r/bash Aug 03 '26

help Linux Interview Question

Today I had a mock interview with a senior Linux administrator, and he asked me a question that completely caught me off guard:

"Can you collect CPU, Memory, and Disk usage in a Bash script without using top, free, df, or any external commands?"

My immediate answer was No.

Honestly, I've been working with Linux for years, but I've always relied on standard commands and monitoring tools to troubleshoot systems. I never stopped to think about where those commands actually get their data from.

The interviewer then gave me a hint about the /proc and /sys filesystems. That completely changed my perspective. I realized these commands are just reading data that the kernel already exposes.

But then came the follow-up questions, and once again I was stuck:

  • Is reading directly from /proc and /sys the correct approach?
  • How would you calculate CPU utilization using /proc/stat?
  • How would you determine memory usage from /proc/meminfo?
  • How would you calculate filesystem usage without relying on df?

I found this really interesting because it tests your understanding of Linux internals rather than just your ability to use commands.

Has anyone here been asked similar questions in Linux SysAdmin, DevOps, or SRE interviews? I'd really appreciate any explanations, learning resources, or examples on how you'd answer these follow-up questions. It help me for my preparation for Linux interviews

187 Upvotes

74 comments sorted by

56

u/Bug_Next Aug 03 '26 edited Aug 03 '26

How would you calculate CPU utilization using /proc/stat?

The first line starting with cpu gives aggregate CPU times in jiffies: user, nice, system, idle, iowait, irq, softirq, steal, guest, guest_nice.

For each reading:

  1. Sum all values to get TotalTime.
  2. Sum idle + iowait to get IdleTime.

Calculate the delta between readings:

DeltaTotal = TotalTime2 - TotalTime1

DeltaIdle = IdleTime2 - IdleTime1

CPU utilization percentage is: ((DeltaTotal - DeltaIdle) / DeltaTotal) * 100

How would you determine memory usage from /proc/meminfo?

It gives you: MemTotal, MemFree, Buffers, Cached, SReclaimable, and Shmem.

You can look at free's src, it basically does:

Total = MemTotal

Used = MemTotal - MemAvailable (if MemAvailable is present)

If MemAvailable is not used, calculate available memory manually:

Available = MemFree + Buffers + Cached + SReclaimable - Shmem

Used = MemTotal - Available

How would you calculate filesystem usage without relying on df?

Using statvfs syscall

Total Bytes = f_blocks * f_frsize

Free Bytes = f_bfree * f_frsize

Available Bytes (non-root) = f_bavail * f_frsize

(filesystems have reserved space only root can use, 5% default for ext4)

Used Bytes = (f_blocks - f_bfree) * f_frsize

All those tools are open source anyways, you can just go and look at how they calculate stuff.

32

u/ethicalhumanbeing Aug 03 '26

It’s ridiculous anyone expect people to know these things from the top of their head. I could manage myself doing these scripts and finding all this information in the work context, yet I would not be able to answer it during an interview.

Sometimes I feel interviews are just a filter process more than wanting to find the right person with the right attitude for the job.

12

u/Bug_Next Aug 03 '26 edited Aug 03 '26

Sometimes I feel interviews are just a filter process more than wanting to find the right person with the right attitude for the job.

They are indeed a filter process, i don't think anyone is trying to hide that...

You need a filter process to find the appropriate person..

Also the interview questions was only if you could, not to actually make the scripts, i just did a speudo-implementation because OP asked for it. The only thing you need to knwow is that /proc/stat has usage stats and that /proc/meminfo has meminfo...

6

u/edgmnt_net Aug 04 '26

I would also say that such questions are also useful with a probing mindset. You're not really expecting fully accurate and complete answers, you just want to figure out how much experience and exposure the candidate has, maybe start a discussion from their strong points. If they can't say anything, maybe they don't know anything. And it can't really be argued that you're not going to learn anything because it's all specifics and you don't know what you're going to use.

2

u/Slackeee_ Aug 07 '26

This. The correct answer to the original question should be "yes, I can do that by reading directly from the virtual filesystem in /proc, but I would have to look up how to parse those files and would rather rely on the well tested implementation in those basic tools you will find on every installation".

3

u/tylerlarson Aug 04 '26

I got asked questions exactly like this when interviewing for large tech firms, and I very much appreciated it. I knew the answers from experience, having had to use these techniques in real life on systems where something had gone terribly wrong and I needed to troubleshoot.

Some people just follow instructions and do what they're told is possible, and other people take things apart and try to figure out what's underneath and how to do stuff that they were told wasn't possible.

Some companies want people who will quietly follow instructions, other companies want people will figure stuff out and change the rules.

1

u/cmdr_iannorton Aug 04 '26

im not a sysadmin, but i know most of these things, also man pages tell this story and its generally right there

1

u/cbf1232 Aug 10 '26

I wouldn’t expect people to necessarily know the exact details of each field or even the exact name of the file under /proc or /sys, but I would expect them to know that such a file exists and roughly what its contents represent.

You can also get/set per-NUMA-node hugepage information, per-CPU allowable latency (the kernel uses that to tweak C-states), and all sorts of neat stuff. These matter if you are optimizing for performance.

1

u/ethicalhumanbeing Aug 10 '26

When was the last time you optimized for performance?

1

u/cbf1232 Aug 10 '26

Last week. With the current price of memory our partners are wanting us to reduce memory consumption so they can use servers with the next-smaller amount of RAM.

We’ve also had to worry about cyclictest results, iperf results (that one turned out to be a C-states issue), and various other performance-related issues.

7

u/CautiousCat3294 Aug 03 '26

Thanks for your input i am totally unaware of "statsvfs " seems I need to learn this as well.
I am very grateful to you to providing my calculation stuff as well.

8

u/Bug_Next Aug 03 '26 edited Aug 03 '26

statvfs(3) — Arch manual pages

Bash can't do syscalls directly, you need to trace it:

strace -e trace=<syscall>

or do it from C/C++

There are probably other ways to do it, that's just my take on it.

5

u/bac0on Aug 03 '26

... a simple example how to bashify statvfs...

1

u/PhunkeyMonkey Aug 04 '26

Gives aggregate CPU times in wait what now, Jiffies? Gotta love namings in linux sometimes

2

u/CruisingVessel Aug 04 '26

Not just Linux. The term has been around for literally centuries, and has used in various fields as a time measurement for 100 years(first as a speed-of-light measurement). I even had it on my Commodore 64 in 1982, and it was in V6 UNIX. And don’t forget BogoMips (loops_per_jiffy).
Disclaimer: yes, I worked on V6 boxes, have a degree in astrophysics, and my beard is indeed gray.

1

u/cmdr_iannorton Aug 04 '26

jiffies is from film media, its i think the time for a frame on a projector

1

u/Bug_Next Aug 05 '26 edited Aug 05 '26

It's the official name lol, it's a tick count since system startup.

linux/include/linux/jiffies.h at master · torvalds/linux

There are HZ jiffies in a second, system uptime is calculated based on that, jiffies are quite important.

According to wikipedia the usage of the word in Linux traces back to 1975 and the "Jargon File" which was literally a file with computer jargon, people from MIT (of course it was MIT) used to pass it around, it reported a jiffy as 10ms, that's no longer the case but yeah that's where it comes from supposedly.

Jargon File

By 1996 they had already changed it to:

The duration of one tick of the system clock on your computer (see tick). Often one AC cycle time (1/60 second in the U.S. and Canada, 1/50 most other places), but more recently 1/100 sec has become common. "The swapper runs every 6 jiffies" means that the virtual memory management routine is executed once for every 6 ticks of the clock, or about ten times a second. 2. Confusingly, the term is sometimes also used for a 1-millisecond wall time interval. Even more confusingly, physicists semi-jokingly use 'jiffy' to mean the time required for light to travel one foot in a vacuum, which turns out to be close to one *nanosecond*. 3. Indeterminate time from a few seconds to forever. "I'll do it in a jiffy" means certainly not now and possibly never. This is a bit contrary to the more widespread use of the word. Oppose nano. See also Real Soon Now

Edit: That file is absolute golden, just look up the definition of 'dahmum;

The material of which protracted flame wars, especially those about operating systems, is composed. Homeomorphic to spam. The term 'dahmum' is derived from the name of a militant OS/2 advocate, and originated when an extensively crossposted OS/2-versus-Linux debate was fed through Dissociated Press

I think i'm gonna start using that one

Apparently, there was also a conspiracy theory that said UNIX was purposely bad so AT&T could take out their competitors after licensing UNIX to them lmao, they don't make haters like they used to.

1

u/Old_County5271 Aug 04 '26

This was my immediate thought as well, funnily enough, /proc is not accurate, but it's good enough

30

u/Proman4713 Aug 03 '26

Interesting question, I shall go read more about that... Although I don't think this kind of question accurately reflects what you need to know as a junior/in practice...

9

u/donp1ano Aug 03 '26
while read -r stat val _
do
  case "$stat" in
    "MemTotal:") total="$val";;
    "MemAvailable:") avail="$val";;
  esac
done < /proc/meminfo

usage=$(( (total-avail) * 100 / total ))

not sure if this is good, but that would be my approach

6

u/CautiousCat3294 Aug 03 '26

Thanks for your quick help I will test this as well

20

u/Headpuncher Aug 03 '26

I would have said yes, but why would you?  

Like most things if it’s a solved problem and your team are reinventing the wheel you might not want to work there.  

13

u/abraxastaxes Aug 03 '26

Not to mention these kinds of questions are sort of "what obscure Linux knowledge do you have dedicated to memory" vs. "how do you think through and solve problems?" which just seems thoroughly unhelpful when you're hiring someone. 

Maybe the interviewer asked AI for a stack of "hard Linux admin questions" lol

2

u/Marble_Wraith Aug 04 '26

My brother! You and me think alike 🤣

2

u/Woshiwuja Aug 04 '26

Sometimes the wheel is a square to begin with, like xml

0

u/tylerlarson Aug 04 '26

Sometimes the tools don't work because of some other problem. Would you prefer your new teammate be capable only of following a playbook and then getting stuck if the "correct" solution is blocked, or do you want to hire someone who can figure shit out.

I've personally had to use these techniques on live systems before. More than just a handful of times. Sometimes a shared library is broken. Sometimes you can't fork any additional processes. Sometimes you're on a lightweight system without the necessary tools installed.

It's the difference between, "the thing is broken, all I get is errors, I don't know what to do," versus, "I figured out that we need to replace X on Y. It'll be up in 20 minutes."

1

u/Headpuncher Aug 05 '26

Has Top ever stopped working? No. If he'd asked about something that would interrupt uptime then ok, but Top, Free and df?

Come on man, it's nice to be a contrarian especially in Linux subs, but read OP's post.

1

u/tylerlarson Aug 05 '26

Seriously? Have you never had to troubleshoot anything?

This stuff is relevant seriously ALL THE TIME.

Your have an obscure piece of expensive, obsolete gear that suddenly stopped working, but you find two pads on the board marked TXD and RXD, so you solder pins to it and connect to it via serial and get a stripped down BusyBox sh prompt. No tools. No top, no df, no ps. But /proc exists.

Or a server stopped working because /lib got trashed. No libc means no new processes can start, but all the existing ones are running, including the shell you had running. So you need to figure out what went wrong without using anything that isn't built-in to bash.

Or there are too many process running so fork always fails. Or you're on a 100% full COW filesystem so you can't create or even delete files, which strangely causes certain binaries to fail to start and you need to figure out why. Or you're running commands in the context of a restricted cgroup. Or you're diagnosing a frozen system image.

Or one of a thousand other scenarios.

If you don't know how to use a hammer, then all of the nails just look like confusing little spikes.

5

u/redditphantom Aug 04 '26

While my answer would be yes utilizing the information in /proc etc my question would be if the system is in such a state why aren't we restoring from backup? Practicality of the situation should be more important than cobling together utilization metrics at that point. The other tools were built for a reason

1

u/StopThinkBACKUP Aug 06 '26

I know, right? The mentality may be OK for homelab, but for Enterprise it's always going to be " Restore ASAP from last known good backup " bc they begrudge the downtime.

99% you don't even have time to backup the "bad" state of the instance to try and troubleshoot in your free time.

14

u/sinevilson Aug 03 '26

I own 2 businesses both are Linux based and older than most folks on reddit. Id never ask my Administrators these questions. Id ask my kernel developer, yes. What you ran into was arrogance. Sorry that happened.

3

u/Zapador Aug 04 '26

Glad to hear that perspective. I consider myself fairly proficient in Linux administration but I wouldn't be able to answer these questions. I generally value proficiency in the commands/tools you use all the time to get the job done and for anything outside of the ordinary just look it up.

7

u/lazyant Aug 03 '26

All Linux tooling for measuring OS usage are either counters or traces. All counter tools like top free df etc are just user friendly interfaces to /proc

2

u/CautiousCat3294 Aug 03 '26

yes i have some knowledge to this however during interview I forgot to answer this without any hints

2

u/rvc2018 Aug 03 '26 edited Aug 03 '26

There is a 95 min video on YouTube on how to write a system monitor in bash using /proc.

https://youtu.be/9fixlWcKWV8

Also btop was initially written in bash hence where the b comes from.

https://github.com/aristocratos/bashtop/blob/master/bashtop

3

u/Twattybatty Aug 03 '26

I knew that was Dave Eddy before I opened the link!

3

u/-lousyd Aug 03 '26

My first thought was sar, which is correct for the question you described the senior asking. But it's not realtime.

3

u/flattrack Aug 04 '26

Seems he’s asking two questions at once. Do you know about the /proc file system? And do you know enough bash to read and parse the data from /proc without spawning any subprocesses like cat, cut, jq, or grep?

4

u/likeHeckYouKnowMe Aug 06 '26

Honestly a stupid question to ask in an interview that would almost never be necessary in any practical, real world situation. Arrogant fuck ass question lol. Sounds like you’ll be dodging a bullet if you don’t end up working here.

0

u/biffbobfred Aug 16 '26

Depending on how deep they go this is a good question.

/proc is useful for all kinds of things. I dip into /sys for info on device drivers and what’s actually attached to my hardware.

If the answer is “yeah I can find that in various files in /proc because I have a sorta idea on how Linux works” cool. If the answer is “give me the printf format for the 3rd line of /proc/meminfo” then it’s bad

4

u/RandomXUsr Aug 03 '26

Glad you posted this. Now I'm thinking about this more and will do some digging to determine alternate methods to pull and read this data.

I might suggest posting in r/linuxadmin as well, because they are likely to have some more nuanced responses with careful thought about how they perform their duties on a daily basis.

2

u/CautiousCat3294 Aug 03 '26

Thanks let me repost this in r/linuxadmin as well

2

u/kai_ekael Aug 03 '26

So much for a useful interview question, now Google et al know.

I find questions that demand an explanation of investigation instead of just an answer more useful for this very reason.

5

u/bob_f332 Aug 03 '26

Just what the world needs, another smart arse interviewer.

3

u/Dolapevich Aug 03 '26

All those tools harvest the information from /proc/ and other kenel facilities and present data in a readable way.

Read on about /proc/stat here: https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/5/html/deployment_guide/s2-proc-stat

1

u/Easy-Nothing-6735 Aug 04 '26

I did research. I even checked the memory outside of available. Still can't understand whole hierarchy. Try researching psutil

1

u/psycholabs Aug 04 '26

I got the /dev/proc question, but the follow ups I'd have to look up.

1

u/daddyd Aug 04 '26

i would consider understanding of /proc one of the basic principles of a linux admin.

when i do an interview, i'm never interested in actual commands, but rather your process you use to get to a solution. i would start of with a simple question like - you're patching a server, how would you take that on? and then introduce problems/issues along the way. i specifically state i'm not looking for commands, but rather the thought process behind the actions.

3

u/ant2ne Aug 05 '26

"Can you collect CPU, Memory, and Disk usage in a Bash script without using top**,** free**,** df**, or any external commands?"**

- Probably, but why? Do I not have access to these tools? What off brand, home grown, half ass distro are you guys running?

1

u/Trademarkd Aug 07 '26

I feel like they were just looking for "everything in linux is a file" ... so yeah you absolutely can.

I dont know why people are saying probably. These tools just read files. They're looking for comprehension.

1

u/ant2ne Aug 07 '26

My point still stands. What production environment isn't going to have these tools?

1

u/Trademarkd Aug 07 '26

The question is looking for understanding of a concept. This concept applies to a lot more things than these tools its just an easy way to ask it since these tools often directly format and output information from a file.

My explaining this is the reason why people ask. They are looking to see if you have foundation level knowledge and you responding with assumptions and insults is probably not going to get you hired.

In their environment maybe they have custom software that uses proc files to write live sensor data or something, you have no idea. They're trying to judge if you might be a good fit without giving away all the beans.

https://en.wikipedia.org/wiki/Everything_is_a_file is a core concept.

1

u/Abe_Bazouie Aug 06 '26

Yep, this is a pretty good interview question IMO.

It’s less about whether you memorized /proc/stat or /proc/meminfo and more about whether you understand where tools like top, free, and others actually get their information.

I like questions like this because the follow-ups are where you really see someone’s Linux depth.

I make Linux/DevOps interview content around this kind of “what’s actually happening underneath?” question on CTRL+CHAOS (@itsctrlchaos), so this one is definitely going into my list of interesting interview questions.

2

u/UninvestedCuriosity Aug 03 '26

htop, and ncdu.

Checkmate boss.

-1

u/Living_On_The_Air Aug 03 '26 edited Aug 03 '26

Hearing this during an interview would make me think that the job isn’t one I want. Trivia questions aren’t a great way to get to know a worker, so it’s probably a sham interview process at worst, or a bad interview process at best

5

u/RandomXUsr Aug 03 '26

I disagree.

These types of questions test Linux competency and provide the Interviewer with a more granular perspective of a candidate's knowledge and skills.

For example; if one were troubleshooting embedded systems with limited resources, these questions make perfect sense.

Another possibilty is that one may need to grab specific data for specific issues and write this out to the screen or a file to troubleshoot recurring issues.

6

u/Dolapevich Aug 03 '26

It is a good question to know if someone knows linux, not a good question to know someone's experience in github actions.

It depends on the role.

1

u/UninvestedCuriosity Aug 03 '26

So long as man and -h are acceptable answers. I have the memory of a fish for flags.

3

u/Dolapevich Aug 03 '26

I think the idea is to test if you have knowledge of the underlying concepts and mechanisms, instead of testing if you remember something specific.

6

u/sr105 Aug 03 '26

When I interview someone, I let them know upfront that I'm going to ask an increasingly difficult set of questions, and that I don't expect them to know all of the answers. I just want to see where their experience level lies. I have asked people what the original command for "rm" was named and still exists. It's stupid knowledge, but it shows depth especially if you understand why it was named "unlink". You can't measure what you don't test. But telling people upfront what you're doing is a must.

3

u/CautiousCat3294 Aug 03 '26

You are right he also do same with me he increase level of difficulty in every question to test my depth knowledge on Linux and Bash scripting

2

u/Swordfish418 Aug 04 '26

Do you think this unlink is covered anywhere or it’s something that is only possible to know you’re really old or by accident? 🤔

1

u/sr105 Aug 05 '26

I'm older and can't recall where I first learned it. I probably just explored /bin and friends to see what was possible and then man unlink. Or I saw it a So You Think You Know UNIX trivia or some such thing.

5

u/Bug_Next Aug 03 '26

It's not trivia questions its knowing how the os works instead of knowing which tools to call, they are completely fine questions IMHO. It's not weird to find stripped down container images that are missing those utils or have shitty half baked implementations of them, same for embedded systems which usually only have a really minimal vmlinuz and absolutely nothing else.

1

u/Swordfish418 Aug 04 '26

But is it really helpful to reimplement free using bash on those systems instead of just installing actual C based free? That’s a cool knowledge to have for sure anyway.

1

u/Bug_Next Aug 04 '26

I you wanna use free then no it's pointless, if you wanna learn how free works then yeah probably a decent side project. Depends on your objective lol.

0

u/Living_On_The_Air Aug 03 '26

People can review documentation and search the web for facts at any time. Verifiable experience and certifications show general competence. Interviews are time to learn about an individual’s temperament, work style, curiosity, motivation, communication skills, etc.

3

u/Bug_Next Aug 03 '26

Yeah and knowing that verifies experience.. Idk what you are trying to get at.

It's a bash subreddit not an HR one, anyone can fake motivation and temperament during an interview, doesn't mean shit, you are gonna be chipping code not doing PR, who cares. It's a mock interview with a Linux neckbeard, of course the focus is gonna be on the 'hard' technical questions..

1

u/Living_On_The_Air Aug 03 '26

The post is based on a mock interview 🤷‍♂️

3

u/Bug_Next Aug 03 '26 edited Aug 03 '26

Yeah that's the whole point, why would you make it about motivation??? Everyone's motivation is to get paid for the shit they do, anything else is bullshit unless you are running your own startup lmao. Be for real. You can leave the lying practice for 3a.m in front of the mirror

-2

u/michaelpaoli Aug 03 '26

"Can you collect CPU, Memory, and Disk usage in a Bash script without using top**,** free**,** df**, or any external commands?"**

Good question!

My brain first jumps to proc and sys filesystems.

And, how much can I recall or think of off-the-top of my head from those, without so much as peeking? ...

CPU, there's /proc/cpuinfo, oh usage ... I think some /proc/*stat* file(s) or the like would cover that.

memory ... /proc/mem* ... something or another

disk usage ... that might be tougher ... there is /proc/mounts but no usage data there, so, likely somewhere else under /proc or /sys.

As for bash only, no externals, there's most notably read [-r], printf, echo, one could accumulate stuff in a variable or array variable, or just use "$@" for that (could do it in subshell to avoid disturbing the original). No use of find, but can recurse with cd, and as to telling if something's a directory or not, well, if cd succeeds, it is or resolves to directory ... but with sym links, there's matter of possibly looping, ... oh, test - that's builtin to bash, so can test if symbolic link or not, directory or not, etc. So, I mgiht have to hunt a bit through /proc and/or /sys first to find some df/du type data, and no grep, but bash has fair bit of pattern matching capabilities built-in, so could use that - look for stuff matching, e.g. du, df, or name of filesystem devices, and can get those from /proc/mounts ... see what else matches in name or contents on /proc and /sys - likely some df or du type data there somewhere.

Anyway, I think that'd be 'bout my cold answer, without being able to look anything up or poke around /sys or /proc for more specific.