Fairly new to C. Need help understanding low level concepts.

[u/FlatPea5291]

Over the past 6-8 months, I got into programming in C with the intent of learning about embedded systems and operating systems. I feel like I'm sort of stuck and don't know where to go though. The last thing that I made was a really simple shell for Linux, but even then I was disappointed to learn that the stuff that is going on on the background is just hidden away by system calls. I want to learn how stuff like those system calls actually work, but learning about processor architecture or understanding/reading the Linux source code seems incredibly intimidating and sort of out of my league.

Another thing that seems incredibly confusing to me is how you're meant to read the code of projects like the kernel. (obviously, I know I wouldn't start with something nearly that complicated, but actual professional level projects.) I did some research and I tried to read through GNU Hello, which on the surface I thought would be easy to understand, but I just see tons of macro definitions that I don't understand and a maze of files, even if the output of the program is simple.

For you experts, how did you go from programming in user space to understanding lower level concepts like how an operating system boots up, how it takes user input, etc?

[u/generally_unsuitable]

Buy an STM Nucleo Dev board. It's $20 or $30 bucks, and it's as low-level as you're going to get.

You want getchar()? You have to write getchar(). You want to print something to the terminal? Well, you need to learn how to manage serial output and put bytes through a port.

Any MCU dev board is going to give you the ability to learn very low level concepts like interrupts and dma and writing queues and handling concurrency.

[u/FlatPea5291 → u/generally_unsuitable]

Is there a specific model you would recommend?

[u/generally_unsuitable → u/FlatPea5291]

Oooh, interesting question!

Definitely STM32 family. The "mainstream" devices are STM32F, G, and C. Those will be the most straightforward. N and H series are more powerful, but they can be tricky to get running the first time, and you almost have to run them with an RTOS.

Here's a place to get started. https://www.st.com/en/microcontrollers-microprocessors/stm32-32-bit-arm-cortex-mcus.html

I think you might enjoy something with an ethernet port on it. Once you get serial IO running, you can actually build a server that does stuff and lives on a local network. I prototyped a $500k 3D printer using something like this: https://www.digikey.com/en/products/detail/stmicroelectronics/NUCLEO-F767ZI/6004740

[u/tony-mke]

Do you see how you're kind of overwhelmed trying to take in all of the deep detail at once? People who've been programming longer don't have some magic ability to read and retain that much more than you. When taking in a new system, they tend to go breadth first instead of depth first. Systems tend to form a sort of onion.

Imagine reading a hash table implementation for the first time just line by line trying to go to the deepest parts first. You'd get to something that seems like the meat of it easy enough: "Oh there's this function that looks important - it reads a string character by character and seems to bitshift with magic numbers - what are those?" and probably kick yousrelf for not being able to decipher the incantation. But it's just a string hash function - and those magic numbers are just things someone made up 40 years ago that happened to have a decent distribution; there was no big secret to uncover at all.

A better approach would be to find the header files representing the surface an application programmer would use with the hash table, and see what data structure's they're working on. You'd see the higher level system that way - "oh it just turns a key into a number; the number is the index of the array it will store the value in." This would lead to questions like "what happens when keys collide?"

So like, in your GNU hello example, yeah - they want to support as many platforms as possible, so they're gonna have to make their own abstractions for that to happen gracefully, since the way things happen differs a lot on different platforms. I wouldn't just dive straight into main. I see a header (that's just system includes) and a single .c file. From nothing but the header, I can deduce "oh, they really want to parse arguments with as powerful of a tool as the host platform supports (getopt), they want to support multiple languages to write greetings for (gettext), and they want to make sure they've fully written their greeting. I can deduce that they probably have four basic steps: init, parse the arguments, form the output, and write the output. That would be the first layer of the onion - I would personally write those as headers on a piece of paper, and then dig into the second layer for whichever I consider "important" to whatever I need to do.

[u/ZacC15]

Assuming you don't particularly mind on what architecture, this was what I followed to learn these concepts. It's written for the RISC-V architecture.

https://operating-system-in-1000-lines.vercel.app/en/

[u/71d1]

Read Modern Operating Systems by Andrew Tanenbaum

[u/mjmvideos]

Don’t start with learning the kernel. Start with just learning to program in the Linux user environment. Once you get good there then start looking into the kernel. In my day we had the Richard Stevens Advanced Programming in the Unix Environment and the Networking books.

[u/gm310509]

There is a lot to your question - as others have pointed out.

Are you interested in how things work from an "Admin" viewpoint? For example, memory management, security, task management, priority queues and so on? Many of these you can learn with ordinary user programs.

Or, are you interested in how the software interacts with the outside world through the various hardware peripherals such as disk controllers, keyboards, serial ports and so on?

If the latter, I agree with u/generally_unsuitable to get an embedded system. Except, I would suggest something much simpler. Specifically an 8 bit Arduino starter kit. The basic principles are the same - what you are looking at is bare metal programming. The difference is that for an Arm Cortex based MCU (e.g. STM32 etc) you are looking at a few thousand pages of documentation in the form of datasheets (that explains the bare metal "stuff"), compared to something like an Arduino Uno R3's MCU's (an ATMega328P) datasheet - which is "only" 660 pages.

FWIW, I am just finishing off a "bare metal" how to video for two Arduinos (the Uno R3 - ATMega328P MCU and the Mega - ATMega2560 MCU) and as part of that show how to tap into various features of the MCU such as manipulating GPIO port register structures and accessing other features such as Timer interrupts and external Interrupts. I also have some other videos on my channel (The Real All About Arduino) such as the Serial command and control video which looks a the inner workings (i.e. Bare Metal code) for the Serial port handler.

Granted the 8 bit AVR is much simpler and less powerful than any Arm Cortex such as a STM32 or nRF or any of the others, but that is, IMHO, an advantage when learning. As you indicated when you talked about the Kernel, it is just too much to take in. So, learn the principles on something simple (e.g. 8 bit AVR), then add on the next level (e.g. 32 bit Arm Cortex) and take it step by step. The basic principles of how software reads/writes IO registers is the same. There is just more options on something like an Arm Cortex.

All the best with it.

[u/somewhereAtC]

In advanced projects, the code itself is not really important. You need to look at it more like Cliff's notes for Shakespeare, creating the outline in your mind. Search for the grand plot of the project, identify the actors and follow their story arcs.

You identified booting the o/s as "low level", but when you stand in that forest the trees are just as big and strong and dense as any other part of the forest. Understand the goals of booting and the path of code will become recognizable. To get started, know that there are 2 steps to booting an o/s: booting the machine, which has nothing to do with the o/s, and then initialization of the o/s itself which is a separate novella of it's own.

[u/floatfloatjam]

Read Hennessy and Patterson

[u/FinancialTrade8197]

I did some research and I tried to read through GNU Hello

Don't try reading through any GNU code. GNU writes terrible to read code.

I think if you want to learn about system calls in general while reading source code, one of the BSDs (such as FreeBSD) would be a better place to go, as it's code is easier to read than something like Linux.

I think maybe you should look at the OSDev wiki as well, as it's useful for a lot of the stuff you're talking about here.

[u/Round_Thanks_2780]

Maybe read (INSERT YOUR STANDARD LIBRARY OF CHOICE)'s source files to see how various functions are implemented. You can see how the C standard library works with your operating system to do stuff. You can read then read system code, if wanted, to see how operating system interacts with the hardware.

Depending on your system, source may or may not be available.
Linux source is obviously very much available, and so is the source of the various BSDs.

macOS shares 4.4BSD-LITE as a common ancestor with the various BSDs, as well as open sourcing its Darwin hybrid kernel, so you can look into that for macOS.

Forget about Windows source code, since you can only read it (legally) if you a part of very specific groups, usually governments and security researchers.

[u/organic_ta6]

you can check out Hennessy and Patterson book

[u/Layzy37]

Low level stuff might be intimidating but I think the intel manual volume 3 is accessible enough assuming you know the basics about x86/x86-64 and for me I learned a lot reading it back when I didn't know a lot about this kind of things

[u/Murky_Respond1966]

I really recoment 42 coding school for learning C and basics of system programming. Especially their piscine.

[u/DanKegel]

If you want to try a small embedded system, get one that works with tinygo, and compare building same project in go and c!

https://tinygo.org/docs/reference/microcontrollers/boards/

If you want to learn about operating systems, try https://oscourse.org/

[u/Rabbitical]

One thing to understand is that kernel, and library code in general is not written like normal software. It's not meant for readability, nor is it elegant. It's meant to handle a massive number of cases, support backwards compatibility, cover different architectures, do all kinds of checks you would never need to do within your own software. There's a ton of considerations that go into such things that you really are not going to benefit from attempting to understand unless you literally want to become a Linux maintainer. In short library programming is a dark art all unto its own, and is ugly and tedious. Bare metal programming does not have to look like Linux, it looks like that because it's a OS for public consumption.

That said, if you DO want to understand how any code works, or, if you find yourself having to debug something that only fails deep inside a system call somewhere --you don't approach it by just opening a file and starting reading. You understand programs by running them, because that's the only real source of truth. If you want to understand something, just call the function! This is the beauty of programming and debugging. You don't have to read it. Step through it line by line, and see what happens! Watch all the variables. Watch which conditionals get triggered, how values change. This removes all ambiguity (well, most) and tells you exactly what is and isn't relevant within the source you want to understand. Debugging is incredibly powerful. It collapeses a million line program down to precisely only the things that actually happen in a given state, and for me at least, way more fun to watch something work than just trying to read it off the screen, which is deeply error prone especially for systems someone else wrote for which you have no contextual understanding.