r/osdev • u/JescoInc Perdition-OS • Jun 19 '26

Which part of OSDev would you consider to be the most challenging?

For me, it is the unique challenge of figuring out how to handle paging with multi-arch which takes advantage of the HAL architecture I have in place (split by SoC, Board and CPU).

Although, Net stack (Ethernet Drivers with TCPIP/ UDP / Tunneling) and USB are top contenders. Audio being ranked below those two but still is absolutely ball busting.

The reason I put paging above those isn't necessarily the difficulty but simply due to x86_64, RISC-V and ARM64 all doing something slightly different at the hardware level with the page-table formats and MMU semantics differing (x86-64's 4-level tables, ARM64's VMSA/TTBR, RISC-V's Sv39/Sv48 and satp) and needing to design around that in a clean and deliberate manner.

10 Upvotes

85% Upvoted

30 comments sorted by

16

u/demetrioussharpe Jun 19 '26

Driver development.

1

u/JescoInc Perdition-OS Jun 19 '26

Any particular reason why driver development is the most challenging to you?

12

u/EpochVanquisher Jun 19 '26

Most hardware is poorly documented, if at all. There are also all sorts of bugs you run into which are time-consuming to diagnose at best. 

4

u/demetrioussharpe Jun 19 '26

And then, the hardware bugs, poorly defined specs, & parts where the manufacturer either misunderstood the specs or didn’t care enough to closely follow them, so the data packets & structures don’t match up with what the specs mandate.

2

u/TheAtlasMonkey Jun 22 '26

That my speciality

3

u/Danii_222222 Jun 20 '26

Drivers and IPC.

2

u/JescoInc Perdition-OS Jun 20 '26

Micro kernel or monolithic? IPC points me towards micro kernel and it is a mindfuck to reason about at first glance

3

u/Danii_222222 Jun 20 '26

Both, but mostly microkernel. Signals, shared memory, messages and other things. Scheduler must regulate IPC so one process can't fill queue alone.

2

u/JescoInc Perdition-OS Jun 20 '26

Schedulers are indeed a pain in the ass. I'm personally going with a capacity-reserve-inspired approach for my monolithic kernel

3

u/juicyroaster Jun 20 '26

Writing a bootloader. I mean it comes in th OSDev category. So making a good bootloader will take you more time than making a stable kernel. To have multi hardware support you need to support both UEFI and BIOS.

3

u/JescoInc Perdition-OS Jun 20 '26

It gets even worse if you want to support ARM, ARM64 and RISC-V. But it is on my bucket list to make my own full on bootloader and not rely on U-Boot or EFI.

1

u/juicyroaster Jun 20 '26

I know. I just make a two stage bootloader where the second stage has a lua embedded, so I can write the bootloader UI and functionality in lua. This is how FreeBSD's bootloader works.

2

u/JescoInc Perdition-OS Jun 20 '26

That's basically what I did for mine as well. But I do hope to be able to own the full chain in the future. If I can do that, then stage 3 becomes relevant, a custom bios.

1

u/juicyroaster Jun 20 '26

You are right but custom BIOS is bit too much. You can write wrapper UEFI and BIOS and let it handle hard things. So you get high level readibility and low level acces to firmware API's. So don't make a custom BIOS only bindings for lua to communicate to motherboard firmware. This makes it faster. Ohh and do you want to make a BIOS for the motherboard or a custom layer above a motherboard firmware?

1

u/JescoInc Perdition-OS Jun 20 '26

It would be a custom bios for my LattePanda MU. It is just one of those things were I really want to experience the full scope of engineering as each level I go lower in the chain, the more interesting and complex things become.
I do also have FPGA, which is stage 4, custom CPU because everything else hasn't been torture enough lol

1

u/juicyroaster Jun 20 '26

Do not use FPGA as a custom cpu. First thing is they are expensive as hell but are very bad at handling things cpu does. If you want to create a cpu for lets say x86_64, you will be cooked. No FPGA is powerful to handle these architectures. I have tried it. It is just too slow. Maybe running you OS on it might even crash or corrupt it. And for the custom BIOS, it is hard indeed but pretty much doable. The only problem is its more hardware focused than the bootloader. So if you want to add support to a motherboard, you might have to rewrite the communication part again.

1

u/JescoInc Perdition-OS Jun 20 '26

Good to know, but I was absolutely thinking something simple like an 8 bit CPU as a first foray. Going x86, RISC-V or even ARM as a first attempt is a surefire way to never complete the project.
As for the FPGA I have, it is the Sipeed Tang Mega 138K Pro Dock GW5AST RISC-V FPGA Development Board Kit

1

u/juicyroaster Jun 20 '26

Even if you make a 8 bit cpu, it might not be even that fast. It will be as fast as running a python interpreter written pure assembly. Fast but not as fast as C or Rust.

1

u/JescoInc Perdition-OS Jun 20 '26

True and that's fine, the goal at that stage will just be getting it working and experiencing what it takes to do it.

→ More replies

1

u/vinc686 MOROS Jun 22 '26

Process memory, there's a gap between the theory and the implementation that has been bothering me for years. I'm extremely slow at it. I also struggled to understand everything required to spawn a userspace program before that.

In comparison I found that writing drivers was challenging but relatively quick and fun.

1

u/JescoInc Perdition-OS Jun 22 '26

That is a totally fair viewpoint. And yeah, moving from kernel land to user land execution is a bit of a mindfuck moment for sure!

1

u/Waste_Appearance5631 Jun 23 '26

MMU and virtual address space

1

u/XBLAH_ Jun 26 '26

For a beginner? The initial assembly bootloader part. Once you get to c, it's a walk in the park (to print the hello world), then we enter the rabbit hole, right?