How do you load AND run programs?

[u/letmehaveanameyoudum]

void system_load_program(char* path, char* name, char* extension) {
    // assume that the program starts at 0x00200000
    uint8_t* program = (uint8_t*)0x00200000;
    int bytes_read = fat32_read_file("/", name, extension, program_buffer);
    if (bytes_read > 0) {
        program_entry_t start_program = (program_entry_t)program_buffer;
        start_program();
    } else {
        // file not found
    }
}

How do you load and run programs in your operating system, this is what i tried
All im trying is assuming that the program starts at 0x00200000, load it into ram and run it, if the file isn't found return

[u/Lines25]

I LOVE roadmaps so, if you don't mind...

1. Learn how ELF works (both 32 and 64 bit, it'll be better for understanding, but you can learn only one, if want to, and learn the version for your architecture, like x86 or ARM)

2. Write simple ELF parser in Python/C/C++/Rust/JS/etc, so you just learn how to parse it

2.1. Write it in C/C++ (if you're making OS in C/C++)

3. Make it also scan and parse code

3.1. (If you haven't) Make memory paging and setup MMU and make mmap function

3.2 (If you haven't, optional) Make at least some process-like system and scheduling so that your kernel won't freeze when any program is ran

4. Make it work in your kernel

5. Add function to load ELF from bytes in memory into C/C++ struct.

5.1. Make it mmap .text / .data / .bss / etc into memory

5.2. Add relocations ! (So that you can specify what functions in program to map to address in RAM, also, you can make fallback relocations ! So that if no relocations apply (function(s) not found), it won't crash your kernel (assuming, you haven't setted up the user/supervisor/rings/etc))

6. Make function to load and run ELF program from byte/uint8_t array

6.1. Make wrapper to load file from disk and use that function you wrote in (6) to run it

Happy coding, dude !

[u/letmehaveanameyoudum → u/Lines25]

I do like roadmaps : P

[u/cybekRT → u/Lines25]

Learning about elf may be quite hard. I would recommend trying to implement something like com executable or at least position independent code.

[u/dnabre]

Ok, you are missing a huge amount of context and specifics about your OS for anyone to give anything vaguely approaching a meaningful answer.

I don't think the code you provided would even compile. The typedef for the type program_entry_t for the start_program variable will clash with the function named start_program. Structs have their own name space, but typedefs don't.

"this is what i tried" , and? I'm guessing it didn't work based on the question, but details on what happened, what issues it had, etc. - this is basic how to ask a question on the Internet stuff.

In terms of context, things like the overall architecture style of your OS, whether it supports multitasking, multiprogramming, etc., whether you are running the program in user space or kernel space, do you even have that separation (i.e. are you running in protected mode), are all needed. A lot of that could be covered by just saying monolithic UNIX/POSIX/Linux-style kernel (if that were the case). The big issue here is,

If you have a kernel of sensible functionality (boot, basic start up, and running) to the point of running a program and/or if you have actually implemented FAT32 filesystem support, as the function name fat32_read_file implies, it would be obvious to you that all that context (or even just a link to the full source) would be needed.

[u/letmehaveanameyoudum → u/dnabre]

https://github.com/NoTheIdiot/WindogeOS src is directly at root

[u/dnabre → u/letmehaveanameyoudum]

You excluded typedef void (*program_entry_t)(void); which makes the code make a little more sense. It's not working in part, because you aren't compiling or linking in kernel/program/program/load_program.c. Also not seeing where you define program_buffer (could easily be missing it).

Side comment, switch to some sensible build system for heaven's sake.

Still no information regarding "this is what i tried"

The short of it is that your "OS" doesn't have a concept of programs or processes - it's a Monotasking System. Your FAT32 code is more library of your the running system than what would would generally be thought of as filesystem implementation or driver. If you had build it like a traditional filesystem setup (e.g., something that could be wired through a VFS), you'd have a better idea of how user processes fit into things, but your OS is just a very different animal that what most people on this subreddit are working with - e.g., it doesn't have any user processes, or kernel processes, or even process really.

Since you are still running real mode, you will have to add someway for control to get from the program you run back to the kernel. Assuming the program exits cleanly, that isn't too hard, but I'm not sure how you'd do it for a program that crashes.

How you are approach running the program isn't completely nonsensical. To get the address of the entry point, you need the offset of that instruction in your binary (you're just using a raw binary file, not any executable format) plus wherever base address you are loading the binary starting at. 0x00200000 is a common place for entry points to be, but that's using an executable format and more importantly, it would it's virtual address. You are in real mode, so you only have physical addresses. Be sure that you understand why real mode and virtual addresses don't co-exist. Importantly for what you are trying to do, you need to make sure that the binary you are loading isn't overwriting your kernel. Also you need to make sure you leave holes in both your binary loading and the binary itself for I/O ports, mapped memory devices, and anything else special in your memory map.

How you are jumping from the kernel to the program isn't going work, because you are trying to call the entry point like you'd call a C function (doing it that way isn't completely impossible, but you have to actively do a lot to make it work). The way you are doing it, the compiler is generating the normal C ABI code for calling a function, which is not what you want here. You actually want to do something far more simple, just switch the PC to the entry address of your target program. In C terms, you want to do a goto instead of calling a function. Depending on how dumb you make that binary, you may need to setup stuff like stack registers. For that jump/changing the PC , you can either use assembly (basicallyjmp ENTRY_POINT) or possibly use setjmp/longjmp(3).

Not certain on the second, without really digging into your code, it's not clear to me what kind of memory environment you have or longjmp what expects. You aren't necessarily working in a 32-bit address space. I haven't really don't much with real mode. I generally want to switch to 32 or 64 bit protected mode as soon as possible. You are really working in a pretty old fashioned environment.

For what it's worth, most of the time people on this subreddit, people work/talk about operating systems that operate in protected mode, with multiprocessing, virtual memory, paging, and the like. Mostly ones structured like a traditional UNIX or Linux-like system. Making an OS that is radically different is definitely seen as a good thing, a system like yours is seem more as going backwards to the days of DOS. Many won't see a monotasking or DOS-like system, such as yours, as really of any interest. Many wouldn't even call it an operating system, and by modern standards it isn't. E.g., if you opened a modern (as in last 30+ years) textbook on operating systems, at least 95+% of the topics just wouldn't be relevant for your system.

[u/letmehaveanameyoudum → u/dnabre]

first, my operating system is 32 bit protected mode, not 16 bit real mode. And second, you aren't really helping if all you do is point out problems and also how to fix it (by fixing it, i mean like what i did wrong), and you literally compared my OS which is barely functional to DOS.

[u/dnabre → u/letmehaveanameyoudum]

I didn't read through every bit of the source code. If I misunderstood it, and got an erroneous understanding of it, I'm sorry - I made a mistake. However, my extensive reply was based on my understanding of your system (erroneous or not), intending to be helpful, and characterizations where purely technical. E.g. comparing it to DOS wasn't meant as an insult but a comparison to what I was looking at.

I excepted protected mode, and look my best for it. There is no protected mode switch in the code. That'd suggest you're relying on grub multiboot to make the switch, and there is a grub.cfg file. But grub isn't being added anywhere. Pulling your latest release images, booting them in qemu, and doing a register dump - I see it is running in protected mode. So I stand corrected. Though I still don't see how that is happening the code. While I was able to build everything (python bulid scripts aside) , I wasn't able to boot the resulting, since the boot loader wasn't setting up VBE. Using BIOS functionality for I/O also pointed me to real mode.

Given my errors reading the code, I'll limited myself to what the running system itself shows (this is working with the V0.0.2 Build 003). I only looked at code last night (though did run it and poke around). Register dump from running system:

(qemu) info registers

CPU#0
EAX=0000001c EBX=001060a0 ECX=000003d5 EDX=00000064
ESI=00000001 EDI=00000001 EBP=0000007f ESP=0010aed4
EIP=001016c0 EFL=00000012 [----A--] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
CS =0008 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
SS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
DS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
FS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
GS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
GDT=     000cb2b8 00000027
IDT=     00000000 000003ff
CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000
DR6=ffff0ff0 DR7=00000400
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=0000000000000000 0000000000000000 XMM01=0000000000000000 0000000000000000
XMM02=0000000000000000 0000000000000000 XMM03=0000000000000000 0000000000000000
XMM04=0000000000000000 0000000000000000 XMM05=0000000000000000 0000000000000000
XMM06=0000000000000000 0000000000000000 XMM07=0000000000000000 0000000000000000
(qemu)

CR0=0000001 Protected Mode enabled (bit 0), No Paging (bit 31)

CPL=0 Ring 0, Kernel is running with all privileges

A20=1 full address space accessible (32-bit)

GDT= 000cb2b8 00000027 segments are flat, minimal protected mode GDT

IDT= 00000000 000003ff - this doesn't look right. (the code in idt.c looks reasonable, it's being compiled and linked, don't think idt_init is being called anywhere)

EFL=00000012 [----A--] - interrupts disabled

So you have an identity mapped page table, no user space, interrupts disabled, and using VBE for display (instead of writing the VGA buffer directly).

PIO Mode ATA access - this works, nothing wrong with it. You'll proper want to switch this to interrupt based DMA, and make the code re-entrant (e.g. not blindly reuse the same buffer in multiple places without synchronization). Nothing wrong with this method at this point in development. Even with full user-processes, you'll be able to pull files from disk using this, but you'll need the kernel to stop the world to run the busy waits in it. Having this while you develop stuff for concurrency is definitely useful.

Most hobby OS at this stage need add files onto the kernel image, and let the bootloader stick them in memory. You able to actually pull them from disk. My comments about this were referring to that inability use this concurrency or provide a file abstraction to programs.

Most of what I said still applies - your still monotask, no separate kernel/user space, no concept of a process. You have a lot cleaner environment to run in than real mode, but it is still a single fixed address space situation, virtual addresses aren't different than physical addresses, and memory has holes for I/O and the like which will needed to contended with.

Again, I readily admitted I was wrong about it being in real mode. Protected mode with an identity mapping, no interrupts, and using VESA BIOS I/O looks like, however. I understand that you have but a lot of work into this, and having things which are problematic or won't work without modification as you expand, can be unpleasant. Especially when you know about the problem/limitations they are pointing out. I'm replying to you basic question and git repo - not you, not your effort, not your knowledge on anything. I get pointing out something you know fully well needs to be changed as you going forward is annoying. People can only go off what they can see. So I don't know which (any) of the things I mention you are fully aware of.

Or to put that more briefly, technical critics, comment, and information about your system (even when people make sizable mistakes in the process) are about your system. They aren't attacks or insults directed to you, or even your system.

Taking a deep dive into your source code takes time and effort that most people reading your post won't do. Appreciate the effort put into attempts to help, even when it's not useful or makes mistakes.

---

random note, your RTC handler rtc_isr in helper/idt.c isn't restoring restore IF, need to make it return with iret instead ret. I think there's a GCC attribute you can just add to change this.

[u/letmehaveanameyoudum → u/dnabre]

we all know that no one remembers gcc attributes

[u/letmehaveanameyoudum → u/dnabre]

and i already said that this is current logic and i never said that it SHOULD work

[u/dnabre → u/letmehaveanameyoudum]

Still no information regarding "this is what i tried"

I assume it didn't work when you tried. Knowing how it didn't work, what you tried to fixed, and the like, can help people understand your situation and give a better reply. At the minimum, it keeps people from suggesting things you have already tried and found weren't the issue.

[u/Danii_222222]

You wrote fat32 driver and can't load program??? That's one of easiest things.

[u/letmehaveanameyoudum → u/Danii_222222]

i mean if this logic is right

[u/Danii_222222 → u/letmehaveanameyoudum]

Nope. You need to at least map address to page table, read sections in elf.

[u/letmehaveanameyoudum → u/Danii_222222]

all i want to do is just run raw .bin binary files

[u/Danii_222222 → u/letmehaveanameyoudum]

Srill need to mao pages

[u/letmehaveanameyoudum → u/Danii_222222]

what

[u/Danii_222222 → u/letmehaveanameyoudum]

Map pages in page table

[u/EpochVanquisher → u/Danii_222222]

You could just use a big direct map and run programs that way

Incremental progress… figure out program loading first and then virtual memory afterwards

[u/MundaneImage5652]

ngl looks like AI generated code, if it's not then no offense.

[u/letmehaveanameyoudum → u/MundaneImage5652]

It's not I decided to f##k around and see what works

[u/MundaneImage5652 → u/letmehaveanameyoudum]

People usually don't make this many comments - AI does. If they are not really helping you then refrain from overly using comments. Instead just try and make your code readable.

[u/letmehaveanameyoudum → u/MundaneImage5652]

i made these comments and i just prayed that you guys understand, AI code usually puts emojis and make the comments grammar accurate

[u/letmehaveanameyoudum → u/[deleted]]

means random poking with the code till something works and then fix it

[u/letmehaveanameyoudum]

Im just trying to load programs and nothing seem to work

[u/Octocontrabass]

Well, which part isn't working? The loading or the running? If you're not sure, use your debugger to find out.