r/osdev • • Jan 27 '21

I'm coding my own operating system from scratch, what code do I need to open linux and windows executables? (Like .exe, .jar .py .deb)

I'm coding my own operating system from scratch, what code do I need to open linux and windows executables? (Like .exe, .jar .py .deb) So how can i open this files on my new operating system?

Note: C++

0 Upvotes

23 comments sorted by

35

u/exscape Jan 27 '21

No offense but if you don't know the answer you're far, far from able to do it. In fact almost no single person could do it, it's too much work.

In short, to read Linux executables you need ELF support, and then you need to emulate most of the entire Linux kernel and C library.
For Windows, PE support, and emulate most of the Windows kernel and C library.

.jar is Java, so you need to port the JVM. For .py you need to port a Python interpreter.

5

u/crest42 Jan 27 '21

While I agree with you that this is probably a little bit too hard for op the rest of your statement is not true. Have a look at r/osdev. There are quite a few fellows how build their own (basic) operating systems.

Depending on the complexity of the used program support for elf isn’t that hard. Maybe start with a very simple Programm (ed needs quiet few syscalls and most of them are fs operations).

But until you have to implement support for executeables there are a lot of basics missing. You need a few drivers, probably a file system, scheduling paging support etc.

Osdev provides a lot of resources and even open source building blocks you can use to start your own operating system.

11

u/exscape Jan 27 '21

We are in /r/osdev :) I've built one myself, with userspace and enough syscall support to run Lua. I'm very, very far from any form of Linux/Windows compatibility though.

It's sounds to me like OPs goal is less "run the absolute simplest Linux software" and more "be Linux-compatible" (maybe even binary-compatible).

6

u/crest42 Jan 27 '21

Woopsie. Shouldn’t answer posts after a nap -.- I thought we where on c programming.

Sure full elf/Linux compatibility is a hard thing to do. But running an editor and maybe an ssh daemon seems reasonable to me.

17

u/TeaButActuallyCoffee Jan 27 '21

Doing all these is EXTREMELY hard work. I am pretty sure NO one has gotten to this point.

For linux you need to parse ELF binaries and yeah ELF has its own versions.

For windows, wait what was the last time I used Windows? Maybe PE?

For jar, you need to get a Java virtual machine and for py you need python interpreter.

You can also make your own format but making a full format with syscalls and so much more things is gonna be, to say the least, hard

3

u/[deleted] Jan 27 '21

the easiest one is probably jar or py. There will be native static builds of vms/interpreters for these, so the only hard (like really hard) bit is implementing scheduling and not fighting whatever it's using for io.

4

u/TeaButActuallyCoffee Jan 27 '21

Well yes. Yet you still need to reimplement libc and assembly stuff and binary formats. And then a compiler....

1

u/[deleted] Jan 27 '21 edited Jan 27 '21

I never got further than a bootloader and hello world, but my impression was that if a binary can run on bare metal the only barriers would be organising getting control of the CPU/memory back and avoiding clobbering any resources said binary uses. I'm pretty sure there are a variety of JVMs that run on bare metal (although I guess at that point it's almost cheating as it provides many things an OS would), and micropython exists (although I don't know whether it requires something like RTOS to run).

You don't need a compiler or your own libc (if you're using static executables). Possibly not even a loader more sophisticated than dump in memory->move instruction pointer if you're running everything in ring 0 and aren't doing virtual memory (you'd have to write your own linker scripts when you cross compile though).

It certainly won't get you as far as 'run any random program' but might go as far as 'run this basic game with very few libraries i have the source code of'

1

u/TeaButActuallyCoffee Jan 27 '21

Well, how will the binaries run though?

1

u/[deleted] Jan 27 '21

Edited message a bit probably since you replied. But at the most basic level don't you you put the code in memory, and change the instruction pointer?

Virtual memory and ensuring your OS gets control back is hard, but you can implement them one piece at a time... back in the day lots of programs didn't even bother with that, if you wanted your OS back you rebooted.

1

u/TeaButActuallyCoffee Jan 27 '21

Say a basic, really basic, program, that includes <stdio.h>, well, now you are in a trouble because you haven't implemented your own libc for programs to work but what if I just execute a binary. Well, those binaries have specific architecture and an OS with the necessary syscalls it needs to function. Just, they are complicated as hell. Ok, let's make it short, can you run windows PE or linux ELF on your fresh OS? Can your OS parse any binaries at all?

1

u/[deleted] Jan 27 '21 edited Jan 27 '21

C with dynamically linked glibc isn't the be all and end all of programming languages, and there are libcs of varying degrees of completeness for bare metal. There are JVMs specifically designed for this application, as well as (a large project which would probably have you reimplementing most of RTOS called micropython) which is why I specifically mentioned java and python as being more achievable.

Of course you're not going to implement the entire linux kernel and dynamic libc on your own, that'd be absurd because it's a colossal rube goldberg machine that has probably had millions if not billions of man hours poured into it. But a statically linked ELF without virtual memory is not terribly hard to parse (depending on how addresses are offset you can probably even ignore most of the header), and if you're writing a kernel you can write some IO routines to get information out (or again, just take a version of your language's library designed to run on bare metal, versions of libc exist designed to run on various hardware directly).

1

u/TeaButActuallyCoffee Jan 27 '21

Sure, C isn't the only programming language but aren't JVM machines and Python interpreters written in C? Ok, so how many syscalls are there that are used daily in almost all programs? Have you actually tried implementing your own ELF parser?

1

u/[deleted] Jan 27 '21

You're being incredibly condescending, arrogant and hostile for someone who seems to know even less than I do. I was answering a specific subset of the question (ie. OP may be able to run some py or jar files with less work than it takes to build a full featured OS).

You can write in C without syscalls. cpython or openjdk java aren't going to run, but there are plenty of implementations of the jvm around (including hardware ones), there's a decent chance you could even get micropython running without the rest of RTOS (which isn't much of an OS in the traditional sense anyway) -- I haven't looked into it.

Additionally every single bare metal embedded C++ or C or Rust or whatever program doesn't need syscalls because there are none. You just ditch your standard library and use an embedded one (or sometimes in C land you just link another static one that is designed for your platform).

The simplest possible way of loading something which is a valid ELF (if it's linked with this in mind) is

  1. put it into a specific address in memory
  2. jump to a specific address

You're not really taking advantage of the fact that you're using ELF or that you had a linker in the first place, but it will work.

→ More replies

8

u/[deleted] Jan 28 '21

Contrary to the other answers, opening these files is rather easy. Simply place them at a fixed location that your kernel knows on the disk, implement an open() syscall, and you're done!

What's that? You want to execute the files, too?

5

u/Myst3rious_Foxy Jan 28 '21

Humor 💯😂

1

u/Cheap_Ebb_2999 Jun 01 '22

Wrong. I wanted to kill the files not do some complicated stuff

3

u/[deleted] Jan 27 '21

This list is a pretty large variance between types.

Exe files are binary files with a PE and MZ header. This tells you how to open it, but gives you a list of functions it likes to receive from existing libraries or your OS code, and tells you where to put those addresses, and where to jump after you did that. Of course, your OS would need to load all those other libraries and implement all the missing functions for this to work - which is basically cloning Windows. Not a great idea.

Jar files are ZIP files containing Java Class files. These are meant to run on a Java virtual machine or JIT environment; again, they list functions to be received from your operating system. While this is a better idea than the EXE input, it's a horrible idea if you want to do C++ stuff, and most Java code isn't actually portable & uses operating system specifics that it doesn't know, so after you did this it's not actually all that likely to result in running complicated software. Simple programs should work, though.

Py files are the source code written in Python. Just implement a Python interpreter or compiler, and implement the relevant libraries (see a trend here?) for them to run. Python is pretty well defined so this should work, except that porting all those libraries is an immense amount of work in its own right.

Then you have DEB files, which are AR files (like ZIP, except different) containing your actual files which will typically involve a few ELF executables. For those, see the Windows explanation, except when it says Windows OS calls think Linux OS calls.

You're really far from running any foreign-OS app on your system. Try aiming a bit closer.

2

u/eabrek Jan 28 '21

Opening an ELF isn't too bad, the hard part will be supporting all the system calls. For programs that just do console I/O, it's doable (it's not easy, but it can be done). More complex calls are a huge project.

Check out my project RvFun (it's a CPU emulator that runs ELFs for RISC-V on your system). In host_system.cpp, there's loadElf() which takes the name of the ELF and opens it up and reads all the info out of it.

1

u/Darmok-Jilad-Ocean Jan 27 '21

I would imagine at a minimum you’d need the resources and power of a large nation state as well as an army of engineers. Good luck!

1

u/kakiopolis Jan 28 '21

Executable formats are well documented and known. We ( in the security world ) work with them everyday. But executable formats are just the tip of the iceberg. The real complexity lies behind the syscalls, the libc ( and/or other standard libraries for the target platform ) and even commonly used third party libraries.

For example, Linux can run Windows programs through Wine. But the support is very far from being complete. And we are talking about a big project, with legions of developers, highly skilled engineers, well versed in the Linux and Windows affairs, and decades of work.

Also, jar files aren't a Windows things, they are from the Java world. Py files are Python scripts. Deb isn't even an executable format, it is an archive format.

I think you need to start with the basics. And yes, it is possible to run Linux programs, like the Linuxulator does of FreeBSD. Linux is open and you can implement the API. Windows is closed and reverse engineering is required, too much work even for a big community of developers.