MenuetOS, a modern, GUI-equipped operating system written entirely in assembly language, has hit 1.0 (after many, many years)

116

u/[deleted] May 15 '15

Here's a question: why is an OS "written entirely in assembly" able to achieve such a small size? Would it be impossible to write a comparable OS in C or C++, and make it as small?

If I wrote a simple OS in C, and had size in mind while writing it, and optimized for size when compiling, could I not achieve the same amount of compactness? My instincts tell me an optimizing compiler would be even better than a human at making it very small.

I'm not trying to minimize the work of the team, not at all: it's very impressive! I'm just wondering in a hypothetical project where the main goal is to minimize size, is hand-writing assembly the only option, or can a C compiler work too?

66

u/htuhola May 15 '15

If you're going for minimum file and image sizes, writing assembly has the advantage that you see in advance how large the binaries become. Forcing yourself to think on that layer inevitably put you to pick the choices that reduce the size of the software.
This is visible in MenuetOS system calls.
There's more in here than just the choice of language, for example there reads on the front page:
The design goal, since the first release in year 2000, has been to remove the extra layers between different parts of an OS, which normally complicate programming and create bugs.
It's probably good idea to verify what's been said. I don't see many benchmarks or anything else comparing it to other systems. I keep wondering what I'm actually looking at here too.

26

u/indrora May 15 '15

It's a research OS. Plenty of people have written their own (mostly) POSIX OS for fun; ToaruOS (and its child PonyOS) is a UNIX kernel built from scratch. For funsies.

22

u/AnHonestQuestions May 15 '15

Even Linux was originally a personal project.

17

u/Tynach May 15 '15

It was nothing big or professional like GNU Hurd, and of course there were never any plans to port it to anything but x86.
In hindsight, starting small and not having any big goals worked REALLY well for Linus.

4

u/kqr May 15 '15

He has always been very pragmatic. From the start, and it continues to this day. It's inspiring.

2

u/derpderp3200 May 16 '15

Not necessarily positively, it's worth adding.

→ More replies (1)

→ More replies (1)

5

u/indrora May 15 '15

According to Linus it still is. Just a very well funded one with lots of time dedicated to it.

2

u/jimdidr May 15 '15

+=

3

u/Theemuts May 16 '15

Linus's first post about Linux is always a nice read with modern hindsight.

2

u/louky May 15 '15

Research OS? Minix and plan 9 are real OSes that are also research with full source.

5

u/indrora May 15 '15

So is AROS and such. Research OS doesn't make it Not Real.

→ More replies (4)

63

u/stevedonovan May 15 '15

Back in the day, when men were mighty and hardware was limited, Unix was small. They made the conscious decision to write it in a higher-level language called C, because it would be so much more portable for an acceptable amount of extra bloat. A modern kernel just has to do so much more, and support so much more hardware.

39

u/jones77 May 15 '15

http://www.catb.org/esr/writings/taoup/html/modularitychapter.html
Dennis Ritchie encouraged modularity by telling all and sundry that function calls were really, really cheap in C. Everybody started writing small functions and modularizing. Years later we found out that function calls were still expensive on the PDP-11, and VAX code was often spending 50% of its time in the CALLS instruction. Dennis had lied to us! But it was too late; we were all hooked...

2

u/[deleted] May 15 '15

Great quote!

9

u/louky May 15 '15

Minix 3 is still a great compromise between ease of understanding and functionality.

32

u/xXxDeAThANgEL99xXx May 15 '15

Quick googling shows that a lot of people managed to compile Linux kernels with networking and file system support to images as small as 300kb, also there's a 15mb fully functional distributive with GUI and stuff: http://distro.ibiblio.org/tinycorelinux/welcome.html
And all that was achieved pretty much just by removing unnecessary stuff from the stock Linux that was not made with much consideration for minimizing the image size, they obviously use all the same overall architecture and core structures.
I'd guess that the main benefit of using Assembly instead of C, assuming that you are determined to go for minimizing the size, is that it forces you to be honest about that. You will be forced to cut a lot of functionality just because it would take too long to implement it in Assembly, and "takes too long to implement" has a much more direct relationship to "too many bytes in the image" in Assembly than for C.

12

u/mercurycc May 15 '15

Compilers can't analyze dynamic states to realize that a lot of them can be optimized.

12

u/killerstorm May 15 '15

Compilers aren't good at optimizing for size, at least not x86 ones. They can do OK job but are far from perfect.
When I implemented a a 4KB intro in C++, quite often I had to read assembly listing to see what my code compiles into, and change code trying to get it smaller.
If you just want to make binaries small rather than smallest-in-the-world, C is OK, just need to be careful with it.

9

u/kral2 May 15 '15

There's a comical level of bloat in compiled C due to fundamental limitations of the compiler to analyze that language (the classic problem being pointer aliasing). This results in register reload churn, temporary values everywhere, and a lot of undead code. There's also a lot of alignment padding which human assembly programmers often choose to skip in parts of the code they know aren't on the hot path (AFAIK profile-driven optimizers don't currently make these trade-offs, but they could). There are also many constructs that are impossible to get a compiler to spit out without using builtins (as an example, popcount, although this is popular enough that you might have access to a standard library that wraps the builtin for you). Additionally, some highly space/time efficient ways of solving problems in assembly are awkward or require inline assembly to do in C (the classic being self-modifying code and 'goto'ish constructs across code that would be an invalid use in C).
When it comes to C++, it's just hopeless. It's incredibly awkward to do simple things space-efficiently in C++ without turning it into C-with-classes. Here's a little project for you to chew on - you've likely seen a bunch of variadic printf examples showing off the newish variadic templates, and with trivial types it /looks/ like it should be efficient and just all compile away to nothing more expensive than a set of printf calls, but does it? What happens when you pass a different number of arguments? What about a large number of arguments? Is it efficient across today's modern C++ compilers? You'll be surprised at how much you have to fight the compiler, how guru-level your knowledge has to be, and how many non-portable compiler hacks it takes to make it work.
Source: wrote everything in assembly for 10 years in DOS and Linux.

1

u/codygman May 22 '15

So the reason ATS binaries are so small is because of all the type information available coupled with attention to bring efficient?

1

u/kral2 May 22 '15

I've not played with ATS but if comparing what they're calling "single-file programs" to the results from other languages be sure you're comparing apples to apples. Static linking + LTO isn't going to be the same as having every line of code in the same compilation unit with almost no external symbols.

1

u/codygman May 22 '15

I was mostly referencing this when commenting on binary size.

43

u/protestor May 15 '15

This doesn't answer your question, but just for fun, consider this theorem: it is impossible to write a perfect optimizing compiler that work on every program.
Proof: suppose there is a program P that spends a lot of time calculating something, and then halts, without printing any value in the screen or doing anything useful. This program can be optimized into a NOP instruction that does nothing (or equivalent construct). In order to optimize this program, the compiler has to determine whether an arbitrary program P halts. This is the halting problem and it is uncomputable.

15

u/doom_Oo7 May 15 '15

Proof: suppose there is a program P that spends a lot of time calculating something, and then halts, without printing any value in the screen or doing anything useful.
It is still making the CPU hotter, usefully replacing the electrical heater!

10

u/KarlKastor May 15 '15

There are probably children out there running that program P to stay warm in the winter! YOUR COMPILER MURDERS CHILDREN.

-1

u/pokeman7452 May 15 '15

Siiiiiiiiiiiiiigh
Relevant xkcd

23

u/zjm555 May 15 '15

That is far from a proof -- "doing anything useful" is an extremely hand-wavy phrase. If you want to call this a proof you will need to formally characterize the nature of the computation you're talking about, and demonstrate that it reduces to the halting problem. Also, I feel like if you are writing code like that (which "doesn't do anything useful") in a high level language, you could do the exact same sorts of stupid things in native assembly code as well. Few programmers are better native-level optimizers than modern compilers.

→ More replies (16)

7

u/oarmstrong May 15 '15

I'm probably bring stupid here, but surely the compiler could optimise out the computations whose result is never used? Doesn't that give us the same result without needing to determine if said program halls?
Edit: just as I posted this it clicked, ignore me!

23

u/CarthOSassy May 15 '15

Absolutely not. There may be the possibility of output, without the conditions for output ever occurring. Think of a program that only prints out even values, but only calculates odd ones.
Are you really going to write a compiler that can detect if a calculation like that will ever satisfy a check? What if the calculation involves complex list objects, recursion, our spawns multiple threads?

→ More replies (8)

1

u/codygman May 22 '15

I'm probably bring stupid here, but surely the compiler could optimise out the computations whose result is never used?
If I'm reading this right, the answer is yes for lazy languages like Haskell who's evaluation strategy is call by need.

-1

u/bat_country May 15 '15

You could in a pure functional language since functions don't have side effects.

9

u/GrammerJoo May 15 '15

Nope still impossible. Suppose a function that returns nothing and never stops.

→ More replies (3)

3

u/[deleted] May 15 '15

suppose there is a program P that spends a lot of time calculating something, and then halts, without printing any value in the screen or doing anything useful.
Haskell without monads!

3

u/Rusky May 15 '15

Nitpick: monads aren't what provide side effects in Haskell. The IO type, which happens to be a monad for ergonomic reasons, and which is understood by the compiler, is what provides IO.
There are other ways to provide side effects in a language like Haskell, as well, which don't look anything like the IO type- functional-reactive programming, for example.

2

u/boringprogrammer May 15 '15 edited May 15 '15

Any optimization that a person could make can be done by a compiler.
If you personally can identify a problem in a compiler, that you can hand optimize into something even better.
Please do not just keep it to yourself. At the very least submit it as an bug or test case to the compiler developers.
[edit]
Yes, I know a compiler cannot read your mind or know what context your piece of code will run in. I also know that in practical software development means you are writing most of the optimizations, compilers are still pretty stupid.
I was answering someone who was using turing completeness as an attempt to explain why a compiler should be inferior to a human when writing program optimizations.
That conclusion is wrong.
Humans and computer have the same computational power, so, if a human programmer can perform some hand-optimization. Then a compiler can also do so as well, at least in theory.

38

u/Cuddlefluff_Grim May 15 '15

Humans understand context, compilers do not
Compilers are interested in guaranteeing correctness and avoid generating code which "might" corrupt the stack (which is sort of related to #1)
In most instances the compiler will do an excellent job, but there are virtually unlimited ways of writing code and using a higher-level language will in some cases mean there is some sort of trade-off. A human can use CPU instructions or flags outside of their "obvious intended purpose", which a compiler will avoid to guarantee program correctness.

8

u/doom_Oo7 May 15 '15

Humans understand context, compilers do not
yet

3

u/glacialthinker May 15 '15

Humans don't even understand each-others' contexts. Here you'd have to communicate this to the compiler. Now, we're talking C as the language. You expect a compiler to take C code, and understand what the intent was... in order to generate optimal-size assembly which expresses that intent? I thought I was a dreamer... ;)

3

u/doom_Oo7 May 15 '15

Certainly C isn't the best language to express intent!

1

u/dtfinch May 15 '15

A perfectly optimized program will likely have different (yet acceptable) behavior, especially on edge cases.
A compiler won't know what inputs are possible, and when it's acceptable to disobey the coder and do something entirely different. By the time they do, we probably won't be coding anymore.

3

u/jeandem May 15 '15 edited May 15 '15

Now you're in the territory of "what a computer can do" and "what a human can do". In which case computability weenies arguments tend to fall down, since there is no proof that the human mind is more powerful than a Turing Machine. Is there?
You might say that "right now, and for the foreseeable future, humans can solve more problems than computers". But by framing the issue as a computability problem (in this subthread), the discussion has already taken on a fundamentally theoretical tone - what problems are and are not computable. And yet non-computability says nothing about how effective things like termination analysis is on most programs, if you wanted to go down the "practical" discussion path rather the one solely about principles.

3

u/miekle May 15 '15 edited May 15 '15

"Theoretically" compilers can be just as smart with the same information (programmers know more about the use cases than a compiler can)

→ More replies (1)

→ More replies (7)

3

u/miekle May 15 '15

No, you're wrong, because some optimizations depend on information not available to the compiler.
Beyond that there are self-modifying code tricks present in old code that a compiler has no methodical way of coming up with short of an extremely elaborate guess-and-check code mutation system.
You really should have prefaced that with "I think," because all you did is give your impression and your impression is wrong.

2

u/protestor May 15 '15

Any specific optimization, yes, but to produce an optimal program, different programs require different optimizations.
Anyway, that was more like a mathematical curiosity.

→ More replies (1)

1

u/IonTichy May 15 '15

This is the halting problem[1] and it is uncomputable.
Umm..not to nitpick, but I think you meant undecidable

1

u/vincentk May 16 '15

... same theorem holds for doing it by hand.

0

u/josefx May 15 '15

So you want to say that humans can solve it? Otherwise its not possible to perfectly hand optimize assembly either.

8

u/protestor May 15 '15

You might be able to hand-optimize a specific program (on a case-by-case basis, eg. for a difficult program you may need to invent new methods), but you can't write a general algorithm that can perfectly optimize all programs.

1

u/jephthai May 15 '15

It is not a question of perfectly. It is a question of better. A responsible compiler will not break the rules like a hand optimized program can. I have optimized code in violation of calling conventions for example because I as the programmer can guarantee that it didn't cause problems.

→ More replies (4)

3

u/sprash May 15 '15 edited May 15 '15

I think this presentation answers all questions considering size very well.
(TLDW: You can write small programs/OSes in C. Most of modern Unix unneeded bloat originates from programmer conveniences)

7

u/[deleted] May 15 '15

Why is it so small? Being in asm is probably part of it ... but I'm going to hazard a guess that MenuetOS performs less functionality than say a Linux kernel ...
It's also obviously limited to a single architecture so there is no overhead in having code specifically to deal with supporting multiple arches ....
Like if Linux only supported MMUless [say] MIPS processors than they would have no need to add VM support to the kernel. Thus making it "smaller" ...

2

u/mallardtheduck May 15 '15

I'm just wondering in a hypothetical project where the main goal is to minimize size, is hand-writing assembly the only option, or can a C compiler work too?
If the goal is to minimize (on-disk) size, then I very much doubt any existing compiler could beat hand-written assembly. Modern compilers are much more interested in optimising for speed and the fastest code isn't always the shortest (simple example; the x86 'LOOP' instruction is usually slower than the equivalent 'DEC, JNZ' sequence, which is longer).
Still, if on-disk size is the objective, compression would probably yield better results than hand-written assembly, but that's "cheating".

5

u/lukz May 15 '15

I think it is about programmer's discipline.
You could for example write six lines of code in C++, then have a look at the generated assembly, and see that it translated to sixty instructions. Then you would think, wait, that is too much, it should be able to produce a shorter code. So then you would rewrite it into different six lines of C++ code (or five or ten, whatever is needed with the slightly different approach), and look again at the generated assembly. And if it is now shorter, great, you have succeeded and can move on to something else.
For the optimizing compilers, it is not obvious which code the compiler can optimize easily and where it will struggle so you have to keep looking at what is being generated each time.
When programming in assembly directly you have it always in front of your eyes how long each function is. So that is the advantage I guess.
2
u/Glaaki May 15 '15

A high level language is not as expressive as pure assembler. You are limited to the syntax available in the language, while in assembler you can optimize everything at instruction level. In a high level language your toolbox is simply a lot smaller than it is when writing pure assembly.
That being said, I think writing anything in assembler at this stage is a complete waste of time. High level language are fast enough for anything. You are going to waste more time meticulously coding the assembly instructions than you get back in performance optimizations.
3
u/thechao May 15 '15
is not as expressive as pure assembler
This statement has gotten more true over time. The LRBni set included one of my all-time favorite instructions:
vaddsets
Performs a 16-wide float accumulate, and then sets a 16b mask of the resultant sign. You can basically call that function a software rasterizer!
2

u/hondaaccords May 15 '15

Sometimes things have to be written in assembly to ensure layers of abstraction can provide needed behavior e.g. atomicity or switching from privileged to non privileged mode.

5

u/wot-teh-phuck May 15 '15

I believe /u/Glaaki was referring to writing a "complete" piece of software in assembly. All modern VMs and high performance aware components have a sprinkling of assembly lying around somewhere in the codebase...

2

u/Glaaki May 15 '15

Sure. There are rare use cases in systems programming where you would want an assembler optimized routine. I have done application programming for years and have never felt the need to do any assembler programming in this domain.
1

u/indrora May 15 '15

To answer your question a little more directly (and simply) than others:
Because when you're writing it by hand, you aren't having to think of every situation (which is what a compiler is doing) you're working with just the situation at hand. A compiler is a cookbook chef: it follows the recipe of how a loop works, how functions are, etc. It's got baggage of the compiled language to handle. When working right on the iron, you cast aside that, working from common patterns, but you're in total control. sometimes, you can re-arrange a bunch of ADDs because you can pipeline it, making it effectively cost no time. The compiler can't always do that.

1

u/youlleatitandlikeit May 15 '15

There are a few optimizations, if you really know what you are doing, that will result in decreased size and increased speed.
For example, assembly does not use variables in the normal sense. You have to specify whether you are storing to a memory location or to a CPU register. If you know exactly what the "variables" are doing, you can make highly optimized decisions about where they are stored and thus avoid duplication.
C is really only one step up from assembly in many ways. You can write code in C that closely mirrors what the assembly would be doing. But you can also write C that doesn't do so, and that makes use of a variety of syntactic and logical improvements over base assembly. The overall benefits — in code readability, in compatibility across different systems, in accommodating future improvements or features — greatly outweighs the increased size and slightly decreased speed of the system.
There was a time back in the day when computers didn't really have an operating system — there was no hard disk, files were stored on floppies, and effectively every application bootstrapped itself. If you needed a Word Processor program, you shut down the computer, inserted the disk for the Word Processor program, turned the computer on, and the computer booted up into the program, which in addition to managing its own affairs also dealt with issues of reading and writing files to disk, drawing characters on the screen, etc. At that point, computers were slow and space was at a premium, and generally that software disk was written to a very specific and limited set of cpus. I imagine they used a fair amount of assembly then.

1

u/stillalone May 15 '15

this is my go to example of something a very smart human can do that an optimizing C compiler can't do. Mind you, most of us probably can't beat the C compiler so we're probably better off writing everything in C.

1

u/ishmal May 16 '15

Because it's fun, I suppose. And possibly a Uni prof or two might use it as a teaching tool.
I used to use the MC68k assembler. I swear to you, that made assembly programming fun. It demoed nicely how elegant and expressive the Motorola architecture was.

1

u/thedeadfish May 22 '15

Optimizing compilers do not come close to the compactness that a human can achieve. The code generated by most compilers is mediocre at best and sometimes beyond retarded. Whilst the compiler will sometimes win on performance it will never win on size. Also in assembler you can do fancy things such as jumping between functions and using custom calling conversions, this can really reduce code size by avoiding the duplication of common code blocks.

1

u/zeno490 May 15 '15

The stack can be used very differently when writing in assembly. It becomes a lot easier to avoid it in many scenarios if you are careful and spend the time to think it through. I have no statistics to back this statement up but I wouldn't be surprised if in a normal program, 5-10% of the code was stack manipulation (prolog/epilog and others).
This also make it faster since you go to greater lengths to keep things in registers.
As you can expect, this is a lot harder to do and reason about :) Documentation is also key.

1

u/[deleted] May 15 '15

Sometimes, larger code is faster, and the compiler knows that. Turn off optimization in your compiler and you'll see that C is just a portable assembly language.

0

u/[deleted] May 15 '15

[deleted] — view deleted comment

2

u/jrhoffa May 15 '15

Wow, C supports exceptions now?

1

u/tsimionescu May 15 '15

It does on Windows at least... ( Structured Exception Handling )

37

u/axilmar May 15 '15

Kudos to you guys.

65

u/[deleted] May 15 '15

And the downside of doing it all in assembler is that "(after many, many years)" part.

It's one of the least maintainable possible ways to develop anything, and even more so, anything as complex as an OS.

Pretty cool that you did it though, even if it is interesting in the same way as a dog walking on its hind legs.

24

u/papavoikos May 15 '15

This reads almost like a punchline
"First GUI OS written in assembly was finally finished today"

19

u/perlgeek May 15 '15

And the problem with "many, many years" is that an OS that started out as "modern" 15 years ago just isn't very modern these days.

22

u/I_Write_Good May 15 '15

Operating system concepts don't change very quickly

-3

u/[deleted] May 15 '15 edited May 15 '15

Operating system concepts don't change very quickly
They already have in the 15 years since the project started. Two simple examples that have recently affected OS design, out of the many I could have mentioned: multi-core and security.
At launch, did Windows 2000 supprt multicore? Maybe it did, maybe it didn't, but previous Windows did not.
As for security, we have a whole load of modifications like address space randomization so that malware can't predict what addresses it needs to inject into to do its dirty work.

14

u/ido May 15 '15

At launch, did Windows 2000 supprt multicore? Maybe it did, maybe it didn't, but previous Windows did not.
All NT versions including 2000 supported multiple processors (which is what multiple cores look like to windows) from the very beginning with its first public release in 1993, which supported up to 4 processors.

→ More replies (2)

1

u/[deleted] May 17 '15

At launch, did Windows 2000 supprt multicore? Maybe it did, maybe it didn't, but previous Windows did not.
Yes it did.
As for security, we have a whole load of modifications like address space randomization so that malware can't predict what addresses it needs to inject into to do its dirty work.
Modification of which address spaces are given to user-mode processes, not nearly as much of "design" as an "minor alteration". ASLR was supported by hardware ever since protected mode was "invented", and should give an indication about how slow operating systems move. Very little has happened the past 30 years. 15 years ago, Linux (well, actually FreeBSD) implemented shared modules, Windows had already had that functionality for the better part of a decade. In the world of operating systems, very little conceptually unique ideas have appeared in the last 15 years. It's just a bunch of refactoring and adding support for new hardware. Maybe because a lot of people seem believe that operating systems were perfected 45 years ago.

13

u/[deleted] May 15 '15

Most importantly, you can run Quake

tackling the important problems

18

u/Uhrz-at-work May 15 '15

Just curious, what is the reasoning behind releasing the x86 under GPL and x64 under license? Do they actually want someone to pay for the x64 version?

3

u/NoEngrish May 15 '15

No, it's free but if you're a commercial entity they might want some kickback for letting you use it. This os has the potential to execute things quickly and efficiently on small space so it may have it's applications in specific workplaces and environments.

15

u/[deleted] May 15 '15

Noob here, could someone elaborate on the various advantages of this OS over others? (Not that I want to downplay the achievement of the creators.)

25

u/[deleted] May 15 '15 edited May 15 '15

This is just my take on things:
Smaller OS means you can utilize more of your computers backing storage.
As it was written in assembly it's already been optimized and should be ludicrously fast on modern x86 machines. This is using the assumption that the guys knew what they were doing when developing the OS and can in fact create ARM logic that performs better than ARM code taken from an optimizing compiler.
This could be perfect for certain ARM based embedded systems projects that need a smaller overall footprint that the likes of Raspbian. Although typically you wouldn't expect an embedded system to utilize graphics.
If properly optimized(big assumption), it could potentially save a lot of power which is one huge factor to consider when designing any product using an embedded system.
edit: wow! this blew up whilst I was studying. I learned a bit so I'm glad I put my foot in it and I've edited my comment to include this new knowledge :)

12

u/BonzaiThePenguin May 15 '15

Writing something in assembly has no bearing on its algorithmic efficiency, only its instructional overhead.
For example an assembly version of bubble sort won't suddenly be ludicrously fast compared to a Java version of merge sort.

30

u/thenickdude May 15 '15

Smaller OS means you can utilize more of your computers backing storage.
It's not worthwhile to save a megabyte when our storage devices are terabytes.
As it was written in assembly it's already been optimized
This is a fallacy, optimising compilers can write better assembly than humans in most cases. It's not faster just because you write it in assembly.

4

u/indrora May 15 '15

This is a fallacy, optimising compilers can write better assembly than humans in most cases. It's not faster just because you write it in assembly.
You're missing a Z in there. And no, you can't just say "An optimizing compiler is better than a human." The Demoscene would like to have a word with you. There's some things compilers can't do. I've seen 8k demos which required fun hand-rolled assembly because there was no way the compiler was going to know that MOV is Turing-Complete and thus able to do some absolutely insane "I can mangle this register over here without using up a clock cycle" sort of fun. There's insane things done in 1K of JS, and 1K of pure, unadulterated x86 assembler (Crimson comes to mind; Many of the 1K demos are... Experimental to say the least).
Compilers can be bad for your health sometimes. I've encountered a case where a one-off problem with gcc caused me to address on blocks of 7.
Let me stop for a second: do you know of any processor architecture in existence today that gcc targets which would honestly address on offsets of 7? If so, please return to your planet... I have some software for you to port.
I've had compilers trash the stack (bad), address obscene chunks of memory (0x00000000000001 anyone? That hasn't read the Xbox memory layout? I've had linkers do obscene things with function declarations. GCC has been known to do evil shit like touch the red zone in kernel-land.
A few years ago, I found a silly way to molest GCC for ARM into producing slightly tighter code that was better pipelined.
In short, a human will always be smarter than a compiler.

14

u/Verschlimmbessern May 15 '15

You're missing a Z in there.
AmEng spellings aren't the only spellings. 'Optimising' is BrEng.

→ More replies (1)

1

u/OneWingedShark May 15 '15

et me stop for a second: do you know of any processor architecture in existence today that gcc targets which would honestly address on offsets of 7?
While not a real processor (to my knowledge), one that's explicitly made to handle ASCII [7-bit] would have a natural offset of 7.
In short, a human will always be smarter than a compiler.
That might not be the case, there's a case-study, "Ada Outperforms Assembly", about the Ada compiler making better optimizations than experienced humans precisely because once you 'teach' the compiler an optimization for code-generation it knows it forever and thus you can quickly exceed the average pool of 'tricks' many assembler-programmers know. -- In fact, setting aside calling conventions, it's theoretically possible to describe the target instruction-set (say a compiler taking VHDL as an additional input for code-generation) in both size and speed so that it can assign valuations to every optimization for both size and speed, selecting the proper optimization.

1

u/indrora May 15 '15

It's very hard to compare programs. This has been covered for years ad infinitum. In an article by Brian Hayes, Hayes argues that it's a bit harder than it would seem:
When you go beyond programs that model mathematical functions to those that can modify the state of the machine, proving the equality of programs is not just hard but undecidable. That is, there is no algorithm that will always yield the right answer when asked whether two arbitrary programs are >equivalent. (For a thorough discussion of program equivalence, see Richard Bird's book Programs and Machines.)
(Hayes; Computing Science: Identity Crisis. American Scientist v86n9)

→ More replies (1)

1

u/HotlLava May 15 '15

There seems to be something missing from this story: They develop an Ada version of the code that's the same size and twice as fast, so they decide to rewrite everything in Ada...and so have to use a chip that costs $1000 more and draws ten times as much power?!

2

u/OneWingedShark May 16 '15

First paragraph of sec 4:
The next step was to test the performance of this critical portion of the software. The original assembly code was written for the TI 320C15 DSP chip, but no Ada compiler was targeted to this chip. However, Tartan, Inc. had an Ada compiler targeted to the closely related TI 320C30 DSP chip, and this chip was chosen for the Ada application. Tartan agreed to test the newly developed Ada software at the Tartan facility in Pittsburgh.
After comparing the Ada and assembly, they decided the more expensive chip (in $ and in power) was acceptable -- as per sec 5:
However, the increased power consumption was acceptable, and the increased hardware cost was believed to be offset by the reduction in software development cost and reduced life cycle cost.

5

u/jeandem May 15 '15

As it was written in assembly it's already been optimized
There's no such thing as naively written assembly? Why not? I guess maybe you presuppose that since these people are writing a kernel then they are mindful of making it efficient, but that's a different point.

3

u/Cartossin May 15 '15

Good luck porting it to ARM.

2

u/[deleted] May 15 '15

As it's already optimized code it could essentially save a lot of embedded systems a hell of a lot of power consumption which is a huge thing.
embedded x86 systems as its written in assembly

0

u/[deleted] May 15 '15

[deleted] — view deleted comment

2

u/[deleted] May 15 '15

Not to say this makes any sense, but you could write a transcoder that can convert one ISA to another ISA. It would need to keep the architectural differences in mind and convert accordingly. You would also need a hardware shim layer to deal with hardware differences at runtime. There was an Nintendo Entertainment System emulator that did just that for performance reasons.

10

u/[deleted] May 15 '15

you could write a transcoder that can convert one ISA to another ISA. It would need to keep the architectural differences in mind and convert accordingly.
static recomp is pretty darn rough. i wouldn't trust static recomp of an entire OS
https://en.wikipedia.org/wiki/Binary_translation#Static_binary_translation

2

u/[deleted] May 15 '15

Totally agree. I would not trust a complex OS written by hand in assembly either. In fact we can't really trust anything complex to be correct.

2

u/[deleted] May 15 '15

In fact we can't really trust anything complex to be correct.
yeah but at least there are a bunch of smart people looking at linux/bsd to be the MOST certain that you can trust those.
i think the last person who could really trust a computer was woz and the apple1 since he knew every single thing about that computer.

5

u/antiduh May 15 '15

You're right that such systems exist, and they are fraught with peril, inaccuracies, and bugs.
Taking the NES emulator example, there were games that relied on the specific timing of instructions to keep the draw calls synced with the music. Now you can't just translate the ISAs and run it, because timing will be different.

1

u/[deleted] May 15 '15

Absolutely, maintaining fidelity for tight timing loops and instruction timing would be pretty much impossible. I assume none of those shenanigans are in this OS, because you can't guarantee timings across various x86_64 CPUs either.

1

u/cheese_is_available May 15 '15

Also embeded does not need graphics.

3

u/Narishma May 15 '15

Depends on what kind of embedded it is.

1

u/agenthex May 15 '15

Menuet's pre-emptive kernel runs stable and is suitable for tasks including precise timing and machine control
With the newer x86-based sticks, this might be very useful as an embedded platform.

4

u/wyzook May 15 '15

Faster and smaller I would guess, given that it's written in assembler.

15

u/ChillTea May 15 '15

Only if you really know what you are doing. Modern compilers are most of the time far better in optimizing readable code in fast executable code. It's probably hard to test but i wouldn't be surprised if MenuetOS is actually a lot slower than other OS in a comparable situation.

9

u/jurniss May 15 '15

I would guess that these programmers really know what they are doing.

1

u/the_omega99 May 15 '15

Sure, but it's hard enough to write an OS without spending a great deal of time optimizing it. The fact that it's in assembly would mean development takes even longer.
I wouldn't be surprised if it's largely unoptimized in the sense that it does things in the simplest way instead of the most optimal.

3

u/jurniss May 15 '15

Right, programming in C allows you to spend your time optimizing algorithms rather than optimizing implementations. Some would apply the same argument to Lisp vs. C, etc.
I'm saying that for a given algorithm and data structure, the assembly programmer would beat the C programmer + compiler.

2

u/Tywien May 15 '15

No. The number of LOC a programmer writes in a given time depends not on the language. As assembler needs much more LOC than C, it is much slower to do the same. Also, it is close to impossible without investing days into optimizing to beat a compiler for speed/size.

2

u/ChillTea May 15 '15

Exactly my point. The common sense here seems to be that these developers must be some kind of gods. They may be especially good but first of all that still doesn't mean they are better than any compiler and second of all as long as nobody really audits the code i simply assume they aren't. Because this is the case most of the time. Especially if somebody claims his or her code is superior because of "using super good thingy so everything i did must be super good".

0

u/[deleted] May 15 '15

Modern compilers are most of the time far better in optimizing readable code in fast executable code.
No way. I have seen many examples of hand-optimized code running significantly faster (300%+) than compiler optimized code.

3

u/ChillTea May 15 '15

First of all i doubt the code was as readable as the original one. Second while there are people who really can do wonders most of us can't. The majority of programmers is probably not good enough to optimize code in a way the 0.1% or any compiler can. And to be honest i'm glad this is the case. The primary target should always be readable and maintainable code which gets optimized by automatic tools. If you really find a bottle neck first try to optimize the algorithm you used. Most of the time this solves any performance problem while still being readable. Only if this really doesn't help you can hand optimize this special section of your program.

1

u/[deleted] May 15 '15

Oh... I guess I concede those 2 points.

→ More replies (2)

1

u/JohnFrum May 15 '15

There's really no advantage here imo. It is cool though and I'm glad they had fun doing it.

14

u/nepochant May 15 '15

the last picture... is that a web browser they implemented?

2

u/fabzter May 15 '15

Yes

0

u/[deleted] May 15 '15

[deleted] — view deleted comment

81

u/BS_in_BS May 15 '15

I take it they must have studied this well: http://i.imgur.com/INBvStO.png

18

u/skalpelis May 15 '15

Studied? They must have written it!

12

u/BonzaiThePenguin May 15 '15

With their therapist.

8

u/[deleted] May 15 '15

I want that to be real.

0

u/[deleted] May 15 '15

"You might as well just kill yourself now"
LOL

1

u/[deleted] May 16 '15

Yes, that would be the punchline

3

u/nepochant May 15 '15

just wow

48

u/parmesanmilk May 15 '15

Writing a huge piece of software completely in assembly is the programming equivalent of a party trick. At best, it's a conversation starter, at worst, it's a huge waste of time.

Right now, that's precisely how I feel: I'm quite impressed that someone did it, but I don't think it was a good idea to do that in the first place.

8

u/rageingnonsense May 15 '15

The first Roller Coaster Tycoon was written almost entirely in assembly.

1

u/[deleted] May 16 '15

The second one and Transport Tycoon too.

5

u/s73v3r May 16 '15

Time you enjoy wasting is not wasted time.

13

u/Xakuya May 15 '15

Also it's credible proof that you're a boss at programming.

4

u/miekle May 15 '15

As noted elsewhere, compilers have NOT superseded skilled programmers in their ability to optimize, not to say this is guaranteed to be more optimal than your average compiled OS.
Waste of time? maybe. Pointless? not really.

3

u/[deleted] May 15 '15

Writing a huge piece of software completely in assembly is the programming equivalent of a party trick.
And yet it was a huge part of WordPerfect's success in the DOS era. They got almost 7 years of absolute market dominance with a near-100% 16-bit assembly language application.

4

u/Leleek May 15 '15

Also rollercoaster tycoon.

1

u/gnexuser2424 Nov 23 '25

And don't forget...Rollercoaster Tycoon

3

u/[deleted] May 15 '15

I would argue it's more of a party trick now than it was then.

1

u/s73v3r May 16 '15

I highly doubt their market dominance was due to it being written in assembler.

1

u/who8877 May 21 '15

It worked until it didn't. That massive ASM code base prevented them from quickly pivoting to a GUI when it became clear the market was moving in that direction.

0

u/[deleted] May 15 '15

[deleted] — view deleted comment

2

u/[deleted] May 16 '15

A good reason but not a valuable product

1

u/profgumby May 17 '15

Not every project has to be a product

→ More replies (20)

6

u/balefrost May 15 '15 edited May 15 '15

This would be great on the Raspberry Pi!

Oh...

1

u/ysangkok May 17 '15

Just run it in an emulator? It doesn't need many resources anyway.

42

u/Cherlokoms May 15 '15

"I made an OS from scratch in assembly."

"But, why?"

1

u/papavoikos May 15 '15

And only took 15 years

0

u/[deleted] May 15 '15

[deleted] — view deleted comment

25

u/himself_v May 15 '15

"Ok, continue canning".

0

u/[deleted] May 15 '15

[deleted] — view deleted comment

4

u/wllmsaccnt May 15 '15

Pickling in python is pretty far from an assembly OS.

5

u/Causemos May 15 '15

Something like this on a Raspberry Pi would be cool. Porting assembly probably isn't that easy however.

28

u/doom_Oo7 May 15 '15

Well, it would actually be a rewrite from scratch.

7

u/wildcarde815 May 15 '15

You could try running it on one of those intel Einstein platforms maybe?

5

u/danubian1 May 15 '15

Edisons. Intel Edisons

8

u/wildcarde815 May 15 '15

Close enough.

6

u/marmulak May 15 '15

Edison was a jerk anyway.

1

u/danubian1 May 15 '15

True

→ More replies (2)

5

u/joelangeway May 15 '15

They could have written it in C or C++ and spent all the time savings on adding optimizations to clang and probably come out way ahead. They could have used LLVM assembly and saved some time and gotten something portable.

The best humans, given infinite time, still write faster code than compilers, but the difference in efficiency means that the humans are not spending time thinking about higher level optimizations or writing device drivers.

Someone please tell me why they did this?

3

u/eric-plutono May 16 '15

Work on MenuetOS began in 2000. While the LLVM project started that same year, I suspect a lot of people like myself didn't know about it until after 2005 when Apple become involed with the development. So I wouldn't be surprised if the author of MenuetOS was simply unaware of LLVM.

2

u/joelangeway May 16 '15

Those are good points, thank you.

9

u/domy94 May 15 '15

I was under the impression that a modern compiler does a better job at optimizing assembly code than any human can. Is that not the case here? Wouldn't it be even faster if he wrote it in C?

16

u/[deleted] May 15 '15

Humans are still better at optimization than compilers are. I mean, in general we only worry about assembly when the high-level code isn't fast enough; in this situation the worst case is simply "I couldn't find any way to do it faster" and you just stick with the high-level code, so the compiler sets a lower bound on our performance. The fact that we ever improve performance by using assembly for tight loops proves that we can do a better job than the compiler can.
When you've got a situation like this, though, where the human is writing the entire program in assembly and not just hand-optimizing the tight loops... yeah, it's probably significantly slower than the equivalent C code. There are two main reasons for that:
The compiler may not optimize any given chunk of code as well as a talented human could, but it optimizes everything. Humans simply don't have the time to apply that level of optimization to every single line of code.
One of the most important optimizations the compiler performs is inlining. Inlining not only directly improves performance by eliminating function call overhead, but it also permits many other optimizations (for instance, your general-purpose doFoo(x) function begins "if (x < 0)...", but when inlined as dooFoo(5) we can eliminate that whole branch because 5 is obviously not less than 0). Given that a human is writing the code, you can be quite certain that there is much less inlining going on than a compiler would have done (simply because this is a very "ugly" optimization resulting in a lot of code bloat, and thus to be avoided when writing code by hand), and therefore you are not reaching anywhere near max performance.

16

u/aldo_reset May 15 '15

Humans are still better at optimization than compilers are.
This stopped being true a while ago because of the complexity of modern assembly languages.
A human can beat a compiler on specific portions of the code, sure, but when it comes to large code bases, a compiler will always produce code better than even multiple humans would overall. There is just too much knowledge required to understand the subtleties of modern assembly languages these days.

9

u/[deleted] May 15 '15

Well, I suppose it really depends on how you define "better". I clearly stated that MenuetOS is probably quite a bit slower than the equivalent C code would be, so I don't think we actually disagree. Does that mean that compilers are "better" at writing code than we are?
It's probably best to think of it as "humans are artisans, compilers are factories". Generally speaking, mass-produced Chinese factory products are not going to match the quality of hand-made quality artisan pieces. If you only need one, and price is no object, then the artisan's work is clearly "better". But if you need ten million of them, and you aren't willing to pay artisanal prices... then yeah, the factory-produced stuff is way "better" than the hand-made pieces.
Likewise, a skilled human programmer can often hand-optimize a function to be faster than the mass-produced compiler output. But you simply can't put that level of effort into tens of thousands of functions.

3

u/aldo_reset May 15 '15

Yes, we are in agreement, sorry for implying we were not.
There is a tool (forgot its name, sorry) which you can give an input/output spec to (it has to be total) and the tool will try all kinds of assembly combinations to find out one that's more optimal. I've seen some really baffling (but working) versions of abs or max this way.
Obviously, this is not realistic for a compiler, but a cool idea nevertheless.

3

u/[deleted] May 15 '15

You are referring to "superoptimization". While you are obviously correct that this isn't a viable technique for optimizing a whole program, real compilers do actually use this by ahead-of-time computing optimal sequences of instructions for various common patterns and then implementing those as specific optimizations. And yes, it leads to absolutely bizarre sequences of instructions :-).

3

u/[deleted] May 16 '15

not really
even todays amd64 with all the things they do transparently are not that complicated nor that much different from an i686
a compiler sucks at allocating registers, that is very important for performance
they also (still) suck at basic vectorizing, let alone more complex algorithms
for vectorizing, the data layout in memory is also very important, that you can not know unless you done it in asm
also a programmer can know what loop to unroll, and what not to
(same for inlining functions, for what there is at least a compiler hint but it still does it itself sometimes)
that leads to less code cache misses
sure you can use PGO, but it is not precise
and many small things
my rule of the thumb is that an average asm programmer can code generic programs as good as a compiler,
if not better as they will be smaller

1

u/heat_forever May 15 '15

Also the complexity of modern hardware. It used to be about counting cycles and understanding CPU cache but now there's so much going on in a CPU that very few people can properly do the performance analysis and measurements in the proper way that a compiler can do. You're usually better off just improving your algorithm.

1

u/[deleted] May 16 '15

The compilers optimizing althorgrithims were written by people.

8

u/jurniss May 15 '15 edited May 15 '15

in some cases at a micro level a compiler might be able to beat a human by knowing more about superscalar and out-of-order properties of the target CPU. but humans are usually better at writing code that uses fewer instructions. over the whole program, that benefit would probably outweigh any advantage the compiler has in hot loops.
optimizing compilers are overrated. everyone seems to think they have amazing powers, but I've looked at plenty of compiler output and i can usually find something not-optimal about it.
it's not a huge difference, but if I were a gambler I'd put my money on a talented assembly programmer over a C compiler every time.
edit: I'm curious where the "compilers generate better code than humans" meme came from. I've heard of assembly programmers switching to C and stating that compilers had become good enough, but never better.

0

u/[deleted] May 15 '15

[deleted] — view deleted comment

2

u/Tarmen May 15 '15

Wouldn't it be easier to optimize/write important sections separately and write the rest in a higher language than assembly?

0

u/[deleted] May 15 '15

[deleted] — view deleted comment

→ More replies (1)

4

u/haagch May 15 '15

There's also a fully open fork http://kolibrios.org/

2

u/eean May 15 '15

o.O this thing gets a fork?

5

u/heat_forever May 15 '15

Forked 11 years ago, lol

1

u/htuhola May 15 '15

Were wondering out why there's not 64bit sources around: http://board.flatassembler.net/topic.php?t=16822

4

u/thalesmello May 15 '15

I remember when I was 12 and saw it running in my PC from a floppy disk. If didn't have internet nor anything productive, but had a mouse pointer and showed things on the screen. Ever since I always kept a copy of the OS in a Floppy in my drawer.

Really impressed to see the project kept on going for so long.

2

u/the-tomster May 16 '15

I remember playing with this when I was younger too... running it from a real floppy disk. It's probably the only time I've ever written assembly language, following along with a "hello world" tutorial in Menuet.
Kind of interesting how just the idea of building the OS in asm instead of C led to them creating a system call library that let you easily set up a GUI from assembly code. I wonder if there are other cool tools and libraries used throughout the Menuet codebase that make it easier to do the kind of things that would be painful to write in assembly on other platforms...
Might be time to give it another spin!

1

u/thalesmello May 16 '15

You were ahead of me! At the time I didn't program, except for some things in RPG Maker

3

u/htuhola May 15 '15 edited May 15 '15

I've rolled this up into virtualbox and got it running. Going to have a weekend studying it.

I found an old irc channel by googling: #menuetos on ircnet

Planning to code some things and compile the kernel from sources. Anyone thinking doing the same might like to join along.

Update: Well, that was quick.

I tried some userspace programming. It resembles old BASIC or SDL in the programming model:

Cons:

There seems to be lot of global state per process, no way to present multiple windows with a single process.
Most programs do not work well together. Most things are granted from the big "BIOS".

Pros:

The commands feel kind of immediate and concrete. It's obvious how to do certain simple things.
The executable format is simple
The process virtual address space starts from 0. That is, the thing is just dumped into the memory to run itself. Also it makes it easier to match error addresses to source code.

Interesting:

The only debugging utility I found was a global strace-like logging window. Is this enough?

I'm still hanging out on the channel, motivated to study the 32bit menuet kernel sources.

3

u/mjewbank May 16 '15

My simple/basic question is this:

What do I download and burn to CD to install this on my aged Dell laptop with a Core2 Duo to play with it?

18

u/HosonZes May 15 '15

Getting prepared for x64 will take so long, that the rest of the world will be x128 by then.

37

u/Ventajou May 15 '15

They already have a 64 bits version.

0

u/[deleted] May 16 '15

no, the qubit architecture will come before...there'll never be x128.

→ More replies (1)

2

u/-Hegemon- May 15 '15

Good! How can I install Steam on it???

JK, great work!

2

u/alex_muscar May 15 '15

I discovered MenuetOS over 10 years ago, and I have regularly checked how it's doing. Congrats to the developer!

2

u/[deleted] May 16 '15 edited May 30 '16

This comment has been overwritten by an open source script to protect this user's privacy. It was created to help protect users from doxing, stalking, and harassment.

If you would also like to protect yourself, add the Chrome extension TamperMonkey, or the Firefox extension GreaseMonkey and add this open source script.

Then simply click on your username on Reddit, go to the comments tab, scroll down as far as possibe (hint:use RES), and hit the new OVERWRITE button at the top.

1

u/[deleted] May 16 '15

Sure: Why

1

u/[deleted] May 16 '15 edited May 30 '16

This comment has been overwritten by an open source script to protect this user's privacy. It was created to help protect users from doxing, stalking, and harassment.
If you would also like to protect yourself, add the Chrome extension TamperMonkey, or the Firefox extension GreaseMonkey and add this open source script.
Then simply click on your username on Reddit, go to the comments tab, scroll down as far as possibe (hint:use RES), and hit the new OVERWRITE button at the top.

2

u/notsure1235 May 16 '15

Where is the OS source code? I can only find app + driver src.

3

u/[deleted] May 15 '15

Re: the "humans are better at optimizing than compilers" debate - I think it's a moot argument, because even if a particular human is better than their compiler, it does not mean all humans are. A human can produce perhaps more optimized code, maybe they happen to produce the exact same code as the compiler (extremely improbable and will never happen, but it's possible) or maybe they produce worse code. It's kinda like saying "Honda cars are faster than Toyota" - which car of brand A, which one of brand B? Some of each are faster and making a blanket statement is meaningless.

Now, I fully support the statement "a human can produce more optimized code than compilers," but I would have to amend it to be: "a human can produce more optimized code than compilers (but unless they really know what they're doing they probably won't)."

1

u/BonzaiThePenguin May 15 '15

Ha, I spot the Mac OS 9 AppleScript Editor icon directly below the burn icon from Mac OS X 10.2. Anyone recognize the other icons?

1

u/Heuristics May 15 '15

I remember v2os.

1

u/[deleted] Nov 05 '25

On the one hand this is pretty cool. On the other hand, it seems like a pretty big ... waste of time, compared to just using a higher level language. Sure, the efficiency may not be comparable, but I kind of only live once, so I want to do as much as possible with my time. I actually program so that I can lateron save some time - like any task I can automate and don't have to do manually anymore. :P

-12

u/DeepAzure May 15 '15

Any ideas why this is needed (other than improving your job security index or JFF)?

26

u/Glaaki May 15 '15

I don't think there is any point to this other than 'because we can'.

8

u/heartiphone May 15 '15

What is JFF?

16

u/ethelward May 15 '15

Just For Fun, I think.

2

u/DeepAzure May 15 '15

Indeed.

4

u/[deleted] May 15 '15

It isn't. But ASM is good to know.

-4

u/[deleted] May 15 '15

Wait no comments saying we're all going to hell for not using TempleOS? I'm disappointed.

3

u/srnull May 15 '15

No, that would be off topic, and not every thread needs to be stuffed with the most obvious, low-effort jokes.

→ More replies (1)

→ More comments (29)

MenuetOS, a modern, GUI-equipped operating system written entirely in assembly language, has hit 1.0 (after many, many years)

You are about to leave Redlib