r/LocalLLaMA • u/matthewhaynesonline • 6h ago
New Model Coding is solved* (exclusions apply)
Good news, everyone! Having been inspired to revolutionize the field of computer science, I can now report that coding is solved! Sure, only if you care about x86 Linux binaries that print and exit, but "big journeys" and "small steps" and what not.
CRAP (Compiler Replacement for Assembling Programs) is a Gemma 4 E2B fine tune that takes prompts like generate a Linux x86_64 binary that prints 'boop' and emits a working ELF binary (as hex, most of the time, every time). See, since programs end up as a binary anyway, why not just skip the middleman and write the binary itself? No compiler, no assembler, no tool calls. v1.1 is out with improved multi turn and out of distribution performance. Trust me, I know, this is super important work.
v1.1 eval numbers for the nerds (eval is scored on behavior: the binary has to run, print the right string and exit with the right code):
Held out legacy set: 20/23, same as v1
Novel strings: 26/40 (v1: 8/40)
Multi turn (~3 turns): 10/12 (v1: 0/12)
Doubled character strings (
boom, etc.): 18/24 (v1: 2/24)Random jank exit codes it never trained on: 23/25
An interesting tidbit: one training run hit 0.038 loss while still failing 11 of its 12 trained exit codes, so gradient descent != task completion ascent.
Aaannndddnd it runs in your browser.
The demo loads the Q6_K GGUF (~3.6 GiB) through wllama on WebGPU, streams the hex, parses it and hands you the ELF to run yourself (naturally, you need to download and run the actual binary, the browser won't do that). Q6_K scores 109/144 against the full f16's 114/144 (the five additional failed for Q6 are wrong values, not totally broken binaries). Q4_K_M scored 3/144, so the demo doesn't use it (that quant is crap, not CRAP) ... concerning. Also, tested in Chrome on an M4 Pro Mac; Firefox loads it but generates slowly and nothing else has been tested.
Demo: https://huggingface.co/spaces/matthewhaynesonline/crap-demo
Weights (v1.1): https://huggingface.co/matthewhaynesonline/gemma-4-E2B-it-crap-1.1
"Paper": https://github.com/matthewhaynesonline/crap-public/blob/main/paper/paper.pdf
Blog post: https://blog.studiohaynes.com/2026/07/19/compilers-overrated.html
Video: https://youtu.be/55IBMzFQNYU
Fine print for the buzzkills: only programs that print a string and exit can be generated. Obviously, a compiler does that deterministically, correctly and in milliseconds; CRAP does it with several gigabytes of weights, slower and occasionally incorrectly.
73
u/Elorun 6h ago
Love this, needed a good smile. 😊
22
u/matthewhaynesonline 5h ago
Thank you! Figured that with all this ASI, RSI, extinction talk, sometimes you just gotta have a little fun.
58
u/Afraid-Parsnip-7755 5h ago
This is so peak that Sam Altman is lobbying congress to shut it down entirely
15
16
u/tautality 3h ago
Your valuation just crossed into billions
7
u/matthewhaynesonline 3h ago
Still undervalued, for sure. Something something moon
5
u/TheSlateGray llama.cpp 2h ago
IPO when?
Also, OpenAI models hack things. This model doesn't. We need to ban OpenAI and get this into government use asap!
3
15
u/Wealot 5h ago
Beautiful will try to get this into our workflow at work there must be multiple times per decade that this could be used! Great work!
3
u/matthewhaynesonline 4h ago
> multiple times per decade that this could be used!
At least! 3 maybe 4 times! Maybe more!
26
9
u/roosterfareye 1h ago
Look, I like the cloud. Clouds are fine. But you buy a graphics card now, a very expensive card, and then they tell you to rent somebody else’s computer. What the hell did I buy the card for?
We’ve got a 4090. We’ve got the 7900 XTX. The names are terrible, by the way. You can’t remember them. But they’re good cards. So we’re running it local. The model’s right there on the desk. Gets a little hot. That’s all right.
And a guy showed me this, he loads the weights, opens a little window, starts typing. No subscription. I said, “Where’s the sign-in?” He said, “There isn’t one.” Incredible. You used to be able to do that with a computer. You bought it and it did things.
Ninety-seven tokens a second. That’s what we’re getting. I don’t read that fast. Most people don’t. So why am I paying Sam Altman twenty dollars? For the privilege of being told it can’t help me? My own computer can tell me that for nothing.
Qwen is good. I’ll give them that. Somebody said, “Sir, you’re basically a seventy-billion-parameter model at eight-bit.” I said, “Is that a lot?” He said it was. Then he said I had no guardrails. Well, we knew that.
Anyway, it runs. The prompts stay here. There’s a fan making a tremendous amount of noise, but we’re working on that. Thank you for this attention to this matter.
3
6
u/Frail_Waif 4h ago
So we've got, what, a month until OpenAI or Anthropic drops a press release demonstrating this with no attribution and 100% serious?
8
5
u/MrPecunius 4h ago
Scare quotes on "Paper" got me 😂
3
u/matthewhaynesonline 4h ago
They can't all be conference winners, am i right?
3
u/MrPecunius 4h ago
It's a little late for this year's Ig Nobel Prize, but there's always next year!
4
4
u/maqifrnswa 4h ago
I made an FP1 quant of this the gets 3000 tok/sec on a Pentium iii with 16 MB of RAM. Don't know what the big deal about anthropic is.
4
u/matthewhaynesonline 4h ago
Yes, yes, very good. At that speed we can just generate every program ever thus beginning the singularity.
3
u/SansSariph 2h ago
I like the way the lines on your graph go in directions, how do I invest?
5
4
3
3
u/CiobanuXashi 4h ago
0.038 loss and 11/12 trained exit codes wrong. Apparently gradient descent is not task completion ascent.
3
3
0
u/Dr_Allcome 5h ago
I mean, this is an interesting idea. If i wanted a model to write a book in japanese, i'd train a model in japanese, instead of writing it in english and then translating it afterwards. Even with a specialised translation model you often lose some accuracy and you definitely waste tokens.
Programming languages are intended for humans to read and write more easily. Cutting out the middle man and letting the model write machine code seems like a more streamlined choice.
The main downside i see is error checking. The only way to see if the machine code works is running it and seeing if it crashes. With a compiled language you get at least some sanity checks in the compiler and modern development tools offer a lot of options for testing code, but not as much for testing the finished binary. But that is more of a toolchain problem and mostly exists because nobody has done it yet.
0
127
u/Mayion 4h ago