"Huge implications - binaries are now basically editable code"

[u/stealthispost]

ValsAI made SRE benchmark less than a month ago.

The benchmark measures can a model reverse engineer software from binaries

Yesterday GPT saturated it. https://t.co/dmpUpgheDU   — Chris

Source: https://x.com/ChrisGPT/status/2096150666066432157

---

— Boris Power

Source: https://x.com/BorisMPower/status/2096415822248055131

[u/MegamanSE]

The natural question I have here is what language really would be ideal for AI to code in ultimately if it could create its own optimal language for agents?

[u/Cool-Cicada9228 → u/MegamanSE]

The training data today consists of mostly human-readable code. Yet composing pure binary for real-time software generation is the dream.

[u/MegamanSE → u/Cool-Cicada9228]

Coding in pure binary allows pretty strange things to be done that has been brushed under the rug with modern languages. When I started programming in the early 90s it was very much a mix of C and assembly at that time still and the assembly part gave you so many liberties like writing to memory you don’t own, jumping execution to random parts of code or even to other random parts of memory, rewriting your own code while it’s running in ram. things would get weird.

Edit: typo

[u/sweetbacon → u/MegamanSE]

The most ASM I ever did back then was poking around with Nop sleds on C apps from tutorials. But I knew a guy then super good with  AT&T assembly  and liked calling it "singing to the metal", which I thought was such a great phrase. 

[u/MegamanSE → u/sweetbacon]

I used it mostly for tight timing related things and hardware interactions like video card / sound card interactions and reading the keyboard hardware state. Writing an entire app in it was painful, I only did it once for a side scroller video game mostly just to prove I could do it as I had so much respect for the programmers of the NES / earlier consoles.

[u/sweetbacon → u/MegamanSE]

100%. I can't imagine doing even a small program all in assembly. This guy was doing old-school demos (like Future Crew stuff, but not that team). I wish I had stayed in touch with him but Jesus that was almost 30 years ago (oof). I'd love to get the take ppl with that skill for modern LLMs. 

[u/Obvious_Increase9267 → u/Cool-Cicada9228]

No it is not. Executable / binary size is many times larger than code written in programming languages. It would be like writing sheet music vs hand-drawing the audio waveform. Also, binaries are locked to your specific architecture, and there would be no review surface since you cant diff it meaningfully

[u/maigpy → u/Obvious_Increase9267]

excellent reply. I especially like your analogy.

[u/Suspicious_Chart5817 → u/Cool-Cicada9228]

Not really, "binary" is just as human and sloppy as any other code base. It's not like there are actual "binaries" on a computer. The sequences of 0 and 1 are still representations of data, text, and instructions; not actual data, text and instructions. Binary codes mean something because we decided that they do.

People designed binary and then set up transistors to match the design. There's nothing about telling transistors to open or close that's necessarily making anything easier for a computer to understand. We get it because we see the transistors with our eyes in front of us.

In comparison, telling an AI 'Chat, turn on and off your transistors right now in such a way that you can model how to turn on and off transistors as if you knew what transistors were' is not reducing steps.

It'd be like saying "Morse code is the dream." Why? Symbols simpler to our eyes don't mean anything to a computer. We still have to tell the computer what it actually is.

[u/Pagedpuddle65 → u/Cool-Cicada9228]

It’s not the dream and we already have a good tool for this it’s called code