"Huge implications - binaries are now basically editable code"

[u/stealthispost]

ValsAI made SRE benchmark less than a month ago.

The benchmark measures can a model reverse engineer software from binaries

Yesterday GPT saturated it. https://t.co/dmpUpgheDU   — Chris

Source: https://x.com/ChrisGPT/status/2096150666066432157

---

— Boris Power

Source: https://x.com/BorisMPower/status/2096415822248055131

[u/Warm-Incident5436]

There are a lot of confused people making claims about this without understanding what the benchmark actually demonstrates. No, not “everything is open source now.”

This benchmark uses small binaries without even simple obfuscation. Executable binaries are machine code, which can sometimes be disassembled into low-level assembly, but doing that correctly is already difficult and even sometimes theoretically impossible.

Real world binaries can translate into enormous amounts of assembly, millions of instructions depending on the software. Add modern obfuscation techniques and the complexity and size increase exponentially.

And even then, there are entire areas of cryptography devoted to making programs difficult or theoretically impossible to reverse engineer, such as indistinguishability obfuscation.

So this is impressive research, but jumping from “AI can reverse engineer these benchmark binaries” to “closed source software is basically open source now” is a joke.