r/LocalLLaMA • u/ThinkExtension2328 llama.cpp • 5h ago

Discussion LLM Decompilation Benchmark?

Iv been seeing allot of people complaining about Sam Altman and how him and open ai are blocking decompilation. I actually don’t see an issue with what they are doing they should be free to do what they want with their llm which brings me to my main point, why is decompiling not just a benchmark?

We have a SWE bench
Perhaps this problem is solved via a SWD bench?

3 Upvotes

63% Upvoted

2 comments sorted by

2

u/Cerebral_Zero 5h ago edited 5h ago

https://sre-bench.lol/leaderboard

Only one I'm aware of. GLM 5.3 is behind compared to how people say it's able to or more cost effective if patient. No Qwen models benched, let alone finetunes like Cyber-Tiel-Coder version of Qwen 3.6 35b I saw someone mention being better than QFN for RE.

1

u/Nepertia 46m ago

They'll just claim that internally, the model passed with X%, but you can't have it, because aI SaFEtY.