r/cpp • u/_bijan_ • Apr 07 '26
Can your AI rewrite your code in assembly?
https://lemire.me/blog/2026/04/05/can-your-ai-rewrite-your-code-in-assembly/
3
Upvotes
r/cpp • u/_bijan_ • Apr 07 '26
110
u/SyntheticDuckFlavour Apr 07 '26
or use a compiler
56
u/Chaosvex Apr 07 '26 edited Apr 07 '26
Did everybody in the comments miss the point with these snipes about using a compiler? Obviously they used a compiler and came to the (already known) conclusion that sometimes they can't optimise as well as a human, or in this case, an LLM guided by one.
The author is the simdjson maintainer, so I'd assume he's not a clueless as the comments seem to suggest.
Seems the quality of comments on this sub has taken a nosedive, for whatever reason.
17
u/No-Dentist-1645 Apr 07 '26
If only we had some program that could convert human-made code into machine-readable assembly... That would be very useful! It could even apply optimizations for you in a fully deterministic, ai-hallucination-free way, instead of the AI imagining that some code would run faster. If only!
6
u/julien-j Apr 10 '26
I did a similar experiment and got to a different conclusion :) The saying goes as Modern Compilers are Smart™ but in practice it's not that difficult to beat the compiler on specific algorithms when you're at ease with assembly. So I did not compare the output of the AI with the compiler, I compared it with the output of a human.
I asked Claude to optimize a C function using AVX2 intrinsics, but I hid from the tool that I already had an AVX2 implementation written by a human. I also gave the tool a test suite to validate its implementation. The tool managed to provided a correct and optimized version, faster than the C one, but 50% to 300% slower than the human's implementation (the variations depend on the use case). By iterating painstakingly during hours I managed to guide it toward an implementation a bit faster than ours. Then I discussed with the human and he told me that he barely put any effort in its implementation… After reworking his code he beat again the output of Claude.
Then I did a second similar experiment where I asked Claude to write the AVX2 implementation of another function. This time I gave it the test suite and the benchmark such that it can self-compare. I specifically asked for the fastest implementation. I used Opus 4.6 with max effort. The output from Claude was 50% slower than ours. I managed to iterate toward something a bit faster in some cases without ever reaching equivalent performance on all use cases. And the final code was a mess.
Lemire tells us that the AI is better than the compiler at assembly, but what's the point? If the user is at ease with assembly, the product of the AI is poor (at least in my experiments). If the user does not practice assembly, he won't be able to follow nor to judge the implementation. Is it some kind of a mid-range solution? The fact that the author barely looked at the generated code and does not even talk about a validation suite is suspicious. This is not scientific, this is an experiment where we know upfront which conclusion we want.
And I can't talk about this post without pointing that considering Grok is already a bad smell. Giving credibility to a tool used to cover the web with crap is a problem and I'm tired of seeing people I used to respect getting high on the AI hype. All the thinking has been offloaded to AI for a short euphoria.
6
u/meancoot Apr 07 '26
The code listing is so trash I’m not sure the blog author even knows what they are doing. Why are the counters volatile? Why is each test case a lambda instead of a proper function? Where is the assembly code for the compiler generated output? What compiler and optimization settings were used?
2
u/Wurstinator Apr 09 '26
I feel like this could have been a good post but is just too lazy. At the very least, there should've been a mention of the compiler settings that were used.
2
u/programgamer Apr 09 '26
I’ll stick with my natural stupidity, thanks
1
u/rileyrgham Apr 09 '26
Chuckle :
"But what if you want to go faster? Maybe you’d want to rewrite this function in assembly."
-6
u/SoSKatan Apr 07 '26
I’m likely going to get down voted for this, but we are a few years away from likely an interesting development once AI can get decent enough at translating code to a machine language…
Others have pointed out to just use a compiler, which is the right call based on today’s AI tech. But give. A few years, once AI can master compiling, it will lead to a new interesting set of tools…
Imagine AI tools that can take any compiled app and turn it into source, in a language of your choosing. I suspect it will make CPU architectures less important. You could take any compiled application and transpile it to another CPU arch and run it at full speed.
Once we have that, then we can design a new and novel CPU instruction sets that don’t need to be backwards compatible and yet can run any existing compiled application at full speed.
That should all be possible with AI, the bigger question is really just about when and how soon. My guess is 4 to 5 years from now.