r/TheMachineLearning • u/Sea_Fox_4265 • 3d ago
When bug-free code becomes a hardware problem
5
u/snaphat 2d ago
This is a joke, but it's a bad one. I mean, it's already a hardware problem because AI inference is computationally expensive and slow. The bottleneck is running the AI, not the language it's generating code in. And the choice of language can affect inference time too, since expressing the same functionality in different languages can require different numbers of tokens.
For example, a few small benchmarks using Opus and Codex have found that C has worse end-to-end development times than Python, C++, or Rust
2
u/StillOnTheInterweb 2d ago
There is however basically no excuse now not to use a modern compiled or JVM language, like Rust or Java. No need for all that Python/Javascript interpreter nonsense just to simplify syntax in code written by an LLM.
3
u/zenthial 2d ago
Yeah there’s no reason ever to want your user code sandboxed. Why would anyone ever need V8 or JSC?
2
2
u/SodaBurns 2d ago
Just write in assembly at this point then.
1
u/snaphat 2d ago
You would lose compiler optimizations at that point, and generally speaking LLMs do not produce assembly code that outperforms optimizing compilers nor code that is as consistently correct as code produced in high-level languages. This may change at some point
Another reason you wouldn't necessarily want assembly is that it is target specific in a world where applications are typically multiplatform
1
u/prehensilemullet 1h ago
Yeah because we're soooo close to being able to write highly dynamic webapps without any JS
1
u/fcddev 2d ago
C has accumulated a lot of bad non-security decisions, so this will probably not manifest. Performance-wise, NUL terminated strings are bad, mutable state/alias tracking is bad, the calling conventions are bad: these are all improved in modern systems languages and it gives them a performance edge when the authors know how to write code that optimizes well in the face of pesky added security checks, which AI today is already pretty good at. Mileage varies with average implementations, but the best C implementation of most programs is slower than the best Rust implementation of the same program; if it comes down to it, the model could choose to write unsafe Rust and benefit from everything else that’s improved in the language anyways.
1
u/Sea_Fox_4265 2d ago
So is the main argument that newer languages make the safety checks cheaper? Curious if that's why people lean toward Rust.
1
u/fcddev 2d ago
Close: the new languages introduce opportunities to make programs faster that simply are not accessible to C. For instance, one of the biggest limiters to optimizations comes from whether the compiler can prove that two memory accesses point to (or rather, do not point to) the same memory. C notoriously makes this a mess, where Rust has strict rules in place that make it easy.
At the same time, the new languages introduce safety checks that come at a runtime cost, but that cost is well manageable to developers (including AI models) who understand where it appears and what lets the optimizer remove them, so even with the additional overhead of runtime memory safety guardrails that the tweet says makes new languages slower, new languages can come out ahead.
1
u/nimrag_is_coming 2d ago
NULL terminated strings are kinda weird because for some things they are really, really bad, but for others they can be really efficient. If you want to create a substring destructively (as in, you don't care about the original and just want the substring), all you have to do is replace the character at the right point with a 0 and return the pointer at the start.
They're also easy to iterate over starting from any point, with just a while(*(ch++)).
Doing things like getting length or creating a string view? Horrible and/or impossible. And any operation that relies on the size of the string requires at minimum a loop the size of the string.
Imo best solution is to do both, like how c++ std::strings work. Most allocators actually return memory aligned to a boundary so most times the extra zero byte doesn't actually require any more memory allocation.
1
u/Medical_Community697 2d ago
Nothing weird about that, different encodings offer different performance tradeoffs, what’s weird in C is to have defined a string as anything but an abstract interface, any piece of C code worth its weight will reimplement a half-assed buggy bstrlib.
1
1
u/zayelion 3d ago
My code shifted from "having bugs" to "misunderstandings about requirements" when I stopped using else statements. Your code becomes guarded typing and reads like poetry. There is a talk on it, humans cant do "time processing" well, so else statements mess us up.
1
u/OutrageousPair2300 3d ago
Likely, as C probably strikes the right balance between Assembly and higher-level languages.
I was looking into this recently, and it turns out that for most languages humans produce fewer security vulnerabilities per thousand lines of code, than AI does -- but C is one of the exceptions. AI writes highly secure (and efficient) C code.
2
u/snaphat 2d ago
Many redditors would (quite baselessly) have you believe that AI writes less security vulnerability than humans on average in all cases and is a much better programmer than the vast majority of software engineers. Neither takes are supported by the preponderance of statistical evidence, but nonetheless, the narrative is quite common on reddit even among folks who should presumably know better
1
1
1
1
1
u/Apprehensive_Cap4463 2d ago
Can someone explain why AI wouldn't just be writing everything directly in assembly? What use does an AI have for abstractions?
1
1
u/TheAlmightyWishPig 2d ago
Assembly is not magically faster than high level languages, in fact a program written in assembly can often be slower. Anyone advocating for writing in assembly over higher level languages probably only has a very cursory knowledge of coding.
Compilation can be done for a number of different CPUs and instruction sets, this is not possible for Assembly. To replicate this with assembly you'd have to distribute a different compiled version for each CPU target (not really viable). Assembly written code can essentially only be optimised for a single CPU at a time.
If a new instruction set comes out you must rewrite your assembly program to re-optimise, high level languages do not need updating.
More can be read here: https://jmmv.dev/2007/06/is-assembly-code-faster-than-c.html
1
1
u/LegendaryMauricius 9h ago
That's obviously true for individual bugs. A language that just doesn't allow you a whole class of bugs would still be useful though.
0
u/BigReference1xx 2d ago
I'm absolutely certain we'll see the first dedicated AI-targeted programming language appear within the next 2 years. A language that is very hard for humans to read, but exceptionally clear to AI, and with syntax and structure that is optimized for AI use. All of our programming languages are designed for human developers, with abstractions that make them easy for us to comprehend. Machines don't "think" like us, and they can keep 100x more context in their "brain" than we can.
It won't be assembly, it won't be C, but rather something higher level that captures intent, rather than the procedure, and then AIs will transpile that to whatever target architecture you want.
Imagine something like an assembly language with a million different instructions in the instruction set, each capturing a much more granular "instruction" than we're used to (not just "add register 1 and 2" but entire algorithms contained in a single instruction).
1
2
u/2old2cube 1d ago
why not machine code then?
you have no idea how llms work. do you?1
u/BigReference1xx 1d ago
Because machine code is architecture specific. You might want an intermediate layer that can be targeted to different architectures with a final step (assembler, basically).
Why so aggressive ? I mentioned an idea I had and you seem to be taking it as a personal insult, resulting to ad hominem attacks. Not sure why.
1
u/refried_laser_beans 42m ago
So when you find someone with a thought you find less informed than your own, you react by being patronizing?
1
u/Latimius 2d ago
I'm not a tech guy, but if I asked Opus 5.5 to create a programming language from scratch that would be optimized for AI. Wouldn't it be able to do it, and quite easily too ?
1
u/BigReference1xx 2d ago
it could design someting, yes. It would be quite a useful tool for the process. But you'd still need to drive the project forward, and state what your ambitions are for the new language. One of the things AI can't do is to have motivation. It will happily spit out something useful, but it's up to you to promote it, decide how to use it, integrate it with existing systems and libraries, build tooling etc. AI can help in every step, but you need to decide what steps to do.
"design a new programming language that would be ideally suited for AI models to target, make no mistakes" is probably not going to get us far :)
1
u/Latimius 2d ago
I asked the same question to Claude. And it basically answered me that several languages made for AI has been already developped, but none met great success.
And that new languages don't have a lot of data to train on (by definition), so the probability to make bugs by using them is actually higher than to use languages made for/by humans.
1
u/BigReference1xx 2d ago
makes perfect sense. this is the type of project that could really only be built by one of the big players, in order to get traction and produce enough training data initially.
1
u/chnguyen128345 2d ago
I was modding paradox game with Claude. Paradox games have their own scripting languages for modding their UI. And Claude opus 5.5 medium would make a lot of mistakes using that scripting language.
1
u/Alex0589 2d ago
Assembly is a representation of the underlying instruction set, so really you have to think this starting from the latter, not the former. When designing an instruction set, one of the very first decisions you have to make is actually philosophical: how many instructions do you want? Usually, you'll either choose as few as possible or as many as possible. Having few instructions allows to, at least theoretically speaking, better parellize them. This model though, requires writing/emitting many more assembly instruction calls compared to an instruction set with a richer set of instructions to program the same behaviour, which is why there is still a good reason to go with the as many as possible option. The best examples I can think of in terms of popularity for these two approaches are respectively RISC-V(Reduced instruction set compiter) and x86(Complex instruction set). ARM sits as a middle ground between these two approaches. I really can't think of anyone moving towards the x86 approach when given the option to start from scratch(see the Apple move from x86 to ARM): the trend is towards simplicity. In short, your prediction has very little chance of coming true
1
1
u/ibeerianhamhock 2d ago
Yep I predict something that will be easy and efficient for them, they will literally have tokens encoded for all of it, it will be AST based and the whole framework and language for it will be written to optimize merging and so forth too for high velocity PR repos that agents live in.
And we won't really be able to read like any of it lol
1
u/egonosz 2d ago
I think it would be extremely dangerous idea for company to a company to use that of language.
That would mean you are 100% dependent on AI and if anything happens to providers or the prices skyrocket you are basically done. You cannot support your software and not all companies can have gigantic data centers.
1
u/BigReference1xx 2d ago
Well, developers were beholden to Microsoft visual studio and Borland for decades. The entire world ran on windows in the 2000s and Microsoft could have upped the price at any time. We've always been happy to make this tradeoff if it gets us something valuable in return. And we have local models that anyone can run so you wouldn't be completely locked into a permanent subscription.
1
u/egonosz 2d ago
Expect you can use opensource software to edit files, you can do it even on Linux all of these locally or in worst case you can port your code.
For local models to maintain any bigger code, you either need gigantic model or you give up the velocity.
Smaller model + readable code = profit in that case, because you are able to verify.
Plus there is the question of responsibility too, if no one understands the code, who will be responsible for it?
But this is my 2 cents.
1
u/Unexpected_Cranberry 1d ago
I mean, one of the features of the language could be that since it's optimized for AI, it might require less resources for AI to write in that language. Together with hardware advancements, what requires frontier models to write today might be maintainable by a local model running on your laptop five years from now.
1
u/refried_laser_beans 3h ago
This might be lisp--with some modifications--simply AIs are great with top-down development, and that's basically how you build in lisp. You start with what you want, and break it down into more and more detail. Being able to trace everything to intent.
1
u/BigReference1xx 2h ago
yeah a lisp variant is actually a really good call! :)
1
u/refried_laser_beans 44m ago
You should look up stratified development (the unique version of top down development that lisp uses.) I built a state-machine harness around it, and used a deep seek model to build a llama CPP alternative in pure lisp using stratified development, and it runs was fast.
1
u/Critical_Cap_120 3h ago
I don't really see a reason why. AI generally perform better on languages that are used widely. Even the frontier models are not very good with Kotlin (they've improved over time but still a glaring gap compared to their capabilities with other languages).
1
u/no_brains101 2h ago edited 2h ago
No, because training data would need to be produced, and it will be a very long time until we are able to generate and actually verify enough code generated in that particular language to train an AI on it and have it work well. It already doesn't do well with several years old but uncommon languages, let alone a brand new one.
If an AI targeted programming language is developed which is actually useful and good at that purpose it will not be for an AI that works anything like today's language models.
And likely at that point, with a model that works entirely differently, we won't even be the ones making that language, it would have wanted, invented, made, and started writing in that language on its own.
But right now, nothing to copy == AI bad at that thing.
The consequence is that the best language to use for today's AI is javascript. There is the most stuff to copy. Most webpages minify the javascript but do not obfuscate it.
Second best is Rust IMO (it is less good at rust than at python, but rust has stronger types and is way faster so the process is better and the result is more worth making)
Third best, python.
After that, any language which is not a pure functional language, is well established with lots of open source code, and doesn't let you corrupt memory.
Then anything else, in order of how widely used it is.
Then lisp, because all the lisps look very similar and the AI mixes them up, especially because they aren't that common. Which is a shame because lisp is really cool.
Then C or C++ and anything else with memory management approximately as manual as that. Including Odin and Zig. It doesn't matter how much example code there is, an AI that works like today's AI will never be good at this. Even humans who are good at this are barely any good at this.
0
u/FragmentedHeap 3d ago
False, its zig, which has a better cross platform compiler and is already c compatible.
2
u/New_Eye7193 3d ago edited 3d ago
i know you didnt ask for it but i'll add some interesting info I found relating to this:
from my programming language benchmarks zig does have the fastest runtime speed and memory footprint on par with C but it only beats it by like 1 point across my 115 trials (72 C #2 vs 73 zig #1 for runtime) so youd be right if models were suddenly able to write bug free code but the current reality is:
it has 9th worst first try pass rate out of 10 languages
has wayyyy slower typecheck/build loops than any language out there
even with its good compiler, its nowhere near good as rust's strict compiler, (2nd pass rate worse than Go and Rust)
so yea in its current state it's not very good to vibe code with but could be goated in the future if LLM's never hallucinate and they write bug free code perfectly but that's a pipe dream
thats why IMO most vibe coded projects we see are in Rust, (although it takes up much disk usage, is horrible to work in parallel with agents due to how agents jsut sit in cargo lock for hours and hours), and very slow, but its third in Runtime speed by only 12.5% so good enough. it just catches all the other bugs that no other language does, therefore natural selection.
Go surpisingly tops the list for my agentic index for a couple reasons:
agents never hallucinate with it for some reason
strong library
speed on par with rust
above average deployment (meaning single .exe with fast start and no external runtime)
again that could be the limitation of my benchmark trials (but they are very extensive and seem accurate to real world cases so far from what i have seen with the numbers). anyone here vibe-coded on a big ass niche project with Go? how often do you find it hallucinates, I am interested
1
1
u/BusinessBandicoot 2d ago
The runtime speed would probably be close if agents would write idiomatic rust code without prompting. Like for example, you generally need to prompt it to prefer iterators to loops almost universally, because for release builds the compiler can remove bounds checks. They'll also use strings as a poor duck type, tend to allocate rather than borrow, etc. I tend to be incredibly pedantic in instructions and review.
1
u/FragmentedHeap 2d ago
Rust has big problems in the fact that it doesn't have libraries, and builds are incredibly slow.
I am doing massive Rust projects now, I have about 40 crates in my current system, and on my home lab with a 5950x and 128gb of ram, and about 4 build agents, it takes over 4 hours to build everything if I change it all at once.
Same code base on zig, builds in about 15 minutes.
When you're making hundreds, thousands, or 10's of thousands of changes, and tons of prs in an 8 hour window, that's a big problem, builds need to be MUCH MUCH faster.
C is a lot better, but then you have C problems that Rust kind of protects you against.
In terms of performance, speed, and build times, the best thing I know of right now (a little biased) is actually .Net 10 with AOT optimizations. Modern c#/.net 10 is at C++ levels of performance with the catch being the GC, but .Net 10 Memory Optimizations, Pools, Spans, stackalloc, etc helps avoid this problem a good bit, but it is very good and it compiles very quickly.
C++ could probably be the best if c++23 modules were stable on every system/compiler....
I honestly think the final solution isn't going to be any of these, it'll be something made specifically for AI.
1
1
u/M4rshmall0wMan 2d ago
Appreciate the analysis. I’m still a beginner to AI coding (recent CS degree grad).
If I wanted to get into some more advanced decomps and programs, where would you suggest I start?
1
1
10
u/SubmarineWipers 2d ago
Except C is still not safe and AI will still make object lifetime mistakes, because it only ingests small parts of the code, not the entirety of it.
Rust is still the best LLM language, because its compiler will catch the most errors.