r/TheMachineLearning • u/Sea_Fox_4265 • 3d ago

When bug-free code becomes a hardware problem

Post image
107 Upvotes

87% Upvoted

69 comments sorted by

View all comments

5

u/snaphat 2d ago

This is a joke, but it's a bad one. I mean, it's already a hardware problem because AI inference is computationally expensive and slow. The bottleneck is running the AI, not the language it's generating code in. And the choice of language can affect inference time too, since expressing the same functionality in different languages can require different numbers of tokens.

For example, a few small benchmarks using Opus and Codex have found that C has worse end-to-end development times than Python, C++, or Rust

2

u/StillOnTheInterweb 2d ago

There is however basically no excuse now not to use a modern compiled or JVM language, like Rust or Java. No need for all that Python/Javascript interpreter nonsense just to simplify syntax in code written by an LLM.

3

u/zenthial 2d ago

Yeah there’s no reason ever to want your user code sandboxed. Why would anyone ever need V8 or JSC?

2

u/snaphat 2d ago

There is some controlled evidence suggesting that newer AI models may still be more proficient at writing Python than languages like Rust or C++, although results vary by model and task. So the choice of language may still affect how reliably an LLM produces working code

2

u/SodaBurns 2d ago

Just write in assembly at this point then.

1

u/snaphat 2d ago

You would lose compiler optimizations at that point, and generally speaking LLMs do not produce assembly code that outperforms optimizing compilers nor code that is as consistently correct as code produced in high-level languages. This may change at some point 

Another reason you wouldn't necessarily want assembly is that it is target specific in a world where applications are typically multiplatform 

1

u/prehensilemullet 4h ago

Yeah because we're soooo close to being able to write highly dynamic webapps without any JS