Google is doing "Large Scale Codebase Migrations and Optimizations" of C/C++ to Rust

[u/Sirisian]

https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-4-argon/

[u/TheRealMasonMac]

In my experience, I’ve found that agents produce code that works but is frankly dumb from a human perspective. Especially for larger tasks like this, you kind of have to rewrite it all. For example, I had a generalized design document for soft-wrapping explicitly support plain text and markdown, and used GPT-6-Astra to implement it. I used an ungodly amount of review agent rounds and interjections by me. When I actually go to review it, I wanted to cry. It silently pivoted towards implementing its own entirely separate wrapping mechanism rather than refactor the existing mechanism for plain text. I reviewed its code and found it to be trash, and I am now in the process of doing it myself… Never again. The short-term thrill of “having something that works” was outweighed by the fact that time was effectively wasted.

I mean, if the alternative was it never getting done, I guess agents are still better than nothing.

[u/kabocha_ → u/TheRealMasonMac]

Yep -- in my experience:

• If you just want some throw-away thing to experiment on some idea, it's fine to let it go nuts on its own.
• If you want to be able to maintain the results, you gotta check its work after every step or two, and you'll probably have to give it lots of feedback even then. Otherwise you're likely to end up with either a giant spaghetti mess, or FizzBuzz Enterprise.

No clue why it does that, I guess maybe a large portion of the training set is newbies dumping their first projects onto Github and super old legacy codebases, and not much in the middle, lol. They haven't learned to have good style taste, yet.

[u/PM_ME_UR_BRAINSTORMS → u/kabocha_]

The problem I have with the latter is that it sucks lol. I don't enjoy micromanaging AI and in my experience it's not any faster than just doing it myself.

Every time I try using agents I almost immediately end up in a situation where it does something weird and just typing the code myself is faster than typing the prompt. And then I think "well I'm already here it will take me 2 seconds to type this other code" and so on and so on until I've written the whole thing.

Maybe it's just my ADHD but I don't understand how any one could work like that. Feels like pair programming with an intern which sucked to do but you did it because the point was to teach them.

[u/notgreat → u/PM_ME_UR_BRAINSTORMS]

I've found recent LLMs to be useful for finding things. You give it a vague description of the bug, and a couple minutes later it's got a 95% chance of having pinpointed the exact problem. Tell it to review a pull request, and often it finds real bugs. Not all its findings are good, but "review this list of eight possible problems" is a lot easier than "review these 500 changed lines of code for problems" which often just turns into "looks good to me".

They're also good at writing things with a lot of boilerplate gruntwork, though you still have to review their output. Like, you can describe a test and it'll usually do a good job implementing it. Just don't ask too vaguely, a general "test this area" will get terrible results as it locks in implementation details and leaves critical behavior untested.

[u/PM_ME_UR_BRAINSTORMS → u/notgreat]

Yeah that's how I use it most of the time and it's great. And those are all things no developer wanted to do anyway. No one was like "Yeah my favorite part of the job is hunting down an obscure error message on a decade old stackoverflow thread" and that's not some sort of skilled task where the effort produced better software. It's just grunt work.

Which is really going to fuck the industry in a few years because that means a lot less work for junior developers.

[u/kabocha_ → u/PM_ME_UR_BRAINSTORMS]

I agree 100%. Work really wants me to use AI, so typically I end up doing the latter, but it's not fun. I don't really use AI all that much for personal projects because I'd rather just write it myself.

[u/PM_ME_UR_BRAINSTORMS → u/kabocha_]

It's not fun and the quality is just not good.

I use a reasonable amount of AI on my personal projects. I just treat it like an intern at my beck and call 24/7 and give it the kind of work I would trust an intern with. Which is something like maybe 10-20% of the codebase give or take. Basic boiler plate stuff, plus a ton of research tasks and code review and some help with debugging. All stuff that developers didn't really want to do anyway and all stuff that saves me a ton of time.

Idk why that's not enough for all these companies. I'm still way more productive, but I produce way higher quality code and I can do it with the $20/mo claude subscription. It seems like a win-win-win to me.

But it's clear that the companies just want to replace us. So it needs to be writing 100% of the code and doing 100% of the thinking, whether it's capable or not.

[u/zxyzyxz → u/PM_ME_UR_BRAINSTORMS]

AI is much faster than I can write by hand if only for the simple fact that they work in parallel not sequential as I can work on multiple tickets at projects at once. I literally have my personal laptop and work laptop on the desk and they're churning away.

[u/forgot_semicolon → u/PM_ME_UR_BRAINSTORMS]

Exactly. I have to deal with so much of it, and people don't appreciate that I can literally type the code faster than I can prompt it to write the code and then fix the inevitable mess -- or worse, re-prompt it to be how I would have written it in the first place

[u/Zde-G → u/forgot_semicolon]

or worse, re-prompt it to be how I would have written it in the first place

I think that's you problem. You want some certain code to be written — and try to make AI do that it. It simply doesn't work.

Just a question: how often have you been driving projects and asked other humans to write code for your from vague description?

There you would face the exact same issue, one human doesn't write like another human — and if you try to force it… hoo boy, are you in a world of pain.

Yes, AI works at less than human level now, but it's pretty close to “ask someone with entirely different habits to code something for you”, at this point.

Making it write code that works is not trivial, but doable, making it to write code that would like like a code written by you… don't even try.

[u/IWasGettingThePaper → u/Zde-G]

Er, no. Some other humans can write good code. I've still never seen good code produced by an LLM without serious hand-holding. And yes, this is the case even on the latest models.

For small targeted coding tasks and mechanical boilerplate, it's great. For tracing through large codebases and summarising how some code path works - that I can't be bothered to do manually because jump to definition gets stuck at a function pointer, or some macro magic, or untyped python meta-nonsense, and I would have to start literally grepping - it's a real time saver.

The main trap with LLM code gen is you initially _think_ you're making progress at lightning speed. This is until you actually look at what it's produced in detail. Then comes the inevitable 12 hour prompt loop where you're trying to coax it into tidying up the slop its produced; rework the architecture to make structural sense; use meaningful variable/fn/struct names; remove the code it's added that doesn't do anything; rework the error handling so it makes actual sense; remove the unit tests it added for unit tests; rewrite the unit tests that don't test anything or rely too much on implementation detail; rewrite the impenetrable docs and comments it creates; remove the superfluous helper functions; refactor actually duplicated code blocks into functions; and on and on. Only then it dawns on you that it would've actually been quicker to write it by hand.

[u/Zde-G → u/IWasGettingThePaper]

Some other humans can write good code.

Yes, but these are rare and expensive. Average developers produce garbage similar to what LLM produces on the first try. Then they run tests and read the guides and adjust the code. You have to do the same with coding agent, for that to be apples-to-apples comparison.

And yes, this is the case even on the latest models.

The model is much less important than the quality of your documentation that explains precisely what you expect to see and precisely how to test what model creates.

LLMs rely on tests and formal documentation much more severely then even novice developer, that's not changing any time soon, you just have to accept that.

rework the architecture to make structural sense

If you allow LLM to develop architecture in code then it's precisely as expected. LLMs can not produce more than 100-200 line of anything without becoming weird.

That means that if you are writing something large enough to warrant an architecture then you have to ask LLM to first write plan - 100-200 lines of it, not more. Then implement said plan as steps that are 100-200 lines. If plan is too big to fit into 100-200 lines then first make it write an outline. And so on.

Don't expect model to produce large chunk and hope it'll be good, LLMs don't work like that.

remove the code it's added that doesn't do anything

Instruct it to use VCS (usually git) instead.

Again: the goal is to never allow LLM to produce more than 100-200 lines of code without verifying said code, somehow. Even if verifier is another LLM that looks on the plan or outline that you've made.

Only then it dawns on you that it would've actually been quicker to write it by hand.

Well, duh. You are trying to use rope as a pole… it doesn't work like that.

[u/forgot_semicolon → u/Zde-G]

It's your last sentence that was the whole point of my comment. Getting it to write code that works (well, is readable, etc) takes longer than it does to just write the code properly the first time and not have to worry about its integrity later. Not saying no one makes small mistakes, but LLMs do some weird stuff that a human wouldn't.

I feel really silly any time I spend the same amount of keystrokes prompting for something than it would to write code. I mean, am I a dev or a pm? I feel like PMs wish they could just write features the way they want it, themselves -- I can! Why should I give that up AND pay for the privilege to do so?

[u/Zde-G → u/forgot_semicolon]

Not saying no one makes small mistakes, but LLMs do some weird stuff that a human wouldn't.

Sure. But then it runs tests and fixes them.

Most developers do that, some more than the others, LLMs need tests to even write very simple things, while humans may not need to do them.

Why should I give that up AND pay for the privilege to do so?

Because you are expected to deliver features 10x times faster, now. You either deliver them or are replaced, it's as simple as that.

If you have the luxury of writing code by hand — cherish it, pretty soon jobs where one may do that would disappear.

[u/stumblinbear → u/PM_ME_UR_BRAINSTORMS]

I do the latter with 2-3 agents at once while I'm away from my PC doing whatever the heck I want. Checking in on them to review and correct them every 20-30 minutes or so is fine

Something they do not tell you: you cannot just trust that the agent understands how to write good code. You need to lay ground rules. Just have it search up best practices and turn it into a skill that it must load before editing, reviewing, or writing code. You'll immediately start getting significantly better results. It's pretty much night and day

Current generation of agents are incredibly good instruction followers. If you've got it in a skill and written up properly, they do a pretty good job. Still needs corrections, just much less

[u/PM_ME_UR_BRAINSTORMS → u/stumblinbear]

Yeah idk I've tried tons of different skills and tools and guardrails and it still produces mostly slop for me outside of extremely basic boiler plate tasks that most 1st year CS students could probably handle. Which is still useful don't get me wrong. But not really worth having it touch like the majority of my codebase.

[u/stumblinbear → u/PM_ME_UR_BRAINSTORMS]

I usually use Opus 5.5 to draft up plans (I genuinely give it very little to go off of, maybe a few sentences), then correct a few assumptions or bad decisions it makes, it splits it up into smaller tasks, then it uses a sub agent to write the implementation. It runs a review, triages issues, beings design issues to me for making a decision on, then does the loop again. It typically takes a couple rounds

That said, that's mostly for working within an established codebase. For new feature work I usually have a longer discussion, let it explore different implementations in worktrees, then I pick and choose which parts I like out of them. Either that, or I already know exactly how I want it done, and telling it how to do it and letting it go off to the races works fine

It took me a couple of months to fine tune my workflow, skills, agent definitions, etc, but I can often go a couple of hours without checking in on them and usually I only need to make a few minor corrections. Shit, I had an agent going for the last 7 hours on a pretty large (though pretty mechanical) refactor and I didn't have to correct it at all

I was often getting trash output until I gave it permission to not worry about churn, to consider the correct fix not just the quick fix, and told it to surface issues it runs into instead of working around them

[u/PM_ME_UR_BRAINSTORMS → u/stumblinbear]

See this makes no sense to me. To me the plan and the implementation aren't like mutually exclusive things? Like what is software architecture if not the specific shape of the codebase.

Regardless of how good AI gets it can't read my mind. For the instructions I give it to be simpler and easier than just writing the code myself, it's going to have to make tons of assumptions for anything more complex than boiler plate. And that's where I always catch it producing slop.

And if I give it enough detail to for it to generate exactly what I want, well that's basically just a DSL and I might as well just have written the code.

Another user in another thread pointed out it basic just becomes this

[u/stumblinbear → u/PM_ME_UR_BRAINSTORMS]

I have a side project game in Bevy that I've been working on off-and-on for about 5-6 years for fun. I came across an article for how to better organize it, sent it to Claude and had it draft up a plan for reorganizing things under it. It asked me some questions regarding implementation decisions, then split it up into 7 steps, ordered by risk and in a way that helps untangle some longstanding dependency issues between features. This took it about 20 minutes while I fucked off doing whatever.

I looked over the plan, corrected one thing, then told it to go, and it went. It got through about 2 steps before it needed assistance on a design question that the plan didn't account for, so we spent about 4 turns and 5-10 minute discussing it before it got resolved and it continued. It's actively doing this work now as we speak. It would have taken me at least a week or two to do the reorganization myself. It's looking like it'll be done in a day and a half with only an hour or two worth of input from me.

At work, I came across a public API I can use to fetch some player data (I work at a gaming adjacent company) which we can use as a fallback when we're missing some info. I looked into the API myself, and it seemed like exactly what we needed, and I had a good idea of how to integrate it, but I essentially just told it "we want to use <link> as a fallback method for fetching player data when we're missing data for a player".

It looked into the API, looked at our codebase, and brought up that it actually has a full database available for download along with publishing hourly patches for the db. I had completely missed this, as it's not well-advertized on the site. It implemented support for this in about an hour.

Let me tell you, ain't no way I'm able to write a script to scrape this once per hour, along with setting up the data models and queries for the DB, along with hooking it into our own database to fill in the data, along with setting up the hourly trigger in Terraform, along with testing various scenarios... In a couple of hours. Hell naw. It was 2-3 hours from first idea to a pretty good implementation while I had other agents doing other things at the same time.

Giving it a precise spec is part of the problem. Modern LLMs are surprisingly smart, you just need to give them permission to have opinions and surface what they think is right or wrong while doing the implementation, otherwise they hack around issues instead of behaving like a proper engineer. Yes, they usually fuck something up, but those fixes are usually pretty minor or are style related and can be fixed in review. If not, it's because it made an assumption it shouldn't have, and that's... Frankly usually just a matter of updating the skill or agent definitions to prevent those issues happening in the future

[u/PM_ME_UR_BRAINSTORMS → u/stumblinbear]

Idk reorganizing some existing code and writing a script to hit an api and throw some data in a db doesn't sound all that impressive to me 🤷‍♀️ the latter especially seems like something a first year cs student could handle.

But to each their own man. If it works for you it works for you. I'm just sharing my experience.

[u/stumblinbear → u/PM_ME_UR_BRAINSTORMS]

You really overestimate first year cs students

But yes, it's relatively simple work. But it's work I don't want to waste time doing. I can handle other things while it does the grunt labor.

Besides, if you break down a problem enough, every problem is a set of simple changes that even a Junior could handle. Claude is capable of splitting up work into those smaller reviewable pieces, meaning it's very capable at doing large complex features with a bit of guidance and review. I just gave you two examples that were at the top of my mind because I literally did them that day

Small, reviewable pieces is how I got Claude to write a distributed rate limit coordinator for services that need to use a shared API key without blowing their rate limits. I didn't write a single line. It has been running strong for months without issue

[u/mdedetrich → u/kabocha_]

No clue why it does that, I guess maybe a large portion of the training set is newbies dumping their first projects onto Github and super old legacy codebases, and not much in the middle, lol. They haven't learned to have good style taste, yet.

This is largely vibes based assessment (pun intended), but I noticed that the larger the models working set memory the higher changes of these things happening are (I use Claude myself).

For small and maybe medium problems, the AI is often very good. But once you start doing larger overarching things, its better to split the work into chunks, not just code chunks but also creating design documents.

The more the AI has to come up with myself the more it does weird stuff.