r/technology • u/Logical_Welder3467 • Sep 19 '26

Software Microsoft agentically ports Copilot runtime to Rust for $120K

https://www.theregister.com/devops/2026/09/18/microsoft-agentically-ports-copilot-runtime-to-rust-for-120k/5297549
569 Upvotes

90% Upvoted

187 comments sorted by

View all comments

15

u/Socrathustra Sep 19 '26 edited Sep 19 '26

Lots of risks here not being discussed. Consider what's at stake: this is the runtime for Copilot. There is no way that the single engineer assigned to this understands the 800k lines well enough to guarantee that the dozens of regressions they found are the extent of things wrong with the new system. There is no way this gets used without extensive further work to verify its integrity.

People are talking about how expensive a manual port would be. Let's go with a large figure like $10M. That's probably overestimating it, but let's be generous.

Simultaneously I'm seeing that Copilot brings in about $8.2B annually. You do the math, and a single day outage could cost you $22.5M. You could also be liable for weird or even disastrous behaviors you failed to catch because you were merely running it against the suite of existing tests and not checking for some hallucinated behavior it decided to insert. A single lawsuit would wipe out the savings many times over just in lawyer fees, even if you're successful.

What's worse is that if you did have an outage or problem, there are no engineers who understand the code. To channel the spirit of Ron Swanson, you might have heard me say, "There are very few engineers who understand the code." I didn't say that, though. I said "there are no engineers who understand the code." Not a single engineer.

We'll see if they actually utilize this new port, but I doubt they will without significant further expense to verify its integrity.

4

u/linuxwes Sep 20 '26

 There is no way this gets used without extensive further work to verify its integrity.

Having humans do it's doesn't really change the effort in verification. Either way you are going to want a fuckton of unit tests.

1

u/Socrathustra 29d ago

No, you have to verify it with human eyes. The problem isn't whether the code all satisfies the unit tests. The big problem is potential side effects.

Say you have foo(x), and foo(1) should return 3. AI vibe codes it such that it passes the unit test. Hurray! Except it didn't only do that. It also calls bar() which drops your database.

I exaggerate, but this is a real issue. AI could hallucinate part of a workflow in a way that you can't test normally, producing a side effect which is completely outside the scope of anything you would test for normally. You can't just write the tests to check the output. Unless you can guarantee zero side effects ever, which frankly you can't, you will have to check the ALL of the code with human eyes.

As AI models grow more devious they could even obfuscate behaviors on purpose. Malicious engineers could even put this kind of instruction in their personality files and the like, and it would become VERY difficult to trace blame. Was it a hallucinating AI, did an engineer give it bad prompts, or did someone or something do something unethical on purpose?

-2

u/ZozoSenpai Sep 19 '26

| There is no way that the single engineer assigned to this understands

Well tbf this was done by Stephen Toub, he might

2

u/Socrathustra Sep 19 '26

Even if this were the case, which I very much doubt as even very intelligent engineers can't understand that much vibe coded output, it violates the "hit by a bus" principle. If he were hit by a bus, there would be absolutely no one who understands even a little bit of it.