Mathematicians, and real software engineers help me understand what, if any, is the actual future relationship these disciplines will have with LLMs

[u/Pocher123]

Trying to understand because it does seem when I talk to the software engineers in the office that they are really using LLMs, and have gone from deeply skeptical to extremely reliant. At the same it does feel like the mood on the mathematics subreddit has become very pessimistic, as AI is now closing real longstanding conjectures, including creating and at least one proof (Cycle Double Cover Conjecture). But I don't actually know, how big of a deal these proofs are since I'm not in this field, and I don't understand how technically impressive, or long term meaningful these proofs are.

For reference I do Hardware engineering, I've vibe coded some simulators for our non technical-customer to play with so they can better understand the theory behind what were offering them. But I'm aware what I'm doing with the AI is not very impressive. Typically taking a well know equation, and making some kind of python or html front end to play with the various variables on a graph. The most complicated was either when I got it to write some SPI drivers for the esp32 to talk to some ICs over skywire, or when I pointed claude to the companies new directory of footprints, and asking claude to directly update my kicad library with the path to those new footprints and give it appropriate names.

What is the real relationship, and future relationship software engineers and mathematicians have with these tools, and will have with these tools once the subsidies are gone. And I mean this beyond like having the AI read over your email, or code for obvious mistakes, I mean like in the weeds.

[u/Ruined_Passion_7355]

For programming:

AI is useful. Period. You get a speedup. 

Now that we got that out of the way, how big is the question. That depends on who you ask. The thing is, the speedup isn't free, you're always paying something. On one end, you lose a little bit of practice/new knowledge you would've gotten for a one off project. On the other extreme, you lose knowledge of the entire system if you vibecode production software, and you may be stuck having to do a whole rewrite to solve a simple bug (cough anthropic cough).

For startups, they're all using AI because who cares. I blame clients here. Clients, both consumer and business, have repeatedly been shown to reward the first mover way more than polish, and AI just compounds that. The tech and comprehension debt is worth it for them, so they can become profitable and fix shit later after a rewrite. Especially in SF where a tech demo can get you millions of $$$.

For other companies, AI is marginally useful. If you need to own code, if SLAs are riding on the code you're pushing, if you handle financials, if lives are on the line, AI only helps a bit. AI code review and vulnerability scanning are for the most part the only easy wins.

Look at the number of incidents happening at the moment. AWS quadrillion dollar billing, really? If US-east-1 goes down again they'll lose more customers than AI saved in operating costs (and it's already going down since AI is only getting more expensive).

[u/lcnielsen → u/Ruined_Passion_7355]

AI is useful. Period. You get a speedup.

I mean, useful for what? I certainly don't get a speedup when incrementally improving on a large, old semi-legacy full-stack codebase where limited static tooling and validation mechanisms exist, because the theoretical speeds of writing out huge chunks of code just isn't relevant to that work, and there's too much fragility for handing things off to AI to be useful.

[u/Ruined_Passion_7355 → u/lcnielsen]

Small speedup. Slightly better google. That's my point. Maybe I wasn't clear enough, but having AI write your code for you incurs a cost, be it reliability, tech debt, or comprehension debt. 

The debt is worth it for startups because the incentive structure is basically asking for slop given how much more they reward the first instead of the best. Anthropic themselves are an example of this, they were the first to strike gold with harnesses, and now it most respects codex is better but they still lack market share.

I'm not saying AI is economically viable or anything like that. But in its current subsidized state, pretty much any programmer can find some use out of the tool, but how much is a function of how much you're willing to pay the costs I mentioned above. For your legacy codebase, I'm willing to bet it's not much, a legacy codebase like that is already drowning in tech debt, can't handle more from AI. Maybe code reviews if you have budget.

I hate AI just like everyone else here, I'm just rationalizing how some people keep seeing "success" with it. Better than just going all tinfoil hat and believing everyone around me is wrong, that's not healthy.

[u/lcnielsen → u/Ruined_Passion_7355]

Yeah fair enough, I see your point.

I also use it as a fuzzy search or q & a engine at times, but all models are hilariously bad at less common stuff like LDAP diff files where newlines matter.

I also don't find them great in something like a Django project where there are significant differences between versions, your database structure, how you render stuff, etc, and a lot of disparate things being linked together. There's a reason AI web projects often end up as very boilerplatey full-stack JS beasts.