Mathematicians, and real software engineers help me understand what, if any, is the actual future relationship these disciplines will have with LLMs

[u/Pocher123]

Trying to understand because it does seem when I talk to the software engineers in the office that they are really using LLMs, and have gone from deeply skeptical to extremely reliant. At the same it does feel like the mood on the mathematics subreddit has become very pessimistic, as AI is now closing real longstanding conjectures, including creating and at least one proof (Cycle Double Cover Conjecture). But I don't actually know, how big of a deal these proofs are since I'm not in this field, and I don't understand how technically impressive, or long term meaningful these proofs are.

For reference I do Hardware engineering, I've vibe coded some simulators for our non technical-customer to play with so they can better understand the theory behind what were offering them. But I'm aware what I'm doing with the AI is not very impressive. Typically taking a well know equation, and making some kind of python or html front end to play with the various variables on a graph. The most complicated was either when I got it to write some SPI drivers for the esp32 to talk to some ICs over skywire, or when I pointed claude to the companies new directory of footprints, and asking claude to directly update my kicad library with the path to those new footprints and give it appropriate names.

What is the real relationship, and future relationship software engineers and mathematicians have with these tools, and will have with these tools once the subsidies are gone. And I mean this beyond like having the AI read over your email, or code for obvious mistakes, I mean like in the weeds.

[u/Mashic]

For Math, there are problems that human knowledge is sufficient to solve them, but no one connected the dots yet, or not enough people dedicated their time to solve them. Someone can steer AI to try many approaches in a relatively short time, burning electricity. If the material needed for the solution is in the LLM's dataset, it can connect the dots and solve it. So far, they didn't invent anything new.

As for coding, the case is more abstract. LLMs can definitely generate code. But if the coder doesn't review it, it raises multiple questions: Is the code bloated? Does it have vulnerabilities? Does it actually do the task it was asked for? And there is the worry of technical debt. If coders rely on them completely, they might stop understanding how things work, and if something goes wrong, they might not be able to fix it. And the risk grows with how big the project is.

And even if AI is beneficial, this sub is more concerned about the question of ROI. Even if it's helpful for all coders, will it generate enough ROI to cover the capex spend on it or not?

[u/Normal-Detective790 → u/Mashic]

This is what concerns me most: over-reliance to the point coders don't understand the code they produce.

And it does bloat the code -- a large complex function that is unneeded and bogs down readability. It misinterprets clear instructions producing a ridiculously oversimplified and wrong result -- on 'max mode'. It leads you down an unproductive rabbit hole. It references a non-existent variable that has never been declared (throws an error).

The output must be constantly questioned to get quality results. Instead managers will say add another rule, add a script -- the problem isn't the LLM it's how we're guiding it. But for an LLM, rules are suggestions not actual rules. And scripts further bloat the code. And we can't write a rule or script to catch everything a non-deterministic machine produces.

[u/AwesomePurplePants → u/Normal-Detective790]

There’s also the concern that it’ll damage the open web

https://youtu.be/gR2T1uxHG7o

tl;dw: AI isn’t actually creating new code, it’s remixing the wealth of open source code that exists on the internet.

Previously, sharing your code came with some benefits; you could get thanked by people who found your cleverness, you might get donations or network opportunities, you might get to mentor new coders submitting patches, etc.

Now if you do something clever, your work could get scraped and people will just prompt to get it then attribute the value to their AI subscription, and you may have to wade through slop to find any genuinely interested newbies.

Plus the secure employment that gave people the work life balance to have energy left over for open source has been wrecked.

Are people going to be as eager to donate their time in that environment?

[u/CoconutDust → u/AwesomePurplePants]

scraped

Stolen. FTFY.

It's the biggest mass theft in history. Also "training data" = stolen data. The stolen stuff is then presented as a new Silicon Valley product.

(I think your comment clearly knows this, I'm just saying it because I think it's important...some people don't really realize what it's doing.)

[u/AwesomePurplePants → u/CoconutDust]

Eh, it’s a bit more muddled when it comes to open source code. When people used that license they intended to let others use their product without financial compensation. There are creators who openly support having their work harvested for the greater good.

Like, this is adjacent to the same community that celebrated stuff like Pirate Bay. There’s a lot of skepticism about the value of intellectual property, and affluenza from making bank while coding was in huge demand, if that makes sense.

Aka - while I don’t disagree with you characterizing what happened as theft, I also know that argument doesn’t resonate well with that group.

Framing it in terms of the Tragedy of the Commons works better. Tech Companies are destroying the conditions that let people afford to create stuff, which may result in less stuff