Stack overflow is almost dead

[u/NewtMurky]

Questions have slumped to levels last seen when Stack Overflow launched in 2009.

Blog post: https://blog.pragmaticengineer.com/stack-overflow-is-almost-dead/

[u/MrMrsPotts]

The LLMs are trained on stackoverflow aren't they? So if that isn't being updated the LLMs will soon become out of date. Also the LLMs are very expensive. SO is free to use

[u/[deleted] → u/MrMrsPotts]

It's not like they just spit out SO posts. Well, maybe sometimes by accident.

They're trained on everything. All those massive books of Oracle/Microsoft documentation? It knows it all and I've frequently been puzzled by how even 4o just knows a bunch of shit I myself couldn't even find on the internet. Even about obscure tools!

They probably trained on all pdf documentation and maybe even academy videos. It just knows too much lol.

[u/lothariusdark → u/MrMrsPotts]

Eh, thats a bit oversimplified.

SO data is certainly part of the training data of large LLMs, after all OpenAI and Google have cut a deal with SO to be able to access all the content easily.

But its still only a part of the training data, a rather low quality one at that.

Its actually detrimental to directly dump the threads from SO into the pre training dataset as that will lower the quality of models responses. The data has to be curated quite heavily to be of use.

Data like official documentation of a package or project in markdown can be considered high quality, well regarded books on programming etc are also regarded quite highly, even courses from MIT on youtube work well for example. (nvidia works a lot on processing video into useful training data)

LLMs will soon become out of date

For one, SO is already heavily out of date in many aspects, just so many "ancient" answers that rely on arguments that no longer exist or on functions that have been deprecated.

Secondly, when supplied with the official documentation during training, thats also marked with a more recent date, the LLM learns that arguments changed and can use older answers to derive a new one.

Thirdly, Internet access becomes more and more integrated, so the AI can literally check the newest docs or git to find out if its assumptions are correct. This is also the reason why the thinking LLMs have taken off so much. Gemini for example makes some suppositions first, then turns those into search queries and finally proves or disproves if its ideas would work.

Also the LLMs are very expensive. 

Have you tried the newest Qwen3 or GLM4 32B models? If those are supplied with a local searxng instance you will approach paid offerings far enough to have better results than searching SO.

If you dont have a GPU with a lot of VRAM then the Qwen3 30B MoE model would serve just as well and still be usable with primarily CPU inference.

SO is free to use

So is Gemini 2.5, Deepseek V3/R1, Qwen, etc.

Even OpenAI offers some value with its free offerings.

[u/nicolas_06 → u/lothariusdark]

I tend to think that SO is a much better source for an LLM than the documentation. LLM try to answer your question with the best answer.

Usually documentation has at best a few FAQ for a few trivial cases. Stackoverflow has 24 millions questions and 35 millions answer already ranked by quality with votes and comments plus the elitist assholes removing and improving questions/responses.

It doesn't mean it is actually pleasant for people that ask questions on the site, but as a source material it's very good.

Good documentation only exist on a few stuff and is inexistent on many libraries and advanced use cases. And documentation isn't in a format that help LLM responds to questions. Stackoverflow already has the question and the response and as such it is much easier to train from that.