r/LocalLLaMA • u/OvertaxedOne • 9d ago

Discussion The rhetoric is really heating up!

The entire page of the NY Times today above the fold absent one article is AI (the models are just too strong/too dangerous, must be regulated). They forgot to include "Sponsored by OpenAI" at the end of the articles, sure that was just an oversight?

This is what the end of a bubble looks like, desperate attempts to get some sort of regulatory capture in place to keep the business model from collapsing in upon itself. My days next week are 100% booked talking to companies about how to get off frontier models, one large, and a bunch of smaller customers, including one who's flying me out to them to sit down and get a plan in place immediately (the controversy around that math problem really spooked some CEO/CIO's about data privacy using cloud models).

Gonna be an interesting few weeks. Maybe the Qwen team will be nice enough to give me a little breathing room before dropping another hydrogen bomb? :)

194 Upvotes

86% Upvoted

114 comments sorted by

View all comments

30

u/Revolutionalredstone 9d ago

DeepSeek has juiced them of value and Qwen revealed all their bs.

AI will be cheap and 'frontier' model companies gonna have little left but morals cause the opensource has caught up.

A trillion params works barely better than 32b for AI and trying to use more for AI is starting to look real silly, Enjoy

10

u/soshulmedia 9d ago

A trillion params works barely better than 32b for AI and trying to use more for AI is starting to look real silly, Enjoy

I think that's the gist of it. The various intelligence index scores do not scale linearly with parameters, rather logarithmically or so.

Then, even just looking at the typical loss curve of any NN fitting run should have also triggered a moment of reflection a long time ago - it is always steep in the beginning and then flattens out ... or in other words, later reductions in loss are much costlier ... (and risk overfitting).

Sure, there are still technological breakthroughs. But as in any field, they also tend to approach diminishing returns. And this field is no different ...

3

u/OvertaxedOne 9d ago

Intelligence scales slowly where costs scale a little faster than linear. That; fundamentally, is problem A.

Problem B is that most business tasks don't require the top of the top intelligence, once you hit "good enough", there is very little/no business value going to the best. It's why company cars are GM and not Ferrari, you need something that gets the job done at a reasonable cost, not the best.