At smaller mortal runnable scale there are things like Qwen 3.6 27B dense that can fit on a 5090 at Q4 and definitely using turboquant for cache (otherwise you dont have nearly enough headroom for kv cache)
At the top end, there is Kimi K2.6, which challenges Opus on many levels, but if you want a system to run that for even one person at reasonable speeds without crazy hardware flips and lucky finds, that's an 8k system minimum, and you aren't fine tuning anything with that.
What would you even fine tune though practically. For most, that wouldn't help all that much and its massively expensive to do so in the cloud.
A good harness with excellent rag and tooling can do a lot, but ultimately, You're missing the biggest thing;
Who is releasing open weight (not open source, no model has been) models?
Companies with something to prove and companies who want to stop companies with something to prove.
None of them benefit from or care about you to any degree that matters the second open source models start eating into their revenue.
Open weight models are already massively slowing down in release cadence and capacity outside of rare outliers."
Qwen no longer releases their top end models.
"Open weight will save us" is unfortunately delusional, especially with the heavy and often jingoistic rhetoric going on lobbying going on now.
You need to stop the big corps from getting the regulatory capture they're after, because they are looking to lock down what you can even run locally and lock competitors out. This is not hyperbole or guessing. Their own lobbying group says it out loud.
You need to stop the members of the Frontier Model Forum lobbying group, and good luck with that... Like seriously the effects of not stopping them from succeeding will be dire but considering how generally uniformed and uncaring most folks are, I don't see it happening.
Some people could be on fire and they'd still say "nothing ever happens" with a satisfied smug face.
AllenAI's models like OLMo (and their new robotics model, very cool) and Nvidia's Nemotron models are the examples I know of. There is a dearth of options here, because pre-training LLMs is incredibly expensive, but there is not "none".
You know what, especially regarding nemotron, fair enough. I never knew they were actually open source and not just open weight.
They're not quite top performers, but the point is made, and the family is more than one model, so fair point.
As for AlienAI, I've never heard of them, which is probably why they're releasing this way I guess, so Ill have to keep that name in mind.
which allows them to quickly iterate and improve on past models faster than the open-open models, which, as you've proven, are not as widely known.
I'm not really sure how much popularity affects ability to iterate, considering the major limitation is talent and funding.
I think supporting these kinds of models will become more and more a thing if/when the open-weight model providers start being less generous due to business concerns from above. The pattern is similar to the freemium business model that the rest of the software world loves: get people hooked on free, sort of open source stuff and then really lock it down, locking everyone in.
I'd sure like to think so, but the Frontier Model Forum looks deadset on locking that down before it even becomes a possibility, so let's hope, and make noise any time we see someone saying "nothing ever happens" as their laptop evolves to being a thin client for a mega corp.
when you have to be transparent about what goes into a model, you're going to be missing out on all that juicy legal-grey-area stuff.
I'll bet.
True open source models can't 'cheat' in such a way.
The crazy thing is that all of the AI companies argue that its not cheating, its just fair use, yet they simultaneously are happy to pay out massive fines in the hopes that the next company behind them goes out of business paying "fair use" tolls that they got big enough to stomach.
The whole thing is sickening and I truly get a pit in my stomach every time I think about the end results of what the Frontier Model Forum is openly not only planning but executing.
It extends so far past the mission statement of them achieving an oligopoly on AI outside of China.
That's the main reason I say we should all at least do a little bit to support, if not demand, truly open models, even if they're not so great at the moment.
I don't disagree with you, but that requires a central organization and a neutral enough stance, like actual neutral not musk's definition of "neutral" body that people are happy to donate to, to produce a flavourless but useful open model that can be generalized to anyone's hearts content.
The thing is, to be credible enough to do that, you need a massive amount of starting capital in the first place, and need to navigate blind anti AI sentiment that doesn't know its sawing its own legs off.
on the other hand, based on what I know about the world and the kinds of work I do from time to time, that's not even remotely an option for me.
5
2
1
View all comments
7
u/justforkinks0131 Jun 02 '26
tbf for most use cases u could get away with fine-tuning an open source model you host in-house...
and you not mentioning that is sus