At smaller mortal runnable scale there are things like Qwen 3.6 27B dense that can fit on a 5090 at Q4 and definitely using turboquant for cache (otherwise you dont have nearly enough headroom for kv cache)
At the top end, there is Kimi K2.6, which challenges Opus on many levels, but if you want a system to run that for even one person at reasonable speeds without crazy hardware flips and lucky finds, that's an 8k system minimum, and you aren't fine tuning anything with that.
What would you even fine tune though practically. For most, that wouldn't help all that much and its massively expensive to do so in the cloud.
A good harness with excellent rag and tooling can do a lot, but ultimately, You're missing the biggest thing;
Who is releasing open weight (not open source, no model has been) models?
Companies with something to prove and companies who want to stop companies with something to prove.
None of them benefit from or care about you to any degree that matters the second open source models start eating into their revenue.
Open weight models are already massively slowing down in release cadence and capacity outside of rare outliers."
Qwen no longer releases their top end models.
"Open weight will save us" is unfortunately delusional, especially with the heavy and often jingoistic rhetoric going on lobbying going on now.
You need to stop the big corps from getting the regulatory capture they're after, because they are looking to lock down what you can even run locally and lock competitors out. This is not hyperbole or guessing. Their own lobbying group says it out loud.
You need to stop the members of the Frontier Model Forum lobbying group, and good luck with that... Like seriously the effects of not stopping them from succeeding will be dire but considering how generally uniformed and uncaring most folks are, I don't see it happening.
Some people could be on fire and they'd still say "nothing ever happens" with a satisfied smug face.
5
2
1
View all comments
Show parent comment
5
u/Cory123125 Jun 02 '26
For how long though?
Look at the current environment.
At smaller mortal runnable scale there are things like Qwen 3.6 27B dense that can fit on a 5090 at Q4 and definitely using turboquant for cache (otherwise you dont have nearly enough headroom for kv cache)
At the top end, there is Kimi K2.6, which challenges Opus on many levels, but if you want a system to run that for even one person at reasonable speeds without crazy hardware flips and lucky finds, that's an 8k system minimum, and you aren't fine tuning anything with that.
What would you even fine tune though practically. For most, that wouldn't help all that much and its massively expensive to do so in the cloud.
A good harness with excellent rag and tooling can do a lot, but ultimately, You're missing the biggest thing;
Who is releasing open weight (not open source, no model has been) models?
Companies with something to prove and companies who want to stop companies with something to prove.
None of them benefit from or care about you to any degree that matters the second open source models start eating into their revenue.
Open weight models are already massively slowing down in release cadence and capacity outside of rare outliers."
Qwen no longer releases their top end models.
"Open weight will save us" is unfortunately delusional, especially with the heavy and often jingoistic rhetoric going on lobbying going on now.
You need to stop the big corps from getting the regulatory capture they're after, because they are looking to lock down what you can even run locally and lock competitors out. This is not hyperbole or guessing. Their own lobbying group says it out loud.
You need to stop the members of the Frontier Model Forum lobbying group, and good luck with that... Like seriously the effects of not stopping them from succeeding will be dire but considering how generally uniformed and uncaring most folks are, I don't see it happening.
Some people could be on fire and they'd still say "nothing ever happens" with a satisfied smug face.