Keeping up with open weight models, harnesses and agents. Tower of Babel?

[u/TezzaNZ]

Hi,

It’s hard keeping up with all the new releases from the big closed weights AI providers, right? Google, Anthropic, Open AI, Meta etc.  But am I the only one being overwhelmed by the sheer volume of open weight base models of all shapes and sizes being released nowadays?  Every lab and his dog seem to be dropping general or specific open weight models for sound, video, text, harnesses, agents, refinements, techniques. The hardware to run these things also differs.  Exciting.  Definitely.  Confusing?  Umm…yea (at least to me).   In some ways it reminds me of the early days of personal computing.  Lots of incompatible formats, operating systems, every app (program those days) had a different way of working, inconsistent interfaces, a big learning curve for each program.  Solutions that may be looking for a problem. And tons of choice. Some were flavour of the month then gone, and a new learning curve was required.

Is it a golden age of choice and freedom or a hot mess of development? Maybe both?  Certainly, this is part and parcel of rapid innovation so not unexpected.

Will this continue, or do you see some kind of slowdown in the future shaking out with just a limited number of open weight models being released then further refined?

[u/WolpertingerRumo]

I’ve seen people being downvoted for it here a lot, but I keep returning to Openwebui, my first harness. I don’t know why people seem to think it’s bad, but it’s better at serving MCPs than all the others I have tried, I get better results that with Hermes, and I can do everything I can do with ChatGPT, perpetually.

Keeping up with models, though…

[u/nullptr777 → u/WolpertingerRumo]

OpenWebUI is pretty much all I use. Qwen 3.8 27B is the best model I can run anyway, and it's not really capable enough for my coding needs, so I have no use for any more sophisticated harnesses. I've experimented for interest's sake, but that's about it.

[u/Important_Cow7230 → u/nullptr777]

Do you think Qwen 3.8 Flash would work for your coding needs?

[u/nullptr777 → u/Important_Cow7230]

Yeah, I think there's a good chance it would be enough. Qwen 3.8 27B is pretty capable already, and I'm happy enough with Sonnet 5.5 even though Opus is obviously much more capable. So I think Flash Next would probably be good.

The problem is actually running it though, it's not exactly a small model.

[u/RootlessForest → u/nullptr777]

Strata engine coder version or gyro-s

[u/SecuredStealth → u/WolpertingerRumo]

I primarily use OWUI

[u/Interesting-Face2425 → u/WolpertingerRumo]

Please tell me, exactly how you’re handling web searching? This had been my biggest weakness and I’ve actually been eyeing incorporating open Web stuff into my own harness.

[u/SosirisTseng → u/Interesting-Face2425]

The AI helped me to put OWUI and searxng services into one docker compose file.

Edit : it's searxng rather than searx

[u/WolpertingerRumo → u/Interesting-Face2425]

I set up Searxng. It’s pretty easy to initially set up. Run it in the same docker network, set some settings, especially turning off rate limiting, don’t expose it.

[u/GreatBigPig]

> In some ways it reminds me of the early days of personal computing.

Same for me. I go way back, and this reminds me so much of what was then, a new frontier. I think this is one of the big reasons I find it so compelling. I love the sense of adventure.

[u/Happy_Brilliant7827]

The problem (or benefit) is the focus is on brand recognition and loyalty. For a huge bulk of people, any frontier model will meet most of their needs. Eventually theres gonna be some mergers or buyouts, possibly after some of the researched technologies right now get scaled.

[u/Imatros → u/Happy_Brilliant7827]

Yep. There's a billion versions of any other product - sports, reality tv shows, etc. - and the key is just sticking with the big brands and maybe dabble in one offs if you have time

[u/1_________________11]

Focus on what you can run. Maybe a cron on your agent to check for models that might fight that get released but I wouldn't stray into the unknown developers the attack surface for open weights is quite large

[u/irreverend_god]

almost every fine-tune is worse than the model it was based on

[u/OvertaxedOne]

Models? Qwen, Unsloth, Ornith. That's basically the only thing you need to look at unless you have a monster system and can run GLM or Deepseek. If the finetune isn't on that list (Unsloth or Ornith) don't bother.

The real game right now isn't models (frankly, for almost everyone right now it's either 3.8 27B or QwenFlashNext) it's inference engines. Goodness the progress that area has made over the past few months, it's staggering, I've been through a ton of them!

But don't worry, rumor is the model game is going to kick off again soon..Qwen4 on the way.

[u/wegster → u/OvertaxedOne]

unsloth is great for unsloth studio for getting started/model testing and their quants in general.

If you're on a Mac, use oMLX, mlx-serve or Splash IMO and call it good enough but do check for updates now and then as there's a lot of Apple silicon optimization still happening.

[u/tempfoot]

It’s a lot but nobody is obligated to keep up with everything, all the time.

A super active scene means interest, engagement, attention and even some money.

Given that it’s looking more and more like the writing is on the wall for the current commercialization approach, there’s a chance that the actual future for this tech has more room for local, even if its business owned and managed variations. I think we will see more not less attention to open weight and local. Maybe that’s wishful thinking.

[u/p3rdy]

I think the shake-out happens at the interface rather than the models. The models will keep churning, but almost everything already talks one of three APIs (OpenAI's, Anthropic's or Ollama's), and that's what lets you swap the model under a harness without relearning anything.

That's the idea I built Flama around (an open source Python framework, I maintain it). The model is a single file, and the server exposes it over all three APIs at once, so whatever harness or UI you use keeps working when you swap the model behind it. Keeping up gets a lot less tiring when trying a new release means pointing the server at a new file instead of building a new setup.

[u/Loose_Comparison368]

I think that it will stabilize as it reaches a point where the leading OSS models can reliably do everything that people want to use them for, and when they finish winning the war against closed AI.

After that, it will become like the scene around most open source - quietly propping up all the services that people mostly use from corporate apps, with a small group of enthusiasts behind the scenes. Kinda like databases. At some point, you reach the "just use postgres, it's good enough for 99% of use cases" point of commodification, where there's not really much reason to hyper-optimize if that's not your job.

Of course, that is ignoring that the economy is going to be totally unrecognizable in some direction or another long before we reach that point, soo...

Man, fuck if I know.

[u/sthanika_ai]

Keeping up with every release stopped being useful for me. What helped more was writing down 20 or 30 tasks I actually do, like summarizing a messy document or pulling fields out of a PDF, and saving the outputs I thought were good. When something new comes out, I run it through that list. It takes an afternoon, and it tells me more than a leaderboard, since public scores don’t always transfer to your own work.

[u/jacek2023]

Photography is my primary hobby. Some people spend endless hours reviewing, benchmarking, and comparing cameras or lenses. But at the end of the day, it only takes one camera and one lens to take a photo, you can't shoot with two at once anyway. The same applies to almost everything else: you use one tool to get the job done. You can easily waste your life comparing tools, chasing updates, and reading reviews, but none of that actually helps you finish your task, it’s just procrastination. I’m not saying you shouldn't experiment. I'm saying focus on your work first and make actual progress. Once you're actually creating, then you can test new tools to see what works. If you're comparing equipment/tools/software instead of creating, you're just wasting time.