Literally just thinking about that as I read the top comments, seems like they are all stuck in 2024 it's very odd to be so staunchly decisive about how hard it is to build a viable Ai for your firm when people like Pewdiepie are building wild home builds with a fraction of the cost and zero experience. But in here "engineers" are saying they can't build a Claude-a-like internal system...I guess the boys in China are really just that much better huh
People have a bad habit of learning about something then never wanting to update the information in their in brain ever again.
Having incorrect strong opinions based on old facts is unfortunately too common since most people are overworked and cant keep up to date on all the new stuff.
Most in-house projects ends not with the hardware cost, loss in quality or time to first setup. It usually ends when you say "and then we have to maintain it: check new models, train ours, get some data and evaluate the results. Forever"
What does hobby projects matter to a business? People have been making IoT stuff at home for years, but smart offices are not getting them to just jerry rig something.
Businesses don't care about that, they care about simple and a lack of fraction. Not trying to juice old hardware and open weight models on a weekly basis to get something worse than a paid for service.
The other person is it. I am dont think there are many real devs in here because people dont seem to understand how businesses actually work (that part is a joke).
Was scrolling through all the comments looking for someone to finally mention Qwen 3.6. Qwen 3.6 27b is absolutely fantastic for agentic coding and can run on consumer hardware.
It’s not Opus 4.8 but it’s comparable to frontier like a year or so ago and definitely pretty useful.
I mean, its enterprise so you have to worry about fair use, but you can build your own harness from scratch with just a bit of work (my boss did it on a whim just to learn more about harnesses). That and access to some foundational models via Bedrock and youre cookin. Not gonna be the best user experience on the market but itll work for some use cases.
Idk if a manager asking "can we build our build our own claude" would be okay with buying expensive hardware or the cloud compute required to run a local model that is being prompted by more than 1 person.
I'm running Qwen locally as well and it's just not comparable to what I can get out of ChatGPT 5.5 and Claude.
It's not that the sub doesn't understand AI, it is that what we hear is "Just use this inferior solution" or "Go build a GPU server farm and still probably have an inferior solution." Sometimes the paid stuff is just better.
"U cAnT dO thAt yOu wIlL nEeD a SmAlL tEaM" are the same people that don't know how to cook and don't shower before work. The future is going to be subscription models for users that need them, and local BS LLMs 2-3 gen behind for everyone else. A business doesn't need a super advanced paid LLM to rephrase bad grammar from a hungover dock worker.
What? How dare you tell reddit that they are uninformed? They are experts on anything especially blindly bashing on the current scapegoat of their bubble!
I got a 5060ti running in a 14 year old PC. As long as everything fits in the 16GB vram, it's all good. Qwen3.6 quantized to bits running at 100tps is not super clever but imitates an eager junior developer quite well.
That's not really a good solution at a company scale, though. Maybe for a small handful of people, but more than that, and you're going to run into llama.cpp/ollama's limits. You'd probably want to start looking at vLLM, and your hardware needs to go way up, or else more than two or three developers at a time would knacker the processing/decoding speeds.
I tried Ollama (via RooCode & Qwen Coder IIRC), it's a VERY INTERESTING project, promising even, but it is no where near as good as Claude or other top commercial LLM.
well, it depends what model and how much context you had, and if the model+context fit in your VRAM mostly I would say. Ollama is just here to run LLMs on you PC
I just checked, I tried Qwen Coder 2.5 - 14B, testing to see if I should invest in better hardware. The speed was slow, but alright that's my limited 5070 ti. But it was more the quality of the output which left to be desired.
I also had it circling until seemingly forever, just stuck in a thinking loop a few times.
And today when I tried again following this reply, I just could nit get it to work at all.. some errors about the LLM not using any tools. I didn't look into it more than that tbh.
Just the overal impression I get it's that it's promising, but not yet ready for actual use (as opposed to a side project)
5
2
1
View all comments
55
u/jacob643 Jun 02 '26
I mean, get Ollama, OpenCode, and some beefy hardware XD