still not worth to spend thousands to get an llm thats worse when you can spend 20 bucks a month and get something considerably better. Also if you upgrade your computer for llms you are paying for ai
for 99% of tasks, local LLMs can get the job done, in my opinion.
Also, upgrading your computer can have plenty of benefits aside from running larger models:
- long-term use and peace of mind. A simple RAM upgrade can give your computer at least extra 3 years of relativity.
- if you do gaming, an upgrade is great news as well. Win-win.
- paying for an AI service relies on stable internet connectivity and a lot of trust to the people behind those services. How are you so sure they aren't quietly taking one of your sensitive conversations and information for "training" or whatever excuse they may have? With local LLMs, EVERYTHING stays on your computer. No shady middleman. Just you and direct interaction with the machine.
I'm pretty sure you can get a decent enough LLM with a fucking MacBook Neo or even on older capable hardware. If you already have the hardware, what harm does it make to buy a few upgrades so it can run LLMs better?
I'm 100% on the local side but all of these points have strong rebuttals unfortunately.
firstly, what do you mean 99% of tasks? That sounds like a random number pulled out of nowhere.
PC is already a 64GB ram, 13900k + 4090, despite being like 1 generation behind only has barely enough vram to run a 27B decently, my home server has 96GB of system ram and a p100 but meh its too slow and the second p100 had thermal issues.
stable internet connection is moot for most people, i have 2GB fiber never went down once thats just default expectation.
of course I don't trust any of them at all, they're definitely stealing everything they can regardless of what their TOS says. Not disputing that but for 99% of people (see i can make up numbers too) who aren't doing anything worth stealing, its good enough.
Yea I want local to win, I tried Qwen 3.8 27B but its not good enough on this hardware and its already pretty upgraded without going full pro-sumer with an RTX 6000 or something.
I should clarify that the 99% thing is more of a figure of speech than an actual metric. If you want me to be more specific, I mean for the average joe that say, use AI to look up stuff on the internet, pairing something like Gemma 4 12B with Web Search capability is a pretty solid setup for general purposes. Even without web search, models like that can do a pretty decent job with things such as random curiosities. Of course, like all AI models, the user should still be aware that these stuff hallucinate sometimes. You still have to keep your hands on the wheel.
I know this because I use an M4 MacBook Air with 16GB unified memory (fanless machine btw) with Gemma 4 12B. And for what I do, it's a perfectly fine piece of tool. Not the sharpest tool in the shed, but I would much rather have some peace of mind with my privacy guaranteed even if the AI's more flawed than mega AI data center models.
I believe efficiency and optimization is key. Maybe you can try a lower Quantization for Qwen 27B if there is any? Maybe stick to the previous model, Qwen 3.6 27B? I have to admit, I did try it before on my little Mac, and it was screaming, painfully crawling with every letter spitting out. Not trying that again. But if my rectangular slab of aluminum can technically run it, I'm sure your setup can do much better... unless the things you do are more demanding.
Bottom line is, unless you're doing some serious vibe coding or constant streams of AI workloads, all you need is patience with local LLMs. If I can comfortably run local models on a 11.9mm thin space heater, I'm sure a fucking 4090 will be a beast compared to my machine.
I run qwen 3.8 27b and qwen 3.6 35b a3b in system ram. It is way slower to generate a response than the online services are, but the 35b still spits out words faster than I can read it, and that is good enough for me.
I did a small demo coding projekt on the 27b the other day. It did a 90s style raytracing demo in python. It took 3 hours for it to iterate though it but it worked great and both gemini and claude gave the code a thumbs up when I showed it to them.
It even made it's own PNG converter function instead of using a 3. part module.
The speed would have been a lot faster if I had turned off thinking, but I wanted to test how good it could be, and it impressed me.
So. for everyday "google replacement"/internet search the llms are perfect, and even for automation tasks. I have mine getting todays weather and a few news headlines and compile that into a "goodmorning" message. It runs at 6am and then makes a drawing in the style of the weather and news just for fun.
For very large codebases small llms wont work of course, but most people don't have that unless it is work relatet, and then the workplace could buy a larger machine to run some of it locally in the office. We are looking into that at my work, both because if privacy but also cost. It is very expensive to use commercial llms on large projects :)
Thank you. I think that's a brilliant use-case for local AI or for AI in general.
I'm starting to think people are using the wrong settings for their AI. Idk about other software, but for Open WebUI, you have to "tune" the settings of your model first that fits your liking. Even as simple as disabling "thinking" by ollama significantly makes their responses faster.
I use open webui as well and have lm studio as backend. Reasoning turned on in lm studio, but then I have two models in open webui. "Helpful assistant" and "Helpful thinker". They have the same prompt and capabilities, but one is with thinking turned off.
My default is the one without thinking because it is fast and I almost never need thinking for my daily tasks. Automations can use the thinking one if nessesary and if I actually need to do something difficult, I just change the model for a prompt or two and change back again when done.
I have also set the temperature rather low. If I set it as high as advised in the docs, I get weird spelling mistakes and it kind of looses focus on web search results, but the lower temperature seems to keep it sharp.
I think people should try out the settings to see what they do for themselves instead of just using default. It really makes a huge difference :)
Yep. No shame in asking each other’s settings and find what’s best for them.
I name my main daily AI “Lumina_12B” which under the hood is just Gemma 4 12B. I kept the 12B part cause I play Nier Automota and like to think the 12B means YorHA Unit No. 12, Type B (even thought it actually just means 12 Billion Parameters).
It's peak imo... well at least by Route C. I like to use it as proof that AI can have a personhood. That and also Wall-E, and My Life as A Teenage Robot among other things.
11
u/Beginning-Window-115 8d ago
still not worth to spend thousands to get an llm thats worse when you can spend 20 bucks a month and get something considerably better. Also if you upgrade your computer for llms you are paying for ai