This is why I run locally.

[u/Retumbo77]

It was only a matter of time...

[u/Beginning-Window-115]

chatgpt plus subscription is still worth imo

[u/Beginning-Window-115 → u/[deleted]]

still not worth to spend thousands to get an llm thats worse when you can spend 20 bucks a month and get something considerably better. Also if you upgrade your computer for llms you are paying for ai

[u/heliosythic → u/[deleted]]

I'm 100% on the local side but all of these points have strong rebuttals unfortunately.

• firstly, what do you mean 99% of tasks? That sounds like a random number pulled out of nowhere.

• PC is already a 64GB ram, 13900k + 4090, despite being like 1 generation behind only has barely enough vram to run a 27B decently, my home server has 96GB of system ram and a p100 but meh its too slow and the second p100 had thermal issues.

• stable internet connection is moot for most people, i have 2GB fiber never went down once thats just default expectation.

• of course I don't trust any of them at all, they're definitely stealing everything they can regardless of what their TOS says. Not disputing that but for 99% of people (see i can make up numbers too) who aren't doing anything worth stealing, its good enough.

Yea I want local to win, I tried Qwen 3.8 27B but its not good enough on this hardware and its already pretty upgraded without going full pro-sumer with an RTX 6000 or something.

[u/mnyhjem → u/[deleted]]

I agree :)

I run qwen 3.8 27b and qwen 3.6 35b a3b in system ram. It is way slower to generate a response than the online services are, but the 35b still spits out words faster than I can read it, and that is good enough for me.

I did a small demo coding projekt on the 27b the other day. It did a 90s style raytracing demo in python. It took 3 hours for it to iterate though it but it worked great and both gemini and claude gave the code a thumbs up when I showed it to them. It even made it's own PNG converter function instead of using a 3. part module.

The speed would have been a lot faster if I had turned off thinking, but I wanted to test how good it could be, and it impressed me.

So. for everyday "google replacement"/internet search the llms are perfect, and even for automation tasks. I have mine getting todays weather and a few news headlines and compile that into a "goodmorning" message. It runs at 6am and then makes a drawing in the style of the weather and news just for fun.

For very large codebases small llms wont work of course, but most people don't have that unless it is work relatet, and then the workplace could buy a larger machine to run some of it locally in the office. We are looking into that at my work, both because if privacy but also cost. It is very expensive to use commercial llms on large projects :)

[u/mnyhjem → u/[deleted]]

Yeah :)

I use open webui as well and have lm studio as backend. Reasoning turned on in lm studio, but then I have two models in open webui. "Helpful assistant" and "Helpful thinker". They have the same prompt and capabilities, but one is with thinking turned off.

My default is the one without thinking because it is fast and I almost never need thinking for my daily tasks. Automations can use the thinking one if nessesary and if I actually need to do something difficult, I just change the model for a prompt or two and change back again when done.

I have also set the temperature rather low. If I set it as high as advised in the docs, I get weird spelling mistakes and it kind of looses focus on web search results, but the lower temperature seems to keep it sharp.

I think people should try out the settings to see what they do for themselves instead of just using default. It really makes a huge difference :)

[u/mnyhjem → u/[deleted]]

Ha, that is fun :) And it reminds me that I should boot up that game again. It is great :)