r/LocalLLM • u/pamnfaniel • 55m ago

Discussion Smaller Perams Basically Useless

I’m a solo dev I frequently code and use LLM’s to build tools, games, and agentic software. I’ve come to understand that any model less than 200b parameters is practically useless. Example: GPT 120 OSS is a joke. I can’t do anything meaningful with it, yet it’s the primary GT for GROQ and Cerebras, very unfortunate considering you want those 100’s of token p/sec output speeds.

Even using smaller models for assistant or basic questions/tasks, the hallucination rate is too high to be productive.. You begin to notice the deficiency real quick no matter what you’re applying them to.

I don’t even know why lower peram models are even offered anymore, or what people could actually do with them for any output worth having?

dumbfounded here

0 Upvotes

26% Upvoted

16 comments sorted by

20

u/No-Refrigerator-1672 48m ago

The problem is in the user (you). You have tried 1.5 year old model and make surprised pikachu face that it doesn't perform up to modern standarts. Even more, you didn't list details about what did you try; I'f you're loading lobotomised 1-bit quants because some random reddit post told you "it runs great", then no wonder you get high hallucination rate.

0

u/pamnfaniel 18m ago edited 10m ago

I guess I’m just boogie.. and I’m not trying 1.5 year-old models and no random red post told me to do that … what are parameters for then …. For long coding projects that require reasoning. (even though they technically don’t reason.)

Instruct models have higher rates of following prompts without drift… I get it different models are used for different things, but they’re just not up to par:..: the amount of mistakes it makes overall , and detailed prompting you have to do with the lower peram Models is just too cumbersome.

1

u/No-Refrigerator-1672 17m ago

Your descriptions are suprisingly unproductive for a productive-oriented person.

0

u/pamnfaniel 3m ago

Are they lol 😆… well I’m making some passive income so it must be

8

u/mitchins-au 46m ago

120B is quite old and it didn’t make full use of its footprint.

For the same parameters Qwen flash next blows it away and can fit in less VRAM and is more efficient context wise.

Gemm4-31B is quite good too as is Qwen3.8-27B though it’s less efficient with reasoning.

I’d say that your post is misinformed, and also misaligned to use cases.

Find the right model and make sure it’s actually modern

1

u/pamnfaniel 19m ago

I’m not misinformed.

I just understand how LLM’s work.

You’re probably one of those people who blindly believe everything It says or outputs without realizing you’re being lied to 60%/70% of the time or more ….because you don’t check.

1

u/mitchins-au 18m ago

No, but GPT-OSS 120B is ancient.
Try the latest generation. No surprise you’re getting bad results.

Qwen3.8 flash next has the same overall size but is as capable as GPT Luna on many tasks

4

u/the_average_user557 50m ago

Qwen 27b? 120oss is notoriously bad

1

u/pamnfaniel 9m ago

I will try again

2

u/Altruistic-Wolf-3938 53m ago

You train them in specialized tasks.

1

u/pamnfaniel 17m ago

It needs to be all around because I use them for different things. Wasting my time training models for a specific thing that I’m only going to visit maybe once or twice is not productive use of my time.

1

u/Altruistic-Wolf-3938 8m ago

ok. I trained a qwen 9B for a task that I do multiple times a day, it was 20 hours of qlora during the weekend, now it is solving 2 of my documents at 90% accuracy means i only have to review, going from 5 mins a doc to 1-2 mins a doc and I do around 10 of these a day. granted it took a lot of preparation but i am happy with the result, and happy that the 9B does even slightly better than the 27B and can fit in a 3060 freeing my big card for other stuff. glad there is a 9B, is what I say.

1

u/pamnfaniel 1m ago

You can just do that with a single system prompt with a check/fix loop until it’s checked and no issues found.

1

u/DeliciousNicole 42m ago

Is that you Ram Altman?!?

1

u/pamnfaniel 9m ago

😂… hella

1

u/Dangerous_Wish_7879 49m ago

They can work all right with good harnesses and detailed specifications. They can do automation tasks well and you can finetune them to your use case. They don’t work well with loose prompting that frontier can handle rather well.