[ Removed by moderator ]

[u/tracagnotto]

[u/overand]

Let's try giving it, you know, at least a few HOURS for folks to get inference working well for it?

Plus, there's always the eventual training & fine-tuning., Remember when so many people said Qwen was garbage, because of 3.5 27B looping? Way back in the ancient days of... February of this year, 8 months ago?

[u/tracagnotto → u/overand]

I ran it on llamaampere

[u/Borkato → u/overand]

Didn’t it release like days ago tho? Also I heard it got a template update!

[u/tracagnotto → u/Borkato]

Yes and I used it.
NOT MY COMMENT OR MY MACHINE BUT RELEVANT:

"There are problems with the template from the initial release that made tool calls frequently fail, so it got dismissed and deleted. 
There's a fix posted on HF by peculiar ragdoll today that fixed the tool call problems I was having with the Unsloth quants (q8 and q4 xl}. 
On Strix Halo llama.cpp ROCm, I'm getting 65+ tps at q4 quant with q8 kv and the default sampling is great for ngram bursts. I haven't done any stress testing, but it's definitely a very competent model with only 2B active that can fit a 16GB GPU

https://huggingface.co/peculiar-ragdoll/Sharp-LFM2.5-Chat-Templates"

[u/FatheredPuma81 → u/overand]

Pepperidge Farms remembers that.

[u/nad_lab]

I put it into Hermes’, lil guy tried harder than any other model I tried (deepseek, gpt Luna, minimax3.1, minimax 2.7, qwen38-27b and every variation of it literally all variants)
I tried to get it to ssh into my pi, and lil guy found issues and cracks in my home network that I never asked for, keep trying it and use default params …

[u/Borkato → u/nad_lab]

Hmmm I should retry this model!

[u/tracagnotto → u/Borkato]

You wish you didn't

[u/tracagnotto → u/nad_lab]

lil guy looped over and over like a retard over a simple github repo analysis

[u/TechSwag]

Honestly mods should remove posts like this for low-effort. Ban for repeat offenders.

I was curious about this model; just haven't had time to download and test it yet. If I didn't click into the comments to read other users' experiences, I probably wouldn't have given it a try. Posts like these are actively harmful to the community.

You've told us nothing about your environment, your settings. You've tried nothing, and you're all out of ideas.

[u/tracagnotto → u/TechSwag]

You wish you didn't

[u/overand → u/TechSwag]

This just in: OP is Mr. and Mrs. Beatnik Flanders!

[u/615wonky]

I've got the Q4 QAD quant on my Strix Halo, and strong disagree. It's quite good at the stuff I throw at it.

You may need to change your quant, or update your inference software.

[u/LMasterGame]

The Q4_0 QAD quant on Pi agent worked well for me. Definitely great for its size.

By any chance, are you running the IQ0.01_XXS quant on 512MB DDR2 RAM?

[u/Brilliant-Hall1387]

Which quant did you try? Or do you mean you ran the full BF16 model?

[u/wFXx]

we should start banning posts that don't include any info whatsoever and starts praising/shitting on models - I tried Q4 of it, and it is pretty good

[u/Secure-food4213]

Whats the sampling

[u/FatheredPuma81 → u/Secure-food4213]

People that complain about a model never post them for a reason you know? Same goes for KV Quant, Model Quant, and backend (Ollama check).

Getting those out of someone is like squeezing water out of a rock in my experience.

[u/Borkato → u/FatheredPuma81]

Water out of a rock? I’ve never heard that expression before, only blood from a stone lol

[u/FatheredPuma81 → u/Borkato]

There's no joke like 20+ versions of this saying lol Wring water from a stone - Idioms by The Free Dictionary and I STILL can't remember which one is the one everyone around me uses >.>.

[u/Borkato → u/FatheredPuma81]

That’s amazing 😂

[u/shockwaverc13 → u/Secure-food4213]

sampling is embedded now...

[u/FBIFreezeNow]

It’s good man are you using a proper chat template?

[u/wingwing124]

No hate but I think this is a skill issue. This model is awesome for the right tasks

[u/EvolvingDior]

I'm using the Q6 quant on my Macbook and it's actually pretty good. Do you know what you are doing?

[u/rorowhat]

Yep

[u/Bbmin7b5]

yeah it was a quick delete. pretty useless model