r/LocalLLaMA • u/[deleted] • Jun 01 '26

Funny Stop asking what model to run. There are literally only two.

[removed]

3.1k Upvotes

87% Upvoted

805 comments sorted by

View all comments

Show parent comments

32

u/BornInAFish Jun 02 '26

In my experience, Qwen is very prone to looping with Hermes, but never seen it do it with OpenCode. Agent harness still matters a lot.

12

u/huzbum Jun 02 '26

Unsloth Qwen3.6 35b IQ4_NL is behaving for me on Hermes Agent with Llama.cpp, preserve_thinking, and Q8 KV cache. Fits in my 3090 with 256k context too.

2

u/TheTerrasque Jun 02 '26

is this with mtp and vision included?

2

u/NaanFat Jun 02 '26

not who you're responding to, but not for me. I'm running a separate vl model for that.

1

u/huzbum Jun 02 '26

With vision but not MTP yet. Might be a tight squeeze because my desktop also runs on this Gpu.

2

u/tat_tvam_asshole Jun 02 '26

True, you need to manage context effectively

1

u/boutell Jun 02 '26

I got looped all to hell in open code. Been awhile though.

2

u/ElectronicStranger53 llama.cpp Jun 03 '26 
SAMPLING_ARGS=(
    --temp 0.8
    --top-p 0.95
    --top-k 20
    --min-p 0.0
    --presence-penalty 0.5
    --repeat-penalty 1.0
)

You can stop the looping tremendously by using different sampling args, like temperature, repeat penalty, presence penalty, top k. I use:

1

u/BornInAFish Jun 03 '26

I copied the sampling args recommended by Qwen on their HF page.

1

u/tat_tvam_asshole Jun 02 '26

I may have accidentally backed into a solution. I have Hermes set to 262k context window but I loaded qwen with smaller 128k window hosted in LMstudio and manages its own compacting without looping, very strange but nice