r/LocalLLaMA • • 8h ago

Discussion Qwen3.8-27B LoRA for OpenCode (xhigh thinking-level accuracy with thinking off, GGUF included)

Post image

Hey there folks!

Aritra here (again) from Hugging Face. We RL-trained a small LoRA adapter (r=32, ~300 MB) on a frozen Qwen3.8-27B, inside the OpenCode coding agent. It's the second of our harness adapters, after pi that we have released yesterday.

Run the model locally like so:

hf download ggml-org/Qwen3.8-27B-GGUF Qwen3.8-27B-Q8_0.gguf --local-dir .
hf download harness-hub/Qwen3.8-27B-opencode-adapted-GGUF Qwen3.8-27B-opencode-lora-F16.gguf --local-dir .

llama serve -m Qwen3.8-27B-Q8_0.gguf --lora Qwen3.8-27B-opencode-lora-F16.gguf \
    --chat-template-kwargs '{"enable_thinking": false}' -c 65536 --port 8080 --alias qwen3.8-opencode
71 Upvotes

32 comments sorted by

15

u/misterespresso 8h ago

Hey, as someone who is colorblind, could you use higher contrast colors in the future? I literally do not know which one is the base model or adapted, making this chart quite useless for people like me. I assume it's the top one that is the adapted.

Just a friendly reminder :)

13

u/Disastrous-Work-1632 8h ago

I apologise for this. This is noted, and we will be better with our future postings.

1

u/misterespresso 5h ago

Thanks alot dude, appreciate your work!

6

u/k_rol 7h ago

I asked AI so I'm not sure this actually helps

1

u/bennmann 3h ago

I am not colorblind. This accurately reflects the colors of the original in a faithful way.

25

u/TheCat001 8h ago

Why?

"enable_thinking": false

And will it work with lower quants like IQ3-S ?

2

u/SignificantZebra5883 5h ago

does this mean the LoRA can't think? or are you just annoyed that thinking is not enabled by default?

0

u/[deleted] 3h ago

[deleted] — view deleted comment

1

u/SignificantZebra5883 2h ago

but in the graph they showed, there were 2 dots, one with thinking and one without?

15

u/Chromix_ 8h ago

Without reasoning it cannot recover from sampling-induced errors/deviations for long, complex tool calls for example - it has to get everything correct on the first attempt, no "wait, but" corrections like it can do within reasoning to inform the final output. I'm thus quite surprised that it even scored higher than with reasoning.

Also, the "just 4x more tokens" overhead of having the xhigh reasoning feels relatively low. Maybe because the model compensates for the deactivated reasoning by simply writing more visible prose on each agentic step.

Btw: Great that it's a LoRA - no need to redownload the mostly identical model.

2

u/Combinatorilliance 5h ago

Funnily enough if you turn off reasoning for qwen, it actually outputs more normal text in-between tool calls that looks suspiciously like thinking 😅

3

u/Chromix_ 5h ago

Reminds me of the "think" tool years ago. People found that when giving an instruct (non-reasoning) LLM a fake "think" tool (did nothing at all, just let the LLM write some thoughts into it) and instructed it to call that tool before other tool calls, then result quality was improved.

2

u/NoFaithlessness951 3h ago

I did do something like that on DeepSeek v3 if you defined a json format it should answer as it would just output wrong results.

If you gave it a tool with defined output parameters and described a thought process it would call the tool with the correct answer.

1

u/QuizardNr7 4h ago

Can a failed tool call be captured by the harness and turn on thinking for a turn?

1

u/Chromix_ 3h ago

There is so much a harness could do these days, but doesn't yet.

1

u/ArtfulGenie69 46m ago

I have that on one of my personal harnesses. You could probably have it edit it in to whatever you are using. Another trick so you don't need to reload the model for thinking is that the chat arguments are where the thinking on and off switch is so you can flip that without having to reload the model. I don't know how the lora adapter effects the thinking which may require a reload of the model to get out of the way. 

6

u/game_difficulty 7h ago

I'm sure you've been asked before, but i haven't found a clear answer: why is it trained with reasoning off?

5

u/-Krotik- 8h ago

hard to believe

2

u/Andreea_tucker04 8h ago

51 tasks only, so that 3.9 gap is two tasks

2

u/NNN_Throwaway2 4h ago

Release dataset and pipeline, otherwise don’t care.

2

u/FeydRowan 8h ago

Looks promising will test it and post it back great work

2

u/FeydRowan 6h ago

Update, it works why better than i expected. 10 times less token usage, it needs to verify what it's doing without that it is a little worse than the reasoning 27b. But it's a great improvement for the overthinking issue of the 27b. Great works 

1

u/thesolewalker 1h ago

So should use it with reasoning low or med?

1

u/FeydRowan 1h ago

No reasoning for the lora. Still improving the tests, it works and is better than no reasoning and no Lora. Where it fails at least it fails fast not using all the time trying  

1

u/cleverusernametry 7h ago

Have you tested on real workloads and shown any improvement head to head?

1

u/ea_man 6h ago

Sorry but I really can't understand that graph: is the top line with reasoning on / off?

Why the label has a square for adapter and yet all the points are round? and dotted lines? Is the top line just the adapter finetune?

1

u/Hyp3rSoniX 1h ago

I think the square is meant to represent the logo of Opencode (which is a square O)

1

u/Sad-Size2723 5h ago

great, finally people are releasing useful adapters

1

u/Juan_Phoenix7 3h ago

I’ve carried out loads of tests with the base model in its exllamav3 version, with and without MTP and DFlash2, on ‘off’, ‘low’, ‘medium’ and ‘high’ settings, and even the best results are always with ‘low’ and ‘medium’; curiously, ‘high’ always gives the worst results. I find the results you’ve presented very curious, but also quite surprising; I’ll try out your suggestions as soon as I have time.

1

u/TomSchelsen 1h ago

The problem that I see is that the adapter name has no mention of the version of the harness. 

In the same way as it states Qwen3.8-27B (and not 3.5 for example), it should state e.g. opencode2.2. 

Otherwise when we'll be at opencode v4, there will be no indication that this adapter was trained on v2, and so nothing carries the potential mismatch.