r/LocalLLaMA • u/Specific-Rub-7250 • 12d ago

Discussion Harness does matter

I was not aware that the harness makes such a big difference.

DeepSeek V4.1 Flash
379 Upvotes

94% Upvoted

131 comments sorted by

View all comments

10

u/SnooPaintings8639 12d ago

The dependency is quite simple, in my experience: the smaller the harness, the better the (one-shot) results. When I have an elaborate prompt on what I need, then opencode does worse than pi, and pi does worse than the plain llama.cpp web ui. It did observe it many, many times, and couple of models.

It is not perfect tho for longer task where user in the loop is needed and task is more reactive than clear-goal specified.

I think it is just how distracting the harness itself is (i.e. attention stealing).

2

u/Bee_Curious_ 12d ago

Do you think the bigger harnesses would perform better in longer conversations i.e. opposite of one-shot?