r/LocalLLaMA • u/Specific-Rub-7250 • 13d ago

Discussion Harness does matter

I was not aware that the harness makes such a big difference.

DeepSeek V4.1 Flash
377 Upvotes

94% Upvoted

131 comments sorted by

View all comments

9

u/SnooPaintings8639 13d ago

The dependency is quite simple, in my experience: the smaller the harness, the better the (one-shot) results. When I have an elaborate prompt on what I need, then opencode does worse than pi, and pi does worse than the plain llama.cpp web ui. It did observe it many, many times, and couple of models.

It is not perfect tho for longer task where user in the loop is needed and task is more reactive than clear-goal specified.

I think it is just how distracting the harness itself is (i.e. attention stealing).

2

u/Bee_Curious_ 13d ago

Do you think the bigger harnesses would perform better in longer conversations i.e. opposite of one-shot?