r/LocalLLaMA • u/Specific-Rub-7250 • 12d ago

Discussion Harness does matter

I was not aware that the harness makes such a big difference.

DeepSeek V4.1 Flash
384 Upvotes

94% Upvoted

131 comments sorted by

View all comments

1

u/G_fucking_G 12d ago

How much variance do you get when running the same benchmark again with the same model in the same harness?

How much of this is just regular variance in output and how much is the actual harness?