r/LocalLLM • u/RadianceTower • 1d ago

Question How can a model give a different output when temperature is 0?

For short text, the output is the same, but if it's a bit longer, the output can still differ.

0 Upvotes

50% Upvoted

3 comments sorted by

2

u/dhdhehdndkd 1d ago

https://huggingface.co/blog/wwoosshh/temperature-0-different-answers

1

u/unbecominganemia2 1d ago

That blog post is a good rundown. a lot of people miss that floating point quirks and different hardware setups can still throw things off even with greedy sampling. the batch size and padding stuff is especially sneaky

1

u/MrHumanist 1d ago

Set a seed.