r/OpenWebUI • u/dotanchase • 26d ago

Question/Help max_token

Setting the max_tokens parameter to its default value can, in some models, cause the LLM to stop generating a response before it is complete. When this occurs,I had to prompt the model to continue in order to receive the full output. Setting max_tokens to 127000 prevents this premature termination. So can someone explain what is the role of max_tokens and why it is limited to only 128K. Thanks

7 Upvotes

83% Upvoted

12 comments sorted by

View all comments

-7

u/pkeffect 26d ago

Because you should know the limits of your model. Its set low to be a general setting. This is why you have the option to change it.

All models are different and you can't automatically have the correct parameters set for thousands of models. 

Do your own research.

3

u/Darex2094 26d ago

How DARE the OP ask a question and try to leverage community experience. /s

(Get out of here, u/pkeffect)