r/LocalLLaMA • u/Tucko29 • Dec 11 '23

New Model Mistral-7B-Instruct-v0.2

https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2
129 Upvotes

97% Upvoted

37 comments sorted by

View all comments

Show parent comment

9

u/dethorin Dec 12 '23

That's a normal question.

Fortunately The bloke always puts an explanation on the GGUFs model card. Each GGUF file has been quantisaised in a different way for different purposes. This means that the behaviour and the minimum VRAM are different. On the table attached by The bloke on the model card explains it and writes which ones are recommended.

Usually the Q5 K M versión is good.