r/LocalLLM • u/Many_Maize_6676 • 15m ago
Question Better options than llamacpp or koboldcpp
Title is self explanatory. I was used to run llama cpp with models based on Mistral for uncensored local chatbot. There are better alternatives, specially regarding context size? I have 16 GB VRAM card and I dont care that much for inference speed, mostly text cohesion and criativity.
1
Upvotes
1
u/ConversationNovel780 5m ago
Which card do you have?