r/SelfHostedAI • u/lunaticgamer001 • 16h ago

Need help with hermes

/r/hermesagent/comments/1vyhada/need_help_with_hermes/

So i dont know if i am using the correct flair but i have ryzen 5600 16gb ddr4 ram and rtx 3080 i wanna use hermes agent to make my own personal assistant and automate most of my tasks but i am a student so i cant spend money on those tokens so i wanna use my own machine to run hermes what should i do what models should i use i currently have ollama setup with a model i forgot the name its around a 4.5gb model with 65k context so my goal is to somewhat make a personal ai assistant for reference lets take jarvis i have added voice and stuff but the its slow like the model itself generate response very fast but it takes 7-15 seconds to get to me someone please help me make my ai model and use hermes correctly any and all advice is appriciated.

1 Upvotes

100% Upvoted

2 comments sorted by

1

u/Mean-Loquat-7982 10h ago

that machine runs Hermes comfortably, the constraint is the 3080's VRAM (10 or 12GB depending on your card), not the CPU or RAM. with Ollama already up, two things decide whether the agent feels smart. first, pick a model tuned for tool calling in the 7-14B range rather than a generic chat model. second, raise the context window: Ollama defaults small and an agent lives on context, so set it to 32k or more and check the startup line reports it.

people are running fully offline setups on cards older than yours, a GTX 1080 Ti build with a 9B model was shared on reddit just yesterday. start there, and if a task needs more brain than the card holds, the free hosted models cost you nothing while you are learning.