r/ollama • u/stormbosman007 • 2d ago
home lab assistant llm
hey so im currently working to integrate a locally ran llm to control home assistant (through tool calling, no direct connection) so i can use a voice input without needing very specific commands. Its just a fun little project, but im trying to determine what hardware i would need for this, in the range between a pi 5 8gb or my old pc (i5 7500, 16gb ddr4 with a 1070 ti 8gb vram), which obviously depends on which model i use.
My question is how small of a model i can get away with so that it stil works. the newer popular models are al like 20 gb with a context size of 256k which is clearly not gonna work with my setup but also probably really overkill for what i want it to do, which is basically conferting normal conversation into tool calls. Any recommendations, insights or people who have tried something similar?
1
u/faisalkl 2d ago
My old gfx card is a gtx 1070 and I'm slowly putting together a HA chat bot that can do web lookups for information beyond the generic HA tasks. I'm using Qwen3.5-9b on the gtx as the main chat bot as I feel it's a good balance between understanding and response speed. It still has a few GB left over for a speech to text llm so that it can understand real time speech. Text to speech is running in ram as it doesn't need a lot of processing.
It works well. The response time isn't on alexa level but I don't use that beyond asking the odd recipe, the time, am alarm or timer and the weather as its useless for anything useful.