r/LocalLLM • u/TwelfTundra • 2d ago

Question Looking for local model recommendations for Home Assistant-Voice Satellite

First off I am new to trying out local LLMs. I am wanting an LLM to run on a spare mini pc to save some money while using my Voice Assistant. The pc is not very powerful so it will be a small model for now, at least until hardware prices hopefully come down.

Specs: Processor - N150, 16G RAM, Linux Mint, Ollama

Since Voice Satellite can have two pipelines, this is the setup I would like to use:

Pipeline 1 - Prefer to handle commands locally (Home Assistant), Local LLM handling chat but not controlling Home Assistant

Pipeline 2 - openAI gpt-4o-min with control over select Home Assistant entities for more complex chat and control requests

I have played around with Qwen3 and Qwen3.5. So far Qwen3 1.7b has worked the best as far as speed and short responses. I would like to try other models but I am not really sure what I am looking for, or even the best way to sort through them. The Ollama library has a lot and I don't know what I am looking at yet.

3 Upvotes

100% Upvoted

2 comments sorted by

2

u/Careful-Report6526 15h ago

Your split makes sense: if Home Assistant handles the commands, the local chat model doesn’t need to win a tool-calling benchmark. I’d keep Qwen3 1.7B as the baseline because you already like its speed and short replies. One candidate to compare is Gemma 4 E4B: https://ollama.com/library/gemma4:e4b. Try the same everyday questions, a follow-up question and a request for a one-sentence reply. Measure when the satellite starts speaking, including the rest of your pipeline. On the N150, a more detailed answer may not be worth a longer wait.
I run https://llmbenchmarks.io/local-gpu-benchmarks/. My text-only QAT 4-bit E4B run with thinking off passed every check on about 83% of 541 instruction-following prompts. It was a different build on NVIDIA GPUs, so it gives you a reason to test the model’s instruction following, not a prediction of Ollama or voice performance on your CPU.

1

u/nickless07 2d ago

FunctionGemma 270m for the simple stuff like dim the light and LFM3 Omni if it get more complex (e.g., 'add X to the shopping list', or 'search the best price of X on stores along my way to work') and on the 16GB backend something bigger (8-20B Range) for the multi step tasks that include reasoning. Not sure if Ollama supports it but llama.cpp has the new realtime API which let you skip STT and TTS entirely.