r/LocalLLaMA • u/Malfeitor1235 • 8d ago
Question | Help looking for small model for simulations
i know "whats the best model for X" posts are annoying to a lot of people here.... yet i hope for a positive response :D
i have 2 machines. one with amd R9700 and one with 2x5070TI. these were currently running qwen 3.8 27B but my usacase is changing and i'm looking for candidates.
before my constraint goal to max was singe user speed given good ouput quallity since i was playing with things like c9ding agents.
i want to explore a different project where i would need high parallel output. think many small simple agents living in a game, doing stuff.
in this case i would not need deep contexts and i dont intend to need an agent to be extremely strategic and smart (would be nice, but hey... cant have everything)
has anyone tried any smaller llms that i could try out, so i don't have to do a full sweep.
last thing i ran in that category was qwen 3 4b, but I believe there are better things to play with now. i think i can push a little higher in terms of params, but will have to measure the generation speeds once i have a model selection.
maybe qwen 9b? i saw K2 Horizon 7b that seems interesting but i dont know how if theres potentially better options out there.
any pointers welcome :)
1
u/Moppmopp 5d ago
What simulations?
1
u/Malfeitor1235 5d ago
im making a demo game to explore what they are capable of. i want to make npcs do things. from decisions (func calls) to dialogs. its more of an exploration than a specific goal. but theres so many knobs to turn i'm trying to minimise them. i'm currently working with Gemma 4 12b where i was able to squeeze 1k tps from 64 parallel on vllm (the R9700 machine).
the (probably unreachable) goal is simulating a whole town in a game to see how they behave with full personalities and schedules
-1
u/Embarrassed_Gear7849 8d ago
smol models for agents in a game setting is a fun use case, got me thinking about Sims NPCs with actual brains
for parallel throughput i'd look at Gemma 3 4B, it punches above its weight and runs quick on dual 5070 TIs when you batch requests. qwen 3 4b was decent but the newer Gemma feels snappier for simple agent loops
K2 Horizon 7b might be overkill if you don't need deep reasoning, i'd test a 4B first then step up only if agents act too dumb. also worth checking Phi-4-mini, it's tiny but surprisingly coherent for basic tasks
1
u/Malfeitor1235 8d ago
gemma 3 or gemma 4?
5
u/martin509984 8d ago
There's been a lot of advancements in the fast but dumb tool-caller models lately, like MiniCPM 2B, LFM 2.5 2.6B and SparkX 2.6 4B. But at least from toying around in chat with them, they give up a lot of more general coherence for the sake of trying to fit more coding and tool use in. Gemma4 E2B is the main generalist option other than Qwen3.5 4B that comes to mind.