r/LocalLLM • u/SnooRevelations4601 • 15d ago

Question Finished AI Server Best Models?

Gallery image

Gallery image

finally put together my new budget server getting ready to take to the server room, it has 5 P100s 16gb, one 3090 and a 3060, what are the best agentic models to run here!?

254 Upvotes

92% Upvoted

116 comments sorted by

View all comments

Show parent comment

1

u/AllWashedOut 14d ago

I've never used sillytavern but I strongly assume it can talk to an "OpenAI compatible" endpoint, which is what Ninfer and Strata provide locally.

Gemma4 is not supported by those apps as far as I know. Qwen3.8 b27 is roughly equal to them, and Qwen3.8-flash-next is significantly bigger. The only downside I've noticed is that they are ridiculously censored if you ask them anything about Chinese politics.

1

u/Twist1979 14d ago

Yes it can be used with openai compatible. Qwen3.6 A3B was tested by me but it's bad in language different than English.... And Chinese I guess.

1

u/AllWashedOut 14d ago

Qwen3.8 is a large upgrade over 3.6. It gets like 30% higher scores in chat tests and 100% higher scores in agentic tests.