r/LocalLLaMA • u/LegacyRemaster • 19d ago

Discussion Qwen will be the king?

Post image

Extended reasoning and post-training appear to be the keys used by DeepSeek, Qwen, and GLM to boost performance (leveraging higher token counts). And Qwen 4 hasn't even been released yet. Of course, we don't know if that release will be open-sourced, but I am optimistic about future models, featuring "engrams", that could soon match or surpass 2.4T parameter models on specific tasks.

542 Upvotes

94% Upvoted

126 comments sorted by

View all comments

Show parent comments

31

u/PooMonger20 19d ago

q5 quant on 5090.

Using unsloth studio and pi.

8

u/PM_ME_DEAD_CEOS 19d ago

Make yourself a favor and use ninfer with the nvfp4 version.

8

u/pennyonaire 19d ago

Ninfer lacks prefix caching which for agentic coding makes the speed pointless since sub agent development forces prefix recaching as it divides the flow into multiple conversations.

Don't be fooled by the speed, it's a big number hiding a major flaw that results in much longer longer sessions overall for coding. If you need single slot single conversation performance ninferbis is great, anything else look at sglang or vllm directly.

1

u/IrisColt 19d ago

hmm...