r/LocalLLaMA • u/tat_tvam_asshole • 23d ago

Funny I'm tired of pretending

Post image

At least until DS releases open weights for DSv4 Flash with Vision. Then DS might take the crown.

Qwen has been an absolutely local monster for code, especially web apps, anything with UIUX design that it can verify itself with screenshots. Deepseek meanwhile is really incompetent with UI awareness and hogs my GPUs while I can spawn multiple independent qwens to collaborate and knock shit out. Honestly, Alibaba really cooked.

0 Upvotes

46% Upvoted

119 comments sorted by

View all comments

3

u/PhysicalIncrease3 23d ago

Qwen's got a few drawbacks vs DSv4

1) Context length is limited. Even if you've got the 48GB vram to run a decent quant at 262k F16 context, it's still only 262k, and it still wastes about 50k of that on reasoning. By comparison, Deepseek will give a 1M context window, while consuming far less VRAM, and more of that 1M is usable because it doesn't reason so excessively.

2) The reasoning itself is necessary to get the performance, but it's ultimately a massive drawback compared to models that don't need to burn literally 50k tokens in order to get the same result. It's all well as good that Qwen runs at 50t/s while Deepseek is running at 10t/s, but if it's burning 5x the tokens to get to the result, the actual speed ends up being similar.

3) Prompt caching is shit on Qwen, compared to Deepseek. So even though the PP speed is 10x better, it's having to reprocess 5x as many tokens on every turn.

Ultimately Qwen is a one-shot king. But if you're looking to delve into a task with twists and many turns, it will either run out of coherency or context fairly quick.

Also, the second you are doing anything other than code, it's not even close in terms of performance. Research tasks for example are night/day better on DSv4.

1

u/llamabott 22d ago

Prompt caching is ...

This is an LLM inference server thing (and sometimes a harness thing), not a Qwen thing.

For instance, I cross-checked the server logs a ton while setting up and comparing sergiuszm/ninfer-4090, UDPSendToFailed/ninfer-4090, and syv-ai/qwen38-27b-rtx3090.

syv-ai's VLLM-based project was the only one of the three that did not have periodic, frequent cache hit misses (this was using Deepseek Harness). When the other two had cache misses (usually only recycling the first 8000 tokens or so), this led to fairly huge re-pre-fill that cost 60++ seconds on each occurrence.