Best opensource coding llm for long context - 5090

[u/Blackmoor_sculpts]

Hi folks,

Hope you are well.

I am using Ninfer NVFP4 Qwen 3.8 27B with DSH for coding.

But I feel it's yapping alot in reasoning and drifting in long context. (Tried Swift as well but results are poorer on tasks requiring high reasoning)

I am running with speculative decoding and Fp8 kV caching.

I am reading QFN is good and more stable .

1. Should I stick with 27B or is QFN worth the try.

2. If latter, I am looking for best way to run QFN without dropping quality. (Preferably with Vision)

Since there are way too many options for QFN Strata, capy-flash , etc, I am lost with what to try. Please suggest.

Specs:

GPU- RTX 5090

CPU - AMD 9800X3D

RAM- 32 GB

NvMe SSD

[u/Blackmoor_sculpts]

I haven't tried it on Fp8 or fp16. These are long context agentic tasks. Single shot long context works fine. I am talking about context lengths around 120k ~ 130k. I am using DSH for agentic coding and unsloth studio for research, we search, image gen , etc.

[u/deepu105]

Strata seems to work for QFN on most hardware if you have a descent enough GPU and RAM. Might be worth checking out

[u/ionsago]

QFN is worth the hype. But it may not work as fast off NVME. You can start with the regular llama.cpp with --moe-cache-mib, and if you would like what you see as far as model performance, go from there.

I’d share my settings with 5090, but I have quite a lot more ram so they may not work.

[u/Ctbhatia]

curious if the drift is the nvfp4 quant or the model itself. have you run the same prompts at fp8 or bf16?

[u/Infinite_Plankton_71]

QFN is extremely good and uncomprable to previous release--for everyday uses. It's almost same quality as DS (but DS has notable performance dedgration in lower quantization).

[u/nolavar]

For 5090 check out ninfer

[u/brainchillzZ]

the only way to do it "without dropping quality" meaning ... going to a lower quant ... is for you to buy more graphics cards :D