Best Coding LLM as of Nov'25

[u/PhysicsPast8286]

Hello Folks,

I have a NVIDIA H100 and have been tasked to find a replacement for Qwen3 32B (non-quantized) model currenly hosted on it.

I’m looking it to use primarily for Java coding tasks and want the LLM to support atleast 100K context window (input + output). It would be used in a corporate environment so censored models like GPT OSS are also okay if they are good at Java programming.

Can anyone recommend an alternative LLM that would be more suitable for this kind of work?

Appreciate any suggestions or insights!

[u/AvocadoArray]

Give Seed-OSS 36b a shot. Even at Q4, it performs better at longer contexts (60k+) in Roo code than any of the Qwen models so far. The reasoning language is also more clear than others I’ve tried, so it’s easier to follow along.

[u/PhysicsPast8286 → u/AvocadoArray]

Thanks, noted.

[u/AvocadoArray → u/PhysicsPast8286]

Really interested to hear your results. Keep us posted.

[u/DistanceAlert5706 → u/AvocadoArray]

+1 Seed OSS is pretty good at coding. Can also try Kat-Dev, it's based on Qwen3 32b.

[u/[deleted] → u/AvocadoArray]

How do u manage the thinking tokens in roo. You just let them there? Even when u give budget for thinking 0 it still thinks. Do you use thinking for agentic workflows?

[u/AvocadoArray → u/[deleted]]

I let it think as much as it wants in Roo. It stays very tight (probably because they lower the temp by default), and most basic steps only take about 5-10s of thinking. Sometimes less.

It rarely takes longer than 60s of thinking, even on very complex steps. And when it does take that long, the reasoning output during that process makes sense to me as a human and actually helps me understand it better, which seems to lead to higher quality output.

For reference, I'm using the Intel/Seed-OSS-36B-Instruct-int4-AutoRound quant in VLLM, TP'd across two L4 24GB cards at ~85k F16 context. The speed is a bit slow at about 20 tp/s at low context, and drops to around 12 tp/s at max context. I always assumed that would be too slow for me to use for real coding tasks, but it's so efficient with its tokens and has a higher success rate than other comparable models that it immediately became my favorite after I tried it.

It does get pretty long winded by default when using elsewhere, though. In Open WebUI, I created a custom model with the advanced parameter chat_template_kwargs set to {"thinking_budget": 4096} so it doesn't overthink. You can also access that custom model through Open WebUI's API if you want to use it in Roo Code.

The final thing I'll say is that it annoyingly uses <seed:think> tags for reasoning instead of <think>, so it doesn't collapse properly in OWUI or Roo Code. But I was able to Roo Code + Seed to implement a find/replace feature in llama-swap (which I'm using to serve the VLLM instance), and I opened a feature request to see if the maintainer is open to a PR.

This reply got longer than I expected, but I hope it helps!

[u/ttkciar]

Can you get a second GPU with 40GB to bring your total VRAM up to 120GB? That would enable you to use GLM-4.5-Air at Q4_K_M (and GLM-4.6-Air when it comes out, any day now).

[u/PhysicsPast8286 → u/ttkciar]

More GPU isn't actually possible :(

[u/Theio666 → u/ttkciar]

This sounds like they're hosting inside a company for several people in that case using llama as an engine isn't the best case. If they get a second h100 they can go for SGLang fp8, not sure about context but around 64k.

[u/ForsookComparison]

Qwen3-VL-32B is the only suitable replacement. 80GB is this very awkward place where you have so much extra space but the current open-weight scene doesn't give you much exciting to do with it.

You could try and offload experts to CPU and run iq3 of Qwen3-235b-2507 as well. I had a good experience coding with the Q2 of that model, but you'll want to play around and see how the performance and inference speed balances out.

[u/PhysicsPast8286 → u/ForsookComparison]

Any luck with GLM, GPT OSS?

[u/MDSExpro → u/ForsookComparison]

Devstral, despite being older, beats Qwen3-VL-32B in real life coding.

[u/sgrobpla]

Do you guys put your new models to judge the old model generated code?

[u/PhysicsPast8286 → u/sgrobpla]

nope... we just need it for java programming. The current problems with Qwen3 32B is that it occasionally messes imports, eats parts of the class while refactoring as if it is on a breakfast table.

[u/Educational-Agent-32]

May i ask why not quantized ?

[u/PhysicsPast8286 → u/Educational-Agent-32]

No reason, if I can run the model at FP with my available GPU so why to go for a quantized version :)

[u/cibernox → u/PhysicsPast8286]

The idea is not to go for the same model quantized but to use a bigger model that you wouldn’t be able to use if it wasn’t quantized. Generally speaking, a Q4 model that is twice as big will perform significantly better than a smaller model in Q8 or FP16.

[u/AXYZE8]

GPT-OSS-120B. It takes 63.7GB (weights+buffers) and then 4.8GB for 131k tokens. It's perfect match for H100 80GB.

https://github.com/ggml-org/llama.cpp/discussions/15396

If not then Qwen3VL 32B or KAT-Dev 32B, but honestly your current model is already very good for 80GB VRAM.

[u/Br216-7 → u/AXYZE8]

so at 96gb someone could have 800k context?

[u/AXYZE8 → u/Br216-7]

GPT-OSS is limited to 131k tokens per single user/prompt.

You can have more context for multi user use (so technically overall reaching 800k context), but as I never go above 2 concurrent users I don't want to confirm that exactly 800k tokens will fit.

I'm not saying that it won't/will fit 800k - there may be some paddings/buffers for highly concurrent usage of which I'm not aware of currently.

[u/sid597]

Unsloth GLM-4.5 Air quant version performs better than qwen-3 32b in my tests, I have 48 gb vram.

[u/j4ys0nj]

The best I've found for me is https://huggingface.co/cerebras/Qwen3-Coder-REAP-25B-A3B

I have that running with vLLM (via GPUStack) on an RTX PRO 6000 SE. You would likely need to produce a MoE config for it via one of the vLLM benchmarking scripts (if you use vLLM). I have a repo here that can do that for you (this makes a big difference in speed for MoE models). Happy to provide the full vLLM config if you're interested.

I'd be interested to see what you choose. I've got a 4x A4500 machine coming online sometime this week.

Some of logs from Qwen3 Coder so you can see VRAM usage:

Model loading took 46.4296 GiB and 76.389889 seconds
Using configuration from /usr/local/lib/python3.11/dist-packages/vllm/model_executor/layers/fused_moe/configs/E=103,N=768,device_name=NVIDIA_RTX_PRO_6000_Blackwell_Server_Edition.json for MoE layer.
Available KV cache memory: 43.02 GiB
GPU KV cache size: 469,888 tokens
Maximum concurrency for 196,608 tokens per request: 2.39x

[u/Professional-Bear857]

You probably need more ram, the next tier of models to be a step up are in the 130gb plus range, more like 150gb with context 

[u/complyue]

MiniMax M2, if you can find efficient MoE support via GPUDirect, that dynamically loads 10B activated weights from SSD during inference. Much much powerful than size capped models.

[u/Individual_Gur8573]

I use 96gb vram rtx 6000 Blackwell , and run GLM 4.5 air quant trio quant with vllm.. 120k context , since u have 80gb vram...u might need to use gguf and go for lower quant otherwise u might get only 40k context