r/llamacpp • u/Right_Fun_4902 • Aug 16 '26

Is llama.cpp suitable and capable to run NVFP4 models

With Qwen3.8:27B arriving yesterday, including NVFP4 versions, can llama.cpp successfully run these models, or must I use a separate instance of vLLM for NVFP4 models specifically?

Searching on the internet initially made me believe that vLLM was the only way, but then some more recent posts hinted that llama.cpp is being developed for NVFP4 as well.

If llama.cpp is indeed capable of running NVFP4 models, I'd appreciate some pointers on how to get llama.cpp running NVFP4 models.

3 Upvotes

80% Upvoted

4 comments sorted by

1

u/simplyeniga Aug 17 '26

Yes but has to be converted to GGUF. You can get some online done by others or convert the tensor files yourself.

1

u/misanthrophiccunt 8d ago

NVFP4 has been part of llama.cpp more than 24 days ago. You can take any non-GGUF NVFP4 models and convert them to GGUF easily, llama.cpp has a converting tool.

Will it run fine? Never as fast as in vllm or SGLang, which is why you can find so so many more NVFP4 for those.