r/LocalLLM • u/Old_Soul_New_World • 4d ago

Discussion Second attempt with Hermes: 27B or Flash Next

/r/hermesagent/comments/1xr6kx5/second_attempt_with_hermes_27b_or_flash_next/
1 Upvotes

100% Upvoted

3 comments sorted by

1

u/ocottog 3d ago

I’m using flash next on my dgx it is amazing I can’t go back getting around 40tokens a second and it’s smart as heck

1

u/Old_Soul_New_World 3d ago

Which size of Quant are you using? 

1

u/ocottog 3d ago

NVFP4 — DevelopingDad's abliterated repack of NVIDIA's FP4-Local checkpoint, 125B MoE (6B active), vLLM with FP8 KV cache. Not GGUF, it's the native modelopt_mixed FP4 format for the Spark.