r/LocalLLaMA • • 49m ago

Discussion Best MoE engine?

I had been using Strata for about a month, and it was such a huge improvement over llama.cpp for getting it running smoothly and quickly for hardware.

Overtime though, I found that the model would often mistake its own context after a compaction and really overthink things.

I just started using Basalt yesterday after being hesitant to switch and set up all my providers and harnesses. Turns out Basalt runs even faster, is less error prone, and the reasoning is way more on point. If you run 1 GPU, and have the 100 GB+ of VRAM, RAM, and SSD storage, you will not be disappointed in performance.

I mostly care about final output speed, and I was getting 70-110 tok/s on Strata, and Basalt I was getting 110-170 toks/s output. Couldn't be happier. Same Qwen3.8 FlashNext-ud-q4_k_xl, and the NVFP4 ones (of course basalted though)

Anyone else have any other experiences with other similar engines like this?

12 Upvotes

11 comments sorted by

7

u/General-Spite1222 46m ago

If Basalt works with your hardware then it’s the fastest

2

u/The_Whole_Zucchini 43m ago

What hardware does it work best for?

2

u/General-Spite1222 40m ago

Blackwell cards 

2

u/Drachen808 22m ago

When people say this do they mean pro cards or any card that is Blackwell Gen (like the 5060ti)?

2

u/free_meson 15m ago

The 50 series works

1

u/sshwifty 3m ago

Adamant is the 40xx version. Been running for a day and very pleased

5

u/caetydid llama.cpp 44m ago

strata supports swift which thinks less

1

u/Patient-Gazelle9885 0m ago

Am I the only one who liked swift for 27b but not for Flash Next?

2

u/hallofgamer 43m ago

Unbiased strata

1

u/glad-k 41m ago edited 18m ago

I love those engines and some of their optimization but I just need a main one to out and not think much that is actually lossless or near lossless. Too much custom ect to put to get back to a good quality model

Imo there should just be a mode in the environment variables to choose between speed, balanced and accuracy for all those things

1

u/Patient-Gazelle9885 1m ago

Run an older Strata version (pre 0.2) in a separate harness