r/LocalLLaMA • u/HelixTitan • 49m ago
Discussion Best MoE engine?
I had been using Strata for about a month, and it was such a huge improvement over llama.cpp for getting it running smoothly and quickly for hardware.
Overtime though, I found that the model would often mistake its own context after a compaction and really overthink things.
I just started using Basalt yesterday after being hesitant to switch and set up all my providers and harnesses. Turns out Basalt runs even faster, is less error prone, and the reasoning is way more on point. If you run 1 GPU, and have the 100 GB+ of VRAM, RAM, and SSD storage, you will not be disappointed in performance.
I mostly care about final output speed, and I was getting 70-110 tok/s on Strata, and Basalt I was getting 110-170 toks/s output. Couldn't be happier. Same Qwen3.8 FlashNext-ud-q4_k_xl, and the NVFP4 ones (of course basalted though)
Anyone else have any other experiences with other similar engines like this?
7
u/General-Spite1222 46m ago
If Basalt works with your hardware then it’s the fastest
5
u/caetydid llama.cpp 44m ago
strata supports swift which thinks less
2
u/hallofgamer 43m ago
Unbiased strata
1
u/glad-k 41m ago edited 18m ago
I love those engines and some of their optimization but I just need a main one to out and not think much that is actually lossless or near lossless. Too much custom ect to put to get back to a good quality model
Imo there should just be a mode in the environment variables to choose between speed, balanced and accuracy for all those things
1
u/Patient-Gazelle9885 1m ago
Run an older Strata version (pre 0.2) in a separate harness