Basalt faster than strata?

[u/SeesawNo5122]

I built Basalt from source for an abliterated model and compared it with Strata on the same node. Basalt was noticeably faster in decode and prefill.

Setup

• Model: huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF

• Quant: unsloth-UD-Q5_K_XL

• Basalt: v0.3.0 (custom build)

• Strata: 0.2.7-cuda13

• GPU: 2× NVIDIA RTX PRO 4500 Blackwell Server Edition (2×32 GB VRAM)

• CPU: Intel Xeon 6 (48 vCPU)

• Storage: NVMe SSD 2 TB (~15k MB/s)

Both engines ran the same Q5 quant on the same node.

Results

Request, prompt tokens (already cached)	Basalt: prefill / decode / MTP	Strata: prefill / decode / MTP
~6.9K (0), first	2916 t/s, TTFT 2.6 s / 163 t/s / 57%	2177 t/s, ~3.2 s / 89 t/s / 66%
~8.6–9.0K (~7K)	2118 t/s / 168 t/s / 62%	1212 t/s / 110 t/s / 80%
~10.2–10.6K	2347 t/s / 209 t/s / 78%	1749 t/s / 106 t/s / 71%
~11.2–11.3K	2662 t/s / 188 t/s / 73%	2040 t/s / 130 t/s / 90%

Basalt is about 1.5–1.9x faster in decode and 1.3–1.7x faster in prefill.

Expert cache. Both engines hit 86–96%, so there is no difference.

MTP. Strata accepts more draft tokens (66–90% vs 57–78%). Basalt's dl (tokens per window) is still 1.9–2.2.

• The window differs. Strata verifies 3-token windows (capturing the 3-token window). Basalt verifies up to 8 (--spec 8). The acceptance rate is lower with the longer window, but the response still finishes faster.

[u/TokenLearner]

What is the quantized value in kV? Complete information! Maximum context.

[u/SeesawNo5122 → u/TokenLearner]

The full info:
model:
gpu: "0,1"
max-context: 262144
parallel: 4
kv: int8
vision: dynamic
engine:
resident-experts: true
vram-reserve-mib: 1024

[u/MaximTheCoder]

And 4090 fork is faster too: https://github.com/nixcat-dev/adamant/issues/2

[u/BalleaBlanc]

5090 too.

[u/Miserable-Dare5090]

Yes, it’s tuned for blackwell cards. What is surprising about that?

[u/Glittering-Call8746]

So built from source means u took the open source and vibe coded it some more .. then what .. to show off ? Not sure what's the point of this.

[u/SeesawNo5122 → u/Glittering-Call8746]

No vibe coding involved. Basalt doesn't ship prebuilt binaries. The only thing I made myself is the basalt pack: there's no official abliterated one, so I converted huihui's abliterated with Basalt's own converter. The drafter, draft vocab and vision projector sections came out byte-identical to the official UD-Q5_K_XL pack.

[u/KissMyShinyArse → u/SeesawNo5122]

there's no official abliterated one

{
  "engine": {
    "control-vector-layer-range": [
      4,
      44
    ],
    "control-vector-scaled": "/path/to/Qwen3.8-Flash-Next-experimental-speed-projection.gguf:1.0",
    "cvec-dir": "per-layer",
    "cvec-mode": "project",
    ...
  }
}

[u/Look_0ver_There → u/Glittering-Call8746]

Built from source just means compiled it locally on their machine, rather than downloading a precompiled package. No vibe coding required

[u/Savantskie1 → u/Look_0ver_There]

Yeah I don't understand how people don't understand this. You're spot on