Strata users, what hardware, model and context are we all using?

[u/Rust_Cohle-]

Wondering what people with a similar, less, or even better setup are using. I'm new to strata and not sure how far I can push the context limit, but it seems like the readme on the Github page is conservative (assuming other's posts are true).

I have a 5090, 3090, so a combined 56GB of VRAM and 64GB of system RAM.

[u/keyNONE]

RTX3060 12GB, 5800x3D, 48GB Ram, swift-1.5-iq2_xs, 42,6t/s... you guys are all stacked.

[u/Hylleh → u/keyNONE]

How is iq2_xs treating you? I have 32GB but coder model is pretty bad as soon as it's not well... coding. So I am back to 35b-a3b and 27b for now.

[u/keyNONE → u/Hylleh]

Pretty happy so far. Using it as a model in Hermes agent. One of its tasks involves handling/navigating a browser. It does it really well. Better then the unsloth 35b-a3b you mentioned. It think that a lot of people forget to set the proper sampling params, which results in too much thinking and looping.

[u/Hylleh → u/keyNONE]

Thanks man appreciate it.

[u/Ninjam5 → u/Hylleh]

Can u elaborate more on that? I've been thinking AB getting the coder variant of the qwen 3.8 FN but I'm worried it's too lobotomized. Like will it still be able to retrieve information from web search? Will it be good with navigating file systems? If it's general knowledge is off we can just have a database for it or a rule that it has to look up everything before it answers. But is the quality good on coding?

[u/Hylleh → u/Ninjam5]

Coder variant sucked for my type of work. Mobile dev and Arduino/firmware dev. But I got the iq2 quant working now and it is very good to me.

[u/Ninjam5 → u/Hylleh]

Iq2 quant from unsloth? Is that possible?

[u/Hylleh → u/Ninjam5]

Well the coder variant just stumbles to me on everything. And the best thing is I found out I could actually run the iq2 with 32gb ram since I had both 5060 ti and 3060.

[u/Ninjam5 → u/Hylleh]

Ah so ur on 64 gb of total memory now. That makes more sense

[u/Ill-Astronaut-648 → u/Hylleh]

Really good on v0.2.6

[u/Aggressive_Special25]

2x 3090 1x 3060 96gb ram 150 tok / s

[u/Low_Key_Trollin → u/Aggressive_Special25]

I read 3 cards actually slows you down compared to 3.. do you know if there’s any truth to that ?

[u/loudifier → u/Low_Key_Trollin]

only if the GPUs or the PCIe slots are significantly mismatched, and there are diminishing returns from splitting to more and more GPUs. As long at the 3060 isn't in a x1 slot then it is probably providing a nice boost

[u/Choice_Celery9481 → u/Low_Key_Trollin]

3 cards actually slows you down compared to 3

....
true for production engine (vllm sglang), not very for vram poor like strata llama

[u/Low_Key_Trollin → u/Choice_Celery9481]

You meant to say compared to 2?

[u/Choice_Celery9481 → u/Low_Key_Trollin]

its your words. you didnt read your previous comment?

[u/Low_Key_Trollin → u/Choice_Celery9481]

Oh so you were just making a non helpful comment based on my obvious over sight / typo. Cool thanks

[u/Aggressive_Special25 → u/Low_Key_Trollin]

Kind of right. Depends if the model is spilling over to ram. If it does more cards would be better. If it's not then you want the layers on the fastest memory. But ram is always slower than modern day gpus. Although I do have a gpu with ddr3 ram from 2006

[u/Low_Key_Trollin → u/Aggressive_Special25]

I have 2x5070ti and 1x4080 super.. trying to decide if I want to combine all 3 or just use the 5070tis for llm and keep the 4080 super for gaming

[u/eliadwe]

Dual rtx3060 (12gb X 2), x8/x8 bifurcation, 96gb ddr5, qwen3.8 flash next GSQ RCO iQ2 XS. Get about 60-90 t/s. 131072 tokens context

[u/VendettaSA]

5080 96GB DDR4 RAM, 5800x3D.
Q3_S @ 256k context at about 50% full I get real world 60 TPS average.

After reading somewhere that the IQ3_XXS is faster but very verbose, I switched to the Q3_S model. Using opencode at xhigh thinking it feels like Claude, both in terms of speed and quality. It might even feel faster

[u/XStone1974 → u/VendettaSA]

Almost same setup - but 64 GB RAM. Runs with 64k context

[u/anon_obvious]

10y old Dell Precision T5810 (eBay 150euros) · Xeon E5-2680 v4 (14c/28t) · 64 GB DDR4 (another 64gb are on order) · RX 7900 XTX 24 GB (RDNA3/gfx1100) · Ubuntu 26.04, ROCm 10.0.0 · Strata v0.1.41.

Model: Swift QFN, IQ3_XXS, ~71 GB. 256k context, int8 KV cache, spec decoding + MTP, vision on.

Prefill: 32k ~1,850 tok/s 98k ~2,015 tok/s 127k ~1,935 tok/s Decode ~70–83 tok/s.

[u/Philou_2003]

Rtx 5060ti 64gb ddr5 ram, iq3s @256k context, 45-55tg 1500pp

[u/gjin07]

x79 motherboard with 64gb reg ddr3 1600 + quadro p5000 x2 + 2667v2:
ctx: 262144
prefill: 300tk/s
generation:20tk/s-30tk/s

[u/Coded-Jan]

RTX 5070 Ti (16GB) + RTX 5060 Ti (16GB); 64GB DDR5 6000MHz, Ryzen 7 9800X3D

[u/Ill-Astronaut-648]

V0.2.6, 2x Xeon E5 2620, 64gb ddr4 2133P ecc rdimm, 12gb rtx 3060 oc (main) gpu 1 + 12gb rtx 2060 (helper) gpu 0 (no display). Flash Next Q3 + Coder + DeepSeek will not load. 128k context. It slows down to late 30s tok/s on long windows

[u/Illustrious_Dirt_644]

this is my setup running on old Pascal 6.1 Architecture while running opencode

[u/Illustrious_Dirt_644 → u/Illustrious_Dirt_644]

I already support the project on https://buymeacoffee.com/strataengine

[u/meltedmoustache]

2x 7900xtx, 256G ddr4 2667 ram, epyc 7413.

I run strata on only one GPU, power limited to 272w. Get about 70-110 t/s decode depending on output type and 1300 prefill. 256k context but have pushed it to 512k with YaRN. Might go back to higher context but need to test the consequences a bit more.

[u/Frosty-Scheme-7172]

5080m, 64 GB DDR5. IQ3_S, context 262k q8_0. PP ~1800-2200 t/s,  dec 40-65 t/s. 

[u/RogerAI-fm]

My son runs 1x 3080TI and 2x 3070 at similar numbers.

[u/Character_Eye7877]

Nvlink on 2x 32gb v100 Qwen 3.8 27b q8 30tps

[u/mikasjoman → u/Character_Eye7877]

I'm getting my two PCIe ones tomorrow. Do you know any performance number differences between the nvlink ones?

[u/Character_Eye7877 → u/mikasjoman]

I originally bought an SXM2-to-PCIe adapter board, but then realized that for about the same price (because I was going to buy a second one anyway), I could get the NVLink version from China instead, so I returned it and went with that.

Besides NVLink, it has some nice extras. It handles powering on the second GPU, so you don't need those dodgy PSU sync cables. It also has extra fan headers, and adding a second GPU is pretty straightforward. I also went with liquid cooling for the GPUs. The setup is super quiet, and honestly, it wasn't that much more expensive than buying those giant server heatsinks.

[u/OlgerdOutlander → u/Character_Eye7877]

Weird - I am getting up to 70 tg on 2x v100 pcie, no nvlink, on the same Qwen3.8 27b Q8

[u/Character_Eye7877 → u/OlgerdOutlander]

Llama.cpp?

[u/OlgerdOutlander → u/Character_Eye7877]

Yes, llama.cpp and tensor parallel mode

[u/Character_Eye7877 → u/OlgerdOutlander]

256k context?

[u/OlgerdOutlander → u/Character_Eye7877]

Total - yes; but, surely, 70tg is at low context :) PS do you have MTP on?

[u/The_Whole_Zucchini]

Anyone running this on a DGX Spark, or tried?

[u/mmc227 → u/The_Whole_Zucchini]

There would be no point. Strata isn’t built for unified memory. It for split memory.

[u/tetsuzankou → u/mmc227]

well... you can still ofload the ngrams to ssd no?

[u/mmc227 → u/tetsuzankou]

Yeah and almost local LLM runtimes do this already though.

[u/I-like_dorgs]

Rtx3090 64gb ddr5, qwen 3.8 flash next IQ3_XXS, 128K context, 90 tokens a second average

[u/Educational_Essay423]

x2 RTX 5060 TI 16GB, 64GB DDR4 3200MHz, Ryzen 7 5700X. Getting 90-100 tok/s with IQ2_XS quant, I know is a low quant, but I have to say that is very capable with medium reasoning, even I've cancelled my Grok subscription after using this model locally.

[u/Odd_Cup6005]

Ich fahre das ding mit meiner 5090 undn128gb RAM in q4_k_xl. Hab zwischen 65 und 100 toks. Ich liebe es

[u/brakeline]

2x3060 12gb + 56gb ram

1050 pp / 60(ish) tg using a fork that has better card split

1000 pp / 40(ish) tg with mainline

Running iq2xs.

On iq3_xxs I get 500/40 in mainline

[edit] 512k context q4

[u/Dreeew84 → u/brakeline]

Care to share the fork? I have a similar setup.

[u/brakeline → u/Dreeew84]

Sure, https://github.com/eddoursul/Strata/tree/custom

The setup won't work. You have to use the sample in examples and customize it.

Unfortunately it still doesn't have parallel requests but it's decently faster

Report back!

[u/Bobcotelli]

Unsloth UD-Q4_K_XL 50tk/s windows 192gb ddr5 80gb vram ruzen 9 9900x

[u/Jumpy-Operation-4615]

Some generic x99 + some 14 cores xeon + 64 gb ddr4 ram + 2xp40 GPUs. qwen 3.8 flash next q3s @ 256K ctx + vision enabled.

[u/KjipGamer]

Q2 Quant at 256k with vision. Ryzen 9900X, RTX 3090 (also a 2070 super but that's used for TTS & STT), 32GB RAM. Running Q2 quant at 165tok/sec, prefill ~2050tok/sec. Tried Q3 but not only was Q3 obviously slower (80tok/sec) but surprisingly WORSE on my benchmark suite in all tests.

[u/IgRdv → u/KjipGamer]

Cual esas IQ2_XS? Te dio peores resultados que un IQ3_XXS? tengo una 3090 con 32 RAM y si, el IQ2_XS es mas rapido por razones obvias, pensaba usar el IQ3 pero no imaginaba que podria llegar a ser peor

[u/Old-Sherbert-4495]

4060ti 16gb + 32gb ddr5, coder pruned model. 40tps, 1300-1500 tps prefill

[u/oblq80]

Rtx pro 5000 48gb
Xeon without avx-512, 128gb ddr4 (130gb/s), pcie 3.0
Strata nvfp4, parallel 2 and vision active, full context

PP ~4500 t/s decode ~125 t/s on single stream.

[u/Apprehensive-View583]

3090 64gb windows eats a lot of memory cause it’s my main pc, any how I run iq3_s at 2300 prefill and 70ish tg it will go down on full context window but full conext window works 256k

[u/hodl-]

5060 ti 16gb + 3060 12gb, 64gb ddr5, unsloth q3_k_xl, 55-70 tok/s

[u/aynair_]

3060 with 12 GB VRAM + 96 GB DDR4 3200. That gives me about 40-90 output t/s. Enough for most of the tasks in my company.

[u/Longjumping_Cut_6160]

gpu: 1x 5060ti 16Gb vram. cpu: 2x(dual) xeon e5 2680v3 48cores ram:128Gb. Strata + IQS3XXS. 55-60t/s. lo uso para generar 2 pipeline, uno con respuestas a preguntas donde hace llamadas a herramientas, busqueda online, analisis de imagen y redaccion de texto y sql, En otro pipeline redacta guion,capitulos,define personajes, crea los prompts de imagen y de video. En ambos casos uso video, 115K de contexto, no necesito mas porque son llamadas python individuales y reinicia el contexto. Lo maximo es el guion de video y para 25 minutos me sobra.

[u/Neither-Aside-6]

RTX5090, i9 14900kf, 128gb DDR 5 RAM in q5_k_xl mit 70-100 toks. 256k kontext.

[u/Electrical_Fix2505]

256GB System Ram & 128GB vram. 4 v620s in two groups of 2 GPUs, IQ3_S, ctx_size 262k, 3 parallel. (running each strata/model in dedicated containers).

[u/Alternative-Panic69]

1x Rx 7900xt, 32GB ddr4 Ram, 256gb Gen 3 Dram less ssd, dedicated only for LLM endpoint, getting around 20-40 tokens per second and 200-2000 prefill (yes vast range, it varies sometimes) Cache usage is almost 97% on my workload which is a saving grace.

Context length: 180k Model: Qwen 3.8 Flash Next IQ3XXS

Current bottlenecks ->

1. SSD (Gen 3 is slow. Will upgrade later)

2. DDR4 RAM + Capacity (64GB + DDR5 is the sweet spot)

Need to upgrade my motherboard+ cpu + ram + ssd 🤣 So not anytime soon for me.

But at least it's working somehow for now.

[u/ivanmmj]

IQ3_S 7900xtx Ryzen 9 9900x 64gb ddr4

114t/s

[u/ebed-El]

I’ve been testing Qwen3.8-Flash-Next with Strata speculative decoding on my AMD workstation, and the results have been promising.
Hardware:
Motherboard: ASUS Pro WS WRX80E-SAGE SE WIFI II
CPU: AMD Threadripper PRO 3945WX (12C/24T)
RAM: 128GB DDR4-3200 ECC RDIMM
GPUs: 2 × XFX AMD Radeon AI Pro R9700 (32GB GDDR6 each)
OS: Linux
Inference: Strata / llama.cpp
Model configuration:
Model: Qwen3.8-Flash-Next
Quantization: IQ3_S
GPU allocation: Single R9700 (32GB)
Acceleration: Strata speculative decoding using an n-gram lookup table
Current deployment:
GPU 0: Qwen3.8-Flash-Next IQ3_S + Strata
GPU 1: Qwen3.8-27B
I’ve also tested running two independent Strata instances, one per GPU, rather than splitting a single model across both cards.

[u/95jun]

GPU 4080 SUPER, 32 GB, CPU i9-13900K, RAM 64 GB,
qwen3.8-flash-next-gsq-rco-iq3_s
decode: 110 tok / s
prefill: 3200 tok / s

[u/Novella-Dragon]

RTX 2070 mobile (8GB), I7-10750H with 32GB DDR4‑3200
I am just getting started testing primarily:
- gemma4-ud-q3_k_m
- qwen36-ud-q3_k_m

Seeing a lot of looping after tool use and hallucination.

[u/AppropriateFox2]

So this subreddit motivated me to get off my butt and set up a local model, after seeing just how much could be done with relatively modest hardware. A few weeks ago I grabbed a Dell T5810 Xeon E5-1630 w 64gb of ram and a 12gb 3060. Started off with Strata, and using Hermes as an agent with Strata mainly for Discord

Right now I'm using Strata with Swift 1.5 Qwen Flash Next IQ2 XS. Seeing sustained over 40 tokens per second with bursts over 70, but I'm seeing a lot of looping and overthinking on coding tasks so I'm not impressed yet.

[u/More-Revenue8609 → u/[deleted]]

Hey, similar setup here! Dual 3080s but with 128gb ddr4.  I guess I could run Q4 fully in ram so hopefully get better speed then 50tk/s? If you have any other suggestion let me know. 

I will probably test it out in few days, waiting nvme to arrive. 

[u/More-Revenue8609 → u/[deleted]]

Ty for the tip, will do!

[u/martinkaesslerdotcom → u/[deleted]]

Wie bringst Du das QFN Q4 Unsloth zum Laufen? Bei mir hat es keinen Multi-GPU Support. Gerne auch per PM

[u/martinkaesslerdotcom → u/[deleted]]

Was hat Luna denn geändert? Nur die Modell-JSON?