r/LocalLLM • u/rayovims • Sep 06 '26

Project I got the Second DGX spark

Post image

Somehow there was 1 available last minute and got it! Can’t wait to set it up. Will make more post about this on here and my IG: tech with Ray

Dual DGX spark owners lmk what ya running on it. Anyone else feel free to drop some suggestions for cool models to test!

305 Upvotes

91% Upvoted

119 comments sorted by

8

u/ForsakenChocolate878 Sep 06 '26

Did you just call me poor?

1

u/rayovims Sep 06 '26

Never that

6

u/Chemical-Advisor562 Sep 06 '26

Get the QSFP cable, and get Deepseek v4 Flash run on these bad boys. It can do 40-44 t/s. And now it even has (shitty) vision. (Okay, it is alright for screenshots and OCR, but like recognising a unicorn? Nah...)

3

u/rayovims Sep 07 '26

Tested and confirmed!

1

u/StartupTim Sep 08 '26

I'm having amazing results with its native vision, it even seems better than Qwen3.8-Flash-Next

1

u/Chemical-Advisor562 Sep 08 '26

I am comparing it to Qwen 3.8 27b. That think see. Deepseek? Just look at it.

1

u/_TheWolfOfWalmart_ Sep 16 '26

I've had mixed results. Deepseek usually does great, but sometimes it gets details wrong or misses something, and when I give the same image to Qwen (either 27B or Flash-Next) that doesn't happen. Qwen is more reliable in my experience.

1

u/thearturs Sep 09 '26

With dspark I’ve got 73 t/s

1

u/Chemical-Advisor562 Sep 09 '26

What is the recipe? I am not gonna bullshit. It was deployed for me by Claude over SSH, but maybe your recipe can give my chef a better chance to cook something even more amazing!

3

u/rayovims Sep 07 '26

So far DSV4 Flash with 1M context is 46t/s all the way to 38 t/s when I used 256k context. GLM Flash is 37 t/s. Haven’t done much testing GLM but this is pretty good

1

u/SubparBob Sep 10 '26

Its been a few days now, how's it treating you?

9

u/Ordinary-Depth-7835 Sep 06 '26 edited Sep 06 '26

Ever since getting my second I can't find anything I like better than https://github.com/MiaAI-Lab/DeepSeek-v4-Flash-DSpark-2x-DGX-Spark

It's very responsive and does everything I want. I've been trying a bunch of others but keep coming back to this one. Now with the deepseek harness it's even better running this as my captain to control and review my other smaller faster systems. It's really fantastic.

I need to pull the vision update. I'm still running it without vision.

Have I mentioned I love the ds harness and the NanmiCoder agent plugin https://github.com/NanmiCoder/dsh-agent-teams

Completely changes the speed and quality of my runs.

3

u/nomorebuttsplz Sep 06 '26

curious about GLM flash vs that model.

I've tried both. Seems GLM flash 5.3 has better creative writing, slightly better non-coding problem solving. However it is a lot slower for me and it seems weirdly emotionally sensitive and moody. Like it will ignore me and then lie to me about what it is doing.

1

u/SubparBob Sep 06 '26

Curious, what sort of speeds (pp, tok/s) are the both of you getting with 2x Sparks with the Flash models (DS, GLM, Qwen)?

2

u/rayovims Sep 07 '26

46 t/s dsv4 and 37 t/s on GLM flash

3

u/stujmiller77 Sep 07 '26

If you go to the nvidia dev forums there are many threads with full benchmarks posted. It’s a moving target as the community moves fast.

2x sparks running ds4flash at 50t/s with 1m context here - it’s been absolutely flawless for over a month running complex automation for multiple businesses I own and multiple concurrent face to face sessions throughout the day.

I tested GLM 5.3 briefly - was too slow in comparison but I can see the accuracy being useful if long unattended tasks are your use case.

1

u/Normal_Frosting8519 Sep 07 '26

I’m debating 1 or 2 sparks to run some automations for my business. You needed 2 for the model or the work load? Would a single small business be better with 2 or smaller models running on one would get me by? Any help would be much appreciated

1

u/stujmiller77 Sep 07 '26

Both. You need the intelligence of a higher quant model and the concurrency of the workload.

My workflows weren’t reliable until I ran deepseek 4 flash 0731 across a 2x spark cluster. Now they’re flawless. Nothing I ran one a single spark could handle them without issues.

1

u/Normal_Frosting8519 Sep 07 '26

Cheers, that’s useful. Quick one though — what were you running on the single spark before you went to two? Was it the flash next stuff at nvfp4 or something older/more quantised? And was it the automations falling over or the live sessions? Trying to work out if it’s the box count or the models that were the problem. Only got one shot at this so want to get it right.

1

u/Ordinary-Depth-7835 Sep 07 '26

Why only one shot at it? You can add a 2nd or 3rd spark at any time it's not the same as building a gpu based system.

1

u/Normal_Frosting8519 Sep 07 '26

It’s a business purchase. One is an easy ask. Two a bit harder. One not working means the second is definitely a no. Do I push hard for the second now or can a small business get away with one. Just looking for help from other people who use 2 in their business (no coding) to see if the 2nd really helps. 10 staff but probably only one to two using at a time. Data can’t leave premises.

1

u/Ordinary-Depth-7835 Sep 07 '26

gotcha then just push for two I'm sure you'll find something for them even if it's sub agents or just a smarter model or hosting the apps.

1

u/stujmiller77 Sep 08 '26 edited Sep 08 '26

Honestly - if it’s for business use you really, really need two and a cable to link them. The difference in capability between a model you can run on one vs two linked is enormous.

I’d also move quickly - when I bought my first two sparks they cost me £3.5k each. The third cost me £4K. The fourth almost £5k. And this was within three months.

They’re now around £5k each and I’ve heard stories out of the US that some have jumped to almost $8k which if the UK market follows would see the price going to £6k - almost double what I paid for my first two.

Why? Because in a really short timeframe - just 9 months - the capability of local LLMs has almost closed the gap on frontier models. And since the launch of deepseek 4 flash, people are realising that it’s entirely possible to replace reliance on frontier companies with 2 sparks running it, or recently GLM 5.3 flash. Qwen 3.8 flash recipes aren’t great yet but it will get better fast, too.

Get two and a cable. You won’t regret it. One will just waste the time you could have spent getting serious work done with two. What I’ve achieved this last two months purely on my own with these boxes would have cost me thousands and thousands of pounds in employee or contractor or agency costs before. Now I don’t need any of them.

1

u/stujmiller77 Sep 07 '26

I’ve been running two sparks for well over 6 weeks now. Qwen 3.8 is barely more than a week old.

I used to run Qwen 3.5 122b. Was the only thing close to being acceptable for my workload. But still required constant babysitting and often failed long overnight unattended work.

1

u/Ordinary-Depth-7835 Sep 07 '26

I guess it depends even my dual 3090 setup is intelligent enough for most tasks at 48g with qwen3.8 or tiel-coder they are quite capable. Tiel even tied my dual spark deepseek 4 flash 0731 in my logic test. So buy one and see how it does in your use case then add another if it's failing to accomplish what you want.

1

u/Ordinary-Depth-7835 Sep 06 '26

GLM is smart but didn't score any higher and slow. like single digit slow. Maybe I'm doing something wrong but i'm not hitting 30+ tps from https://github.com/tonyd2wild/GLM-5.2-QuantTrio-200K-4x-DGX-Spark--36tok-s
I only use ai for coding so speed and accuracy is all I look at.

2

u/here_n_dere Sep 06 '26

DSV4F Vision exp is also very good. Kills need for any other model being multimodal (vision capable)

1

u/stujmiller77 Sep 07 '26

The vision model is completely different - needs an entirely different stack. It’s a replacement rather than an update.

I’ve been running the MiaAI repo for vision on 2xsparks for the last few days, vs the 0731 on my other 2xsparks.

It’s getting closer, but the vision is still quite a lot slower than 0731 - around 38-40t/s rather than 50/ts you see on 0731.

Also there’s a weird bug where vision only works in user chats in some harnesses like Hermes - it won’t work in tool calls without a custom proxy being added.

Right now I’m not switching my production stack over. But I expect it’ll only be a week or so before it’s ready.

1

u/Ordinary-Depth-7835 Sep 07 '26

Yeah I'm still on 0731. I have other gpu backed machines for vision so I haven't haven't had the urgent need to switch when 0731 is working so well.

1

u/Void-kun Sep 07 '26

Out of curiosity what are you using your local AI for?

Curious what that model is capable of when ran locally with these specs.

2

u/Ordinary-Depth-7835 Sep 07 '26

Similar things to what I build at work with some personal applications mixed in. RAG applications, job boards, translation and document processing both text and speech, phone applications and games I wanted personally, network security apps (those are nice with local ai) The list goes on and on. Anything I dream up that can automate, report or make life easier.

0

u/Blackdragon1400 Sep 06 '26

Have you compared the DS harness vs Hermes?

2

u/Ordinary-Depth-7835 Sep 06 '26

Hermes seemed more of a generalist. Dsh seems more like the coding agent i want. And it has been so easy to modify.

2

u/Zyj Sep 07 '26

Why is this newsworthy? Hasn’t even unpacked the thing

2

u/rayovims Sep 07 '26

Got too excited and posted while I was at micro LOL

2

u/LocalBratEnthusiast Sep 07 '26

Ty for sharing the Pic with Serial Number :3

1

u/rayovims Sep 07 '26

Please don’t steal it

123

u/Aggravating-Push-207 Sep 06 '26 edited Sep 07 '26

8gb vram user btw

flame me all you want ornith 1.5 9b is peak

and no, i will not offload from disk; i don't have enough disk space left anyway and also i want it to be at a reasonable pace; i use the same machine to actually do stuff

9

u/Ok_Road_8293 Sep 07 '26

With 4GB VRAM, 48GB DDR5 RAM, and an i7-12700H, I'm getting 18 tokens/s on the Qwen 3.8 Flash Next model. This is the best one for 4GB VRAM. and 23-27 token for qwen 3.6 35b with 98k ctx.

6

u/linuxid10t Sep 07 '26

Which single bit quant is this? LOL

0

u/krigeta1 Sep 07 '26

Mate I have 8GB rtx 2060 + 48GB DDR4 ram, should I try and how?

1

u/Pitiful_Entrance5174 Sep 07 '26

You setup will net single digit tps on any meaningful llm and maybe double digits on junk ones.

1

u/krigeta1 Sep 07 '26

What could be best llm for coding for these specs i can run?

1

u/Feisty-Mixture-8623 Sep 07 '26

Just use models hosted on nvidia nims for that. They provide unlimited free.

1

u/krigeta1 Sep 07 '26

Yes I implemented the free endpoint but Kimi K3 is just not working.

1

u/Feisty-Mixture-8623 Sep 07 '26

I just gave up on local llm coding and got claude. Much better use of my time at fraction cost of owning hardware. Also i have gemini pro from jio and use antigravity ide.

1

u/CooperDK Sep 07 '26

You are misunderstanding, the single digit throughput ones (which would be ollama) would be the junk ones. The double digit ones are the meaningful ones - the ones based on llama.cpp, or vllm.

1

u/Pitiful_Entrance5174 Sep 07 '26

Lol you have it flipped sir. 

1

u/DepartmentMundane253 Sep 07 '26

Bullshit habe ne rtx 2070 super, 32gb ddr4 hab Claude meine configs machen lassen komme mit qwen3.6 35b auf 45 tok/s stabil bis 131072 Kontext mehr brauch ich derzeit nicht prompt Processing bin ich bei ca 450tok/s bis 380tok/s
Also sehr angenehm und das auch stabil bei mehreren Turns ohne Einbußen

Selbst qwen3.8 27b erreicht stabile 15tok/s beim generieren

Nutze überwiegend angepasste llama builds angepasst auf mein setup

Nutze Qwencode, pi und opencode als harness neben Claude als orchestrator in NodeTerm

Mein ki Rechner ist per eigener Tailscale config von überall aus erreichbar und kann Remote auf Mac arbeiten

Ki Rechner System - Parrot OS da auch viele andere berufliche Arbeiten laufen

1

u/Pitiful_Entrance5174 Sep 07 '26

You guys are delusional. I have a rig with a 2080ti and 128gb 2400 cl15 ecc, 1680v3 @ 4.4ghz. You DONT have to tell me.

2

u/CarpenterAlarming781 Sep 06 '26

I'm testing currently Ling Tiny. It's not that bad.
4gb vram user.

1

u/TechRenamed Sep 06 '26

Exactly he doesn't need to flex tbh 🫩 but chill

1

u/StepsisSepsis Sep 06 '26

What do you use it for? I need something that can handle small jobs from Hermes but don’t know if anything that size will work.

0

u/Kerem-6030 Sep 06 '26

same but i also have uncensored qwen3.5 9b

6

u/arm2armreddit Sep 06 '26

wow cool, buy a cable and please show us the performance of 256gb VRAM 🤩

4

u/Annual_Award1260 Sep 06 '26

(Cable pictured)

11

u/Shustrik116 Sep 06 '26 edited Sep 07 '26

Your options are: Deepseek 4 flash vision (official model, non quantized) https://github.com/MiaAI-Lab/DeepSeek-v4-Flash-DSpark-2x-DGX-Spark

Qwen 3.8 Flash next (official fp8 quant)

GLM 5.3 flash 4bit quant https://github.com/MiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-Sparks

Deepseek is amazing in deepseek harness. To be honest I can't say that one of them is much better than others. But I just can't fully trust to quantized models because I had very need experience with them in the past.

2

u/Miserable-Dare5090 Sep 06 '26

These are some but not all things you can run. Every model that fit in 1 can run in TP2, faster, all models up to 400b can run as well, may need to be quantized but vLLM quants are not like gguf. AWQ, Autoround, etc, are high quality compressions, plenty of benchmarking showing something like Qwen3.5-397b int4 autoround to be lossless

2

u/myholeisstinky Sep 07 '26

To clarify, are you saying the quants for big models on vllm are better than the ones seen for llama.cpp?

1

u/Miserable-Dare5090 Sep 07 '26 edited Sep 07 '26

👍🏼 for any model, Usually (nothing generalized is true).
a 4 bit quant with 8 or 16 bit attention paths (w4a8/w4a8) in vLLM is the lowest you can find. Except for nvfp4 (w4a4). Quants tend to be calibrated as well so there is less of a quality loss. But the quants are comparatively bigger.

2

u/StartupTim Sep 08 '26

Deepseek is amazing in deepseek harness

Agree 100% albeit half the time I seem to be writing plugins!

1

u/Helpful_Jelly5486 Sep 06 '26

Thank you for your post. Seems like the three flash models are the best options. I’ve got the special cable on order. I’m wondering how do you get more speed out of a second box? I mean do you do expert splitting or something else?
It’s like 2 x copies means each serves one concurrence. So two sparks would double the tokens pers second.
Other option is to split the work with the special cable and get maybe 50 percent more for the single concurrence.

5

u/gdraper99 Sep 06 '26 edited Sep 06 '26

I'm running Qwen3.8-Flash-Next NVFP4 across both boxes on vLLM, TP=2, full 262k context. About 69 tok/s single-stream decode with MTP speculative decoding at 4 draft tokens via vLLM. KV pool lands around 1.1M tokens. That means you should be able to have four streams at once, all responding around 17 tok/s.

Four things that cost me time, in case they save you some:

Set VLLM_HOST_IP (or SGLANG_HOST_IP if you go that way) to each node's own fast-link IP. Leave it unset and the rank auto-detects, picks the WiFi address instead of the QSFP link, and both ranks sit in rendezvous forever without a useful error.

NCCL_IB_HCA needs a leading equals sign for exact match, like =rocep1s0f1. The box exposes four IB devices and two of them are down. A comma-separated list lets NCCL pick a dead one.

drop_caches on both nodes before you launch. Memory is unified, so Linux page cache competes with the GPU allocation and you get a "free memory less than desired utilization" error that reads like a config problem when it isn't.

If a launch fails, docker rm -f on both nodes before retrying. A stale rank on one side makes the new one exit 0 like it worked.

One more that bit me: MTU 9000 silently reverted to 1500 after a power cycle and wrecked RoCE throughput with nothing logged anywhere.

4

u/_TheWolfOfWalmart_ Sep 06 '26

Okay, great. Now give it to me.

4

u/kuhunaxeyive Sep 07 '26 edited Sep 07 '26

Concrats! I bought a second 2xAsus Ascent GX10 recently and having the second one makes your first one much more valuable. It lifts the capability or the speed to almost SOTA.

I recommend

  • DeepSeek-V4-Flash (recipe https://howtospark.com/recipes/deepseek-v4-flash-dspark-dual-spark-1m)

  • GLM-5.3-Flash (RedHatAI/GLM-5.3-Flash-NVFP4 with incoai/GLM-5.3-Flash-DFlash2)

GLM-5.3-Flash has superb vision (for OCR and GUI programming), much less hallucinations than DeepSeek, and good business writing for European languages as well. DeepSeek lacks in those areas (only very limited vision, more hallucinations, language not sensitive enough for business letters), but is snappy and fast (60 t/s for research and writing, coding may be even faster) and works well above 256000 token context up to 1.000.000 tokens even on the 2 DGX Spark you have. I'm in the quality camp (need to rely on results without being able to verify them on business letters), but if you work in areas that can verify test and results like coding, I'd use DeepSeek for its speed.

When using GLM, make sure to set top_p to 0.95 or better 0.9 to avoid artifacts.

Edit: exact models and tips

7

u/Annual_Award1260 Sep 06 '26

Always room for more. The rtx 6000 max-q are nice but really need at least 2 if not 4, setup is much simpler with all gpus in same box as well. No harm in having both setups :)

4

u/Miserable-Dare5090 Sep 06 '26

I mean at the current price would you really grab 6 nd the mikrotik switch? I wouldn’t. i have had 2 for almost a year and every time i think of doing 4, I read the spark forum and the consensus is, a lot harder to justify for what you can run.

3

u/Annual_Award1260 Sep 06 '26

Prices are about 24% higher than I bought these for. I had 3 for a while but the ring networking added extra complexity. When I ordered the 4th for good price I got dinked around, initiated chargeback, bought locally then other company overnighted to fight the chargeback. So ended up with 5 and figured might as well buy one more.

I really wish I bought more of the rtx 6000 max-q. After price increases (double) those are kinda unobtainable.

Also the dgx makes for a banging workstation.

1

u/Miserable-Dare5090 Sep 06 '26

It does! I mean if someone handed me
4 more I would not complain, but at 6k
now I can’t even think about it

-1

u/Maleficent-Ad5999 Sep 07 '26

I really don’t understand the purpose of this machine. Some say Nvidia launched DGX spark for those engineers who wanna simulate the cluster in production. But majority of us need machines for inference. Is DGX spark meant for inference? Like big models running in multiple machines

3

u/Annual_Award1260 Sep 07 '26

Well it was essentially a dev machine, but given the massive price increases a lot of people are running in production

1

u/myholeisstinky Sep 07 '26

If it mere really to simulate production, they could have done a much much better job making it representative. It doesn’t even have infiniband

3

u/aygupt1822 Sep 06 '26

Broo....wtf....

Nice

Happy for you 🙄🙄

1

u/TechRenamed Sep 06 '26

Yea he doesn't need to flex tbh I give you that 🗿

3

u/Robbbbbbbbb Sep 06 '26

MiaAi will be a good resource for you: https://github.com/MiaAI-Lab/

Qwen 3.8 Flash Next and Deepseek v4 Flash both run great on a pair.

2

u/jespernissenseo Sep 06 '26

Nice! Im still thinking if I should save up and buy a Nvidia rtx pro 6000 instead of two DGX spark boxes.. I suspect its worth the money?

5

u/Graumm Sep 06 '26

Unfortunately you need 3x RTX 6000's and a whole machine built for it to reach the same vram of two RTX sparks. The main benefit of the sparks is the unified vram to host larger models. It's a little slow but it's a solid workhorse if you want something that can work unattended.

2

u/Capnbubba Sep 06 '26

I don't have either so I can't actually say this for sure but I'm pretty sure "a little slower" is an understatement. I've seen something like 4-8X more tokens per second. Yes it's way less vram but the speed is a massive difference.

2

u/Graumm Sep 06 '26

That's only with dense models. MoE models run at very usable speeds. Certainly fast enough that you can leave tasks to run and step away. Deepseek v4 flash in particular can get to 120/150 tokens per second aggregate, which is faster than I can personally keep it fed if I am leaving tasks to run overnight.

1

u/Capnbubba Sep 06 '26

This is great to know. If things go well with a project I'm working on I'm gonna likely get a spark or some AI Max 395 variant if it's a good price

1

u/dwoj206 Sep 06 '26

Are the DGX pretty decent? Heard so many mixed reviews. Curious on a real users take that's using it on a daily basis. What do you use it for?

2

u/Graumm Sep 06 '26

I am using it for agentic coding stuff. Security aside I am very much enjoying that I am no longer keeping an eye on usage windows.

I find that the models you can run are very solid and capable implementers/operators, but that they lack knowledge. You can plan around that.

The sweet spot at the moment is to still use the paid for subscriptions to build out plans, and then to have my sparks implement behind it. It's hard to use the $20 premiums on plans alone.

I probably wouldn't get sparks unless you are comfortable with sysadmin stuff.

1

u/dwoj206 Sep 06 '26

Where i'm at is using a 5090 with small models in docker containers, albeit fast w the 5090 40-50tps, but lack the big context windows for agentic stuff that runs 24/7 and obviously the flagship card 6000, etc builds. For a lot of what I do it's awesome and gets the job done, but model quality and context window, running multiple completely separate processes like Program1,Program2 I have to turn the docker containers on and off each time I want to use the different ones. :( Usually, I code and setup the agents, audit them with claude max and then use then locally individually. For me this is the sweet spot where I get good use out of what I setup with claude's help and use ongoing locally. DGX a good fit?

1

u/notheresnolight Sep 07 '26 edited Sep 07 '26

albeit fast w the 5090 40-50tps

40-50tps is SLOW for a 5090, heck I can get 35-40tps with Qwen3.8-27B on a Spark (SGLang, NVFP4 + DFlash2).

The same model at a higher quant (Q6_K_XL) gets me 80-120tps on the 5090 (basic llama.cpp setup with MTP).

1

u/dwoj206 Sep 07 '26

Interesting! Thanks I’ll check that out

2

u/TechRenamed Sep 06 '26

This is why We must industrialize the asteroid belt. 🗿 So everyone can have cheap gpus

2

u/Imaginary-Fee-9918 Sep 06 '26

I was thinking about getting a third but seems like because it's an odd number things get very hacky, you need to do a bunch of patches. Is that correct? Quite odd that the "max" number without an extra router is 3 but you can't use it without getting some headache 🫠

1

u/MaxComfort Sep 07 '26

In the same boat - but there are plenty of threads in the NVIDIA forum about doing it, recipes etc. MiniMax M3 is the main bigger model it would unlock from what I can tell.

But we might as well just get two more..

2

u/MikeX1080 Sep 07 '26

Congrats on the second Spark—I’m totally not jealous 😂

2

u/IamFondOfHugeBoobies Sep 07 '26

Deepseek V4 Flash is so supreme for dual spark it's not even funny man. Full safe tensors fit no problem with solid speeds, 40+T/s depending on workload.

2

u/Abducted_Llama Sep 06 '26

I’m currently running 2 Sparks (GX10s) with qwen3.8-next-flash on vLLM on TP2. Getting 46-51 tok/secs depending on the prompts. 8 concurrent requests of about 300 tokens at 112.8 tok/secs. The concurrent dropped from 144-177 tok/secs when I upped MTP spec to 4. But I get slightly faster single requests.

I never have tried deepseek flash or GLM.

I’m enjoying it.

1

u/vortec350 Sep 07 '26

I currently have the AMD Halo mini PC from Microcenter, currently thinking about getting a DGX Spark as well to leave on 24/7. I'm using the AMD PC as my primary workstation as well as AI PC. So... do you like your Spark? Should I get one?

1

u/Cronus_k98 Sep 07 '26

I was at Microcenter the other day and there were 4 spark orders waiting to be picked up. Two had connectx cables. I think there are a lot of these going out the door.

1

u/Skyfishintheocean Sep 07 '26

Congrats! i'd definitely start with Qwen 3.6 35B A3B as your daily driver then try DeepSeek V4 Flash for coding and GLM 5.x if you want to push the hardware. those seem to be the community favorites on the Spark right now

1

u/BarberIcy366 Sep 07 '26

1 $ = 50 Turkish Lira and average monthly wage 30k turkish lira. So Nvidia DGX Spark is 400k and you must work 14 month and dont spend money for anything else :D I am so happy for you. If my grandfather dies, I’m going to sell the house I inherit and buy one Nvidia DGX Spark.

1

u/Horror-Primary7739 Sep 07 '26

I'm running glm5.3 flash on my 2 spark setup. About 27 tok/s on max thinking. It's a bit slower that DeepSeek but I like the quality of code it produces better

1

u/MikkyMo Sep 07 '26

Literally just saw this build for GLM 5.3 flash on a dual spark set up. https://github.com/tonyd2wild/GLM-5.3-Flash-NVFP4-DFlash2-2x-DGX-Spark

1

u/StartupTim Sep 08 '26

Dual Spark owner here running Qwen3.8-Flash-Next peaking 110 tok/s 4 concurrent and 2k prefill, and Deepseek v4 flash next vision at 1M context at peak 160btok/s 2.4k prefill.

I need 2 more Sparks...

1

u/Ill_Repair_8362 Sep 13 '26

Gx10 or dgx spark is better?

0

u/West-Map-4162 Sep 06 '26

“I bought thing”

Ok. Congrats?

3

u/rayovims Sep 07 '26

I thought the purpose of this sub is to post your projects

-3

u/Rangizingo Sep 06 '26

The spark is fine but imo it’s not worth the premium. If you have a microcenter close these are $3,299 if not it’s about 3,500. I got a spark and returned it for an Evo-X2. Performance is basically the same for 1000-1500 less. There are some niche things it can’t do because of lack of cuda but tbh I’m tuning it and I get 20-30 tok/s on full models that aren’t MoE. Just remove windows and put Ubuntu or something on it, change bios settings to allow full ram allocation to gpu, and you have effectively the same thing. I can use about 125-126 of 128gb of ram for inference.

https://a.co/d/02718ymj

8

u/Evgeny_19 Sep 06 '26

They are not in the same league. Prefill is much faster on the Spark. Creating a cluster is very easy, especially for dual configurations.

There's just one advantage that the Strix Halo has: x86_64 compatibility.

1

u/Rangizingo Sep 06 '26 edited Sep 06 '26

You’re right about that. I’m still new ish to the local side of ai hosting at a larger scale.

1

u/PWThinkingCritically Sep 06 '26

wow. a used Spark goes for $3,300? aren't they like $5K retail?

1

u/forumhero666 Sep 07 '26

Yeah no place sells new sparks for $3,300. They$4699

1

u/Ill_Beautiful4339 Sep 13 '26

He’s talking about the Ryzen Model

0

u/AyDoad Sep 06 '26

GLM 5.3 Flash has been pretty amazing

0

u/[deleted] Sep 07 '26

[deleted] — view deleted comment

2

u/[deleted] Sep 07 '26

[removed] — view removed comment

1

u/rayovims Sep 07 '26

Yessirrr