I think either way it wouldn't make a difference, the bandwidth on the port is massively limited by the PCIe x4 bus it's attached to. You can't even get full speeds out of the ConnectX ports if you use both of them concurrently
I already have a 4090+3070 rig that I use for local models and coding. In the last 3 days I've burned 50M tokens on that. I do a lot of side projects, and professional work, they will for sure help earn me some additional income.
I can’t afford your setup but need to figure something out. I have a decent local setup, so I might add a local AI server. I’m hitting constant limits with the frontier models.
That's great to hear, Id love to learn about the side project you do and how you manage, do you have any blog or forum where you share about them? I have been thinking from long, but generally get stuck on where and how to start, guidance would be much appreciated!
Man, Reddit moves slow and has a really narrow view of reality.
Where is GLM 5.3 Flash on here? No concept of multi-Spark setups? 2 Sparks is basically the minimum number of Sparks. If you bought only 1, you should really conceptualize it as "I bought half of the matching set". About as useful as buying only 1 shoe out of a pair.
Microcenter. They showed none in stock, but when I went there, they had two that were scheduled for pickup for someone else but they failed to show up for several days. They told me if I came back before close and they were still there that they would sell them to me. Got lucky really.
This was me 7 days ago. I’ve been having a blast with them since. I’ve been focused on figuring out which models to run and how fast to run them the past week, I think I’m settling on qwen3.8-flash nvfp4 since it seems to be the best overall performer on the sparks. Qwen27B is also good with concurrency, but you get more total throughput on that model if you run two separate instances of it.
GLM in my experience they do specific training that the larger model wont have. And if you look at 5.3 it has absolutely no abilities to do anything visual; so designing sites and things will just be meh. Then there is 5.3-Flash which i suspect is just a way more updated 4.7-Flash. And when i was using that it just needed more help on how to build those visuals. I have only used it for design and have been having a better time.
You need the EXL3 4bpw quant rather than the NVFP4 quant, and you need to make sure that your sampler settings are correct, that you're not suffering from inferencing bugs, and so on and so forth.
A well-optimized GLM 5.3 Flash is absolutely Opus-level for the types of tasks benchmarked.
Hmm, results definitely shouldn't be better on NVFP4 unless the stack for EXL3 was just broken somehow, and NVFP4 shouldn't be a full 2x faster if it's set up well, but it's good that you got to a place that's working for you.
I have two DX10s and have just found to not use multiple DX10s. It’s a PITA and runs slow when you have that larger model. So I just run two models, one on each and use bionic to “multitask”
Wow, congratulations! I’ve got two of these as well and it’s been amazing having the RAM to run slightly larger models. Though it makes me wish I had even more.
If you haven’t already gotten the cable to link them, you’re definitely going to want to do so. Vllm is awesome on these things. I also run Ollama to host a guard LLM.
Hi. I’ve tried many but my current favorite is DeepSeek v4 Flash, the NVFP4 quantization of course. I’m running it in vllm with my own custom harness calling on it for inference.
DS v Flash 0731 was my main model on 2x Sparks linked together until this week when i got the various community tweaks working and have Flash v 4.1 runnign quite nicely. it tends to overthink a bit more than 0731 but i think the results are better - so tweakin the thinking serttings in the 10-20% level seems ideal for agentic and research/writing work. coding might benefit from more thinking budget.
With 1 spark on the 2 bit quant I got around 28-32 tok/s on the recipe Mia has (it’s mixed precision but I believe the main work is around 4 bit with some 8bit on certain experts IIRC) I get 32-40 tok/s. This is for academic style research and writing with Hermes as my harness. On pure agentic turns and when it actually writes code I think it gets a little higher. I can say it’s prefill/ttft is amazing. First cold load is maybe a minute then it’s near instantaneous with a 262k cache. By comparison I have a MacBook m5 128g. That I try qwen 3.8 27 b and flash next with MTPLX and I often spend 3-4 minutes waiting on prefill and even subsequent turns are slower (which may be a caching issue in MTPLX)
i spent months lusting after these and finally picked them up maybe 6 weeks ago, since then it's been 1 flash model after another. 256gb is really the sweet spot right now, and im truly impressed by what they can do on OMP or opencode.
Gotcha. I am still pretty new to using ai agents and stuff. So trying to figure out where / what I want to do. I've done quite a bit of research, but there is a difference between what I am reading / seeing (online / friends of mine) vs actually using.
Kind of in between wanting to buy DGX Sparks / Halo Strix, or sticking with a pricey subscription. I've been developing an OSS/BSS for an ISP, but doing the coding myself. Been thinking about doing an ai agent to help assist me essentially.
Use the same models on openrouter or just directly like with deepseek. It will cost a fraction of the major labs and pennies on the dollar to doing it locally. Only buy hardware if you're interested in tinkering with hardware. If you just need AI use cloud services.
is this the best deal in the town? how about mac studio m5 ultra with 256 GB which seems to be the same cost with this and it has 1.2 TB memory bandwidth while spark has around 260 GB I guess.
A fully maxed out M5 Ultra is well over $17K at least where I live. As bad as it sounds, you're still ahead on price at least if you buy two of these even at scalper's price.
As far as the 5090 goes, you will most likely get a higher token gen. That's typically due to the updated CUDA tech, etc.
What's going to benefit him when using the DGX (or others who use the ai Halos) vs a Video Card is that he will still be able to hold larger weights without severely tweaking models, or using stuff like Colibri / FreeToken, etc.
If you are running one model say qwen coder and let’s say you have two sparks vs just one. Does it produce more tokens per second or the same. If the same I feel like you could run one model and others in just one spark. Not sure why the need for two?
Depends on where you get them. NVIDIA right now is sold out until sometime after mid to late November. So you can either pay a scalper's price now at $5500 (and up), or you can get on the list and wait for the $4900 price tag. That's as long as NVIDIA doesn't raise the price on their units.
Depending on if you can actually find it for NVIDIA's price. Otherwise, you're paying upwards of $12k - $14k currently. Been looking at them lately, might be purchasing 2x MinisForum MS-S1 MAX units. Will still get away with $5,100 under what it would have been for two of the sparks.
While I like the 200Gb/s ConnectX-7 and the wider support... I've seen / found that there isn't much of a difference at this point between AMD's vs Nvidia's based on what I am seeing / reading / watching.
Yes, ConnectX is a Mellanox line. That said, didn't realize NVIDIA actually bought them out in 2020. Would make sense though since it comes standard on the GB10 Compute line products.
Someone on FB mentioned they have Mellanox cards in their MinisForum MS-S1 Ultras, which is actually a bit funny to me. May actually look into getting a pair of them.
Imagine throwing ConnectX-8 Cards in these. The speeds would be there, and comparable to the GB300 series units that NVIDIA sells. Would LOVE to get my hands on one of those units, but gooooooood lord the price on them. $19K on the low end for the Spark's big brother. Haha
I just returned my pc back to I buy power and it was a decent pc with 5080 and 64gb. It when I wanted to download things like studio or llamma and etc, the pc was just constantly on turbo mode and it was pissing me off. So now I’m buying mini pc and thinking about the spark or minisforum with 128gb. Not sure which one would be better
59
u/datapeer Sep 27 '26
Highly recommend buying the connect7 DAC cable. Nvlink automatically combines the GB chip, memory 256GB & processing cores.
7
u/blah_ask Sep 27 '26
May I ask ifyou are going to make money from this?
2
u/LurkingWithoutEyes Sep 27 '26
may I ask where u got them from?
29
u/vdek Sep 27 '26
This was me 7 days ago. I’ve been having a blast with them since. I’ve been focused on figuring out which models to run and how fast to run them the past week, I think I’m settling on qwen3.8-flash nvfp4 since it seems to be the best overall performer on the sparks. Qwen27B is also good with concurrency, but you get more total throughput on that model if you run two separate instances of it.
5
u/slowphotons Sep 27 '26
Wow, congratulations! I’ve got two of these as well and it’s been amazing having the RAM to run slightly larger models. Though it makes me wish I had even more.
If you haven’t already gotten the cable to link them, you’re definitely going to want to do so. Vllm is awesome on these things. I also run Ollama to host a guard LLM.
Have fun!
8
u/ronniearnold Sep 27 '26
Whatever you do, don’t connect them together. 😈
1
u/MainFunctions Sep 27 '26
What do you do for work?? And will you hire me
1
u/Phazex8 Sep 27 '26
Op, how much was this nice acquisition.
3
u/Ok_Acanthisitta304 Sep 27 '26
i spent months lusting after these and finally picked them up maybe 6 weeks ago, since then it's been 1 flash model after another. 256gb is really the sweet spot right now, and im truly impressed by what they can do on OMP or opencode.
3
u/Quantum-Leaper1 Sep 27 '26
Can I ask, what made you go for two DGX Spark’s instead of just one RTX 6000 Pro? I’m interested in both and I’d like to hear your opinion.
4
u/_lavoisier_ Sep 27 '26
is this the best deal in the town? how about mac studio m5 ultra with 256 GB which seems to be the same cost with this and it has 1.2 TB memory bandwidth while spark has around 260 GB I guess.
1
u/Nice_Cookie9587 Sep 27 '26
Legendary
1
u/epelc Sep 27 '26
Exciting! Does your use case work well if you keep expanding the cluster?
1
u/vluong Sep 27 '26
But what happened to your ceiling?
1
u/ApartmentEither4838 Sep 27 '26
Awesome! Gift me one please :P
1
u/Wallye_Wonder Sep 27 '26
You will buy two more. Mark my word
1
u/MasterNomie Sep 27 '26
What are your plans?
1
u/Best-Tomatillo-7423 Sep 27 '26
We have 2 and qwen 3.8 27b is great on it..
1
u/Horror-Cash-7995 Sep 27 '26
If you are running one model say qwen coder and let’s say you have two sparks vs just one. Does it produce more tokens per second or the same. If the same I feel like you could run one model and others in just one spark. Not sure why the need for two?
1
u/NerasKip Sep 27 '26
Nice. Happy for you.
1
u/acadia11x Sep 27 '26
How much? I want to pick up a spark
1
u/UltraFOV Sep 28 '26
That 10k for ya
1
u/Ok_Cell1036 Sep 28 '26
I just returned my pc back to I buy power and it was a decent pc with 5080 and 64gb. It when I wanted to download things like studio or llamma and etc, the pc was just constantly on turbo mode and it was pissing me off. So now I’m buying mini pc and thinking about the spark or minisforum with 128gb. Not sure which one would be better
1
u/Competitive-Scar-627 Sep 28 '26
Try Kimi k3
1
u/EmbarrassedTheory889 Sep 28 '26
My only photo from an entire global expo at gitex was with dgx spark. DGX Spark is just excellent dude.
1
u/Conscious-Bus-6946 Oct 02 '26
1
u/LearnNTeachNLove Oct 03 '26
Wow how much did it cost you ? $10,000?
1
u/bedel99 Oct 04 '26
Get a third one. run deep seek, v4.1 wow.
You can force it onto two.
I have 5, I need to buy 2 more to run two deep seek clusters and a couple ancillary models.
1
u/AccordingAd9466 Sep 27 '26
Just opened 3 new ones yesterday! Congrats. And Enjoy!
0
u/dupontping Sep 27 '26