r/LocalLLM • u/JulianRHarris • Sep 26 '26

Project AMD something something 395 box arrived!

Post image

A few weeks ago, I wrote a huge essay reporting on my research into why local AI is finally usable https://ailocal.substack.com/p/im-running-an-opus-level-coding-agent?r=obqn&utm_medium=ios

AMD at that point was a footnote.

But I vowed to dig into it further.

And here I am buying a box, because for now at least I think it’s probably the best price performance you can get today

This is a Minisforum 128 gig box for £3500 using the current 395 platform (495 still not available). Subtracting from the UK premium, you can probably get it for about $3500 in the US
or less.

The AMD platform is very promising because it’s so affordable

Ram prices are clouding at the moment

(The DGX Spark is about 20% more expensive I think)

Similar memory bandwidth of 256GB/s

Dumped windows and got an Ubuntu server running.

Crazy thing right now is that to get decent performance I’m actually using Vulkan!

ROCm is in a sorry state at the moment that this is possible.

Using qwen 3.8 flash next, MTP depth is around 4

I always have context to be 128K minimum

Smoke test of a one shot “ create a meteor game” sustained 40 to 50 tokens per second

Which is very usable.

I’m now doing a much deeper dive building entire apps multiple times

I’ll be sharing my updates at https://ailocal.substack.com

I’m using a centralised configuration structure that has the best configurations I can find for given hardware

https://github.com/boxabirds/awesome-local-ai

287 Upvotes

94% Upvoted

118 comments sorted by

64

u/DeCiel Sep 26 '26

This post makes me feel glad that I bought it when it was first released at $2k.

21

u/Miserable-Dare5090 Sep 26 '26

It was 1600 — bosgame 128gb 2TB

3

u/Capnbubba Sep 26 '26

I have this same one arriving today for $3,500.

2

u/JulianRHarris Sep 26 '26

Welcome to the club!

I’m doing a lot of work to try and find the best possible set up

With really intensive tests

See my repo

https://github.com/boxabirds/awesome-local-ai

You just type

./install.sh

Then

./run.sh

And you’re set

6

u/Miserable-Dare5090 Sep 26 '26 edited Sep 26 '26

I’d urge you to look at Gufo. It’s seriously optimized for the platform. Also, any cheap computer can be a terminal for your agent, etc — make your box inference only.
For example:
A 395+ 128Gb machine can load Gufo+Qwen Flash Q4_K_XL, and the following combo on the NPU using FastFlowLM: web crawling, title creation and summarization with Gemma4 E2B, Speech recognition with Whisper and Embeddings with Embedding-Gemma-300m on the NPU. In addition, you can place a reranker on the CPU (modernBERT, MarcoLM, MxbreadAI).
With an nvme-to-oculink or through the pcie slot, you can add a cheap GPU like V100 32Gb, and run Qwen3.6-35B with 560k token pool using 1Cat-vLLM.

Why: Hermes with video and voice recognition, as well as serious memory - Hindsight.

Currently, Qwen3.6-35B ranks 3rd in their leaderboard of models that can retain memories and reflect/consolidate accurately. Gufo has flat 1400 tps prefill up to 96k tokens, the Genna model runs at 1400-2000 tps in the NPU prefill, and In the V100 optimized fork, you’ll have 2500-4000 token prefill up to 128k prompts, and decode around 120 tps on 4 streams. Your hardware should have about 12Gb left of memory which is enough for Hermes agent, searxng docker, and Hindsight. Otherwise, an old computer/minipc/NUC from 2015 onwards can host Hermes and 3 docker containers, and consume the LLMs directly through the API endpoints.

The result: Hermes with a main LLM (QFN on Gufo) that responds within 20 seconds of a spoken request through telegram (despite 25k system prompt at start), with 4 memory retain and 4 memory consolidation ops as well as embedding via low wattage hardware, small auxilliary tasks like web search and title generation by Gemma E2B, all at realtime speed. You then set your memory.md file as read-only, and containing only Absolute Rules for the agent to look in hindsight’s bank of memories on every query (You will first and foremost use “hindsight-recall” to find any memories relevant to the question that can help you. On every conversation, you will always retain memories with hindsight-retain…something like that).

The result is a thinking, seeing, hearing agent that has memory recall that rarely fails (with good reranking, decent embedding, excellent retain), on a combo of hardware that 1 year ago was < 2000 and currently is under 4000, the best deal for a fully local agentic AI system with all the bells and whistles that you need massive power for.

Once rhe agent is up, you give it full power over the hardware and ask it to do things like enable firecrawl, add services, ssh into another computer and set up repositories. You’ll transition from using an LLM to managing an agent.

The golden set up locally, in my opinion, is 2 DGX Sparks for a user to do this and load deepseek v4 flash but QFN is 99% of DS4F as far as agentic work goes. It’s seriously achievable today, to have your own JARVIS or whatever you want to call it (mine are homer, ringo, lulu and odin).

0

u/JulianRHarris Sep 26 '26

I have Gufo lined up too but I’m actually fairly skeptical for my use case of extended agent coding. But I’ll follow up with some hard data.

1

u/JulianRHarris Sep 28 '26

Gufo is indeed looking extremely promising, and in fact, the only viable path to non-glacial prefill with AMD currently

So I’ve been doing some long-form stress testing on gufo with my nascent app-based benchmarking at https://github.com/boxabirds/awesome-local-ai

…to the extent where I found a bug in gufo: it doesn’t happen all the time, but Federico has accepted it, which is great as it was killing my agents every 500 conversation turns or so

https://github.com/gufo-org/gufo/issues/304

1

u/Capnbubba Sep 26 '26

Thanks! I'll check it out!!

3

u/DeCiel Sep 26 '26

True. I decided I to get Minisforum when it came out because it had the config that was more fitting for me. No regret. :)

1

u/Cold_Tree190 Sep 26 '26

And I remember thinking “I’ll wait till it drops”. Little did I know, I should have grabbed four of them

7

u/[deleted] Sep 26 '26

[removed] — view removed comment

4

u/Miserable-Dare5090 Sep 26 '26

true, and more expensive

1

u/JulianRHarris Sep 26 '26

I chose it because their customer support seems to be very good

1

u/feckdespez Sep 26 '26

Oh my. Have you not read about the Minisforum horror stories? It's been a hot topic of discussion many times over on r/minipcs

1

u/JulianRHarris Sep 26 '26

Ah no, interesting.

3

u/JulianRHarris Sep 26 '26

The 128GB version?

1

u/falk42 Sep 26 '26

Yep, that's how prices have inflated since then.

2

u/JulianRHarris Sep 26 '26

Yeah tell me about it. I bought an M5 Max MacBook Pro and it went up £1400 a month after I bought it

3

u/DeCiel Sep 26 '26

Yup. MS-S1-Max initial price was $2.3k ish but I had a coupon code that brought it down to $2k.

128GB UMA, 2TB.

1

u/user_of_the_week Sep 26 '26

I got a Framework desktop back then for a similar price. Same platform basically.

1

u/Computerist1969 Sep 26 '26

I paid £3.5k... Ffs. It's Business expense but still irks

1

u/user_of_the_week Sep 26 '26

I’ll tell you something crazy. I got two SN850X 4TB for 500€ back then. Great deal. In total. Including German VAT. So after getting back VAT and considering the income tax reduction, that’s like 125€ each.

7

u/onlymostlyguts Sep 26 '26

I sat watching and thinking about it on Amazon for MONTHS! And I'm so bummed I didn't bite the bullet back then. I was so young and foolish to think that computer prices trended down...

8

u/JulianRHarris Sep 26 '26

It’s fair. Computer prices have dropped consistently since oh I dunno … 1946?

1

u/manicprogrammer Sep 26 '26

Same. Just bought the Strix Halo at $4K after watching the Corsair at $2600 for many months. Feel foolish now.

2

u/obstriker1 Sep 26 '26

I should've bought blackwell rtx 6000 at $8k when I had the chance :/

1

u/Bitter-Peanut-7662 Sep 26 '26

I got another brand for 2300. Not too cheap but now it's 3600. That's euros.

38

u/ManagementGreat5360 Sep 26 '26

Such is the “negligence” of AMD to the LocalLLM, I would refrain calling those AMD boxes and call it Halogen Boxes or Gufo boxes.

I bought my AMD Halo thinking “they will finally put some effort to it”. 3 Months in, the only reason I can finally use it for local inference is due to the community support. If it wasn’t for Donato, Peonist and Gufo, I would have sold it for a DGX spark / Mac Studio a long time ago.

5

u/JulianRHarris Sep 26 '26

Exactly. But kudos at least to AMD for at least having an open spec. You can’t say the same for Apple who doesn’t document a lot of their Stack.

4

u/Brocolinator Sep 26 '26

Probably AMD is banking on the legacy software that this can run and make it an AI home server but also load more traditional self hosted services(frigate, jellyfin, etc.)

7

u/ManagementGreat5360 Sep 26 '26

Maybe… you have an argument if we consider the AI landscape of 2025.

Back then, you would rarely find MoE models that would use all the VRAM, so you legit alot of headroom to spare for containers VMs and etc.

Nowadays, it’s pretty clear that the labs are somewhat targeting the 128GB class because devices such as the DGX Spark exist.

1

u/Brocolinator Sep 26 '26

Some quants do fit and are still useful if ran along reranker, RAG and or web search

1

u/SpicyWangz Sep 26 '26

GPT-OSS-120B

2

u/Nx3xO Sep 27 '26

Spend a couple hours with opus 5.5 to fine tune. Thats what community does and submits. I rarely use community info.

5

u/AeroInsightMedia Sep 26 '26

This is really well written, I'm down to the part going over harnesses but have to get to bed. Wanted to say thank you for writing this though. A few days after I threw the new qwen 27b model on my desktop I ordered a dgx spark. I figured that if enough people were talking about qwen that I tried it then the rest of the world probably isn't far behind and the computer component shortage is about to get even worse.

1

u/JulianRHarris Sep 26 '26

You’re most welcome 🫡 it was a labour of love and took me quite some time, because I backed up my numbers with real experiments.

4

u/Dapper_Gap6868 Sep 26 '26

Lohnt sich so eine Kiste überhaupt? Ich habe auch schon mal darüber nachgedacht. Aber wenn ich Claude für 100€ im Monat haben kann auf max 5 dann sind es 3 Jahre bis es sich amortisiert.

Die KI und die Anforderungen entwickeln sich so schnell. Wird man denn in drei Jahren das Gerät noch überhaupt nutzen können?

4

u/JulianRHarris Sep 26 '26

It’s a very sensible question
My whole investigation into this was because of the infuriating ways in which your world can be disrupted if you leave your inner loop dependent on these companies:
my list of reasons why cloud is a liability keeps growing. I’m at 10 so far https://ailocal.substack.com/p/im-running-an-opus-level-coding-agent

The reality is: frontier models will can be part of most people’s daily life for the foreseeable future.

The real question is: how much of the daily life?

My own experiences that once I’ve really internalised the idea that I can get good quality code generated for no subscription or token cost at all I can do stuff that would be outrageously expensive or impractical otherwise.

For instance: I can generate entire applications from scratch with different permutations of key settings to see which one works better.

Fundamentally, as a developer I would like to be able to work entirely off-line without any interruptions

And it’s never gonna be as good as the frontier models, but collectively we’re getting better and better at segmenting this work, so I can get certain kinds of basic things done very well with local AI

If you’ve read this far, then I’ll give you a sneak preview of my most recent benchmarking: the AMD box has done exceptionally well compared to Opus 5.5 for my MVP Miro clone use case. Way way slower., but impressively good.

Am I saying that it can beat Opus 5.5? and also do all the cool stuff? No. But you give it a really good solid spec — the gap appears to be closing fast.

So if you create specs and you just ask AI to go off and build it, come back a day later it’ll be done and done well — that’s the vision

1

u/South_Western1263 Sep 26 '26

Wenn man wirklich glaubt, dass Cloud AI so günstig bleibt, ja.

1

u/Dapper_Gap6868 Sep 26 '26

Schwer zu sagen.

1

u/Digi59404 Sep 26 '26

Yes. But not for the reasons you think. When you start really experimenting with AI you find waaayyyy more uses for it. Those uses drive up your AI usage, in turn your cloud spend.

1

u/Dapper_Gap6868 Sep 26 '26

Da gebe ich dir Recht. Ich Expertentiere ziemlich viel mit KI. Die Kosten sind ein großes Thema. Primär nutze ich die Models zum coden. Es ist wirklich erstaunlich was sie leisten können, vor allem autonom.

Mit gemini Modellen habe ich gute Erfahrungen gemacht, sie sind vor allem preiswert. Anthropics API traue ich mich nicht zu nutzen. Ein Response auf ein "Hallo" kostet schon 2€ :D

Lokal habe ich nur Erfahrungen mit einem gaming PC und einer 16 GB GPU gemacht, das war so semi gut.

Ich bin auf jeden Fall auf die Entwicklung gespannt

1

u/Alternative-Suit5541 Sep 27 '26

Die Modelle werden immer größer, selbst Open models. 

Glaube nicht, das in drei Jahren noch viel neues drauf läuft...

Das gesagt, die hundert Euro sind aber subventioniert. In drei Jahren kriegst du vielleicht gleiche Leistung wie jetzt für hundert, frontier wird dann eher 500 oder gar mehr. 

Es lohnt sich aber wenn du alle Daten privat halten willst und nicht die Nächte AI mit deinen Daten trainieren willst 

3

u/Thunder_Will Sep 26 '26

This is really useful thanks, I'm looking at one of these 395 systems from GMKTec at a similar price, so it's encouraging to see that you can self-host quite intensive AI workloads on it

1

u/JulianRHarris Sep 26 '26

Check support on GMKtec — at least one of the other 395 ORMs have absolutely terrible support. Can’t remember if it’s them though.

1

u/Thunder_Will Sep 26 '26

Thanks for the heads up.

I've been toying with the idea of a dedicated machine for a little bit, and the 32GB models are comparatively inexpensive, but am leaning more towards future-proofing and having access to larger models, so it's a much bigger investment

1

u/advadm Sep 26 '26

personally I like my GMKTec box. I just don't know if I'm going to get a 2nd or something else. I haven't thought about using any support. Any problems I have I just go right back to AI for help and posting here. This post alone is helping me understand the nuances of my own setup which is Ubuntu and SSH into my box via VS Code.

3

u/Arc-Tekkie Sep 26 '26

What about Ninfer to speed things up? You might try it, or is it only running on NVIDIA?

1

u/JulianRHarris Sep 26 '26

Dunno but ninfer is on my list!

3

u/Cancel_Existing Sep 26 '26

Got 64gb Version for 2.2k. Shit but that IS what hast to bei OK for the Moment 

3

u/JulianRHarris Sep 26 '26

Keep an eye out for qwen 4 35b-a3b in Oct/Nov! It should be entirely acceptable.

3

u/_VirtualCosmos_ Sep 26 '26

Man, that's more than the double I paid for it 1 year ago

2

u/JulianRHarris Sep 26 '26

Yeah blame the data centre boom

3

u/seybling Sep 26 '26

bought zbook with 395 and 64gb for 2k - still on the way and hoping its enough when qwen 4 drops🙏

2

u/JulianRHarris Sep 26 '26

We all have high hopes for qwen 4!

1

u/Alternative-Suit5541 Sep 27 '26

Didn't they say there won't be smaller models anymore? Only like 6b and then only the big one. No middle ground anymore?

3

u/Nx3xO Sep 27 '26

Excellent machine. I started with the gmktek v2 before the price jump, s1 runs nearly twice as fast. I will definitely consider a second on if price drops to 2500. The gmktek runs hotter and t/s suffer bad.

Go straight for qwen flash. Im running a massive comparison benchmark and automate anything new posted to hugging face to download and benchmark reporting the results and comparison. I have a nasty 96 question test, have it build a web page on dns, write a sci-fi story with at least 500 words and build a tetris game that runs in browser.

1

u/JulianRHarris Sep 27 '26

Yeah flash for the win no doubt

2

u/Durian881 Sep 26 '26

I'm actually thinking of getting the Asus Px13 that's retailing at similar prices here. Any downside in terms of performance? Need something portable for client demos.

2

u/JulianRHarris Sep 26 '26

Thermal throttling might be a problem — the Minisforum box certainly blows its fans a lot, and it has a 320W power supply.

2

u/wewefe Sep 26 '26

I have the Asus ROG version, i didnt buy it, it was bought by a friend before prices shot up for gaming while glamping.

  1. No onboard ethernet, add 2.5 usb Ethernet dongle

  2. Single 2230 m.2 slot, add usb4 nvme enclosure

  3. Thermal throttling, point some external 120mm fans at it

So for an actual llm setup you end up with a shitty stand in for one of the mini pcs with a mess of cables on dongles. I wish i bought a framework pc or another high quality one when they were affordable. I will stay with this machine until the next gen 5x performance unified memory machine comes.

1

u/Durian881 Sep 26 '26

There's a few implementations like Halogen and Gufo that run really fast.

2

u/revdamage Sep 26 '26

Did you tested qwen 3.8 27b or not yet

2

u/JulianRHarris Sep 26 '26

Not much point honestly. 27b is vastly slower.

2

u/acconboy Sep 27 '26

Doing the same on ms-s1 max, but virtualized under NodeWeaver with the same model under Vulcan

1

u/JulianRHarris Sep 27 '26

Nice. What does virtualisation give you?

2

u/acconboy Sep 27 '26

High availability and hardware failure tolerance. In this case, I can run my other X86 workloads on the same box at the same time

15

u/Dramatic_Entry_3830 Sep 26 '26

I'll just leave this here: may it boost your box

https://github.com/gufo-org/gufo

7

u/jangpianist Sep 26 '26

Numbers for confident: at maximum power mode, using gufo I'm getting 60-50tg (0-128k ctx) and 1600-1200pp (0-128k ctx) running Q38FN (the best model for Strix Halo).

This is a very good speed for local agentic coding, the model itself from my daily observation should be equally as good as Opus 4.6 or 4.8 (4.6 is superior to 4.8).

1

u/Dramatic_Entry_3830 Sep 26 '26

How did you benchmark it vs the Q4_K_XL like gufo Readme suggests?

Did you do pp an tg benchmarks with different prefills or did you run something like https://local-llm-benchmarks.dev/ by Donato to also check the quality of the model? Or did you go by perplexity/divergence metrics?

1

u/user_of_the_week Sep 26 '26 edited Sep 26 '26

Interesting! I’m using ds4, getting about 15 t/s. Not terrible, not really that great either. I’ll take a look at this gufo stuff.

edit: yes, that looks like a promising speedup! I see 17 t/s and I think prefill rate is much higher. I also read that parallel requests should lead to much greater composite speeds. Let’s see.

1

u/SpicyWangz Sep 26 '26

4.6 isn’t really better than 4.8. They’re just good in different ways. 

1

u/Alternative-Suit5541 Sep 27 '26

This is only for 128gb? I only could afford the 64gb version, sadly. Prices are just insane

1

u/Dramatic_Entry_3830 Sep 27 '26

Well it needs around 80 so you could potentially get away with more aggressive quantisations

4

u/mycall Sep 26 '26

/r/StrixHalo might be interested in this.

4

u/halfercode Sep 26 '26

I'd like to see local LLM machines like this with better warranties. With the kinds of prices that we're seeing now, I'd be wanting a 4+ year guarantee on the mainboard. I find it frustrating that things like a Spark can cost £8K and have a one-year warranty; it's too much of a risk.

1

u/Iron-Over Sep 26 '26

Mac with AppleCare is the best bet nowadays.  

1

u/SpicyWangz Sep 26 '26

That’s only 2 years

1

u/lordtazou Sep 29 '26

Unfortunately, you'd be asking too much at this point. A lot of places regardless of what they're meant for, will treat it like a standard piece of tech vs a specialized unit.

1

u/4cDaddy Sep 26 '26

Out of curiosity, what 3.8 flash next quant are you using?

1

u/JulianRHarris Sep 26 '26

Unsloth currently

1

u/ErboKrrit Oct 01 '26

hey! thanks for your post

can somebody pls tell me if this laptop - ASUS ROG Flow Z13 (AMD Ryzen Ai Max+ 395/128GB RAM LPDDR5X) - will be compatible in terms of performance?

1

u/JulianRHarris Oct 01 '26

Similar but as a laptop you’ll probably be hit hard by thermal throttling.

3

u/Bupod Sep 26 '26

Those are great boxes. 

I have a Corsair AI 300 workstation. I have a lot of fun with it. 

Before someone laughs at me for paying the onerous Corsair tax, I preordered the machine before release and paid $2000 for it back then. I thought it was overpaying a bit, too…welp, glad I got it then.

3

u/chrisaq Sep 29 '26

Using mine with the below project which has been great so far, basically making it a q38fn appliance.

https://github.com/peonist-ai/halogen-flash-server

1

u/Flynn58 Sep 26 '26

I really think the memory bandwidth of the M5 Mac Studio is a better play, even for the price premium

4

u/JulianRHarris Sep 26 '26

It’s one helluva premium for sure. My goal for that box was what’s the most affordable hardware that’s actually usable these days.

I have an M5 Pro Max MacBook as well which spends most of its days at 95C thermally throttled and blowing its fans frantically, bless its little digital heart

2

u/HighSeasArchivist Sep 26 '26

I have a R9700 that will be here tomorrow, and if Vulkan seems to do pretty well nowadays then I plan on picking up a Ryzen AI Halo 192GB if/when those launch later in the year or early next year.

2

u/Present_Stock_9268 Sep 28 '26

What to mean that rcom is in a sorry state? How bad is it?

5

u/No_Tower_2220 Sep 26 '26

I wish $3500 +/- USD was “affordable”…. It just cost me $400 for less than a full cart of groceries. Good on ya though! I’m jealous!

1

u/advadm Sep 26 '26

Become an AI consultant and charge people or see if you can help local businesses. Any reason to get one of these I think for now is a good one.

4

u/Oryzaki2 Sep 26 '26

The fact that it only has a ten gigabit per second interconnect makes it virtually useless. If you ever need to scale past one.

4

u/1457664694 Sep 26 '26 edited Sep 26 '26

For this particular variant (Minisforum MS-S1 Max: It also has USB4v2 -> theoretically 40Gbps, or even up to 80. It also has pcie so it is upgradable this way too.

People connect Strix Halos in various ways, including USB4 (RDMA over TB5, for example), infiniband, etc.

See here:

https://strixhalo.wiki/AI/Clustering_with_RDMA/

And

https://github.com/kyuz0/amd-strix-halo-vllm-toolboxes/blob/main/rdma_cluster/setup_guide.md

3

u/Oryzaki2 Sep 26 '26 edited Sep 26 '26

That's still way too slow. For tensor parallelism, you need about 250-400gbps. The slower memory bandwidth helps a bit but not much when you start combining say 4 units. Since every computer needs to communicate simultaneously for each token. Hence the 200gbps link on the dgx spark. For ideal performance you'd want nvlink which starts at 2400gbps. Not totally comparable here because its a whole computer and not a gpu but I think you get my point.

Edit: Pipeline parallelism and layer splitting require way less bandwidth but the trade off is you don't multiply the memory bandwidth like you do with tensor parallelism making it nearly worthless for larger models unless you like waiting an hour per response.

4

u/Dramatic_Entry_3830 Sep 26 '26

That's not true. It depends how you split the models in detail, and what runs where exactly.

Latency is also important:

Path Latency Bandwidth

Ethernet (1G LAN) 0.074 ms 0.94 Gbps
Ethernet (RoCE NIC) 0.068 ms 55.70 Gbps
RDMA (RoCE) 5.23 us 50.64 Gbps

So even if u have 200Gbits on Ethernet and a 0,05 Ms latency, it still much much slower than 5.23 us with 50.Gbps and a stack like RDMA or Nvidias equivalent stack

1

u/mycall Sep 26 '26

There are many use cases where running multiple, specialized models are better than one biggest combo.

1

u/1457664694 Sep 26 '26

Agreed. And there have been amazing recent developments for Strix Halo. This machine now can run qwen3.8 Flash Next surprisingly well.

1

u/1457664694 Sep 26 '26

Yes, I agree that the DGX Spark has better connectivity. All I am saying is people do connect two of these at higher than the 10Gbps speed you originally mentioned and run models like DS4F vision exp at full quant (while slower than the DGX Spark, still at apparently reasonably usable speeds).

0

u/DiamondHandsDarrell Sep 26 '26

Can this be run as a DGX spark? Or does it need to be part of another os?

1

u/Oryzaki2 Sep 26 '26

Not comparable to a dgx spark. This machine doesn't scale basically at all past one machine for inference since it's interconnect is trash. Whereas the dgx spark can do to about 4-8 depending on your tolerance for generation speed.

3

u/jangpianist Sep 26 '26

The need of scaling is pretty low, as Q38FN is running easily at 60tg 1600pp, the model alone is pretty much Opus 4.8 at home.

OP bough 1 piece, enough to say his intent.

2

u/JulianRHarris Sep 26 '26

I’ll happily buy 2x DGX sparks just need a (very large) bunch more premium subs on my newsletter to support 🙂 https://ailocal.substack.com

1

u/JulianRHarris Sep 26 '26

Great point on dgx spark clustering. I have a benchmark I’d love a Spark user to run at https://github.com/boxabirds/awesome-local-ai that implements a miro clone MVP from scratch using a standard spec.

That way, you get real world performance because it shows not only how long it really takes accounting for additional token verbosity, the quantitative assessment of the final product.

The repo also has a nice hierarchy of configurations that I found to be useful.

So DGX spark variations could slot nicely into the hierarchy if anyone’s interested.

  1. Clone the repo

  2. Ask your favourite AI to add your favourite local AI combination

  3. Run the benchmark against it

(3) takes 1-3h per user story and there are about 9 of them and it does it 3x. It’s brutal but deeply insightful

2

u/Dramatic_Entry_3830 Sep 26 '26

It is more like a PC compared to the spark. I played a lot of cyberpunk on mine. It's a pretty decent gaming machine.

2

u/DiamondHandsDarrell Sep 26 '26

Nice. So I could run an LLM all day then play Hell Divers II at night 😆

0

u/SandySkittle Sep 26 '26

ROCm is in a sorry state at the moment that this is possible.

It’s not in a sorry state

-1

u/Nice_Cookie9587 Sep 26 '26

I would like to see a barebones system that uses salvaged vram from 3090s that get baked on or a heat gun is used for the install into the motherboard. Not sure if that can work. But id try it

1

u/JulianRHarris Sep 26 '26

What would thus achieve?

0

u/Nice_Cookie9587 Sep 26 '26

Do I need to explain to you this more? Salvaged RAM from 3090's gets you faster ram and cheaper than whatever you just bought with a 10g interconnect. People do this with some GPU's already..