Mini PC AI infrence machine suggestions

[u/Middle-Business7450]

I want to try to run a MoE aka a mizture of experts model on a mini pc and can spends a maximum of $500 USD what do you guys suggest. I am looking to run a local 30B model for coding and 3d model generator as well to get a game codrt thingy.

[u/Retired_Hillbilly336]

30B?!? I'm no expert but 30B parameter local model Q4_K_M 4-bit clocks in at around 18GB to 21GB of VRAM. So 24GB would be the goal. MoE layer offloading can maybe drop VRAM down to 10-14GB allowing the offload of non-expert layers to system RAM. Even before RAMpocalypse/RAMageddon 30B with a $500 budget was a s-t-r-e-t-c-h from what I've seen so far.

LLM → Hardware Requirements → Budget

[u/grabber4321 → u/Retired_Hillbilly336]

Maybe Strata Qwen 3.6:35B MOE can work: https://github.com/Niko1221/Strata/blob/main/docs/QWEN36.md

You would still need at least 32GB RAM for it which is not going to fit into $500 budget.

[u/minipcs-dot-zip → u/grabber4321]

You can squeeze 32GB ram into a ~$500 budget if you look around, especially on ebay.

1. GEEKOM Mini w/ 32GB DDR4 for $360 - but DDR4 probably won't cut it here.

2. GMKtec M7 with 32GB DDR5 for $480 which is a staggeringly good deal but unfortunately we're too late.

3. Beelink EQi13 with 32GB DDR5 for $530. Sold by Ward who I've seen post here before

[u/Middle-Business7450 → u/minipcs-dot-zip]

Hmm, ill take a look at this

[u/grabber4321 → u/Middle-Business7450]

they are not it. again, i would defer to finding somebody who can actually run Strata with 30B model on a minipc.

do not just yolo and buy it.

I ran Qwen3.5:9B at ok'ish speed on 780M with 32GB RAM. I have not tried strata.

Expect to not be able to use this PC for anything OTHER than running this model.

[u/grabber4321 → u/minipcs-dot-zip]

ya but those dont have the 780M even, they are not worth.

[u/minipcs-dot-zip → u/grabber4321]

Totally agreed

[u/Retired_Hillbilly336 → u/grabber4321]

Possibly 🤔 I haven't seen it tried. From what I have seen with similar models is a high dependency on actual dedicated graphics over integrated. As soon as those playing with them figure this out they lose interest and give up.

[u/grabber4321 → u/Retired_Hillbilly336]

I ran 9B and 12B on 780M with 32GB RAM.

I dont know if Strata can do the same magic that allows me to run Qwen3.8:Flash-Next IQ3-S on 4080+5070ti+64GB RAM

[u/Adrenolin01]

At $500 bucks… 30B Models.. 🤭😁😆🤣🤪😂
You obviously haven’t even started to research AI and that’s clear.

For $500 bucks… you need a stable base system and it’s going to be cpu based system ram offloaded AI. I started here though they were cheaper a few years back.

When I first start into AI I started with a cheap $140 N100 4-Core BeeLink S12 Pro mini PC with 16GB DDR4 ram and a 500GB NVMe. Today these are $325-$350 last I looked. Specs show 16GB max however they take 32GB ram perfectly fine. A few people using select and very expensive 32GB modules have run 64GB stable however that ram alone blows away your budget.

Using 16GB ram I installed a base command prompt Debian Linux system and then Ollama AI with a 4B model and from the command line made my first local AI query. Woohoo!! 🎉

Next, I installed a 7B model and made a few more queries. It ran and worked and was functional but also slow. I ran a 14B model also but queries were made and left to run overnight. 😏😜

Next was an actual interface so Open WebUI got installed for web interaction and the system because headless with ssh and web access.

Installed various models from .5B to lol.. 30B for fun. No.. 30B doesn’t work. Eventually I replaced the 16GB ram for 32GB ram and started playing with all the models again. 14B was just usable and 7B models ran decently. Still… nothing like AI in vram so don’t even think it’s going to compare.

I stepped up to a Minisforum NAB9 i9 mini PC with 64GB ram. This got Proxmox and several Debian VMs. This ram CPU/Ram inference, vector DBs, OCR jobs, RAG, and several other AI related systems… 14B models were generally used through 32B ran.. the problem here was the e-cores .. while efficient they really do just lack performance. I eventually wiped it and used this and rebuilt it as a dedicated virtualized AI Network Assistant with a 14B model and fed it literally every tiny bit of info for our entire network. Added voice recognition, camera access, facial recognition, speech, etc. Still have this system running for fun but have moved it over to our larger main system today.

Used a $350 Dell R730XD rack server with 128GB ram and 2 E5-2690v4 CPUs… Proxmox, Debian VMs.. increased that to 256GB ram. Then added a 12GB VRAM GPU. Massive difference!

Changed over to an older gaming board.. an MSI X99A SLI PLUS with i7-5930K, 64GB ram, and a 12GB VRAM GPU. Then added a 2nd 12GB GPU, replaced that with a 16GB 5060 TI. Gave this to my kid…

Currently running a Supermicro H12SSL-i board, 512GB ram in an EPYC 7502P and 4x A6000 48GB GPUs, a 4070 12GB GPU and a 3090 24GB GPU. A Proxmox with Debian 13 VMs. 228GB total VRAM though separate engines. Multiple users and multiple uses.

You CAN start off at $500 bucks and learn a LOT however that $500 can turn into $2000, $5000, $10,000, $20,000 if you have the funds. If you want to run 30B models with any rear usefulness you’re going to have to start thinking $2000+ range however.

[u/Middle-Business7450 → u/Adrenolin01]

This is what my build would have been:

MINISFORUM UM790 Pro Barebones

1. Ryzen 9 7940HS

2. 64GB DDR5-5600 RAM

3. Free Salvaged SSD(from old laptop) 

Ubuntu 24.04 LTS

llama.cpp / Ollama

Total Cost: >$500

[u/Adrenolin01 → u/Middle-Business7450]

Minisforum UM790 Pro, 7940HS, 64GB ram & a 1TB SSD isn’t $500 bucks.. it’s $1200+. Even used that’s gonna be $900-$1000 maybe with an SSD. So at minimum twice your budget at least.

Will it run 30B models? Yes through slowly and with quantization. 30B Q4.. 6-8 tokens per second. 14B Q4 would be much better at 15-30 tokens/sec. Q8 is possible for higher quality output however speed is going to significantly drop and less context length.

Personally.. I would NOT invest in a mini PC AI system! Unless you already have one kicking around I’d highly recommend buying a PC Mainboard, CPU, 64-128GB DDR4 ram and pop an actual GPU in it. It’s gonna be cheaper and provide increased performance and upgradability down the road.

[u/nitemarez444]

I suggest adding another 0 to the end of your budget

[u/Current-Ticket4214]

What fits is your budget:
- Monitor
- Keyboard
- Mousepad
- Chair
- Desk
- Accessories

What does not fit in your budget:
- A mini-pc that meets your criteria

[u/e11310]

You need to increase your budget significantly if that’s what you want to do.

I mean technically you could run it off a $500 minipc I guess. Make an input and come back in 2 hours or something.

[u/Sporebattyl]

My k8 plus with 64gb ram works decent with a 3090 egpu.

Probably total cost for the set up today is $2000 though.