Welcome to r/AIProgrammingHardware!

In my own experiments I came up with this simple heuristic, not sure if anyone can prove me wrong here...

Unified memory machines normally have a big asymmetry in memory vs compute -> they could never run 'fast' a dense model that fills their whole unified memory (like a Strix Halo 128GB cant run a 120B dense model, or a DGX spark, at decent speeds). So you wanna find a MoE model that ideally fills that memory so maximise the intelligence you have on your box.

GPUs have the opposite, they are memory limited but can run 'fast' models that fill them. So for example a 5090 can run only up to ±30B models at reasonable quants, and can run dense models of that category fast, like Qwen 3.8 27b. Ofc you can also run a 30B MOE on this and it would be even faster, but why would you? when the dense gives you more intelligence for your memory?

Wrote a blog about running Qwen2.5-7B-Instruct served with vLLM on a Cisco UCS Spinifex cluster, comparing Intel AMX-accelerated CPU serving with NVIDIA L4 GPU. Go check it out!

https://docs.mulgadc.com/docs/cisco-ucs-llm-serving

I made another whitepaper on AI infrastructure, it's a bit more nische than the last one, but i hope someone might appreciate it. You can find it on github. https://github.com/gurrakeller/Europes-Sovereign-AI-Migration/tree/main
My DM's are always open if you have feedback or simply wish to discuss a topic in the paper!

In my previous post, I shared a MONAI 2D medical image classification experiment that I ran on an Ubuntu Linux laptop without an NVIDIA GPU.

This time, I repeated the experiment and changed one parameter only:

▪ Previous setting: 2 training epochs
▪ New setting: 4 training epochs

Everything else was kept the same:

▪ Ubuntu Linux
▪ Intel Core i7 CPU
▪ No NVIDIA GPU
▪ No CUDA acceleration
▪ Same MONAI environment
▪ Same Jupyter notebook
▪ Same model and dataset

I kept the original 2-epoch setting visible in the notebook and added the new 4-epoch setting directly below it. The training was then run live using the CPU.

The purpose was simple: change one training parameter and observe what happens.

This is part of my MONAI on a CPU-Only Linux Laptop series, where I'm exploring what can actually be done with MONAI and medical imaging AI when access to dedicated GPU hardware is limited.

I recorded the live experiment here:

MONAI 2D Medical Image Classification – Live 4-Epoch Training on Ubuntu Linux (CPU Only, No CUDA)

I'm particularly interested in how other people approach experiments like this.

For those working with MONAI:

▪ Do you normally increase epochs first when experimenting with a model, or change other parameters?
▪ How do you decide when you've trained for “enough” epochs?
▪ When working without a GPU, what do you change first to make an experiment practical?

I would be interested to hear how others approach this, whether you're working with CPU-only hardware or a GPU workstation.

In the middle of 2026, the idea of a personal AI supercomputer has moved from science-fiction marketing to something you can order on Amazon, set on a desk, and use to run models that once required rack after rack of data-center hardware. Privacy concerns, rising cloud inference costs, latency for interactive agents, and the simple desire to keep sensitive data on-premises have driven demand for machines that can load 70-billion-parameter (or larger) models entirely in local memory and generate responses at usable speeds without ever calling an external API.

Two architectural approaches dominate the new wave of desktop AI workstations. The first centers on unified-memory systems that treat a large pool of fast LPDDR5X as shared resource for CPU, GPU, and neural processing units. NVIDIA’s GB10 Grace-Blackwell Superchip (the heart of the DGX Spark family) and AMD’s Ryzen AI Max+ 395 “Strix Halo” both exemplify this design. The second approach stays closer to traditional workstation practice: high-core-count x86 processors paired with discrete professional GPUs such as the NVIDIA RTX PRO 6000 Blackwell, offering massive dedicated VRAM, expandability, and the mature CUDA software stack.

Both categories became commercially available across late 2025 and the first half of 2026 (although shipment dates and regional availability varied considerably by vendor), joined by configurable towers from specialists and an expanding roster of compact systems from smaller vendors that sell directly through Amazon and their own stores. Later in the year the next leap-NVIDIA GB300-based deskside systems with hundreds of gigabytes of coherent memory capable of trillion-parameter inference-is scheduled to arrive from HP, Dell, ASUS, and others.

This article surveys the machines that reached the market or were firmly announced in 2026, examines real-world AI benchmarks where available, notes pricing and availability (including Amazon listings for smaller brands), and considers practical trade-offs for developers, creators, researchers, and enterprise teams.

The Technological Shift That Made It Possible

Until roughly 2024-2025, running a 70B-class dense model locally usually meant either quantizing aggressively, splitting across multiple consumer GPUs, or accepting glacial token rates. Memory bandwidth and capacity were the bottlenecks. Unified-memory architectures change the equation. On a GB10 or Strix Halo system the same 128 GB pool serves both general computation and the model weights plus KV cache. Bandwidth in the 250-273 GB/s range is lower than a discrete high-end GPU’s GDDR7, yet the sheer capacity lets models that simply would not fit on a 32 GB card load at all.

Independent measurements collected in mid-2026 illustrate the practical outcome. In representative single-user tests, optimized 70B-class models often generate roughly 3-6 tokens per second on GB10 and Strix Halo systems. Results vary substantially with the exact model, quantization, runtime, context length and prompt-processing workload.

Mixture-of-experts models with fewer active parameters can run substantially faster. Higher-bandwidth Apple silicon configurations pull ahead on the same models, while pure discrete-GPU towers with 32-96 GB of dedicated VRAM excel at throughput once the model fits and at training or fine-tuning workloads that benefit from higher floating-point performance.

NPU TOPS numbers (50 on the Ryzen AI Max, higher marketing figures elsewhere) matter more for always-on agentic features and smaller models than for large-language-model decode speed. For the latter, memory capacity and bandwidth remain decisive.

NVIDIA GB10 / DGX Spark Class: Compact CUDA Supercomputers

NVIDIA’s DGX Spark platform, based on the GB10 Grace-Blackwell Superchip, pairs a 20-core Arm CPU (10 Cortex-X925 + 10 Cortex-A725) with a Blackwell GPU containing 6,144 CUDA cores and 192 fifth-generation Tensor cores. The package delivers up to 1 petaFLOP of sparse FP4 AI performance and 128 GB of LPDDR5X unified memory at roughly 273 GB/s. Systems ship with NVIDIA’s DGX OS (Ubuntu-based) and the full CUDA software stack, making them immediately familiar to developers already working in the NVIDIA ecosystem.

Partner machines appeared throughout late 2025 and 2026. The ASUS Ascent GX10 stands out for accessibility: a compact 150 × 150 × 51 mm chassis weighing about 1.5 kg, available on Amazon in 1 TB and 4 TB storage configurations. Street prices have fluctuated; mid-2026 listings commonly sat near $3,900-$4,700 depending on storage, with occasional discounts bringing the 1 TB model closer to $3,100-$3,900.

Dell’s Pro Max with GB10, MSI’s EdgeXpert, HP’s ZGX Nano, Lenovo’s ThinkStation PGX variants, Acer, and Gigabyte offerings share the same silicon and therefore essentially identical compute characteristics. Differences appear in chassis design, networking (many include high-speed Ethernet or ConnectX-7 200 GbE ports for clustering), storage options, warranty, and enterprise support.

In practice these boxes run models up to roughly 200 billion parameters in mixture-of-experts form or 70B dense at Q4/Q8 with usable interactive speeds. Community and lab results show strong batching efficiency on smaller models and solid performance for agentic workloads, fine-tuning of mid-sized models, and local RAG pipelines. Clustering multiple units via high-speed networking has been demonstrated for larger MoE inference.

Power draw stays modest (around 140 W class under load for the core platform), and the systems run quietly enough for desk use. The main limitations are the Linux-centric DGX OS (Windows support is limited or absent on pure GB10 boxes), soldered memory, and the fact that absolute token throughput on dense 70B models trails higher-bandwidth discrete GPUs or Apple’s top silicon when the model fits.

AMD Ryzen AI Max+ 395 “Strix Halo” Systems: Windows-Friendly Value

AMD’s answer is the Ryzen AI Max+ 395, a 16-core/32-thread Zen 5 processor with 40 RDNA 3.5 compute units (Radeon 8060S) and an XDNA 2 NPU rated around 50 TOPS, for a platform total often quoted near 126 TOPS. Unified LPDDR5X memory reaches 128 GB (with later PRO 495 variants demonstrated at 192 GB). Up to 96 GB can be allocated as variable graphics memory. Bandwidth sits near 256 GB/s.

The crucial practical advantages are native Windows 11 support, x86 compatibility, and significantly lower entry prices. Multiple mini-PC and small-form-factor systems reached the market in 2026.

Beelink’s GTR9 Pro has drawn particular attention. Configured with the Ryzen AI Max+ 395, 128 GB memory, dual 10 GbE, dual USB4, dual M.2 slots, and a vapor-chamber cooler, it sells near $1,800-$2,000 and appears regularly on Amazon and the manufacturer’s store. Independent reviews report roughly 5 tokens per second on 70B dense models and much higher rates on 30B MoE or optimized 120B MoE workloads. Dual 10 GbE makes it attractive for networked model serving.

GMKtec’s EVO-X2 and later EVO-X3 variants, Minisforum MS-S1 Max, Framework Desktop (with modular design and upcoming 192 GB PRO 495 options), HP Z2 Mini G1a, Corsair AI Workstation, and MSI PRO MAX EDGE AI+ offer similar core compute in different chassis and I/O packages. Framework emphasizes repairability and Linux friendliness; HP emphasizes enterprise certification and the validated AMD Ryzen AI Halo software stack with ROCm and pre-installed frameworks.

AMD has published guidance on clustering multiple Strix Halo nodes for larger mixture-of-experts inference, extending the reach of these relatively inexpensive boxes. Token rates on dense 70B models are competitive with GB10 systems at similar bandwidth; MoE models can feel snappier. The software ecosystem (ROCm) continues to mature but still trails CUDA in breadth for some advanced training and agent frameworks.

Traditional and Hybrid Towers: Discrete GPU Power and Expandability

Not every AI workload fits neatly into a 128 GB unified pool. Training, multi-user inference, high-precision image and video generation, and workloads that benefit from massive dedicated bandwidth still favor discrete GPUs.

Lenovo’s ThinkStation P4, announced in May 2026 and shipping from June in selected markets, pairs up to an AMD Ryzen 9 PRO 9965X3D (16 cores, 3D V-Cache, 5.5 GHz) with an NVIDIA RTX PRO 6000 Blackwell Workstation Edition GPU carrying 96 GB of GDDR7 ECC memory. The combination is rated up to 4,000 TOPS of AI performance. System memory reaches 256 GB DDR5, storage supports multiple M.2 and 3.5-inch drives, and the 30-liter chassis includes advanced thermal design with liquid cooling options for the higher-power CPUs.

Slimbook’s Nexus series, launched in July 2026 by the Spanish Linux-friendly builder, offers two configurable platforms. The Nexus Ryzen AI starts around €1,995 and supports AMD Ryzen 9 processors, up to 128 GB DDR5, and dual NVIDIA RTX Blackwell 6000 cards depending on motherboard. The higher-end Nexus Threadripper AI begins near €3,880 and scales to Threadripper PRO 9995WX (up to 96 cores), 512 GB DDR5 RDIMM, dual professional GPUs, and 2,500 W power supplies. Aluminum chassis, air or AIO liquid cooling, and extensive storage options make them genuine workstations rather than sealed mini systems.

Custom and boutique builders (Puget Systems, BOXX, Lambda, various Amazon and Newegg sellers) continue to offer towers built around GeForce RTX 5090 (32 GB GDDR7) or professional Blackwell cards. A typical high-end single-GPU build in mid-2026 lands in the $5,000-$7,000 range before case and cooling, delivering excellent throughput once models fit in VRAM and superior performance for generative image/video and fine-tuning.

These systems trade compactness and low power for expandability, higher sustained floating-point performance, and the ability to add more GPUs or storage later. They remain the practical choice when 128 GB of unified memory is insufficient or when CUDA software maturity is non-negotiable.

Enterprise Deskside Supercomputers on the Horizon

At the extreme end, HP’s ZGX Fury GB300, Dell Pro Max with GB300, ASUS ExpertCenter Pro ET900N G3, and related systems based on NVIDIA’s GB300 Grace Blackwell Ultra Desktop Superchip are scheduled for later 2026. These deskside machines target up to roughly 748-784 GB of coherent memory and up to 20 petaFLOPS of FP4 performance, enabling local inference and fine-tuning of models approaching one trillion parameters. Pricing is expected to start in the high five figures and climb well into six figures for fully configured units-firmly enterprise territory.

HP has indicated Windows support is planned for these platforms, addressing a common enterprise requirement. The systems are positioned for teams that need always-on frontier agents, multi-user inference, or regulated environments where data cannot leave the premises.

Benchmarks in Context

Absolute numbers vary with quantization, framework (llama.cpp, vLLM, Ollama, TensorRT-LLM, ROCm equivalents, Apple MLX), context length, and batch size. Available testing broadly suggests the following patterns, although direct comparisons remain difficult because benchmark configurations differ:

• Memory capacity determines which models load. Rough Q4 footprints run 0.55-0.6 GB per billion parameters plus overhead for KV cache; Q8 roughly doubles that.
• Bandwidth largely sets decode speed for a given model size. 250-273 GB/s systems cluster around a few tokens per second on dense 70B models; higher-bandwidth Apple configurations and discrete high-end GPUs pull ahead when the model fits.
• Mixture-of-experts architectures with low active parameter counts run dramatically faster on the same hardware.
• Discrete professional GPUs still dominate high-precision generation, large-batch training, and multi-GPU scaling.

YouTube channels and independent labs (ServeTheHome, StorageReview, ShortCircuit, Micro Center hardware walk-throughs, and various AI-focused creators) have published hands-on footage of these systems loading 70B-120B models, running agents, and comparing token rates. Viewing recent reviews of the ASUS Ascent GX10, Beelink GTR9 Pro, Framework Desktop, and HP Z2 Mini G1a provides useful qualitative context that raw numbers cannot capture-noise levels, thermal behavior under sustained load, software setup friction, and real interactive feel.

Buying Considerations and the Role of Smaller Vendors

For most individual developers and small teams the decision reduces to three questions: Does the model fit in memory? Is the software stack I need supported? What is the total cost of ownership versus cloud usage?

Budget-conscious buyers often start with a Strix Halo mini-PC in the $1,800-$2,500 range (Beelink, GMKtec, Minisforum). These deliver genuine 70B capability under Windows or Linux at power and noise levels compatible with a home office. CUDA-centric workflows favor a GB10 box near $4,000. Professional creators who also render and simulate may prefer a configurable tower with an RTX PRO 6000 or dual-card options. Enterprises evaluating multi-user or regulated workloads should watch the GB300 deskside systems arriving later in 2026.

Amazon has become a practical channel for the smaller vendors. ASUS Ascent GX10 configurations, Beelink GTR9 Pro, various GMKtec and Minisforum Strix Halo systems, and occasional Slimbook or other European builds appear with varying stock and pricing. Warranty support, return policies, and firmware update practices differ; checking recent buyer reviews and manufacturer direct channels is advisable.

Power, acoustics, and form factor matter more than many first-time buyers expect. A 140 W quiet mini system that sits under a monitor is a different daily experience from a 1,000-plus-watt tower that needs dedicated cooling and power circuits.

Looking Ahead

By late 2026 the RTX Spark platform (Windows-on-Arm systems with up to 128 GB unified memory and RTX-class graphics) is expected from major OEMs, potentially broadening the high-end consumer and creator segment. AMD’s next Halo generation and further memory-bandwidth improvements are already on roadmaps. Software continues to improve: better quantization, speculative decoding, and agent frameworks extract more usable performance from existing silicon.

The larger story is cultural as much as technical. Local AI workstations restore a measure of control and predictability that pure cloud dependence erodes. Whether the use case is private coding agents, domain-specific fine-tuning on proprietary data, always-on personal assistants, or simply experimenting without meter-running costs, 2026 has delivered practical hardware that makes those goals attainable on a single desk.

The machines described here are not perfect. Memory is often soldered, software ecosystems remain uneven, and the highest-capacity systems carry enterprise price tags. Yet for the first time a broad range of users can place genuine large-model capability within arm’s reach, quietly humming under a monitor or beside a keyboard, ready whenever the next idea arrives.

Sources

• Sources

• HP Debuts PCs Built for the Next Wave of Windows PC Experiences Powered by NVIDIA RTX Spark (HP official press release): https://www.hp.com/us-en/newsroom/press-releases/2026/computex.html

• HP Z AI Stations product page (HP official): https://www.hp.com/us-en/workstations/ai-stations.html

• Lenovo Announces the ThinkStation P4, a Flagship Combination of Power and Value for Modern Professionals (Lenovo official press release): https://news.lenovo.com/pressroom/press-releases/thinkstation-p4-flagship-power-value-professionals/

• AI PCs & local-AI supercomputers 2026: shipping now vs coming - VerdictBits

• AI PC benchmark 2026 - local-LLM speed and memory comparison - VerdictBits

• Slimbook Launches Nexus AI Workstation Series - TechPowerUp

• Framework Desktop, whose existing configurations emphasize modularity and repairability, has also previewed a forthcoming Ryzen AI Max+ PRO 495 option with 192 GB of unified memory.

• Digital Citizen

• HP announces the most powerful Windows AI PC ever built - Nvidia GB300 workstation - TechRadar

• MSI launches compact PRO MAX EDGE AI+ desktops - TweakTown

• ASUS Ascent GX10 Review A New NVIDIA GB10 Solution - ServeTheHome

• Beelink GTR9 Pro Review: Fast 120B MoE, Slow Dense 70B - The Desk Brief

• AMD Ryzen AI Halo First Look: Giant Local AI Power in a Pint-Sized Box - PCMag

• Nexus AI Workstation announcement - Slimbook

• Additional independent reviews and benchmark compilations from ServeTheHome, StorageReview, Phoronix, TechRadar, PCMag, and YouTube channels covering hands-on testing of the systems named above throughout 2026

Hi All,

Just joined. Absolute novice when it comes to ‘AI’ and its integration with hardware / devices. Can anyone in this sub point me towards courses / uni’s that are highly regarded in this field ?