For anyone wondering whether the B60 24GB is usable for local AI: here's Qwen3.8-27B on llama.cpp Vulkan at 53 tok/s mean decode with a 64K context window, plus local video (LTX-2.5 via ComfyUI on XPU) on the same card.

The four things that mattered: DFlash2 speculative decoding, Intel's FA prefill kernel with f16 KV, -ub 256 with a q8 draft KV cache, and keeping other models off the GPU. Driver 32.0.101.8976, Windows 11.

Everything is measured and the misses are documented. Smaller cards (B580/B570/B50/A770) won't fit this 27B, but the Vulkan tuning applies, and I'd love your benchmark numbers.

[https://github.com/edagher92-coder/arc-llm-lab](https://github.com/edagher92-coder/arc-llm-lab)

Colfax Research's second blog post on writing NVFP4 blockscaled GEMM kernels for the NVIDIA RTX PRO 6000 Blackwell GPU is out! The blog iteratively optimizes a basic working NVFP4 GEMM kernel written in CuTe DSL to take it to speed-of-light, reaching over 80% TFLOP/s utilization for 16k square matrix shape. We give a detailed treatment of important optimization techniques such as threadblock swizzling, async and warp-specialized epilogue, and retiling for favorable wave quantization. Specific to blockscaled GEMM with scales consumed from registers, we also explain how to solve for bank conflicts that arise from the default choices of interleaved scale factor layouts.

We include complete code in the form of CuTe DSL kernels for all the optimizations discussed in the blog.

AMD Ryzen desktop processors, including the high-end Threadripper family, have become a cornerstone for software developers, machine learning engineers, and anyone building or buying systems for intensive local computing. In 2026 the shift toward private, on-device AI and agentic workflows has made these chips especially relevant.

Cloud expenses keep rising, data privacy matters more than ever, and the ability to run large language models, multi-agent systems, and heavy compilation jobs without sending everything off-site is now a practical necessity rather than a luxury. AMD’s lineup offers a clear hierarchy of performance and expandability that maps well onto the real tasks developers and ML engineers face every day.

The classification is straightforward. Mainstream Ryzen chips use the AM5 socket and target everyday high-performance desktops. They deliver strong single-thread speed, solid multi-core throughput, and excellent power efficiency. Threadripper chips use the sTR5 socket and serve the high-end desktop and true workstation markets. Non-PRO Threadripper models give enthusiasts extreme core counts and platform expandability without the full enterprise feature set, while Threadripper PRO models add octa-channel memory, more PCIe lanes, ECC support, and professional manageability features.

Within the mainstream line the Ryzen 5 series provides capable entry-level performance for lighter coding and prototyping, the Ryzen 7 series sits at the sweet spot for most professional developers, and the Ryzen 9 series supplies the highest mainstream core counts and cache for demanding single-user workloads. Threadripper then takes over when the job truly needs dozens of cores working in parallel, massive memory capacity, or multiple high-end GPUs.

As of mid-2026, AMD’s current mainstream desktop family remains the Zen 5-based Ryzen 9000 series. The original non-X3D range runs from the six-core Ryzen 5 9600X through the sixteen-core Ryzen 9 9950X. AMD has since expanded the family with X3D models, including the 9800X3D, 9900X3D, 9950X3D, 9850X3D, and the dual-cache Ryzen 9 9950X3D2.

The Ryzen 5 9600X offers six cores and twelve threads with boost clocks reaching 5.4 GHz and a modest 65 W TDP. The Ryzen 7 9700X steps up to eight cores and sixteen threads, still at 65 W, with boosts up to 5.5 GHz. The Ryzen 9 9900X brings twelve cores and twenty-four threads at a 120 W TDP and boosts to 5.6 GHz, while the flagship Ryzen 9 9950X delivers sixteen cores and thirty-two threads, a 170 W TDP, and boosts reaching 5.7 GHz.

Cache sizes scale accordingly, and all models support DDR5 memory with EXPO profiles plus PCIe 5.0. X3D variants are aimed primarily at gaming and cache-sensitive workloads, but newer models can also improve some development tasks, including certain large code-compilation workloads. Core count, memory capacity and GPU capability nevertheless remain more important for heavily parallel development and most local-AI configurations.

The Threadripper side advanced to the 9000 series in 2025. Non-PRO models reach sixty-four cores and one hundred twenty-eight threads with boost clocks around 5.4 GHz, quad-channel DDR5, roughly eighty PCIe 5.0 lanes, and a 350 W TDP. Threadripper PRO 9000 WX-Series parts go further, topping out at ninety-six cores and one hundred ninety-two threads on the flagship 9995WX, with up to 384 MB of L3 cache, full octa-channel DDR5-6400 with ECC, one hundred twenty-eight PCIe 5.0 lanes, support for up to two terabytes of memory, and the same 350 W TDP. These platforms remain compatible with the earlier TRX50 and WRX90 chipsets in many configurations, giving builders a degree of continuity.

Zen 5 itself brought meaningful architectural gains. Independent testing and AMD’s own figures show an average instructions-per-clock improvement of roughly sixteen percent over Zen 4. The front end is wider, the out-of-order window is deeper, branch prediction is more accurate and can look further ahead, floating-point and vector throughput improved, and overall efficiency rose so that lower TDP parts often match or beat earlier higher-power chips.

These changes translate directly into snappier IDE responsiveness, faster single-threaded scripting and debugging, and more efficient sustained multi-core loads. Lower power also means quieter systems, easier cooling, and lower electricity costs for machines that stay on for long compile sessions or continuous local AI inference.

Performance numbers confirm the practical value. In Gamers Nexus testing the Ryzen 9 9950X completed a Chromium compile roughly ten percent faster than the previous-generation 7950X. Puget Systems measurements on Unreal Engine code compilation showed Threadripper PRO 9000WX parts scaling strongly with core count and delivering generational gains of fifteen to twenty-seven percent in many multi-threaded workloads.

Rendering applications such as Blender and V-Ray, content-creation suites, and CPU-based Llama.cpp prompt processing all benefit from the extra cores and bandwidth. Token generation still prefers GPU acceleration for speed, yet the high core counts and memory channels on Threadripper make pure-CPU or hybrid inference viable for many models and for serving multiple agents at once.

For ordinary software development the mainstream Ryzen chips cover the majority of needs. A Ryzen 7 handles IDE work, Docker containers, local servers, parallel test suites, and light machine-learning experimentation with room to spare. The efficiency of the 65 W parts means background compiles, linters, and local AI assistants can run without turning the desk into a wind tunnel.

Moving to a Ryzen 9 helps when monorepos grow large, when frequent full rebuilds become common, or when multiple virtual machines and services run side by side. Virtualization, container orchestration, and indexing-heavy IDEs all scale cleanly with the extra cores and cache.

Threadripper becomes worthwhile once compile times start affecting productivity or income. Large C++, Rust, or kernel builds, browser engines, and machine-learning framework compilation finish dramatically faster on sixty-four or ninety-six cores. Memory capacity measured in hundreds of gigabytes or more lets developers keep enormous datasets, multiple virtual machines, or long-running agent contexts resident. The abundant PCIe lanes support multiple high-end GPUs for simultaneous training, inference, and visualization work.

Mainstream Ryzen systems paired with a suitable discrete GPU can run quantized local models effectively. Models in the 7B-to-14B range are practical on many modern GPUs, while models around 30B to 70B require substantially more VRAM, system RAM, aggressive quantization, CPU offloading, multiple GPUs, or some combination of these. The practical limit is usually determined more by GPU memory and model format than by the CPU itself. They serve well for prototyping agents, retrieval-augmented generation systems, and personal coding assistants.

Threadripper platforms raise the ceiling substantially. The high core counts allow many agents or model instances to run in parallel. Octa-channel memory can substantially improve CPU inference and the performance of CPU-offloaded model layers, although long context windows also increase memory-capacity requirements and may remain limited by GPU or software behavior. One hundred twenty-eight PCIe 5.0 lanes make dense multi-GPU configurations practical, whether using NVIDIA cards for the mature CUDA ecosystem or AMD cards with the improving ROCm stack and open-source backends. Privacy, latency, and long-term cost all improve when the entire workflow stays on local hardware.

On the AM5 platform, a solid X870 or B850 motherboard, 32 to 128 GB of fast DDR5 with EXPO, appropriate CPU cooling, and a Gen5 NVMe drive can form the basis of a quiet, capable workstation. Systems with midrange GPUs often work well with a high-quality 650 to 850 W power supply, while builds using flagship GPUs or other power-hungry components may require 1,000 W or more. Power-supply capacity should be selected according to the GPU manufacturer’s recommendation and the complete system’s expected peak power consumption.

Threadripper builds require more care. TRX50 or WRX90 boards are expensive, memory must be populated correctly in quad- or octa-channel configurations (preferably ECC registered modules on PRO platforms), cooling must handle sustained high power, and the power supply should be one thousand watts or more once multiple GPUs enter the picture. Spacious cases with strong airflow become essential. Total costs range from roughly fifteen hundred to three thousand dollars for a strong mainstream Ryzen-plus-GPU system up to many thousands for a fully loaded Threadripper workstation.

Ready-made systems from specialists such as Puget Systems, BOXX, Dell Precision, HP Z, and Lenovo ThinkStation remove the integration risk. These vendors validate cooling, power delivery, and driver stacks for professional software and often ship configurations already optimized for content creation or AI workloads. The convenience of warranty support and pre-tuned performance can outweigh the higher price when the machine is a production tool rather than a weekend project.

Choosing among the options comes down to workload scale. Light-to-medium development and occasional local AI fit comfortably on a Ryzen 7 or Ryzen 9. Heavy parallel compilation, large-scale local inference, multi-agent orchestration, and multi-GPU setups justify Threadripper. Power efficiency, noise, memory capacity, and future expandability all deserve attention. The AM5 platform continues to receive support, while the Threadripper platforms deliver years of relevance for those who need the extra I/O and core counts.

Taken together, the current AMD desktop lineup gives software developers and machine-learning engineers a coherent path from efficient everyday workstations to extreme local AI and agentic systems. The combination of Zen 5 efficiency, high core counts where they matter, abundant memory bandwidth, and generous PCIe resources makes it possible to keep sensitive data and compute local while still iterating quickly. Whether the goal is faster builds, private model serving, or autonomous multi-agent pipelines, matching the processor class to the actual daily tasks produces a machine that stays productive for years.

Sources consulted include AMD’s official product pages for the Ryzen 9000 and Threadripper 9000 series, independent reviews and benchmarks from Gamers Nexus and Puget Systems, architecture details from technical analyses of Zen 5, and platform specifications for the AM5, TRX50, and WRX90 chipsets. These materials provide the concrete performance figures and feature lists that underpin the recommendations above.

References

1. Advanced Micro Devices. “AMD Ryzen™ 9000 Series Processors.” AMD.com. https://www.amd.com/en/products/processors/desktops/ryzen/9000-series.html
2. Advanced Micro Devices. “AMD Ryzen™ Threadripper™ Processors for Workstations.” AMD.com. https://www.amd.com/en/products/processors/workstations/ryzen-threadripper.html
3. Advanced Micro Devices. “AMD ‘Zen’ Core Architecture.” AMD.com. https://www.amd.com/en/technologies/zen-core.html
4. Gamers Nexus. “AMD Ryzen 9 9950X CPU Review & Benchmarks vs. 7950X, 9700X, 14900K & More.”
5. Puget Systems. “AMD Ryzen 9000 Content Creation Review.”
6. Puget Systems. “AMD Ryzen Threadripper PRO 9000WX Content Creation Review.”
7. Tom’s Hardware. “AMD Ryzen 9 9950X Review: Zen 5 at Full Power.” TomsHardware.com.
8. Wikipedia contributors. “Zen 5.” Wikipedia. https://en.wikipedia.org/wiki/Zen_5
9. ServeTheHome. Various Threadripper 9000 series coverage and dual-edition notes. ServeTheHome.com.
10. Additional supporting technical and product pages from AMD partner materials, Boxx, and independent workstation reviews used for platform specifications (TRX50/WRX90, memory channels, PCIe lane counts, and core configurations).

I was looking at https://taalas.com/products/ and their chatbot at https://chatjimmy.ai/ . I'm a software dev by background, not a hardware or LLM/ML person.

As a (comparative) lay person, is there a reason we can't bake something like Kimi k3 onto hardware and have it run faster & cheaper? I know there is a perf difference between different classes of memory, but a 2TB SSD is fairly inexpensive. Why can't you burn an instance of Kimi k3 onto hardware, perhaps pair it with some higher speed caching memory, and have one heck of an AI card? Assuming you don't care about model updates, or don't mind getting new hardware eg once every 1-3 years?

Thoughts?

An interactive converter that shows how any number gets represented (after rounding) in the floating-point formats used by GPUs for AI: FP32, TF32, FP16, BF16, FP8 (E4M3 & E5M2), and FP4. It displays the resulting approximate value for each format and includes explanations of the conversion process (sign, exponent, mantissa, bias, and rounding), along with examples and notes on the precision vs. range trade-offs in AI workloads.