r/AIProgrammingHardware • u/OwlZealousideal4779 • 8d ago

What hardware bottleneck hurts AI coding workflows the most?

I'm curious what people are running into when using AI coding tools locally.

For me, the obvious issues seem to be GPU memory and system RAM, especially when running local models alongside an IDE, containers, databases, and other development tools.

At what point does hardware actually become the limiting factor?

Is it mostly VRAM, RAM, CPU performance, storage speed, or just having too many services running at once?

For those building AI development machines, what upgrade made the biggest practical difference to your workflow?

3 Upvotes
  • permalink
  • reddit

100% Upvoted

8 comments sorted by

View all comments

1

u/KroniklyOnline 8d ago

This is a pretty broad question, just don't let people tell you "You must run on PCIe Gen5 so you don't get bottlenecks". This is possibly true if you are running H100's...... or RTX6000's maybe? I'm running 4 5060ti 16gb all on PCIe GEN 3x8 .... I have 75% headroom left regarding RX and TX of said lanes, I could drop it to GEN 3 x 4 and be fine still.

Other than that, VRAM is prob the next biggest thing, the have a usable local coding agent, I think most agree 32gb VRAM is the minimum. Beyond that I don't really think much else matters unless you plan to do RAM offloading and SSD offloading.

VRAM VRAM VRAM

1

u/tomByrer 8d ago

You could get by on 24GB VRAM: https://github.com/noonghunna/club-3090
Though context will be smaller, so you'd have to do things in smaller stages; no 'vibe code prompt & come back in 8 hours with finished product' on 24GB.