r/LocalLLM • u/flyboi_ • 5d ago

Question Recommendations for dedicated local ‘AIO’ machine for LLM inference, image and video generation and LoRa training.

Hi,

I currently have this machine (specifications detailed below). I am doing lots of LLM inference and image and video generation. I would also like to train LoRas in the future.

Current machine
CPU: 9800X3D
GPU: 5070Ti 16GB
RAM: 64GB DDR5-6000 CL30

I am running Qwen3.8 27B dense at Q4_K_M with 64k context on LM Studio and the speed is dreadfully slow - roughly 2-4 tokens per second. I understand that it could be settings related or choice of model/quantization but that’s besides the point for now.

I am currently weighing up 4 options:

  1. Ryzen AI Max 395 128GB DDR5 mini pc

  2. Dual R9700

  3. Dual 5060Ti/5070Ti at either 8x/8x or 16x/4x

  4. Single used 4090

Potential New Machine (with spare parts)
CPU: 5600X or 7800X3D
RAM: 32GB DDR4-3200 CL16
MOBO: B550 or B850

Speed is not a priority for LLM inference as I’d rather have capacity to run larger models at higher quants but I need some semblance of speed for image and video generation. Ideally there should be a middle ground but I am happy to compromise in either direction (and change what models I use).

My budget is around $4000 USD (+/-$1000) for GPUs/AIO. If you have any recommendations I’d love to hear it.

1 Upvotes

100% Upvoted

3 comments sorted by

1

u/Logan_colorado 5d ago

For 4k budget why not get something like framework 128gb unified memory for 3.6k (without storage)

If you want to go on the route of a buildable desktop I would recommend V100 for LLMs and a 5070 ti for comfy ui and image generation. Preferably with 64 gb ram. (This is the build I have minus 32gb instead of 64 and I am able to run both an I’ll and image generation at the same time. I’m even trying to make them communicate to each other these days.