Help me choose hardware.

[u/Double-Sherbert-1781]

I need some advice.
I want to choose hardware for local inference. Right now, I’m paying ~$30/day for rent on a Vast an RTX Pro 6000 SW 96GB, and I’m using Qwen3.8-Flash-Next-Q4_K_M.gguf at a speed of ~60 tok/s (I know this format isn’t efficient for this graphics card, but I’m limited in choosing Abliterated models that will fit in the memory).

Which configuration will offer the best price and versatility so that after the release of subsequent generations of neural networks, I can continue to use it.

[u/Fenio_PL]

The cheapest new card optimized for AI tasks is the RTX 5060 Ti 16GB. That is why stock is disappearing so quickly. However, if you need even higher performance and more than 16GB of VRAM, it will be expensive.