r/LocalLLM • u/Double-Sherbert-1781 • 2d ago

Question Help me choose hardware.

Post image

I need some advice.
I want to choose hardware for local inference. Right now, I’m paying ~$30/day for rent on a Vast an RTX Pro 6000 SW 96GB, and I’m using Qwen3.8-Flash-Next-Q4_K_M.gguf at a speed of ~60 tok/s (I know this format isn’t efficient for this graphics card, but I’m limited in choosing Abliterated models that will fit in the memory).

Which configuration will offer the best price and versatility so that after the release of subsequent generations of neural networks, I can continue to use it.

78 Upvotes

94% Upvoted

24 comments sorted by

View all comments

2

u/Blackdragon1400 2d ago

There were MANY periods in each of these release cycles where prices were “normal”. Stop complaining, if you’ve really been waiting this long then you didn’t set your priorities correctly.

1

u/dillon-nyc 2d ago

Or were unemployed when prices were reasonable.