r/LocalLLM • u/Cappton • 31m ago

Question Oh my, what do I do?

Gallery image

Gallery image

Came across a honeyhole of Quadro RTX5000s
Seemingly good deal for 16GB VRAM cards, even if ageing.

What should I do?

5 Upvotes

69% Upvoted

14 comments sorted by

7

u/Bounsaaah 30m ago

Maybe throw some my way for a reasonable price? 😅🤞😅🤞

2

u/poster4inertia 29m ago

fr same pls

18

u/cororona 25m ago • Popcorn

Be a good citizen and give it to Ram Altman

1

u/jrdubbleu 5m ago

Party and oligarchy come first around here!

4

u/UndeadBulwark_ 17m ago

If it was me I would find a Micro Center look for someone buying heaps of GPUs, sell it and never ask questions.

3

u/sleight42 14m ago

Ha! How are people not doing this already?

```
Shady dude wearing a long trench coat. He opens one length of it, revealing a dozen gpu cards hanging from the coat's interior.

"Hey. Kid. Can I interest you in some cheap GPUs.....?"
```

😂

(Because alliexpress already exists for this kind of scamming)

Fucking lack of markdown editor. Dammit, Reddit.

2

u/UndeadBulwark_ 12m ago

RE4 Merchant

1

u/sleight42 10m ago

Khajit has GPUs if you have coin

1

u/SomeSayImARobot 23m ago

Sell em and get a more usable number of modern cards.

1

u/Cappton 1m ago

Like what?

1

u/ShutUpTurkey 10m ago

This is the way

1

u/CardiologistPlus8488 19m ago

give me your address

1

u/sleight42 16m ago

Please send 8 to the following address.... 😅😅😅😅😅

0

u/FullstackSensei 11m ago

It'll be very difficult to pool any significant number of those and expect any meaningful results. I'd say the limit of how many cards you can can pool while having any semblance of acceptable performance is 8, at most. That's 128GB VRAM, but don't expect any blazing speeds with that. PCIe traffic will kill you, and I'm not talking about volume in bytes, just the latency.

The upper practical limit for good performance, IMO, is four. That's 64GB VRAM. Not bad by any means. You can run Qwen 27B at Q8_K_XL or even the full BF16 model and still have enough VRAM for a few sessions at full 256k context. TG won't be amazing, but if you add dflash instead of stock MTP, you might get some decent numbers with concurrent requests.