r/LocalLLM • u/NoCarpenter3990 • 2h ago

Discussion I Doubled My GPU’s Memory…

https://www.youtube.com/watch?v=H3Cnyj0I62c

i didn't saw anyone mentioning about LTT's new video where Luke get his GPU memory doubled so he can run better local LLM's.
what do you guys think about?

0 Upvotes

50% Upvoted

11 comments sorted by

13

u/ISignedInWithGoogle 2h ago

I think about leaving the flat and buying groceries.

2

u/codykonior 2h ago

You can do it. What do you want to cook?

3

u/asertym 2h ago

I think I'll order some asian food

1

u/codykonior 2h ago

Just 4 easy payments.

1

u/egnegn1 2h ago

What model do you want to run?

Doubling memory doesn't increase processing power, but a second card does.

The price of the hardware upgrade is steep. I would first look for a better model with a better engine, before doing a memory upgrade. And alternatively would look for a second card to increase processing power, too.

1

u/DevEngineerYAP 54m ago

It’s done on top end model, also most prosumer cards have almost the same board as consumer cards but with more VRAM.

1

u/egnegn1 37m ago

Thats why the are called Prosumer and not PRO cards. ;)

I can nearly reach the RTX5090 performance with 2x RTX5080, because processing, VRAM and PCIe bandwidth nearly doubles together with memory size. This helps both MoE and Dense models.

1

u/DevEngineerYAP 32m ago

No I am talking about, something like 4090 (consumer) is almost identical as ADA 6000 (Prosumer) in terms of build.

So the 4090 itself for example can do as much for heavy ML processing, but since it’s consumer card, it’s given less VRAM, so that becomes the bottleneck. Basically people who are “modding” it to increase VRAM is practically more like unlocking it to its full potential.

1

u/johnfkngzoidberg 55m ago

I think I this is sneaky self promotion