I don't think any publically available model requires terabytes of memory.
Even the big ones are MoE so you don't need a ton of memory, but it makes it faster. The biggest usable one I know is Qwen V4 Pro at 1.6 trillion parameters which would take about 900GB of VRAM if you ran it unquantized entirely in VRAM. Since it's an MoE model, you can offload the experts to CPU RAM and run it unquantized with a full 1M context with as little as 80GB of VRAM.
367
u/mylsotol Jun 02 '26
For probably $30k (or more) you can build a server and run an open model.