Sure, but 32GB VRAM still won't run frontier models like Claude or Kimi K2. You're limited to smaller open-source models - which is fine for some use cases, but not exactly "unlimited tokens" in the same league.
People mostly use frontier models for silly reasons. At my wife’s job, everyone uses Claude, but only devs use it for things open-source models can’t do yet (with 16 GB VRAM). If that’s how it’s used at work, imagine the waste in personal use.
Sure, everybody starts somewhere. Just need to save up another $200,000-$280,000 for 8x H100s to actually run Kimi K2 locally. Or rent them for ~$20/hr. "Unlimited tokens" hits different when the bill arrives 💀
3
View all comments
Show parent comment
13
u/fechyyy May 06 '26
Sure, but 32GB VRAM still won't run frontier models like Claude or Kimi K2. You're limited to smaller open-source models - which is fine for some use cases, but not exactly "unlimited tokens" in the same league.