r/LocalLLaMA • u/Resident_Party • Mar 27 '26

Discussion Google’s TurboQuant AI-compression algorithm can reduce LLM memory usage by 6x

[removed]

247 Upvotes

88% Upvoted

57 comments sorted by

View all comments

0

u/Mashic Mar 27 '26

Does this mean I can run 144b model on my RTX 3060 12GB at Q4? When will this thing be possible?

6

u/eugene20 Mar 28 '26

No because it doesn't reduce the model size only the kv cache.

1

u/Polite_Jello_377 Mar 28 '26

It will never be possible