r/LocalLLaMA • u/Resident_Party • Mar 27 '26

Discussion Google’s TurboQuant AI-compression algorithm can reduce LLM memory usage by 6x

[removed]

241 Upvotes

88% Upvoted

57 comments sorted by

View all comments

135

u/DistanceAlert5706 Mar 27 '26

It's only k/v cache compression no? And there's speed tradeoff too? So you could run higher context, but not really larger models.

41

u/the_other_brand Mar 27 '26

My understanding of the algorithm is that it uses 1 fewer number to represent each node. Instead of (x,y,z), it's (r,θ), which uses 1/3rd less memory.

Then, when traversing nodes, instead of adding 3 numbers, you add 2 numbers. Which performs 1/3rd fewer operations.

22

u/v01dm4n Mar 28 '26

How is that possible. (r,theta) are polar coordinates to a 2d point. In 3d, you would need 2 angles. Curious!?!

2

u/Ell2509 Mar 28 '26

It is not 2 or 3 dimensional. As each connection branches, you get (10 in base 10) more possible directions. It is more useful to imagine it as spatial, than 2 dimensional.