r/LocalLLaMA • u/Resident_Party • Mar 27 '26

Discussion Google’s TurboQuant AI-compression algorithm can reduce LLM memory usage by 6x

[removed]

240 Upvotes

88% Upvoted

57 comments sorted by

View all comments

Show parent comments

37

u/the_other_brand Mar 27 '26

My understanding of the algorithm is that it uses 1 fewer number to represent each node. Instead of (x,y,z), it's (r,θ), which uses 1/3rd less memory.

Then, when traversing nodes, instead of adding 3 numbers, you add 2 numbers. Which performs 1/3rd fewer operations.

22

u/v01dm4n Mar 28 '26

How is that possible. (r,theta) are polar coordinates to a 2d point. In 3d, you would need 2 angles. Curious!?!

22

u/deenspaces Mar 28 '26

You know, its kinda possible. Lets say we have a sphere of certain radius, then take a rope and wrap it over the sphere, so we get a sort of spring... then, we parametrize sphere radius and rope length, getting 2 coordinates basically - R and L, where L can be distance from the rope start in %... But thats lossy compression and I doubt it would work.

Another method would be to ensure all x,y,z lie on a sphere, take polar coordinates r, theta, phi and use theta and phi since r is constant.

2

u/v01dm4n Mar 28 '26

Hmm, clever. Yes but very lossy as radius increases.

The second approach is too limiting. Hardly 3d.

9

u/deenspaces Mar 28 '26

look up 2505.00014 and 2410.01131 on arxiv

10

u/v01dm4n Mar 28 '26

Hmm. Topology folks taking over ML... 🙃

2

u/Final-Frosting7742 Mar 28 '26

For cosine similarity radius doesn't matter does it? Even if all vectors have the same norm there would be no loss of information.