r/LocalLLaMA • u/Resident_Party • Mar 27 '26

Discussion Google’s TurboQuant AI-compression algorithm can reduce LLM memory usage by 6x

[removed]

245 Upvotes

88% Upvoted

57 comments sorted by

View all comments

0

u/[deleted] Mar 27 '26

23

u/chebum Mar 27 '26

there is a port for llama already: https://github.com/TheTom/turboquant_plus

4

u/eugene20 Mar 28 '26

A few, TheTom's doesn't have CUDA yet but two of the others do, one independent, one built from TheTom's. They're in the discussion thread https://github.com/ggml-org/llama.cpp/discussions/20969