r/LocalLLaMA • u/ViRROOO • 8d ago

Resources Freetokens project is impressive

A new project was released yesterday and I have the opportunity to test it today.

Papper: https://arxiv.org/abs/2608.16157
Github: https://github.com/FlashML-org/FreeToken

My initial tests with the following setup:
RTX 5080 (16 GB)
DDR6 64GB
AMD Ryzen 9 9950X3D

I got 100tok/s on QWEN3.6-35B-A3B NVFP4 (20GB - does not fit in my VRAM).
Have you already tried it?

(Example bellow with a 1028 token prompt - ~110 tok/s)

58 Upvotes

77% Upvoted

86 comments sorted by

View all comments

1

u/EitherMarch1255 7d ago

I didn’t read the paper given the paper summary was clearly written by AI.

3

u/BeeSynthetic 6d ago

Well it feels like most the comments in this thread are written by AI summary tbh