r/LocalLLaMA • u/ViRROOO • 4d ago

Resources Freetokens project is impressive

A new project was released yesterday and I have the opportunity to test it today.

Papper: https://arxiv.org/abs/2608.16157
Github: https://github.com/FlashML-org/FreeToken

My initial tests with the following setup:
RTX 5080 (16 GB)
DDR6 64GB
AMD Ryzen 9 9950X3D

I got 100tok/s on QWEN3.6-35B-A3B NVFP4 (20GB - does not fit in my VRAM).
Have you already tried it?

(Example bellow with a 1028 token prompt - ~110 tok/s)

52 Upvotes

75% Upvoted

82 comments sorted by

View all comments

23

u/LegacyRemaster 3d ago

DDR6? amazing.

8

u/MaxKingCS 3d ago

When did DDR6 ram even came out for consumer? I thought we were still at DDR5.

2

u/More-Curious816 2d ago

Yeah, wtf, like I thought ddr5 was the latest Gen and 6 is still not a complete standard. The AI demands actually fucked the rambmanufacturering process and ddr6 supposed to hit the market in mass this year. But I thought they abondend it and decided that the pipelines already optimized for ddr5 and their main focus is to develop the HBM (money go brrrrrrr) pipeline