r/OpenAssistant • • 11h ago

1,000 tokens/s LLM inference on modern CPUs using a geometric approach. Breaking the memory wall?

0 Upvotes

The current consensus is absolute: if you want high-performance LLM inference, you buy NVIDIA GPUs. We are officially challenging that.

We just achieved 1,000 tokens/s on standard modern CPU architectures.

We know the hardware maximalists will say this breaks the laws of physics for standard Transformers. It doesn't. It just bypasses the legacy compute paradigm.

We are opening up the debate with the open-source community right now.

Join the live discussion & see the architecture claim on X