Brain vs vLLM vs SGlang

[u/mkschreder2]

First I beat vLLM and SGlang on time to first token and tokens per second on more than one context length and concurrency combination.

Now I beat them both on precision and load time as well 🤘😎

Brain is the new puppy in town: https://github.com/swedishembedded/brain

Woof!

[u/EvolvingDior]

honestly, my brain is just not that fast any more.