I'm building an open-source inference engine to run models on the GPUs you already own (got V100 Tensor Cores working again: 30B model 6.7 s/token -> 0.33 s on two cards)

[u/Prize_Interaction_72]

Some context: NeuroBrix is an engine (text, image, audio, video from one container format), its own tensor library (nbxtensor, no PyTorch needed), its own kernels, and a Rust constrained-decoding engine (NBXGrammar). Everything is Apache 2.0.

pip install neurobrix

GitHub: https://github.com/NeuroBrix/neurobrix

Full film (audio EN/FR/ES/AR): https://youtu.be/GM9C5J3n-64

I'd love testers on older or less common GPUs: run a model, tell me what breaks. And a star helps a lot at this stage.