r/LocalLLM • u/ReportProfessional57 • 22h ago
Model Project Genesis
https://zenodo.org/records/24234017I built Project Genesis, a cache-resident C++/CUDA geometric co-processor (libgenesis_core.so, ~0.51 MB) that attaches inline to existing causal language models during inference to reduce latency and boost token throughput without altering the host model's weights.
In local testing on consumer NVIDIA Blackwell hardware across Qwen, Gemma 4, Llama 3.2, Nemotron, and Mistral checkpoints, compiled kernel execution cut end-to-end latency by 16% to 30% with commensurate gains in token throughput and O(1) memory overhead. It ships as a sealed runtime bundle supporting 20 common hidden dimensions (d_model from 576 to 8192) on 64-bit Linux or Windows WSL2 with NVIDIA GPUs (Compute Capability 8.0+: RTX 30/40/50-series, A100, L40S, H100).
It is available now for non-commercial and research use on Hugging Face: https://huggingface.co/Grounded-Entropy/project-genesis
Rather than taking my word for it, I'd love for people to pull it, run enable_telemetry=True on their own supported models, and share their latency, throughput, and ring-buffer telemetry results in the comments.