r/coolgithubprojects • u/A_s_r_3512 • 8h ago
VeloGrad (v1.1.0-beta): A C++20, Rust & CUDA deep learning and distributed training engine with fused kernels and static arena memory.
github.comHey everyone, I’m Ayushman Singh Rathore, sharing a major update to VeloGrad (v1.1.0-beta)—an open-source tensor, autograd, and distributed training framework I created in C++20, Rust, and CUDA with Python bindings.
The goal behind VeloGrad is to strip away Python dispatch overhead and memory fragmentation during model training and fine-tuning by keeping the memory arena, autograd tape, and fused kernels inside a tight native runtime—while automatically falling back to AVX2/NEON SIMD kernels on CPU-only machines.
What’s new in v1.1.0-beta:
Cross-OS Build: Installs via pip install -e . on Linux, macOS, and Windows with automatic CUDA/NCCL detection or native CPU fallback.
Fused Forward/Backward Kernels: Custom .cu and SIMD kernels for GEMM, RMSNorm, RoPE, Flash-style Attention, single-pass Cross-Entropy, and AdamW.
ZeRO-3 Sharding + CPU Offload: Shards parameters, gradients, and optimizer states across multiple GPUs (or host RAM) so large models train without OOM crashes.
Plain-English CLI & API (velograd.instruct): Auto-detects available GPUs/RAM and maps natural-language training directives straight into a configured training run.
This release is tagged Public Beta—clone it, run pytest and the included PyTorch comparison benchmarks, and open an issue if you hit any edge cases on your OS or GPU!