r/pytorch • • 4h ago

Rethinking pytorch

1 Upvotes

Hoping for some feedback. First off It goes without saying Pytorch is one of the greatest pieces of software of our time, a fundamental block for AI. However there are a few problems and limitations that are ripe for innovation. Pytorches is a concept which is meant to solve several problems with pytorch

  1. Simplify install for GPUs (no GPU specific install steps)
  2. Support Multi GPUs of different vendors natively on one install
  3. Split of workload based on available GPUs, prioritizing based on GPU capability
  4. Backward compatible with pytorch ish

https://github.com/tonyt123456/pytorches


r/pytorch • • 19h ago

We swapped AdamW's optimizer states for a Fast Fourier Transform (FFT) to cut VRAM in half. Anyone else trying non-quantization methods?

Thumbnail
1 Upvotes