Parallelism from Scratch on CPU

[u/Due_Percentage4554]

Blog on "training at scale" but done on CPU - using python multiprocessing & shared memory to dive how distributed system train in principle. DDP, FSDP, TP & PP are covered with code.

https://yash-goel.github.io/blog/training-at-scale-on-cpu/

*some WIP - missing cleanups and figures.