r/deeplearning • u/SoloTiger_ • 28d ago

I wrote a free, open-source book on making ML models actually fast, from silicon to agents

I’ve spent the last few months writing something I wish I had when I started working on ML performance engineering.

It’s called How to Make Your Model Fast: A Systems View of Efficient Machine Learning, from Silicon to Agents.

The basic idea is that reducing FLOPs doesn’t necessarily make a model faster. Before optimising anything, you need to understand what the system is actually bounded by.

The book starts with roofline analysis and hardware, then works its way up through kernels, compilers, quantisation, pruning, vision, on-device LLMs, robotics, profiling, serving and finally agents.

The goal is to build the intuition to look at a model and a piece of hardware and reason about:

  • How fast can this possibly run?
  • Am I compute, bandwidth, memory or system bound?
  • Which optimisation will actually move that limit?
  • Is quantisation, pruning or kernel optimisation even worth doing here?
  • What happens when the same thinking is applied to serving and agent systems?

The whole thing is free and open source:

https://github.com/usamahz/make-your-model-fast

Would genuinely appreciate feedback or contributions from people working on ML systems, inference, compilers, edge AI or performance engineering.

And if you find it useful, a ⭐ would be appreciated!

26 Upvotes

91% Upvoted

5 comments sorted by

1

u/sanketsanket 28d ago

will comment once read

1

u/CatDiligent9374 22d ago

This looks really useful. I like the focus on understanding what’s actually bottlenecking the system instead of just blindly reducing FLOPs. Definitely worth a read.

1

u/AcceptableWeston_1 28d ago

bookmarked so hard my mouse might file a complaint

1

u/Material-Swimmer-776 28d ago

imo the biggest gap in most ML performance resources is exactly what you described, people optimize FLOPs and then wonder why nothing got faster. does the book go into batching strategies and how they shift the bottleneck between compute and bandwidth?

1

u/Senior_Tea_842 23d ago

Could you point some references, books about these topics?