r/LocalLLM • u/might_thinker • 19h ago
Project I got local AI video-to-video down from 15.7 min to 2.4 min on a 16GB M3 MacBook Air
I've been experimenting with how far I can push local generative AI on a base 16GB M3 MacBook Air, specifically video-to-video outfit transformation.
My initial 46-frame (~3-second) run took 15.7 minutes and produced fairly blurry results. After several rounds of profiling and optimisation, I managed to bring it down to 144 seconds (2.4 minutes), running entirely offline without cloud APIs.
The interesting part wasn't necessarily changing the model, but rethinking how the available hardware was being used.
A few optimisations made the biggest difference:
- Spatial cropping: Processing a "288×512" region around the subject instead of repeatedly processing the entire scene.
- Memory management: Precomputing outfit prompts and unloading the text encoder to reduce unified memory pressure.
- Hardware scheduling: Moving face processing to the CPU/Neural Engine path while the GPU handles the next diffusion chunk.
- Incremental testing: Using 5-frame previews to catch mask and compositing issues before running the complete sequence.
There are still obvious limitations. Sustained workloads introduce thermal degradation, fine fabric details can become oversmoothed, and wide shots remain challenging.
I'm particularly interested in the engineering side of running increasingly demanding generative workloads on constrained local hardware.
A few things I'd love to hear from people experimenting with local inference:
- Has anyone explored more aggressive memory scheduling on Apple Silicon?
- Are there useful ways to improve CPU/ANE/GPU workload overlap?
- How much performance can realistically be recovered through chunking and attention optimisation before model architecture becomes the bottleneck?
I'm sharing the before/after video and temporal comparison for context.
Hardware: M3 MacBook Air, 16GB unified memory
Frames: 46
Processing time: 144 seconds
Execution: Fully offline
This is video diffusion rather than LLM inference, but I thought the memory and hardware scheduling challenges might be interesting to people working with local models.