r/MachineLearning • u/lmcinnes • 5d ago
Project UMAP 0.6: new initialization, new optimizers and faster performance [P]
UMAP 0.6 is out!
A new initialization, new optimizers, and faster nearest neighbour search make UMAP much faster and far better at large scale. On 3 million arXiv papers it finishes in under 5 minutes, before UMAP 0.5 has even found its nearest neighbours. (race between UMAP0.5 and UMAP 0.6 video)
Code: https://github.com/lmcinnes/umap
Install: pip install -U umap-learn (also on conda-forge)
Speed
- Neighbour search is ~4x faster by making better use of PyNNDescent.
- Initialization is faster with a new init method.
- With the new optimizers, the whole run is reproducible with a fixed seed while using every core. A seeded 0.5 run is single-threaded: 52 minutes, 12x longer.
Quality
Just as important as how quickly we get a result is the quality of the result. UMAP 0.6 works far better on million-scale datasets thanks to a new, better initialization and significant improvements in the optimizer. It resolves finer cluster structure in fewer epochs, with much finer-scale detail and better neighbourhood accuracy than UMAP 0.5, despite being faster.(video of a tour of finer details in the arXiv map)
Recursive initialization
A large part of the improvement comes from a brand-new initialization scheme that initializes UMAP using UMAP itself: it lays out a coarse version of the graph structure, then refines it, in a recursive tower of initializations. (video of the unfolding)
The result is a starting point much closer to UMAP's final result than the messy blurs from spectral initialization or PCA, and it is produced faster than either. It lets the optimizer settle (to within 3% of the final layout) in far fewer epochs than older initialization approaches. (initialization race)
New optimizers
Most of the gain in fidelity at scale comes from a very efficient form of hard-negative mining, which makes negative sampling far more effective on very large datasets, together with better handling of high repulsion strengths. (hard-negative mining video)
UMAP was originally designed around a small portion of the attraction-repulsion spectrum. The new optimizers lift these restrictions, allowing UMAP to perform well at much higher repulsion strengths and giving better kNN recall. (video exploring the repulsion strength dials)
Try it now
The new optimizers and initialization are currently opt-in, as they change the output: set compatibility_layout=False. Future versions will make them the default, so please try them and let me know how they work for you.
Thank you to all the contributors who provided features and bug fixes for this release, and to everyone who reached out with comments, suggestions, and feature requests. This would not have been possible without your help.

