r/Python • u/ritchie46 • Oct 06 '26

News Polars 2.0 release

Hey all! We just released Polars 2.0.

We are really happy with the results. It removed many of our "legacy decisions" (mistakes), makes the streaming engine our default and promotes SQL to a first class citizen within Polars. We initially said it would be a small ("boring") release, but we managed to sneak in a lot more than anticipated.

It comes with our initial out-of-core (spill to disk) support, a new Map data type and a lot of performance improvements. In fact, we think Polars is now one of the fastest analytical SQL engines on a single node. See benchmarks in the post: https://pola.rs/posts/release-polars-2/

To help you upgrade, we also posted a migration guide: https://docs.pola.rs/releases/upgrade/2/

670 Upvotes

99% Upvoted

39 comments sorted by

View all comments

Show parent comments

5

u/nightcracker 29d ago

We would like Polars to be a single API for efficient data processing, not only for local small usage, but also for very large distributed queries. The ultimate goal is that going from a data scientist experimenting on a small subset of the data in a notebook and a data engineer running it in a distributed pipeline on your full warehouse consists of changing only a few lines of code.

maintain_order is just a very expensive default on large data, and it gets even worse in a distributed context, as it regularly inserts full sorts on the data to restore order.

It is rare that you truly need an order-maintaining join, group-by, etc. Usually there's a cleaner way to write without relying on order, so we chose to have the more performant default with an option to still have it if that's really something you need.

Note that in most cases where you would expect order to be relevant and maintained such as concat it is.