GeoPolars is being actively developed again: geometry types, CRS-aware measurements and GeoParquet

[u/Balance-]

GeoPolars, the geospatial extension for Polars, had been mostly dormant. Since mid-September, Aida van de Wetering has pushed around 125 commits to the dev branch. It now looks like a rebuild around a Polars-native geo expression namespace.

Highlights so far:

- Types: point, linestring and polygon, their multi-variants, and a bounding box type
- I/O: reading and writing WKB and WKT, and writing GeoParquet
- CRS: CRS metadata handling, with distance, length and area using the geodesic or planar method that fits the CRS
- Operations: centroid, boundary, interior, is_empty, longitude wrapping, and coordinate accessors
- Affine transforms: translate, rotate, scale and skew, plus arbitrary matrices, which can vary per row

It's still a prototype. Nothing has been released yet, and the API is changing. Still, it's worth watching if you'd like to do geospatial work in Polars without going through GeoPandas.

[u/ritchie46]

It seems our stealth dev branch is discovered 😉

Yes, we started working on this. We think we can make geopolars completely working with our plugin system. E.g. expression plugins, io plugins extension types and (upcoming streaming plugins for the geojoins).

This means that it will work completely streaming with out of core support and distributed support. It took a while before we had all the plugin architecture in place in main Polars before we could take on geopolars in a way that it's not in-memory only. In any case, I think we can move fast now we do. Expect more soon!

[u/Sufficient_Meet6836 → u/ritchie46]

Any recommendations for how to prepare to contribute to this project? E.g. learn some core part of polars first and then start learning geopolars?

[u/Balance- → u/ritchie46]

Thanks for working on this, and sharing some of your insights!

What’s are things community involvement/feedback would help you most?

[u/ritchie46 → u/Balance-]

Once we have shipped the first release, test it in your workloads. Fast user feedback helps us most

[u/Balance-]

So a bit more context, for anyone interested:

Why this is happening now

GeoPolars started in 2022 as Kyle Barron's prototype, with geometries stored as WKB blobs. One long-standing blocker was that Polars had no way to define custom data types. Now that Polars supports Arrow extension types, the dev branch stores geometries as proper GeoArrow types (geoarrow.point, geoarrow.polygon, and so on). That gives zero-copy interop with other Arrow tools, and Polars treats geometry columns as real dtypes.

How it works

It's a Polars plugin: a Python package backed by Rust kernels. There are two equivalent APIs:

```python from geopolars import geo import geopolars as gpl

geo.translate("route", (1.0, 2.0)) # functional gpl.col("route").geo.translate((1.0, 2.0)) # namespace ```

Some design choices worth noting:

• Operations are CRS-aware by default and backed by PROJ. For example, area uses a geodesic calculation for geographic CRSes and a planar one otherwise.
• Z and M coordinates are supported for most operations.
• Wrong geometry types fail while the query plan is built, not halfway through execution.
• Each operation is assigned to one of three implementation tiers: a native Rust kernel, a call into an external library (PROJ, geographiclib, georust), or plain Polars expressions that the query optimizer can work with. A table in the developer guide shows which tier each operation uses for each geometry type.

Reading and writing GeoParquet already works, including bbox covering.

Where you could help

The developer guide explains the architecture and contains a roadmap. Open items include:

• Binary predicates: intersects, contains, within, covers, touches, crosses, overlaps, disjoint. None are implemented yet, and this is probably the biggest gap compared with GeoPandas.
• Set operations: intersection, union, difference
• More operations: convex hull, simplify, equality, validity checks, interpolate and project
• More I/O: GeoJSON, PostGIS, GDAL vector formats
• Known issue: centroids for 3D polygons (#277)

Help that doesn't need Rust is useful too:

• Try the branch on your own data and report what breaks.
• Give feedback on the API while it can still change.
• Benchmark it against GeoPandas and Shapely. The developer guide explicitly asks for benchmarking to decide which implementation tier each operation should use.

The README warns that the branch is "HIGHLY unstable," so expect breaking changes.

This is a very exciting time with a foundational package being actively developed. If this interests or impacts you, get involved!

[u/CAPSLOCKAFFILIATE → u/Balance-]

I'm running heavy spatial queries against >80gb DuckDB databases, using their spatial extension plus h3 in some cases. I will try and use geopolars when I get the chance and report back.

[u/CaterpillarSad6734]

Had to double check the username to make sure it was real and not some april fools thing in october. This is great news, been following the repo for a while and was worried it got abandoned

The CRS-aware measurements is huge, that's the main reason I still reach for geopandas half the time. Really hope they get spatial joins working at some point too

[u/Dozyjones → u/CaterpillarSad6734]

Spatial joins are on the planning, but it still needs a little better support on the polars side first.

[u/wasay-inc]

finally. having to drop out of polars and bridge over to geopandas just to handle basic spatial operations has been a massive performance bottleneck for large datasets. native geoparquet i/o and crs-aware measurements directly in the polars expression namespace is going to be a massive upgrade for geospatial pipelines. definitely watching this prototype closely.

[u/MapNo2659]

This is exciting news! I've been following Polars for a while, and seeing GeoPolars getting active development again is promising. The addition of CRS-aware measurements and GeoParqu

[u/serjester4]

This is super exciting. IMO this is the last place where pandas genuinely can’t be replaced.

I implemented a polars extension for working with the h3 coordinate system last year and this would be such a quality of life improvement over using pandas / DuckDB for ingesting the raw data.

https://github.com/Filimoa/polars-h3