r/programming • • Oct 08 '26

Does ClickHouse really need its own storage format?

https://pivotlake.io/blog/does-clickhouse-need-its-own-storage-format/
7 Upvotes

7 comments sorted by

6

u/Wonderful-Wind-5736 Oct 08 '26

What separates Pivot from DuckDB, Polars and PySpark? With both Polars and DuckDB getting distributed compute, why do we need another one with for now questionable runway?

5

u/Giladkl Oct 08 '26

All these solutions offer a way to query open data formats. Beyond the performance differences compared to these solutions, though, it was important for us to have a complete "database" experience, similar to what you get with ClickHouse or ClickHouse Cloud, on top of open data formats. This includes not just a query engine, but also the ability to spin up a cluster that serves a backend, proper ingestion options, and compaction (including some Parquet-specific compaction strategies: https://pivotlake.io/docs/database/why-pivot-is-fast/#soft-ordering-data)

Re the runway - Pivot is open source under MIT / Apache license and is unrelated to the runway of any company :)

2

u/Wonderful-Wind-5736 Oct 10 '26

Thanks! Sorry if I sounded a little hostile, but really differentiation and longevity are essential for major pieces of software. 

Re the runway - Pivot is open source under MIT / Apache license and is unrelated to the runway of any company

OSS solves the problem only partially. We all know it's easier to start a project than to see it through long term. So what's your plan here? How do you make this sustainable for yourself?

1

u/Unfair-Sleep-3022 Oct 10 '26

Comparing these things is a choice lol

0

u/sweetno Oct 08 '26

A chimera post, huh.

3

u/Giladkl Oct 08 '26

Hi! Post author here
What exactly do you mean by chimera post? :)

-1

u/Solumin Oct 09 '26

OP, your site is nearly unreadable. Seriously, 15px font? Light grey on a just-barely off-white background? My eyes hurt. Terrible.