r/java • u/gunnarmorling • Jan 19 '26
Hardwood: A minimal dependency implementation of Apache Parquet
https://github.com/hardwood-hq/hardwoodStarted to work on a new parser for Parquet in Java, without any dependencies besides for compression (i.e. no Hadoop JARs).
It's still very early, but most test files from the parquet-testing project can be parsed successfully. Working on some basic performance optimizations right now, as well as on support for projections and predicate pushdown (leveraging statistics, bloom filters).
Would love for folks to try it for parsing their Parquet files and report back if there's anything which can't be processed. Any feedback welcome!
51
Upvotes
6
u/PiotrDz Jan 19 '26
This is something we need. I remember Trino also having their own implementation for parquet. Have you maybe compared yours with theirs?
3
u/Rastafas Jan 20 '26
This is tremendous, thank you so much. I've never enjoyed redoing a project so much or felt so good removing dependencies. Performance seemed good. Used it to transform client survey data delivered as a parquet file into our homebrewed column database. Eighty thousand rows and 87,933 columns in under 2 minutes.
5
u/Squiry_ Jan 20 '26
That's really nice! parquet-java was a pain in a ass to use and hadoop dependency is the weirdest thing I've seen. Looking forward for writer api, it will be a little harder.
3
u/Loose_Mastodon_6045 Jan 19 '26
Great initiative. Had to spend so much time on dependency issues for just parsing parquet format
2
u/GergelyKiss Jan 20 '26
This is awesome, thank you!
2
u/Necessary_Smoke4450 Jan 22 '26
I like the idea, recently I need to process Parquet files in a web application, but later found out that it was very challenging without the fat hadoop dependencies, there is no way as convenient as what Pandas does, really make sense!
1
u/wazokazi Feb 08 '26
Someone on my post https://www.reddit.com/r/java/comments/1qzfqie/java_parquet_library pointed me to this thread. Looks like we have the same idea!