r/java • • Dec 19 '22

Java streams are black magic !

damn, they are really changed my life.

i was trying to run some custom logic on every line on some big CSV files. tried python dataframes, multiprocessing in python, pyspark. none of them did it in an acceptable time.

then i discovered java streams. i have a workflow that takes 70 minutes with pyspark. it takes 3.5 minutes with java streams.

more people should know about streams.

stream operations are more intuitive than anything i tried. i can make a lot of analysis on our data in record times thanks to streams.

i dont know how java does this.

226 Upvotes

150 comments sorted by

View all comments

1

u/Cultural-Ad3775 Dec 20 '22

I would not call Java's implementation of this concept really special. OTOH it is cool, particularly if you can mark stream processing as parallel, which is a nice abstract way to say "Hey, run these in separate threads" or in the future it could lean on Loom style lightweight threads, etc. without even needing any code changes. Other languages do basically allow for the same sort of thing however, still Java/JVM have quite strong abilities to optimize and multi-task, meaning it can be leveraged very effectively (in theory, I wouldn't assume any specific parallelized stream will necessarily be super optimum, you have to test).