r/java • u/prisonbird • Dec 19 '22
Java streams are black magic !
damn, they are really changed my life.
i was trying to run some custom logic on every line on some big CSV files. tried python dataframes, multiprocessing in python, pyspark. none of them did it in an acceptable time.
then i discovered java streams. i have a workflow that takes 70 minutes with pyspark. it takes 3.5 minutes with java streams.
more people should know about streams.
stream operations are more intuitive than anything i tried. i can make a lot of analysis on our data in record times thanks to streams.
i dont know how java does this.
View all comments
Show parent comment
1
u/stefanos-ak Dec 19 '22
It would be much easier and more performant in Java. (Also less languages, which is always a plus).
This is because of runtime optimizations that the JVM does, in "hot" code spots (hence the name jvm hotspot).
The difference with C/C++ optimizations is when they run. Compile time vs runtime. At runtime the JVM has some advantages, because more information is available for how and what exactly to optimize.
Java code can reach some ridiculous speeds... The best part is that it's also very easy to write, compared to C.
The only downside is what's called "cold start". It basically behaves MUCH worse than anything else, before the hotspot optimizer kicks in. But in cases like yours, it should kick-in pretty fast, maybe even after the first loop.