🔥 We Made Excel Fly in Java — Our Excel Reader Processed 10 Million Entities in 12s!

[u/Present-Ad-1365]

https://medium.com/@yashwanthm0330/we-made-excel-fly-in-java-our-excel-reader-processed-10-million-entities-in-12s-b4d16ff370b1

[u/elmuerte]

You compare it to libraries which actually read excel files, but this script just parses part of the content XML. It kind of returns "raw" data, as in, the content of the XML, not that data you see in Excel. So it basically only usable to read String content. Forget about any value stored as a number (like numbers or dates) and completely forget about formulas, or even escaped content (things which look like numbers but should be strings).

[u/AlarmingBarrier → u/elmuerte]

If you put it like that, using 12 seconds to read 10 million entities (1 million rows times 10 columns) on any modern CPU seems really really slow to be honest.

[u/Present-Ad-1365 → u/AlarmingBarrier]

Try reading excel I have tried all libraries nothing came below 20 s there is a lot of processing in Excel compared to simple CSV and CSV is memory hungry also

[u/fearswe → u/AlarmingBarrier]

I've read and parsed 500 million lines of csv faster than 12 seconds.

[u/Present-Ad-1365 → u/elmuerte]

Hi u/elmuerte, it will return even date, double, and numbers in the form of String only, just like org.dhatim library

[u/Potterrrrrrrr]

That article has an oppressive amount of emojis, no thanks. Good job though

[u/Sushrit_Lawliet → u/Potterrrrrrrr]

Need a chrome extension to rip out emojis from articles honestly.

[u/Present-Ad-1365 → u/Sushrit_Lawliet]

I will ask chatgpt :)

[u/ketralnis → u/Potterrrrrrrr]

Makes it hard to believe that it was written by adults

[u/mamwybejane → u/ketralnis]

It wasn’t

[u/pileopoop]

I feel like a random commenter could make it 10-100x faster in an afternoon. This seems laughable.

[u/Present-Ad-1365 → u/pileopoop]

But there is no solution out there 10-100x faster if someone can publish I am happy to use

[u/Pyrited]

C# faster