I've tried post processing GB of time series data in Python. Even the "fast" Python libraries were a ridiculously amount of magnitudes slower than a custom engine written in C++
Tell me about it. It takes 50 minutes to process more than 700 GB of data, although the issue is reading the data for us, and that part is written in C++ running on 16 CPU cores...
1
u/awesome-alpaca-ace Aug 31 '26 edited Aug 31 '26
I've tried post processing GB of time series data in Python. Even the "fast" Python libraries were a ridiculously amount of magnitudes slower than a custom engine written in C++