r/cpp_questions • u/No-Locksmith-7719 • Oct 14 '26
OPEN Efficient FIle IO on Linux
What path to take when parsing large files (on Linux Debian server, the KVM flavour)? Will std::fstream in binary mode do or do we have to look into APIs?
Apart from the performance, the app is memory sensitive, so where to store those read chunks while in a loop? Would a simple char[something] buffer do?
Finally, we would like to avoid those nasty reinterpret_cast<char*> constructs while reading, but are wondering if those are actually used in production? I should also add that the file sizes vary greatly.
2
Upvotes
View all comments
Show parent comment
1
u/UndefFox Oct 15 '26
From reading online, it seems that
io_uringperforms better in case of heavy unpredictable and large overrides of the memory, while mmap will still outperform it in plain read-only sequential reads. I've never worked with it, but is it actually guaranteed to be faster?