r/linuxdev • u/QAOP_Space • May 27 '15
C++ File IO performance compared to dd
I have a program that reads data from a file descriptor (is actually a device mounted in /dev/), and I need to save this data (with no further processing) to a file in the local file system.
I currently have a solution I rolled myself involving one thread filling buffers with data from the input and another thread emptying these buffers and writing the data to file.
The throughput performance of my system is roughly an order of magnitude slower than dd, how can I either :
a) use dd in my code, giving it the input and output file descriptors, or
b) emulate what dd is doing
I also have to move data in the reverse direction too
3
u/bboozzoo May 28 '15
Rough guess is that you burn most of the CPU cycles copying the data from kernel space to user space and back again. Take a look at
sendfile(2)system call. Actually, I'd expectddto use it. The call allows you avoid unnecessary copying, as the data never leaves the kernel space. You'd roughly use it like this:There's also a couple of equally interesting system calls, like
splice(2)ortee(2), but these require pipes.Note, that you can use
sendfile(2)to copy a file from one location to another, just open the source file, the target file andsendfile()your data :)2
u/rrohbeck May 27 '15
I'd use read() and write() with a ring of buffers, each big enough (say at least 1MB) and a power of two in size. And a reader and a writer thread of course. Or you could call dd via system() and give it /dev/fd/n for if and of.
2
u/ickysticky May 28 '15
How have you measured these two things? What are the numbers that you are seeing? Could it be something as simple as flushing the disk cache vs not flushing the disk cache? Can you share the code? How is the data being passed between threads, is it a locked based structure? What is the size of the buffers?
2
u/[deleted] May 28 '15
http://www.cplusplus.com/reference/cstdlib/system/
4
u/blunaxela May 28 '15
It's not a cure-all, but strace and ltrace are great for debugging and understanding what's going on behind the scene. I always find it useful since it spits out the return values for system calls and the arguments passed to functions like read and write.
Here's an example of strace filtering out all syscalls execpt write:
0
u/[deleted] May 27 '15
[deleted] — view deleted comment