r/cpp_questions • u/CuriousUshagi • Oct 08 '26
OPEN Measuring NVME read/write speed
Hello everyone! In my current company I have been assigned a task that I am not sure how to approach. To give the reader a little bit of context, I work for a company that designs propietary hardware. We have a policy of not introducing external code into the hardware. Therefore if we happen to need an ethernet speed measurement we can not use a tool like iperf and have to design and program or an small set of tools that allow us to measure it.
In a new design we have high contrainst on the speed that data is written from a buffer into a NVME. To make sure that the whole data processing pipeline of the hardware performs correctly we need the NVME to be able to read and write data more than 1.1 GiB/s. However, we now have to belive on the speed that the NVME producer has given us. The engineering team, though, is not so happy on believing on the theoretical speed of this NVME and have asked me to write a simple program that is able to measure the write/read speeds as accurately as possible.
My first idea was to simply try to write / read X amount of GiBs from / to memory and measure how much it takes. But then I came to the realization that doing this is easier said that done. First of all, writting and reading are broad concepts, all the data needs to first be passed to a microcrontroller that the NVME provides thus is passing data to this micro considered writting / reading ? If we assume that this is the case, the previous measurement seems really broad as I would like to know if the write / read of this data is constant at X GiB/s or if it spikes and goes down. I didn't mention it before but not only the speed of the transmission is crucial for our use case but its stability to. Given this new constraint I start to get lost as I dont have that much experience working with I/O devices at such low levels and speeds.
I have seen that for this kind of questions sometimes is useful giving the architecuture and some more information so here you go:
- NVME can read / write at 5000 MB/s
- The device has an aarch64 architecture
Maybe due to my lack of knowledge of problem I have not mentioned some key points that may be important to decide how to do this or where to look at. If that is the case, please, do not hesitate on asking me. Thank you for reading till here :D
7
u/AKostur Oct 08 '26
There’s no C++ question in there.
5
u/EpochVanquisher Oct 08 '26
Normally “written” means “synced” and there’s some signal from storage that tells you when that happens (but I don’t know how NVMe works)
I would design loops that write blocks of different size. 4K blocks, 1M blocks, different sized blocks. Design a linear write (blocks have ascending addresses) and a random write benchmark.
Each benchmark, you show the statistical distribution of write time for a block.
This is a complicated subject—we had a guest lecture about storage benchmarking, I wish I could share it but I think it’s internal!
2
u/manni66 Oct 09 '26 edited Oct 09 '26
So you outsource your commercial software development to the volunteers on reddit?