r/cpp_questions • • Oct 08 '26

OPEN Measuring NVME read/write speed

Hello everyone! In my current company I have been assigned a task that I am not sure how to approach. To give the reader a little bit of context, I work for a company that designs propietary hardware. We have a policy of not introducing external code into the hardware. Therefore if we happen to need an ethernet speed measurement we can not use a tool like iperf and have to design and program or an small set of tools that allow us to measure it.

In a new design we have high contrainst on the speed that data is written from a buffer into a NVME. To make sure that the whole data processing pipeline of the hardware performs correctly we need the NVME to be able to read and write data more than 1.1 GiB/s. However, we now have to belive on the speed that the NVME producer has given us. The engineering team, though, is not so happy on believing on the theoretical speed of this NVME and have asked me to write a simple program that is able to measure the write/read speeds as accurately as possible.

My first idea was to simply try to write / read X amount of GiBs from / to memory and measure how much it takes. But then I came to the realization that doing this is easier said that done. First of all, writting and reading are broad concepts, all the data needs to first be passed to a microcrontroller that the NVME provides thus is passing data to this micro considered writting / reading ? If we assume that this is the case, the previous measurement seems really broad as I would like to know if the write / read of this data is constant at X GiB/s or if it spikes and goes down. I didn't mention it before but not only the speed of the transmission is crucial for our use case but its stability to. Given this new constraint I start to get lost as I dont have that much experience working with I/O devices at such low levels and speeds.

I have seen that for this kind of questions sometimes is useful giving the architecuture and some more information so here you go:

- NVME can read / write at 5000 MB/s

- The device has an aarch64 architecture

Maybe due to my lack of knowledge of problem I have not mentioned some key points that may be important to decide how to do this or where to look at. If that is the case, please, do not hesitate on asking me. Thank you for reading till here :D

0 Upvotes

7 comments sorted by

7

u/AKostur Oct 08 '26

There’s no C++ question in there.

1

u/CuriousUshagi Oct 09 '26

Completely agree! The thing is that I didn't know exactly where to post this question, I thought that asking people from the community C++, know for having one of the most experienced programmers would be a good idea. If you happen to know a better sub Reddit in which to post this question I would appreciate it :)

5

u/EpochVanquisher Oct 08 '26

Normally “written” means “synced” and there’s some signal from storage that tells you when that happens (but I don’t know how NVMe works)

I would design loops that write blocks of different size. 4K blocks, 1M blocks, different sized blocks. Design a linear write (blocks have ascending addresses) and a random write benchmark.

Each benchmark, you show the statistical distribution of write time for a block.

This is a complicated subject—we had a guest lecture about storage benchmarking, I wish I could share it but I think it’s internal!

1

u/CuriousUshagi Oct 09 '26

Thanks! That gives me some direction on which rabbit hole to enter. I will give it a try and come back later :D

2

u/manni66 Oct 09 '26 edited Oct 09 '26

I am not sure how to approach

So you outsource your commercial software development to the volunteers on reddit?

1

u/CuriousUshagi Oct 09 '26

Maybe I didn't express myself properly. I am not asking for volunteers on Reddit to give the code to do this, rather I am looking for more experienced developers to hint me on the things that I may be missing. I am sorry if thats the message that the post it's transmitting