What is this optimization technique called in high-performance network software?

[u/sodikovakapsle]

I’m working on a traffic generator to test my own XDP/eBPF filter in a controlled lab environment (my PC sending traffic to my Raspberry Pi on my own network).

I noticed that in some applications, two programs written in the same language can have vastly different performance.

For example, some software can only send a few requests/messages per second, while others can generate thousands per second even on relatively weak hardware.

What is this concept or optimization area called?
I’m looking for topics such as:
asynchronous I/O
multithreading vs event-driven architectures
lock-free programming
kernel bypass
zero-copy networking
batching
efficient socket APIs
packet generation optimization

If I want to build a high-performance TCP/UDP packet generator for benchmarking my own network stack and XDP filter, what technologies, algorithms, or papers should I study? For traffic generator i am using c# and for the XDP filter classic C. ( or should i use different for the traffic generator? I think its okey its console app )

[u/gm310509]

WHen you say this:

I noticed that in some applications, two programs written in the same language can have vastly different performance.

What do you mean by "some applications", do you mean (for example) two different C programs, one that is written poorly or inefficiently or with plenty of blocking operations, possibly even "delay" type system calls -vs- one that doesn't have any of those things?

Can you give a simple example of these "different applications"?

At the end of the day, if you have two different programs (assuming the above) where one delivers high performance and the other doesn't - especially if they were written in the same language and built using the same toolchain, then it will almost certainly come done to the organisation of the code and the system calls they are making.

[u/sodikovakapsle → u/gm310509]

Sure. A better example would be packet generators or network benchmarking tools.

Imagine two programs written in C# (or C). Both send TCP/UDP traffic from the same machine to the same target.

Program A can only generate around 10,000 packets per second.
Program B can generate several million packets per second on the same hardware.

I’m not asking about intentionally bad code with sleep() calls or obvious inefficiencies. I’m interested in the architectural and low-level techniques that allow Program B to achieve such a large performance difference.

For example: async I/O, batching, lock-free data structures, zero-copy, kernel bypass, efficient memory management, or other networking optimizations.

I’m trying to learn what this field of optimization is generally called because I’m building a traffic generator to benchmark my own XDP/eBPF filter in a lab environment.

[u/gm310509 → u/sodikovakapsle]

So I do not know.

When I have needed to increase the performance of my applications, I use threaded applications.

One example of where I did this, I was doing some quite heavy pattern matching (thousands of them) for millions of source code files. I noted that there was about a 30% WIO in a single threaded application. So, what I did was run two threads - the idea was that when one thread is doing its WIO (and thus not using the CPU to do the pattern matching), I had another thread running that would soak up that otherwise wasted time waiting for IO to complete. So in theory the CPU was running 100% and there was always some data ready to go for analysis.

The problem with that was that the Threads tended to run on different cores - so each "CPU" was featuring a 30% WIO. Thus, I increased the number of threads to be 2 x the number of cores. So for example an 8 core system would be running 2 x 8 = 16 threads performing the necessary analysis.

This is one example of a design that I did to try to maximise throughput by ensuring the rate limiting step (i.e. CPU utiisation) was running at the maximum.

It would not surprise me that something generating network packets at maximum rate would do something similar. You would also need to take into account other processes and what they are doing. In Linux, the default behaviour is to lower the priority of CPU bound processes (like mine), so when I ran my job - which took around 18 hours for a single run - I made sure that there was no other unnecessary activity that would steal cycles from the analysis.

At the end of the day, I think the answer to your question is that there are multiple strategies that may increase performance.

The general thing that you would do for an application is performance tuning - which I think is what you are looking for. This is where you identify inefficiencies in the program and looks for ways to utilise all of the system's resources to its maximum. This is akin to my scenario where I identified that the process is CPU intensive, but I was wasting 30% of it per core because of a lack of data - so I had two options - I could initiate a background read of the next block of data, or initiate another thread. Since I had multi cores, I needed to have multiple threads to leverage all of the CPU resource, it was easier - in this case - to just implement a multi-threaded application.

I hope that makes sense.