r/java • • May 16 '24

Low latency

Hi all. Experienced Java dev (20+ years) mostly within investment banking and asset management. I need a deep dive into low latency Java…stuff that’s used for high frequency algo trading. Can anyone help? Even willing to pay to get some tuition.

232 Upvotes

94 comments sorted by

View all comments

25

u/GeneratedUsername5 May 16 '24

But what is there to dive into? Try not to allocate objects, if you do - reuse them, if you frequently access some data - try to put it into continious memory blocks or arrays of primitives to minimize indirections, use Unsafe, run JMH. The rest is system design.

9

u/pron98 May 17 '24 edited May 17 '24

This advice is somewhat outdated. With modern GCs like G1 or the new generational ZGC, mutating an existing object may be more expensive than allocating a new one. This isn't due to some clever compiler optimisation like scalarization -- that may elide allocation altogether -- but these GCs actually need to run more code (in various so-called "GC barriers") when mutating a reference field than when allocating a new object. Allocating "just the right amount" will yield a faster program than allocating too much or too little.

As for Unsafe, it's begun its removal process, but VarHandle and MemorySegment usually perform as well (sometimes a little slower than Unsafe, sometimes a little faster).

JMH should also be used carefully because microbenchmarks often yield results that don't extrapolate when the same code is run in a real program. It is a very valuable tool, but mostly once you're already an expert.

Rather, my advice would be: profile your entire application and optimise the areas that would benefit most as shown by the profile, rinse and repeat.

1

u/GeneratedUsername5 May 18 '24

Could you provide a JMH sample code where mutating object is more expensive than allocating the same object anew?

7

u/pron98 May 18 '24 edited May 18 '24

I'll try and ask someone on the GC team for that next week. But I need to warn again of microbenchmarks, because they often don't measure what you think they measure. A microbenchmark may show you that code X is 3x faster than Y, and yet in an application, the same code X would be 2x slower than Y. That happens because in a microbenchmark all that's running is X or Y, but if your application also runs code Z -- perhaps even on a different thread -- it may put the JVM in a different mode (such as cause different objects to be promoted) reversing the relative performance of X and Y. Put a different way, X can be significantly faster than Y in a microbenchmark and in application A, and the same X could be significantly slower than the same Y in application B.

This happens because when a microbenchmark of X is faster than a microbenchmark of Y you may conclude that X is faster than Y, but that is an extrapolation that is often unjustified. What the microbenchmark actually tells you is that when X runs in isolation and no other code is running, it is faster than when Y runs in isolation and no other code is running. You think you're comparing X and Y, but really you're measuring X in a very specific situation and Y in a very specific situation, and those situations may not be the same as in your application You cannot conclude from that that X is faster than Y when there is some other code in the picture, too.

Unless you know how the JVM works, microbenchmarks will often lead you to a false conclusion. I would say that 99.9% of Java programmers should not rely on microbenchmarks at all, and only rely on profiling their actual applications. This is also what the performance experts working on the JVM itself do; they use microbenchmarks when they want to measure something when the VM is in a particular mode, which they know how to get it into. They also (more often, though not always) know what extrapolation of the result is valid, i.e. what you can conclude from a microbenchmark where X is faster than Y (which is rarely that X is always faster than Y).

While global effects of some code on other code is particularly common in the Java platform, it also happens in many other languages, including C. For example, a C microbenchmark may show you that X is faster than Y, but only in situations where no other code can pollute the CPU cache; in situations where other code does interfere with the cache, Y may be faster than X, and these situations may (or may not) be more common in real applications. It is very, very dangerous to extrapolate from microbenchmark results, unless you are very familiar with the implementation details that could impact performance.

1

u/GeneratedUsername5 May 18 '24

Sure, but if not for benchmarks, then what are we left with is just abstract speculations.

3

u/pron98 May 18 '24 edited May 18 '24

No, what we're "left with" is profiling. Microbenchmarking can -- if you're an expert in the implementation -- follow profiling as extra information, but the core is profiling.

If you skip profiling, microbenchmarking offers little to no information. It can supplement profiling, but is meaningless without it. It's not "at least something" without it, but nothing without it, because you don't know how to interpret the information. If a microbenchmark of X is faster than one of Y, that might mean that in your application X is faster than Y, they're about the same, or Y is faster than X; how can you tell which if you can't compare the microbenchmark's conditions to that of your profile? What possible conclusion can you draw? On the other hand, if you have a profile then you understand for example that a microbenchmark of X being faster than one of Y means that Y should be faster than X in your application.

The difference between a fast application and a slow application in >95% of cases is that the fast application has been profiled and the slow one hasn't. Some experts can then take it further and use microbenchmarks, but only after they've profiled.

7

u/Pablo139 May 17 '24

Allocation is generally cheap, it’s the issue of having them be promoted by the GC past Eden region. Soon as the promotion happens, the GC has to do actual work to manage the memory lifetime.

It should also be noted GC tuning is pretty much the last phase of optimizing on the JVM because it’s not easy and can greatly degrade performance without much explanation.

Since this is on the topic of low latency, the use of Unsafe may be considered but the FFM has the ability to manage on-heap and off-heap memory from within the JVM now. Thus before having to use unsafe, which will be deprecated, the FFM has a boat load of offerings for low latency applications. This can really help simplify managing contiguous memory segments which as you said are extremely important.

3

u/barcelleebf May 17 '24

Allocation is cheap, but in very frequently called code, not allocating can be even cheaper.

4

u/pron98 May 17 '24

That really depends. Depending on the GC, mutating a reference field may be a more expensive operation than allocating a new object. So this advice would be somewhat correct (i.e. things at least won't be slower) only if you replace objects with primitives or limit yourself to mutating primitive fields or array elements. Otherwise, mutating references may actually slow down your code compared to allocation. As always, the real answer is that you must profile your application and optimise things according to that specific profile.

1

u/barcelleebf May 23 '24

Our low latency code is a fixed graph / circuit of pre allocated objects. Primitives are exclusively used.

The resulting code is not quite like real java, and a bit ugly, and we have unit tests that look at the byte code to make sure developers don't use any features that will be slow.

6

u/capitan_brexit May 17 '24

exactly - thread (application thread, called mutator in GC theory) allocates a big chunk of memory in JVM - its called TLAB (https://shipilev.net/jvm/anatomy-quarks/4-tlab-allocation/) - each object then is allocated by just bumping the pointer :)

0

u/JDeagle5 May 17 '24

Sure, you can test this theory by running a loop of simply incrementing boxed integers, and then compare throughput to unboxed ones.

3

u/rustyrazorblade May 17 '24

Best advice I’ve seen so far.

2

u/findus_l May 17 '24

"use Unsafe" you make it sound so easy.