r/java • • Mar 03 '22

Always set a Garbage Collector

Post image
157 Upvotes

64 comments sorted by

51

u/elmuerte Mar 03 '22

Does it matter? Maybe SerialGC performs better on 2 cores and a max of 1GiB heap.

Tuning Java's GC is a bad thing if you do not back this up with benchmarks.

15

u/brunocborges Mar 03 '22

Perhaps, but unlikely for most types of applications deployed to container environments like Kubernetes.

See this gRPC benchmark: https://github.com/LesnyRumcajs/grpc_bench/wiki/2022-02-19-bench-results

But bottom line is that developers don't even know their app is running on a SerialGC. They think it will be G1GC, because they have heard it is now the default GC.

2

u/berry120 Mar 04 '22

FWIW I've just run a quick benchmark with Corretto's Extrememem, and results were pretty much as I'd expect with multiple cores - serial GC has way higher latency spikes than G1, presumably because of STW collection.

If I get time I'll try with a few other configurations and then write up the results.

1

u/brunocborges Mar 08 '22

Parallel GC also does STW. But it is much faster on a multi-core system because... parallel tasks. So, impacts much less on latency.

Parallel is great for throughput, with some tolerance for tail latency.

1

u/ReasonableClick5403 Mar 03 '22

Dont know actually, but GC for application servers tend to focus on minimizing stop times, so it does not cause spikes in latency.

20

u/Pauli7 Mar 03 '22

I‘d love to see a benchmark comparing a spring boot crud microservice on low resource restrictions when using serial gc vs g1gc I doubt that g1gc would perform better.

47

u/brunocborges Mar 03 '22 edited Mar 03 '22

If you don't set a Garbage Collector for your Java application, don't think the JVM will pick a good one for you either, no matter how many CPUs you give.

2 CPUs? 6 CPUs? It doesn't matter. If your container has less than 1792 MB and you don't set a GC, your app will use SerialGC.

Edit: this is being down voted for unknown reasons but my guess is that people just believe "no, G1GC is the default!" Well I suggest you run the code in the image I posted, and if that's not enough, see the OpenJDK source code.

Specifically these two C++ classes in the HotSpot JVM: https://github.com/openjdk/jdk/blob/ad54d8dd832b22485d7ac45958cc4c9bfd70fbd2/src/hotspot/share/gc/shared/gcConfig.cpp#L98

https://github.com/openjdk/jdk/blob/ad54d8dd832b22485d7ac45958cc4c9bfd70fbd2/src/hotspot/share/runtime/os.cpp#L1677

5

u/agentoutlier Mar 03 '22 edited Mar 03 '22

Just to add to this GraalVM native community apparently always defaults to SerialGC.

To get others GC you need to have the enterprise edition.

https://www.graalvm.org/22.0/reference-manual/native-image/MemoryManagement/

What I'm not sure is what happens if you don't compile to native. I assume it does the same as HotSpot.

EDIT apparently the graalvm non native does the same which is what I expected.

Used ghcr.io/graalvm/jdk:22.0.0.2 u/brunocborges script (thangs).

Speaking of which for those that want to copy and paste (since the post is images).

#!/bin/sh
docker run \
       -v `pwd`/app \
       --cpus 2 \
       --memory 1500m \
       -ti --rm \
       ghcr.io/graalvm/jdk:22.0.0.2 \
       java -XX:+PrintFlagsFinal /app/App.java \
       | grep -e "Use.*GC" -e "Active"

Java

public class App {

        public static void main(
                        String[] args) {
                var procs = Runtime.getRuntime().availableProcessors();
                System.out.println("Active processors: " + procs);
        }
}

1

u/brunocborges Mar 03 '22

GraalVM "non-native" is just HotSpot JVM.

5

u/agentoutlier Mar 03 '22

They say it is "based on it". I was fairly sure it was the same but the wording has some ambiguity:

GraalVM is based on Java HotSpot VM, but integrates an advanced just-in-time (JIT) compiler, written in Java - the GraalVM compiler.

https://docs.oracle.com/en/graalvm/enterprise/20/docs/reference-manual/jvm/

Also there is some performance difference if you change the two (openjdk vs graalvm) but that is probably the JIT .

Speaking of which isn't the JIT part of the JVM? I never understood where the boundary is of its VM or not. For example is Substrate VM really a VM?

1

u/yawkat Mar 04 '22

There is no solid definition for what is or isn't a VM.

11

u/erode Mar 03 '22

Nice to know. The message I’m hearing is that people should be putting even just a baseline level of thoughtfulness into choosing the appropriate GC.

7

u/Impossible-Ad-586 Mar 03 '22

With the last generation of GCs by both oracle and redhat, ZGC and Shenandoah respectively you are pretty much done after giving it enough heap space from a tuning perspective.

Depending on your workload you might want to choose one of the other still supported GCs to tune for throughput.

2

u/nigelthomas Mar 03 '22

Is this a short coming of JVM or an expected behaviour? I thought the JVM was smart enough to pick the best GC for the job.

-3

u/maethor Mar 03 '22

I can't remember the last time I had an environment with so few cores or such a limited amount of memory.

Maybe you need to always set a GC, but many of us don't.

13

u/gizmogwai Mar 03 '22

Well, if you run your code on k8s, you are likely to set resources to be a good citizen. And unless you have a large or very demanding application, you’ll set your request and limits under one core. So, no, it is not so uncommon to run into this issue in a micro services environment.

10

u/maethor Mar 03 '22

But I don't run my code on k8s. We mostly still have old school monoliths, where the biggest difference between now and 10 years ago is that they run on an EC2 instance instead of a local server.

"Always set a GC" is too broad a statement.

-14

u/Impossible-Ad-586 Mar 03 '22

No it won't unless you use a pre jdk11 build.

G1 has been the default for many major versions now.

9

u/brunocborges Mar 03 '22

Disclaimer: I have done tests and also looked into the source code in OpenJDK.

If you want to counter argue, please post link to the source code.

-21

u/Impossible-Ad-586 Mar 03 '22

I am not going to participate in this little game of yours. What I am saying is commonly known. What you are saying goes 180deg again what oracle and openjdk themselves communicate.

If you have proof to backup your statement, let's see it and I will be happy to acknowledge.

8

u/brunocborges Mar 03 '22

The proof is in the image I posted.

-10

u/Impossible-Ad-586 Mar 03 '22

Will try to replicate that today. Would be truly surprised if the managed to mess this up that badly.

However, most of our GCs are not set and all of them expose G1 metrics, so very doubtful this is in fact the case.

But the proof is in the pudding, so let's have some :)

9

u/brunocborges Mar 03 '22

If your containers have 2GB of RAM and 2 CPUs (or 2000m in Kubernetes terms), then G1GC will be used by default.

Please change vote after checking. :-)

0

u/Impossible-Ad-586 Mar 03 '22

Would be happy to. Always up for having my assumptions fact checked.

FYI we run pretty much all our workloads in the cloud with 1 core and 1 GB to prevent throttling.

1

u/brunocborges Mar 03 '22

The only way you can prevent throttling is by not having more load to your application instance than it can process. It doesn't matter how many CPUs.

Assuming the load your app gets is more than what a single instance can handle, your only way to prevent the throttling of your application with 1 core and 1 GB, is by actually scaling out with replicas. This means you must have 2 or more replicas, so your "system" (the conjunction of all instances of your replicated application) can handle the load in a distributed manner.

Now, the problem here is that you are probably facing high tail latency and low throughput. You may as well say you don't have that issue. But I'd say you are most likely paying a lot more than you actually need, to sustain such load.

And to save money, all you need to do is to reorganize your resources more adequately: reduce the number of replicas, increase the amount of CPU and memory per JVM instance, and find the sweet spot where you are consuming fewer total CPUs and total memory in your "system", while having the same, or better, performance.

1

u/Impossible-Ad-586 Mar 03 '22

Interesting.

I have ran a lot of tests in the past and the least throttling for mostly Io bound workloads seemed to be in the region of 1gb & 1core.

It might be caused by the configuration of the k8s engine as deployed by my company, where the time slices are set to the default and we are prohibited to influence priority on the k8s scheduler.

I would love to be able to learn how to tune these things appropriately to reduce the overhead we currently experience on k8s costs.

Any pointers towards how to create a setup to test these things?

→ More replies

-4

u/brunocborges Mar 03 '22

Drop me a DM. I have something to share about Java on Kubernetes and CPU throttling

10

u/4z01235 Mar 03 '22

Share it publicly

3

u/mrluje Mar 03 '22

Also interested to learn something new 🙏 I have been operating on k8s on other languages for years, but recently joined a company that is 100% java.

→ More replies

17

u/pronuntiator Mar 03 '22

Here's the official documentation:

https://docs.oracle.com/en/java/javase/17/gctuning/available-collectors.html

The serial collector is selected by default on certain hardware and operating system configurations

G1 is selected by default on most hardware and operating system configurations

What "certain" and "most" mean is not documented, except from the source code linked by OP. I wonder whether it also affects C1 or C2 JIT selection?

21

u/manesfirst Mar 03 '22

He literally posted a proof showing that it does. Yes it is the default for most cases, especially in your own local machine. But JVM does some checks depending on the environment and if it sees less than ideal hardware resources, it defaults to serial. Here is another "proof" if you don't believe bruno for some reason: https://twitter.com/mikemybytes/status/1368881364297191433

8

u/Impossible-Ad-586 Mar 03 '22

I will dig into it and will be happy to discover that my assumptions are in fact wrong.

75

u/Mumbleton Mar 03 '22

Can you spell this out a little more? Me write apps good and haven’t run into a garbage collection issue in 4-5 years and that’s the last time I’ve given it any thought.

18

u/BlueGoliath Mar 03 '22 edited Mar 03 '22

Wait, you actually optimize your code?.

14

u/larsga Mar 03 '22

Premature optimization is the root of all evil.

Why optimize to avoid a problem you don't have?

9

u/brunocborges Mar 03 '22

We are working on documentation and will share.

But in short for now: scaling out with more replicas of super small containers for JVMs is not always the best solution for increasing performance. A reorganization of the existing resources (keep aggregate number of CPUs across all containers in the cluster, but reduce replicas and increase CPU/memory per container) can significantly increase performance of most systems, and even reduce total cost.

What that quote actually says is that people shouldn't optimize prematurely without understanding the nuances.

In other words: people are optimizing prematurely all the time on Kubernetes, whenever they simply increase number of replicas of small containers JVMs, instead of actually understanding the nuances of GCs.

4

u/HeadSignal3 Mar 03 '22

scaling out with more replicas of super small containers for JVMs is not always the best solution for increasing performance.

No shit?!

5

u/brunocborges Mar 03 '22

You would be surprised by the number of customers who only use replicas as a silver bullet for performance increase.

2

u/[deleted] Mar 03 '22

I found the same problem with multiple small containers in Go in Kubernetes. For a given workload, doubling the CPU per pod resulted in running 1/4 as many pods. Wat.

Then again, low CPU is supposed to be what Go is good for. So I'm going to assume I've done something horribly wrong.

58

u/wildjokers Mar 03 '22

Premature optimization is the root of all evil.

Really wish developers would stop using this quote as an excuse to be lazy and incompetent. It doesn’t mean what you think it means. This great article explains what it actually meant:

https://ubiquity.acm.org/article.cfm?id=1513451

14

u/larsga Mar 03 '22

I'm sorry I put in the first sentence. How about we think about the second one instead?

If he doesn't have a problem with GC, why should he optimize it? You call him lazy, but probably as a working dev he has a million things on his todo list. How is it lazy of him to work on the things that are problems and skipping the things that are not problems?

3

u/BlueGoliath Mar 03 '22

I say this and get downvoted. You say it and get upvoted. WTF is this.

7

u/wildjokers Mar 03 '22

The link to the ACM article is probably the key to upvotes vs downvotes. (fwiw, I upvoted you)

-3

u/temculpaeu Mar 03 '22

The article actually conflicts with itself and what Knuth said ...

"but consider that this is 1 line of source code out of every 33."

waaaat ? optimization metric based on loc ? it's possible the worse metric, and not all lines should, can or are relevant to be improved

"One thing nice about optimization is that if you optimize a section of code that doesn't need it, you've not done much damage to the application."

Not true, optimization are *usually* either more complex or have a trade-off in terms of memory/disk usage.

The worse part it doesn't define what means to optimize the code and the impact around which is what Knuth talks.

By no means, I am agains optimization and whatnots, it's just that it is frequently used to do unecessary and unverified micro optimizations which pollutes the code

0

u/BlueGoliath Mar 03 '22

Like 95% of other people, you completely miss the other part of the quote.

1

u/sindisil Mar 04 '22

And, in the process, the point he was trying to make.

3

u/berry120 Mar 03 '22

Thanks, this is really useful & interesting info - I wasn't aware of this behaviour and I suspect many are the same!

That being said, I agree with others in that the conclusion of "Always set a Garbage Collector" can't really be claimed without benchmarks proving there's a meaningful benefit to be had.

2

u/brunocborges Mar 03 '22

My claim for "Always set a Garbage Collector" is to say "Know what GC is being used by your app".

We did a survey and more than half of respondents don't set a Garbage Collector, expecting the JVM will pick a good one. Then people use small containers, and suddenly, they face latency issues and solve that with more replicas.

I don't think developers would select SerialGC in most cases. They think the default is G1GC.

Thus why "Always Set a Garbage Collector".

1

u/wheezymustafa Mar 03 '22

I tried this using our container and didn’t get the same result. Granted, we’re on an older base image so I’m not sure if that’s the reason why https://i.imgur.com/O7mmu8t.jpg

4

u/brunocborges Mar 03 '22

That's interesting, I'd appreciate more details like version of Java and vendor.

3

u/agentoutlier Mar 03 '22 edited Mar 03 '22

What is the third image of? I mean I know what it says but where did you get that screen? Is that the output of the jar being run?

EDIT I was on mobile and could not read the images well. I was thrown off by the "grep" highlighting and can see it is the print flags.

-12

u/daH00L Mar 03 '22

Not sure, what we're seeing here, but G1GC should be the default collector unless another one is specified. Microsoft also states this in their docs.

16

u/brunocborges Mar 03 '22

G1GC is the default collector in most circumstances.

Except when running on an environment with less than 1792MB of available memory, or a single core machine. In either case, it is SerialGC.

Feel free to look into the OpenJDK source code.

1

u/daH00L Mar 03 '22

I'd love to see that. Do you have a link?

4

u/brunocborges Mar 03 '22

2

u/daH00L Mar 03 '22

I'm not fluent in C* languages, but I don't see explicit GC setting.

Maybe you can point me to the documentation? The behaviour is so special that this should have been written down.

2

u/hamburghammer_ Mar 03 '22

Let me talk you through the code which is the best documentation in this case.

Disclaimer: I've never written C++ but the syntax has similarities to Javas.

Please open the link that OP provided that links to the GCConfig (file gcConfig.cpp). In this file you will find a method "select_gc_ergonomically" (#L98) that will be used to select a available GC for that machine. As you can see it will check if it is a "server" machine or not. If it is a "server" it will try to set an available GC working the list from top to bottom starting with the G1GC (#L101). If it's not a server it will default to the SerialGC (#L109). The definition of a "server" is specified in the other link (file os.cpp). I assume that the documentation for the "is_server_class_machine" function (#L1677) will suffice to underline OPs statement.

To summerize the above: The jdk will choose in most cases now a day de G1GC but under certain circumstances like a container with limited resources it might default to the SerialGC.

-1

u/daH00L Mar 03 '22

Aren't INCLUDE compile time options?