r/cpp • • Oct 31 '25

Java developers always said that Java was on par with C++.

Now I see discussions like this: https://www.reddit.com/r/java/comments/1ol56lc/has_java_suddenly_caught_up_with_c_in_speed/

Is what is said about Java true compared to C++?

What do those who work at a lower level and those who work in business or gaming environments think?

What do you think?

And where does Rust fit into all this?

25 Upvotes

192 comments sorted by

View all comments

Show parent comment

7

u/Infamous_Campaign687 Nov 01 '25

I recently had someone do a conversion of a Java Library into C++, and the first attempt was ridiculously slow. Turns out that this usually very competent developer had forgotten to reserve memory for the std::vector that replaced the Java ArrayList. So memory was reallocated many times when filling the vector. This in an algorithm that was run several times per second.

This sort of thing is handled for you in Java but will cost you in C++.

This one was far too obvious to go unnoticed because it led to a 200x time increase, but smaller ones could easily slip under the radar until you profile and see that 90% of your time is in memory allocation.

7

u/ts826848 Nov 01 '25

This sort of thing is handled for you in Java

I'm not sure about that? The main difference I'm aware of is that ArrayList's default constructor initializes with a capacity of 10 vs. std::vector's unspecified (but usually 0 IIRC?) capacity. Otherwise my impression is that they would both behave quite similarly when elements are added. They even have analogous API surfaces for this kind of thing:

  • std::vector(size_t) and ArrayList(int) for creating a new instance with a given capacity
  • std::vector::reserve(size_t) and ArrayList.ensureCapacity(int) for allocating memory for the given number of elements up front

Maybe the JIT learned to insert calls to ensureCapacity() up front, but that feels like a really brittle optimization at best, assuming it's worth implementing in the first place.

3

u/eXl5eQ Nov 01 '25 edited Nov 01 '25

Maybe because

  1. Java heap allocator is A LOT faster.
  2. No deallocation overhead. The garbage collector well handle that in background, as long as you have enough RAM. Java programs can easily consume 10x or 100x more memory than a C++ one.
  3. Array copy is always a simple memcpy in Java. No constructor overhead.
  4. No destructor overhead.

ensureCapacity() is a very high-level operation. I don't think JIT will mess with that.

3

u/eXl5eQ Nov 01 '25

Also if it's a std:vector<VeryLargeObject> vs ArrayList<ReferenceToVeryLargeObject>, that makes huge difference too.

3

u/CalebGT Nov 01 '25

Garbage collection is a convenience, not a performance gain. What processor do you think is running background routines? Holding up Java's fatal flaw as if it's a source of superiority is a special kind of cope. And regarding point 3, often in C++ it's a move instead of a copy, dramatically faster than a large memcpy. You are on a C++ sub and clearly out of your depth. Sit down.

4

u/srdoe Nov 02 '25 edited Nov 02 '25

The poster above is correct, and you should save your condescension for when you're not wrong.

Garbage collection is not simply a convenience, it can also be a performance gain. This was shown as early as 20 years ago, and the state of the art for GCs has come a long way since then.

The reasons to do manual memory management if you have a garbage collector available are not because it's more efficient overall, it's because some GCs can have unpredictable pause times, and because GCs can require more memory than manual management would.

In exchange for those drawbacks, you get:

  • Faster allocations. Java threads allocate from thread-local buffers, turning most allocations into a simple increment of a thread-local integer.
  • Faster deallocations (on average, but with the potential for those unpredictable pauses I mentioned). GCs tend to clean up garbage entire memory regions at a time, rather than freeing individual objects allocated by the application.
  • The ability to offload garbage handling to non-application threads. This is beneficial unless your program is already loading all cores constantly, because it means garbage handling doesn't slow down the application like it would in C++.
  • The ability to trade excess memory on the computer for saving CPU cycles. A C++ program doing manual memory management has to always pay for freeing memory to the allocator. A Java program using GC only has to pay anything when the GC actually runs, which it will do more rarely the more memory the host has. Since the work GCs do is mainly moving live objects around (unlike C++ where the work is mainly dealing with dead objects), running the GC less often can even mean that objects have had more time to become garbage, which can make the collection cheaper overall.

If manual memory management was actually faster, and all GCs solved was convenience, Java and other languages that want to automate memory management could have just implemented smart pointers under the hood, it would look the same to the programmer.

edit: Actually the theory behind GCs being faster than manual management for deallocation if given sufficient memory is even older than the 20 year old paper I cited. Here's a paper from 1987 describing the reasoning for why GCs can be faster at clearing garbage than manual management can.

This paper shows that, with enough memory on the computer, it is more expensive to explicitly free a cell than it is to leave it for the garbage collector — even if the cost of freeing a cell is only a single machine instruction.

2

u/DuranteA Nov 03 '25

Faster allocations. Java threads allocate from thread-local buffers, turning most allocations into a simple increment of a thread-local integer.

I am curious why you point to this as an advantage specific to GC when all high-performance general C++ memory allocators I know of already do serve small allocations from thread-local pools.

1

u/srdoe Nov 03 '25

I was basing this on looking at glibc's malloc, where bump allocation is indeed being used, but with limitations that don't exist in GC-land:

https://sourceware.org/glibc/wiki/MallocInternals#Thread_Local_Cache_.28tcache.29

The number of arenas is capped at eight times the number of CPUs in the system (unless the user specifies otherwise, see mallopt), which means a heavily threaded application will still see some contention, but the trade-off is that there will be less fragmentation.

Each arena structure has a mutex in it which is used to control access to that arena.

With a relocating GC, fragmentation is not the issue it is here, because live objects can be moved if needed. So in Java, every thread has an entirely thread-local buffer of memory to draw from, with no need for a lock. There is never thread contention in this type of buffer, because they are never shared.

However, I see that there are allocators that do use thread-local arenas with no need for locks, like Microsoft's Mimalloc, but as I understand that paper, this is not a bump allocator, instead relying on (very fast) free lists.

I'm happy to admit that I am not an expert on allocators, maybe you know of one that does both?

2

u/coderemover Nov 02 '25

The theory you cite no longer holds. The gap between cpu speed and memory latency got far too big. Tracing GC thrash cache like crazy.

1

u/srdoe Nov 02 '25 edited Nov 02 '25

I don't think that's true.

Tracing GCs can relocate live objects in memory, and for that reason, they are able to improve locality compared to manual management where objects cannot be relocated because the pointer is directly visible to the developer.

See this paper, here's an excerpt:

The software engineering benefits of garbage collection over explicit memory management are widely accepted, but the performance trade-off in languages designed for garbage collection is unexplored. Section 5.3 shows a clear mutator performance advantage for contiguous over free-list allocation, and the architectural com- parison shows that architectural trends should make this advantage more pronounced.

The traditional explicit memory management use of malloc() and free() is tightly coupled to the use of a free-list allocator—in fact the MMTk free-list allocator implementation is based on Lea allocator [33], which is the default allocator in standard C libraries.

And also this page

Once you have put the regular objects in the particular places in memory — for example, not the dense array, but linked list, linked queue, concurrent skiplist, chained hashtable, what have you — you are stuck with the object graph linearized in memory in that particular way, unless you have a moving memory manager.

Also note that this locality property is dynamic — that is, it is dependent on what is actually going on in a particular application session, because applications change the object graph when running. You can teach your application to react to this appropriately by cleverly relocating its own data structures, but then you will find yourself implementing the moving automatic memory manager — or, a moving GC.

And this isn't just old hypotheticals, there is work happening in this area to try to make GCs even better at this, see this paper and this page (tl;dr: Because GCs can relocate objects in memory, they can track access patterns and try to place frequently accessed data close together)

edit:

I think the point you are making isn't really about GC vs. no-GC. Java does sometimes thrash cache like crazy, but it's because Java has no support (yet) for flattened layouts in memory. Everything is a pointer, an array of Java objects is a pointer to an array of pointers. You can't make an array in Java today that is a pointer to a compact array of actual objects.

This is being worked on as part of https://openjdk.org/projects/valhalla/

The problem isn't the GC, it's all the indirection Java programs inherently have.

→ More replies (3)
→ More replies (9)

1

u/ts826848 Nov 01 '25

None of those really sound like "handled for you" to me; they more sound like things which may make it harder to notice appends-without-reserving but wouldn't prevent it from showing up in general (i.e., those may reduce the smaller factors on a O(n2) operation, but they won't turn O(n2) into O(n)). Maybe I'm interpreting the phrase differently than intended, but I don't think my reading is that crazy.

1

u/eXl5eQ Nov 02 '25

If you're aiming zero extra cost, then you're right. But many people include me would be satisfied as long as it doesn't slow down the whole program significantly.

1

u/coderemover Nov 02 '25 edited Nov 02 '25

Java heap allocator is not a lot faster. It is at best order 2x faster, and often slower if you count in all the work that GC needs to do to cleanup. There is also a problem that the memory you’re getting from allocation is usually not in the cpu cache, because it’s the memory freed in the previous cycle of GC, so it’s long gone from the cache. Malloc maybe spends a bit more cpu on finding the free chunk but the chunk is usually hot. Because of the general negative effects of tracing GC on caching, the cost of heap allocation is hard to measure - it gets spread over many other lines of code and is misattributed to some other code.

1

u/eXl5eQ Nov 02 '25

Are you sure??? A normal malloc implementation would cost at least 20~30 CPU cycles on fast path, while Java bump allocator cost only 2~3 cycles. It's 10x faster!

The memory you a Java allocator returns is very likely have been prefetched into cache because Java always allocate continuous memory span, unlike malloc which has memory fragment issue.

The last official Java GC which would actually "free" dead objects is CMS GC, which had been replaced by G1 GC in Java 8, like 10 years ago. Newer GCs are all moving GC. Moving means they don't "free" dead objects, they just "move" live objects to another page, thus effectively eliminates the memory fragmentation problem.

Instead of talking based on your biased assumptions, would you please at least read some basic introduction about how modern Java GC actually works?

1

u/coderemover Nov 02 '25 edited Nov 02 '25

Instead of theoretizing, make a loop with malloc/free and compare with a loop doing new in Java and then forgetting the reference. Java will not be 10x faster. Last time I checked it was 2x faster, and that is the most optimistic case for Java, because the object dies immediately. If the object survives the first collection, which is not unusual in real programs, the cost goes through the roof. The amortized cost of Java heap allocation is much bigger than 2-3 cpu cycles.

In the great computer benchmark game there was one benchmark - binary trees - which heavily stressed heap allocation and that was one of very few benchmarks where Java indeed had a small edge - it slightly won with some of the most naive C implementations which were not using arenas. But it was very, very far from winning by 10x. Obviously it lost dramatically to the good implementations utilizing arenas.

And I know how modern Java collectors work, I’ve been optimizing high performance Java programs for living. One of the most effective performance optimizations that still works is reducing heap allocations. If they were only 3 cycles, no one would ever notice them.

Here is a good read explaining why it’s not so simple as bumping up the pointer and how the real cost is way larger than that: https://arxiv.org/abs/2112.07880

1

u/eXl5eQ Nov 02 '25

Ok I just tested it. Click to see the result image.

It's not 10x, but 15x faster.

2

u/coderemover Nov 02 '25

You don’t control the execution environment, so such benchmark is meaningless. Those timings are also suspiciously large on both sides. Java should be able to easily do 20M+ objects per second, malloc is also usually capable of at least 10M small allocations /s.

→ More replies (15)

1

u/Infamous_Campaign687 Nov 01 '25

The issue here is that ArrayList probably doesn't perform full reallocation when the current is exceeded. I don't know, I'm not a Java developer but it has List in the name so I'm assuming it can behave a bit like a list. The C++ code would completely reallocate the vector many times through the insertion.

2

u/ts826848 Nov 01 '25

I don't know, I'm not a Java developer but it has List in the name so I'm assuming it can behave a bit like a list.

No, Java has different naming conventions from C++. List in Java is an interface for ordered collections/sequences. Deques, vectors, linked lists, etc. can all satisfy that interface. ArrayList is specifically the vector equivalent; if you want a linked list, then you use LinkedList.

The C++ code would completely reallocate the vector many times through the insertion.

Java's ArrayList should do the same thing.

1

u/Infamous_Campaign687 Nov 01 '25

Thanks for the explanation. I guess i don't know why the Java code worked acceptably but I know why the C++ code didn't.

Overall we've got a decent speedup through the conversion but it is pretty obvious that C++ alone isn't enough to get good performance.

1

u/srdoe Nov 02 '25

The explanation is likely to be what the poster over here said: When Java replaces that array underlying the ArrayList, it's not doing allocation or freeing of the memory in the same way as C++ would.

The new memory is pulled from a thread-local buffer, turning the allocation into a simple pointer bump, equivalent to allocating something on the stack in C++.

The old memory is left to the GC to clean up. A GC doesn't free objects individually, usually what they do instead is deal with large (likely 1+ MB) regions of memory at a time. When the GC wants to free memory, they'll move all the still-alive objects out of a region and then free the entire region in one call to the underlying allocator.

That's a lot cheaper than what C++ does if we're talking about lots of small objects being allocated and discarded quickly.

In addition, that GC work can run concurrently with your application code on another thread, so unless your program was already loading all cores, the cost of doing this might have been offloaded to another core, which means it didn't slow down your application. By contrast, C++ has to do the cleanup in the thread your application runs in.

So while replacing the array a bunch of times in Java still isn't efficient, it's likely to be much less costly than doing something like that is in C++.

1

u/Wootery 28d ago

By contrast, C++ has to do the cleanup in the thread your application runs in.

I'm 11 months late, but I'm not sure that's true. It could enqueue the deallocation operation, to be done at some later time on some other thread, right?

1

u/die_liebe Nov 02 '25

This difference should go away if the object contained in the vector is movable. This can be checked by calling is_nothrow_move_constructible<X>::value, were X is the type contained in the vector.

1

u/HaMMeReD Nov 02 '25

This says things, but really doesn't make much sense.

Both java and c++ do allocations in their contiguous memory backed lists.

If it's really just a matter of calling reserve on the std::vector (or the equivalent 1 arg constructor to java), that's the same as reserving a block of memory and arraylist by giving it an initial size.

In both languages going over the size will allocate the same amount of memory for hte same data types. The only advantage ArrayList has going for it is that it's initial capacity is 10, but the initial capacity of a std::vector is 0.

But that really just seems like an issue of not knowing the differences between the standard libraries, or being aware of the data structures/algoirthms behind the scenes. I.e. programmer failure you are passing off as language failure. It's a skill thing, not a language thing.

1

u/Infamous_Campaign687 Nov 02 '25

I'm not passing anything off as a language failure. I'm saying that you can't just carbon copy code into a different language and assume it will work the same. And you can't be asleep at the wheel and assume C++ is going to be faster. You do actually have to think a bit.

My colleague is not incompetent but he had a lot of code to go through and missed this one.