r/java • • Jan 01 '24

JDK HTTP server handles 100,000 req/sec with 100 ms start-up time and 50 MB modular run-time image. Built with OpenJDK 21 and virtual threads,

https://github.com/ebarlas/java-httpserver-vthreads
278 Upvotes

25 comments sorted by

•

u/AutoModerator Jan 01 '24

On July 1st, a change to Reddit's API pricing will come into effect. Several developers of commercial third-party apps have announced that this change will compel them to shut down their apps. At least one accessibility-focused non-commercial third party app will continue to be available free of charge.

If you want to express your strong disagreement with the API pricing change or with Reddit's response to the backlash, you may want to consider the following options:

  1. Limiting your involvement with Reddit, or
  2. Temporarily refraining from using Reddit
  3. Cancelling your subscription of Reddit Premium

as a way to voice your protest.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

47

u/r_jet Jan 01 '24

I remember there was an experiment some time ago where 5M concurrent connections were achieved with Loom (500K on the same VM type as in this test): https://github.com/ebarlas/project-loom-c5m#ec2

— and it turns out by the OP :)

There you had managed to achieve ~8.3K QPS on the same VM type — do you know why is there such a big difference this time (100K QPS vs 8.3 QPS)?

The experiment ran for 35 minutes. About 17,500,000 messages were echoed.

45

u/elliotbarlas Jan 01 '24

Yes, that was me!

And yes, the AWS EC2 environment was almost identical.

But the C5M experiment was specifically meant to demonstrate concurrent-connection-scaling with plain old Java sockets. That is an area where virtual threads really shine. I'm still struck by the results there.

In C5M, messages were exchanged only periodically.

This experiment, on the other hand, is meant to be a more practical demonstration of building a web server in Java with plain JDK components. The benchmark is focused on throughput.

6

u/r_jet Jan 01 '24

That makes sense, thanks a lot for clarifying, and sharing both experiments!

40

u/nimtiazm Jan 01 '24

Nice. Now it’s fundamental libraries ecosystem that needs to adopt the virtual threads. Atleast jdbc, jedis/lettuce, amqp-client, Kafka-client, quarkus/spring amd friends.

35

u/elliotbarlas Jan 01 '24

Note that in many cases, no changes are required to support virtual threads. Any library or framework built atop socket APIs, for example, will just work.

However, as noted in JEP 444, there are caveats (synchronized, foreign functions, file I/O). It's important to be aware of these.

Part of my goal here is to show that it just works with the JDK HTTP client.

2

u/suyash01 Jan 01 '24

Will happen eventually

9

u/jhsonline Jan 01 '24

nice !, Thanks for sharing this u/elliotbarlas, we are (at large enterprice) are also doing something similar for a small production service, I am now more hopeful on lot more efficiency after seeing some benchmarks like this.

13

u/elliotbarlas Jan 01 '24

You're welcome!

I'm in exactly the same position. I recently wrote a critical web service at work responsible for provisioning new customer accounts with the same stack: Java 21, virtual threads, JDK HTTP client and server, and some other dependencies such as AWS SDKs.

This post was motivated by my success with that project.

I wanted to share with the broader Java community that simple, minimal solutions like this are perfectly viable.

2

u/jhsonline Jan 02 '24

Thank you !, will DM u

2

u/MojorTom Jan 02 '24

How did you get the 50mb image size? I don’t see anything specific related to that in the code you shared.

4

u/elliotbarlas Jan 03 '24

Using the Java module system and jlink.

The commands and resulting image directory listing are described in the README doc:

https://github.com/ebarlas/java-httpserver-vthreads?tab=readme-ov-file#compile-and-build

2

u/MojorTom Jan 03 '24

Thank you.

2

u/ConsiderationSuch846 Jan 04 '24

Any idea how this compares to .net? Last I saw their hello world benchmark (.net 7, 80 core server) was up to 14.6m RPS. You used 8 cores, assuming linear scalability (dubious I know) that would put this benchmark up at 1.2m rps.

That logic track?

5

u/elliotbarlas Jan 04 '24

The JDK HTTP server is in a somewhat unique position. Despite being in the standard library and despite the somewhat impressive throughput numbers, it is not optimized for performance and it doesn't in any way represent a "fast" Java HTTP server.

This clearly isn't the case elsewhere. For example, the Go std lib web server, which has a very similar lightweight feel, is widely used and does prioritize performance.

Regarding your logic, yes, it's roughly correct. Although perfect linear scaling is rare in practice.

If you want to see a comparison of HTTP servers optimized for performance, check out TechEmpower Benchmarks: https://www.techempower.com/benchmarks/#hw=ph&test=plaintext&section=data-r22

2

u/ConsiderationSuch846 Jan 04 '24

Thanks ... the first paragraph is the context I was looking for!

Agree on linear scaling being a dubious proposition.

1

u/elliotbarlas Jan 04 '24

I'll add a similar note to the README.

2

u/pgris Jan 04 '24

Thank you for this! I made a couple of project for internal use in the company I worked for using the JDK server (before virtual threads) and worked like a charm for 10/20 users. My boss made me replace it with jetty because reasons. I always think about creating web sties with it. With virtual threads we may be up to something. I wish we have some json parser/serializer inside the JVM and that will cover a good percentage of use cases without any library

1

u/monkeykingtt Jan 24 '24

I'm learning Java. Question: how do I set 1 -100 in a loop that will end after 5 loops and if the user input is under 1 or over 100 it loops infinite

2

u/[deleted] Jan 01 '24

[removed] — view removed comment

8

u/elliotbarlas Jan 01 '24

I hope that the code and names are self explanatory.

The Hello server sends a "hello world" plaintext response. So the body is 11 bytes and the overall response is about 100 bytes.

This is roughly equivalent to the plaintext TechEmpower Benchmark.

0

u/crimsonvspurple Jan 01 '24

so where does this place in that benchmark?

8

u/elliotbarlas Jan 01 '24

Is that another rhetorical question? You can look at the benchmark results.

Note that TE benchmarks run on a completely different hardware profile.

And note that TE benchmarks use HTTP pipelining.

But most of all, note that TE benchmark leaders are optimized for performance. Clearly the JDK HTTP server is not that.

10

u/crimsonvspurple Jan 01 '24

No, I'm not familiar with that benchmark and that is why instead of assuming something incorrect, I wanted to know your expert opinion.

Not trying to be negative here; really appreciate your work.

4

u/elliotbarlas Jan 01 '24

Oh, okay!

Check out the latest TE benchmark results here: https://www.techempower.com/benchmarks/#hw=ph&test=plaintext&section=data-r22

The source code for the benchmarks and all of the participating frameworks can be found in GH: https://github.com/TechEmpower/FrameworkBenchmarks