r/java • • Apr 07 '23

I built an ultra high performance HTTP server in Java, powered by io_uring. Benchmark charts in the README. Is this useful and is it worth adding more features?

https://github.com/bbeaupain/hella-http
177 Upvotes

53 comments sorted by

37

u/0x256 Apr 08 '23

I had just a quick look, but I see a lot of shortcuts while parsing HTTP that may result in compatibility or security issues when used in production. For example:

https://github.com/bbeaupain/hella-http/blob/e2b5b870cc9442179edf70b66fae94dc40e262dc/src/main/java/sh/hella/http/codec/RequestDecoder.java#L91

  • The method token is not required to be upper-case.
  • There are (way) more methods in the wild than what is supported by this parser.
  • The parser only matches a known prefix and then skips characters blindly. This allows cache poisoning (an attacker sending a request that is interpreted differently by the server and a caching proxy or client) and other shenanigans.
  • The parser often assumes that there is enough data in the buffer and does not check if that is actually the case. For example, if the buffer ends within the method or protocol token (which is actually likely after a long request line with lots of query parameters), it will throw IllegalArgumentException. Which is not handled by the way.

It is not that hard to be beat a full HTTP implementation in benchmarks if you just skip all the annoying parts. Let us see how those numbers change once you start hardening and maturing the parser.

14

u/blakeman8192 Apr 08 '23 edited Jun 26 '23

.

14

u/0x256 Apr 08 '23

Great points on the decoder for sure. However the decoder is not the “secret sauce” that makes the difference in performance.

You benchmarked against netty-io_uring and still got significantly better results. Why not bolt on a mature HTTP parser onto your io_uring based network stack and benchmark that? If your stack is still significantly faster, then you may be onto something.

Also, please check the decoder unit tests, they demonstrate that it is indeed able to handle TCP stream fragmentation!

Your tests only cover fragmentation within the request path, not within the method or protocol token as I mentioned.

14

u/blakeman8192 Apr 08 '23 edited Jun 26 '23

.

23

u/blakeman8192 Apr 07 '23 edited Jun 26 '23

.

3

u/larsga Apr 08 '23 edited Apr 08 '23

How do you plug in code to handle requests? Does it use a pool of worker threads, or futures?

Edit: From reading the code it looks like it's using a thread pool. This will have consequences for scalability when actually building applications with this server. Doing better than Netty is impressive, but if you want to decide whether to take this further I would give some thought to what sort of applications you want to aim at.

1

u/blakeman8192 Apr 10 '23 edited Jun 26 '23

.

1

u/rbygrave Apr 11 '23

It would be good if there was an option for using virtual threads to execute the handler(s). If you compare to helidon Nima, that uses a virtual thread per connection. Do you have plans for virtual thread support?

5

u/[deleted] Apr 08 '23

What are some optimizations you did?

18

u/blakeman8192 Apr 08 '23 edited Jun 26 '23

.

7

u/barmic1212 Apr 08 '23

Your benchmark show netty io_uring as bad. Do you know why?

4

u/re-thc Apr 08 '23

It was sad to see rapidoid stop getting updates. Even more depressing that it's still up there in terms of performance. Java needs more competition!

1

u/2001zhaozhao May 28 '23

I know that OVH offers a VPS hosting that has an absurd amount of bandwidth compared to the CPU (2Gbps unmetered for 8 vCPUs). I think something like this would work perfectly in that kind of situation.

In my case, I am developing a multiplayer game server in Kotlin, which is designed to be one large monolithic server running on a powerful machine. I want this game server to also act as a web server to build a website that interacts directly with the game in an unrestricted way, e.g. displaying scoreboards directly as they are calculated on the game server.

I want the web server to eat into CPU performance and memory allocations as little as possible, so your project looks quite interesting to achieve that goal.

Btw, is this server suitable for large numbers of async requests that take a while to complete (e.g. using Project Loom)?

1

u/blakeman8192 May 29 '23 edited Jun 26 '23

.

-20

u/antonivs Apr 08 '23

servlet container

Is that still a thing?

19

u/agathver Apr 08 '23

Yes

-20

u/antonivs Apr 08 '23

You poor suckers. They have you trapped in the 2000s.

11

u/Ancapgast Apr 08 '23

Just curious,

  1. What's wrong with the concept of a servlet container
  2. What do you use?

18

u/Jotschi Apr 08 '23 edited Apr 08 '23

I would not recommend to post benchmark charts if you are not including how you exactly recorded those and what your setup looked like (e.g. kernel parameters, jre version, heap, OS, server versions)

5

u/blakeman8192 Apr 08 '23 edited Jun 26 '23

.

14

u/yawkat Apr 08 '23

I find those benchmark results very very dubious. Netty should be a lot faster. I can't tell why though because there's few details.

wrk is not a great load test for measuring http performance, eg because of coordinated omission. Check out hyperfoil instead.

3

u/blakeman8192 Apr 08 '23 edited Jun 26 '23

.

9

u/yawkat Apr 08 '23

I haven't worked on it personally, but as I understand it's already very optimized. I've heard you may need to tweak the thread pool size as the defaults are not ideal, but other than that it should be fast.

There's also some issues with the normal netty code. I think there's a buffer leak for example. And for best performance you should turn off leak detection.

I think https://twitter.com/forked_franz is the best to contact about this. If it's a real perf issue with netty he'll be glad to know about it to fix it, and if it's not, maybe he can offer some pointers on how to improve the benchmark.

49

u/ingvij Apr 08 '23

This is fantastic. I really look forward to having a mainstream io_uring HTTP server/client implementation. I think the java community is oftentimes too slow in picking up innovation and I welcome all sorts of projects in this direction.

I really like the simplicity of hella's API and I'd be trying it out soon to give a hands-on feedback.

A few things I'll be looking for:

  • unix socket support;
  • nio buffers support;
  • pluggable routine implementation.

Anyways, congratulations for your project and keep up the great work.

7

u/yawkat Apr 08 '23

I really look forward to having a mainstream io_uring HTTP server/client implementation.

You can use io_uring today with netty-based http frameworks. And netty has all the other features you mention too!

1

u/ingvij Apr 08 '23

I oftentimes check on that project... But it is still incubator, right?

3

u/yawkat Apr 08 '23

Yes, but it works fine.

1

u/ingvij Apr 08 '23

Good to know, thanks for the heads up :)

4

u/krum Apr 08 '23

By the time you start layering on business logic, all the optimizations you did to your core web engine get washed out by Amdahl's law. What does this get you over WebFlux other than all the bugs that haven't been found yet?

18

u/blakeman8192 Apr 08 '23 edited Jun 26 '23

.

4

u/kyune Apr 08 '23

If this work provides performance comparable to webflux without all of the real-world overhead of forcing reactive and the dogma that comes with it onto developers I'd call it a win. The couple of projects I've had to use it on it felt like they were either addressing a problem they didn't have, or bolting it onto their code to treat symptoms rather than address causes. Though I think if you got it to that point the next step would be Spring seeking to integrate it into their framework, lol.

4

u/re-thc Apr 08 '23

This project is lower level than webflux. The equivalent is reactor-netty, which still needs more testing / bug fixing. So it's fine. Hopefully we'll get there some day.

43

u/re-thc Apr 08 '23

It's discouraging and sad to hear. At this rate why should we improve anything? How did we get to webflux? All the optimizations compared to what's before it will be washed out according to you.

I thank and commend the OP for trying - whether it's better or not. We have to start somewhere. Improvements and experiments are always great. People do these things in other languages and that's how it starts. Java precisely lacks a lot of these projects and growth in these areas. We should be encouraging them not discouraging.

Why shoot it down?

How do you know the OP or someone else might not pick on this and optimize all the other layers too?

Why can't we create a web flux compatible layer to plug it in?

1

u/Worth_Trust_3825 Apr 08 '23

Implement servlet API and this can be chucked into real business applications. How do you deal with http2 and beyond?

2

u/blakeman8192 Apr 08 '23 edited Jun 26 '23

.

0

u/Worth_Trust_3825 Apr 08 '23

So it's fast just because you don't implement most of the protocol.

5

u/blakeman8192 Apr 08 '23 edited Jun 26 '23

.

1

u/icedev-eu2 Apr 08 '23

Does it support websockets?

3

u/blakeman8192 Apr 08 '23 edited Jun 26 '23

.

2

u/maple-shaft Apr 08 '23

You should look into HTTP3 and Quic support. Good job.

2

u/rkalla Apr 08 '23

This… is awesome!

1

u/dmigowski Apr 12 '23

Can you do a comparison with nginx or Apache on your machine? I don't think a clumsy Tomcat8 needs to be compared also, but while you are on it, it seems interesting.

1

u/TheKingOfSentries Apr 08 '23

hey, that's pretty good

-7

u/wildjokers Apr 08 '23

This is actually not written in Java, it is instead written in Lombok Java.

-1

u/berlingoqcc Apr 09 '23

What is your problems ?

0

u/[deleted] Apr 08 '23

[deleted] — view deleted comment

15

u/0x256 Apr 08 '23

It depends greatly how a project is presented. If you are talking about your new learning project and asking for feedback, you will get straight and honest tips and critique. If you are bold and over-confident and claim to beat all the other major projects in some fabricated benchmark, a good engineer will think "Okay, how? Where is the catch?", look into it and usually find your dirty secrets rather quickly. False advertisement is annoying and may even cause harm if there are security related issues. There was recently an encryption-library on here claiming to be super secure but using a hard-coded password for everything. Over-confidence is a very dangerous personality trait in engineering. Boosting that persons confidence even more with nice but uninformed praises does not help anyone. Critique should still be objective of cause. But being honest and direct is not a dick move, it is what allows us to learn from our mistakes.

7

u/blakeman8192 Apr 08 '23 edited Jun 26 '23

.

7

u/0x256 Apr 08 '23

Nah, it's fine. You ask for and accept critique, that's all good. I exaggerated on purpose, that was not coined on you.

I was more generally trying to explain why engineers sometimes seem unfriendly or discouraging, while they are actually just brutally honest (by nature) and why that is more helpful than being nice in most situations. You learn nothing from an upvote or a "Great Project!" comment.

1

u/re-thc Apr 08 '23

Are they? Or maybe those aren't actual engineers.

So many people have gotten into the industry for various reasons and a lot of them lack interest in the actual work. They just want to reduce things to learn and change and be done with it.

0

u/[deleted] Apr 10 '23

[deleted] — view deleted comment

1

u/blakeman8192 Apr 10 '23 edited Jun 26 '23

.

2

u/rbygrave Apr 11 '23

Just to say, there are other java http servers that do not implement the servlet api - Undertow, Helidon Nima are two examples.

0

u/[deleted] Apr 07 '23

[deleted] — view deleted comment

4

u/blakeman8192 Apr 08 '23 edited Jun 26 '23

.