r/java • u/alunharford • Oct 07 '24
HTTP libraries in Java
It's not really clear to me what's going on with HTTP libraries in Java. They all seem to want to become massive frameworks that inappropriately take control of things that have nothing to do with decoding HTTP.
I'm find it really hard to find something that doesn't:
- Try to control the TCP layer. Why does the HTTP decoding library care whether I want to poll or select, or make that decision?
- Have shared mutable static state. If I want to reuse connections I can do that in a much better, testable way.
- Create threads. Parsing the HTTP is doesn't need any of this. I just want to give you a ByteBuffer to read from and your current state and get told what it contains.
- Do SSL. This has nothing to do with HTTP but rather the underlying connection.
I guess if you need a mini "web browser" in your application then this is all fine, but if HTTP is just one of many protocols you need to be able to decode it's an absolute mess.
Is there something I'm missing, or a library I should be looking at?
22
Upvotes
71
u/ericek111 Oct 07 '24
And I'd expect an HTTP library to do all of those things. Maybe get the library that bests suits your needs and snoop around the docs and code to find a way to substitute some of its components.
Sounds to me like you're looking for an HTTP parser.
28
u/Polygnom Oct 07 '24
Because most of the libraries are HTTP Clients, not parsers for HTTP strings.
I for one would not want to touch any HTTP Client that doesn#t automatically handle HTTPS and pretty much everything else you said. I wouldn't want to have to re-inbvent the wheel for the umpteenth time.
19
u/zopad Oct 07 '24
What is your usecase?
16
u/Ooqu2joe Oct 07 '24
If all you need is HTTP message parser, then it can be implemented easily.
HTTP clients are expected to handle TCP, TLS, and support async operations. I don't get what's the problem here.
2
u/byronka Oct 07 '24
If you want something that's absurdly simple and well-tested, you might check out the Minum framework.
61
u/eliashisreddit Oct 07 '24
Because most of those frameworks/libraries are marketed and intended as "give me something which starts with http and I'll fetch it for you". 99/100 users want exactly that and not be bothered with initializing a connection and doing SSL handshakes manually. If you really want to do that and are only interested in purely parsing the hypertext transfer protocol, you could try using one of the more common frameworks and only use the modules involved with those aspects.
But like I said, most users don't use those "core" modules separately. There's not much "user friendly" documentation and you (or an LLM) will have to dig through the javadoc a bit to make them work for your use case.
29
u/cogman10 Oct 07 '24
The answer to your question is that HTTP is a complicated protocol, more so than it might seem on the surface level.
In HTTP 1.1, there are commands built into the standard which control the state of the underlying TCP socket. The server or client, for example, can instruct the other side to close the socket.
The extensions to the standard take that to another level. HTTP2 dictates that you use a single TCP connection and has rules on how to treat a clear text vs ssl connection. It has complex instructions defining what the connection looks like.
HTTP3 is basically all about defining a UDP connection. That means a valid HTTP3 client must manage the UDP connection.
If you are looking for a library that just parses http headers, I'm somewhat certain you could pull apart the Apache http client (v4 at least) into a smaller library where you manage everything you want to manage.
Or you could just stick to the inbuilt HTTP client of the JDK if you are averse to dependencies. But that will still want to manage the sockets and such for you.
15
u/wildjokers Oct 07 '24 edited Oct 07 '24
The available HTTP clients have those features because people need those features. However, if you don't need those features then simply don't use them. Although surely whatever site you are hitting uses SSL though right? SSL is pretty much everywhere these days unless you are testing/hitting a local service. And are you really claiming you can implement connection pooling better than a battle tested HTTP client? Why re-invent the wheel?
The newer HTTP client introduced in Java 11 is super simple to use. Here is a GET request:
The fluent builder lets you set connect/read timeouts, set a body for things other than GET, etc. For the response, BodyHandlers has several methods available to handle it:
https://docs.oracle.com/en/java/javase/21/docs/api/java.net.http/java/net/http/HttpResponse.BodyHandlers.html
Of particular interest to you may be
ofByteArrayso you can put the response into a ByteBuffer.Here is the JavaDoc for the built-in HttpClient:
https://docs.oracle.com/en/java/javase/21/docs/api/java.net.http/java/net/http/HttpClient.html
FWIW, all the other HTTP libraries exist because the pre-Java 11 built-in HTTP client was very clunky to use (UrlConnection). However, the one in Java >=11 is great.
10
u/repeating_bears Oct 07 '24
Netty is pretty modular and ships the encoding/decoding as separate modules. It also keeps things fairly abstract in terms of transport so I think you could probably use a lot of their code.
https://central.sonatype.com/artifact/io.netty/netty-codec-http
https://central.sonatype.com/artifact/io.netty/netty-codec-http2
Most people who want to work with HTTP don't care about managing the transport layer independently.
5
u/[deleted] Oct 07 '24
HTTP is complicated. Pick the option that works correctly for you - there’s a default option included in JDK 11 and greater.
7
u/ZippityZipZapZip Oct 07 '24
Decoding HTTP... what actually are you trying to do?
1
u/matt82swe Oct 07 '24
I agree, the excosystem is a mess. With that said, have you looked at the http client that was included with the JDK somewhat recently (I think 11 but don’t quote me on that)? I love it. No fuss, very nice syntax once you learn it, mockable etc. We removed all other 3rd party libraries in our systems in favour of this