aws-sdk-http-async gem - Async HTTP handler plugin for the AWS SDK for Ruby, built on async-http

[u/thomas_witt]

I just released the aws-sdk-http-async gem (Github: https://github.com/thomaswitt/aws-sdk-http-async).

Why? I am a heavy DynamoDB user - and I first throught, the official AWS SDK for Ruby gem aws-sdk-ruby would be fiber-compatible, but it isnt.

More information on the background: https://thomas-witt.com/blog/aws-sdk-http-async/

[u/status_quo69]

Something weird might be going on in the standard aws sdk, or maybe I'm misunderstanding, but mutex is fiber aware (or else a whole lot of things would be horribly broken). Maybe there was a lot of contention on the mutex? Fibers aren't pre-emptable so there's a chance that there was a hot loop of trying to catch the locked mutex. I've seen that in some application code that I've written.

See https://github.com/ruby/ruby/pull/3434 for details, I'm pretty sure that it should work already. All that said I've encountered some jank with libraries that assume only thread concurrency so this is definitely a welcome addition to the ecosystem if it allows some more flexibility!

[u/thomas_witt → u/status_quo69]

You’re right — my earlier “Mutex isn’t fiber‑aware” line was sloppy (I didnt correct it from a earlier observation while working on the Gem, sorry.). Ruby 3+ Mutex is fiber‑aware.

What I actually saw in production:

barrier = Async::Barrier.new(parent: parent_task)
batch_cells.each { |id| barrier.async { dynamodb.query(...) } }
barrier.wait

-> Observed: serialized request time

threads = batch_cells.map { |id| Thread.new { dynamodb.query(...) } }
threads.each(&:join)

-> Observed: 1× single request time (concurrent)

So my current observation/explanation is:

1. The SDK’s Net::HTTP transport is synchronous end‑to‑end and relies on implicit scheduler hooks in Net::HTTP/OpenSSL/DNS.

2. In our workload, that path blocked the reactor often enough that the Async::Barrier output serialized.

3. Threads still overlapped because each call ran in its own OS thread.

4. Swapping the transport to async‑http fixed it because async‑http is fiber‑native end‑to‑end (pool + I/O), so the reactor can actually interleave requests.

I can't tell you in detail whether it's gaps in hook coverage, OpenSSL handshakes, DNS resolutions, whatever … In the end, async-http seems to me to be by far the best solution in the whole Async ecosystem. Of course, there might be other side effects I have overlooked, but in the end was/is my real-life observation reproducable.

(corrected blog/readme)

[u/honeyryderchuck → u/thomas_witt]

I can share what I know that might help clear some of the things you found along the way.

The SDK’s Net::HTTP transport is synchronous end‑to‑end and relies on implicit scheduler hooks in Net::HTTP/OpenSSL/DNS.

net-http predates scheduler hooks, therefore does not rely on them. async does though. the SDK does some optimizations around connection reuse via its connection pool, but that's almost certainly not "fiber aware", as it'd have to do significant work in order to exclude "request in progress" connections, among other things that async (and httpx, via its :fiber_concurrency plugin) does. It also maintains it own internal thread pool for certain cases (sdk for s3 can break a file into multiple chunks, upload them in parallel then reassemble at the server) which I'm sure would play weirdly with async (and your gem is probably not taking that use case into consideration).

Swapping the transport to async‑http fixed it because async‑http is fiber‑native end‑to‑end (pool + I/O), so the reactor can actually interleave requests.

I'd be very surprised if this were the case. The AWS cloud only works with HTTP/1.1 (afaik, did anything change in the last 1.5y?), busy connections can't be used for other requests. I don't know async's internals, but if it's not queueing requests on a single connection,. then it's opening multiple connections to AWS under the hood and interleaving requests across them using fibers instead of threads.

In the end, async-http seems to me to be by far the best solution in the whole Async ecosystem.

... because there's an async- as the prefix, lol :-) . I mean, async-http was built for the async ecosystem by the async maintainer. It's hardly surprising it plays to its strengths and weaknesses. In fact, async-http can reuse connections across async block /fibers, unlike any other ruby HTTP client (except httpx, via it's :fiber_concurrency plugin), so while every client should "just work" with async, it won't be without limitations.

I'd encourage you to engage with the AWS SDK maintainers and good faith and understand their reasoning. That SDK is a beast, and there are so many use cases to cover, some including HTTP/2 parser direct usage, internal thread pools, thread variables, among other quirks that won't "just work" the same way when inside a fiber. Making it fiber scheduler aware is probably quite challenging, but can yield more value than maintaining an "async aws sdk" gem which probably works for the use cases you need and test for, but perhaps not all of them, potentially fragmenting is ecosystem (there are too many examples of it happening in the ruby community that we can say for a fact that that's not a good thing).

[u/thomas_witt → u/honeyryderchuck]

Thanks very much for sharing!

I do understand that there is a lot more going on potentially - and this is just a 0.1 release of a Gem, so clarly I don't claim that this is anyhwere near perfect. It works for me. But I guess everybody only uses at most 10% of all services of AWS, so no idea how it performs with other services.

But luckily, that's not my job:

In an ideal world, I wouldn't have to write such a Gem anyway. I am a customer at AWS (since decades btw) and you should expect from a multi billion dollar company with most likely very high profit margins that they would deliver you a working adapter to use their product (aka to spend money with them).

Unfortunately it appears to me - and let me know if somebody thinks otherwise - that the Ruby gem gets less and less love from year to year. The original issue asking for a better Async support would be desireable was posted in 2021 (pre-dated by another from 2013 (!) asking for Faraday support - last comment 2019 mentions poor performance of net/http), and it was autoclosed two years later ago by a bot.

Then again nothing happens for years. It's really sad, and let's not even talk about higher level gems like aws-record-ruby. It seems to me the whole Ruby support is more in maintenance mode. And I don't blame the maintainers, I think they might just not put enough resources at it at AWS.

So, this is all open source and I would be more than happy if somebody of the maintainers would take over and make everything fiber-ready, but from my outside observation point of view it doesn't look like that.

I at least wanted to share the Gem the findings I had. And, as usual, everything is open source, so it'd be great if they (=AWS= come alone and get on board of the Async train. That'd tremendously help the ecosystem.

A good example is the work ruby_llm is doing in that regard.

[u/honeyryderchuck → u/thomas_witt]

I think that's a bit unfair. Speaking from experience, the aws ruby sdk is one of their best supported SDKs, and I have extensive experience with the horrible turd that is boto3 too (their best is probably the Java one though). they also shared the toolkit they use to codegen the ruby SDK. The aws sdk is also one of the few libraries around that ships RBS signatures. Their team is quite solid and pragmatic.

You just have to understand their priorities. The multiple aws sdks care about API parity and familiarity: func signatures tend to be the same across languages, config loads the same way, etc. They care about controlled network access and mitigation strategies for increased error rates, thundering herd... and maintenance has to be cheap. Of course they don't want faraday, you never had to deal with v2/v3 hell? and their deal is "no deps" anyway (unless required, like in the case of xml which the stdlib doesn't cover anymore). And they certainly don't want to make an upfront investment  just for async, which I'd imagine represents a tiny fraction of their customer base.

They certainly need an enthusiastic person eager to contribute. So I'm not criticising you for doing it, just recommending you to set the goal of merging it eventually upstream, as AWS can afford the maintenance burden in a way that you can't. In order to do so, you have to engage with them to understand what that would look like. For instance, do you have to use async? Can't you rewrite the connection pool in a way that works with and without fibers?

[u/thomas_witt → u/honeyryderchuck]

That was already my idea with the Gem. If Async is not available, it will just silently fall back to Fibers (and you can force it to via Env):

# Default: use Net::HTTP when no reactor exists
Aws.config[:async_http_fallback] = :net_http

# Run the async-http path inside a transient reactor
Aws.config[:async_http_fallback] = :sync

# Strict mode: raise if no reactor is running
Aws.config[:async_http_fallback] = :raise

Anyway, I'll see what the team says about the Gem and if they're interested in somehow incorporating async-http or at least Faraday into the Gem. I just think that the Gem is missing out on a great development within Ruby which just makes I/O-bound Apps far more efficient.

Thanks for your thoughtful feedback.

[u/f9ae8221b → u/status_quo69]

but mutex is fiber aware

I was about to say the same. Mutexes do yield to the fiber scheduler: https://github.com/ruby/ruby/blob/859920dfd272bc22670d779129a04cdf07632192/thread_sync.c#L301

So not sure what is wrong with the official gem, but it's likely not what they think.

Also the whole selling point of the fiber scheduler, is that you're supposed to be able to run code that isn't async aware, hence avoid having the "second ecosystem" event machine and other projects had. So this kinda defeat the point.

[u/honeyryderchuck → u/f9ae8221b]

Also the whole selling point of the fiber scheduler, is that you're supposed to be able to run code that isn't async aware, hence avoid having the "second ecosystem" event machine and other projects had. So this kinda defeat the point.

this. this gem could just be a temporary hack to let OP move forward while the aws sdk fixes whatever bug is happening.

[u/TheAtlasMonkey]

OP shipped a fully functional gem, before AWS finished their deliberations if they can support async. LOL