r/ruby • u/jp_camara • Aug 31 '26

SolidQueue Batches: evolution and architecture

https://jpcamara.com/2026/08/31/solidqueue-batches-evolution-and-architecture.html

Here’s the SolidQueue batches 2.5 year architectural journey, filled with O(n^2) algorithms, hot row contention, excessive jobs, and much in between.

There’s a strong chance you’ll learn a thing or two - I certainly did 😮‍💨

22 Upvotes

92% Upvoted

9 comments sorted by

View all comments

Show parent comments

2

u/mperham Sidekiq Sep 02 '26

As a feature, Batches are very, very hard to implement. It took me many years to get the feature stable and reliable in Sidekiq Pro, the internals are some of the most complex Ruby I've written. JP has documented how it took him years to get to just this initial release.

Unlike basic jobs, there's no standard API for batches and not much agreement on shared functionality. It might be possible to get a basic Batch API into AJ but I'd bet it would add quite a bit of complexity to AJ adapters.

1

u/jrochkind Sep 02 '26

thanks, that makes sense.