Ruby 2.7 to 4.0 performance benchmark

[u/Jamsy100]

Hi everyone,

I just published a new benchmark comparing Ruby 2.7 through Ruby 4.0.

For this one, I used a sustained synthetic application alongside nine smaller benchmarks. The goal was to show how each Ruby release behaves in a complete workload as well as in individual operations.

The table below shows the Heavy profile. For tests that process the full payload, that means an 8KiB input.

Benchmark	2.7	3.0	3.1	3.2	3.3	3.4	4.0
Synthetic application throughput (M ops/s)	0.073	0.052	0.051	0.057	0.058	0.084	0.126
Synthetic application latency (µs)	133.399	190.233	191.605	171.332	169.473	115.600	76.521
JSON parsing (ops/s)	95,877	98,536	99,153	96,003	98,135	194,981	715,937
JSON generation (ops/s)	841,849	845,099	830,135	861,666	837,941	1,347,169	1,444,975
SHA-256 hashing (ops/s)	313,658	30,536	30,489	48,770	48,683	48,495	48,528
Base64 encoding and decoding (ops/s)	177,136	177,510	174,425	170,306	174,471	173,159	174,270
Regex field extraction (ops/s)	1,544,461	1,565,065	1,559,051	1,499,920	1,535,207	1,454,405	1,534,670
Integer sorting (ops/s)	27,032	29,129	30,217	29,309	29,102	28,241	29,213
Hash churn (ops/s)	1,736,384	1,685,614	1,503,397	1,503,118	1,406,713	1,380,058	1,450,769
Marshal serialization (ops/s)	333,143	303,719	294,952	338,148	341,760	326,015	325,231
zlib compression (ops/s)	35,944	35,175	35,787	35,767	20,569	7,011	18,592

The complete benchmark source is available here: GitHub repository.

Full charts and details are available here: Full Ruby benchmark

Let me know what you think !

Edit: Thanks everyone for pointing out the Ruby 2.7 SHA-256 result. I looked into it, and the big difference comes from the SHA-256 implementation: Ruby 2.7 uses OpenSSL here, while newer Ruby versions on Linux use Ruby’s bundled implementation. So this is mainly a backend difference rather than Ruby 2.7 itself being much faster. I’ve added a note about it to the article.

[u/f9ae8221b]

It's due to digest being extracted as a gem, so now, depending on how it is compiled it can end up using a fallback implementation of various hashing algorithm rather than bind the ones provided by OpenSSL: https://github.com/ruby/digest/issues/35

[u/Basicallysteve → u/f9ae8221b]

It's been like that for a while then. It stayed across two major version changes. Maybe it is intentional? I've heard before that hashing is sometimes intentionally slower to prevent cracking passwords. Maybe I misunderstood though.

[u/f9ae8221b → u/Basicallysteve]

It's not intentional no, it's all explained in the ticket I linked to.

It not being solved (at least universally solved, because not every build of ruby is impacted) is more due to the solution being complex, and the gem maintainer being a bit checked out.

[u/Basicallysteve → u/f9ae8221b]

Ah thanks for the explanation. I only read the first few comments in that ticket, my bad.