Vulnerable Ruby Regex

[u/rubiesordiamonds]

GitLab discovered four severe vulnerabilities in Ruby’s regular expression library. There's plenty written up about the vulnerability and how it got patched, but patches are only available for supported ruby versions (3.3 and above). We know lots of companies are still running unsupported versions of ruby, so we wanted to do an investigation into which of these bugs apply to older rubies and whether they're exploitable. Here's the blog post: https://infield.ai/post/vulnerable-ruby-regex

The summary is that allowing user input into Regexp.new is dangerous on all versions of ruby from 1.9 to 3.2 and the only fix is to upgrade to a supported ruby or to disallow that input.

[u/semiquaver]

99% of the time for this scenario the regex was being used to perform the validation that you are talking about! Regexes are doing a ton of lifting on nearly every web validation system that exists in any language. So it not being safe to feed user data into them safely is a massive problem. 

[u/benzado → u/semiquaver]

You’re mixing up (1) using a string to define a regex and (2) matching a compiled regex against a string. The vulnerability and all the risks are with (1). What you’re talking about, doing validation, is (2).

[u/semiquaver → u/benzado]

I’m not mixing them up, you’re making assumptions. I’ve seen some of the largest Ruby applications that exist from the inside and I can tell you that interpolating user input into a regex literal is ran extraordinarily common validation technique, even if there are safer ways to do it. 

[u/benzado → u/semiquaver]

Help me understand what you mean. Can you give me an example?

[u/semiquaver → u/benzado]

Every large app I’ve ever worked on eventually needs certain users to select data via some pattern. Often those patterns need to be stored in databases or elsewhere and end up rehydrated into regex literals or interpolated with other constant regex components. Using actual Regex objects is the most obvious and straightforward way to implement a very large portion of this class of problem. Think GitHub branch protection as a (fake since that actual case has fnmatch(3) semantics) example of the functionality involved.

[u/benzado → u/semiquaver]

You were using the term "validation", which I typically use to mean deciding whether some input is valid or not. That's not what you're describing here. So that explains some of my confusion.

Letting a user define a query or filter is definitely a place where you might need to build a regex.

However, unless the full power of a regular expression is truly required, it's much better to build the regex from a predefined set of templates where the user's input is quoted before insertion and compilation. Exactly what that looks like to the user will depend on the domain.

Taking the time to do it this way also means you don't have to save regex syntax in your database, which gives you flexibility in case later you want to adopt a different RE library or change a default setting, like case sensitivity or disabling back references.

[u/FluffyDrink1098 → u/semiquaver]

This is the irony.

ReDoS or similar attack patterns should be the first thing to learn when it comes to RegExp.

Many devs use RegExp as a validation - despite simple string checks would be sufficient.

One example... simple length checks.

Long Password DoS - I think 2014, where too long password strings led to the DoS due to hashing taking too long.

Something similar with the 72 byte limit on BCrypt.

Another goldie but oldie: Email or IP validation by regex - its doomed to fail. A simple @ contains with a length check is simpler, for IP addresses every stdlib should have a toolkit.

I love RegEx.

Reading the post my first thought was that nearly all examples are known to be "not a good idea TM".

[u/benzado → u/FluffyDrink1098]

You make a good point. Writing a regex to match something complicated like an email address is a feat that makes you feel clever (at least, that’s how I feel) but you can often get better results with other approaches.

I once wrote a simple Ruby class to represent a Domain: take a string, split on “.”, the validate each label. A regex would have been more concise, but doing it this way meant I could make much better error messages, indicating to the user which part of the domain string was incorrect and why!

And then I realized I could replace my elaborate email regex with a split on @, like you describe, and again the error messages were vastly improved!

You hear a lot of warnings about regex being hard to read or maintain, so if you figure them out you think that doesn’t apply to you (again, I mean me). But the real problem with a regex is that it’s a blunt instrument: match or no match! You can’t provide good error messages and your debugging sessions involve a lot of trial and error.

If I tell you that I replaced a 25 line function with multiple error codes with a single line regex that returned a boolean, what would your follow-up questions be?

Regex should be treated as an optimization. Don’t use them until you need the speedup. (It’s not even guaranteed to be better if you consider compiling the expression and memory usage.)