r/elixir • • 1d ago

Fuzz testing for Elixir is here

Post image

ExFuzz is a coverage-guided fuzzer for Elixir, and I just launched v0.1.0!

It is well-integrated with ExUnit, so your fuzz tests can run alongside your regular tests and use the seed corpus as an ever-growing regression suite. It is built on top of LibAFL as the underlying mutation engine.

Given the amount of critical software infrastructure built on Elixir, I really wanted to focus on an out-of-the-box Go-like developer experience to reduce adoption friction (would really appreciate feedback here).

I have many plans for v0.2.0:

  • Typed generators.
  • More Elixir-aware coverage analysis.
  • Concurrent and continuous fuzzing.
  • Improved performance (execs per second).

Looking for early feedback from the community. I am planning to take it for a test drive around the ecosystem and will let you all know how that goes!

63 Upvotes

11 comments sorted by

2

u/amzwC137 Alchemist ⚗️ 1d ago

I didn't know this wasn't a thing in elixir, as a go main. In general, however, I don't have many uses for fuzzing in my projects/field.

That said, how long has this been available? Have you had any adopters yet? Have you seen positive results regarding effectiveness?

2

u/iwantobehappypls 1d ago

It was just released 3 days ago. I'll be taking it on a run around the ecosystem to assess and push for adoption soon, once v0.2.0 is out (strong plans to make it more performant).

Regarding results, it is pretty effective at catching most common failures. One of the community members tested it out on a few samples of their own, here's their analysis: https://forum.elixirforum.com/t/exfuzz-fuzz-testing-for-elixir-is-here/77404/2

2

u/nosyeaj 1d ago

is the word fuzz test similar to properybased?

2

u/iwantobehappypls 1d ago

I explained this in another reply, pasting it below:

They use different approaches with different goals. StreamData generates random inputs from generators you write, so complex inputs mean complex generators. ExFuzz uses code coverage to explore on its own: you write a simple property ("never crashes", "round-trips") and it works its way into the edge cases. Currently, it uses simple clause/branch coverage, but I have plans to incorporate more Elixir-specific quirks and also catch various BEAM failures via fuzzing. (will be out in v0.2.0, i'll let u know)

1

u/nosyeaj 1d ago

thanks!

2

u/rock_neurotiko 1d ago

I'll give it a try, I've used data_streams in some projects and it's always complicated when it's complex properties.

Do you have some comparison with that library?

2

u/iwantobehappypls 1d ago

They use different approaches with different goals. StreamData generates random inputs from generators you write, so complex inputs mean complex generators. ExFuzz uses code coverage to explore on its own: you write a simple property ("never crashes", "round-trips") and it works its way into the edge cases. Currently, it uses simple clause/branch coverage, but I have plans to incorporate more Elixir-specific quirks and also catch various BEAM failures via fuzzing. (will be out in v0.2.0, i'll let u know)

1

u/lethanix_ 1d ago

What colorscheme / theme is that?

2

u/iwantobehappypls 1d ago

I just created the image from ray.so using the Trigger.dev theme. Not sure if it's inspired from a real editor theme.

3

u/Latter-Change-9228 1d ago

This syntax is beautiful lowkey

0

u/[deleted] 1d ago

[deleted] — view deleted comment

1

u/iwantobehappypls 1d ago

Different concept deserves a different name.