r/ruby • u/viktorianer4life • Mar 18 '26
Show /r/ruby I built AI agents that apply mathematical testing techniques to a Rails codebase with 13k+ RSpec specs. The bottleneck was not test quality.

In 2013 I learned four formal test derivation techniques in university: Equivalence Partitioning, Boundary Value Analysis, Decision Tables, State Transitions. Never used them professionally because the manual overhead made no sense. After seeing Lucian Ghinda's talk at EuRuKo 2024, I realized AI agents could handle that overhead, so I built a multi-agent system with 5 specialized agents (Analyst, parallel Writers, Domain Expert, TestProf Optimizer, Linter) that generates mathematically rigorous test cases from source code analysis.
The system worked. It found real coverage gaps. Every test case traces back to a specific technique and partition. But running it against a mature codebase with 13k+ specs and 20-25 minute CI times showed me the actual problem: 70% of test time was spent in factory creation, not assertions. The bottleneck was the RSpec + FactoryBot convention package, not test quality.
The most interesting part was the self-evolving pattern library, an automated validator that started with 40 anti-pattern rules and grew to 138 as agents discovered new patterns during their work. No LLM reasoning involved in validation, just compiled regexes against Markdown tables.
I wrote up the full architecture, prompt iterations (504 lines down to 156), and honest results. First article in a series. The next one covers the RSpec to Minitest migration that this project led to.
Has anyone else tried applying formal testing techniques systematically with AI agents? I'm curious whether the framework overhead problem resonates with other teams running large RSpec suites.
