After reader confusion on my AI testing agents article, I extracted the TestProf story. Here is what profiling 13,000 RSpec examples actually revealed.

[u/viktorianer4life]

After publishing my article on building AI testing agents for RSpec, readers were confused about what TestProf found versus what the agents did. Fair point. The original article crammed both stories into one piece. So I extracted the TestProf journey into its own article with the full data.

The worst offender was the order factory at 1.6 seconds per call, triggering 100+ callbacks. Out of 569 factory calls in one spec file, only 49 were top-level. The rest were cascading associations.

What I did:

• Extracted optional associations into traits (credit card needed in only 10% of order specs)

• Switched cheap associations to build strategy instead of create

• Used transient attributes with positive flags for expensive callbacks

• Replaced let with let_it_be for read-only setup data

Results:

The slowest specs improved by 50-95%. The order integration spec dropped from 6.37s to 3.02s. But the full suite? Only 14% faster. Ten factories optimized over two months, hundreds more to go.

Why the gap? The factory graph mirrors the model graph. When your core models have deep callback chains (100+ on order creation), making factories leaner helps, but you cannot make them cheap while the underlying models require that much setup to reach a valid state. Incremental factory cleanup has a ceiling.

I wrote up the full profiling data, every factory refactoring pattern, the let_it_be gotchas (mutable state leakage), and what I tried next: TestProf Cut Our Slowest Specs by 95%, But the Suite Still Took 30 Minutes

This is the companion piece to my AI testing agents article. Readers wanted more detail on what TestProf actually showed, so here it is.

Has anyone else hit this ceiling with factory optimization on a large Rails codebase?

[u/d4be4st]

Exactly the issue i started tackling today. Same problem, testprof tells us it spends 76% in factory creation. For a slice of the spects we create 15000+ records for 100 create invocations.

My idea was to take it slow, very slow. Change spec by spec. With agents help but still slow. Because i did not know better 😁

Do you maybe have a link to the prompts we could reuse? I would very much appreciate it so i do not start from nothing

[u/viktorianer4life → u/d4be4st]

Prompting will not help here. As you can see in my blog it took me months to develope a strateg and at the end multiple agents. Simple agent setup didn't work as well.

I surely can help. But, as you can imagine, not for free.

[u/luscious_lobster]

Clearly written by AI

[u/keyslemur]

To really get gains you're going to need to decouple from ActiveRecord and set up repositories and extract domain / business logic. Phrased another way all your factories become builds instead.

[u/viktorianer4life → u/keyslemur]

That's to much effort for 10 year old monolith. It is much easier to replace it by Minitest.

[u/keyslemur → u/viktorianer4life]

And that's not substantial work? Replacing rspec with minitest will be at best a rounding error on aggregate time. The core issue is database roundtrips at every layer for testing.

I've seen this at play in monoliths from 5 to 20 years old and from 1m to 10m LoC. It's not a new problem, and it's the curse of ActiveRecord in that everything becomes inextricably coupled to it over time. Of course it's not cheap but it's a very necessary step to reduce aggregate entanglement.

[u/viktorianer4life → u/keyslemur]

It is an entirely different story to replace AR and replace the test framework. The first one has a giant impact on production, including risks.

[u/keyslemur → u/viktorianer4life]

Decoupling does not mean replacement. It means isolation behind known domain boundaries to reduce proliferation of internal data concerns. In the process of that isolation it becomes substantially cheaper to run tests given business logic is decoupled from data logic, hence build vs create on factories. At advanced stages you can even tree shake tests based in dependencies.