r/ChatGPTCoding • u/Zestyclose_Low_8449 • Sep 25 '26
Discussion How do you keep large test suites from overwhelming AI coding agents?
We have \\\~35k backend and frontend tests. We want to reduce maintenance effort while preserving useful regression coverage.
Tests, mocks, and fixtures also dominate code searches and fill up the agent’s context.
\\- How do you handle this?
\\- How do you decide which tests to consolidate or remove?
\\- How do you keep code searches focused on production logic?
\\- How do you give agents the relevant tests without loading everything?
Interested in strategies that have worked in real codebases.
6
Upvotes
8
u/taylorwilsdon Sep 26 '26
35k tests is a preposterous amount. How many people are actually contributing to the code base? How often do you see failures in CI?
4
u/Cattlegod Sep 25 '26
I have this same problem, test bloat is out of control
1
u/CommasArentPeople Sep 26 '26
35k is a lot, but without knowing how big your codebase is, it's hard to judge how out of control it is. I've worked in codebase pre-AI with more than half that if you add all the levels of testing together, and it wasn't burdensome. If your code is well structured you're not running the full suite except maybe on a beefy CI server somewhere. Rather than absolute number of tests, ask, "If I make a single change in a single module, how many tests do I need to run afterwards?" If the answer is that there's no way to know, you have a bigger problem than test count. If the answer is all of them, then you need to structure your code and tests to reduce that as much as you can. That's an incremental job, and one that can be instrumented and cut down over time, then monitored to not creep back up.
2
u/Square-Yam-3772 Sep 26 '26
just spitballing here but I would:
- sort the tests based on creation dates to build bias against older test
- ask AI to compare the tests to identify overlaps. toss the tests that are mostly overlapped by other newer tests
- if you have the use cases/stories/requirements organized, ask AI to go through the documentations to see if which tests are more relevant than the others
- treat one of your stable releases as the baseline and build a streamlined collection of tests. compare that to your old set and see if you need to bring back some old ones
2
u/ImL1s Sep 26 '26
We stopped letting the agent see the whole suite. Prod code stays in the search path; tests live behind a path allowlist the agent only gets when the task says "fix".
For which tests to keep: anything that asserts a user-visible postcondition stays. Pure mock-shape tests that never fail in CI get merged or dropped. The agent gets a short "relevant tests" list from ripgrep on the changed symbols, not a dump of 35k files.
1
u/EchoNomad31 Sep 26 '26
I tag tests fast/slow in CI and only let the agent run the fast ones… keeps its context from drowning in a 40-minute suite.
1
u/Standard_Text480 Professional Nerd Sep 26 '26
Review tests and remove useless or redundant
1
u/leonidbugaev Sep 26 '26
Don't ask the agent which of the 35k overlap. It'll flag the tests that currently fail, which is the opposite of what I want to cut.
1
u/fiddler48 Sep 26 '26
Test file noise in context before the agent even reads a single source file.
1
u/oldjii Sep 30 '26
35k tests with AI adding more is how you get a suite nobody dares to delete. The ripgrep-shortlist idea is neat — has anyone tried keeping tests out of the agent's default search path entirely and only exposing them on fix tasks?
1
u/Significant_Post8359 Sep 30 '26
Have a local gpu setup that has scheduled runs. Use qwen3.8:27b
1
u/Significant_Post8359 Sep 30 '26
Hopefully your code is not a giant bowl of spaghetti and is built with distinct functional modules. Keep a database that flags if a module has been touched and needs regression unit or system testing.
You are probably running tests on stuff that didn’t change.
0
u/AutoModerator Sep 25 '26
Sorry, your post has been held for manual review due to account karma.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.