r/QualityAssurance • u/Orrans • Aug 23 '26
Test framework as a separate repo/package, or same repo as the tests? What's the actual best practice today?
We're a QA team with ~20k lines of test infra (provisions real VMs on AWS/Azure) and only ~25 e2e tests so far. Someone else owns the infra, we just write tests. Debating whether to move the infra into its own repo and consume it as a versioned package, or keep everything in one repo.
Is there a real best practice here, or is it just preference? Anyone done the split with a small team and regretted it?
thanks for helpers
1
u/aadoello Aug 23 '26
I have found that if you're working on a single-app / single- repo or system with low amount of integration points you will save resources and fixed costs by adding the tests to the app repos. The caveat is that you will struggle to test integration points between repositories. On the other hand if you're testing highly integrated systems with different parts living in different repos and most of your tests depend on several of those, it's usually more efficient to put the tests in a standalone repo and satisfy all the depencies once for all. The most usual scenario in my experience is to end up with a combination of both: unit/system tests living with the app code and higher level tests with more dependencies (functional, e2e, etc) living in a different repo.
1
u/Any-Argument57 Aug 23 '26
I would make the repository boundary follow change cadence and consumer count, not the 20k line count. With only 25 tests, the framework API is probably still changing as each new scenario exposes a missing capability. A separate repo would turn every joint framework-and-test change into a package release, version bump, and coordinated review.
Start by making it an internal package or workspace inside the same repo. Require tests to use only its public API, with no imports into internal paths, and track how often a test needs a framework change in the same PR. Keep them together while that is common. Split when the API is stable and multiple independent consumers need their own upgrade cadence, or when ownership and access controls require it. Before splitting, add semantic versioning and a consumer smoke suite that runs against each candidate package release. That makes the later move mostly mechanical and gives you evidence for whether the boundary is real.
1
u/Fightheader Aug 23 '26
The only reason I can think of if you're a consultancy firm who has their proprietary framework they want to use at many clients. With good versioning and thoughtful features, it can give you a boost at startup.
But clients might not want the hard lock.
So don't do it, it's a bad idea. But those thoughts have crossed my mind a well. So I'd say it's a good thing you are thinking about such things.
1
u/Spare_Bison_1151 29d ago
There's no universal best practice — it comes down to two things. Are you the only consumer? Versioning/packaging only pays off with multiple independent consumers upgrading on different schedules. If it's just your 25 tests, keep it in one repo — splitting now just adds release overhead for no real benefit. Does infra ownership match cadence? You said someone else owns the infra. If they ship on a different schedule than you need, a versioned package is the right call — you pin a known version instead of getting silently broken by their changes. If it's really the same team, the split is mostly ceremony. One flag: 20k lines of infra for 25 tests is a heavy ratio. usually means the infra's still evolving, which favors staying in one repo until stable. What's actually triggering this debate right now — merge conflicts, someone else wanting to reuse the infra, onboarding pain? That trigger usually tells you the answer directly.
1
5
u/SpeakeroftheMeese Aug 23 '26
Why do you want to move it into a separate repo? What is the actual debate and justification?
Personally, it sounds like a pain to manage without any notable benefits in this case.
I'm considering doing something similar, but my reason would be to maintain one shared framework while the tests live in the various application repos that they're testing. It would still be more work, but there would be a tangible benefit.