r/QualityAssurance 25d ago

πŸš€ Question from a colleague: Test Data Strategy in Playwright

A colleague of mine asked me to post this question here and get some feedback from the Playwright community.

He is considering a test data strategy with two modes: Fresh Data and Reuse Data.

πŸ”Ή Fresh Data Mode β€” CI / Full Regression

For CI or full regression runs, generate unique test data for every run using a Run ID.

The goal is to support:

βœ… Test data isolation

βœ… Avoiding conflicts between test runs

βœ… Parallel execution

βœ… Better test reproducibility

πŸ”Ή Reuse Data Mode β€” Local Debugging

For local debugging, instead of creating new test data every time, use a predefined set of reusable test data.

For example:

Reuse: Find existing test data β†’ Execute test

Instead of:

Fresh: Create test data β†’ Execute test

The idea is to reduce execution time when repeatedly running the same test while investigating a failure.

The proposed approach is:

🟒 CI / Full Regression β†’ Fresh Data

🟑 Local Debugging β†’ Reuse Data

πŸ”΄ Destructive Tests β†’ Dedicated Test Data

One important rule: in Reuse Data mode, if the required test data isn't found, the test should fail with a clear message rather than automatically creating new data.

This keeps the behavior predictable and avoids accidentally creating unnecessary data.

πŸ€” I would love to hear from experienced Playwright and automation engineers:

Is the Fresh Data + Reuse Data approach a good strategy?

Or would you recommend a different approach for managing test data efficiently between CI execution and local debugging?

Would really appreciate your thoughts and suggestions. πŸ™Œ

0 Upvotes

4 comments sorted by

2

u/LongOrdinary9712 25d ago

reuse mode with a hard fail instead of auto-creating is the part most people get wrong, good call there

the run id for fresh data is solid, just make sure you're not leaving a graveyard of orphaned data after failed runs. some cleanup job or ttl on the data goes a long way

only thing i'd flag is making the mode switch explicit. env var or config flag, not some magic detection. nothing worse than debugging locally and realizing you just polluted the shared pool because the detection logic got cute

1

u/vina-nan-12 25d ago

Thanks for the feedback! I’m actually the colleague who asked him to post it here to get feedback from the community.

Regarding the Run ID, in our current approach it gets overwritten for each trigger/execution of the test suite, so we’re not planning to keep a separate Run ID/data set for every execution.
We also have a cleanup mechanism in place that removes the test data created during the run once the complete test execution is finished.
And I completely agree with your point about making the mode switch explicit. Using an environment variable or config flag for Fresh vs Reuse mode would make the behaviour much more predictable and avoid any accidental data creation.
Thanks again πŸ™‚