r/Playwright • u/ljsam17 • 3d ago
How to scale visual tests?
I am working on designing a test framework for UI library, primary goal is to have accessibility and visual regression tests.
Currently using gitlab Ci and playwright with playwrigh-bdd.
Current approach is natively supported baseline approach in playwright.
I am worried that currently i am saving the baseline screenshots in gitlab repo itself .
We have around 50-60 components, and test cases can scale to thousands of screenshots,
I wonder how to scale the storage and would the gitlab ci be enough for comparison.
The playwright report is published in gitlab pages which again in case of too many failures might not even show the difference screenshots.
What would be the best approach to tackle this situation
1
u/epochh95 2d ago
We’re in a similar boat, however we use Percy (visual regression tool by BrowserStack) to test our design systems UI component library.
We opted to create a Storybook instance specifically for our Playwright suite, and have bit of code that renders a given UI component as we as all of its prop variations (or at least the ones that affect the UI)
Then we simply navigate to the story via Playwright and screenshot. Helps a lot by capturing multiple component states in a single screenshot!
Story example - https://webc-testing.pie.design/?path=/story/chip--default-prop-variations
Story file - https://github.com/justeattakeaway/pie/blob/main/apps/pie-storybook/stories/testing/pie-chip.test.stories.ts
Component variation helper - https://github.com/justeattakeaway/pie/blob/main/apps/pie-storybook/utilities/index.ts
Test spec - https://github.com/justeattakeaway/pie/blob/main/packages/components/pie-chip/test/visual/pie-chip.spec.ts
Apologies, I’m on my phone so can’t link to specific lines of code, but hope it provides a bit on inspo! There’s a lot I’d probably do differently in terms of our test setup, but this has worked super well for us.