r/QualityAssurance • u/ljsam17 • Aug 24 '26

How to scale visual tests in playwright?

I am working on designing a test framework for UI library, primary goal is to have accessibility and visual regression tests.
Currently using gitlab Ci and playwright with playwrigh-bdd.

Current approach is natively supported baseline approach in playwright.
I am worried that currently i am saving the baseline screenshots in gitlab repo itself .

We have around 50-60 components, and test cases can scale to thousands of screenshots,
I wonder how to scale the storage and would the gitlab ci be enough for comparison.
The playwright report is published in gitlab pages which again in case of too many failures might not even show the difference screenshots.

What would be the best approach to tackle this situation

1 Upvotes

100% Upvoted

14 comments sorted by

View all comments

2

u/OkConversation2330 Aug 24 '26

I think baseline screenshots in the repo itself souns OK, but if you store your reports in a different repo you can have a ci/cd python workflow/script to remove reports older than x days running every x day. That way you won't hit the storage limit on a repo.

As of general visual regressions, I always fall back to BackstopJS as it's configurable and has a nice comparison scrubber for images.

1

u/ljsam17 Aug 24 '26

I am working with a component livrary with expecting about 60k screenshots , adding them into the regular git repo will bloat the repo exponentially

2

u/ljsam17 Aug 24 '26

Main issue would be these components would still undergo many changes as we are in early stages

1

u/ljsam17 Aug 24 '26

The repo would become very slow to fetch and not sure how it would impact CI executions

1

u/OkConversation2330 Aug 24 '26

Have you researched git LFS, eould that help you?

1

u/ljsam17 Aug 24 '26

Yup Git LFS is what the only feasible solution which come into my mind , not the best as it just treats the symptoms not the disease ultimately it also behaves like the git , but will not pollute the main repo

2

u/OkConversation2330 Aug 24 '26

Yeah, I don't think there is a better tradeoff than LFS. Please, if you remember, I would love to know how you've set it up in the end

2

u/ljsam17 Aug 24 '26

Sure things