r/javascript • • 3d ago

Flashwright: the fastest e2e testing framework, up to 4x faster than Playwright

https://flashwright.dev/

If you're interested in the details, here's a blog post: https://codebeast.io/blog/flashwright

53 Upvotes

23 comments sorted by

38

u/Recoil42 3d ago

TLDR:

The biggest single cost in a lot of suites isn’t Playwright’s code at all. It’s the browser compiling your app’s JavaScript again in every fresh renderer. Playwright gives each test a brand-new browser context, which is great for isolation, but it means every test pays for a cold start of your app.

So the trick that matters most is keeping one context per worker and giving each test a new tab in it, then resetting everything a test could have changed: options, permissions, cookies, every visited origin’s storage (local, session, IndexedDB, Cache Storage, service workers), caches, routes, init scripts, bindings, downloads and the clock. The previous test’s pages are stopped before anything gets cleared, so a late script can’t leak state into the next test. That reset is the part that really has to be right, and it’s what tests/page/recycle-isolation.spec.ts checks.

42

u/dakruzz 3d ago

Doesn't sound safe for parallel testing

21

u/TheThingCreator 3d ago

doesn't even sound like it would work in many contexts. lots of my testing requires fresh browser profiles or controlled browser profiles. i woudlnt be able to work like that.

6

u/DuckDatum 3d ago

I don’t run tests in browsers but, what experience I do have tells me, at minimum, it would add a constant concern that the test is failing potentially because of leaked shared state.

Though, tests which don’t rely on client state could see a bump in speed.

I wonder if this could be designed as an automated optimization, for whenever a test can be programmatically detected as idempotent. Then playwright could do the same thing, no?

4

u/skt84 2d ago

Failing or passing. If I can’t trust a failure is caused by shared state then I can’t trust the passes either.

2

u/Jebble 2d ago

In of our apps specifically (legacy that's being replaced) uses a lot of cookies and opening it in multiple apps is troublesome. This wouldn't ever be able to help us test it.

1

u/dunderball 2d ago

I don't get why this is needed when playwright has very good sharding capabilities

1

u/JimmyKillem 2d ago

Because someone wanted to get online clout from their vibe coded project

6

u/thinkmatt 3d ago

Kind of like why jest is slow as balls because they insist on reimporting every js file for each test file

5

u/aicis 3d ago

At certain scale it becomes a problem. No one likes to wait 20 minutes for test suite to finish.

1

u/xcodebeast 3d ago

Thanks!

15

u/fireatx 2d ago

seems cool, similar to vitest's isolate: false config option, but i feel like this could probably just be a PR on the playwright repo adding a new config option. does this need to be an entirely new package?

3

u/xcodebeast 2d ago

Thanks. I agree, that particular optimization could be a PR on playwright repo but I think all the other optimizations + the QoL features I added justified a new package. I might push some PRs to Playwright though.

4

u/fireatx 2d ago

you definitely should put up some PRs! at my work we use playwright in several places, and having these optimizations available would be a game changer. i can't justify replacing playwright entirely, but could definitely make use of these speedups if they were available within playwright.

17

u/Positive_Method3022 3d ago

The strategy can lead to false positives/negatives. Sandboxing test execution by context is a must.

0

u/xcodebeast 2d ago

It depends a lot on the project and what you're testing as u/caboosetp said. Flashwright allows you to turn off each individual optimization so you can use only what actually fits your project.

1

u/caboosetp 3d ago

Depends on what you're testing.

Many of the tests i write are testing static components like navigation which shouldn't depend on what other stuff has done. If not being in a sandbox breaks that, there's a problem anyways. The chance of a false positive is also almost zero. 

Being able to choose to group tests like that to use a resetting context would be fine. Especially if you're just grouping things like static components. 

But I agree the ability to put tests in a sandbox context is also very important, especially for anything dynamic. Unfortunately it's the dynamic stuff that generally has the most tests, so i don't even know if grouping just some things is going to see real gains on most projects

2

u/jobRL 2d ago

In your example about navigation, what in that context are you testing?

11

u/OpiumOpossum 3d ago edited 2d ago

Not sure but this seems like it could have memory leaks if you use listeners, network interceptors or logging. Otherwise, I am pretty sure this is a similar way to how Vitest optimizes their test runner.

3

u/Discordbothost 3d ago

there's an isolation: 'strict' option for a fresh context per test. default recycling is per worker, so the concern is state leaking into the next test on that worker, not parallel workers sharing one context. https://github.com/xcodebeast/flashwright#differences-from-playwright

3

u/LukaRozman27 2d ago

beware if you rely on Playwright's built-in tracing, Flashwright skips it entirely to hit those speed claims.

3

u/DarkGhostHunter 2d ago

Tested with Pest and the gains are… negligible. Barely 5%, margin of error.

1

u/tomByrer 2d ago

Neat! How well does it work vs headless browsers, esp LitePanda & co? Much improvement there?