Why your Playwright suite is flaky, and the three causes behind almost all of it

A flaky suite is worse than no suite. Not because the tests are wrong — they might be finding real bugs — but because a suite that fails intermittently teaches your team to ignore failures. Once people habitually re-run a build until it goes green, you are paying for CI minutes and getting nothing back. The safety net is decorative.

The good word is that flakiness is rarely mysterious. In almost every suite we have inherited, it traces back to three causes.

Cause 1: tests share state

This is the big one, and it is usually invisible until you enable parallelism.

Two tests both use the user test@example.com. One updates the profile name; the other asserts the profile name. Run serially in a fixed order, they pass forever. Run them in parallel — or after someone reorders the file — and they fail perhaps one time in eight.

The instinct is to add test.describe.serial. That is not a fix, it is a lock on the door of a room you have decided not to clean. It also makes your suite slower for as long as it exists.

The fix is that every test creates the data it needs and owns it exclusively:

// Each test gets its own user. No sharing, no ordering assumptions.
const userFixture = base.extend<{ user: TestUser }>({
  user: async ({ request }, use) => {
    const user = await createUser(request, {
      email: `qa+${crypto.randomUUID()}@example.com`,
    });
    await use(user);
    await deleteUser(request, user.id);
  },
});

Two things matter here. Create the data through the API, not through the UI — signing up through five screens to test a settings page is slow and gives you a second thing that can fail. And make the identifier unique per test, because a timestamp is not unique when eight workers start in the same millisecond.

If you take one thing from this post: a test that cannot run twice concurrently against the same environment is not finished.

Cause 2: timing assumptions the code does not state

Playwright’s auto-waiting is genuinely good, and it handles the common case so well that people assume it handles every case. It waits for the element you addressed to be actionable. It cannot know about things you did not tell it about.

// Passes on a fast machine. Fails on loaded CI, roughly one run in twelve.
await page.click('#save');
await expect(page.locator('.row')).toHaveCount(3);

The click fires a request. The list re-renders when the response lands. Between those two moments the old list is still on screen with three rows — so occasionally the assertion passes for entirely the wrong reason, which is worse than failing.

Wait for the thing you actually care about:

await Promise.all([
  page.waitForResponse((r) => r.url().includes('/api/rows') && r.ok()),
  page.click('#save'),
]);
await expect(page.getByRole('row')).toHaveCount(3);

And never reach for waitForTimeout. A hard sleep is a bet that the machine will not be slower than it was on the day you wrote it. You will lose that bet on a Monday morning when everyone pushes at once. Every waitForTimeout in a suite is a scheduled future failure.

Animations deserve a mention: an element can be visible and still moving. page.click handles most of this, but for drag interactions and custom transitions, disabling animations in your test environment removes the whole category:

@media (prefers-reduced-motion: reduce) {
  *, *::before, *::after { animation: none !important; transition: none !important; }
}

Cause 3: locators coupled to markup instead of meaning

page.locator('div.mt-4 > div:nth-child(2) > button.btn-primary');

This does not describe a button. It describes today’s DOM. It will break when a designer adds a wrapper div, and the failure will say nothing about what went wrong.

Address elements the way a user perceives them:

page.getByRole('button', { name: 'Save changes' });
page.getByLabel('Email address');
page.getByTestId('invoice-total'); // when nothing semantic exists

Role and label locators have a second benefit worth more than the stability: if getByRole('button', { name: 'Save changes' }) cannot find your button, that is frequently a real accessibility defect. The test is telling you something about the product.

Where the DOM offers nothing meaningful, add an explicit data-testid. That is a contract — a developer who sees it knows it is referenced by tests. A CSS class is not a contract; it belongs to whoever last touched the stylesheet.

Making flakiness visible

You cannot fix what you do not measure, and a flaky test hides by passing on the retry.

Set retries in CI, but treat a retry as a recorded event rather than a success. Playwright’s flaky status exists for exactly this. Publish the number weekly. Our working thresholds:

Flake rate	What it means
Under 0.5%	Healthy. Investigate individual cases.
0.5% – 2%	Degrading. Allocate time now, before trust erodes.
Above 2%	People have already stopped believing the suite.

When a test flakes twice in a week, quarantine it with a linked ticket and an owner. Quarantine is not deletion and it is not muting — it is a promise with a name attached. A suite with three quarantined tests and a published flake rate is in far better health than one with zero quarantined tests and a culture of hitting re-run.

The uncomfortable part

Most flakiness is not a Playwright problem. It is a test-design problem, and occasionally it is your application genuinely being non-deterministic — in which case the test is correct and you have found a real bug worth more than the test suite cost you.

Before blaming the framework, check the three causes above in order. In our experience they account for the overwhelming majority of intermittent failures, and all three are fixable with ordinary engineering discipline rather than a new tool.