stablyai/playwright-test-data-isolation

Playwright Test Data Isolation

Use this skill when teams run Playwright tests in a shared QA/staging environment with:

• A shared database (often production-connected)
• Shared test user accounts
• Parallel CI and local test runs
• Flaky tests caused by data collisions, cross-test interference, or unsafe cleanup

Goal: make tests parallel-safe and operationally safe while staying standard Playwright.

When To Use This Skill

Use this skill for tasks like:

• "Our Playwright tests share a database and keep colliding"
• "How do we run E2E safely with shared accounts?"
• "How should we isolate test data in CI?"
• "How do we clean up test data without deleting real data?"
• "How do we keep parallel Playwright runs stable?"
• "How do we design fixtures for safe create-and-cleanup?"

Core Contract (Non-Negotiable)

Every test must follow this contract:

1. Create the entities it needs (test-owned data).
2. Mutate only entities it created.
3. Treat shared baselines (accounts, workspaces, org settings) as read-only.
4. Clean up by exact IDs captured during the test (never broad filters).

If a test cannot follow this contract, move it to a serial suite.

Non-Negotiable Guardrails

• Treat shared users and shared workspaces as read-only containers.
• Never update shared workspace settings, membership, billing, or global toggles in parallel suites.
• Prefix test-created records with a unique namespace (e2e-<run>-<worker>-<uuid>).
• Delete by exact IDs; never run broad delete filters in test teardown.
• Keep a scheduled stale-data janitor as backup, not as the primary cleanup mechanism.

Recommended Architecture

Use shared accounts for authentication only, then isolate all mutable data by namespace.

• Shared accounts: identity only
• Test data: fully owned by one test via namespace
• Cleanup: deterministic, ID-based, reverse order
• Shared-state mutation suites: serialized
• Scheduled cleanup: removes stale e2e-* artifacts older than a threshold

Account Strategy

Pick the highest isolation level your environment can support:

1. Best: one account per worker (pool accounts and map by workerIndex).
2. Acceptable: shared account for sign-in only, test-owned child data for all writes.
3. Avoid: many tests mutating top-level account/workspace settings in parallel.

Implementation Pattern (Playwright-Native)

1) Reuse auth state for shared accounts (setup project)

// auth/setup-auth.ts
import { test as setup } from "@playwright/test";

setup("login as shared qa account", async ({ page }) => {
  await page.goto(process.env.BASE_URL!);
  await page.getByLabel("Email").fill(process.env.E2E_USER_EMAIL!);
  await page.getByLabel("Password").fill(process.env.E2E_USER_PASSWORD!);
  await page.getByRole("button", { name: "Sign in" }).click();
  await page.context().storageState({ path: "playwright/.auth/qa-user.json" });
});

Wire this into a setup project dependency in playwright.config.ts so parallel projects reuse a stable signed-in state.

2) Add namespace + cleanup tracker fixtures

// tests/fixtures/test-data.ts
import { test as base } from "@playwright/test";
import { randomUUID } from "crypto";

type CleanupFn = () => Promise<void>;

type IsolationFixtures = {
  namespace: string;
  trackCleanup: (fn: CleanupFn) => void;
};

export const test = base.extend<IsolationFixtures>({
  namespace: async ({}, use, testInfo) => {
    const runId = process.env.CI_RUN_ID ?? process.env.GITHUB_RUN_ID ?? "local";
    const ns = [
      "e2e",
      runId,
      testInfo.project.name,
      String(testInfo.workerIndex),
      randomUUID(),
    ].join("-");
    await use(ns);
  },

  trackCleanup: async ({}, use) => {
    const fns: CleanupFn[] = [];
    await use((fn: CleanupFn) => fns.push(fn));

    for (const fn of fns.reverse()) {
      try {
        await fn();
      } catch (err) {
        console.error("cleanup failed", err);
      }
    }
  },
});

export { expect } from "@playwright/test";

Notes:

• runId + project + worker + uuid keeps names unique across CI, shards, and local runs.
• Reverse-order cleanup (LIFO) avoids dependency-order teardown bugs.

3) Create child resources in shared containers only

// tests/e2e/projects.spec.ts
import { test, expect } from "../fixtures/test-data";

test.use({ storageState: "playwright/.auth/qa-user.json" });

test("creates isolated project safely", async ({ page, request, namespace, trackCleanup }) => {
  const workspaceId = process.env.E2E_SHARED_WORKSPACE_ID!; // read-only container
  const projectName = `e2e-${namespace}`;

  await page.goto(`/workspaces/${workspaceId}/projects`);
  await page.getByRole("button", { name: "New project" }).click();
  await page.getByLabel("Project name").fill(projectName);
  await page.getByRole("button", { name: "Create" }).click();

  const projectId = await getProjectIdByName(request, workspaceId, projectName);
  trackCleanup(async () => {
    await deleteProjectById(request, workspaceId, projectId);
  });

  await expect(page.getByText(projectName)).toBeVisible();
});

async function getProjectIdByName(
  request: import("@playwright/test").APIRequestContext,
  workspaceId: string,
  name: string,
) {
  const res = await request.get(
    `/api/internal/workspaces/${workspaceId}/projects?name=${encodeURIComponent(name)}`,
  );
  if (!res.ok()) throw new Error(`project lookup failed: ${res.status()}`);
  const body = await res.json();
  return body.items[0].id as string;
}

async function deleteProjectById(
  request: import("@playwright/test").APIRequestContext,
  workspaceId: string,
  id: string,
) {
  const res = await request.delete(`/api/internal/workspaces/${workspaceId}/projects/${id}`);
  if (!res.ok()) throw new Error(`project delete failed: ${res.status()}`);
}

4) Isolate true shared-state tests in serial suites

import { test } from "@playwright/test";

test.describe.configure({ mode: "serial" });

Only use this for tests that must mutate global/shared settings. Keep most tests fully parallel by using test-owned child resources.

5) Add a stale-data janitor project (backup safety net)

// playwright.config.ts
import { defineConfig } from "@playwright/test";

export default defineConfig({
  projects: [
    {
      name: "e2e",
      testDir: "tests/e2e",
      use: { storageState: "playwright/.auth/qa-user.json" },
    },
    {
      name: "cleanup",
      testDir: "tests/cleanup",
      workers: 1,
      retries: 0,
    },
  ],
});

// tests/cleanup/stale-e2e-data.spec.ts
import { test, expect } from "@playwright/test";

test("delete stale e2e data older than 24h", async ({ request }) => {
  const res = await request.post("/api/internal/cleanup/e2e", {
    data: {
      prefix: "e2e-",
      olderThanHours: 24,
    },
  });

  expect(res.ok()).toBeTruthy();
});

Run janitor on a schedule (nightly or multiple times/day), but keep per-test cleanup as the primary defense.

Collision Debugging Checklist

When a test collides, collect these first:

• Namespace used by each failing test
• Worker index and project name
• IDs created by each test before cleanup
• Which shared entity was mutated (if any)

If retries make failures disappear, treat that as a signal of shared-state coupling, not success.

Optional: If Team Uses Stably

Stably can help with orchestration and operations, but this strategy does not depend on it.

• Use npx stably test as a Playwright-compatible runner wrapper when you want hosted reporting/ops.
• Use Stably environments to inject shared account credentials by environment.
• Use Stably schedules/cloud workers for janitor jobs and large parallel runs.
• Keep the same isolation contract regardless of runner.

Anti-Patterns To Block

• Parallel tests writing to the same shared entity
• Shared mutable fixtures reused across files
• Cleanup using broad filters like "delete all e2e rows"
• Relying on execution order
• Using retries to mask shared-data collisions
• Treating nightly cleanup as the only cleanup

Agent Output Requirements

When this skill is activated, the agent should:

1. Confirm environment constraints (shared DB, shared accounts, parallelism level).
2. Produce or update a namespace fixture and deterministic cleanup tracker.
3. Classify tests into parallel-safe vs shared-state-mutation suites.
4. Refactor at least one target test to create-and-own data.
5. Add or propose a stale-data cleanup project and schedule.
6. Return a short risk report listing remaining unsafe tests.

Rollout Plan

1. Add namespace + cleanup fixtures.
2. Migrate flaky mutable tests to create-and-own data.
3. Add cleanup project and schedule.
4. Move shared-state mutation tests to serial mode.
5. Increase workers only after collision-free runs.

Success Criteria

• Parallel runs no longer collide on shared data.
• Teardown never touches non-test data.
• Flake rate drops for data-dependent tests.
• Shared environment remains stable for manual QA and CI.