PramodDutta/Playwright test.step & Reporting

Playwright test.step & Reporting

This skill makes the agent produce tests that explain themselves in the report. Instead of a flat wall of actions, the agent groups logical phases with test.step, attaches screenshots/JSON/diffs via testInfo.attach, records annotations for traceability, and uses expect.soft to collect multiple failures in one run. When a test fails in CI, a human should understand what happened from the HTML report alone.

Use this skill when writing non-trivial flows, when a test is hard to debug from its output, or when the user mentions reports, steps, annotations, attachments, or issue traceability.

Core Principles

1. Every logical phase is a test.step. Steps appear as a collapsible tree in the HTML report with timings, turning a failure into "which step failed" instead of "which line number."
2. Attach evidence, do not just log it. console.log is invisible in the report; testInfo.attach puts screenshots, JSON, and text into the report next to the step.
3. Use expect.soft to gather multiple defects in one execution, but end critical paths with a hard assertion or expect.poll so the test still fails.
4. Annotate for traceability. Link tests to issues/requirements with annotation, and use test.info().annotations to surface skips/known-issues in the report.
5. Steps should be named like a test plan — imperative, business-readable ("Add Pro plan to cart"), not "click button #3".
6. Box internal helper steps so a failure points at the caller, not deep inside the helper.

Workflow / Patterns

Pattern 1 — Structure a flow with test.step

Steps nest and report their own duration. Return values from a step to chain them.

import { test, expect } from '@playwright/test';

test('checkout flow', async ({ page }) => {
  await test.step('Sign in', async () => {
    await page.goto('https://shop.example.com/login');
    await page.getByLabel('Email').fill('[email protected]');
    await page.getByLabel('Password').fill('Secret123!');
    await page.getByRole('button', { name: 'Sign in' }).click();
    await expect(page.getByText('Welcome back')).toBeVisible();
  });

  const orderId = await test.step('Place order for Pro plan', async () => {
    await page.getByRole('link', { name: 'Pro plan' }).click();
    await page.getByRole('button', { name: 'Buy now' }).click();
    await page.getByRole('button', { name: 'Confirm purchase' }).click();
    const id = await page.getByTestId('order-id').textContent();
    return id!.trim();
  });

  await test.step(`Verify order ${orderId} in history`, async () => {
    await page.goto('https://shop.example.com/orders');
    await expect(page.getByText(orderId)).toBeVisible();
  });
});

Pattern 2 — Attach evidence with testInfo.attach

Attachments render inline in the HTML report. Attach a screenshot, the API response, or a computed diff at the moment it matters.

test('attaches evidence to the report', async ({ page }, testInfo) => {
  await page.goto('https://example.com/dashboard');

  await test.step('Capture dashboard state', async () => {
    // Screenshot attachment (shown inline in the report).
    await testInfo.attach('dashboard.png', {
      body: await page.screenshot({ fullPage: true }),
      contentType: 'image/png',
    });

    // JSON attachment — the raw API payload behind the screen.
    const widgets = await page.evaluate(() => (window as any).__WIDGETS__ ?? []);
    await testInfo.attach('widgets.json', {
      body: JSON.stringify(widgets, null, 2),
      contentType: 'application/json',
    });

    // Plain-text attachment for a human-readable note.
    await testInfo.attach('environment.txt', {
      body: `Project: ${testInfo.project.name}\nBase URL: ${page.url()}`,
      contentType: 'text/plain',
    });
  });
});

Pattern 3 — Soft assertions to collect multiple failures

expect.soft records the failure and keeps going. End the test with expect(test.info().errors).toHaveLength(0) or a hard check so it still fails — and the report shows every problem at once.

test('validates a form with soft assertions', async ({ page }) => {
  await page.goto('https://example.com/profile');

  await test.step('Verify all profile fields at once', async () => {
    await expect.soft(page.getByLabel('Display name')).toHaveValue('Ada Lovelace');
    await expect.soft(page.getByLabel('Email')).toHaveValue('[email protected]');
    await expect.soft(page.getByLabel('Timezone')).toHaveValue('UTC');
    await expect.soft(page.getByRole('img', { name: 'Avatar' })).toBeVisible();
  });

  // Hard gate: fail the test if any soft assertion failed.
  expect(test.info().errors).toHaveLength(0);
});

Pattern 4 — Annotations for traceability and known issues

Annotations attach metadata to a test; they show up in the report and JSON output. Use them to link issues and to document why something is skipped.

test('payment retries on gateway 503', async ({ page }) => {
  test.info().annotations.push(
    { type: 'issue', description: 'https://github.com/acme/app/issues/4821' },
    { type: 'suite', description: 'payments-regression' },
  );

  await page.goto('https://example.com/pay');
  await expect(page.getByRole('button', { name: 'Pay' })).toBeEnabled();
});

test('legacy export still works', async ({ page }) => {
  test.skip(process.env.LEGACY !== 'on', 'Legacy export disabled in this env');
  test.info().annotations.push({ type: 'known-issue', description: 'Slow > 5s, tracked in #5102' });
  await page.goto('https://example.com/export');
  await expect(page.getByText('Export ready')).toBeVisible({ timeout: 30_000 });
});

Pattern 5 — Box helper steps so failures point at the caller

A boxed step collapses its internals in the report; the error is reported at the step call site, not buried in shared helper code.

import { test, expect, type Page } from '@playwright/test';

async function loginAs(page: Page, email: string, password: string) {
  await test.step(
    `Log in as ${email}`,
    async () => {
      await page.goto('https://example.com/login');
      await page.getByLabel('Email').fill(email);
      await page.getByLabel('Password').fill(password);
      await page.getByRole('button', { name: 'Sign in' }).click();
      await expect(page.getByTestId('user-menu')).toBeVisible();
    },
    { box: true }, // failure surfaces at this step, internals collapsed
  );
}

test('admin can open settings', async ({ page }) => {
  await loginAs(page, '[email protected]', 'Admin123!');
  await page.getByRole('link', { name: 'Settings' }).click();
  await expect(page.getByRole('heading', { name: 'Settings' })).toBeVisible();
});

Pattern 6 — Configure rich reporters

Wire up reporters in playwright.config.ts. The HTML reporter consumes the steps, attachments, and annotations above; JSON/JUnit feed CI dashboards.

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['html', { open: 'never', outputFolder: 'playwright-report' }],
    ['json', { outputFile: 'test-results/results.json' }],
    ['junit', { outputFile: 'test-results/junit.xml' }],
    ['list'],
  ],
  use: {
    // Auto-capture trace + screenshot on failure — they attach to the report.
    trace: 'retain-on-failure',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
});

Best Practices

1. Wrap each logical phase in test.step with a business-readable, imperative name — the report becomes a living test plan.
2. Attach the artifact that explains the failure (screenshot, response JSON, diff) at the step where it is relevant, via testInfo.attach.
3. Set trace: 'retain-on-failure' and screenshot: 'only-on-failure' so failing tests carry full debugging context automatically.
4. Use expect.soft for grouped, independent checks, then add a hard expect(test.info().errors).toHaveLength(0) gate.
5. Push issue/known-issue annotations to keep traceability between tests and tickets visible in the report.
6. Box shared helpers so a failure inside login/setup points at the calling test, not the helper internals.
7. Return values from steps to pass IDs/tokens forward and keep the flow linear.

Anti-Patterns

1. A 60-line test with no steps. A failure gives a line number and nothing else; reviewers cannot tell which phase broke.
2. console.log for evidence. Logs are not in the HTML report. Use testInfo.attach.
3. expect.soft everywhere with no hard gate. The test goes green while assertions silently failed. Always end with a hard check.
4. Steps named after mechanics ("click", "fill input") instead of intent ("Submit signup form").
5. Attaching full-page screenshots on every step of a passing test — bloats the report. Attach on failure or at key checkpoints.
6. Skipping tests with bare test.skip() and no reason — the report shows a skip with no explanation. Always pass a reason string.
7. Deep nesting of unnamed anonymous steps that obscure rather than clarify the flow.

When to Trigger This Skill

• "Make my Playwright test report readable / easier to debug"
• "Group my test into steps" / "use test.step"
• "Attach a screenshot or JSON to the Playwright report"
• "How do I do soft assertions in Playwright?"
• "Add annotations / link tests to Jira or GitHub issues"
• "Why can't I tell which step failed in CI?"
• "Set up the HTML / JUnit / JSON reporter"
• "Capture a trace only when a test fails"