The-Utopia-Studio/evidence-driven-testing

Evidence-Driven Testing

Record annotated screen-recording proof of UI behavior, then attach it to the PR and tracker issue.

Inputs

• Test targets (required): The behaviors/flows to verify, phrased as testable statements.
• PR / issue (optional): Where to post the evidence. If omitted, deliver to the requester only.

Instructions

1. Prepare the screen

• Maximize the browser/app window; close popups, notifications, and extra panels.
• Navigate to the starting state (logged in, correct page) BEFORE recording, unless setup itself is under test.

2. Start recording

• Begin the screen recording before the first meaningful action.
• Add a setup annotation describing the starting context, e.g. "Logged in, navigating to connectors page".

3. Annotate as you test

• At each named test's start, add a test_start annotation in Jest style: It should execute the tool directly when permission is 'always'.
• After each check, add an assertion annotation with result passed, failed, or untested.
• Rules for assertions:
  • One assertion per meaningful state change — consolidate, don't annotate per UI label.
  • Use "Precondition: ..." assertions to establish starting state.
  • Keep under ~80 characters, high-signal.
  • If a test cannot run (missing prerequisite, expired auth window), mark it untested with the reason — never skip silently.

4. Stop and review

• Stop recording after the final assertion.
• Confirm the recording captured the key moments before sharing.

5. Post the evidence

• Write a short report: what was tested, environment + exact commit, pass/fail per test, caveats.
• Post the video + summary as a PR comment (embed in the PR description if it's your PR).
• Attach the same video to the tracker issue (Linear/Jira) with a one-line result.
• Send the report + recording to the requester.

Guardrails

• Never record a half-covered or tiled window — maximize first.
• When verifying a fix, show or reference the old failure alongside the new success.
• Always state the exact commit/branch/deployment tested against.