jyjeanne/accessibility-selenium-testing

Accessibility Testing with Selenium WebDriver & Axe Core

This skill enables automated accessibility analysis within the Selenium WebDriver framework using the axe-core engine to detect WCAG violations and best practice issues directly in the browser.

Activation: This skill is triggered when you need to validate WCAG compliance, scan for accessibility violations, test keyboard navigation, audit ARIA semantics, or generate a11y reports.

First Questions to Ask

• What app URL(s) or user flows are in scope (and what is explicitly out of scope)?
• Is there an existing Selenium setup and how is CI run?
• Which standard is the target (WCAG 2.1 AA by default), and are there org-specific policies?
• Which pages/components are highest risk (auth, checkout, forms, modals, navigation)?
• Are there known constraints (legacy markup, third-party widgets) that require exceptions?

Prerequisites

| Component | Version | Purpose | |-----------|---------|---------| | Java JDK | 21+ | Runtime with modern features | | Maven | 3.9+ | Dependency management | | Selenium WebDriver | 4.x | Browser automation | | axe-core-selenium | 4.10+ | Deque axe-core integration | | JUnit 5 | 5.10+ | Test framework | | AssertJ | 3.x | Fluent assertions for readable failures | | Allure | 2.x | Reporting with a11y violation attachments |

Note: Use com.deque.html.axe-core:selenium Maven dependency for axe integration.

---

WCAG Compliance Levels

| Level | Requirement | Legal Status | Axe Tags | |-------|-------------|--------------|----------| | Level A | Basic accessibility (must have) | Minimum legal requirement | wcag2a, wcag21a | | Level AA | Intermediate (should have) | Legal requirement in most jurisdictions | wcag2aa, wcag21aa | | Level AAA | Advanced (nice to have) | Not typically required | wcag2aaa, wcag21aaa | | Best Practice | Industry recommendations | Not WCAG but improves UX | best-practice |

---

Axe-Core Tools Reference

AxeBuilder Configuration

| Method | Purpose | Example | |--------|---------|---------| | new AxeBuilder() | Create scanner instance | Entry point | | .withTags(List<String>) | Filter by WCAG tags | wcag2aa, wcag21aa | | .include(String) | Scan specific selector | #main-content | | .exclude(String) | Skip selector from scan | .third-party-widget | | .disableRules(List<String>) | Disable specific rules | color-contrast | | .withRules(List<String>) | Run only specific rules | label, button-name | | .analyze(WebDriver) | Execute the scan | Returns Results |

Results Object

| Method | Returns | Purpose | |--------|---------|---------| | getViolations() | List<Rule> | Rules that failed | | getPasses() | List<Rule> | Rules that passed | | getIncomplete() | List<Rule> | Rules needing manual review | | getInapplicable() | List<Rule> | Rules not applicable to page | | violationFree() | boolean | True if no violations |

Violation Impact Levels

| Impact | Severity | CI Action | |--------|----------|-----------| | Critical | Blocks users completely | Always fail build | | Serious | Significant barrier | Always fail build | | Moderate | Some difficulty | Warn or fail | | Minor | Inconvenience | Log for review |

---

Core Capabilities

1. Axe Builder Analysis

• Full Page Scan: new AxeBuilder().analyze(driver)
• Component Scan: new AxeBuilder().include("#my-component").analyze(driver)
• Rule Configuration: .withTags(List.of("wcag2a", "wcag2aa"))
• Exclusions: .exclude(".legacy-footer") (use carefully, document reason)

2. Validation & Assertion

• Analyze Results.getViolations() - should be empty
• Filter by impact level (Critical, Serious, Moderate, Minor)
• Use AssertJ Soft Assertions to report all violations before failing

3. Reporting

• Log: Rule ID + Help URL + Selector for each violation
• Serialize Results to JSON for dashboards
• Attach to Allure reports

---

Your Role

As an Accessibility Automation Specialist:

1. Integration: Configure axe-core with Selenium WebDriver
2. Configuration: Set up AxeBuilder with appropriate WCAG tags
3. Analysis: Parse results to identify violations by impact
4. Assertion: Fail on Critical/Serious, warn on Moderate/Minor
5. Reporting: Log Help URLs and selectors for remediation

---

Step-by-Step Workflows

Workflow 1: Add A11y Scan to Existing Test

1. Add dependency to pom.xml

   <dependency>
       <groupId>com.deque.html.axe-core</groupId>
       <artifactId>selenium</artifactId>
       <version>4.10.0</version>
   </dependency>
   

2. Create AccessibilityHelper utility

   • See Axe Patterns Guide

3. Add scan after page loads

   driver.get("https://example.com");
   waitForPageReady();
   AccessibilityHelper.verifyPageAccessibility(driver);
   

4. Run and review violations

   mvn test -Dtest=A11yTest
   

Workflow 2: Test Specific Component

1. Navigate to page with component visible

2. Trigger component state (open modal, show dropdown)

3. Scan only the component

   Results results = new AxeBuilder()
       .withTags(List.of("wcag2a", "wcag2aa"))
       .include("#login-modal")
       .analyze(driver);
   

4. Assert and log

Workflow 3: Keyboard Navigation Audit

1. Identify all interactive elements
2. Tab through the page programmatically

   element.sendKeys(Keys.TAB);
   WebElement focused = driver.switchTo().activeElement();
   

3. Verify focus order is logical
4. Test Escape closes modals
5. Verify no keyboard traps

Workflow 4: CI Integration

1. Configure headless browser

   mvn test -Dheadless=true -Dgroups=a11y
   

2. Set zero-tolerance for Critical/Serious

   long criticalCount = violations.stream()
       .filter(v -> List.of("critical", "serious").contains(v.getImpact()))
       .count();
   assertThat(criticalCount).isZero();
   

3. Generate JSON report for tracking

---

Code Patterns

Basic Full-Page Scan

@Step("Verify page accessibility - WCAG 2.1 AA")
public void verifyPageAccessibility(WebDriver driver) {
    Results results = new AxeBuilder()
        .withTags(List.of("wcag2a", "wcag2aa", "wcag21a", "wcag21aa"))
        .analyze(driver);

    logViolations(results.getViolations());

    assertThat(results.violationFree())
        .as("Accessibility violations found on: %s", driver.getCurrentUrl())
        .isTrue();
}

Component-Specific Scan

@Step("Verify component accessibility: {selectors}")
public void verifyComponentAccessibility(WebDriver driver, String... selectors) {
    AxeBuilder builder = new AxeBuilder()
        .withTags(List.of("wcag2a", "wcag2aa"));
    
    for (String selector : selectors) {
        builder.include(selector);
    }
    
    Results results = builder.analyze(driver);
    logViolations(results.getViolations());
    
    assertThat(results.violationFree())
        .as("Component accessibility check failed")
        .isTrue();
}

Filter by Impact Level

@Step("Verify no critical accessibility violations")
public void verifyCriticalViolations(WebDriver driver) {
    Results results = new AxeBuilder()
        .withTags(List.of("wcag2a", "wcag2aa"))
        .analyze(driver);

    List<Rule> criticalViolations = results.getViolations().stream()
        .filter(v -> List.of("critical", "serious").contains(v.getImpact()))
        .toList();

    if (!criticalViolations.isEmpty()) {
        logViolations(criticalViolations);
    }

    assertThat(criticalViolations)
        .as("Critical/Serious accessibility violations found")
        .isEmpty();
}

With Documented Exclusions

/**
 * Scan with exclusions for known issues.
 * Exclusions must be documented with ticket reference.
 */
@Step("Verify accessibility with documented exclusions")
public void verifyWithExclusions(WebDriver driver) {
    Results results = new AxeBuilder()
        .withTags(List.of("wcag2a", "wcag2aa"))
        .exclude(".third-party-chat-widget")  // JIRA-1234: Vendor limitation
        .exclude("#legacy-footer")            // JIRA-5678: Scheduled for Q2 fix
        .analyze(driver);

    assertThat(results.violationFree()).isTrue();
}

Violation Logger

private void logViolations(List<Rule> violations) {
    if (violations.isEmpty()) {
        log.info("✓ No accessibility violations found");
        return;
    }

    log.error("✗ Found {} accessibility violations:", violations.size());
    for (Rule violation : violations) {
        log.error("  [{}/{}] {}", 
            violation.getImpact().toUpperCase(),
            violation.getId(),
            violation.getDescription());
        log.error("    Help: {}", violation.getHelpUrl());
        
        for (CheckedNode node : violation.getNodes()) {
            log.error("    Target: {}", String.join(", ", node.getTarget()));
            log.error("    HTML: {}", truncate(node.getHtml(), 100));
        }
    }
}

JUnit 5 Test Class

@Epic("Accessibility")
@Feature("WCAG 2.1 AA Compliance")
class AccessibilityTest extends BaseTest {

    @Test
    @Tag("a11y")
    @Severity(SeverityLevel.CRITICAL)
    @DisplayName("Homepage should meet WCAG 2.1 AA standards")
    void homePage_shouldBeAccessible() {
        driver.get(ConfigReader.get("base.url"));
        waitForPageReady();

        Results results = new AxeBuilder()
            .withTags(List.of("wcag2a", "wcag2aa", "wcag21a", "wcag21aa"))
            .analyze(driver);

        attachResultsToAllure(results);

        SoftAssertions.assertSoftly(softly -> {
            softly.assertThat(results.violationFree())
                .as("Page should have no accessibility violations")
                .isTrue();
        });
    }

    @Test
    @Tag("a11y")
    @DisplayName("Login modal should be keyboard accessible")
    void loginModal_shouldBeKeyboardAccessible() {
        driver.get(ConfigReader.get("base.url"));
        
        // Open modal
        driver.findElement(By.id("login-btn")).click();
        waitForVisible(By.id("login-modal"));

        // Scan modal only
        Results results = new AxeBuilder()
            .withTags(List.of("wcag2a", "wcag2aa"))
            .include("#login-modal")
            .analyze(driver);

        assertThat(results.violationFree()).isTrue();

        // Test keyboard navigation
        WebElement modal = driver.findElement(By.id("login-modal"));
        WebElement firstInput = modal.findElement(By.cssSelector("input:first-of-type"));
        
        assertThat(driver.switchTo().activeElement())
            .as("Focus should be inside modal")
            .isEqualTo(firstInput);

        // Test Escape closes modal
        modal.sendKeys(Keys.ESCAPE);
        assertThat(isDisplayed(By.id("login-modal"))).isFalse();
    }
}

---

Troubleshooting

| Problem | Cause | Solution | |---------|-------|----------| | Axe returns empty results | Page not fully loaded | Add explicit wait for page ready state | | False positives on contrast | Dynamic themes | Test both light and dark modes | | Violations in third-party widgets | Cannot modify vendor code | Use .exclude() with documented ticket | | Incomplete rules | Requires manual review | Log for manual audit, don't auto-fail | | Different results between runs | Async content loading | Ensure deterministic page state before scan | | CI fails but local passes | Different viewport/browser | Use same headless config as CI |

---

Best Practices Checklist

✅ Wait for page ready - Ensure DOM is stable before axe analysis ✅ Scan unique states - Test modal open, form error, empty state separately ✅ Zero tolerance for Critical/Serious - Always fail CI on these ✅ Use specific tags - Define wcag2aa vs best-practice to reduce noise ✅ Log Help URLs - Developers need the link to fix issues ✅ Document exclusions - Every .exclude() needs a JIRA ticket ✅ Test keyboard navigation - Tab order, focus traps, Escape key ✅ Attach JSON reports - Enable tracking violations over time ✅ Combine with manual audit - Axe catches ~30-50% of issues

---

Guardrails (Important Limitations)

⚠️ Automated tooling cannot prove full WCAG conformance - only the presence of certain issues ⚠️ Use automation to prevent regressions - use manual audits for complete coverage ⚠️ Prefer native HTML semantics - use ARIA only when required ⚠️ Never disable rules globally - scope exceptions narrowly with documentation

---

Triage by POUR Principles

| Principle | Focus Areas | Common Violations | |-----------|-------------|-------------------| | Perceivable | Text alternatives, captions, contrast, structure | Missing alt text, low contrast, missing labels | | Operable | Keyboard access, focus order, bypass blocks | Keyboard traps, no skip link, focus not visible | | Understandable | Labels, predictable behavior, error handling | Unclear instructions, unexpected changes | | Robust | Valid HTML, ARIA, name/role/value | Invalid ARIA, duplicate IDs, missing roles |

---

Running Tests

Maven Commands

| Command | Purpose | |---------|---------| | mvn test -Dgroups=a11y | Run all accessibility tests | | mvn test -Dtest=A11yTest | Run specific test class | | mvn test -Dheadless=true | Run headless (CI mode) | | mvn allure:serve | View Allure report with violations |

CI/CD Integration

- name: Run Accessibility Tests
  run: mvn test -Dgroups=a11y -Dheadless=true
  
- name: Upload A11y Report
  uses: actions/upload-artifact@v3
  with:
    name: a11y-report
    path: target/a11y-results/

---

References

• Axe Patterns Guide - AxeBuilder patterns and helpers
• WCAG 2.1 AA Checklist - Manual audit checklist
• Deque Axe Rules - Rule descriptions
• W3C WCAG 2.1 - Official specification
• WAI-ARIA Practices - Widget patterns

---

Quick Reference

| Task | Code Pattern | |------|--------------| | Full page scan | new AxeBuilder().withTags(List.of("wcag2aa")).analyze(driver) | | Component scan | new AxeBuilder().include("#selector").analyze(driver) | | Exclude element | new AxeBuilder().exclude(".ignore").analyze(driver) | | Check violations | results.getViolations().isEmpty() | | Filter critical | .filter(v -> v.getImpact().equals("critical")) | | Get help URL | violation.getHelpUrl() | | Tab navigation | element.sendKeys(Keys.TAB) | | Get focused element | driver.switchTo().activeElement() |