rshankras/test-spec

Test Specification Skill

Generate comprehensive test specification and QA plan for iOS/macOS app testing.

Metadata

• Name: test-spec
• Version: 1.0.0
• Role: QA Engineer
• Author: ProductAgent Team

When This Skill Activates

This skill activates when the user says:

• "generate test spec"
• "create QA plan"
• "write testing guide"
• "generate test cases"
• "create test specification"

Description

You are a QA Engineer AI agent specializing in iOS/macOS app testing. Your job is to transform product requirements and implementation details into a comprehensive test specification that ensures quality, identifies edge cases, and provides clear test cases for both automated and manual testing.

Prerequisites

Before activating this skill, ensure:

1. PRD exists (from prd-generator skill) with features and acceptance criteria
2. IMPLEMENTATION_GUIDE exists (from implementation-guide skill) with code structure
3. UX_SPEC exists (from ux-spec skill) for UI testing scenarios

Input Sources

Read and extract information from:

1. docs/PRD.md

   • All features with acceptance criteria
   • User stories (Given/When/Then format)
   • Success criteria
   • Non-functional requirements

2. docs/IMPLEMENTATION_GUIDE.md

   • All ViewModels to test
   • All data models to test
   • API endpoints to test
   • File structure for organizing tests

3. docs/UX_SPEC.md

   • All user flows
   • All interactions
   • All states (empty, loading, error)
   • Edge cases documented

4. docs/ARCHITECTURE.md

   • Testing strategy overview
   • Tech stack (for choosing testing tools)

Output

Generate: docs/TEST_SPEC.md

Structure:

# Test Specification: [App Name]

**Version**: 1.0.0
**Last Updated**: [Date]
**Status**: Draft / In Review / Approved
**QA Engineer**: QA Engineer AI
**Platform**: iOS [Version]+

---

## 1. Test Strategy

### 1.1 Test Pyramid

Our testing approach follows the test pyramid:

     /\\
    /  \\  UI Tests (10%)
   /    \\  Critical user journeys, happy paths
  /------\\
 /        \\  Integration Tests (20%)
/          \\  API integration, data persistence, service layer

/------------\ / \ Unit Tests (70%) / \ ViewModels, Models, Utilities, Business Logic /------------------\

**Rationale**:
- **Unit Tests (70%)**: Fast, reliable, easy to maintain. Focus on business logic.
- **Integration Tests (20%)**: Test component interactions (API + Database, ViewModel + Service).
- **UI Tests (10%)**: Slow and brittle, only for critical user flows.

### 1.2 Testing Levels

**Level 1: Unit Testing**
- **Scope**: Individual functions, methods, ViewModels, Models
- **Tools**: XCTest
- **Run Frequency**: On every commit (CI/CD)
- **Target Coverage**: 80%+ code coverage

**Level 2: Integration Testing**
- **Scope**: Multiple components working together
- **Tools**: XCTest with mock/stub services
- **Run Frequency**: On every PR merge
- **Target Coverage**: All critical data flows

**Level 3: UI Testing**
- **Scope**: End-to-end user journeys
- **Tools**: XCUITest
- **Run Frequency**: Before release
- **Target Coverage**: All P0 user flows

**Level 4: Manual Testing**
- **Scope**: Exploratory testing, edge cases, UX validation
- **Tools**: TestFlight beta
- **Run Frequency**: Before each release
- **Target Coverage**: Full app walkthrough

### 1.3 Test Environments

**Development**:
- Local Xcode testing
- In-memory database (SwiftData)
- Mock API responses
- Fast feedback loop

**Staging**:
- TestFlight internal testing
- Staging API environment
- Real backend integration
- Pre-production validation

**Production**:
- Phased rollout (10% → 50% → 100%)
- Real user monitoring
- Crash analytics
- Performance monitoring

### 1.4 Testing Tools

| Tool | Purpose | When to Use |
|------|---------|-------------|
| XCTest | Unit & integration tests | Always |
| XCUITest | UI automation tests | Critical flows |
| TestFlight | Beta testing | Pre-release |
| Xcode Instruments | Performance profiling | Optimization phase |
| Accessibility Inspector | Accessibility audit | Every release |
| Network Link Conditioner | Network testing | Edge case testing |

---

## 2. Unit Test Cases

### 2.1 Data Model Tests

Test all `@Model` classes from ARCHITECTURE.md.

#### Test Suite: User Model

**File**: `[AppName]Tests/ModelTests/UserTests.swift`

| Test Case | Setup | Input | Expected Output | Priority |
|-----------|-------|-------|-----------------|----------|
| testUserInitialization | None | name: "John Doe", email: "[email protected]" | User object created with UUID, timestamps set | P0 |
| testUserInitializationWithEmptyName | None | name: "", email: "[email protected]" | User created but isValid returns false | P1 |
| testEmailValidation_Valid | User instance | email: "[email protected]" | isValid returns true | P0 |
| testEmailValidation_Invalid | User instance | email: "invalid.com" | isValid returns false | P0 |
| testEmailValidation_Empty | User instance | email: "" | isValid returns false | P0 |
| testDisplayName_SingleName | User with name: "John" | Call displayName | Returns "John" | P1 |
| testDisplayName_FullName | User with name: "John Doe" | Call displayName | Returns "John" | P1 |
| testInitials_SingleName | User with name: "John" | Call initials | Returns "J" | P2 |
| testInitials_FullName | User with name: "John Doe" | Call initials | Returns "JD" | P2 |
| testUpdateProfile_Name | User instance | updateProfile(name: "Jane") | name updated, updatedAt changed | P1 |
| testUpdateProfile_Email | User instance | updateProfile(email: "[email protected]") | email updated, updatedAt changed | P1 |
| testCodable_Encoding | User instance | Encode to JSON | Valid JSON with snake_case keys | P1 |
| testCodable_Decoding | JSON data | Decode from JSON | User object created correctly | P1 |

**Implementation Example**:

```swift
import XCTest
@testable import [AppName]

final class UserTests: XCTestCase {
    var sut: User!  // System Under Test

    override func setUp() {
        super.setUp()
        // Setup runs before each test
        sut = User(name: "Test User", email: "[email protected]")
    }

    override func tearDown() {
        // Cleanup runs after each test
        sut = nil
        super.tearDown()
    }

    // Test: User initialization creates valid object
    func testUserInitialization() {
        // Given: Setup in setUp()

        // When: User is initialized (done in setUp)

        // Then: Verify properties
        XCTAssertNotNil(sut.id, "ID should be generated")
        XCTAssertEqual(sut.name, "Test User")
        XCTAssertEqual(sut.email, "[email protected]")
        XCTAssertNotNil(sut.createdAt)
        XCTAssertNotNil(sut.updatedAt)
        XCTAssertTrue(sut.items.isEmpty, "New user should have no items")
    }

    // Test: Valid email passes validation
    func testEmailValidation_Valid() {
        // Given
        sut.email = "[email protected]"

        // When
        let isValid = sut.isValid

        // Then
        XCTAssertTrue(isValid, "Valid email should pass validation")
    }

    // Test: Invalid email fails validation
    func testEmailValidation_Invalid() {
        // Given
        sut.email = "invalid.com"  // Missing @

        // When
        let isValid = sut.isValid

        // Then
        XCTAssertFalse(isValid, "Invalid email should fail validation")
    }

    // Test: Display name returns first name only
    func testDisplayName_FullName() {
        // Given
        sut.name = "John Doe"

        // When
        let displayName = sut.displayName

        // Then
        XCTAssertEqual(displayName, "John", "Display name should be first name only")
    }

    // Test: Initials are correctly generated
    func testInitials_FullName() {
        // Given
        sut.name = "John Doe"

        // When
        let initials = sut.initials

        // Then
        XCTAssertEqual(initials, "JD", "Initials should be JD")
    }

    // Test: Update profile changes name and timestamp
    func testUpdateProfile_Name() {
        // Given
        let originalUpdatedAt = sut.updatedAt
        sleep(1)  // Ensure timestamp changes

        // When
        sut.updateProfile(name: "Jane Doe")

        // Then
        XCTAssertEqual(sut.name, "Jane Doe")
        XCTAssertNotEqual(sut.updatedAt, originalUpdatedAt, "updatedAt should change")
    }

    // Test: Codable encoding produces valid JSON
    func testCodable_Encoding() throws {
        // Given: User from setUp

        // When
        let encoder = JSONEncoder()
        let data = try encoder.encode(sut)
        let json = try JSONSerialization.jsonObject(with: data) as? [String: Any]

        // Then
        XCTAssertNotNil(json)
        XCTAssertEqual(json?["name"] as? String, "Test User")
        XCTAssertEqual(json?["email"] as? String, "[email protected]")
        XCTAssertNotNil(json?["id"])
    }
}

Repeat for all models: Item, Category, etc.

---

Test Suite: Item Model

[Similar structure for ItemTests.swift]

Key Test Cases:

• Initialization
• Validation (title not empty, description not empty)
• Status updates
• Relationships (owner, category)
• Codable conformance

---

2.2 ViewModel Tests

Test all ViewModels from IMPLEMENTATION_GUIDE.

Test Suite: HomeViewModel

File: [AppName]Tests/ViewModelTests/HomeViewModelTests.swift

| Test Case | Mock Setup | Action | Expected Result | Priority | |-----------|------------|--------|-----------------|----------| | testInitialState | New ViewModel | None | items empty, isLoading false, showError false | P0 | | testLoadItems_Success | Mock API returns 3 items | Call loadItems() | isLoading becomes false, items count = 3, no error | P0 | | testLoadItems_Failure | Mock API throws error | Call loadItems() | isLoading false, showError true, errorMessage set | P0 | | testLoadItems_EmptyResponse | Mock API returns empty array | Call loadItems() | items empty, no error | P1 | | testRefresh | 3 items already loaded | Call refresh() | isLoading true then false, items refreshed | P1 | | testDeleteItem_Success | 3 items loaded | Delete item[1] | items count = 2, deleted item removed | P0 | | testDeleteItem_Failure | Mock API throws error on delete | Delete item | showError true, item not removed | P1 | | testToggleItemStatus_Active | Item with status .active | toggleItemStatus(item) | item.status becomes .completed | P0 | | testToggleItemStatus_Completed | Item with status .completed | toggleItemStatus(item) | item.status becomes .active | P0 | | testSearchItems_MatchFound | 3 items, searchQuery = "Test" | Set searchQuery | items filtered to matching only | P1 | | testSearchItems_NoMatch | 3 items, searchQuery = "XYZ" | Set searchQuery | items empty | P1 | | testSearchItems_EmptyQuery | searchQuery was "Test" | Set searchQuery = "" | All items shown again | P1 | | testLoadItems_LocalFirst | Mock delays API response | Call loadItems() | Local items shown immediately, then synced | P2 |

Implementation Example:

import XCTest
@testable import [AppName]

final class HomeViewModelTests: XCTestCase {
    var sut: HomeViewModel!
    var mockAPIClient: MockAPIClient!
    var mockDataManager: MockDataManager!

    override func setUp() {
        super.setUp()
        mockAPIClient = MockAPIClient()
        mockDataManager = MockDataManager()
        sut = HomeViewModel(
            apiClient: mockAPIClient,
            dataManager: mockDataManager
        )
    }

    override func tearDown() {
        sut = nil
        mockAPIClient = nil
        mockDataManager = nil
        super.tearDown()
    }

    // Test: Initial state is correct
    @MainActor
    func testInitialState() {
        // Then
        XCTAssertTrue(sut.items.isEmpty)
        XCTAssertFalse(sut.isLoading)
        XCTAssertFalse(sut.showError)
        XCTAssertTrue(sut.searchQuery.isEmpty)
    }

    // Test: Loading items succeeds
    @MainActor
    func testLoadItems_Success() async {
        // Given
        let mockItems = [
            Item(title: "Item 1", description: "Desc 1", owner: mockUser()),
            Item(title: "Item 2", description: "Desc 2", owner: mockUser()),
            Item(title: "Item 3", description: "Desc 3", owner: mockUser())
        ]
        mockDataManager.itemsToReturn = mockItems
        mockAPIClient.itemsToReturn = mockItems

        // When
        await sut.loadItems()

        // Then
        XCTAssertEqual(sut.items.count, 3)
        XCTAssertFalse(sut.isLoading)
        XCTAssertFalse(sut.showError)
    }

    // Test: Loading items handles error
    @MainActor
    func testLoadItems_Failure() async {
        // Given
        mockDataManager.shouldThrowError = true

        // When
        await sut.loadItems()

        // Then
        XCTAssertTrue(sut.showError)
        XCTAssertFalse(sut.errorMessage.isEmpty)
        XCTAssertFalse(sut.isLoading)
    }

    // Test: Deleting item succeeds
    @MainActor
    func testDeleteItem_Success() async {
        // Given
        let mockItems = [
            Item(title: "Item 1", description: "Desc 1", owner: mockUser()),
            Item(title: "Item 2", description: "Desc 2", owner: mockUser())
        ]
        mockDataManager.itemsToReturn = mockItems
        await sut.loadItems()
        let itemToDelete = sut.items[0]

        // When
        await sut.deleteItem(itemToDelete)

        // Then
        XCTAssertEqual(sut.items.count, 1)
        XCTAssertFalse(sut.items.contains(where: { $0.id == itemToDelete.id }))
    }

    // Test: Toggle item status changes status
    @MainActor
    func testToggleItemStatus_ActiveToCompleted() async {
        // Given
        let item = Item(title: "Test", description: "Desc", owner: mockUser())
        item.status = .active
        mockDataManager.itemsToReturn = [item]
        await sut.loadItems()

        // When
        await sut.toggleItemStatus(item)

        // Then
        XCTAssertEqual(item.status, .completed)
    }

    // Helper: Create mock user
    private func mockUser() -> User {
        return User(name: "Mock User", email: "[email protected]")
    }
}

// MARK: - Mock APIClient

class MockAPIClient: APIClient {
    var itemsToReturn: [Item] = []
    var shouldThrowError = false

    override func get<T: Decodable>(_ endpoint: String) async throws -> T {
        if shouldThrowError {
            throw APIError.serverError(500)
        }
        return itemsToReturn as! T
    }

    override func delete<T: Decodable>(_ endpoint: String) async throws -> T {
        if shouldThrowError {
            throw APIError.serverError(500)
        }
        return EmptyResponse() as! T
    }
}

// MARK: - Mock DataManager

class MockDataManager: DataManager {
    var itemsToReturn: [Item] = []
    var shouldThrowError = false

    override func fetch<T: PersistentModel>(
        _ type: T.Type,
        predicate: Predicate<T>? = nil,
        sortBy: [SortDescriptor<T>] = []
    ) throws -> [T] {
        if shouldThrowError {
            throw DataManagerError.fetchFailed(NSError(domain: "test", code: 1))
        }
        return itemsToReturn as! [T]
    }

    override func delete<T: PersistentModel>(_ model: T) throws {
        if shouldThrowError {
            throw DataManagerError.deleteFailed(NSError(domain: "test", code: 1))
        }
        // Remove from mock array
        itemsToReturn.removeAll { ($0 as AnyObject) === (model as AnyObject) }
    }
}

Repeat for all ViewModels: ItemDetailViewModel, AddEditItemViewModel, SettingsViewModel, etc.

---

2.3 Service Tests

Test API client, DataManager, and other services.

Test Suite: APIClient

Key Test Cases:

• Successful GET request
• Successful POST request with body
• 401 Unauthorized error handling
• 404 Not Found error handling
• 500 Server error handling
• Network timeout handling
• JSON decoding error handling
• Invalid URL handling

Test Suite: DataManager

Key Test Cases:

• Create model instance
• Fetch all instances
• Fetch with predicate filter
• Fetch by ID
• Update model instance
• Delete model instance
• Delete all instances
• Specialized queries (fetchItems(byStatus:), searchItems(query:))

---

3. Integration Test Cases

Test multiple components working together.

3.1 Data Persistence Integration

| Test Case | Setup | Action | Expected Result | Priority | |-----------|-------|--------|-----------------|----------| | testSaveAndFetch | Empty database | Create user, save, fetch all | User retrieved successfully | P0 | | testUpdateAndFetch | User in database | Modify user, save, fetch | Changes persisted | P0 | | testDeleteAndFetch | User in database | Delete user, fetch all | User no longer exists | P0 | | testRelationshipCascade | User with 3 items | Delete user | All items also deleted | P0 | | testQueryWithPredicate | 5 items (2 active, 3 completed) | Fetch active only | Returns 2 items | P1 |

3.2 API Integration

| Test Case | Setup | Action | Expected Result | Priority | |-----------|-------|--------|-----------------|----------| | testFetchItems_APIToDatabase | Empty database, mock API returns 3 items | Call fetchItems() | 3 items saved to database | P0 | | testCreateItem_DatabaseToAPI | Mock API | Create item locally, sync | POST request sent to API | P0 | | testSyncConflict | Local and remote modified same item | Sync | Conflict resolved (last write wins) | P1 | | testOfflineMode | No network | Create item locally | Item saved locally, queued for sync | P0 | | testOnlineRestore | Previously offline | Network restored | Queued items synced to API | P1 |

---

4. UI Test Cases

Test end-to-end user journeys with XCUITest.

4.1 Critical User Journey 1: Onboarding and First Item Creation

Priority: P0 (Must pass before release) Estimated Duration: 30 seconds Frequency: Run on every release candidate

| Step | User Action | Expected UI State | Assertions | Priority | |------|-------------|-------------------|------------|----------| | 1 | Launch app (first time) | Welcome screen shown | "Welcome to [AppName]" exists, "Get Started" button exists | P0 | | 2 | Tap "Get Started" button | Registration screen shown | Email and name fields visible, "Continue" button exists | P0 | | 3 | Enter name: "Test User" | Name field populated | Name field value = "Test User" | P0 | | 4 | Enter email: "[email protected]" | Email field populated | Email field value = "[email protected]" | P0 | | 5 | Tap "Continue" button | Onboarding Step 1 shown | Progress dots visible, skip button visible, next button exists | P0 | | 6 | Swipe left (or tap Next) | Onboarding Step 2 shown | Different content shown, progress indicator updated | P1 | | 7 | Tap "Skip" button | Home screen shown with empty state | "No Items Yet" message visible, "+" button in navbar | P0 | | 8 | Tap "+" button in navbar | Add Item sheet presented | Title field, description field, save button visible | P0 | | 9 | Enter title: "My First Item" | Title populated | Title field value = "My First Item" | P0 | | 10 | Enter description: "Test description" | Description populated | Description field value = "Test description" | P0 | | 11 | Tap "Save" button | Sheet dismissed, Home screen shown | Item appears in list with title "My First Item" | P0 | | 12 | Verify item visible | Item card visible in list | Card contains "My First Item" and "Test description" | P0 |

Implementation:

import XCTest

final class OnboardingUITests: XCTestCase {
    let app = XCUIApplication()

    override func setUpWithError() throws {
        continueAfterFailure = false

        // Reset app to first-time user state
        app.launchArguments = ["--uitesting", "--reset"]
        app.launch()
    }

    func testOnboardingAndFirstItemCreation() throws {
        // Step 1: Verify welcome screen
        let welcomeTitle = app.staticTexts["Welcome to [AppName]"]
        XCTAssertTrue(welcomeTitle.waitForExistence(timeout: 2))

        let getStartedButton = app.buttons["Get Started"]
        XCTAssertTrue(getStartedButton.exists)

        // Step 2: Tap Get Started
        getStartedButton.tap()

        // Step 3-4: Fill registration
        let nameField = app.textFields["Name"]
        XCTAssertTrue(nameField.waitForExistence(timeout: 2))
        nameField.tap()
        nameField.typeText("Test User")

        let emailField = app.textFields["Email"]
        emailField.tap()
        emailField.typeText("[email protected]")

        // Step 5: Continue to onboarding
        app.buttons["Continue"].tap()

        // Verify onboarding screen
        XCTAssertTrue(app.staticTexts["Onboarding Step 1"].waitForExistence(timeout: 2))

        // Step 7: Skip onboarding
        app.buttons["Skip"].tap()

        // Step 8: Verify Home empty state
        let emptyStateMessage = app.staticTexts["No Items Yet"]
        XCTAssertTrue(emptyStateMessage.waitForExistence(timeout: 2))

        // Step 9: Tap + button
        let addButton = app.navigationBars.buttons["Add Item"]
        XCTAssertTrue(addButton.exists)
        addButton.tap()

        // Step 10-11: Fill add item form
        let titleField = app.textFields["Title"]
        XCTAssertTrue(titleField.waitForExistence(timeout: 2))
        titleField.tap()
        titleField.typeText("My First Item")

        let descriptionField = app.textViews["Description"]
        descriptionField.tap()
        descriptionField.typeText("Test description")

        // Step 12: Save
        app.buttons["Save"].tap()

        // Step 13: Verify item appears
        let itemCard = app.staticTexts["My First Item"]
        XCTAssertTrue(itemCard.waitForExistence(timeout: 3))

        // Verify subtitle
        let subtitle = app.staticTexts["Test description"]
        XCTAssertTrue(subtitle.exists)
    }
}

---

4.2 Critical User Journey 2: View and Edit Item

[Similar detailed test case with steps, assertions, and implementation]

4.3 Critical User Journey 3: Delete Item

[Similar structure]

4.4 Critical User Journey 4: Search Items

[Similar structure]

---

5. Accessibility Testing

5.1 VoiceOver Testing

Manual Test Cases:

| Screen | Element | Expected VoiceOver Label | Expected Behavior | Priority | |--------|---------|-------------------------|-------------------|----------| | Home | Add button | "Add new item, button" | Announces label, tapping activates button | P0 | | Home | Item card | "[Item title], [subtitle], [status], button" | Announces all relevant info, tapping navigates | P0 | | Home | Search field | "Search items, search field" | Announces hint, typing works | P1 | | Detail | Edit button | "Edit item, button" | Clear action announced | P0 | | Add/Edit | Title field | "Title, text field, required" | Indicates required field | P0 | | Add/Edit | Save button | "Save item, button" | Clear action, disabled state announced | P0 | | Settings | Toggle | "Notifications, on, toggle switch" | State announced, double-tap toggles | P1 |

Automated VoiceOver Tests:

func testVoiceOverLabels() {
    // Enable accessibility features
    app.launchArguments += ["-UIAccessibilitySpeakScreenEnabled", "1"]
    app.launch()

    // Navigate to Home
    let addButton = app.buttons["Add Item"]
    XCTAssertEqual(addButton.label, "Add new item")
    XCTAssertTrue(addButton.isAccessibilityElement)

    // Check item card
    let itemCard = app.buttons.matching(identifier: "ItemCard").firstMatch
    XCTAssertTrue(itemCard.label.contains("Item title"))
    XCTAssertTrue(itemCard.isAccessibilityElement)
}

5.2 Dynamic Type Testing

Test Cases:

| Size | Expected Behavior | Pass Criteria | Priority | |------|-------------------|---------------|----------| | Extra Small (XS) | All text readable, buttons not truncated | No overlap, all elements visible | P1 | | Default | Standard layout | Reference design | P0 | | Extra Large (XL) | Text larger, layout adjusts | Scrollable, no truncation | P0 | | Accessibility XXL | Text very large, stacked layout | Fully readable, scrollable | P1 |

Test Implementation:

func testDynamicType_ExtraLarge() {
    // Set large text size
    app.launchArguments += ["-UIPreferredContentSizeCategoryName", "UICTContentSizeCategoryAccessibilityXL"]
    app.launch()

    // Verify text is not truncated
    let title = app.staticTexts["Item Title"]
    XCTAssertTrue(title.exists)
    XCTAssertFalse(title.label.hasSuffix("..."), "Text should not be truncated")

    // Verify buttons are still tappable
    let addButton = app.buttons["Add Item"]
    XCTAssertTrue(addButton.exists)
    XCTAssertGreaterThan(addButton.frame.height, 44, "Button should be at least 44pt tall")
}

5.3 Color Contrast Testing

Automated Checks:

| Element | Foreground | Background | Ratio | WCAG AA Pass | Priority | |---------|------------|------------|-------|--------------|----------| | Primary button text | #FFFFFF | #007AFF | 4.5:1+ | ✅ Yes | P0 | | Body text | #000000 | #FFFFFF | 21:1 | ✅ Yes | P0 | | Secondary text | #3C3C43 (60%) | #FFFFFF | 4.6:1 | ✅ Yes | P0 | | Link text | #007AFF | #FFFFFF | 4.5:1+ | ✅ Yes | P0 |

Tool: Xcode Accessibility Inspector → Color Contrast

---

6. Performance Testing

6.1 Performance Benchmarks

| Metric | Target | Measurement Method | Priority | |--------|--------|-------------------|----------| | App launch time (cold) | < 1.5s | XCTest measure block | P0 | | App launch time (warm) | < 0.5s | XCTest measure block | P1 | | Home screen load | < 500ms | Time to first content | P0 | | Scroll FPS | 60 fps | Instruments (Core Animation) | P0 | | Memory usage (idle) | < 50MB | Instruments (Allocations) | P1 | | Memory usage (active) | < 150MB | Instruments (Allocations) | P1 | | Network request time | < 2s (95th percentile) | Analytics | P1 |

Test Implementation:

func testLaunchPerformance() {
    measure(metrics: [XCTApplicationLaunchMetric()]) {
        app.launch()
    }
    // Xcode will report average launch time
}

func testHomeScreenLoadPerformance() {
    app.launch()

    measure {
        // Navigate to Home
        app.tabBars.buttons["Home"].tap()

        // Wait for content
        _ = app.tables.cells.firstMatch.waitForExistence(timeout: 5)
    }
}

6.2 Load Testing

| Scenario | Data Volume | Expected Behavior | Pass Criteria | Priority | |----------|-------------|-------------------|---------------|----------| | Large list | 1000 items | Smooth scrolling, no lag | 60 fps maintained | P1 | | Heavy images | 50 high-res images | Progressive loading | No memory warnings | P1 | | Large search | Search in 1000 items | Results in < 1s | Fast filtering | P2 | | Rapid scrolling | Scroll 100 items quickly | No crashes, smooth | No dropped frames | P1 |

---

7. Edge Cases & Error Scenarios

7.1 Network Conditions

| Condition | User Action | Expected Behavior | Verification | Priority | |-----------|-------------|-------------------|--------------|----------| | No network (airplane mode) | Launch app | Cached data shown, offline banner | Data visible, banner "No internet connection" | P0 | | No network | Tap refresh | Error message, cached data remains | "Can't refresh. Check connection." | P0 | | Slow network (3G) | Load items | Loading indicator, timeout after 30s | Spinner shows, then timeout message | P1 | | Network timeout | API call exceeds 30s | Timeout error, graceful fallback | "Request timed out. Try again." | P1 | | Network restored | Was offline, now online | Auto-sync, banner dismissed | Data refreshes, "Back online" toast | P1 | | Intermittent connection | Random disconnects | Retry with exponential backoff | Eventually succeeds or clear error | P2 |

Test with: Settings → Developer → Network Link Conditioner

7.2 Data Validation

| Invalid Input | User Action | Expected Behavior | Verification | Priority | |---------------|-------------|-------------------|--------------|----------| | Empty required field | Tap Save with empty title | Error message, save disabled | Red outline, "Title is required" | P0 | | Invalid email format | Enter "invalid.com" | Inline error shown | "Please enter a valid email" | P0 | | Too long input | Enter 500 character title | Character limit enforced | Max 150 chars, counter shows "150/150" | P1 | | Special characters | Enter "Item <script>" in title | Sanitized or rejected | No script execution, saved safely | P0 | | Duplicate entry | Create item with existing title | Warning shown, allow override | "Item exists. Continue?" | P1 | | Invalid date | Select date in past (if not allowed) | Error shown, date reset | "Date must be in future" | P1 |

7.3 Permission Handling

| Permission | Denied State | User Action | Expected Behavior | Priority | |------------|--------------|-------------|-------------------|----------| | Camera | Denied | Tap "Add Photo" | Alert: "Camera access needed. Open Settings?" | P0 | | Camera | Denied | User opens Settings | Deep link to app settings | P1 | | Photos | Denied | Tap "Choose Photo" | Photo picker not shown, request permission | P0 | | Notifications | Denied | App needs to notify | App works without notifications | P1 | | Location | Denied | Feature requires location | Feature disabled, explanation shown | P1 |

7.4 Device Conditions

| Condition | Expected Behavior | Verification | Priority | |-----------|-------------------|--------------|----------| | Low battery mode | Reduced animations, background tasks paused | Respects system setting | P1 | | Low storage | Can't save images | Error: "Not enough storage" | P1 | | Rotation (if supported) | Layout adapts | No clipping, all elements visible | P1 | | Multitasking (iPad) | App works in split view | Responsive layout | P2 | | Background mode | App suspended, returns | State preserved | P0 | | App killed by system | Force quit, relaunch | State restored from last save | P0 |

---

8. Security Testing

8.1 Data Security

| Test | Action | Expected Result | Priority | |------|--------|-----------------|----------| | Keychain storage | Store auth token | Token encrypted in Keychain | P0 | | API token exposure | Inspect network traffic | Token in Authorization header only | P0 | | Sensitive data logging | Check console logs | No passwords/tokens logged | P0 | | Screenshot protection | App contains sensitive data | Blur/hide when app switching | P1 |

8.2 Privacy

| Test | Action | Expected Result | Priority | |------|--------|-----------------|----------| | Privacy manifest | Check PrivacyInfo.xcprivacy | All data collection disclosed | P0 | | Third-party SDKs | Inspect network requests | No unexpected tracking | P0 | | Data export | User requests data | All data provided in readable format | P1 | | Account deletion | User deletes account | All data removed from server | P0 |

---

9. Regression Testing

Run before every release to ensure new changes don't break existing functionality.

9.1 Smoke Test Checklist

Duration: 15 minutes Frequency: Every build

• [ ] App launches without crash
• [ ] Login/registration works
• [ ] Home screen loads
• [ ] Can create new item
• [ ] Can view item detail
• [ ] Can edit item
• [ ] Can delete item
• [ ] Search works
• [ ] Settings accessible
• [ ] Can log out

9.2 Full Regression Suite

Duration: 2 hours Frequency: Every release candidate

• [ ] All P0 unit tests pass (100%)
• [ ] All P0 UI tests pass (100%)
• [ ] All critical user journeys tested manually
• [ ] Accessibility audit passed (VoiceOver, Dynamic Type)
• [ ] Performance benchmarks met
• [ ] No memory leaks (Instruments check)
• [ ] Network error scenarios tested
• [ ] Edge cases from Section 7 verified
• [ ] Security checklist completed
• [ ] Privacy manifest up to date

---

10. Beta Testing (TestFlight)

10.1 Beta Test Plan

Participants: 20-50 external beta testers Duration: 2 weeks before App Store submission Platforms: iPhone (all sizes), iPad (if supported) iOS Versions: [minimum deployment target] – latest

Recruitment:

• Internal team members
• Trusted users from target audience
• Mix of tech-savvy and average users
• Geographic diversity (if localized)

Feedback Collection:

1. In-app feedback form: Button in Settings → "Send Feedback"
2. TestFlight feedback: Native TestFlight feedback
3. Crash reports: Automatically collected
4. Survey: Email survey after 1 week of use

10.2 Beta Testing Focus Areas

Week 1: Functionality Testing

• [ ] All features work as expected
• [ ] No critical bugs
• [ ] Performance is acceptable
• [ ] UI/UX is intuitive

Week 2: Real-World Usage

• [ ] App fits into users' workflow
• [ ] Identify missing features
• [ ] Collect feature requests
• [ ] Assess onboarding clarity

10.3 Beta Success Criteria

Before proceeding to App Store submission:

• [ ] Crash-free rate > 99.5%
• [ ] Average rating from beta testers > 4.0/5.0
• [ ] No P0 bugs reported in last 3 days
• [ ] All critical feedback addressed or planned for v1.1
• [ ] At least 75% of testers completed onboarding
• [ ] At least 50% of testers used app 3+ times

Feedback Analysis:

• Categorize feedback: bug, feature request, UX issue, question
• Prioritize: P0 (fix before launch), P1 (fix in v1.1), P2 (backlog)
• Track resolution: open, in progress, fixed, won't fix

---

11. Test Automation

11.1 CI/CD Integration

Platform: Xcode Cloud / GitHub Actions / Fastlane Trigger: On every pull request and merge to main

Pipeline:

name: iOS CI

on: [pull_request, push]

jobs:
  test:
    runs-on: macos-latest
    steps:
      - uses: actions/checkout@v3

      - name: Select Xcode version
        run: sudo xcode-select -s /Applications/Xcode_15.0.app

      - name: Run SwiftLint
        run: swiftlint lint --strict

      - name: Build
        run: |
          xcodebuild clean build \
            -scheme [AppName] \
            -destination 'platform=iOS Simulator,name=iPhone 15'

      - name: Run Unit Tests
        run: |
          xcodebuild test \
            -scheme [AppName] \
            -destination 'platform=iOS Simulator,name=iPhone 15' \
            -only-testing:[AppName]Tests

      - name: Run UI Tests
        run: |
          xcodebuild test \
            -scheme [AppName] \
            -destination 'platform=iOS Simulator,name=iPhone 15' \
            -only-testing:[AppName]UITests

      - name: Generate Code Coverage
        run: |
          xcodebuild test \
            -scheme [AppName] \
            -enableCodeCoverage YES \
            -derivedDataPath ./DerivedData

      - name: Upload Coverage to Codecov
        uses: codecov/codecov-action@v3

11.2 Test Reports

Generate HTML test report:

xcodebuild test \
  -scheme [AppName] \
  -resultBundlePath TestResults.xcresult

xcrun xcresulttool get --format html TestResults.xcresult > test-report.html

Code Coverage Report:

• Target: 80%+ overall coverage
• Critical paths: 95%+ coverage (authentication, data persistence)
• View in Xcode: Product → Show Code Coverage

---

12. Bug Tracking

12.1 Bug Report Template

File in issue tracker (GitHub Issues, Jira, etc.):

**Title**: [Concise, descriptive bug title]

**Severity**: Critical / High / Medium / Low

**Priority**: P0 / P1 / P2 / P3

**Affected Version**: 1.0.0

**Environment**:
- Device: iPhone 15 Pro
- iOS Version: 17.5
- App Version: 1.0.0 (Build 42)

**Steps to Reproduce**:
1. Launch app
2. Navigate to Home
3. Tap on item "Test Item"
4. Observe crash

**Expected Behavior**:
Item detail screen should open without crash.

**Actual Behavior**:
App crashes immediately when tapping item.

**Screenshots/Videos**:
[Attach if available]

**Crash Logs**:

[Paste crash log from Xcode Organizer]

**Additional Context**:
Happens only with items that have nil thumbnailURL.

**Possible Fix**:
Add nil check in ItemDetailView line 47.

12.2 Bug Severity Definitions

Critical (P0):

• App crashes on launch
• Data loss
• Security vulnerability
• Cannot complete core user flow
• Fix immediately, hotfix if needed

High (P1):

• Feature completely broken
• Frequent crashes (> 1% of sessions)
• Major UI issues
• Fix before next release

Medium (P2):

• Feature partially broken
• Rare crashes (< 0.1% of sessions)
• Minor UI issues
• Workaround exists
• Fix in next minor update

Low (P3):

• Cosmetic issues
• Feature enhancement
• Edge case issues
• Fix when convenient

---

13. Post-Launch Monitoring

13.1 Metrics to Track

Crash Analytics (Xcode Organizer, Firebase Crashlytics):

• Crash-free rate: Target > 99.5%
• Top crashes by volume
• Crash trends over time

Performance Monitoring:

• App launch time: p50, p95, p99
• Screen load time
• API response time
• Memory usage

Usage Analytics (App Analytics, Firebase Analytics):

• Daily/Monthly Active Users (DAU/MAU)
• Session length
• Session frequency
• Feature usage rates
• Funnel completion rates

App Store Metrics:

• Rating: Target > 4.5 stars
• Reviews: Respond to all reviews
• Download trends
• Conversion rate (impressions → downloads)

User Feedback:

• Support ticket volume
• Common complaints
• Feature requests
• Positive feedback themes

13.2 Alerting Thresholds

Critical Alerts (Page immediately):

• Crash rate > 1%
• App Store rating drops below 3.0
• API error rate > 5%
• Server downtime

High Priority Alerts (Check within 1 hour):

• Crash rate > 0.5%
• App Store rating drops below 4.0
• API error rate > 2%
• Support ticket spike

Medium Priority (Check daily):

• Crash rate > 0.1%
• Feature usage < 10% of users (for key features)
• Session length decreasing trend

---

14. Test Documentation

14.1 Test Case Format

Each test case should document:

• ID: TC-001
• Feature: Feature name from PRD
• Priority: P0/P1/P2
• Type: Unit/Integration/UI/Manual
• Preconditions: State before test
• Steps: Numbered steps
• Expected Result: Clear pass criteria
• Actual Result: Filled after execution
• Status: Pass/Fail/Blocked
• Notes: Any observations

14.2 Test Execution Tracking

Test Run Spreadsheet:

| ID | Test Case | Priority | Type | Status | Notes | Date | Tester | |----|-----------|----------|------|--------|-------|------|--------| | TC-001 | User login success | P0 | UI | Pass | | 2024-01-15 | QA | | TC-002 | User login invalid email | P0 | UI | Pass | | 2024-01-15 | QA | | TC-003 | Create item success | P0 | UI | Fail | Crash on save | 2024-01-15 | QA |

Test Coverage Matrix:

| Feature (PRD) | Unit Tests | Integration Tests | UI Tests | Manual Tests | Coverage % | |---------------|------------|-------------------|----------|--------------|------------| | User registration | 12/12 ✅ | 3/3 ✅ | 2/2 ✅ | 5/5 ✅ | 100% | | Item CRUD | 20/20 ✅ | 5/5 ✅ | 8/10 ⚠️ | 10/10 ✅ | 90% | | Search | 8/8 ✅ | 2/2 ✅ | 3/3 ✅ | 3/3 ✅ | 100% |

---

15. Release Readiness Checklist

Before submitting to App Store:

15.1 Testing Complete

• [ ] All P0 unit tests pass (100%)
• [ ] All P1 unit tests pass (> 95%)
• [ ] All P0 UI tests pass (100%)
• [ ] No P0 bugs open
• [ ] No P1 bugs open (or accepted as known issues)
• [ ] Code coverage > 80%
• [ ] Performance benchmarks met
• [ ] Memory leaks checked (0 leaks)
• [ ] Accessibility audit passed
• [ ] Security audit passed

15.2 Manual Testing

• [ ] Full regression suite completed
• [ ] All critical user journeys tested on real devices
• [ ] Tested on iPhone SE (smallest screen)
• [ ] Tested on iPhone Pro Max (largest screen)
• [ ] Tested on iPad (if supported)
• [ ] Tested on iOS 17.0 (minimum version)
• [ ] Tested on latest iOS version
• [ ] Dark mode tested
• [ ] Localization tested (all supported languages)

15.3 Beta Testing

• [ ] TestFlight beta completed (2 weeks)
• [ ] Crash-free rate > 99.5%
• [ ] Beta tester rating > 4.0/5.0
• [ ] All critical beta feedback addressed
• [ ] No new crashes in last 48 hours

15.4 Final Checks

• [ ] App Store assets ready (from RELEASE_SPEC)
• [ ] Privacy manifest accurate
• [ ] App Store guidelines compliance verified
• [ ] Legal review completed (terms, privacy policy)
• [ ] Marketing materials ready
• [ ] Support infrastructure ready (email, FAQ)

---

Appendix A: Test Data

A.1 Test Users

{
  "valid_user": {
    "name": "Test User",
    "email": "[email protected]",
    "password": "Password123!"
  },
  "invalid_email": {
    "name": "Invalid",
    "email": "invalid.com",
    "password": "Password123!"
  },
  "empty_fields": {
    "name": "",
    "email": "",
    "password": ""
  }
}

A.2 Test Items

{
  "valid_item": {
    "title": "Test Item",
    "description": "This is a test description",
    "status": "active"
  },
  "long_title": {
    "title": "[150 character string...]",
    "description": "Normal description",
    "status": "active"
  },
  "special_chars": {
    "title": "Item with <script>alert('XSS')</script>",
    "description": "Testing sanitization",
    "status": "active"
  }
}

---

Appendix B: Glossary

• P0: Critical priority - must be fixed before release
• P1: High priority - should be fixed before release
• P2: Medium priority - can be deferred to next release
• P3: Low priority - backlog
• SUT: System Under Test
• Mock: Simulated object for testing
• Stub: Predefined test data
• Regression: Re-testing to ensure new changes don't break existing functionality
• Smoke Test: Quick test of basic functionality
• Sanity Test: Quick test after bug fix to verify it's resolved
• Coverage: Percentage of code executed by tests

---

Document History: | Version | Date | Author | Changes | |---------|------|--------|---------| | 1.0.0 | [Date] | QA Engineer AI | Initial test specification created |

---

## Execution Instructions

When activated, follow these steps:

1. **Read All Specification Documents**

• Read docs/PRD.md (features, acceptance criteria, user stories)
• Read docs/IMPLEMENTATION_GUIDE.md (all ViewModels, models, services)
• Read docs/UX_SPEC.md (all screens, user flows, states)
• Read docs/ARCHITECTURE.md (testing strategy)

2. **Extract Testing Requirements**
- List all features to test (from PRD)
- List all ViewModels to unit test (from IMPLEMENTATION_GUIDE)
- List all models to unit test
- List all user flows to UI test (from UX_SPEC)
- Identify edge cases from UX_SPEC error states

3. **Generate Unit Test Cases**
- For each Model: test initialization, validation, methods, relationships, Codable
- For each ViewModel: test loading, errors, user actions, state management
- For each Service: test API calls, error handling, data operations

4. **Generate Integration Test Cases**
- Database + API sync
- ViewModel + Service integration
- Multi-component workflows

5. **Generate UI Test Cases**
- Critical user journeys (from PRD P0 features)
- Include all steps with assertions
- Provide complete XCUITest implementation

6. **Generate Accessibility Tests**
- VoiceOver labels for all interactive elements
- Dynamic Type scenarios
- Color contrast checks

7. **Generate Performance Tests**
- Launch time benchmarks
- Load testing scenarios
- Memory usage targets

8. **Document Edge Cases**
- Network conditions (offline, slow, timeout)
- Data validation (empty, invalid, too long)
- Permission handling (denied, restricted)
- Device conditions (low battery, low storage)

9. **Create Beta Testing Plan**
- Participant criteria
- Duration and timeline
- Feedback collection methods
- Success criteria

10. **Write Complete Test Spec**
 ```
 Write to: docs/TEST_SPEC.md
 Target length: 2000-3000 lines
 Format: Markdown with code examples
 ```

11. **Present Summary**
 ```
 ✅ Test Specification generated!

 🧪 **Test Spec Summary**:
 - Unit test cases: [X] tests across [Y] test suites
 - Integration test cases: [Z] scenarios
 - UI test cases: [N] critical user journeys with full implementation
 - Accessibility tests: VoiceOver, Dynamic Type, Contrast
 - Performance benchmarks: Launch time, memory, FPS
 - Edge cases documented: [M] scenarios
 - Beta testing plan: 2-week plan with [P] participants

 **Coverage**:
 - All features from PRD have test cases
 - All ViewModels have unit tests
 - All critical flows have UI tests
 - Accessibility fully covered
 - Performance targets defined

 **Next Steps**:
 1. Review test cases in docs/TEST_SPEC.md
 2. Implement unit tests as development progresses
 3. Set up CI/CD pipeline for automated testing
 4. Execute manual test cases before release
 5. Run beta test program via TestFlight

 **Developer can now**:
 - Write tests alongside implementation
 - Achieve 80%+ code coverage
 - Ensure all features are tested
 - Catch bugs before release

 Ready to proceed to release specification?
 ```

12. **Iterate Based on Feedback**
 If user requests changes:
 - Add missing test cases
 - Expand specific areas (e.g., more edge cases)
 - Provide more implementation examples
 - Adjust priorities

---

## Quality Guidelines

1. **Be Comprehensive**: Cover every feature from PRD
- Unit tests for all business logic
- UI tests for all critical flows
- Integration tests for key workflows
- Edge cases for all error scenarios

2. **Be Specific**: Test cases must be actionable
- Clear steps
- Clear expected results
- Clear pass/fail criteria
- Include test data

3. **Provide Code Examples**: Not just test case descriptions
- Show complete XCTest implementations
- Show XCUITest implementations
- Show mock objects
- Show test setup/teardown

4. **Think Like QA**: What could go wrong?
- Empty states
- Invalid inputs
- Network failures
- Permission denials
- Edge cases

5. **Consider Real Users**: Beta testing is crucial
- Real devices, real usage patterns
- Diverse user base
- Real-world network conditions
- Actual user feedback

6. **Enable Automation**: CI/CD integration
- Tests should run automatically
- Fast feedback loop
- Code coverage tracking
- Fail fast on critical issues

---

## Integration with Workflow

This skill is typically:
- **Fifth step** in specification generation
- Activated after PRD, Architecture, UX, and Implementation Guide are complete
- Followed by release-spec
- Critical for ensuring quality before App Store submission

The test specification ensures the app works correctly and provides confidence for release.

---

## Notes

- Every feature in PRD must have corresponding test cases
- Test early, test often - don't wait until the end
- Automated tests save time in the long run
- Manual testing still crucial for UX validation
- Beta testing provides real-world feedback
- Bug tracking is part of the testing process
- Performance and accessibility are not optional
- Security testing must be thorough
- Document all test cases for repeatability