tachyon-beep/quantitative-management

Quantitative Management

Purpose & Context

Implements CMMI MA (Measurement & Analysis), QPM (Quantitative Project Management), and OPP (Organizational Process Performance) through data-driven decision making, statistical process control, and predictive analytics.

Core Principle: Measure what matters, use data to drive decisions, distinguish signal from noise.

Avoid: Measurement theater (tracking without action), vanity metrics (looks good but no value), gaming metrics (optimizing numbers vs processes).

---

When to Use This Skill

Triggers:

• "What should we measure?"
• "How do I know if this variation is a problem?"
• "How do I establish process baselines?"
• "What's the difference between Level 2, 3, and 4 measurement?"
• "How do I use DORA metrics?"
• "How do I implement statistical process control?"
• "How do I use metrics for project decisions?"

Use this when:

• Establishing a measurement program
• Moving from Level 2 → 3 → 4
• Implementing DORA metrics
• Analyzing process performance
• Detecting process instability
• Predicting project outcomes
• Making data-driven decisions

Do NOT use for:

• Platform-specific metric collection → See platform-integration skill
• One-time reporting → Just write the report
• Metrics mandated by compliance → See governance-and-risk skill

---

Quick Reference: Finding What You Need

| You Want To... | Reference Sheet | Key Content | |----------------|-----------------|-------------| | Plan what to measure | measurement-planning.md | GQM methodology, cost/value analysis, anti-patterns | | Choose metrics | key-metrics-by-domain.md | Quality, velocity, stability, deployment metrics | | Implement DORA | dora-metrics.md | 4 DORA metrics, collection automation, baselines | | Analyze variation | statistical-analysis.md | Control charts, SPC, trend detection | | Establish baselines | process-baselines.md | Historical data analysis, baseline maintenance | | Make data-driven decisions | quantitative-management.md | QPM, prediction models, Level 4 practices | | Scale L2→L3→L4 | level-scaling.md | Maturity progression, requirements by level |

---

Maturity Level Guidance

Level 2 (Managed): Basic Tracking

Focus: Capture data for visibility

Measurements:

• Counts (defects found, tests run, deployments)
• Dates (when tasks completed, releases shipped)
• Simple averages (average lead time, average velocity)

Tools: Spreadsheets, basic dashboards, manual collection acceptable

Example: Track number of bugs found per sprint, average time to close bugs

Limitation: No statistical analysis, no baselines, reactive only

Level 3 (Defined): Organizational Baselines & Trends

Focus: Establish norms, detect trends

Measurements:

• Organizational baselines (typical velocity, typical defect density)
• Trend analysis (improving or degrading?)
• Comparative analysis (this project vs org average)

Tools: Analytics platforms, automated collection, historical databases

Example: Know that your organization typically delivers 30 story points/sprint ± 10, and current project is trending 25 (slightly below average)

Advancement over L2: Baselines provide context, trends show direction

Level 4 (Quantitatively Managed): Statistical Process Control

Focus: Predict outcomes, control variation

Measurements:

• Statistical process control (control charts, control limits)
• Prediction models (effort estimation, defect prediction)
• Process performance objectives (quantitative targets)
• Variance analysis (special cause vs common cause)

Tools: Statistical packages (R, Python), control chart software, Monte Carlo simulation

Example: Use control charts to detect when defect rate exceeds upper control limit (special cause), trigger root cause analysis. Use regression model to predict project completion date with 90% confidence interval.

Advancement over L3: Predictive, not just reactive; distinguishes noise from signal

---

Measurement Anti-Patterns

| Anti-Pattern | Symptom | Better Approach | |--------------|---------|-----------------| | Measurement Theater | Tracking many metrics, no action taken | Use GQM to link metrics to decisions | | Vanity Metrics | Numbers look impressive but don't drive improvement | Focus on actionable metrics with clear business value | | Gaming Metrics | Teams optimize numbers instead of processes | Measure outcomes, not activities; use multiple metrics | | Lagging-Only | Only measure results after the fact | Balance lagging with leading indicators | | Analysis Paralysis | Too many metrics, can't make decisions | Focus on critical few (3-5 key metrics) | | Flying Blind | No metrics, decisions based on opinion | Start with Level 2 basics, build up | | Over-Precision | Measuring to 3 decimal places when ±20% is noise | Match precision to decision granularity |

---

Integration with Other CMMI Process Areas

REQM (Requirements Management):

• Metric: Requirements volatility (changes per sprint)
• Metric: Requirements traceability coverage (% with tests)
• Use baseline to detect abnormal churn

CM (Configuration Management):

• Metric: Branch protection compliance
• Metric: Baseline stability (changes after freeze)
• Use SPC to detect configuration drift

VER/VAL (Verification/Validation):

• Metric: Test coverage, defect density, escape rate
• Metric: Code review effectiveness
• Use control charts for quality gates

RSKM (Risk Management):

• Metric: Risk exposure (probability × impact)
• Metric: Risk burndown (risks closed over time)
• Use trends to predict risk realization

DAR (Decision Analysis & Resolution):

• Use measurement data as input to DAR
• Track decision quality (outcomes vs predictions)

Cross-references:

• requirements-lifecycle - Metrics for requirement quality
• design-and-build - Metrics for development process
• governance-and-risk - Risk and compliance metrics
• platform-integration - Metric collection automation

---

Getting Started (Quick Start Scenarios)

Scenario 1: First-Time Measurement Program (Level 2)

Goal: Start tracking basic metrics for visibility

1. Read ./measurement-planning.md - Learn GQM methodology
2. Pick 3-5 metrics using GQM (don't start with 20+)
3. Set up manual or automated collection
4. Track for 4-8 weeks (establish baseline data)
5. Review monthly, adjust what you track

Example metrics for first program:

• Deployment frequency (how often we ship)
• Lead time for changes (commit to production)
• Defect escape rate (bugs found in production)

Scenario 2: Implementing DORA Metrics (Level 2→3)

Goal: Industry-standard DevOps metrics

1. Read ./dora-metrics.md - Understand the 4 metrics
2. Read ./measurement-planning.md - Ensure DORA aligns with goals
3. Automate collection (see platform-integration skill for GitHub/Azure DevOps)
4. Establish baselines (4 weeks minimum)
5. Set quarterly improvement goals
6. Review weekly/monthly

Scenario 3: Statistical Process Control (Level 3→4)

Goal: Detect process instability automatically

1. Read ./statistical-analysis.md - Learn control charts
2. Establish process baselines (Level 3 requirement)
3. Calculate control limits (mean ± 2-3 standard deviations)
4. Plot data on control charts
5. Investigate points outside control limits (special cause)
6. Update baselines quarterly

Example: Defect escape rate control chart with UCL=15%, LCL=2%, current value 18% → investigate

Scenario 4: Data-Driven Project Management (Level 4)

Goal: Use quantitative data for project decisions

1. Read ./quantitative-management.md - Learn QPM practices
2. Read ./process-baselines.md - Understand baseline usage
3. Set process performance objectives (e.g., defect density < X)
4. Use prediction models for estimates (see statistical-analysis.md)
5. Monitor against objectives using SPC
6. Adjust process when out of control

---

Common Questions

Q: How many metrics should I track? A: Level 2: 3-5 basic metrics. Level 3: 5-10 organizational baselines. Level 4: 3-5 with statistical control. Focus beats breadth.

Q: How long to establish a baseline? A: Minimum 4 weeks for initial baseline. Prefer 12 weeks (1 quarter) for statistical validity. Update quarterly or when process changes.

Q: What's the difference between MA and QPM? A: MA (Level 2+): Measurement & Analysis - defining, collecting, analyzing metrics. QPM (Level 4): Using statistical process control and prediction models to manage projects quantitatively.

Q: Do I need Level 3 before Level 4? A: Yes. Level 4 requires organizational baselines (Level 3). Can't do statistical process control without knowing what "normal" looks like.

Q: Leading vs lagging indicators - what's the difference? A: Lagging: Measure results after the fact (defects found, deployment time). Leading: Predict future outcomes (code review coverage, test coverage). You need both. See ./key-metrics-by-domain.md for examples.

Q: How do I avoid measurement theater? A: Use GQM methodology (see ./measurement-planning.md). Every metric must answer a question that drives a decision. If you can't explain the decision, don't track the metric.

Q: When should I use control charts? A: When you have 20+ data points and want to distinguish normal variation from abnormal (special cause). See ./statistical-analysis.md for guidance.

---

Reference Sheets

1. measurement-planning.md - GQM methodology, cost/value analysis, measurement repository design
2. key-metrics-by-domain.md - Quality, velocity, stability, deployment metrics with selection guidance
3. dora-metrics.md - Deployment Frequency, Lead Time, CFR, MTTR implementation
4. statistical-analysis.md - Control charts, trend detection, confidence intervals, SPC
5. process-baselines.md - Baseline establishment, maintenance, usage for estimation
6. quantitative-management.md - QPM practices, prediction models, process performance objectives
7. level-scaling.md - Level 2→3→4 progression with concrete requirements

---

Related Practices

• ../requirements-lifecycle/SKILL.md - Requirements management metrics
• ../design-and-build/SKILL.md - Development process metrics
• ../governance-and-risk/SKILL.md - Risk and compliance metrics
• ../platform-integration/SKILL.md - Metric collection automation in GitHub/Azure DevOps

---

Last Updated: 2026-01-25