back1ply/kpi-checklists

KPI Checklists Skill

Source: Distilled from KPI Checklists by Bernie Smith, Metric Press, 2013. ISBN: 978-1-910047-00-2

Applicable when designing KPI or measurement systems; aligning measures to strategy; engaging stakeholders around measurement; building KPI Trees; shortlisting and prioritising measures; defining KPIs precisely; prototyping dashboards and reports; rolling out measurement systems; troubleshooting data quality, trust, or delivery issues; sustaining established KPI systems; or improving meeting effectiveness with data.

---

Core Concepts

What Is a KPI?

A KPI (Key Performance Indicator) shows how you are doing at a particular activity to achieve a particular outcome. KPI, metric, and measure are interchangeable — "key" simply implies importance.

• Leading indicators show what is coming (windscreen view)
• Lagging indicators show what happened (rear-view mirror)
• A good dashboard has a mixture of both, financial and non-financial

The ROKS Framework

The Results Orientated KPI System is a 7-step methodology for developing KPIs from strategy through to sustained production:

| Step | Name | Purpose | |------|------|---------| | 1 | Clear Strategy | Confirm strategic objectives are specific and measurable | | 2 | Engage | Identify stakeholders, build communications plan, assess situation | | 3 | Longlist | Use KPI Trees to generate all candidate measures | | 4 | Shortlist | Rate measures on Importance vs Availability, classify Use/Aspire/Discard | | 5 | Define | Create precise KPI definitions and meeting Terms of Reference | | 6 | Prototype | Design, test, and approve dashboards and reports | | 7 | Go Live | Buy-in, data collection, analysis, process mapping, sustain |

---

Step 1 — Clear Strategy

Without clear strategy, you only achieve objectives by accident. Strategic objectives must be specific, not woolly.

Strategy Fitness Checklist:

• Is the strategy written down and accessible?
• Does it make sense to a non-specialist reader?
• Is there broad management consensus?
• Are objectives linked, specific, and non-dated?
• Are there fewer than seven strategic objectives?
• Are key decision-makers working to the strategy?

Red flags: Words like "excellence", "synergy", "outstanding", "empowerment" — they sound great but are unmeasurable.

For the complete strategy and stakeholder checklists, see Strategy & Engagement Checklists.

---

Step 2 — Engage Stakeholders

Identify all who have involvement in production, review, or reward of measures. Use a RACI matrix:

| Role | Description | |------|-------------| | Responsible | People who do the work | | Accountable | Person who signs it off | | Consulted | Subject matter experts (two-way) | | Informed | Kept up to date (one-way) |

Stakeholder types: Data originators, Data aggregators, Data packagers, Internal customers.

For communications plan checklists and stakeholder interview guides, see Strategy & Engagement Checklists.

---

Step 3 — KPI Trees (Longlist)

A KPI Tree is a graphical tool showing linkages between strategic objectives and everyday measures. It decomposes from Strategic → Theme → Tactical → Measure level.

Three link types:

• Cause-effect (grey line) — One activity directly influences another
• Conflict (red line) — Activities work against each other (e.g., minimise AHT vs maximise First Touch Resolution)
• Companion (double line) — Measures overlap or one is a subset

Key benefit: Prevents "going straight for the obvious measure" and exposes unintended consequences.

For detailed KPI Tree building instructions and shortlisting methodology, see KPI Trees & Shortlisting.

---

Step 4 — Shortlist

Rate each candidate measure on two criteria:

| Criterion | Scale | Description | |-----------|-------|-------------| | Importance | 0-10 | How significant is this measure to the business? | | Availability | 0-10 | How easy is it to get the data? |

Plot on a 4-box matrix:

| | Hard to Collect | Easy to Collect | |---|---|---| | Important | Aspire — Develop these | Use — Put on first dashboard | | Not Important | Forget — Park with rationale | Caution — Don't include just because you can |

Always record the Discard list with reasons — it defends choices and prevents report bloat.

For the complete shortlisting process, see KPI Trees & Shortlisting.

---

Step 5 — Define KPIs

Every KPI needs a precise definition answering:

| Field | Purpose | |-------|---------| | KPI name | Clear "what-it-says-is-what-it-is" name | | Measurement intent | Why are you measuring this? | | Formula | Could a numerate stranger calculate it? | | Frequency | How often is it calculated? | | Units | Dimensionless ratio or real measure? | | Assumptions | Document known flaws openly | | Data source | Painful detail: server, directory, spreadsheet cell | | Target | What score do you want? Why that level? | | Owner | Person responsible for tracking and reporting |

For the full KPI Definition Canvas and meeting Terms of Reference checklists, see KPI Definition.

---

Step 6 — Prototype Dashboards

Never go straight to production. Prototype collaboratively. Three approaches:

1. Offline options — Present pre-built mockups (lowest risk, slowest)
2. Live session — Build with decision-maker in real time
3. Hybrid — Present options, then tune together (recommended)

Dashboard design principles: Simplicity, restraint, visual clarity. Avoid 3D charts, double Y axes, RAG without criteria, mixed chart types, red-green colour only.

For the 53-point Brilliant Dashboards Checklist, see Dashboard Design.

---

Step 7 — Go Live

Buy-In Stages

1. Create engagement — Involve stakeholders in KPI Tree sessions
2. Build a case — Show real-world benefit (but rational argument alone won't build engagement)
3. Remove obstructions — User guides, clear processes, easy compliance
4. Public displays of importance — Senior sponsor endorsement
5. Develop good habits — Consistent heartbeat, quick problem fixes, positive reinforcement

Common Data Problems

| Problem | Solution Approach | |---------|-------------------| | Data in islands (spreadsheets, ad hoc DBs) | Map data locations, define ownership | | Contradictory datasets | KPI definition sheets, investigate incidents | | Data in different forms | Standardise, use data marts | | Lack of trust | Survey stakeholders, investigate, remediate | | Collection delays | Publication timetable, SMED cycle-time reduction |

Sustaining KPIs — What Kills Established Systems

• Rapid strategy change without KPI review
• Loss of senior sponsor
• "Flat earthers" who reject data-based management
• Loss of key specialists (single-person dependencies)
• Mistrust of output
• Entropy — gradual slide into disorder

For detailed go-live checklists, SMED methodology, FMEA risk management, and sustainability guidance, see Go-Live & Operations.

---

Risk Management — FMEA

Failure Mode and Effects Analysis scores risks as:

FMEA Score = Probability (1-10) × Severity (1-10)

Optionally add Detectability: Score = Probability × Severity × Detectability

Use calibrated scales with specific examples at 1, 5, and 10. Identify mitigations, then rescore.

---

Improvement Cycle — PDCA

KPIs sit in the Check step of Plan-Do-Check-Act:

| Step | Purpose | |------|---------| | Plan | Clear objectives, ownership, resources, baseline measures | | Do | Execute, gather data along the way | | Check | Analyse actual vs planned, present clearly, share in time | | Act | Put things right, allocate resources, document changes |

The cycle's effectiveness is limited by its weakest element. Perfect KPIs are worthless if you don't act on them.

For PDCA checklists and meeting effectiveness guidance, see Meetings & Improvement Cycle.

---

Anti-Patterns to Avoid

| Anti-Pattern | Why It's Bad | |---|---| | Using "off-the-shelf" KPIs without customisation | KPIs must reflect your specific business and aspirations | | Woolly strategic objectives | "Excellence" and "synergy" lead to endless debate | | Skipping the Aspire list | You miss important but hard-to-collect measures | | No KPI definition sheet | Creates "common sense" problem — everyone assumes differently | | Fox guarding the henhouse | Don't ask people to incriminate themselves with data | | Omission censorship | Charts quietly disappear from reports | | Skipping the "circle of tedium" | Jumping to IT solutions without iterating the manual process | | Ignoring middle managers | Any dissent from line managers will destroy data collection | | Changing process frequently | Destroys habit formation needed for reliable collection |

---

Decision Framework

When building a KPI system:

1. Validate strategy → Use strategy fitness checklist
2. Identify stakeholders → Build RACI matrix, create communications plan
3. Generate candidates → Build KPI Trees in workshops
4. Prioritise → Importance/Availability matrix → Use, Aspire, Discard
5. Define precisely → KPI definition sheet for every measure
6. Prototype → Collaborative dashboard design with stakeholders
7. Launch → Pilot first, preflight checks, user guides
8. Sustain → Regular review, habit formation, feedback loops, PDCA

---

Mindset for KPI Success

• Persistent constructive scepticism
• A bias towards simplicity
• Desire for structure and order
• Constant mindfulness of organisational objectives
• Always remembering that your end customer is a human being

---

Reference Files

For detailed checklists and extended guidance, consult:

• references/strategy-engagement.md — Strategy fitness, strategic objectives, communications plans, RACI, stakeholder interviews and workshops
• references/kpi-trees-shortlisting.md — KPI Tree building workshops, link types, shortlisting matrix, Aspire list development
• references/kpi-definition.md — KPI Definition Canvas, measurement intent, formula precision, target setting, meeting Terms of Reference
• references/dashboard-design.md — 53-point Brilliant Dashboards Checklist, prototyping approaches, design principles, readability standards
• references/go-live-operations.md — Buy-in stages, data problems and solutions, SMED cycle-time reduction, FMEA risk management, data collection methods, sustainability
• references/meetings-improvement.md — PDCA improvement cycle, meeting behaviour, Terms of Reference, lightning rounds, relevancy matrix, meeting rules

---

Quick Reference: When to Use Each Reference

| User Request | Reference File | |---|---| | Strategy alignment, strategic objectives, fitness check | strategy-engagement.md | | Stakeholder identification, RACI matrix | strategy-engagement.md | | Communications plan for KPI rollout | strategy-engagement.md | | Interview questions for stakeholders | strategy-engagement.md | | Building KPI Trees, workshop facilitation | kpi-trees-shortlisting.md | | Conflicting measures, companion measures | kpi-trees-shortlisting.md | | Shortlisting method, Importance/Availability matrix | kpi-trees-shortlisting.md | | KPI definition sheet, measurement precision | kpi-definition.md | | KPI Canvas template, definition database | kpi-definition.md | | Meeting Terms of Reference, meeting inputs | kpi-definition.md | | Target setting, who sets targets | kpi-definition.md | | Dashboard design review, design checklist | dashboard-design.md | | Prototyping approaches, stakeholder sign-off | dashboard-design.md | | Report readability, colour use, chart design | dashboard-design.md | | Getting buy-in, engagement strategies | go-live-operations.md | | Data quality problems, contradictory data | go-live-operations.md | | Speeding up report production, SMED | go-live-operations.md | | Risk management, FMEA scoring | go-live-operations.md | | Data collection methods, Excel vs SharePoint | go-live-operations.md | | Process mapping KPI production | go-live-operations.md | | Sustaining KPIs, what kills KPI systems | go-live-operations.md | | Handover to BAU team | go-live-operations.md | | PDCA improvement cycle, continuous improvement | meetings-improvement.md | | Meeting effectiveness, chairing meetings | meetings-improvement.md | | Lightning rounds, meeting rules | meetings-improvement.md | | Meeting relevancy matrix, reducing meeting length | meetings-improvement.md | | Ad hoc reporting requests | meetings-improvement.md |