Metrics that matter - reverse your approach to create KPIs that provide value
By Rae Ann Bruno, President of Business Solutions Training
If you want to know whether you are measuring the right key performance indicators (KPIs), stop distributing your KPI reports and wait to see if anyone, whether they are business partners, other IT managers, your team, or the CIO, asks for them. Believe it or not, this is a good litmus test to assess the usefulness of the reports that you are distributing, determine whether the metrics you are sharing are truly KPIs, and learn whether there is value in the metrics and reports. If they truly are KPIs, your stakeholders will ask for them!
Key performance indicators or standard metrics?
A key performance indicator, by definition, is 'an important metric used to evaluate the success in meeting an objective'(ITIL® Foundation, ITIL 4 edition, 2019). In other words, KPIs align with stakeholder goals and are not determined by IT alone. Are your KPIs measuring success in meeting objectives? If you cannot answer that question or your answer is no, you may be providing standard metrics and not KPIs. Standard metrics are 'metrics that only store and display one metric value for each date range' (Databox).
Too often, I see cases where KPI reports were inherited from previous leaders, written to satisfy a request from one person (ages ago), or used because they are provided by a tool. I have also seen KPI reports that are built on 'easy-to-collect-metrics' or are showing a single metric instead of telling a story. It is unlikely that these reports provide sufficient information to drive action or show business value. Furthermore, these reports are usually written from an IT perspective and show how efficiently we are operating instead of aligning with goals and measuring progress or showing the delivered value (from a stakeholder perspective).
ITIL 4 focuses on the value system, the value chain, and value co-creation. Not only is the value co-created, but it is also monitored and measured so that it can evolve as needed. Properly aligned KPIs are essential in value realization, innovation, and continued growth and improvement.
Too often, to define KPIs we start with the data and canned reports that our tools provide. This usually results in reports and metrics that have limited value. Through aligning KPIs with business goals, at a minimum, IT should be able to:
Site specific business tasks that IT services have facilitated or improved
Quantify the amount of time that IT has helped the business save
Measure productivity gains facilitated by IT
Show cost savings or an increase in profits or customer retention (at the organizational level)
Demonstrate that employees are working efficiently and cost-effectively.
If your KPIs are not currently providing this type of information, you may need to reverse your approach to defining and aligning your KPIs.
Reverse your approach to create KPIs that provide value
Step 1 - Ask the questions.
Reversing your approach starts with the right questions (before considering the data). For example:
What do your stakeholders want to know?
What are the business goals in various areas of the organization?
What are critical business processes and how do they impact the business?
What IT services align with critical business processes?
What problems is an initiative or process trying to solve?
What goals have been set for the initiative (practice, procedure, project, etc.)?
What results have we seen from recent improvements (and their impact on the business)?
Keep value co-creation in mind. Any questions regarding business goals, needs or questions require answers from all stakeholders. In other words, ask, do not assume. Most likely, your stakeholders already have questions they would like to have answered or goals they are working toward in their respective areas.
Step 2 - Understand the current workflow.
Observe the current workflow and business practices (I suggest you do the Gemba Walk) and learn, from the stakeholders, about what works well and what is needed to do their jobs and meet their goals more consistently and effectively. This will lead to identifying needed KPIs to measure progress and success.
Step 3 - Look at existing metrics and reports and determine how to monitor, measure, and effectively communicate KPIs.
Once workflow, goals, and value are defined and understood and KPIs are identified, it is time to look at existing metrics and reports. Are you measuring progress toward these goals today? Are you currently measuring and promoting value realization? Often, you have the data, but have not combined different metrics to 'tell the story'. Another possibility is that you have trending reports that answer the question at a high level and need to take a 'deeper dive' into metrics to provide more specific answers.
If you do not currently have reports that answer the questions, determine what data would be needed to 'tell the story' or provide the answers. Then, design KPI reports that measure against the goals.
To demonstrate the approach, let's review a scenario for a non-profit organization that provided food to an area devasted by a hurricane. The organization's mission was to feed the families in the area and 'leave no one hungry'.
Watch the video and identify metrics that are used to measure success: (©Supermarché)
Let's apply our reversed approach to this scenario:
Ask the questions.
The consultants interviewed the leaders and workers to understand the goals and current challenges. They visited the place of work (performed a Gemba Walk) and assessed current workflow. They also performed a Kaizen (to eliminate waste) and found ways to overcome challenges.
Understand the current workflow and goals.
The focus areas were:
a. Eliminate wasted box space
b. Simplify the workflow for filling the boxes, thus making it easier on the workers and reducing the time to fill the boxes
c. Have enough boxes to feed all families
d. Reduce the time families had to wait to get their boxes.
Look at existing metrics and reports and determine how to monitor, measure, and effectively communicate KPIs.
In the video, empirical data and metrics were not being used. Therefore, the consultants established a baseline in all areas before making the improvements (food per box, wasted space per box, time and efficiency to load boxes, work/effort for employees, time to load the truck, boxes packed per truckload, time to distribute, number of families fed, number of families who did not receive food).
They identified ways to measure success (KPIs) and communicated these to the workers and leaders.
a. The workflow was highly optimized and simplified and it was much easier for the workers (reduced effort and waste). Packing boxes went from 3 minutes per box to 11 seconds.
b. More boxes fit in the truck and a conveyor was used to move the boxes from the truck to the workers to hand them out to the families.
c. Distribution time went down from 3 hours to 1.2 hours (handing out food to families).
d. No families were left empty-handed and 400 more families were fed in ½ the time.
This scenario shows the connection that is needed between goals, practices, procedures, optimization, and metrics. Once 'metrics that matter' were being used, it became evident that changes to the workflow needed to be made. Adding structure and measuring improvements not only motivated the workers, but it helped the organization to improve work conditions for the workers as well as the experience for the families in need. Although they identified further improvements to make, measuring against the goals showed that the changes were a huge success.
Keys to KPI success
Regardless of the industry or business drivers, it is essential for IT to partner with stakeholders to focus on value and business goals instead of defining 'metrics in a vacuum'. Although operational and efficiency metrics are important, they do not stand alone. KPIs are necessary to show progress, success and value. By starting with the right questions, IT can more accurately and successfully provide 'metrics that matter' and truly measure goal achievement and business success.
About the author
Rae Ann Bruno is the President of Business Solutions Training, Inc (BST), an organization focused on training and consulting in various areas of ITIL, Knowledge-centered Service (KCS), Service Desk Improvement, Internal Marketing, Metrics and ITIL Process Improvement.
She was inducted into the HDI Hall of Fame in 2017 and was named one of Cherwell's IT Legends that same year. She holds several ITIL and HDI certifications and is a Business Associate for HDI ITIL and Support Center certification courses and consulting.
Rae Ann is a member of the HDI International Standards committee and is a frequent speaker at leadership and support conferences. She is the author of the HDI focus books: Translating IT Metrics into Business Benefits and What Have you Done for Me Lately? Creating an Internal Marketing Culture.
ITIL Foundation, ITIL 4 edition
The first step on the ITIL 4 pathway provides IT professionals with an understanding of the ITIL 4 framework and its practical application to the modern digital world while serving as expert reference guidance for solving day-to-day problems.
ITIL 4 Direct, Plan and Improve
Ideal guidance for IT professionals who are responsible for managing the direction and strategy of their IT team or organisation. This book helps you understand the Direct, Plan and Improve module towards Managing Professional, as well as provide daily expert reference guidance for day-to-day problems.