It has been said that what gets measured gets done. But this is not strictly true – managers today are deluged by a sea of metrics that are completely irrelevant to the task at hand. In fact, as early as the 1960’s, academic researchers were discovering that top executives rarely if ever used or relied upon the stacks of reports that were painstakingly produced and dutifully delivered to their desks each month. Instead they were much more likely to rely upon the news, hallway conversations, rumours and gossip in making business decisions.
The truth is that what gets managed gets done, and choosing which metrics to manage is the most highly levered strategic decision that an organization of any size can make. This indeed is “the hard part”. Choosing what is important is both science and art and requires attention to an uncommon mix of disciplines – psychology, business strategy, engineering, systems science, finance etc. And you must choose. As a rule of thumb, the human short-term memory limitation of 7 items plus or minus 2 is an excellent guideline. You should choose a total of somewhere between 5 and 9.
Too many measures
Focusing on between 5 and 9 measures does not mean that a management team should only look at or analyze fewer than 10 numbers. Analysis and learning should be encouraged on every bit of available data. However, the key focus – the measures that raise the questions to be answered by analysis must be limited to a set that the human mind can easily grapple with, and where the linkage to the overall vision and purpose of the organization is direct and clear. We’ve seen organizations who cannot clearly state their purpose generate scorecards with between 50 and 75 measures. It’s clearly impossible to keep all of these top of mind, or even to discuss them all in a collaborative management setting.
Too few measures
Selecting less than 5 measures indicates a lack of engagement with the process of improving performance. For example, focusing on profitability alone provides no guidance concerning the unique competitive advantage of the organization, and does not balance short and long-term considerations at all. Furthermore, profitability is a lagging indicator – providing information only about the performance that has been achieved to date, not what the organization is doing to improve performance in the future.
Side-effects
Like pharmaceutical drugs, measures have side-effects. For a worst-case example, consider a manufacturing process where quality control inspectors are paid for finding problems at the end of the line. Quickly, an internal economy appears where quality control inspectors share their bonuses with the people who can introduce defects. Almost all measures have side-effects. Selecting measures that avoid the worst of these is of critical importance.
Negative side-effects can also be exaggerated or mitigated by organizational culture and behaviour norms. A workforce aligned on delivering a quality product in an open and trusting culture would at once reject the compensation scheme described above.
Selecting Great Measures from the top down
Selecting great measures involves grappling with challenging questions. Those who have the luxury and burden of setting measures from the top of an organization have a particular responsibility to “go deep” when selecting measures because their impact will be wide and immediate, and will communicate much about the management culture, style and agenda in the organization. Some questions executives can begin with include:
- What business are we in?
- What is our strategy for growth?
- If we could choose only one aspect of our business to optimize, which one would be most consistent with our vision and drive the greatest results
- What single metric could we use as a proxy to determine whether this aspect of our business is doing well
- What single metric would predict whether we will do well in the next quarter or year?
- What important result does our team too often forget in the flurry of day-to-day management?
- Can we set a target for this metric that will energize and motivate our people
- What will we loose sight of by focusing on this metric? What aspects of our business will falter? What bad tradeoffs and negative behaviours might people engage in if we focus on this alone?
Selecting Great Metrics from the Bottom Up
Managers and analysts who are initiating a performance management program lower in the organization have the advantage of detailed knowledge of what issues are important “on the ground”. Frequently this perspective is useful in predicting the strategy the organization will later adopt, and thus the metrics selected at the bottom pre-empt large-scale top down programs and show results sooner. Questions to consider at this level include:
- Of all the data I receive or have access to, which metrics would be most interesting to analyze and learn from?
- Which of the metrics I have access to would be most useful in causing people to ask interesting questions and learn about what is going on in our operation?
- Are there any surprising metrics in my daily reports that merit further investigation?
- How would posting a graph of this metric on the wall affect my team’s morale and focus?
The bad and the good
The bad news is that even by answering these questions, you will still be likely to get it wrong – wildly wrong – like the call center in which support representatives are rewarded for increasing the number of calls handled, and thus hang up on their customers as quickly as possible.
The good news is that selecting and using metrics, whether right or wrong is a deeply instructive process over time. Each iteration of metric selection will uncover new fallacies in your thinking about and understanding of the business you are in, and create new opportunities to improve the performance of your organization. Set the example by holding the performance management process lightly yet persistently in your search for excellence. Encourage a sense of humour about what works and what doesn’t in the process. Remember that all metrics are just proxies for the truth – they are there to guide your thinking and learning – not to end it.
Disciplines to help you select great metrics
Truth telling: There are lies, damn lies and then there are statistics. Do not fall into the trap of choosing numbers that make the situation look better than it is. The more transparent a metric makes your organization, the better it is. Follow Jack Welch’ s lead and see things for what they are.
Systems thinking: Follow Peter Senge’s advice and look for the first, second and third order effects of optimizing your business around a particular metric. In the example above, if call volume is increased at your call center, what will the impact on customer satisfaction be? What will the effect of that change on customer loyalty? What will the impact of customer loyalty be on transaction costs? Etc.
Undying curiosity: Understanding why things “are the way they are” is the best way to figure out how to make them better. Sometimes you may want to choose a metric just so that people in the organization will become curious about it. Perhaps the best thing about choosing a particular metric to focus on is that it will inspire passionate debate amongst those who care about making things better.
Story telling: Selecting metrics is part of the process of creating a story that explains why the people in your organization should get up and come to work in the morning. It explains what they should do and why they should do it. The metrics you select become the most tangible, and repeated element of the story as you watch them and talk about them from day to day and quarter to quarter.
Strategic intuition: Often the route to success is not around optimizing an internal operation, but instead requires a complete change in the direction and focus of the organization to access some new dynamic in the external environment. Unfortunately, few people are good at this, a lot are bad, and most winners are simply lucky. Still, it’s useful to have this perspective as you select metrics. What is important is as much about what is outside your organization as it is about what is inside.
Tolerance for imperfection: Numbers will never tell the whole story; they are merely an approximation of reality. Still, these approximations are incredibly useful in focusing attention, and raising questions that drive organizational learning. As you choose your metrics, just be clear that you are looking for the best and most accessible approximation of reality. No set of five numbers will ever completely describe your business. Use them for what they are and move on.
