Forrester View

Where do metrics come from?

Craig Symons and Adam Brown

Selecting metrics to measure is a major challenge in implementing a performance management programme

One of the popular topics in IT management is metrics and measurement. Forrester published a series of reports on the IT Balanced Scorecard and articulated key metrics for each of the four perspectives. However, these only represent a starting point; each organisation will develop metrics that are more specific to their strategic objectives, culture, industry and other relevant criteria. But when all is said and done, which metrics should you choose? It is relatively easy to come up with things to measure, but they may have problems including data that is unavailable or difficult to get, or driving behaviour that is inappropriate. There are several factors to look at when selecting metrics, which can ensure that you select the most appropriate metrics for what you are trying to accomplish.

What do you pick to measure?

Selecting the most appropriate metrics to measure is a major challenge in implementing a performance management programme. There should be a balance between operational-level metrics and strategic metrics. More importantly, metrics must be complementary and not work at cross-purposes. For example, a strategic objective of increasing customer satisfaction may not be in sync with another strategic objective of measuring reduction in inventory. Those being measured on reducing inventory may become overzealous, leading to many out-of-stock instances which causes long delivery times and unhappy customers.

The Data Warehousing Institute recently surveyed 360 executives about their data problems (see Table 1). As expected, analysing corporate objectives came out on top. Somewhat surprisingly, using strategy maps—a Balanced Scorecard device—finished near the bottom. We suspect that this is due to the newness of the concept, but we will expect its adoption to accelerate due to its inherent power to describe strategy. Ideally, the more people included in the metrics development process, the more likely you are to end up with comprehensive measures. The challenge then is to reduce the prospective metrics down to a manageable number that enables you to measure and meet your strategic objectives.

Smart metrics

When developing metrics, a sound approach is to use the SMART technique. SMART metrics have the following characteristics:

Specific: The metric is clear in what it is measuring. This also applies to ownership and accountability for the performance of the metric. A specific metric is, for example, help-desk calls by hour by technician, rather than help-desk volume.

Measurable: Some metrics are not easily measurable, which may be due to the lack of accurate or timely information, or can arise from disagreements about how to measure it. For example, customer satisfaction is a metric, but how do you measure it? A customer satisfaction score from a corporate customer satisfaction survey is clearly measurable.

Actionable: For a metric to have value, it must be actionable. There must be some influence that can be exercised to alter the outcome. A metric over which you have no control is not a good metric; it serves no real purpose. For example, measuring an individual (unless s/he is part of the senior management) on corporate profitability accomplishes nothing. However, measuring an individual sales representative on the profitability of his/her business does.

Relevant: When developing metrics, the focus must be on those with strategic significance and leverage associated with them. By meeting or exceeding these metrics, you can move the organisation forward along one of its strategic objectives. If the goal is to improve project performance, then a good metric will be to measure the percentage of projects that are completed on or ahead of schedule. Measuring the number of projects started will be less relevant.

Timely: Information must be available in a time frame that enables it to be acted on; stale information is of no use. Cycle times in business are only getting shorter, with some vendors and consultants articulating the concept of the ‘real-time enterprise.’ This environment requires that data acquisition and reporting be almost instantaneous to support the critical decision-making that must occur. A good metric will measure the current state of affairs versus the older data (see Table 2).

Ensuring success

The development of an important set of metrics is critical to any successful performance management initiative. However, developing and publishing the metrics is only the beginning.

Metrics must be owned. The owner must be held accountable for the performance of each metric. It is best if the owner is an individual, as opposed to a department or team. The more specific the ownership and accountability, the higher the probability of success.

To be effective, each metric must have a target or goal associated with it. Ideally, in a performance management effort, the goal is to move the organisation forward or even change the culture; this can be accomplished by setting appropriate goals.

In addition to having targets, each metric should have one or more initiatives associated with it that significantly increase the probability of meeting or exceeding the target. There must be specific tasks or programmes associated with the metrics.

A greater probability for success will be realised if organisations follow these guidelines in the development of metrics, whether they are for the entire enterprise or for individual performance plans.