CBR Thought Leaders Series: Beyond IT performance -- measuring true IT effectiveness

Business Intelligence

by CBR Staff Writer| 15 December 2008

Why do traditional IT performance measures fail to tell managers what their real IT capability is – or how IT is serving business goals? Richard Williams and Gordon Miller of Procertis, an independent consultancy and Intellectual Property development company focussed on IT and business inter-relationships, investigate.

Everyone agrees – or pays lip service to the idea – that effective IT is vital to the health of a modern business. True business excellence needs excellent IT to support it, but for most enterprises that's a future goal rather than a current reality.

IT's match to business needs remains stubbornly poor, prompting concerned leaders to explore the gap between goal and reality. But to understand – and begin to close – the gap, decision makers need tools that can help them explore and articulate.

We believe that a rigorous, metric model of business and IT capability is the key tool. This is a model that explicitly states, in hard, absolute terms, what the correct match between business and IT should be. Such a model can also be used to measure the current state of IT provision.

By using a common model to define both the desired and the actual states, managers can produce a meaningful description of the gap they are looking to close. And that means they'll have a concrete basis for a road map that the whole team can share – and that the enterprise as a whole can sign up to.

Counting what counts

Effective management relies on measurement. Today's IT leaders have no shortage of items that they measure, and target, and report on. However, these measures rarely add up to a coherent model of IT's contribution to the business. From the point of view of business management, the effort spent to gather the data might well be wasted. The uses to which the data are put are often suspect as well: meetings “review” the measurements, and pass on.

What are these make-work IT metrics that so many organisations track in the hope that they will somehow help them to manage the business? They are attributes like availability, mean time between failure, service desk ticket volumes, and licence usage. In outsourced environments, a range of metrics like these will be enshrined in contracts and used to control the relationship between IT provider and user.

The common characteristic of these metrics is their focus on performance. They comment on the efficiency of applications, systems and networks – but they don't inform on effectiveness.

This is because effectiveness can only be measured by taking into account the business goals served by the IT. For example, uptime on its own means nothing. It's equivalent to noting when your retail store's doors are unlocked – an interesting, but hardly conclusive, fact.

When you examine the IT metric suites used in most businesses you'll find that at best they describe the basic performance characteristics of the technology base. They tell you if the engine is running properly. They don't tell you if the engine is working well with the vehicle it's meant to drive. Nor do they tell you what the potential capability of the engine is.

These are narrow and historic metrics, designed to ensure localised engineering excellence.

What gets measured, gets reported on. And what gets reported on, gets addressed. The dominance of performance metrics in IT management is a major reason why – despite their protestations – IT leaders struggle to take equal partnership with their non-IT colleagues in the leadership of the business. You can't convincingly state that you're interested in where the ship's going if you spend all your time staring at, and fiddling with, the engine.

The business is interested in how the operation of its assets are adding value for stakeholders. As long as IT fails to demonstrate how the assets it controls and the resources it consumes deliver tangible benefit into the business, it will be seen as a simple cost drain.

IT is widely regarded not as the great enabler of modern business, but an inconvenient cost of doing business – a cost that must be driven down. Until the linkage between IT investment and business enhancement is convincingly and continuously demonstrated, IT leaders will find themselves defending their territory from sustained and indiscriminate attack.

Decision makers must commit to a new mission to explain IT's role, using models that prioritise business outcomes rather than abstract technical excellence.

Made to measure

In order to manage the value of IT correctly, leaders need to turn their thinking on metrics inside out. They need to de-emphasise raw performance metrics, and develop a measurement model that tracks IT's contribution to the business. And that means starting from the business, not the IT.

So, what does the business want of IT? Enterprises use IT to capture business information that supports their processes. Organisations also invest in IT in order to enable business change. Taken together, these two statements dictate that IT's contribution must be at once specific – to support defined business processes – and generic – to enable undefined business change.

Our required measurement model therefore already has a degree of complexity not present in the traditional, one-dimensional, performance-oriented suite. It needs to be a capability model rather than a performance model.

IT leaders often have colleagues who find it hard to take a business-first perspective. For them, a little abstract reverse engineering may be in order. By taking traditional IT metrics and asking what they might mean for the business, managers can often begin to see the limited nature of the time-honoured metrics. Let's return, for example, to uptime. What use is “availability” in the business?

In business terms, availability means that a predefined set of services is usable at the time they are needed. IT teams can ensure availability through a number of strategies.

One solution is to agree on the parameters of “the working day”, and then allocate resources to the technical coverage of this period, and build in redundancy to the IT platform. This solution meets the specific requirement of the business. But it does not address the generic business requirement of IT: that, essentially, every aspect of the specific requirement may become, without notice, invalid. For example, round-the-clock trading and online channels may make “the working day” meaningless.

There's clearly a business meaning of “availability” that is richer, subtler and harder to service than the performance meaning.

The types of performance metrics typically enshrined in Service Level Agreements (SLAs) are fine as far as they go – but that's not far enough. They allow the organisation to understand its performance at a particular point in time, but by their very nature tend to ossify the business model and act as a contractual barrier to change in the business. The performance characteristics measured in traditional SLAs provide no information on IT's capability to support business change. They lack true business context.

What metric or set of metrics could we use to replace the old performance-related “availability” measure with a more meaningful understanding of the business's “on-demandness”? We would need to measure the ability of the infrastructure to change – a feature which might decompose into a number of measures around cost, lead-time, contractual constraints, geographic exceptions and application dependencies. This metric set could also include assessments of the IT team's understanding of the current business dynamics, and the quality of the relationships between IT and business people.

Collecting and presenting these metrics is clearly a much more complex activity than the simple collection of old-style performance statistics. But business is complex: ignore its inherent complexities, and you abrogate your management responsibilities.

The practice of measurement

The vice of performance cultures is that, typically, the wrong things are measured. But the virtue of performance cultures is that measurement is baked into the daily processes of the organisation. If we can make sure that the right attributes are being monitored, then we can exploit the efficiency of existing collection and reporting mechanisms.

One of the risks in transitioning to a capability model-driven approach is that measurement practices get jettisoned alongside the rejected measures. If this happens, the organisation will gain a more useful model but lose the ability to execute it.

We therefore recommend that, as organisations reorientate towards IT capability metrics, they seek new ways of embedding measurement practices in their processes. If this doesn't happen, then the capability model will end up as expensive shelfware.

In terms of driving the business forward, constant and consistent measurement of capability metrics is the key to keeping the business and IT aligned, and to melting the age-old distinction between the business (as an entity) and IT (as a service). It will take time to build a new culture where managing with a rich capability model is the norm.

Communication styles must change, as IT and business colleagues move towards collaborative investigation of capability metrics and leave behind the formalised conflict that usually accompanies performance-centric models.

Credible answers to critical questions

The subtext to any conversation between business and IT is the business's goals, and how IT is – or isn't – helping to further those goals. We need a common language in which such dialogues can unfold and develop, explicitly and objectively.

Change must be driven by business vision: an IT capability model engages vision and attaches it to the motive power of IT. This model must be independent of special interests, and independent of institutional biases towards mechanisms that seem to have “worked” in the past just because they produced outputs.

In one way or another, and with varying levels of confidence, organisations continually ask their IT teams to account for IT capability. The right answer to that question can never be couched simply as an aggregate of performance statistics. Nor is the answer a recitation of the applications portfolio. It's not a clouds-and-spaghetti diagram of the technical landscape. And it's not a smoke-and-mirrors PowerPoint deck that boosts a shopping list of new projects.

The real answer is a set of measurements related to, and projected upon, a framework for operating that encompasses day-to-day delivery, periodic business transformation, and continuous improvement – and which helps different constituencies work together to achieve the business goals they hold in common.

Procertis can be found at www.procertis.com.

If you or your client have an idea for an article for the CBR Thought Leaders Series, email it to jstamper@industryreview.com. Submissions are invited from analysts, independent consultants, standards bodies, user groups, book authors or others who consider themselves thought leaders (but are outside of the IT vendor community).

Post a comment

Comments may be moderated for spam, obscenities or defamation.
Privcy Policy

We have updated our privacy policy. In the latest update it explains what cookies are and how we use them on our site. To learn more about cookies and their benefits, please view our privacy policy. Please be aware that parts of this site will not function correctly if you disable cookies. By continuing to use this site, you consent to our use of cookies in accordance with our privacy policy unless you have disabled them.