Module 19: Monitoring and Evaluation

19.4. More on performance indicators

As it has been mentioned above, performance indicators can be defined as: “variables whose purpose is to measure change in a process or function”. Indicators used by the promoter (the contracting partner, usually local government) must be clearly defined, accessible and transparent.

Quantitative and qualitative indicators

Indicators may be quantitative or qualitative in nature. The average cost incurred and the time taken to complete a particular item of work are both quantitative indicators. By contrast, a community member’s perception of his or her satisfaction with the results of a particular community procurement process is essentially qualitative in nature. For comparison purposes, it will usually be necessary to find some way of placing some quantitative value on such qualitative indicators.

It is arguable that processes that cannot be measured cannot be managed. This statement might be questioned by a group that combines together to provide some basic item of shared infrastructure, for example. However, it is true that in some way such a group does measure costs against perceived benefits. It is certainly widely accepted by governments that quantitative information is needed if choices are to be made between a range of possible development options. A common response to the need to quantify indicators is the use of ranking scales in which, for instance, a person’s satisfaction with a process, activity or situation might be ranked on a scale ranging from very happy, through happy, indifferent and unhappy to very unhappy. Such ranking scales must be set up at the time that indicators are identified and introduced to the overall monitoring and evaluation system.

Process and output indicators

Indicators may relate to processes and the outcomes from those processes.

Process indicators...

... may relate to both the quality of the process and its progress. Indicators relating to the quality of a process might cover such questions as:

– Who is involved?

– What role are they playing? and

– What are the power relationships within the group?

Those relating to progress...

... will be concerned primarily with the question “Is the process occurring as planned?” Managers can use information on the quality of a process as the basis on which to amend and improve that process. Such information can also be used to assess the outcomes of that process and the possibility that this assessment may lead to changes in the design of future initiatives. Indicators of progress are immediately useful to the managers of a project or process, but they may also help in the assessment of the efficacy of that process.

Indicators relating to outcomes...

... are required by managers primarily to assess the success of projects and to measure the achievements of those projects. They are intended to help planners to determine the extent to which the project or process achieved its objectives. At the project level, the concern will be with whether it has achieved both its specified outputs and its overall purpose. At a wider level, there is a need to consider the overall impact of projects and programmes on fundamental indicators – for instance, those that measure the level of poverty in society.

Check what information the performance indicator provides

When choosing performance indicators for service delivery, local government managers need to take account of the following factors for a performance indicator PI(x), which is intended to measure the performance of service X:

– Is X an area which falls under the control of the user?

– Will the PI(x) measure what is needed?

– Will problems in area X be detected by the use of PI(x)?

– Does PI(x) give an idea of the magnitude of the problem?

– Is data available to compute PI(x)?

– Is PI(x) accepted by the people involved?

– Are there any other indicators that can help identify the cause of the problem?

– Who will use PI(x)?

Performance indicators in monitoring and evaluation

Performance indicators are normally used in one of two ways. First, they may be collected at regular intervals to track the way in which a system is performing or an activity is unfolding. Alternatively, they may be used to assess the change resulting from a particular activity or project. In the first case, they are being used to monitor the progress of a process, while in the latter their purpose is to evaluate the outcome of a project or process. Evaluation requires that the situation be assessed both at the beginning and at the end of the project or process. This suggests that there is a need to collect baseline data relating to the proposed performance indicators before a project or process starts [Tool 4]. Information collected before a project or process begins can and should also be used to inform decisions on what is to be done. This information should, wherever possible, include data taken from previous initiatives so that lessons from the past can inform the new venture.

The above uses of indicators are linked strongly with the project cycle of identification, implementation, operation and assessment. The use of information to inform decisions is an essential aspect of project identification. Monitoring systems are required to ensure that implementation and operation proceed as planned, and to provide the basis for corrective action if there is a problem. As already indicated, process indicators will be important in this respect. Evaluation systems are required in order to assess the achievements and impact of a project or programme. Evaluation requires the assessment of change and hence is dependent on the existence of adequate baseline information; this in turn means that performance indicators should be in place at the beginning of any initiative. Timelines are essential in the development and application of indicators.

Performance indicators and planning

A range of stakeholders can use indicators to assist them in planning projects, programmes and policies. Planning starts from the assumption that the evaluation of completed projects, programmes and policies should provide an input into plans for the future. In effect, evaluation of what has gone before becomes part of the appraisal process for what is to come.

Most of the indicators presented in this study are concerned with performance at the level of the individual contract. However, it is important to recognise that, when brought together and analysed, the information obtained through such indicators can be important in shaping programmes and policies. For instance, indicators of the local employment resulting from specific schemes featuring community-partnered procurement might provide a strong rationale for the development of changes in policy to encourage such partnerships. A related point is that, while individual schemes may result in improvements in the local situation, assessment of the cumulative effect of a number of projects is necessary if overall impacts of different approaches to the provision of services are to be assessed.

Professionals involved in service provision have been traditionally interested in the efficiency with which, for example, infrastructure facilities have been provided. Their concern has been with the cost of works, the time taken to complete them and their quality. As already indicated, research is now taking a wider perspective, exploring indicators that relate to the “softer” impacts of service delivery. Some of these impacts are sensitive to the process of service procurement and delivery; some are process independent.

Key points when using performance indicators

To summarise, the following points need to be kept in mind when using performance indicators.

1. They should be truly representative of the quantities and characteristics that they are intended to represent.

2. They should be verifiable; in other words, it should be possible to check that the values of the data or indicators presented are reported accurately.

3. They should provide information that can be used by decision-makers. As already indicated, this will often mean that they are presented quantitatively.

4. The information must be available in time to influence decisions.

5. They should be linked into systems that allow feedback of information into the decision-making process.

© 2004 UNDP,  Manufactured by Margraf Publishers GmbH, Germany