Question

Have the appropriate indicator types been identified?
You are here: Home / Pathfinder / Monitoring and evaluation / Monitoring and evaluation

Have the appropriate indicator types been identified?

Once the purpose and criteria have been established, the next question to address is whether appropriate indicators have been identified.

M&E for adaptation should combine qualitative, quantitative and binary indicators and that on their own, any category of indicator is insufficient (Lamhauge, et al 2012) ). For instance, to establish the successful development of a policy framework you need also to find indicators to assess implementation and sustainability. Policy development indicators thus need to be augmented by quantitative indicators that for example measure the number of projects that have been developed in response to the policy or the number of households that have benefitted from it.

AP interactive decision tree - click any node to select it

The ability to track progress on adaptation and learn lessons relies on the selection of indicators that are capable of isolating and representing the essential changes sought. Going through the process of defining indicators may help in clarifying different perspectives on the outcomes desired and may help to achieve consensus. Outcome indicators are hard to define if there is a lack of clarity in conceiving the outcome, or if an outcome is too vague or ambitious. Where possible, indicators should be derived through dialogue with stakeholders In defining indicators, UNDP guidance advises 'it is important to remember that they should be used to provide approximate answers to a few important questions rather than seek to provide exact answers to many less important questions. Balance is key, in order to prevent the process of defining and monitoring indicators from becoming a major workload'. In order to be usable as indicators a number of critical qualities have to be present. These were described by UNDP as:

  • Validity: does the indicator capture the essence of the desired result?
  • Practicality: are data actually available at reasonable cost and effort?
  • Clarity: do stakeholders agree on exactly what to measure?
  • Clear direction: are we sure whether an increase is good or bad?
  • Owned: do stakeholders agree that the indicator makes sense?
The criteria for identifying indicators were developed for the Bellagio Framework of National Adaptation Functions and are as follows:
  • Broad applicability
  • Flexibility to accommodate national circumstances
  • Logic and straightforwardness
  • User-friendliness and common sense
  • A top-down approach that empowers bottom-up action
  • Comprehensiveness with regard to key national adaptation functions
  • Compatibility with other tools, frameworks, and decision criteria
The OECD review found a wide variation in the level of detail in the data that were being collected for the evaluations they looked at. Some projects had detailed indicators corresponding to every component of an intervention, while others focus on an aggregate assessment of change in climate vulnerability. The authors suggested that a preferred approach is likely to depend on the type and the scale of the activity. It is clearly important to be rigorous and careful in the identification of indicators if they are to be taken seriously for developing credible and effective evaluations that can be used to capture learning and provide accountability. However, there is a danger that too much focus is placed on results measurement and indicators diverting those managing the process from useful and potentially more effective, but less measurable activities.

  • Pathfinder overview diagram
References AP home Glossary

This section is based on the UNEP PROVIA guidance document

Criteria checklist

1. You want to monitor and evaluate implemented adaptation actions.
2. The purpose of the evaluation is clear.
3. The underlying principles and evaluation criteria have been established.
4. As a next step you are faced with the question whether appropriate indicator types have been identified.