You are here

Identifying indicators for monitoring and evaluation

Monitoring and evaluation require two types of indicators to be defined at the planning stage; trigger indicators (which will be used to trigger new action or a change of action) and performance indicators. 

Save 

At a glance

Monitoring and evaluation are critical aspects of adaptation planning and implementation.  There are three aspects of adaptation planning and implementation that need to be monitored. 

  1. Tracking the performance of works that are undertaken during the development of an adaptation plan (e.g. performance of engagement activities at all steps in the planning cycle). 
  2. Tracking pre-identified risk thresholds/trigger levels which identify when adaptation actions should be undertaken
  3. Determining the effectiveness of planned output and outcomes from adaptation actions.

Key to monitoring is to identify the indicators that need to be monitored. This should be done early, during the formation of a detailed adaptation plan, in order to ensure that:

  • the selected indicators are appropriate for the actions being implemented; and
  • triggers to stimulate actions are appropriate and linked to the next possible action in the adaptation sequence or pathway.

Indicators should be linked to specific actions and/or objectives.  It is important that they are considered during the preparation of adaptation plans, and included as part of the plan. In this document we provide information about identifying suitable indicators.

Main text

Monitoring and evaluation involve the development of an assessment plan to measure the performance of adaptation efforts in terms of relevance, effectiveness, efficiency, results and sustainability of the results. Indicators are a core part of this. Indicators are things that we can measure. They help to determine whether we have achieved objectives for a specific program or project, or whether we have reached a threshold or trigger for action.

What makes a good trigger indicator?

Trigger indicators are used to determine when a particular threshold is met and there has been irreversible change. Trigger indicators identify when the specific adaptation action is no longer achieving its intended objectives. At this stage, an alternative adaptation action is required. Trigger indicators should be set at levels that allow sufficient time for the appropriate planning, stakeholder engagement and funding to be achieved for the next action in the sequence.  Depending on the nature of the adaptation plan, trigger indicators can be social or biophysical in nature.

The selected trigger levels or thresholds should reflect real changes that have resulted from a changing climate; they should not be set for changes resulting from a non-climate change-related extreme event or from other drivers. For example, a single beach erosion event should not necessarily be a trigger for a completely new action in an adaptation pathway or sequence.  But an erosion event can be a trigger in instances where beach erosion is occurring at unprecedented levels and rates (e.g. upper dunes being eroded four to six times per year, with potential to impact infrastructure and long-term amenity for beach goers).

Trigger indicators should be developed in conjunction with stakeholders, to ensure that they are accepted and understood and reflect local knowledge.  Working with stakeholders can help to ensure that triggers consider social impacts in addition to just  biophysical impacts (Barnett et al. 2014). 

What makes a good performance indicator?

Good indicators need to be easily understood and meaningful to those who seek to use the information they provide. There is little point in collecting complex information if there is no capacity to analyse it or no intent to use it in the review of adaptation plans.  Indicators should be SMART (specific, measurable, achievable and attributable, relevant and realistic, and time-bound), with the following characteristics (Allen et al. 2012, Hockings et al. 2008, Leagnavar et al. 2015):

  • Specific – indicators should capture the essence of the desired result by clearly and directly relating to the achievement of an objective and only that objective.
  • Measurable indicators must be measurable, with consideration given to the repeatability of assessment, the precision required for measurement, and the resources (funding, time) needed for measurement.

Decision makers should consider how measurement information is likely to be used, and adjust the scope of monitoring appropriately. Additional depth and scope of measurement may be useful in situations where management sensitivities exist. For example, when a council is deciding whether to undertake a beach nourishment program they may consider the level of beach usage as well as business activity in beach-side areas. In this situation, monitoring visitor numbers at different times of the year, and related business activity, may be necessary in addition to monitoring changes in the amount of sand dune erosion.

  • Achievable and attributable – the monitoring and evaluation system, and related indicators, identify what changes are anticipated as a result of the intervention and whether the results are realistic. Attribution requires that changes in the targeted developmental issue can be linked to the intervention.

Indicators should be framed in a way that enables comparison over time. Where appropriate, indicators should be similar in similar contexts to enable comparison, e.g. climate negotiations use the indicator of carbon emissions per unit of GDP.

  • Relevant and realistic – the monitoring and evaluation system and related indicators establish levels of performance that are likely to be achieved in a practical manner and that reflect the expectations of stakeholders.
  • Timely – indicators need to be time sensitive to change. For example, some variables are slow changing and highly variable across years (e.g. climate), meaning that progress towards outcomes can be difficult to assess over the short term. In these situations, it is helpful to identify intermediate outcomes that lead towards longer-term outcomes.
  • Time-bound, timely, trackable, and targeted – the monitoring and evaluation system and related indicators allow progress to be tracked in a cost-effective manner at the desired frequency for a set period, with clear identification of the particular stakeholder group(s) to be affected by the project or program.

Indicators should also be sensitive to change to ensure that they provide the necessary information to detect a response to change and help managers assess the effectiveness of their actions.

Where applicable, indicators should also be linked to standards (e.g. Global Reporting Initiative for business sustainability), as these help to ensure people use the same types of indicators and are able to learn from one another. However it is more important that selected indicators fit with the adaptation objectives of the plan rather than that they are selected because they are used by others.  Figure 1 illustrates steps in indicator selection that help to ensure that indicators are appropriate for their intended purpose.

T4M4_2_Figure-1.gif

Steps to follow when selecting indicators that are suited to the needs of a plan or program

Figure 1: Steps to follow when selecting indicators that are suited to the needs of a plan or program. Source: Adapted from Hockings et al. 2008.

Attribution

It is important to attribute indicators to actions. This ensures that when indicators are measured any changes can be related to that action, or in some cases that series of actions. There are many ways in which this can be done. These include development of influence diagrams (Figure 2), causal networks or by constructing conceptual models.  The more complex the system, the more difficult it is to directly attribute the change of an indicator to one or more actions. 

It is possible to have short-term indicators which show that a particular action is completed, and long-term indicators which show that actions have resulted in changed outcomes. Attributing outcomes to a particular action is more difficult due to the many potential drivers. 

T4M4_2_Figure-2.gif

Simple influence diagrams showing cause and effects and indicator selection

Figure 2: Simple influence diagrams showing cause and effects and indicator selection. Source: © NCCARF 2016.

Using indicators to support monitoring and evaluation

Indicators are only one component of monitoring and evaluation.  Care must be taken in designing monitoring programs and in using the results to undertake evaluation.  Further detail on these important aspects is provided in Monitoring and evaluation

Source material

Allen, W., A. Fenemor, and D. Wood, 2012: Effective indicators for freshwater management: attributes and frameworks for development. Landcare Research NZ Ltd, Christchurch, New Zealand. Accessed 31 May 2016. [Available online at http://www.learningforsustainability.net/pubs/developing-effective_indicators.pdf].

Barnett, J., S. Graham, C. Mortreux, R. Fincher, E. Waters, and A. Hurlimann, 2014: A local coastal adaptation pathway. Nature Climate Change, 4(12), 1103 – 1108.

Hockings, M., C. Jacobson, and R.W. Carter, 2008: Process guidelines for indicator selection in protected area management effectiveness evaluation: Building capacity for adaptive management of protected areas. Report to the Natural Heritage Trust. Brisbane, The University of Queensland.

Leagnavar, P., D. Bours, and C. McGinn, 2015: Good Practice Study on Principles for Indicator Development, Selection, and Use in Climate Change Adaptation Monitoring and Evaluation. Climate-Eval Community of Practice, Washington DC. Accessed 31 May 2016. [Available online at https://www.climate-eval.org/study/good-practice-study-principles-indicator-development-selection-and-use-climate-change].   

Save