Characteristics
In order to understand water endowments in relation to IWRM and to use it as a means towards reaching a wider goal of water security, policy makers need to be able to assess how the policies that are put in place are performing and what outcomes have been achieved. To that end, a coherent system of monitoring and evaluation (M&E) should be developed. Indicators are the basic building blocks of such a system. It helps to answer key questions such as: where are we now, where do we want to go, are we taking the right path to get there, and, finally, are we there yet?
An appropriate and comprehensive M&E system can provide insights into the efficiency of a process and its management and help to reformulate policies and programmes, reallocate resources and guide processes in a more efficient manner. It also supports transparency and gives civil society and governments a way to assess the performance and impact of IWRM processes if information are made openly available. This way, it also serves advocacy and communication purposes. Typically, bodies responsible for assessing water resources and policies are governmental, but this function could also be outsourced or done in partnership with other institutions such as academic institutes and civil society groups (see Tool B1.03), if they have adequate technical capacities.
Monitoring and evaluation presupposes gathering data which has to be reliable and legitimate. In order to measure progress, a baseline has to be established as a reference framework on the basis of which the results can be evaluated. Strategic goals, objectives and targets should also be established as an overarching vision to work towards. In water management, water security would be one such vision.
To evaluate policy performances, indicators need to be developed. These indicators are often selected using the SMART criteria: Specific, Measurable, Attainable, Relevant and Timely. It is important that the indicators are part of a larger evaluation system, in which they relate to each other and to established strategic goals.
The progress towards IWRM can be monitored using indicators of four distinct orders (not necessarily expressing a linear succession):
- 1st Order: process indicators – IWRM Policies and legislation are in place. “The paperwork has been done.”
- 2nd Order: performance indicators – efficiency of process – institutional reforms take place. “IWRM capacity building is taking place and staff in key institutions is acting accordingly.”
- 3rd Order: impact indicators – impacts are felt and issues addressed: “IWRM is influencing conditions on the ground and objectives for water resources are achieved fully or in part.”
- 4th Order: sustainability indicators – sustainability achieved – economic, environmental and equity concerns are balanced/managed. “Societal benefits achieved.”
For monitoring and evaluation to be effective, it should be clear in advance how the results are to be used, reacted to, and by whom. Institutional anchoring of monitoring and evaluation is important and cooperation across institutions is often crucial for the success of M&E systems. Another issue to address is how the results are to be collected and disseminated, and who pays for the monitoring and evaluation. The cost of sustained monitoring must be compared to its expected benefit.
Lessons learned
- Indicators need to be defined and analysed as part of a logical framework of relationships between the goals, objectives, actions, and the intended outcomes and impacts.
- Avoid single parameter indicators. Indicators are best used in clusters, as a combination of indicators will better present the whole picture. The appropriate combination will depend on local circumstances.
- Whilst devising representative indicators is relatively easy, it is often difficult in practice to collect consistent, reliable and meaningful data.
- Stakeholders should be involved in defining indicators and setting up M&E systems. The information provided by the system needs to relate to their concerns and activities.
- Stakeholders can also be involved through participatory monitoring.
- Although simple indicators and evaluation systems may fail to reflect important variations, they are powerful tools for creating awareness and political will.
- Where indicators are used to compare different regions, countries, or water utilities, it is essential that the data elements of the indicator are precisely defined.
- If questionnaires are used for collection of data, objectivity has to dominate subjective valuations.
- Values of indicators or indices should be reviewed critically, i.e. observations that lie an abnormal distance from other values (outlier values) of an index need to be investigated and explained.