Jonas Schoenefeld & Andy Jordan, University of East Anglia
What is needed to make the Paris Agreement a success? This blog post focuses on one of the most central but underappreciated elements – the periodic reviews of progress. States must make ambitious and credible contributions in the first place, but if there is no system to ensure that they are monitored and evaluated the agreement will have very shaky legs.
Article 13 of the Agreement states that the Transparency Mechanism should “provide a clear understanding of climate change action… including clarity and tracking of progress towards achieving Parties’ individual nationally determined contributions… including good practices, priorities, needs and gaps” (p. 27). Almost all countries have already put forward lists of policies to fulfil their commitments. But how will we know whether all of this will add up to limiting global warming to ‘well below’ two degrees Celsius? Doing this will require concerted efforts in monitoring and evaluating climate action – but how may we best organise these activities across the globe?
Our brand-new paper, published in Evaluation, addresses the question on how to organise monitoring and evaluation. This issue matters because the European Union and others are already investing significant amounts of resources in climate policy monitoring and evaluation activities, and many different kinds of actors, such as the European Environment Agency, the European Commission, the Court of Auditors and others have become actively involved. Earlier research has already highlighted important shortcomings in climate policy monitoring and evaluation activities. Our paper argues that the organisation of these activities is another area that has received far too little attention.
In the paper we argue that there are, in principle, two different axes to think about governing or organising monitoring and evaluation activities. The first is a distinction between formal, government-driven or funded evaluation and informal, society-driven evaluation. Earlier evaluation scholars have argued that evaluation driven, funded or even conducted by governments may benefit from more detailed knowledge of the policy process, potentially higher levels of resources to conduct evaluations, and perhaps also greater uptake of the findings. By the same token, governmental actors may be under pressure to ‘look good’ and thus shy away from critically evaluating their own policies, particularly ones that may not be working as envisaged. Thus, more informal societal actors, such as NGOs, foundations, trade unions and others, may be able to step in and provide a more critical eye, provided that they have the interest and resources to do so.
The second way to view the governance of evaluation runs along centralised and de-centralised approaches. On one hand, evaluation may be organised in a centralised way with clear standards that evaluators use everywhere, where essentially one evaluation actor (governmental or non-governmental) organises and potentially funds/conducts evaluation. In other words, resources are pooled in order to conduct evaluation tasks. An advantage of this approach may be higher levels of comparability given more uniform evaluation standards and potentially higher level of resources. On the other hand, scholars from the polycentric governance tradition would argue that such centralised evaluation systems may be prone to failures (what if one picks the wrong or incomplete standards?) and is often insensitive to vital contextual factors from which policies emerge and contribute significantly to success or failure. Thus, another way of organising evaluation may be in a much more decentralised way with many different kinds of actors involved in evaluation.
Taking these two axes, we created a new typology that considers them both at the same time in Figure 1 below. Doing this opens up new combinations and thus ways to think about governing evaluation. We then drew on existing evidence from climate policy monitoring and evaluation in the European Union to assess to what extent we can detect empirical patterns that chime with our typology.
The European Union has been a long-standing leader in efforts to monitor and evaluate climate policies and there are many ongoing efforts, and certainly challenges, in monitoring climate policies. Looking at these activities from the perspective of our typology shows that we can indeed detect patterns that relate to the four quadrants in Figure 1 in the context of the European Union. The new typology is thus a useful way to comprehend evaluation activities, and we hope that it will open up ways of governing them.
For example the Monitoring Mechanism for climate policies and measures, operated by the European Environment Agency, contains negotiated standards and methods, but once these are set operates in a fairly hierarchical way. Both the European Commission (formal) and more informal evaluation organisations have endeavoured to create evaluation standards, which could be extended to the climate domain. A meta-analysis of climate policy evaluation in the EU has revealed a vast number of studies that used a range of different standards and methods (à la carte). As states are currently hammering out the details of transparency in the Paris Agreement, they should keep these different options and trade-offs in mind.
What then is necessary in order to implement the Paris Agreement? We argue that part of the answer to this question lies in considering how to build successful and enduring systems to monitor and evaluate ongoing climate policy efforts, and that it crucially matters how to organise these activities. Our paper reveals that there are range of organisational choices and options in the climate policy monitoring and evaluation domain and that different options have implications for the practice and results of evaluation. It is worth thinking carefully about how and who organises the review processes, in order to generate systems that capture the full extent of knowledge and understanding on our climate policies.
 Ostrom, E. (2010). Polycentric systems for coping with collective action and global environmental change. Global Environmental Change, 20(4), 550-557.
 Huitema, D., Jordan, A., Massey, E., Rayner, T., Van Asselt, H., Haug, C., … & Stripple, J. (2011). The evaluation of climate policy: theory and emerging practice in Europe. Policy Sciences, 44(2), 179-198.
This post has also been published on the EnvironmentalEurope Blog.
Image credit: Eric Huybrechts/Flickr