Jonas Schoenefeld (Tyndall Centre for Climate Change Research, University of East Anglia)
Dr Paula Kivimaa (Science Policy Research Unit, University of Sussex)
With the Paris climate summit around the corner, and substantial research efforts to catalogue and understand an increasing number of climate policy initiatives, there is little doubt that much is already happening to address climate change.[1] But what does all this effort really amount to? And how will we know?
One way to improve our knowledge of climate policy effects is through policy evaluation. This has been defined as the “careful retrospective assessment of the merit, worth, and value of administration, output and outcome of government interventions, which is intended to play a role in future practical action situations”[2]. Despite a relatively long tradition of environmental policy and programme evaluation, there remain substantial questions about how to evaluate climate policy. A key challenge is to assess complex climate policy approaches from a long term perspective in this broad policy area. Therefore, the European Environmental Evaluators Network (EEEN) forum in Florence in September specifically focused on climate policy evaluation and the ‘key findings from retrospective climate and environment policy evaluation to support the road from Paris to 2050’, ‘good examples of attributing policy impact to interventions’ and ways to address gaps in current evaluation approaches.
A clear theme emerging from the conference was the need to better integrate existing evaluation activities. Surprisingly, there is no comprehensive, indexed database of climate policy evaluations anywhere in the world, which makes it nearly impossible to know what is out there and to work cumulatively. While there are some (limited) examples of integrating evaluations in Germany, much needs to be done. The ‘Architectures of Environmental Evaluation’ cooperation between the US Environmental Protection Agency, the European Environment Agency and others will begin to address that issue. Building such databases will enable conducting more syntheses and systematic reviews, which the conference identified as one key area of further development.
What complicates these endeavors is that we need the synthesized information to be timely and produced quickly to address the challenge ahead in a useful form for policy-makers. Meta-evaluations can take time, while the need for information is instantaneous. Moreover, all too often, policy-makers themselves seem keen to report on projections of anticipated policy impact, but there is less interest in looking back to understand what has worked (or not) and why,[3] although learning from past experiences – looking at both successes and failures – is key to future success.
Discussion at the conference also focused on the need to combine different types of evaluation – such as modeling and more policy-science based approaches – in order to generate a more complete picture of the effects of climate policy. This means evaluating – well beyond emissions reductions generated by climate policies (though of course an important component) – a range of intended and unintended side-effects, such as, amongst others, poverty reduction, income distribution and access to resources. Such approaches will likely involve policy-makers and other stakeholder who need to become active participants in evaluation processes, which may be carried out in various compositions from top-down to bottom-up.
Given the importance of context in climate governance, the conference did not generate a great deal of across-the-board recommendations on effective policies. Rather, the discussions focused on how to improve evaluation – and evaluation governance – in such a way as to generate more cumulative insights and build knowledge that is transferable, but yet tied to the experiences from particular contexts. Doing so will be particularly helpful to incorporate the much-needed long-term vision in climate policy evaluation. Rather than conducting ad-hoc evaluation studies, the conference highlighted the need to understand policies over time, and make adjustments as we go along. But how to do so remains an open and crucial question. On the road to 2050, policy evaluation can play a crucial role. But our ‘evaluation systems’ will have to become much more attuned to the current policy needs and questions in order to fulfill that role. In sum, future evaluation-related work should address questions such as: How to produce aggregated analyses of up-to-date evaluations quickly? What does the complexity of climate change and the need for substantial system-level transitions mean in terms of further developing evaluation methodology? How do we design and communicate evaluations to be of interest to policymakers but yet functioning as rigorous analyses of policy?
[1] Jordan, A. J., Huitema, D., Hildén, M., van Asselt, H., Rayner, T. J., Schoenefeld, J., Tosun, J., Forster, J. & Boasson, E. L. (2015). Emergence of polycentric climate governance and its future prospects. Nature Climate Change. Available at: http://www.nature.com/nclimate/journal/vaop/ncurrent/full/nclimate2725.html
[2] Vedung, E. (1997). Public policy and program evaluation. Transaction Publishers.
[3] e.g., Hildén, M., Jordan, A., & Rayner, T. (2014). Climate policy innovation: developing an evaluation perspective. Environmental Politics, 23(5), 884-905.
Photo credit: Windwärts Energie/Flickr
Please note that this publication has been cross-posted on the Environmental Europe Blog.
