Tag: governance

Innovations in Monitoring and Evaluation ‘as if Politics Mattered’,

Date: 17-18 October 2011
Venue: ANU, Canberra, Australia

Concept Note, Chris Roche & Linda Kelly, 4 August 2011

The Developmental Leadership Program (DLP)[1] addresses an important gap in international thinking and policy about the critical role played by leaders, elites and coalitions in the politics of development. At the core of DLP thinking is the proposition that political processes shape developmental outcomes at all levels and in all aspects of society: at national and sub-national levels and in all sectors and issue areas.

Initial findings of the DLP research program confirm that development is a political process and that leadership and agency matter. This is of course not new, but the DLP research provides important insights into how, in particular, leadership, elites and formal and informal coalitions can play a particularly important and under-recognized role in institutional formation (or establishing the ‘rules of the game’), policy reform and development processes[2].

International aid therefore needs to engage effectively with political processes. It needs to be flexible and be able to respond when opportunities open up. It needs to avoid the danger of bolstering harmful political settlements.

Furthermore Monitoring & Evaluation (M&E) mechanisms need to be improved and made compatible with flexible programming and recognize the importance of ‘process’ as well as outcomes. Donors should invest in a range of monitoring and evaluation techniques and approaches which are more aligned with the kinds of non-linear and unpredictable processes which characterise the kinds of political processes which drive positive developmental outcomes. This is important because it can be argued that, at best, current approaches are simply not appropriate to monitor the kinds of processes DLP research indicates are important; or, at worst, they offer few incentives to international assistance agencies to support the processes that actually lead to developmental outcomes Continue reading “Innovations in Monitoring and Evaluation ‘as if Politics Mattered’,”

Measuring Impact on the Immeasurable? Methodological Challenges in Evaluating Democracy and Governance Aid

by Jennifer Gauck, University of Kent, Canterbury – Department of Politics, 2011. APSA 2011 Annual Meeting Paper. Available as pdf

Abstract:

“Recent debates over the quality, quantity and purpose of development aid has led to a renewed emphasis on whether, and in what circumstances, aid is effective in achieving development outcomes. A central component of determining aid effectiveness is the conduct of impact evaluations, which assess the changes that can be attributed to a particular project or program. While many impact evaluations use a mixed-methods design, there is a perception that randomized control trials (RCTs) are promoted as the “gold standard” in impact evaluation. This is because the randomization process minimizes selection bias, allowing for the key causal variables leading to the outcome to be more clearly identified. However, many development interventions cannot be evaluated via RCTs because the nature of the intervention does not allow for randomization with a control group or groups.”

“This paper will analyze the methodological challenges posed by aid projects whose impacts cannot be evaluated using randomized control trials, such as certain democracy and governance (D&G) interventions. It will begin with a discussion of the merits and drawbacks of cross-sectoral methods and techniques commonly used to assess impact across a variety of aid interventions, including RCTs, and how these methods typically combine in an evaluation to tell a persuasive causal story. This paper will then survey the methods different aid donors are using to evaluate the impact of projects that cannot be randomized, such as governance-strengthening programs aimed at a centralized public-sector institution. Case studies will be drawn from examples in Peru and Indonesia, among others. This paper will conclude by analyzing how current methodological emphases in political science can be applied to impact evaluation processes generally, and to D&G evaluations specifically.”

RD Comment: See also the 3ie webpage on Useful resources for impact evaluations in governance which includes a list of relevant books, reports, papers, impact evaluations, systematic reviews, survey modules/tools and website

Measuring Results: A GSDRC Topic Guide

Available as linked pages on the Governance and Social Development Resource Centre (GSDRC), website as of August 2011

The guide is designed to provide a quick and easy way for development professionals to keep in touch with key debates and critical issues in the field of monitoring and evaluation. It will be updated on a quarterly basis.

About this guide
“How can the impact of governance and social development programmes be assessed with a view to improving their efficiency and effectiveness? What particular challenges are involved in monitoring and evaluating development interventions, and how can these be addressed? How can the ‘value for money’ of a particular intervention be determined?

Monitoring and evaluation (M&E) is vital to ensuring that lessons are learned in terms of what works, what does not, and why. M&E serves two main functions: 1) it builds accountability by demonstrating good use of public funds; and 2) it supports learning by contributing to knowledge about how and why programmes lead to intended (or unintended) outcomes. There can sometimes be a tension between these functions.

This guide introduces some of the core debates and considerations for development practitioners involved in designing and managing M&E activities. It introduces key tools and approaches, provides case studies of applying different methodological approaches, and presents lessons learned from international experience of M&E in a range of developing country contexts. While the guide focuses on M&E for governance and social development programmes, it has relevance for all programmes.

The guide was originally prepared by Claire Mcloughlin, and a comprehensive update was undertaken by Oliver Walton in July 2011. The GSDRC appreciates the contributions of Claire Vallings and Lina Payne (DFID) and Hugh Waddington and colleagues at 3ie. Comments, questions or documents for consideration should be sent to enquiries@gsdrc.org.”

Micro-Methods in Evaluating Governance Interventions

This paper is available as a pdf.  It should be cited as follows: Garcia, M. (2011): Micro-Methods in Evaluating Governance Interventions. Evaluation Working Papers. Bonn: Bundesministerium für wirtschaftliche Zusammenarbeit und Entwicklung.

The aim of this paper is to present a guide to impact evaluation methodologies currently used in the field of governance. It provides an overview of a range of evaluation techniques – focusing specifically on experimental and quasi-experimental designs. It also discusses some of the difficulties associated with the evaluation of governance programmes and makes suggestions with the aid of examples from other sectors. Although it is far from being a review of the literature on all governance interventions where rigorous impact evaluation has been applied, it nevertheless seeks to illustrate the potential for conducting such analyses.

This paper has been produced by Melody Garcia, economist at the German Development Institute (Deutsches Institut für Entwicklungspolitik, DIE). It is a part of a two-year research project on methodological issues related to evaluating budget support funded by the BMZ’s evaluation division. The larger aim of the project is to contribute to the academic debate on methods of policy evaluation and to the development of a sound and theoretically grounded approach to evaluation. Further studies are envisaged.

Policy Practice Brief 6 – What Makes A Good Governance Indicator?

January 2011, Gareth Williams

http://www.thepolicypractice.com/papersdetails.asp?code=17

The rise to prominence of good governance as a key development concern has been marked by an increasing interest in measurement and the production of a huge range of governance indicators. When used carefully such indicators provide a valuable source of information on governance conditions and trends. However, when used carelessly they can misinform and mislead. The purpose of this brief is to make sense of the different types of governance indicator and how they are used and misused. It warns against the commission of ‘seven deadly sins’ representing the most common pitfalls. The paper puts forward guidelines to ensure a more careful use and interpretation of governance indicators, and highlights the need for providers of indicators to be subject to greater transparency, scrutiny, evaluation and peer review. From the perspective of political economy analysis the challenge is to make the indicators more relevant to understanding the underlying political processes that are the key drivers of better governance.

Critique of Governance Assessment Applications

GRDC Helpdesk Research Report by Sumedh Rao, Governance and Social Development Resource Centre, July 2010. 16 pages. Available as pdf

Query:  Identify the key literature that critiques the use and application of governance assessments.  Enquirer: DFID

Contents
1. Overview
2. General critiques
3. Critiques of measurement
4. Worldwide Governance Indicators (WGI)
5. African Peer Review Mechanism (APRM)
6. Other assessments
7. Donor Guidance
8. Initiatives for improving assessments

Including a bibliography of 39 annotated references Continue reading “Critique of Governance Assessment Applications”

Survey of Donor Approaches to Governance Assessment

Published by the  Organisation for Economic Cooperation and Development Development Assistance Committee (OECD-DAC),
Size: 34 pages (358 KB)
Full Text here

Executive Summary

Bilateral and multilateral development agencies have engaged intensively in assessing governance over the last decade. To explore opportunities for increased harmonization and alignment in this area, members of the OECD DAC’s GOVNET have commissioned a survey of donor approaches to governance assessments. The survey reported here focuses on general and thematic governance assessment approaches actually used by agencies.

The survey identified 11 agencies having 17 general assessment tools in use and 3 under development, while 6 agencies which presently have no own tools are developing these. 9 agencies reported having 13 thematic tools in use, 4 of these and 3 other agencies are developing new tools. The thematic tools category includes assessment tools related to conflict, human rights, corruption, and sector assessments, as well as tools which focus on particular themes (e.g. financial governance aspects).

Evaluation of the Implementation of the Paris Declaration

Thematic study: The applicability of the Paris Declaration in fragile and conflict-affected situations

Executive Summary

The September 2008 DAC HLF in Accra provides an opportunity to discuss the challenges of applying the Paris Declaration in fragile and conflict-affected situations. This report aims to provide evidence to inform these discussions by:
• Synthesising existing evidence on the aid effectiveness and state-building challenges faced in fragile and conflict-affected situations;
• Exploring the relevance and application of the Paris Declaration and the Fragile States principles in different contexts of fragility and conflict; and
• Setting out the key challenges to improving effective engagement by development partners in fragile situations.

This paper is based on a review of the primary and secondary literature. As part of the review,
four country case studies (Afghanistan, Burundi, the DRC and Nepal) were carried out. These are
included as annexes to the report.
Continue reading “Evaluation of the Implementation of the Paris Declaration”

GRDC Helpdesk Research Report: Monitoring and Evaluation of Participation in Governance

M&E of Participation in Governance: Please identify toolkits, methodologies and indicators for monitoring and evaluating the effectiveness of programmes aimed at improving governance (particularly of urban infrastructure/services). Please highlight methods of relevance to NGOs for monitoring and evaluating poor people’s participation in decision-making processes.

Helpdesk response
Key findings: There is generally very little information available on evaluating the effectiveness of the inclusive/participatory aspects of governance programmes. A particular difficulty is that there is a limited understanding of what improvements in governance actually look like. Nevertheless, some common principles identified in the literature include the need for both quantitative and qualitative indicators and the importance of focusing on purpose, processes, context and perception as well as outputs and outcomes.

Some common indicators for assessing the effectiveness of participatory programmes include:

  • the level of participation of different types of stakeholders
  • institutional arrangements to facilitate engagement
  • active engagement of stakeholders in the programme, and confidence and willingness to get involved in future
  • the extent to which participants are mobilising their own resources
  • transparent access to and use of resources
  • equality of access to decision-making
  • transformation of power through e.g. new relationships and access to new networks
  • level of trust and ownership of the process behavioural changes of stakeholders (values, priorities, aims)
  • level of self-reliance, self-management, capacity and understanding of the issues sustainability and ability to resolve conflict.

Full response: http://www.gsdrc.org/docs/open/HD549.pdf

Produced by the Governance and Social Development Resource Centre

International Aid Transparency Initiative

[From the Development Gateway Foundation]  In support of the Accra Agenda for Action, 14 donors committed to increasing transparency. Participants from developed countries were joined by heads of multilateral and bilateral funding institutions and representatives of foundations in agreement to make information on development more accessible. They promised to establish a common format for the publication of information on aid by 2010. The signatories to the International Aid Transparency Initiative are Australia, Denmark, Finland, Germany, Ireland, Netherlands, Norway, Sweden, United Kingdom, the European Commission, the World Bank, the United Nations Development Program, the GAVI Alliance, and Hewlett Foundation.

See the International Aid Transparency Initiative Accra Statement

Continue reading “International Aid Transparency Initiative”

%d bloggers like this: