Tag: peacebuilding

Evaluation in Violently Divided Societies: Politics, Ethics and Methods

Journal of Peacebuilding & Development
Volume 8, Issue 2 (2013)
Guest Editors: Kenneth Bush and Colleen Duggan

“Those who work in support of peacebuilding and development initiatives are acutely aware that conflict-affected environments are volatile, unpredictable and fast-changing. In light of this reality, evaluation and research in the service of peacebuilding and  development is a complex enterprise. Theories of change and assumptions about how peace and development work are often unarticulated or untested. While much work continues to be done on the theories, methodologies and praxis of peacebuilding, we suggest that the international aid community, researchers and practitioners need to think more deeply and systematically about the role of evaluation in increasing the efficacy of projects and programmes in violently divided societies (VDS).

Core questions that underpin and motivate the articles contained in this special issue include.

• How does the particular context of conflict affect our approaches to, and conduct of, research and evaluation?

• Specifically, how do politics — be they local, national, international, geopolitical — interact with evaluation practice in ways that enhance or inhibit prospects for peace and sustainable development?

• What can we learn from current research and evaluation practice in the global North and South about their impacts in VDS?

• Which tools are most effective and appropriate for assessing the role of context? Should there be generic or global assessment frameworks, criteria and indicators to guide evaluation in VDS, and, if so, what do they look like? Or does the fluidity and heterogeneity of different conflict zones inhibit such developments?

• How can evaluation, in its own right, catalyse positive political and societal change? What theories of peacebuilding and social change should best guide evaluation research and practice in ways that promote peace and sustainable development?

Duggan & Bush on Evaluation in Settings Affected by Violent Conflict: What Difference Does Context Make?

From AEA365:| A Tip-a-Day by and for Evaluators. Posted: 08 Feb 2013 12:51 AM PST

“We are Colleen Duggan, Senior Evaluation Specialist, International Development Research Centre (Canada) and Kenneth Bush, Director of Research, International Conflict Research (Northern Ireland).  For the past three years, we have been collaborating on a joint exploratory research project called Evaluation in Extremis:  The Politics and Impact of Research in Violently Divided Societies, bringing together researchers, evaluators, advocates and evaluation commissioners from the global North and South. We looked at the most vexing challenges and promising avenues for improving evaluation practice in conflict-affected environments.

CHALLENGES Conflict Context Affects Evaluation – and vice versa.  Evaluation actors working in settings affected by militarized or non-militarized violence suffer from the typical challenges confronting development evaluation.  But, conflict context shapes how, where and when evaluations can be undertaken – imposing methodological, political, logistical, and ethical challenges. Equally, evaluation (its conduct, findings, and utilization) may affect the conflict context – directly, indirectly, positively or negatively.

Lessons Learned:

Extreme conditions amplify the risks to evaluation actors.  Contextual volatility and political hyper-sensitivity must be explicitly integrated into the planning, design, conduct, dissemination, and utilization of evaluation.

  1. Some challenges may be anticipated and prepared for, others may not. By recognizing the most likely dangers/opportunities at each stage in the evaluation process we are better prepared to circumvent “avoidable risks or harm” and to prepare for unavoidable negative contingencies.
  2. Deal with politico-ethics dilemmas. Being able to recognize when ethics dilemmas (questions of good, bad, right and wrong) collide with political dilemmas (questions of power and control) is an important analytical skill for both evaluators and their clients.  Speaking openly about how politics and ethics – and not only methodological and technical considerations – influence all facets of evaluation in these settings reinforces local social capital and improves evaluation transparency.
  3. The space for advocacy and policymaking can open or close quickly, requiring readiness to use findings posthaste. Evaluators need to be nimble, responsive, and innovative in their evaluation use strategies.

Rad Resources:

  • 2013 INCORE Summer School Course on Evaluation in Conflict Prone Settings , University of Ulster, Derry/ Londonderry (Northern Ireland. A 5-day skills building course for early to mid-level professionals facing evaluation challenges in conflict prone settings or involved in commissioning, managing, or conducting evaluations in a programming or policy-making capacity.
  • Kenneth Bush and Colleen Duggan ((2013) Evaluation in Extremis: the Politics and Impact of Research in Violently Divided Societies (SAGE: Delhi, forthcoming)

Guidance for designing, monitoring and evaluating peacebuilding projects: using theories of change

CARE, June 2012. Available as pdf

“To advance the use of theory-based inquiry within the field of peacebuilding, CARE International and International Alert undertook a two and a half year research project to develop light touch methods to monitor and evaluate peacebuilding projects, and pilot these in Democratic Republic of Congo (DRC), Nepal and Uganda. This document, Guidance for designing, monitoring and evaluating peacebuilding project: using theories of change emerges from the efforts of peacebuilders who field tested the processes to define and assess the changes to which they hoped to contribute.

The main audiences for this guide are conflict transformation and peacebuilding practitioners, non-governmental organisations (NGOs) and donor agencies. Other actors in the conflict transformation and peacebuilding field may also find it useful.”

Contents page

Acknowledgements
1. Overview
1.1 The problem we seek to address
1.2 The research that developed the guidance
1.3 Definitions
2. Theories of change
2.1 What is a theory of change?
2.2 Why is it important to explicitly state theories of change?
3. Using theories of change for project or programme design
3.1 Carry out a conflict analysis
3.2 Design an intervention
3.3 Develop a results hierarchy
3.4 Articulate the theories of change
4.  Monitoring and evaluating of a project or programme based on  its theories of change
4.1 Identify / refine the theories of change
4.2 Assess a project or programme’s relevance
4.3 Decide what you want to learn: choose which theory of change
4.4 Undertake outcome evaluation
4.5  Design a research plan using the monitoring and evaluation grid to assess  whether the theory of change is functioning as expected, and collect data according to the plan
4.6 Data collection methods
4.7 Helpful hints to manage data collection and analysis
4.8 Analysis of data
5.  Present your findings and ensure their use
Annex 1: Questions to ask to review a conflict analysis
Annex 2: A selection of conflict analysis tools and frameworks
Annex 3: Additional resources
Notes

Peacebuilding with impact: Defining Theories of Change

Care International UK, January 2012. 12 pages. Available as pdf

Executive Summary: “Focusing on theories of change can improve the effectiveness of peacebuilding interventions. A review of 19 peacebuilding projects in three confict-affected countries found that the process of articulating and reviewing theories of change adds rigour and transparency, clarifes project logic, highlights assumptions that need to be tested, and helps identify appropriate participants and partners. However, the approach has limitations, including the diffculty of gathering theory validating evidence.

While they are not a panacea, devoting greater attention to theories of change is a simple and relatively inexpensive means of increasing the quality of peacebuilding interventions. Donors and peacebuilding agencies should review their procedures to encourage and accommodate more widespread focus on theories of change, and ensure adequate resources are set aside to allow appropriate monitoring of these theories throughout the life of an intervention.

A focus on theories of change led to the following key fndings:
• Clarifying project logic helps highlight tenuous assumptions;
• Clearly identifying the aims of activities and measures of success strengthens project design;
• Determining the appropriate actors to work with, and not just the easy-to-reach, enables better programme focus;
• More explicit links need to be made between local level activities and national peace processes for desired changes to occur;
• Confict analysis is critical for determining the relevance of activities but is rarely done;
• Staff often require support in ensuring their theories of change are suffciently explicit;
• Current project planning tools do not help practitioners articulate their theories of change;
• Gathering evidence to validate a theory of change is challenging, particularly in conditions of conflict and fragility;
• Critical review of theories of change needs to be undertaken in conjunction with other forms of evaluation to have maximum value;
• Theories of change can encourage an overly linear approach, when change in con?ict contexts can be more organic or systemic.

Recommendations:
1 Donors should revise their logical frameworks guidance to encourage the use of theories of change, notably to include them within the ‘assumptions and risks’ column of existing logical frameworks or by adding an additional column.
2 Theories of change need to be as precise, nuanced and contextually specific as possible and be based on broad conflict analysis.
3 Practitioners need to articulate theories of change within a hierarchy of results and to review these periodically throughout the implementation of a project, particularly if conflict dynamics change.
4 Donors should encourage funded agencies to review their theories of change throughout the project cycle and make resources available for this.”

Improving Peacebuilding Evaluation A Whole-of-Field Approach

by Andrew Blum, June 2011. United States Institute of Peace,  Available as pdf Found courtesy of @poverty_action

Summary

  • The effective evaluation of peacebuilding programs is essential if the field is to learn what constitutes effective and ineffective practice and to hold organizations accountable for using good practice and avoiding bad practice.
  • In the field of peacebuilding evaluation, good progress has been made on the intellectual front. There are now clear guidelines, frameworks, and tool kits to guide practitioners who wish to initiate an evaluation process within the peacebuilding field.
  • Despite this, progress in improving peacebuilding evaluation itself has slowed over the past several years. The cause of this is a set of interlocking problems in the way the peacebuilding field is organized. These in turn create systemic problems that hinder effective evaluation and the utilization of evaluation results.
  • The Peacebuilding Evaluation Project, organized by USIP and the Alliance for Peacebuilding, brought funders and implementers together to work on solutions to the systemic problems in peacebuilding work. This report discusses these solutions, which are grouped into three categories: building consensus, strengthening norms, and disrupting practice and creating alternatives. Several initiatives in each of these categories are already under way.

About the Report

In May 2010, the Alliance for Peacebuilding in collaboration with the United States Institute of Peace launched the Peacebuilding Evaluation Project. Over the course of a year, the project held a series of four meetings in Washington, DC. The goal of the project was to foster collaboration among funders, implementers, and policymakers to improve evaluation practice in the peacebuilding field. This report is inspired by the deep and far-ranging conversations that took place at the meetings. Its central argument is that whole-of-field approaches designed to address systemic challenges are necessary if the practice of peacebuilding evaluation is to progress.

http://apture.com/?ref=hotspotsbetahgvh7

The Evaluation of Storytelling as a Peace-building Methodology

Experiential Learning Paper No. 5
January 2011

www.irishpeacecentres.org

This paper is the record of an international workshop which was held in Derry in September 2010 on the evaluation of storytelling as a peace-building methodology. This was an important and timely initiative because currently there is no generally agreed method of evaluating storytelling despite the significant sums of money invested in it, not least by the EU PEACE Programmes. It was in fact PEACE III funding that enabled this examination of the issue to take place. This support allowed us to match international experts in evaluation with experts in storytelling in a residential setting over two days. This mix proved incredibly rich and produced this report, which we believe is a substantial contribution to the field. It is an example of the reflective practice which is at the heart of IPC’s integrated approach to peace-building and INCORE’s focus on linking research with peace-building practice. Built on this and other initiatives, one of IPC’s specific aims is to create a series of papers that reflect the issues which are being dealt with by practitioners.

Contents:
Foreword 4
Introduction 5
Presentations, Interviews and Discussions 13
Final Plenary Discussion 52
Conclusions:
a. What we have learned about storytelling 65
b. What we have learned about the evaluation of storytelling 69
c. What next? 73
Appendix 1: Reflection Notes from Small
Discussion Groups 75
Appendix 2: How does storytelling work in violently divided societies? Questioning the link between storytelling and peace-building 112
Appendix 3: Workshop Programme 116
Appendix 4: Speaker Biographies 118
Appendix 5: Storytelling & Peace-building References and Resources 122

PS: Ken Bush has passed on this message:

Please find attached an updated copy of the Storytelling and Peacebuilding BIBLIOGRAPHY.  Inclusion of web addresses makes it particularly useful.

%d bloggers like this: