Tag: participation

Who Counts? The power of participatory statistics

Edited By Jeremy Holland, published by Practical Action. 2013

(from the Practical Action website) “Local people can generate their own numbers – and the statistics that result are powerful for themselves and can influence policy. Since the early 1990s there has been a quiet tide of innovation in generating statistics using participatory methods. Development practitioners are supporting and facilitating participatory statistics from community-level planning right up to sector and national-level policy processes. Statistics are being generated in the design, monitoring and evaluation, and impact assessment of development interventions.Through chapters describing policy, programme and project research, Who Counts? provides impetus for a step change in the adoption and mainstreaming of participatory statistics within international development practice. The challenge laid down is to foster institutional change on the back of the methodological breakthroughs and philosophical commitment described in this book. The prize is a win–win outcome in which statistics are a part of an empowering process for local people and part of a real-time information flow for those aid agencies and government departments willing to generate statistics in new ways. Essential reading for researchers and students of international development as well as policy-makers, managers and practitioners in development agencies.”
Table of Contents
1 Introduction Participatory statistics: a ‘win–win’ for international development Jeremy Holland
PART I Participatory statistics and policy change
2 Participatory 3-dimensional modelling for policy and planning: the practice and the potential , Giacomo Rambaldi
3 Measuring urban adaptation to climate change: experiences in Kenya and Nicaragua Caroline Moser and Alfredo Stein
4 Participatory statistics, local decision-making, and national policy design: Ubudehe community planning in Rwanda  ,Ashish Shah
5 Generating numbers with local governments for decentralized health sector policy and planning in the Philippines , Rose Marie R. Nierras
6 From fragility to resilience: the role of participatory community mapping, knowledge management, and strategic planning in Sudan , Margunn Indreboe Alshaikh
Part II Who counts reality? Participatory statistics in monitoring and evaluation ,
7 Accountability downwards, count-ability upwards: quantifying empowerment outcomes from people’s own analysis in Bangladesh , Dee Jupp with Sohel Ibn Ali
8 Community groups monitoring their impact with participatory statistics in India: reflections from an international NGO Collective , Bernward Causemann, Eberhard Gohl, C. Rajathi, A. Susairaj, Ganesh Tantry and Srividhya Tantry,
9 Scoring perceptions of services in the Maldives: instant feedback and the power of increased local engagement , Nils Riemenschneider, Valentina Barca, and Jeremy Holland
10 Are we targeting the poor? Lessons with participatory statistics in Malawi , Carlos Barahona
PART III Statistics for participatory impact assessment
11 Participatory impact assessment in drought policy contexts: lessons from southern Ethiopia , Dawit Abebe and Andy Catley
12 Participatory impact assessment: the ‘Starter Pack Scheme’ and sustainable agriculture in Malawi , Elizabeth Cromwell, Patrick Kambewa, Richard Mwanza, and Rowland Chirwa with KWERA Development Centre,
13 Participatory impact assessments of farmer productivity programmes in Africa Susanne Neubert
Afterword , Robert Chambers
Practical and accessible resources
Index

Collaborative Evaluations Step by Step

by Liliana Rodriguez-Campos and Rigoberto Rincones-Gomez, Stanford Business Books, 2013 (2nd edition, first was in 2005)

Book website here and available on Amazon. But neither sites show the contents pages or exerpts

Book website says “Collaborative Evaluations is a highly comprehensive and easy-to-follow book for those evaluators who want to engage and succeed in collaborative evaluations. The author presents the Model for Collaborative Evaluations (MCE) with its six major components: (1) identify the situation, (2) clarify the expectations, (3) establish a shared commitment, (4) ensure open communication, (5) encourage best practices, and (6) follow specific guidelines. In clear and simple language, the author outlines key concepts and methods to help master the mechanics of collaborative evaluations.

Each section deals with fundamental factors inside each of the six collaborative evaluation components. In addition, each section provides practical tips for “real-life” applications and step-by-step suggestions or guidelines on how to apply this information. The MCE has emerged from a wide range of collaboration efforts that the author has conducted in the private sector, nonprofit organizations, and institutions of higher education. The author shares her experience and insights regarding this subject in a precise and easy-to-understand fashion, so that the reader can use the information learned from this book immediately.”

Related blog posting: Josey Landrieu on Collaborative Evaluation, on the AEA365 | A Tip-a-Day by and for Evaluators website

Innovations in Monitoring and Evaluation ‘as if Politics Mattered’,

Date: 17-18 October 2011
Venue: ANU, Canberra, Australia

Concept Note, Chris Roche & Linda Kelly, 4 August 2011

The Developmental Leadership Program (DLP)[1] addresses an important gap in international thinking and policy about the critical role played by leaders, elites and coalitions in the politics of development. At the core of DLP thinking is the proposition that political processes shape developmental outcomes at all levels and in all aspects of society: at national and sub-national levels and in all sectors and issue areas.

Initial findings of the DLP research program confirm that development is a political process and that leadership and agency matter. This is of course not new, but the DLP research provides important insights into how, in particular, leadership, elites and formal and informal coalitions can play a particularly important and under-recognized role in institutional formation (or establishing the ‘rules of the game’), policy reform and development processes[2].

International aid therefore needs to engage effectively with political processes. It needs to be flexible and be able to respond when opportunities open up. It needs to avoid the danger of bolstering harmful political settlements.

Furthermore Monitoring & Evaluation (M&E) mechanisms need to be improved and made compatible with flexible programming and recognize the importance of ‘process’ as well as outcomes. Donors should invest in a range of monitoring and evaluation techniques and approaches which are more aligned with the kinds of non-linear and unpredictable processes which characterise the kinds of political processes which drive positive developmental outcomes. This is important because it can be argued that, at best, current approaches are simply not appropriate to monitor the kinds of processes DLP research indicates are important; or, at worst, they offer few incentives to international assistance agencies to support the processes that actually lead to developmental outcomes Continue reading “Innovations in Monitoring and Evaluation ‘as if Politics Mattered’,”

Social Return On Investment: A practical guide for the development cooperation sector

by Jan Brouwers, Ester Prins and Menno Salverda (2010) , Utrecht, Context International Cooperation / Creative Commons, 60pp, ISBN 978-90-77526-06-4.  Available free online

Description
“Related to Cost Benefit Analyses and Participatory Planning, Monitoring and Evaluation is a new methodology called SROI; Social Return On Investment. As the term implies investments are not made to simply generate financial returns; in fact they create returns in many other areas, including poverty reduction or environmental protection. The SROI process attempts to provide insight in multiple values such that better decisions about investment (development) choices can be made.

The Social Return On Investment (SROI) practical guide, published by Context, international cooperation, provides guidelines and follows nine concrete steps in how to implement a SROI analysis. In this manual, the SROI idea is translated into the practice of international cooperation. What is interesting is that the manual is written in a write shop, where some fifteen SROI practitioners from Africa and Asia have shared and written down their practical experience. This results in a very readable guide with clear examples and practical tips of existing case studies and experiences.

Although SROI conceptualizing and practice should still be considered in its infancy, generally practitioners find that SROI can play an important role in creating awareness and appreciation of different (types of) values. In other words, by using SROI, the landscape of values, previously hidden (or externalised from market values) will change with social, environmental, economic and other values integrated.

Moreover, the communities, the development practitioners and all the related stakeholders, develop a TVC perspective (Total Value Consciousness – Economic, Social & Environmental), which helps in shifting mindsets or perceptions to planning and monitoring of development programs. The focus on social & environmental benefits will motivate the communities to participate in the program actively. This in turn creates a shift in the balance of whether a project is considered beneficial, profitable or commercially viable and consequently would lead to opportunities and implementation of new and innovative initiatives that genuinely contribute to positive social change and poverty reduction for all.

For whom?
For anyone interested in (new) methods for planning, monitoring and evaluation of social change or for those who want to try something different. SROI can be applied at an organisation level, at value chain levels or at the level of a more complex initiative at a community level. SROI can be applied in various fields: rural development, health, economic development, environmental economics, leadership programs, physical infrastructure, value of water infrastructure investment projects, etc. ”

Participatory Impact Assessment: A guide for practitioners

Andrew Catley – John Burns – Dawit Abebe – Omeno Suji, Feintein International Centre, Tufts University, 2008. Available as pdf

“Purpose of this guide

The Feinstein International Center has been developing and adapting participatory approaches to measure the impact of livelihoods based interventions since the early nineties. Drawing upon this experience, this guide aims to provide practitioners with a broad framework for carrying out project level Participatory Impact Assessments (PIA) of livelihoods interventions in the humanitarian sector. Other than in some health, nutrition, and water interventions in which indicators of project performance should relate to international standards, for many interventions there are no ‘gold standards’ for measuring project impact. For example, the Sphere handbook has no clear standards for food security or livelihoods interventions. This guide aims to bridge this gap by outlining a tried and tested approach to measuring the impact of livelihoods projects. The guide does not attempt to provide a set of standards or indicators or blueprint for impact assessment, but a broad and flexible framework which can be adapted to different contexts and project interventions.

Consistent with this, the proposed framework does not aim to provide a rigid or detailed step by step formula, or set of tools to carry out project impact assessments, but describes an eight stage approach, and presents examples of tools which may be adapted to different contexts. One of the  objectives of the guide is to demonstrate how PIA can be used to overcome some of the inherent weaknesses in conventional humanitarian monitoring evaluation and impact assessment approaches, such as; the emphasis on measuring process as opposed to real impact, the emphasis on external as opposed to community based indicators of impact, and how to overcome the issue of weak or non-existent baselines. The guide also aims to demonstrate and provide examples of how participatory methods can be used to overcome the challenge of attributing impact or change to actual project activities. The guide will also demonstrate how data collected from the systematic use of participatory tools can be presented numerically, and can give representative results and provide evidence based data on project impact.

Objectives of the Guide

1. Provide a framework for assessing the impact of livelihoods interventions

2. Clarify the differences between measuring process and real impact

3. Demonstrate how PIA can be used to measure the impact of different projects in different contexts using community identified impact indicators

4. Demonstrate how participatory methods can be used to measure impact where no baseline data exists

5. Demonstrate how participatory methods can be used to attribute impact to a project

6. Demonstrate how qualitative data from participatory tools can be systematically”

Helpdesk Research Report: Participatory M&E and Beneficiary Feedback

from the Governance and Social Development Resource Centre

Date: 03.09.2010 Available as a pdf

Query: Please identify the existing literature on participatory monitoring and evaluation, with a particular emphasis on gaining wide-ranging beneficiary feedback. Comment on the coverage, scalability, risks, benefits and applicability. Enquirer: Aid Effectiveness Team, DFID

Contents
1. Overview
2. General Literature on PM&E
3. Beneficiary Feedback
4. The Use of New Technologies in PM&E
5. Additional Information
Continue reading “Helpdesk Research Report: Participatory M&E and Beneficiary Feedback”

The Katine Challenge: How to analyse 540+ stories about a rural development project

The Guardian & Barclays funded and AMREF implemented, Katine Community Partnerships Project in Soroti District, Uganda is exceptional in some respects and all too common in others.

It is exceptional in the degree to which its progress has been very publicly monitored since it began in October 2007. Not only have all project documents been made publicly available via the dedicated Guardian Katine website, but resident and visiting journalists have posted more than 540 stories about the  people, the place and the project. These stories provide an invaluable in-depth and dynamic picture of what has been happening in Katine, unparalleled by anything else I have seen in any other development aid project.

On the flip side, the project is all too common in the kind of design and implementation problems that have been experienced, along with its fair share of unpredictable and very influential external events, including dramatic turn-arounds in various government policies. Plus the usual share of staffing and contracting problems.

Right now the project has completed its third year of operation and is now heading into the fourth and final year, one more year than originally planned.

I have a major concern. It is during this final year that there will be more knowledge about the project available than ever before, but at the same time its donors, and perhaps various staff within AMREF, will be becoming more interested in other new events appearing over the horizon. For example, the Guardian will cease its intensive journalistic coverage of the project from this month, and attention is now focusing on their new international development website

So, I would like to pose an important challenge to all the visitors to the Monitoring and Evaluation NEWS website, and the associated MandE NEWS email list:

How can the 540+ stories be put to good use? Is there some form of analysis that could be made of their contents, that would help AMREF, the Guardian, Barclays, the people of Katine, and all of us learn more from the Katine project?

In order to help I have uploaded an Excel file listing all the stories since December 2008, with working hypertext links. I will try to progressively extend this list back to the start of the project in late 2007. This list includes copies of all progress reports, review and planning documents that  AMREF has given the Guardian to be uploaded onto their website.

If you have any questions or comments please post them below, as Comments to this posting, in the first instance.

What would be useful in the first instance is ideas about plans or strategies for analysing the data. Then volunteers to actually implement one or more of these plans.

PS: My understanding is that the data is by definition already in the public domain, and therefore anyone could make use of it. However, that use should be fair and not for profit. What we should be searching for here are lessons or truths in some form that could be seen as having wider applicability, which are based on sound argument and good evidence, as much as is possible.

Smart Tools: For evaluating information projects, products and services

Produced by CTA, KIT, IICD. 2nd (2009) edition

PDF version available online

“About the Toolkit

The Smart Toolkit focuses on the evaluation of information projects, products and services from a learning perspective. It looks at evaluation within the context of the overall project cycle, from project planning and implementation to monitoring, evaluation and impact assessment, and then at the evaluation process itself, the tools involved and examples of their application.The theme running throughout the toolkit is:

Participatory evaluation for learning and impact Continue reading “Smart Tools: For evaluating information projects, products and services”

Measuring Empowerment? Ask Them

Quantifying qualitative outcomes from people’s own analysis. Insights for results-based management from the experience of a social movement in Bangladesh Dee Jupp Sohel Ibn Ali with contribution from Carlos Barahona 2010: Sida Studies in Evaluation. Download pdf

Preamble

Participation has been widely taken up as an essential element of development, but participation for what purpose? Many feel that its acceptance, which has extended to even the most conventional of institutions such as the international development banks, has resulted in it losing its teeth in terms of the original ideology of being able to empower those living in poverty and to challenge power relations.

The more recent emergence of the rights-based approach discourse has the potential to restore the ‘bite’ to participation and to re-politicise development. Enshrined in universal declarations and conventions, it offers a palatable route to accommodating radicalism and creating conditions for emancipatory and transformational change, particularly for people living in poverty. But an internet search on how to measure the impact of these approaches yields a disappointing harvest of experience. There is a proliferation of debate on the origins and processes, the motivations and pitfalls of rights-based programming but little on how to know when or if it works. The discourse is messy and confusing and leads many to hold up their hands in despair and declare that outcomes are intangible, contextual, individual, behavioural, relational and fundamentally un-quantifiable!

As a consequence, results-based management pundits are resorting to substantive measurement of products, services and goods which demonstrate outputs and rely on perception studies to measure outcomes.

However, there is another way. Quantitative analyses of qualitative assessments of outcomes and impacts can be undertaken with relative ease and at low cost. It is possible to measure what many regard as unmeasurable.

This publication suggests that steps in the process of attainment of rights and the process of empowerment are easy to identify and measure for those active in the struggle to achieve them. It is our etic perspectives that make the whole thing difficult. When we apply normative frames of reference, we inevitably impose our values and our notions of democracy and citizen engagement rather than embracing people’s own context-based experience of empowerment.

This paper presents the experience of one social movement in Bangladesh, which managed to find a way to measure empowerment by letting the members themselves explain what benefits they acquired from the Movement and by developing a means to measure change over time. These measures , which are primarily of use to the members, have then been subjected to numerical analysis outside of the village environment to provide convincing quantitative data, which satisfies the demands of results-based management.

The paper is aimed primarily at those who are excited by the possibilities of rights-based approaches but who are concerned about proving that their investment results in measurable and attributable change. The experience described here should build confidence that transparency, rigour and reliability can be assured in community led approaches to monitoring and evaluation without distorting the original purpose, which is a system of reflection for the community members themselves. Hopefully, the reader will feel empowered to challenge the sceptics.

Dee Jupp and Sohel Ibn Ali
Continue reading “Measuring Empowerment? Ask Them”

How Wide are the Ripples?

Report of the March 2010 workshop Prepared by Louise Clark, Kate Newman and Hannah Beardon

“This report presents reflections from a workshop held in London on 18th and 19th March 2010. The workshop was part of a larger process of reflection and research, supported by IKM Emergent and called ‘How Wide Are the Ripples?’. The process explored how international development NGOs use and manage the information, knowledge and perspectives generated through the participatory processes they initiate or fund. The initial research and report built on a literature review and case studies from five international NGOs (ActionAid, Concern, Healthlink, Panos and Plan), identifying challenges and opportunities to good bottom-up information and learning flows. The workshop invited participants from the original research and others working on and around these issues to reflect further on the challenges and discuss practical solutions based on their own experiences.

Participants came from a mix of large international development NGOs, smaller organisations and included independent consultants . The variety of organisations was not only evident in their size, but also in their different structures and relationships with grassroots processes and organisations, a recurring theme throughout the discussions. The expectation, and commitment from the participants, was that these discussions and experiences would feed into a guest-edited edition of the IIED journal Participatory Learning and Action (PLA), in June 2011. The workshop was therefore organised around two main goals: improving the practice of international development NGOs in relation to information generated through participatory processes, through workshop discussions and by developing a network for support and sharing ideas; and promoting further reflection and learning around specific issues, in particular through developing articles for the edition of PLA.”

%d bloggers like this: