Monitoring and Evaluation NEWS

Measuring Results for Dutch Development Aid, Approaches and Future Directions

Date: October 4-7, 2010
Venue: Royal Tropical Institute, Amsterdam,

The International Institute of Social Studies and The Amsterdam Institute for International Development invite applications / submissions for a training and conference event on Measuring Results for Dutch Development Aid, Approaches and Future Directions with financial support from the Policy and Operations Evaluation Department of the Dutch Ministry of Development Cooperation.

Participation is free of charge, but places are limited.
Deadline for applications: September 10, 2010
Click here to apply

Objectives: Share results from and experiences with impact evaluation in developing countries, and discuss their relevance for Dutch development cooperation.

Target Audiences: Researchers, NGOs, consulting companies and policy makers in the Netherlands conducting or using impact evaluation to study the effectiveness of development assistance.

Confirmed speakers: Dr. Howard White, director of International Initiative for Impact Evaluation(3ie).
Dr. Paul Gertler, Professor of Economics, University of California, Berkeley.
Dr. Sulley Gariba, Executive Director, Institute for Policy Alternatives, Ghana.
Prof. Ruerd Ruben, director of the Policy and Operations Evaluation Department of the Dutch Ministry of Development Cooperation (starting Sept 1).

Submit a paper (optional): Contributed papers are sought in the area of (1) completed impact evaluations, (2) reviews of impact evaluations on a particular sector, (3) position papers on approaches to impact evaluations in relation to decision making.

Selection criteria: Quality of submission and/or professional link with result assessment for development assistance and/or participation in the impact evaluation training.

Maximum number of participants: 100

PROGRAM »

Tags: impact, measurement, results

Posted in Conferences, Types of Events | No Comments » |  Print This Post

(No Ratings Yet)

 Loading ...

Measuring Up: HIV-related advocacy evaluation training pack (draft)

HIV-related advocacy evaluation training for civil society organisations.

Produced by the International HIV/AIDS Alliance (Secretariat), International Council of AIDS Service Organizations (ICASO), July 2010, 38 pages. Available as .pdf

“This training pack is published by the Alliance and the International Council of AIDS Service Organizations (ICASO) and consists of two guides designed for advocacy, monitoring and evaluation staff of civil society organisations (including networks) who are involved in designing, implementing and assessing advocacy projects at different levels. The purpose of these guides is to increase users’ capacity to evaluate the progress and results of their advocacy work. The guides aim to:

1. help users to identify and confront the challenges faced by community-based organisations evaluating HIV-related advocacy
2. introduce new thinking for designing advocacy evaluations
3. give users the opportunity to apply some aspects of the evaluation design process to their specific contexts
4. make users aware that advocacy evaluation is a fast-growing and evolving field, with a large number of publications on advocacy evaluation design, approaches and methods available via the Internet and summarised in the resources section of the learner’s guide.”

Tags: advocacy, Civil Society / NGO, HIV, Training

Posted in Uncategorized | 1 Comment » |  Print This Post

(No Ratings Yet)

 Loading ...

Updated MSC bibliography

This page is intended to provide  an update of the bibliography in the 2005 Most Significant Change technique (MSC) Users Guide

Please feel free to suggest additions to this list, through the Comment facility below, or by emailing the editor (Rick Davies)

Papers

• MSC: Misconceptions, strengths and challenges (2009) Fiona Kotvojs and Carolina Lasambouw. Presented at the Australasian Evaluation Conference  September 2009 Available as pdf
• Most Significant Change Stories from the Challenge Program on Water and Food (CPWF) (2009) Compiled and edited by Cristina de Leon, Boru Douthwaite and Sophie Alvarez, CPWF Impact Project. 92 pages. Available as .pdf
• Most significant change technique and the national training system: Core Ideas (2009) Dr John Mitchell & Suzy McKenna
• M&E Waardewerken. Monitoring en Evaluatie van het innovatienetwerk Waardewerken met de ‘Most Significant Change’ methode. Jong, Daniël de, Sandra Bos, Marlèn Arkesteijn and Jorieke Potters (2008), Wageningen UR ASG. (M&E of Waardewerken. Monitoring and Evaluation of the Innovation Network ‘Waardewerken’(Multifunctional Agriculture) using the ‘Most Significant Change’ method)
• Cambio más significativo (CMS): evaluando sin indicadores (2008) 2 pages
• The Most Significant Change: using participatory video for monitoring and evaluation (2007) by CHRIS LUNCH. 5 pages. Available as .pdf
• The most significant lessons about the Most Significant Change technique (2007) Juliet Willetts, Paul Crawford Development in Practice, Volume 17, Issue 3 June 2007 , pages 367 – 379
• Stories of Significance: Redefining Change. An assortment of community voices and articulations. A report based on an evaluation of a programme on “Community Driven Approaches to Address the Feminisation of HIV/AIDS in India” by means of the ‘Most Significant Change’ Technique. (July 2007)
• Methoden voor monitoring en evaluatie van innovatieprojecten. (Monitoring and Evaluation Methods for Innovation Projects), Arkesteijn, Marlèn, Jorieke Potters and Barbara van Mierlo (2007), Wageningen UR PPO, Lelystad.
  This report examines the usefulness/ applicability of various M&E methods (MSC, Learning Histories, result orientend M&E etc) for innovation networks.
• Evaluation: LandLearn’s most significant change (2006) Lydia Fehring, Jenny Pettenon, Ann Fagan, Kathryn Goyen, Jessica Connor. Paper given at the APEN International Conference 6-8 March 2006 at Beechworth, Victoria, Australia
• Monitoreo con el Cambio Mas Significativo (CMS): La experienca de Ibis en Nicaragua (2006) Ibis, Denmark.
• “Learning from Capacity Building Practice:Adapting the ‘Most Significant Change’ Approach to Evaluate Capacity Building Provision by CABUNGO in Malawi” by Rebecca Wrigley: – Praxix Paper No. 12, INTRAC, Oxford, December (2006)
• ‘Most Significant Change’ Pilot Project Evaluation Report (2004) Prepared by Institute for Sustainable Futures  For ADRA Laos

Powerpoints

• Seven sets of slides, used for 2 day MSC training in Delhi, 2008 by Rick Davies . Available on request , on condition of willingness to share any adaptations made

• Undated Most Significant Change, by Tracey Delaney. 17 slides.

YouTube video

• 2007 Jess Dart on MSC: Part 1, Part 2, Part 3, Part 4, Part 5, Q&A.

Other

• The introductory page on MSC, also on this website
• Clear Horizon’s webpage on MSC

Tags: Most Significant Change (MSC)

Posted in Lists of sources, Lists of sources re ..., MSC bibliography | No Comments » |  Print This Post

(No Ratings Yet)

 Loading ...

Designing Initiative Evaluation A Systems-oriented Framework for Evaluating Social Change Efforts

W. K. Kellogg Foundation, 2007. 48 pages. Available as pdf.

“Purpose”

“This document is designed for use by external evaluators who conduct initiative evaluations for theW.K. Kellogg Foundation (WKKF) – and, hopefully, other foundations and government agencies. It presents a systems-oriented framework and four general designs for initiative and cluster evaluation. The designs are based on systems concepts related to change and the dynamics of systems. The focus is not on considering all ideas about systems that could be applied to initiative evaluation, rather on how different dynamics within systems can serve as the basis for initiative evaluation designs.” Read the rest of this entry »

Tags: cluster, systems

Posted in Uncategorized | No Comments » |  Print This Post

(No Ratings Yet)

 Loading ...

UK Evaluation Society 2010 Annual Evaluation Conference

Evaluation in a turbulent world: Challenges, opportunities and innovation in evaluation practice
Date: 22-23 November 2010
Venue: Macdonald Burlington Hotel, Birmingham

Tags: collaboration, Innovation, methodology, utilisation

Posted in Conferences, Types of Events | 1 Comment » |  Print This Post

(No Ratings Yet)

 Loading ...

The American Evaluation Association annual conference: Evaluation Quality

Date: November 10-13, 2010
Venue: San Antonio, Texas

The American Evaluation Association invites evaluators from around the world to attend its annual conference to be held Wednesday, November 10, through Saturday, November 13, 2010 in San Antonio, Texas. We’ll be convening at the lovely Grand Hyatt San Antonio, right in the heart of the vibrant city and adjacent to the Riverwalk’s nightlife, restaurants, and strolling grounds. Discounted hotel reservations will be available in March.

AEA’s annual meeting is expected to bring together approximately 2500 evaluation practitioners, academics, and students, and represents a unique opportunity to gather with professional colleagues in a collaborative, thought-provoking, and fun atmosphere.

The conference is broken down into 44 Topical Strands that examine the field from the vantage point of a particular methodology, context, or issue of interest to the field as well as the Presidential Strand highlighting this year’s Presidential Theme of Evaluation Quality. Presentations may explore the conference theme or any aspect of the full breadth and depth of evaluation theory and practice.

Proposals are due by midnight in the Eastern time zone, on Friday, March 19, 2010.
For more information: http://www.eval.org/eval2010/10cfp.htm

Tags: quality

Posted in Conferences, Types of Events | No Comments » |  Print This Post

(No Ratings Yet)

 Loading ...

RESOURCE PACK ON SYSTEMATIZATION OF EXPERIENCES

ActionAid International, 2009, 104 pages.  Available as pdf 3.39Mb

See also the associated AAI website on systematization

“Systematization is a methodology that offers a way to do all of the above. It allows us to:

• Organise and document what we have learnt through our work
• Better understand the impact of our work and the ways in which change happens
• Develop deeper understanding about our work and the challenges we face to inform new ways of working
• Capture and communicate the complexity and richness of our work

Tags: Civil Society / NGO, Learning

Posted in Uncategorized | No Comments » |  Print This Post

(No Ratings Yet)

 Loading ...

Critique of Governance Assessment Applications

GRDC Helpdesk Research Report by Sumedh Rao, Governance and Social Development Resource Centre, July 2010. 16 pages. Available as pdf

Query:  Identify the key literature that critiques the use and application of governance assessments.  Enquirer: DFID

Contents
1. Overview
2. General critiques
3. Critiques of measurement
4. Worldwide Governance Indicators (WGI)
5. African Peer Review Mechanism (APRM)
6. Other assessments
7. Donor Guidance
8. Initiatives for improving assessments

Including a bibliography of 39 annotated references Read the rest of this entry »

Tags: governance, indicators

Posted in Uncategorized | No Comments » |  Print This Post

(No Ratings Yet)

 Loading ...

Narrative Research

David Snowden, 2010.  21 pages. Available as pdf, from the Cognitive Edge site

“Narrative Research, … lays the foundation for the use of narrative research and inquiry methods not only in the project but broadly in the field of research and consultancy…. Elements of it together with general material on Complexity Theory will be published as a chapter in a book on Naturalising Decision Making in the Fall of 2010.”

Tags: complexity, stories

Posted in Uncategorized | No Comments » |  Print This Post

(No Ratings Yet)

 Loading ...

Listen First: a pilot system for managing downward accountability in NGOs

Alex Jacobs and Robyn Wilford. Development in Practice, Volume 20, Number 7, September 2010 Available as pdf

“Abstract: This article reports on a research project intended to develop systematic ways of managing downward accountability in an international NGO. Innovative tools were developed and trialled in six countries. The tools comprised a framework, de?ning downward accountability in practical terms, and three management processes. They were successfully used to
(a) encourage staff to improve downward accountability in ways relevant to their context;
(b) hear bene?ciaries’ assessments of the level of accountability achieved and the value of the NGO’s work; and (c) generate quanti?ed performance summaries for managers. Taken together, they form a coherent draft management system. Areas for further research are identied.”

There’s more related material  at www.listenfirst.org,

Tags: accountability, Civil Society / NGO

Posted in Uncategorized | No Comments » |  Print This Post

(No Ratings Yet)

 Loading ...

The Limits of Nonprofit Impact: A Contingency Framework for Measuring Social Performance

Alnoor Ebrahim, V. Kasturi Rangan, Social Enterprise Initiative, Harvard Business School (2010) Working Paper 10-099 Available as pdf

ABSTRACT

“Leaders of organizations in the social sector are under growing pressure to demonstrate their impacts on pressing societal problems such as global poverty. We review the debates around performance and impact, drawing on three literatures: strategic philanthropy, nonprofit management, and international development. We then develop a contingency framework for measuring results, suggesting that some organizations should measure long-term impacts, while others should focus on shorter-term outputs and outcomes. In closing, we discuss the implications of our analysis for future research on performance management.”

Tags: accountability, Civil Society / NGO, impact assessment

Posted in Uncategorized | No Comments » |  Print This Post

(1 votes, average: 5.00 out of 5)

 Loading ...

Smart Tools: For evaluating information projects, products and services

Produced by CTA, KIT, IICD. 2nd (2009) edition

PDF version available online

“About the Toolkit

The Smart Toolkit focuses on the evaluation of information projects, products and services from a learning perspective. It looks at evaluation within the context of the overall project cycle, from project planning and implementation to monitoring, evaluation and impact assessment, and then at the evaluation process itself, the tools involved and examples of their application.The theme running throughout the toolkit is:

Participatory evaluation for learning and impact Read the rest of this entry »

Tags: Information / Communication Technology (ICT), participation, self-evaluation, tools

Posted in Uncategorized | No Comments » |  Print This Post

(No Ratings Yet)

 Loading ...

EVALUATING DEVELOPMENT CO-OPERATION: SUMMARY OF KEY NORMS AND STANDARDS. SECOND EDITION

OECD DAC NETWORK ON DEVELOPMENT EVALUATION, February 2010 Download a pdf copy

“The DAC Network on Development Evaluation is a unique international forum that brings together evaluation managers and specialists from development co-operation agencies in OECD member countries and multilateral development institutions. Its goal is to increase the effectiveness of international development programmes by supporting robust, informed and independent evaluation.

A key component of the Network’s mission is to develop internationally agreed norms and standards to strengthen evaluation policy and practice. Shared standards contribute to harmonised approaches in line with the commitments of the Paris Declaration on Aid Effectiveness. The body of norms and standards is based on experience, and evolves over time to fit the changing aid environment. These principles serve as an international reference point, guiding efforts to improve development results through high quality evaluation.

The norms and standards summarised here should be applied discerningly and adapted carefully to fit the purpose, object and context of each evaluation. This summary document is not an exhaustive evaluation manual. Readers are encouraged to refer to the complete texts available on the DAC Network on Development Evaluation’s website: www.oecd.org/dac/evaluationnetwork. Several of the texts are also available in other languages.”

Tags: bilateral agencies, Multilaterals, standards

Posted in Uncategorized | No Comments » |  Print This Post

(No Ratings Yet)

 Loading ...

DEVELOPMENT EVALUATION RESOURCES AND SYSTEMS – A STUDY OF NETWORK MEMBERS

The DAC Network on Development Evaluation,  OECD, 2010. Download a pdf copy

“Introduction

In June 2009, the Organisation for Economic Co-operation and Development (OECD) Development Assistance Committee (DAC) Network on Development Evaluation agreed to undertake a study of its members’ evaluation systems and resources. The study aims to take stock of how the evaluation function is managed and resourced in development agencies and to identify major trends and current challenges in development evaluation. The purpose is to inform efforts to strengthen evaluation systems in order to contribute to improved accountability and better development results. It will be of interest to DAC members and evaluation experts, as well as to development actors in emerging donor and partner countries.

To capture a broad view of how evaluation works in development agencies, core elements of the evaluation function are covered, including: the mandate for central evaluation units, the institutional position of evaluation, evaluation funding and human resources, independence of the evaluation process, quality assurance mechanisms, co-ordination with other donors and partner countries, systems to facilitate the use of evaluation findings and support to partner country capacity development.

This report covers the member agencies of the OECD DAC Network on Development Evaluation.1 See Box 1 for a full list of member agencies and abbreviations. Covering all major bilateral providers of development assistance and seven important multilateral development banks, the present analysis therefore provides a comprehensive view of current policy and practice in the evaluation of development assistance.

The study is split into two sections: section I contains an analysis of overall trends and general practices, drawing on past work of the DAC and its normative work on development evaluation. Section II provides an individual factual profile for each member agency, highlighting its institutional set-up and resources.”

Tags: bilateral agencies, Multilaterals

Posted in Uncategorized | No Comments » |  Print This Post

(No Ratings Yet)

 Loading ...

Measuring Empowerment? Ask Them

Quantifying qualitative outcomes from people’s own analysis. Insights for results-based management from the experience of a social movement in Bangladesh Dee Jupp Sohel Ibn Ali with contribution from Carlos Barahona 2010: Sida Studies in Evaluation. Download pdf

Preamble

Participation has been widely taken up as an essential element of development, but participation for what purpose? Many feel that its acceptance, which has extended to even the most conventional of institutions such as the international development banks, has resulted in it losing its teeth in terms of the original ideology of being able to empower those living in poverty and to challenge power relations.

The more recent emergence of the rights-based approach discourse has the potential to restore the ‘bite’ to participation and to re-politicise development. Enshrined in universal declarations and conventions, it offers a palatable route to accommodating radicalism and creating conditions for emancipatory and transformational change, particularly for people living in poverty. But an internet search on how to measure the impact of these approaches yields a disappointing harvest of experience. There is a proliferation of debate on the origins and processes, the motivations and pitfalls of rights-based programming but little on how to know when or if it works. The discourse is messy and confusing and leads many to hold up their hands in despair and declare that outcomes are intangible, contextual, individual, behavioural, relational and fundamentally un-quantifiable!

As a consequence, results-based management pundits are resorting to substantive measurement of products, services and goods which demonstrate outputs and rely on perception studies to measure outcomes.

However, there is another way. Quantitative analyses of qualitative assessments of outcomes and impacts can be undertaken with relative ease and at low cost. It is possible to measure what many regard as unmeasurable.

This publication suggests that steps in the process of attainment of rights and the process of empowerment are easy to identify and measure for those active in the struggle to achieve them. It is our etic perspectives that make the whole thing difficult. When we apply normative frames of reference, we inevitably impose our values and our notions of democracy and citizen engagement rather than embracing people’s own context-based experience of empowerment.

This paper presents the experience of one social movement in Bangladesh, which managed to find a way to measure empowerment by letting the members themselves explain what benefits they acquired from the Movement and by developing a means to measure change over time. These measures , which are primarily of use to the members, have then been subjected to numerical analysis outside of the village environment to provide convincing quantitative data, which satisfies the demands of results-based management.

The paper is aimed primarily at those who are excited by the possibilities of rights-based approaches but who are concerned about proving that their investment results in measurable and attributable change. The experience described here should build confidence that transparency, rigour and reliability can be assured in community led approaches to monitoring and evaluation without distorting the original purpose, which is a system of reflection for the community members themselves. Hopefully, the reader will feel empowered to challenge the sceptics.

Dee Jupp and Sohel Ibn Ali
Read the rest of this entry »

Tags: empowerment, participation, qualitative, quantative

Posted in Uncategorized | No Comments » |  Print This Post

(No Ratings Yet)

 Loading ...