The International Institute of Social Studies and The Amsterdam Institute for International Development invite applications / submissions for a training and conference event on Measuring Results for Dutch Development Aid, Approaches and Future Directions with financial support from the Policy and Operations Evaluation Department of the Dutch Ministry of Development Cooperation.

Participation is free of charge, but places are limited.
Deadline for applications: September 10, 2010
Click here to apply

Objectives: Share results from and experiences with impact evaluation in developing countries, and discuss their relevance for Dutch development cooperation.

Target Audiences: Researchers, NGOs, consulting companies and policy makers in the Netherlands conducting or using impact evaluation to study the effectiveness of development assistance.

Confirmed speakers: Dr. Howard White, director of International Initiative for Impact Evaluation(3ie).
Dr. Paul Gertler, Professor of Economics, University of California, Berkeley.
Dr. Sulley Gariba, Executive Director, Institute for Policy Alternatives, Ghana.
Prof. Ruerd Ruben, director of the Policy and Operations Evaluation Department of the Dutch Ministry of Development Cooperation (starting Sept 1).

Submit a paper (optional): Contributed papers are sought in the area of (1) completed impact evaluations, (2) reviews of impact evaluations on a particular sector, (3) position papers on approaches to impact evaluations in relation to decision making.

Selection criteria: Quality of submission and/or professional link with result assessment for development assistance and/or participation in the impact evaluation training.

Maximum number of participants: 100

PROGRAM »

Produced by the International HIV/AIDS Alliance (Secretariat), International Council of AIDS Service Organizations (ICASO), July 2010, 38 pages. Available as .pdf

“This training pack is published by the Alliance and the International Council of AIDS Service Organizations (ICASO) and consists of two guides designed for advocacy, monitoring and evaluation staff of civil society organisations (including networks) who are involved in designing, implementing and assessing advocacy projects at different levels. The purpose of these guides is to increase users’ capacity to evaluate the progress and results of their advocacy work. The guides aim to:

1. help users to identify and confront the challenges faced by community-based organisations evaluating HIV-related advocacy
2. introduce new thinking for designing advocacy evaluations
3. give users the opportunity to apply some aspects of the evaluation design process to their specific contexts
4. make users aware that advocacy evaluation is a fast-growing and evolving field, with a large number of publications on advocacy evaluation design, approaches and methods available via the Internet and summarised in the resources section of the learner’s guide.”

Please feel free to suggest additions to this list, through the Comment facility below, or by emailing the editor (Rick Davies)

Papers

• MSC: Misconceptions, strengths and challenges (2009) Fiona Kotvojs and Carolina Lasambouw. Presented at the Australasian Evaluation Conference  September 2009 Available as pdf
• Most Significant Change Stories from the Challenge Program on Water and Food (CPWF) (2009) Compiled and edited by Cristina de Leon, Boru Douthwaite and Sophie Alvarez, CPWF Impact Project. 92 pages. Available as .pdf
• Most significant change technique and the national training system: Core Ideas (2009) Dr John Mitchell & Suzy McKenna
• M&E Waardewerken. Monitoring en Evaluatie van het innovatienetwerk Waardewerken met de ‘Most Significant Change’ methode. Jong, Daniël de, Sandra Bos, Marlèn Arkesteijn and Jorieke Potters (2008), Wageningen UR ASG. (M&E of Waardewerken. Monitoring and Evaluation of the Innovation Network ‘Waardewerken’(Multifunctional Agriculture) using the ‘Most Significant Change’ method)
• Cambio más significativo (CMS): evaluando sin indicadores (2008) 2 pages
• The Most Significant Change: using participatory video for monitoring and evaluation (2007) by CHRIS LUNCH. 5 pages. Available as .pdf
• The most significant lessons about the Most Significant Change technique (2007) Juliet Willetts, Paul Crawford Development in Practice, Volume 17, Issue 3 June 2007 , pages 367 – 379
• Stories of Significance: Redefining Change. An assortment of community voices and articulations. A report based on an evaluation of a programme on “Community Driven Approaches to Address the Feminisation of HIV/AIDS in India” by means of the ‘Most Significant Change’ Technique. (July 2007)
• Methoden voor monitoring en evaluatie van innovatieprojecten. (Monitoring and Evaluation Methods for Innovation Projects), Arkesteijn, Marlèn, Jorieke Potters and Barbara van Mierlo (2007), Wageningen UR PPO, Lelystad.
  This report examines the usefulness/ applicability of various M&E methods (MSC, Learning Histories, result orientend M&E etc) for innovation networks.
• Evaluation: LandLearn’s most significant change (2006) Lydia Fehring, Jenny Pettenon, Ann Fagan, Kathryn Goyen, Jessica Connor. Paper given at the APEN International Conference 6-8 March 2006 at Beechworth, Victoria, Australia
• Monitoreo con el Cambio Mas Significativo (CMS): La experienca de Ibis en Nicaragua (2006) Ibis, Denmark.
• “Learning from Capacity Building Practice:Adapting the ‘Most Significant Change’ Approach to Evaluate Capacity Building Provision by CABUNGO in Malawi” by Rebecca Wrigley: – Praxix Paper No. 12, INTRAC, Oxford, December (2006)
• ‘Most Significant Change’ Pilot Project Evaluation Report (2004) Prepared by Institute for Sustainable Futures  For ADRA Laos

Powerpoints

• Seven sets of slides, used for 2 day MSC training in Delhi, 2008 by Rick Davies . Available on request , on condition of willingness to share any adaptations made

• Undated Most Significant Change, by Tracey Delaney. 17 slides.

YouTube video

• 2007 Jess Dart on MSC: Part 1, Part 2, Part 3, Part 4, Part 5, Q&A.

Other

• The introductory page on MSC, also on this website
• Clear Horizon’s webpage on MSC

“Purpose”

“This document is designed for use by external evaluators who conduct initiative evaluations for theW.K. Kellogg Foundation (WKKF) – and, hopefully, other foundations and government agencies. It presents a systems-oriented framework and four general designs for initiative and cluster evaluation. The designs are based on systems concepts related to change and the dynamics of systems. The focus is not on considering all ideas about systems that could be applied to initiative evaluation, rather on how different dynamics within systems can serve as the basis for initiative evaluation designs.”

“Cluster and Initiative Grantmaking at WKKF”

“Over the past three decades,WKKF has introduced two types of grantmaking that involve multiple projects, locations, and grantees: (1) “clusters” that typically focus on the exploration and/or development of new approaches to a program issue area; and (2) strategic “initiatives,” designed to create systems changes that will lead to intended long-term, sustainable impact.

As WKKF began to increasingly support clusters and initiatives, it became clear that new opportunities for learning and evaluation arose beyond those available when looking at single projects; thus began cluster and initiative evaluation. (The evaluation designs for clusters and initiatives evaluation are similar. They are termed together as “initiative evaluation” in this document.)”

The American Evaluation Association invites evaluators from around the world to attend its annual conference to be held Wednesday, November 10, through Saturday, November 13, 2010 in San Antonio, Texas. We’ll be convening at the lovely Grand Hyatt San Antonio, right in the heart of the vibrant city and adjacent to the Riverwalk’s nightlife, restaurants, and strolling grounds. Discounted hotel reservations will be available in March.

AEA’s annual meeting is expected to bring together approximately 2500 evaluation practitioners, academics, and students, and represents a unique opportunity to gather with professional colleagues in a collaborative, thought-provoking, and fun atmosphere.

The conference is broken down into 44 Topical Strands that examine the field from the vantage point of a particular methodology, context, or issue of interest to the field as well as the Presidential Strand highlighting this year’s Presidential Theme of Evaluation Quality. Presentations may explore the conference theme or any aspect of the full breadth and depth of evaluation theory and practice.

Proposals are due by midnight in the Eastern time zone, on Friday, March 19, 2010.
For more information: http://www.eval.org/eval2010/10cfp.htm

See also the associated AAI website on systematization

“Systematization is a methodology that offers a way to do all of the above. It allows us to:

• Organise and document what we have learnt through our work
• Better understand the impact of our work and the ways in which change happens
• Develop deeper understanding about our work and the challenges we face to inform new ways of working
• Capture and communicate the complexity and richness of our work

Query:  Identify the key literature that critiques the use and application of governance assessments.  Enquirer: DFID

Contents
1. Overview
2. General critiques
3. Critiques of measurement
4. Worldwide Governance Indicators (WGI)
5. African Peer Review Mechanism (APRM)
6. Other assessments
7. Donor Guidance
8. Initiatives for improving assessments

Including a bibliography of 39 annotated references

1. Overview

Governance  assessments  are  based  on  subjective  indicators  (or  measures),  objective indicators or a combination of  the  two, known as composite  indicators. Composite  indicators are  the most popular and are used by  international organisations, donors,  investors and  the media  (Arndt,  2008).  Of  these  the  most  popular  seems  to  be  the  World  Bank‘s  World Governance  Indicators  (WGIs).  Transparency  International‘s  Corruption  Perceptions  Index (CPI) and  the World Bank/International Finance Corporation‘s Doing Business  Indicators are also  in common use but  less so by  international organisations and donors. These  indicators
face  several  limitations  but  general  reliance  on  a  small  set  of  indicators  has  led  to  greater international  acceptance  and  further  reliance  on  these  indicators.  Many  critics  cite  the numerous failings of indicators and the way such failings or inaccuracies are routinely ignored by those who use them.

The  main  use  of  the  indicators  by  international  organisation  and  donors  is  to  incentivise developing nations  to  improve  their governance and  to  improve  the allocation of aid.   There will  thus  continue  to  be  a  demand  for  summary measurements which  can  be  used  across space and  time. At  the same  time  there  is a  growing  resistance by developing countries  to
indicators  that  are  developed  and  used  by ’outsiders‘.  New  forms  of  assessments  are increasingly country-led, and  in some cases continent-led such as  the African Peer Review Mechanism (APRM).

“Narrative Research, … lays the foundation for the use of narrative research and inquiry methods not only in the project but broadly in the field of research and consultancy…. Elements of it together with general material on Complexity Theory will be published as a chapter in a book on Naturalising Decision Making in the Fall of 2010.”

“Abstract: This article reports on a research project intended to develop systematic ways of managing downward accountability in an international NGO. Innovative tools were developed and trialled in six countries. The tools comprised a framework, de?ning downward accountability in practical terms, and three management processes. They were successfully used to
(a) encourage staff to improve downward accountability in ways relevant to their context;
(b) hear bene?ciaries’ assessments of the level of accountability achieved and the value of the NGO’s work; and (c) generate quanti?ed performance summaries for managers. Taken together, they form a coherent draft management system. Areas for further research are identied.”

There’s more related material  at www.listenfirst.org,