International Course on Participatory Planning, Monitoring and Evaluation – Navigating and Managing for Impact

Date: 1 Mar 2010 – 19 Mar 2010
Venue: Wageningen, Netherlands

Dear colleagues,

I would like to refer you to the succesful course that we are running since 2002 and which this year had 3 parallel courses due to the high number of applications!

This course is organised by Wageningen International, part of Wageningen University and Research Centre. This course focuses on how to manage for impact by integrating strategic guidance, operational management, monitoring and evaluation in a learning environment, whilst navigating the external and internal context. Particular attention is given to designing and institutionalising participatory planning and M&E systems in development initiatives and organisations for continuous learning and enhancing performance. Attention also is paid to the relationship between management information needs and responsibilities and the planning and M&E functions. For more info please visit our website: http://www.cdic.wur.nl/UK/newsagenda/agenda/Participatory_planning_monitoring_and_evaluation.htm or contact us: training.wi@wur.nl or cecile.kusters@wur.nl

If your are interested to receive scholarship – the deadline is 1st September so please be fast!

Kind regards / Hartelijke groeten,

Cecile Kusters
Participatory Planning, Monitoring & Evaluation
Multi-Stakeholder Processes and Social Learning
Wageningen UR, Wageningen International
P.O.Box 88, 6700 AB Wageningen, the Netherlands
Visiting address: Lawickse Allee 11,Building 425, 6701 AN Wageningen, The Netherlands
Tel.  +31 (0)317- 481407
Fax. +31 (0)317- 486801
e-mail:  cecile.kusters@wur.nl
Website: www.cdic.wur.nl/UK
PPME resource portal: http://portals.wi.wur.nl/ppme/
MSP resource portal: http://portals.wi.wur.nl/msp/
www.disclaimer-uk.wur.nl

Evaluation of Conflict Sensibility, Conflict Prevention and Peace Building Programmes

Date: 5-8 October 2009
Venue: Belgium

This annual course is an intermediate- to advanced level course based on the newest guidelines established by the OECD-DAC. It provides methodologies for carrying out assessments of conflict sensibility, conflict situations and, subsequently, evaluating the performance of peace-building and conflict prevention activities in a seminar format with focus on methods and challenges. The course is intended for those with experience in evaluations, and an interest in, and general experience of, conflict situations.

Based on Channel Research’s experience of running training programmes on evaluation, the participants in previous years have come from aid agencies (headquarters and field personnel), donor governments, consultancies and academia. This 4 days (5 nights) course is facilitated by Emery Brusset, Director of Channel Research, Tony Vaux, an expert on conflicts and Koenraad Denayer, expert in conflict sensibility and will take place at Orshof (www.orshof.be) near Brussels.

Please find attached the course outline and application form or on the link: http://www.channelresearch.com/peace-building/evaluation-of-peace-building. For any further information, please contact Maria Bak on bak@channelresearch.com.

You can find more information about Channel Research and our trainings on: www.channelresearch.com

Utilization-focused evaluation for agricultural innovation

Michael Quinn Patton and Douglas Horton
ILAC Brief No 22

Utilization-focused evaluation (UFE) is based on the principle that an evaluation should be judged by its utility. So no matter how technically sound and methodologically elegant, an evaluation is not truly a good evaluation unless the findings are used. UFE is a framework for enhancing the likelihood that evaluation findings will be used and lessons will be learnt from the evaluation process. This Brief, based on the book Utilization-focused evaluation, introduces this approach to evaluation, outlines key steps in the evaluation process, identifies some of the main benefits of UFE, and provides two examples of UFE in the context of programmes aimed at promoting agricultural innovation.

ALNAP 8th Review of Humanitarian Action

The ALNAP Review of Humanitarian Action series aims to advance analysis and understanding of key trends and issues relating to humanitarian learning and accountability as a means of supporting improvement in sector-wide performance. The 8th Review contains three in-depth studies:

Chapter 1: Counting what counts: performance and effectiveness in the humanitarian sector [http://www.alnap.org/pool/files/8rhach1.pdf

Chapter 2: Improving humanitarian impact assessment: bridging theory and practice [http://www.alnap.org/pool/files/8rhach2.pdf]

Chapter 3: Innovations in International humanitarian action [http://www.alnap.org/pool/files/8rhach3.pdf

The first study is on humanitarian performance and provides a wide-ranging overview of the performance agenda – at the heart of ALNAP’s work – drawing on experiences from the private, public and development sectors. The second study focuses on improving humanitarian impact assessment, and provides a comprehensive framework to help bridge theory and practice in operational settings. The third study is a systematic review of innovations in international humanitarian response, which presents ways to think about and strengthen innovations across the sector.

Key Messages from ALNAP’s Eighth Review of Humanitarian Action: [http://www.alnap.org/pool/files/8rhakm-eng.pdf
Key messages in French and Spanish will be available shortly.

Pan African Monitoring and Evaluation Conference

Date: 27 – 31 July 2009
Venue: Premier Hotel, Pretoria, Gauteng, South Africa

The leaders of Africa continue to grapple with service delivery and are looking for ways to improve their capabilities and help them to achieve tangible and sustainable results.

“Now, more than ever, governments are being held accountable to their constituents for their expenditure,” explains Hennie Oosthuizen, CEO of the African Information Institute. “It is prudent for Africa’s leaders to embrace monitoring and evaluation in order for them to accurately assess the quality and impact of their work against their strategic plan.”

Monitoring and Evaluation (M&E) is a public management tool used to improve the way that government and other organizations achieve results. South African President, Jacob Zuma, has prioritised M&E through the establishment of an evaluation, monitoring and planning commission within the presidency, as well as in all government departments from national down to local level.
Continue reading “Pan African Monitoring and Evaluation Conference”

Assessing aid impact: a review of Norwegian evaluation practice

Authors: Espen Villanger ; Alf Morten Jerve
Published in: Journal of Development Effectiveness, Volume 1, Issue 2

June 2009 , pages 171 – 194

Warning: Unfortunately you have to pay to get access to this article in full

Abstract

This article reviews recent Norwegian aid evaluations with a mandate to study impact, and assesses how the evaluators establish causal effects. The analytical challenges encountered in the seven studies reviewed are: (1) the Terms of Reference ask for evidence of impact where this is not possible to identify, (2) the distinction between impacts of the aid element versus other components is often blurred, and (3) the methodological approaches to identify impact are either poorly developed or applied superficially. A main conclusion is that most of the evaluators did not have the necessary time or budget to conduct a proper impact evaluation given the large number of questions raised in the commissioning agency.

view references (26)

NAO Review – DFID: Progress in improving performance management

Publication date: 12 May 2009 . Full report (PDF – 366KB)

Executive Summary

1.  This brief review of the Department for International Development’s (DFID) performance management arrangements during 2008 is a follow-up to our 2002 VFM report on the same topic. It responds to a request from DFID’s Accounting Officer to re-visit the topic periodically, which the C&AG agreed would be valuable. It is based on a desk review of main documents, interviews with DFID staff, and a survey of staff and stakeholder views about evaluation (Appendix 3). We did not undertake a full audit of DFID systems, some of which are in the process of being revised, and we concentrated on those areas of DFID activity most directly related to its performance targets. We drew on recent DFID reviews of monitoring and evaluation, and our findings square well with the results of those reviews.
Continue reading “NAO Review – DFID: Progress in improving performance management”

Learning purposefully in capacity  development 

Why, what and when to measure?

An opinion paper prepared for IIEP,  by Alfredo Ortiz and Peter Taylor, INSTITUTE OF DEVELOPMENT STUDIES (IDS), 25th July, 2008

>>Full text<<

Abstract

Many capacity development (CD) programs and processes aim at long?term sustainable change,
which depends on seeing many smaller changes in at times almost invisible fields (rules, incentives,
behaviours, power, coordination etc.). Yet, most evaluation processes of CD tend to focus on short?
term outputs focused on clearly visible changes. This opinion paper will offer some ideas on how to
deal with this paradox, by examining how monitoring and evaluation (M&E) does, or could, make a difference to CD.  It explores whether there is something different and unique about M&E of CD that
isn’t addressed by predominant methods and ways of thinking about M&E, and which might be
better addressed by experimenting with learning?based approaches to M&E of CD.

Contents
1.  INTRODUCTION—WHAT SHOULD MONITORING &EVALUATION (M&E) TELL US ABOUT
CAPACITY DEVELOPMENT (CD)? …………………………… 1
2.  CAPACITY DEVELOPMENT MEANS AND ENDS—“WHAT ARE WE MEASURING AND WHEN
SHOULD WE MEASURE IT?”  …. 5
2.1.  IN SEARCH OF PERFORMANCE AND IMPACT .. 5
2.2.  STANDING CAPACITY  …. 10
3.  WHAT CAN WE LEARN FROM M & E OF CD DILEMMAS? … 13
3.1.  DEVELOPMENT BEING A PROCESS ALREADY IN MOTION …….. 13
3.2.  LINEAR VERSUS COMPLEX ADAPTIVE SYSTEMS (CAS) THINKING, PROGRAMMING AND MEASUREMENT ……. 14
3.3.  ATTRIBUTION … 17
3.4.  DONOR ACCOUNTING FOCUS VERSUS OPEN LEARNING APPROACHES .. 18
4.  CONCLUDING THOUGHTS  … 24
4.1.  INCORPORATION OF ORGANIZATIONAL LEARNING APPROACHES TO M&E OF CD … 26
4.2.  LARGE-SCALE EXPERIMENTATION AND ACTION RESEARCH . 27
4.3.  USE OF THEORY OF CHANGE (TOC) APPROACHES FOR DESIGNING M&E OF CD SYSTEMS ……… 28
4.3.1.  WHAT CAN A THEORY OF CHANGE OFFER? . 28
4.3.2.  HYPOTHETICAL EXAMPLE OF TOC USE IN EFA ….. 31
4.4.  CONCLUSION ……………………………………. 33
5.  ACRONYMS …………………………………… 34
6.  BIBLIOGRAPHY …………………………………… 35

Guidance on using the revised Logical Framework (DFID 2009)

Produced by the Value for Money Department, FCPD, February 2009.

>>Full text here<<

“The principal changes to the logframe from the earlier (2008) 4×4 matrix are:
•  The Objectively Verifiable Indicator  (OVI) box has been separated into its
component elements (Indicator, Baseline and Target), and Milestones added.
•  Means of Verification has been separated into ‘Source’.
•  Inputs are now quantified  in terms for funds (expressed in Sterling for DFID
and all partners) and in use of DFID staff time  (expressed as Full-Time
Equivalents (FTEs);
•  A Share box now indicates the financial value of DFID’s Inputs as a
percentage of the whole.
•  Assumptions are shown for Goal and Purpose only;
•  Risks are shown at Output and Activities level only;
•  At the Output level,  the Impact Weighting is now shown in the logframe
together with a Risk Rating for individual Outputs
•  Activities are now shown separately (so do not normally appear in the
logframe sent for approval), although  they can be added to the logframe
template if this is more suitable for your purposes.
•  Renewed emphasis on the use of disaggregated beneficiary data within
indicators, baselines and targets.”

BOND Quality Group – Debate on logframes

Date: 2-5.30pm 11th June 2009
Venue: NCVO offices, N1 9RL, London

For more information contact: Alex Jacobs <alex@keystoneaccountability.org>

Motion: this meeting believes that the logframe is the right tool for managing most NGO work

Logframes (Logical Framework Analysis) are very widely used in NGOs. But they split opinion sharply throughout the sector: some people love them, some hate them.

To their supporters, logframes provide a simple short way of summarising a project’s aims and activities. They force staff to map out the intermediary steps that link activities and overall goals. They can be applied at any level, from an entire organisation to one specific project. They help managers and donors alike by providing a guide to action and a set of indicators to monitor progress, which be can conveniently communicated to other people. Many different approaches can be used to create logframes, including participatory methods.

To their detractors, logframes force staff to think in an inappropriate way. They assume that complex social systems can be predicted in advance and that social problems reduced to a single problem statement. They do not take account of different people’s views and priorities (e.g. within communities), and they are based on an inappropriate linear logic (if A happens, then B will happen, then C). In practice, they are inflexible, creating a strait-jacket for relationships with partners and communities, which undermines outsiders’ ability to respond effectively to changing realities on the ground. They create bureaucratic paperwork, and are most useful for donors and senior managers.

What are the arguments and evidence for each side of the debate? Come along, listen to some expert opinion, debate the issues with your peers.

Speakers:

  • Proposing: Peter Kerby (DFID) & Claire Thomas (Minority Rights International)
  • Opposing: Robert Chambers (IDS) & Rick Davies (independent)

Presentations made by:

Voting Results (before and after debate)

Table 1: Votes before the debate
For Against Abstain Total
Women 9 14 1 24
38% 58% 4%
Men 3 5 1 9
33% 56% 11%
Total 12 19 2 33
36% 58% 6%
For Against Abstain Total
Large org 6 4 10
60% 40%
Small org 1 13 14
7% 93%
Total 7 17 24
29% 71%
Table 2: Votes after the debate
For Against Abstain Total
Women 6 13 1 20
30% 65% 5%
Men 2 4 1 7
29% 57% 14%
Total 8 17 2 27
30% 63% 7%
For Against Abstain Total
Large org 2 5 7
29% 71% 0%
Small org 2 11 13
15% 85% 0%
Total 4 16 0 20
20% 80% 0%

See also the summary of the BOND logframe debate, available at the BOND website