Expert seminar with Dr MQ Patton ‘Developmental evaluation’

Date: 29th March 2011
Venue: in the Netherlands

NOW CANCELLED. Further information will be provided when available

With pleasure we would like to announce an expert seminar with Dr. Michael Quinn Patton on ‘Developmental evaluation – new kid on the evaluation block’.

Developmental evaluation is based on insights from complex dynamic systems, uncertainty, nonlinearity, and emergence. World renowned, award-winning evaluation expert Dr. Michael Quinn Patton will discuss the developmental evaluation framework as detailed in his book `Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use’. Patton will illustrate how developmental evaluation can be used for a range of purposes: ongoing program development; adapting effective principles of practice to local contexts; generating innovations and taking them to scale; and facilitating rapid response in crisis situations.

Participant discussions will focus on development evaluation’s value for the development sector. How is developmental evaluation different from existing practice, ongoing reflective monitoring – in development? What new insights can help strengthen the utility of evaluation for development?

During the morning, Dr. Patton will explain developmental evaluation and illustrate it with many examples from his own experience. In the afternoon, participants will debate the practical application of developmental evaluation in development, based on participants’ existing evaluation questions.

For those interested and not to far away around that time, please do come and join this interesting event!

For more info and registration: http://www.cdi.wur.nl/UK/newsagenda/agenda/DevelopmentalEvaluation_MichaelPatton
We suggest to make an early hotel booking (http://www.hofvanwageningen.nl/?language=en) as the hotel is already quite full. Also indicate to the hotel that you are booking a room for the ‘expert seminar with Patton’.

Looking forward to meeting you here!

Cecile Kusters (CDI), Irene Guijt (Learning by Design), Jan Brouwers (Context, international cooperation) and Karel Chambille (Hivos)
Kind regards / Hartelijke groeten,

Cecile Kusters
Participatory Planning, Monitoring & Evaluation – Managing for Impact
Multi-Stakeholder Processes and Social Learning
Centre for Development Innovation
Wageningen UR
P.O. Box 88, 6700 AB Wageningen, The Netherlands
Tel. +31 (0)317 481407 (direct), +31 (0)317 486800 (reception)
Fax +31 (0)317 486801
e-mail cecile.kusters@wur.nl
www.cdi.wur.nl
PPME resource portal: http://portals.wi.wur.nl/ppme/
MSP resource portal: http://portals.wi.wur.nl/msp/
www.disclaimer-uk.wur.nl

Seminar: Complexity-oriented Planning, Monitoring and Evaluation (PME): from alternative to mainstream?

Date: Wednesday 10 November 2010, 13.30-17.00 pm,
Venue: Theatre Concordia, Hoge Zand 42, The Hague, The Netherlands

The HIVA Research Institute for Work and Society of the Katholieke Universiteit Leuven, PSO Capacity Building in Developing Countries, the Flemish Office for Development Cooperation and Technical Assistance (VVOB), and Vredeseilanden/VECO invite you to a seminar organised within the framework of the Development Policy Review Network (DPRN) to discuss how alternative PME approaches such as Outcome Mapping and Most Significant Change may complement the mainstream Logical Framework approach for complex development programmes. The aim is to come up with recommendations for PME policies, based on lessons learned from practical experience with various PME approaches in complex situations.

The seminar forms the closing part of the DPRN ‘Planning, Monitoring and Evaluation in complex social situations’ process, in which the four organisations have worked on various studies about these PME approaches. They analysed the different PME approaches in detail and carried out various case studies of PME approaches which featured as learning histories for various organisations. In addition, the organisers reviewed current PME policy frameworks in Belgium and the Netherlands and organised a public online discussion about the use of Logical Framework versus Outcome Mapping. The insights of these four projects will be shared and discussed at the seminar. For more details see the attached invitation or contact Jan van Ongevalle of HIVA at Jan.VanOngevalle@hiva.kuleuven.be (Jan.VanOngevalle@hiva.kuleuven.be). More information on the process can be found on the website http://pme.global-connections.nl/.

You can register by sending an email to aanmelding@pso.nl (aanmelding@pso.nl) before 1 November, mentioning ‘DPRN seminar 10 November’

Narrative Research

David Snowden, 2010.  21 pages. Available as pdf, from the Cognitive Edge site

“Narrative Research, … lays the foundation for the use of narrative research and inquiry methods not only in the project but broadly in the field of research and consultancy…. Elements of it together with general material on Complexity Theory will be published as a chapter in a book on Naturalising Decision Making in the Fall of 2010.”

Beyond Logframe: Using Systems Concepts in Evaluation

March 2010. Nobuko Fujita (Ed)  Foundation for Advanced Studies on International Development (FASID) Available as pdf

“Editor’s Note: The 2010 Issues and Prospects of Evaluations for International Development employs systems concepts as clues to re-assess the conventional ways of conducting evaluations and to explore how development evaluation can potentially be made more useful.

In Japan, development evaluation predominantly relies on the Logical Framework (logframe) when conducting evaluations. Evaluations based on a logframe often face difficulties. One such difficulty arises from the futile attempt to develop an evaluation framework based on a logframe, which, in many cases, was prepared as part of the early-stage planning of the project and which then does not necessarily reflect a project’s real situation at the time of evaluation. Although a logframe can be utilised initially as a tentative project plan, logframes are rarely revised even when the situation has changed. By the end of the project, the original logframe may not be an accurate embodiment of what the project is about and therefore logframes do not particularly help in terminal or ex-post evaluations.

Still, having been institutionalized by clients, logframe-based evaluations are common practice and in extreme cases, evaluators face the danger of evaluating the logframe instead of the actual project. Although widely used for its simplicity, logframes can end up becoming a cumbersome tool, or even a hindrance to evaluation.

Various attempts have been made to overcome the limitations of the logframe and some aid organizations such as USAID, UNDP, CIDA and the World Bank have shifted from the logframe to Results-Based Management (RBM). Now GTZ  is in the process of shifting to a new project management approach designed on RBM and systems ideas.

In the first article, “Beyond logframe: Critique, Variations and Alternatives,” Richard Hummelbrunner, an evaluator/consultant from Austria, sums up the critique of logframe and the Logical Framework Approach (LFA), and explores some variations employed to overcome specific shortcomings of LFA. He then outlines a systemic alternative to logframe  and introduces the new GTZ management model for sustainable development called “Capacity WORKS.” Richard has dealt with LFA and possible alternatives to LFA at various points along his career, and he is currently involved in GTZ’s rollout of Capacity WORKS as it becomes the standard management model for all BMZ 5 projects and programmes.

What does he mean by “systemic alternative”? In the second article, “Systems Thinking and Capacity Development in the International Arena,” Bob Williams, a consultant and an expert in systems concepts, explains what “thinking systemically” is about and how it might help evaluation. He boils down systems ideas into three core concepts (inter-relationships, perspectives, and boundaries), and relates these concepts to various systems methods.

In December 2009, FASID offered a training course and a seminar on this topic in Tokyo. Through the exchange of numerous e-mails with the instructors prior to the seminar, it occurred to me that the concepts might be more easily understood presented as a conversation. That is what we tried to do in the third article, “Using Systems Concepts in Evaluation – A Dialogue with Patricia Rogers and Bob Williams –.” These two instructors of the FASID training course and workshop explain in simple conversational style where and how we can start applying systems concepts in development evaluation.

This issue also carries a report of two collaborative evaluations of Japanese Official Development Assistance (ODA) projects. The first case presents an innovative joint evaluation conducted collaboratively with Vietnamese stakeholders. The evaluation took place in 2009 – 2010 as the last year of a three-year evaluation capacity development project coordinated by the Japan International Cooperation Agency. The second case covers a joint evaluation study of another Japanese ODA project in Lao PDR with a local Lao administration for which neither logframe nor OECD DAC five criteria was used. Instead, an evaluation framework was developed from scratch, based entirely on the beneficiaries’ interests and perspectives. In both cases, a partner country’s participation in the evaluation necessitated considerable changes in perspectives of evaluation practice. I hope they provide examples of how boundaries and perspectives, as discussed theoretically in the first three articles, relate to development evaluation in practice.”

Meetings on evaluation and complexity

(Via Ben Ramalingam)

“The latest post on Aid on the Edge post compiles presentations and reports from meetings on evaluation and complexity (NORAD, Panos, Mokoro) that have taken place in the past few months and years, as well as planned meetings later this year”

The list:

Conference: “Evaluation Revisited: Improving the Quality of Evaluative Practice by Embracing Complexity.”

PS: Videos of the event are now available:  video_1.wmv video_2.wmv video_3.wmv . The EvaluationRevisited conference website also has a number of interesting post conference comments by different participants.

Date: 20-21 May 2010
Venue: Netherlands

On May 20-21, 2010, a conference on evaluation for development will be held in The Netherlands: “Evaluation Revisited: Improving the Quality of Evaluative Practice by Embracing Complexity.”

This conference focuses on how evaluative practice can be improved, given the need to view much of development as a process of social transformation and, therefore, complex. Current evaluation practice has not yet embraced the full implications of assessing ‘the complex’ and existing approaches often fall woefully short. During the conference, participants can explore concrete evaluation practices that reconcile an understanding of complex societal change processes with quality standards, including rigorous, ethical concerns, appropriateness and feasibility. Continue reading “Conference: “Evaluation Revisited: Improving the Quality of Evaluative Practice by Embracing Complexity.””

Learning purposefully in capacity  development 

Why, what and when to measure?

An opinion paper prepared for IIEP,  by Alfredo Ortiz and Peter Taylor, INSTITUTE OF DEVELOPMENT STUDIES (IDS), 25th July, 2008

>>Full text<<

Abstract

Many capacity development (CD) programs and processes aim at long?term sustainable change,
which depends on seeing many smaller changes in at times almost invisible fields (rules, incentives,
behaviours, power, coordination etc.). Yet, most evaluation processes of CD tend to focus on short?
term outputs focused on clearly visible changes. This opinion paper will offer some ideas on how to
deal with this paradox, by examining how monitoring and evaluation (M&E) does, or could, make a difference to CD.  It explores whether there is something different and unique about M&E of CD that
isn’t addressed by predominant methods and ways of thinking about M&E, and which might be
better addressed by experimenting with learning?based approaches to M&E of CD.

Contents
1.  INTRODUCTION—WHAT SHOULD MONITORING &EVALUATION (M&E) TELL US ABOUT
CAPACITY DEVELOPMENT (CD)? …………………………… 1
2.  CAPACITY DEVELOPMENT MEANS AND ENDS—“WHAT ARE WE MEASURING AND WHEN
SHOULD WE MEASURE IT?”  …. 5
2.1.  IN SEARCH OF PERFORMANCE AND IMPACT .. 5
2.2.  STANDING CAPACITY  …. 10
3.  WHAT CAN WE LEARN FROM M & E OF CD DILEMMAS? … 13
3.1.  DEVELOPMENT BEING A PROCESS ALREADY IN MOTION …….. 13
3.2.  LINEAR VERSUS COMPLEX ADAPTIVE SYSTEMS (CAS) THINKING, PROGRAMMING AND MEASUREMENT ……. 14
3.3.  ATTRIBUTION … 17
3.4.  DONOR ACCOUNTING FOCUS VERSUS OPEN LEARNING APPROACHES .. 18
4.  CONCLUDING THOUGHTS  … 24
4.1.  INCORPORATION OF ORGANIZATIONAL LEARNING APPROACHES TO M&E OF CD … 26
4.2.  LARGE-SCALE EXPERIMENTATION AND ACTION RESEARCH . 27
4.3.  USE OF THEORY OF CHANGE (TOC) APPROACHES FOR DESIGNING M&E OF CD SYSTEMS ……… 28
4.3.1.  WHAT CAN A THEORY OF CHANGE OFFER? . 28
4.3.2.  HYPOTHETICAL EXAMPLE OF TOC USE IN EFA ….. 31
4.4.  CONCLUSION ……………………………………. 33
5.  ACRONYMS …………………………………… 34
6.  BIBLIOGRAPHY …………………………………… 35

Complexity in Aid Workshop series: Strategy in a complex world

Date: January 14, 2009, all day (9-5)
Venue: CAFOD offices in London (Stockwell).

The world is becoming increasingly inter-related, complex and fast-changing and yet many organisations continue to use traditional methods for strategy development, organisation change and leadership – even when they have questionable success. Why is this? What has to happen for strategists and policy makers to give up on behaving as if the world is predictable, measurable controllable? And what should be done instead?

In this latest workshop of the emerging community on “Complexity in Aid” we will review the paradox of complexity and see what it means for organisational and strategic approaches; we will consider how to get people engaged in these ideas and what complexity thinking implies for practice.

The workshop will be led by Dr Jean Boulton. Jean has a PhD in physics and designed and led the teaching on complexity for several years at Cranfield School of Management; she now teaches complexity on the MSc in Responsible Business Practice at Bath School of Management and works
with organisations in the areas of strategy and organisation change; she is currently co-authoring a book, ‘Embracing Complexity’, with Professor Peter Allen, to be published by Oxford University Press in 2009. See www.embracingcomplexity.co.uk

There will be plenty of opportunity during the workshop for discussion and consideration of how these ideas challenge current methods of strategic planning and implementation; we will look at the balance between the formal and informal, the espoused and the actual.
With this is mind, you might like to consider the following questions:
*       How is strategy developed in your organisation? To what extent does it shape practice? How do you know?
*       What ways, formal or informal, global or local, really impact on what actually is done by your organisation? What, in practice, has most influence on the  direction the organisation travels?

Places are limited. If you would like to attend please email
learning@cafod.org.uk

Event: Navigating Complexity – An innovation dialogue

Date: May 26-27, 2008
Venue: Wageningen International, Netherlands.

[Ten presentations given at this event are now available online]

What does complexity thinking mean for development interventions? This innovation dialogue offers a unique opportunity for exploring how emerging insights from the complexity sciences and systems thinking, combined with field practice, could reshape assumptions about the design, monitoring and evaluation of development work. What does it mean to shift from compliance with external standards to investing in capacities for navigating complexity? Continue reading “Event: Navigating Complexity – An innovation dialogue”

ODI Seminar: Exploring the Science of Complexity in Aid Policy and Practice

Date: July 9th 2008
Venue: ODI, London

Dear All,

I am very pleased to be able to share with you all the agenda for the forthcoming Seminar at ODI on “Exploring the Science of Complexity in Aid Policy and Practice” (see below). [July 9th, ODI, London] The aim of this first Seminar is to bring together practitioners and researchers to discuss the key ideas of complexity thinking, and the relevance they have for those working in the international aid sector, and to identify some concrete and practical ways to take this area of work forward, both collectively and within our individual organisations.
Continue reading “ODI Seminar: Exploring the Science of Complexity in Aid Policy and Practice”