Why, what and when to measure?
An opinion paper prepared for IIEP, by Alfredo Ortiz and Peter Taylor, INSTITUTE OF DEVELOPMENT STUDIES (IDS), 25th July, 2008
>>Full text<<
Abstract
Many capacity development (CD) programs and processes aim at long?term sustainable change,
which depends on seeing many smaller changes in at times almost invisible fields (rules, incentives,
behaviours, power, coordination etc.). Yet, most evaluation processes of CD tend to focus on short?
term outputs focused on clearly visible changes. This opinion paper will offer some ideas on how to
deal with this paradox, by examining how monitoring and evaluation (M&E) does, or could, make a difference to CD. It explores whether there is something different and unique about M&E of CD that
isn’t addressed by predominant methods and ways of thinking about M&E, and which might be
better addressed by experimenting with learning?based approaches to M&E of CD.
Contents
1. INTRODUCTION—WHAT SHOULD MONITORING &EVALUATION (M&E) TELL US ABOUT
CAPACITY DEVELOPMENT (CD)? …………………………… 1
2. CAPACITY DEVELOPMENT MEANS AND ENDS—“WHAT ARE WE MEASURING AND WHEN
SHOULD WE MEASURE IT?” …. 5
2.1. IN SEARCH OF PERFORMANCE AND IMPACT .. 5
2.2. STANDING CAPACITY …. 10
3. WHAT CAN WE LEARN FROM M & E OF CD DILEMMAS? … 13
3.1. DEVELOPMENT BEING A PROCESS ALREADY IN MOTION …….. 13
3.2. LINEAR VERSUS COMPLEX ADAPTIVE SYSTEMS (CAS) THINKING, PROGRAMMING AND MEASUREMENT ……. 14
3.3. ATTRIBUTION … 17
3.4. DONOR ACCOUNTING FOCUS VERSUS OPEN LEARNING APPROACHES .. 18
4. CONCLUDING THOUGHTS … 24
4.1. INCORPORATION OF ORGANIZATIONAL LEARNING APPROACHES TO M&E OF CD … 26
4.2. LARGE-SCALE EXPERIMENTATION AND ACTION RESEARCH . 27
4.3. USE OF THEORY OF CHANGE (TOC) APPROACHES FOR DESIGNING M&E OF CD SYSTEMS ……… 28
4.3.1. WHAT CAN A THEORY OF CHANGE OFFER? . 28
4.3.2. HYPOTHETICAL EXAMPLE OF TOC USE IN EFA ….. 31
4.4. CONCLUSION ……………………………………. 33
5. ACRONYMS …………………………………… 34
6. BIBLIOGRAPHY …………………………………… 35
