Category: Books

Working with Assumptions in International Development Program Evaluation

By Nkwake, Apollo M., with a Foreword by Michael Bamberger.  2013, 2013, XXI, 184 p. 14 illus., 7 in color. Published by Springer and available on Amazon

Publisher description

“Provides tools for understanding effective development programming and quality program evaluations Contains workshop materials for graduate students and in-service training for development evaluators The author brings together more than 12 years of experience in evaluation of international development programs

Regardless of geography or goal, development programs and policies are fueled by a complex network of implicit ideas. Stakeholders may hold assumptions about purposes, outcomes, methodology, and the value of project evaluation and evaluators—which may or may not be shared by the evaluators. Even when all participants share goals, failure to recognize and articulate assumptions can impede clarity and derail progress.

Working with Assumptions in International Development Program Evaluation probes their crucial role in planning, and their contributions in driving, global projects involving long-term change. Drawing on his extensive experience in the field, the author offers elegant logic and instructive examples to relate assumptions to the complexities of program design and implementation, particularly in weighing their outcomes. The book emphasizes clarity of purpose, respect among collaborators, and collaboration among team members who might rarely or never meet otherwise. Importantly, the book is a theoretical and practical volume that:

·          Introduces the multiple layers of assumptions on which global interventions are based.

·          Explores various approaches to the evaluation of complex interventions, with their underlying assumptions.

·          Identifies ten basic types of assumptions and their implications for program development and evaluation.

·          Provides examples of assumptions influencing design, implementation, and evaluation of development projects.

·          Offers guidelines in identifying, explicating, and evaluating assumptions

A first-of-its-kind resource, Working with Assumptions in International Development Program Evaluation opens out the processes of planning, implementation, and assessment for professionals in global development, including practitioners, development economists, global development program designers, and nonprofit personnel.”

Rick Davies comment: Looks potentially useful, but VERY expensive at £85.50 Few individuals will buy it but organisations might do so. Ideally the author would make a cheaper paperback version available. And Amazaon should provide a “Look inside this book” option, to help people decide if spending £85.50 would be worthwhile. PS: I think the publishers, and maybe the author, would fail the marshmellow test

Rick Davies postcript: The Foreword, Preface and Contents page of the book is available as a pdf, here on the Springer website.

See also:


What Causes What & Hypothesis testing: Truth and Evidence

Two very useful chapters in Denise Cummins (2012) “Good Thinking“, Cambridge University Press

Cummins is a professor of psychology and philosophy, both of which she brings to bear in this great book. Read an interview with author here

Contents include:

1. Introduction
2. Rational choice: choosing what is most likely to give you what you want
3. Game theory: when you’re not the only one choosing
4. Moral decision-making: how we tell right from wrong
5. The game of logic
6. What causes what?
7. Hypothesis testing: truth and evidence
8. Problem solving: another way of getting what you want
9. Analogy: this is like that.

Integrated Monitoring: A Practical Manual for Organisations That Want to Achieve Results

Written by Sonia Herrero, InProgress, Berlin, April 2012. 43 pages Available as pdf

“The aim of this manual is to help those working in the non-profit sector — non-governmental organisations (NGOs) and other civil society organisations (CSOs) — and the donors which fund them, to observe more accurately what they are achieving through their efforts and to ensure  that they make a positive difference in the lives of the people they want to help. Our interest in writing this guide has grown out of the desire to help bring some conceptual clarity to
the concepts of monitoring and to determine ways in which they can be harnessed and used more effectively by non-profit practitioners.

The goal is to help organisations build monitoring and evaluation into all your project management efforts. We want to demystify the monitoring process and make it as simple and accessible as possible. We have made a conscious choice to avoid technical language, and instead use images and analogies that are easier to grasp. There is a glossary at the end of the manual which contains the definitions of any terms you may be unfamiliar with. This manual is organised into two parts. The first section  covers the ‘what’ and ‘why’ of monitoring and  evaluation; the second addresses how to do it.”

These materials may be freely used and copied by non-profit organisations for capacity building purposes, provided that inProgress and authorship are acknowledged. They may not be reproduced for commercial gain.

Contents
Introduction
I. KEY ASPECTS OF MONITORING
1. What is Monitoring?
2. Why Do We Monitor and For Whom?
3. Who is Involved?
4. How Does it Work?
5. When Do We Monitor?
5. What Do We Monitor?
5.1 Monitoring What We DoII. HOW DO WE MONITOR?
1. Steps for Setting Up a Monitoring S   2. How to Monitor the Process and the Outputs
3. How to Monitor the Achievemen 3.1 Define Results/Outcomes
3.2 Define Indicators for Results
4. Prepare a Detailed Monitoring Plan
5. Identify Sources of Information
6. Data Collection
6.1 Tools for Data Compilation
7. Reflection and Analysis
7.1 Documenting and Sharing
8. Learning and Reviewing
8.1 Learning
8.2 Reviewing
9. Evaluation
Conclusion
Glossary
References

Magenta Book – HM Treasury guidance on evaluation for Central Government (UK)

27 April 2011

“The Magenta Book is HM Treasury guidance on evaluation for Central Government, but will also be useful for all policy makers, including in local government, charities and the voluntary sectors. It sets out the key issues to consider when designing and managing evaluations, and the presentation and interpretation of evaluation results. It describes why thinking about evaluation before and during the policy design phase can help to improve the quality of evaluation results without needing to hinder the policy process.

The book is divided into two parts.

Part A is designed for policy makers. It sets out what evaluation is, and what the benefits of good evaluation are. It explains in simple terms the requirements for good evaluation, and some straightforward steps that policy makers can take to make a good evaluation of their intervention more feasible.

Part B is more technical, and is aimed at analysts and interested policy makers. It discusses in more detail the key steps to follow when planning and undertaking an evaluation and how to answer evaluation research questions using different evaluation research designs. It also discusses approaches to the interpretation and assimilation of evaluation evidence.

The Magenta Book will be supported by a wide range of forthcoming supplementary guidance containing more detailed guidance on particular issues, such as statistical analysis and sampling. Until these are available please refer to the relevant chapters of the original Magenta Book.

The Magenta Book is available for download in PDF format:

An introduction to systematic reviews

Book publishedin March 2012, by Sage. Authors: David Gough, Sandy Oliver, James Thomas

Read Chapter One pdf: Introducing systematic reviews

Contents:

1. Introducing Systematic Reviews David Gough, Sandy Oliver and James Thomas
2. Stakeholder Perspectives and Participation in Reviews Rebecca Rees and Sandy Oliver
3. Commonality and Diversity in Reviews David Gough and James Thomas
4. Getting Started with a Review Sandy Oliver, Kelly Dickson, and Mark Newman
5. Information Management in Reviews Jeff Brunton and James Thomas
6. Finding Relevant Studies Ginny Brunton, Claire Stansfield & James Thomas
7. Describing and Analysing Studies Sandy Oliver and Katy Sutcliffe
8. Quality and Relevance Appraisal Angela Harden and David Gough
9. Synthesis: Combining results systematically and appropriately James Thomas, Angela Harden and Mark Newman
10. Making a Difference with Systematic Reviews Ruth Stewart and Sandy Oliver
11. Moving Forward David Gough, Sandy Oliver and James Thomas

Social Psychology and Evaluation

by Melvin M. Mark PhD (Editor), Stewart I. Donaldson PhD (Editor), Bernadette Campbell PhD (Editor) Guilford Press, May 2011. Available on Google Books.
Book burb “This compelling work brings together leading social psychologists and evaluators to explore the intersection of these two fields and how their theory, practices, and research findings can enhance each other. An ideal professional reference or student text, the book examines how social psychological knowledge can serve as the basis for theory-driven evaluation; facilitate more effective partnerships with stakeholders and policymakers; and help evaluators ask more effective questions about behavior. Also identified are ways in which real-world evaluation findings can identify gaps in social psychological theory and test and improve the validity of social psychological findings–for example, in the areas of cooperation, competition, and intergroup relations. The volume includes a useful glossary of both fields’ terms and offers practical suggestions for fostering cross-fertilization in research, graduate training, and employment opportunities. Each chapter features introductory and concluding comments from the editors.”

Diversity and Complexity

by Scott Page, 2011. Available on Google Books Princeton University Press, 14/07/2011 – 296 pages

Abstract: This book provides an introduction to the role of diversity in complex adaptive systems. A complex system–such as an economy or a tropical ecosystem–consists of interacting adaptive entities that produce dynamic patterns and structures. Diversity plays a different role in a complex system than it does in an equilibrium system, where it often merely produces variation around the mean for performance measures. In complex adaptive systems, diversity makes fundamental contributions to system performance. Scott Page gives a concise primer on how diversity happens, how it is maintained, and how it affects complex systems. He explains how diversity underpins system level robustness, allowing for multiple responses to external shocks and internal adaptations; how it provides the seeds for large events by creating outliers that fuel tipping points; and how it drives novelty and innovation. Page looks at the different kinds of diversity–variations within and across types, and distinct community compositions and interaction structures–and covers the evolution of diversity within complex systems and the factors that determine the amount of maintained diversity within a system.Provides a concise and accessible introduction. Shows how diversity underpins robustness and fuels tipping points .Covers all types of diversity. The essential primer on diversity in complex adaptive systems.

RD Comment: This book is very useful for thinking about the measurement of diversity. In 2000 I wrote a paper “Does Empowerment Start At Home? And If So, How Will We Recognise It?” in which I argued that…

“At the population level, diversity of behaviour can be seen as a gross indicator of agency (of the ability to make choices), relative to homogenous behaviour by the same set of people. Diversity of behaviour suggests there is a range of possibilities which individuals can pursue. At the other extreme is standardisation of behaviour, which we often associate with limited choice. The most notable example being perhaps that of an army. An army is a highly organised structure where individuality is not encouraged, and where standardised and predictable behaviour is very important. Like the term “NGO” or “non-profit”, diversity is defined by something that it is not –  a condition where there is no common constraint, which would otherwise lead to a homogeneity of response. Homogeneity of behaviour may arise from various sources of constraint. A flood may force all farmers in a large area to move their animals to the high ground. Everybody’s responses are the same, when compared to what they would be doing on normal day. At a certain time of the year all farmers may be planting the same crop. Here homogeneity of practice may reflect common constraints arising from a combination of sources: the nature of the physical environment, and the nature of particular local economies. Constraints on diversity can also arise within the assisting organisation. Credit programs can impose rules on loan use, specific repayment schedules and loan terms, as well as limiting when access to credit is available, or how quickly approval will be give.”

See also…

%d bloggers like this: