Tag: New books

Essentials of Utilization-Focused Evaluation

Michael Quinn Patton, August 2011. Sage publications

Publisher’s description:

“Based on Michael Quinn Patton’s best-selling Utilization-Focused Evaluation, this briefer book provides an overall framework and essential checklist steps for designing and conducting evaluations that actually get used. The new material and innovative graphics present the utilization-focused evaluation process as a complex adaptive system, incorporating current understandings about systems thinking and complexity concepts. The book integrates theory and practice, is based on both research and professional experience, and offers new case examples and cartoons with Patton’s signature humor. ”
Continue reading “Essentials of Utilization-Focused Evaluation”

Evaluating the Complex: Attribution, Contribution and Beyond.

Kim Forss, Mita Marra and Robert Schwartz, editors. Transaction Publishers, New Brunswick. May 2011. Available via Amazon

“Problem-solving by policy initiative has come to stay. Overarching policy intiatives are now standard modus operandi for governmental and non-governmental organisations. But complex policy initiatives are not only reserved for the big challenges of our times, but are used for matters such as school achievement, regional development, urban planning, public health and safety. As policy and the ensuing implementation tends to be more complex than simple project and programme management, the task of  evaluation has also become more complex.”

“The book begins with a theoretical and conceptual explanation of complexity and how that affects evaluation. The authors make the distinction between, on the hand, the common-sense understanding of complexity  as something that is generally messy, involves many actors and has unclear boundaries and overlapping roles; and on the hand, complexity as a specific term from systems sciences, which implies non-linear relationships between phenomena. It is particularly in the latter sense that an understanding of complexity has a bearing on evaluation design in respect of how evaluators approach the question of impact.”

“The book presents nine case studies that cover a wide variety of policy initiatives, in public health (smoking prevention), homelessness, child labour, regional development, international development cooperation, the HIV/AIDs pandemic, and international development cooperation. The use of case studies sheds light on the conceptual ideas at work in organisations addressing some of the world’s largest and most varied problems.”

“The evaluation processes described here commonly seek a balance between order and chaos. The interaction of four elements – simplicity, inventiveness, flexibility, and specificity – allows complex platterns to emerge. The case studies illustrate this framework and provide a number of examples of practical management of complexity in light of contingency theories of the evaluation process itself. These theories in turn match the complexity of the evaluated policies, strategies and programmes. The case studies do not pretend to illustrate perfect evaluation processes, the focus is on learning and on seeking patterns that have proved satisfactory and where the evaluation findings have been robust an trustworthy.”

“The contingency theory approach of the book underscores a point also made in the Foreword by Professor Elliot Stern: “In a world characterised by interdependence, emergent proerties, unpredictable change, and indeterminate outcomes, how could evaluation be immune?” The answer lies in the choice of methods as much as in the overall strategy and approach of  evaluation.”

Impact Evaluation in Practice

Paul J. Gertler, Sebastian Martinez, Patrick Premand, Laura B. Rawlings, Christel M. J. Vermeersch, World Bank, 2011

Impact Evaluation in Practice is available as downloadable pdf, and can be bought online.

“Impact Evaluation in Practice presents a non-technical overview of how to design and use impact evaluation to build more effective programs to alleviate poverty and improve people’s lives. Aimed at policymakers, project managers and development practitioners, the book offers experts and non-experts alike a review of why impact evaluations are important and how they are designed and implemented. The goal is to further the ability of policymakers and practitioners to use impact evaluations to help make policy decisions based on evidence of what works the most effectively.

The book is accompanied by a set of training material — including videos and power point presentations — developed for the “Turning Promises to Evidence” workshop series of the Office of the Chief Economist for Human Development. It is a reference and self-learning tool for policy-makers interested in using impact evaluations and was developed to serve as a manual for introductory courses on impact evaluation as well as a teaching resource for trainers in academic and policy circles.

CONTENTS
PART ONE. INTRODUCTION TO IMPACT EVALUATION
Chapter 1. Why Evaluate?
Chapter 2. Determining Evaluation Questions
PART TWO. HOW TO EVALUATE
Chapter 3. Causal Inference and Counterfactuals
Chapter 4. Randomized Selection Methods
Chapter 5. Regression Discontinuity Design
Chapter 6. Difference-in-Differences
Chapter 7. Matching
Chapter 8. Combining Methods
Chapter 9. Evaluating Multifaceted Programs
PART THREE. HOW TO IMPLEMENT AN IMPACT EVALUATION
Chapter 10. Operationalizing the Impact Evaluation Design
Chapter 11. Choosing the Sample
Chapter 12. Collecting Data
Chapter 13. Producing and Disseminating Findings
Chapter 14. Conclusion

Purposeful Program Theory: Effective Use of Theories of Change and Logic Models

by Sue C. Funnell,  Patricia J. Rogers. March 2011. Available on Amazon.

Product Description

“Program Theory in Evaluation Practice is a ground–breaking reference that teaches how to develop an explicit causal model that links an intervention (project, program or policy) with its intended or observed impacts and using this to guide monitoring and evaluation. Peerless in its explanation of why and how to use and develop program theory, the book is rich with examples and alternative approaches. The book is an invaluable resource to faculty and students as well as professionals in professional development programs, education, social work, and counseling. “

From the Back Cover

“Between good intentions and great results lies a program theory—not just a list of tasks but a vision of what needs to happen, and how. Now widely used in government and not–for–profit organizations, program theory provides a coherent picture of how change occurs and how to improve performance. Purposeful Program Theory shows how to develop, represent, and use program theory thoughtfully and strategically to suit your particular situation, drawing on the fifty–year history of program theory and the authors? experiences over more than twenty–five years.

“From needs assessment to intervention design, from implementation to outcomes evaluation, from policy formulation to policy execution and evaluation, program theory is paramount. But until now no book has examined these multiple uses of program theory in a comprehensive, understandable, and integrated way. This promises to be a breakthrough book, valuable to practitioners, program designers, evaluators, policy analysts, funders, and scholars who care about understanding why an intervention works or doesn?t work.” —Michael Quinn Patton, author, Utilization–Focused Evaluation”

“Finally, the definitive guide to evaluation using program theory! Far from the narrow ?one true way? approaches to program theory, this book provides numerous practical options for applying program theory to fulfill different purposes and constraints, and guides the reader through the sound critical thinking required to select from among the options. The tour de force of the history and use of program theory is a truly global view, with examples from around the world and across the full range of content domains. A must–have for any serious evaluator.” —E. Jane Davidson, PhD, Real Evaluation Ltd.

Making Evaluations Matter: a Practical Guide for Evaluators.

Authors: Cecile Kusters with Simone van Vugt, Seerp Wigboldus, Bob Williams and Jim Woodhill. 2011

“Too often evaluations are shelved, with very little being done to bring about change within organisations that requested the evaluation in the first place. This guide will explain how you can make your evaluations more useful. It will help you to better understand some conceptual issues and appreciate how evaluations can contribute to changing mindsets and empowering stakeholders. On a practical level, it presents core guiding principles and pointers on how to design and facilitate evaluations that matter. Furthermore, it shows you how you can get your primary intended users and other key stakeholders to contribute effectively to the evaluation process. This guide is primarily for evaluators working in the international development sector. However, if you are a commissioner of an evaluation, an evaluation manager or a monitoring and evaluation officer, you too will find it useful”.

The book can now be found here as a PDF version, http://www.cdi.wur.nl/UK/resources/Publications/

Social Return On Investment: A practical guide for the development cooperation sector

by Jan Brouwers, Ester Prins and Menno Salverda (2010) , Utrecht, Context International Cooperation / Creative Commons, 60pp, ISBN 978-90-77526-06-4.  Available free online

Description
“Related to Cost Benefit Analyses and Participatory Planning, Monitoring and Evaluation is a new methodology called SROI; Social Return On Investment. As the term implies investments are not made to simply generate financial returns; in fact they create returns in many other areas, including poverty reduction or environmental protection. The SROI process attempts to provide insight in multiple values such that better decisions about investment (development) choices can be made.

The Social Return On Investment (SROI) practical guide, published by Context, international cooperation, provides guidelines and follows nine concrete steps in how to implement a SROI analysis. In this manual, the SROI idea is translated into the practice of international cooperation. What is interesting is that the manual is written in a write shop, where some fifteen SROI practitioners from Africa and Asia have shared and written down their practical experience. This results in a very readable guide with clear examples and practical tips of existing case studies and experiences.

Although SROI conceptualizing and practice should still be considered in its infancy, generally practitioners find that SROI can play an important role in creating awareness and appreciation of different (types of) values. In other words, by using SROI, the landscape of values, previously hidden (or externalised from market values) will change with social, environmental, economic and other values integrated.

Moreover, the communities, the development practitioners and all the related stakeholders, develop a TVC perspective (Total Value Consciousness – Economic, Social & Environmental), which helps in shifting mindsets or perceptions to planning and monitoring of development programs. The focus on social & environmental benefits will motivate the communities to participate in the program actively. This in turn creates a shift in the balance of whether a project is considered beneficial, profitable or commercially viable and consequently would lead to opportunities and implementation of new and innovative initiatives that genuinely contribute to positive social change and poverty reduction for all.

For whom?
For anyone interested in (new) methods for planning, monitoring and evaluation of social change or for those who want to try something different. SROI can be applied at an organisation level, at value chain levels or at the level of a more complex initiative at a community level. SROI can be applied in various fields: rural development, health, economic development, environmental economics, leadership programs, physical infrastructure, value of water infrastructure investment projects, etc. ”

Theory Construction and Model-Building Skills: A Practical Guide for Social Scientists

By James Jaccard PhD , Jacob Jacoby PhD, Guilford Press, 2010. Available on Amazon. Found courtesy of a tweet by EvalCollective.

See also the book review by Brandy Pratt, Western Michigan University, Journal of MultiDisciplinary Evaluation, Volume 6, Number 14, ISSN 1556-8180, August 2010

Amazon book description:  Meeting a crucial need for graduate students and newly minted researchers, this innovative text provides hands-on tools for generating ideas and translating them into formal theories. It is illustrated with numerous practical examples drawn from multiple social science disciplines and research settings. The authors offer clear guidance for defining constructs, thinking through relationships and processes that link constructs, and deriving new theoretical models (or building on existing ones) based on those relationships. Step by step, they show readers how to use causal analysis, mathematical modeling, simulations, and grounded and emergent approaches to theory construction. A chapter on writing about theories contains invaluable advice on crafting effective papers and grant applications.

Useful pedagogical features in every chapter include:

*Application exercises and concept exercises.

*Lists of key terms and engaging topical boxes.

*Annotated suggestions for further reading.”

Research Integration Using Dialogue Methods

David McDonald, Gabriele Bammer, Peter Deane, 2009 Download pdf

Ed: Although about “research integration”  the book is also very relevant to the planning and evaluation of development projects

“Research on real-world problems—like restoration of wetlands, the needs of the elderly, effective disaster response and the future of the airline industry—requires expert knowledge from a range of disciplines, as well as from stakeholders affected by the problem and those in a position to do something about it. This book charts new territory in taking a systematic approach to research integration using dialogue methods to bring together multiple perspectives. It links specific dialogue methods to particular research integration tasks.

Fourteen dialogue methods for research integration are classified into two groups:

1. Dialogue methods for understanding a problem broadly: integrating judgements

2. Dialogue methods for understanding particular aspects of a problem: integrating visions, world views, interests and values.

The methods are illustrated by case studies from four research areas: the environment, public health, security and technological innovation.”

“Monitoring and Evaluating Capacity Building: Is it really that difficult?’

By Nigel Simister with Rachel Smith. Published by INTRAC.

“Whilst few doubt the importance of capacity building, and the need for effective monitoring and evaluation (M&E) to support this work, the M&E of capacity building is as much a challenge now as it was two decades ago. This paper examines both theory and current practice, and aims to promote debate on some of the key barriers to progress.

The paper is primarily concerned with capacity building within civil society organisations (CSOs), although many of the lessons also apply to commercial and state organisations. It is based on a literature review and interviews with capacity building providers in the North and South. Continue reading ““Monitoring and Evaluating Capacity Building: Is it really that difficult?’”

Evaluating Climate Change and Development

From: Juha Uitto , via the MandE NEWS email list

Folks,

I’d like to draw your attention to a new book, ‘Evaluating Climate Change and Development‘, edited by Rob van den Berg and Osvaldo Feinstein. This to my knowledge is the first volume that systematically addresses issues relating to climate change from an evaluation point of view. How do we know that the policies, programs and projects targeted towards mitigation, adaptation and vulnerability actually work? The book reviews evidence and approaches and also contains a wealth of case studies from the developing world.
…end of email…

Amazon Abstract: Climate change has become one of the most important global issues of our time, with far-reaching natural, socio- economic, and political effects. To address climate change and development issues from the perspective of evaluation, an international conference was held in Alexandria, Egypt. This book distills the essence of that timely conference, building on the experiences of more than 400 reports and studies presented. Developing countries may be particularly vulnerable to the expected onslaught of higher temperatures, rising sea levels, changing waterfall patterns, and increasing natural disasters. All societies will have to reduce their vulnerability to these changes, and this book describes how vulnerabilities may be addressed in a systematic manner so that governments and local communities may better understand what is happening. Different approaches are also discussed, including the use of human security as a criterion for evaluation as well as ways to deal with risk and uncertainty. “Evaluating Climate Change and Development” presents a rich variety of methods to assess adaptation through monitoring and evaluation. The volume deals with climate change, development, and evaluation; challenges and lessons learned from evaluations; mitigation of climate change; adaptation to climate change; vulnerability, risks and climate change; and, presents a concluding chapter on the road ahead. Collectively the authors offer a set of approaches and techniques for the monitoring and evaluation of climate change.

Editor PS: See also Evaluating adaptation to climate change from a development perspective: Current state of evaluation of climate change adaptation interventions and next stages‘ by Merylyn Hedger,Tom Mitchell,Jennifer Leavy, Martin Greeley, IDS, 2009. Funded by DFID and GEF

%d bloggers like this: