The Evaluation of the Paris Declaration: Phase II Report

“After the landmark international Paris Declaration on Aid Effectiveness, endorsed in 2005, what have been the improvements in the quality of global aid and its effects on development? And how can a political declaration, implemented across widely varying national contexts, be robustly evaluated?

An independent global evaluation – a fully joint study involving countries and donor agencies – has assessed these efforts to improve the effectiveness of international aid, especially since 2005. The evaluation is the largest aid evaluation ever conducted. It has been a major international effort in itself, comprising more than 50 studies in 22 partner countries and across 18 donor agencies, as well as several studies on special themes, over a period of four years. It has broken some new boundaries in the study of complex evaluation objects.

The study had the unusual object of a political declaration, implemented across very diverse national environments to varied timescales. Its methodology rejected a traditional linear approach to cause and effect, emphasising instead the importance of context. It opted to draw out the programme theory of the Declaration itself (and that of the Accra Agenda for Action) and their statement of intent. Recognising the limits of aid in development, it applied contribution analysis to assess whether and how the commitments, actors and incentives brought together by the Declaration and the Accra Agenda have delivered on their intentions. The methodology traces the logic of how the Declaration is supposed to work and illustrates the complex pathways from development objectives to results. Recognising that development is a journey, it focuses on assessing the direction of travel on each key point, and the pace and distance travelled so far.

The study concludes that the global campaign to make international aid programmes more effective is showing results, giving the best hope in half a century that aid can be better used to help developing countries raise their economic and living standards. Improvements are slow and uneven in most developing countries, however, and even more so among donor countries and aid agencies. The Evaluation report, all the component studies and the Technical Annex – which describes the methodology and process – can be found at www.busanhlf4.org and www.oecd.org/dac/evaluationnetwork/pde. The second phase of the study was managed by UK-based development consultancy IOD PARC.”

For more information email // julia@iodparc.com  IOD PARC, 16-26 Forth Street. Edinburgh EH1 3LH

RD comment: Of additional interest ” Given the importance of the Evaluation of the Paris Declaration, the Management Group commissioned an independent assessment – a meta evaluation – of the evaluation process and outcome to determine whether the evaluation meets generally accepted standards of quality and to identify strengths, weaknesses, and lessons. The Report, by Michael Quinn Patton and Jean Gornick, can be downloaded here” “Evaluation of the Phase 2 Evaluation of the Paris Declaration”

 

Theory-Based Stakeholder Evaluation

Morten Balle Hansen and Evert Vedung. American Journal of Evaluation
31(3) 295-313, 2010. Available as pdf

Abstract
“This article introduces a new approach to program theory evaluation called theory-based stakeholder evaluation or the TSE model for short. Most theory-based approaches are program theory driven and some are stakeholder oriented as well. Practically, all of the latter fuse the program perceptions of the various stakeholder groups into one unitary program theory. The TSE model  keeps the program theories of the diverse stakeholder groups apart from each other and from the program theory embedded in the institutionalized intervention itself. This represents, the authors argue, an important clarification and extension of the standard theory-based evaluation. The TSE model is elaborated to enhance  theory-based evaluation of interventions characterized by conflicts and competing program theories. The authors argue that especially in evaluations of complex and complicated multilevel and multisite interventions, the presence of competing theories is likely and the TSE model may prove useful.”

Purposeful Program Theory: Effective Use of Theories of Change and Logic Models

by Sue C. Funnell,  Patricia J. Rogers. March 2011. Available on Amazon.

Product Description

“Program Theory in Evaluation Practice is a ground–breaking reference that teaches how to develop an explicit causal model that links an intervention (project, program or policy) with its intended or observed impacts and using this to guide monitoring and evaluation. Peerless in its explanation of why and how to use and develop program theory, the book is rich with examples and alternative approaches. The book is an invaluable resource to faculty and students as well as professionals in professional development programs, education, social work, and counseling. “

From the Back Cover

“Between good intentions and great results lies a program theory—not just a list of tasks but a vision of what needs to happen, and how. Now widely used in government and not–for–profit organizations, program theory provides a coherent picture of how change occurs and how to improve performance. Purposeful Program Theory shows how to develop, represent, and use program theory thoughtfully and strategically to suit your particular situation, drawing on the fifty–year history of program theory and the authors? experiences over more than twenty–five years.

“From needs assessment to intervention design, from implementation to outcomes evaluation, from policy formulation to policy execution and evaluation, program theory is paramount. But until now no book has examined these multiple uses of program theory in a comprehensive, understandable, and integrated way. This promises to be a breakthrough book, valuable to practitioners, program designers, evaluators, policy analysts, funders, and scholars who care about understanding why an intervention works or doesn?t work.” —Michael Quinn Patton, author, Utilization–Focused Evaluation”

“Finally, the definitive guide to evaluation using program theory! Far from the narrow ?one true way? approaches to program theory, this book provides numerous practical options for applying program theory to fulfill different purposes and constraints, and guides the reader through the sound critical thinking required to select from among the options. The tour de force of the history and use of program theory is a truly global view, with examples from around the world and across the full range of content domains. A must–have for any serious evaluator.” —E. Jane Davidson, PhD, Real Evaluation Ltd.

Theory Construction and Model-Building Skills: A Practical Guide for Social Scientists

By James Jaccard PhD , Jacob Jacoby PhD, Guilford Press, 2010. Available on Amazon. Found courtesy of a tweet by EvalCollective.

See also the book review by Brandy Pratt, Western Michigan University, Journal of MultiDisciplinary Evaluation, Volume 6, Number 14, ISSN 1556-8180, August 2010

Amazon book description:  Meeting a crucial need for graduate students and newly minted researchers, this innovative text provides hands-on tools for generating ideas and translating them into formal theories. It is illustrated with numerous practical examples drawn from multiple social science disciplines and research settings. The authors offer clear guidance for defining constructs, thinking through relationships and processes that link constructs, and deriving new theoretical models (or building on existing ones) based on those relationships. Step by step, they show readers how to use causal analysis, mathematical modeling, simulations, and grounded and emergent approaches to theory construction. A chapter on writing about theories contains invaluable advice on crafting effective papers and grant applications.

Useful pedagogical features in every chapter include:

*Application exercises and concept exercises.

*Lists of key terms and engaging topical boxes.

*Annotated suggestions for further reading.”

%d bloggers like this: