Tag: impact assessment

The Limits of Nonprofit Impact: A Contingency Framework for Measuring Social Performance

Alnoor Ebrahim, V. Kasturi Rangan, Social Enterprise Initiative, Harvard Business School (2010) Working Paper 10-099 Available as pdf

ABSTRACT

“Leaders of organizations in the social sector are under growing pressure to demonstrate their impacts on pressing societal problems such as global poverty. We review the debates around performance and impact, drawing on three literatures: strategic philanthropy, nonprofit management, and international development. We then develop a contingency framework for measuring results, suggesting that some organizations should measure long-term impacts, while others should focus on shorter-term outputs and outcomes. In closing, we discuss the implications of our analysis for future research on performance management.”

Analyzing the Effects of Policy Reforms on the Poor: An Evaluation of the Effectiveness of World Bank Support to Poverty and Social Impact Analyses

World Bank, 2010

“The World Bank introduced the Poverty and Social Impact Analysis (PSIA) approach in fiscal 2002 to help governments and the Bank anticipate and address the possible consequences of proposed policy reforms, especially on the poor and vulnerable, and to contribute to country capacity for  policy analysis. By fiscal 2007 the Bank had undertaken 156 pieces of analytical work using one or more elements of the PSIA approach (hereafter called PSIAs) in 75 countries and 14 sectors. Total donor support to PSIAs over fiscal 2004–06 was $15 million, which came from the Bank’s earmarked Incremental Fund for PSIAs ($5.8 million), earmarked PSIA Trust Funds contributed by various bilateral donors, and non-earmarked Bank budget and other donor funding.”…

“Although  the Bank has  submitted progress reports  to donors  regarding  the  implementation  of  PSIAs,  it  has  not  yet  completed a  comprehensive  self-evaluation  of  the PSIA  experience.  This  evaluation  by  the Independent  Evaluation Group,  requested by  the Bank’s Board of Executive Directors, represents the first independent evaluation of the PSIA experience.”

Full text available online

Workshop: Understanding the effects of development interventions: Theory-based impact evaluation in practice.

A three-day workshop jointly organised by Maastricht University and the University of Antwerp.

Date: April 28-30, 2010
Venue: Institute of Development Policy and Management, Lange Sint Annastraat 7, 2000 Antwerp, Belgium

Focus

In the past few years development organizations in both North and South have focused on improving evaluations of development interventions. As a result, demand for highquality impact evaluation has increased. Impact evaluation refers to the growing field of evaluative practices aimed at assessing the effects of a broad range of policy interventions. As well as enhancing the accountability of public spending in development, it has the potential to be an important learning tool, allowing us to understand what works and why under what conditions. To help impact evaluation to achieve this potential evaluators and policy makers need to open the ‘black box’ of policy interventions. Attention should be paid not only to the changes caused by an intervention but also to understanding how and why these changes have been brought about. In other words interventions should be considered as theories and evaluations are the tools for reconstructing, refining and testing these theories. This is the essence of theory-based evaluation.
The workshop focuses on the concept and application of theory-based impact evaluation in development. It starts with an overview of the key issues in impact evaluation and development effectiveness. Subsequently, the principles of theory-based impact evaluation will be discussed. Particular attention is paid to how a theory of change provides a framework for further inquiry. Starting out from a theory based perspective, different methodological approaches, from review and synthesis of existing evidence to full-scale empirical inquiry, will be presented and illustrated. These methods and modalities can form the basis for an institutional strategy on impact evaluation and learning, which is the subject of the final part of the workshop. Continue reading “Workshop: Understanding the effects of development interventions: Theory-based impact evaluation in practice.”

Training: “Impact Oriented Project Planning and Monitoring”

Date: 8th to 10th of March, 2010
Venue: Karl Kübel Institute for Development Education – India

Dear Friends,

We would like to inform you about the above mentioned workshop which we offer at the Karl Kübel Institute for Development Education, Coimbatore (www.kkid.org).

Target group:

Participants from organisations dealing with projects and programmes of development cooperation in all fields being responsible or involved in activities of project cycle management. Knowledge of formalized planning procedures like logframe-planning and the use of project cycle management are appreciated.

Objectives:

Theory-Based Impact Evaluation: Principles and Practice

Howard White June 2009 3ie WORKING PAPER 3

Abstract

Calls for rigorous impact evaluation have been accompanied by the quest not just to find out what works but why. It is widely accepted that a theory –
based approach to impact evaluation, one that maps out the causal chain
from inputs to outcomes and impact and tests the underlying assumptions,
will shed light on the why question. But application of a theory-based
approach remains weak. This paper identifies the following six principles to
successful application of the approach: (1) map out the causal chain
(programme theory); (2) understand context; (3) anticipate heterogeneity;
(4) rigorous evaluation of impact using a credible counterfactual; (5)
rigorous factual analysis; and (6) use mixed methods.

Designing impact evaluations: different perspectives

Robert Chambers, Dean Karlan, Martin Ravallion, and Patricia Rogers
July 2009 3ie WORKING PAPER 4

Preface (see text below)
Howard White, Executive Director, International Initiative for Impact
Evaluation (3ie)

Making the Poor Count: Using Participatory Methods for Impact
Evaluation
Robert Chambers, Institute of Development Studies, University of Sussex

Thoughts on Randomized Trials for Evaluation of Development:
Presentation to the Cairo Evaluation Clinic
Dean Karlan, Yale University and Innovations for Poverty Action/ Jameel
Poverty Action Lab Affiliate

Evaluating Three Stylized Intervention
Martin Ravallion, World Bank

Matching Impact Evaluation Design to the Nature of the
Intervention and the Purpose of the Evaluation
Patricia Rogers, Collaboration for Interdisciplinary Research, Consulting
and Learning in Evaluation, Royal Melbourne Institute of Technology

Preface

Debates on approaches to impact evaluation design appear to have reached an impasse in recent years. An objective of the international conference, Perspectives on Impact Evaluation, March 29th to April 2nd, Cairo, organized by 3ie, NONIE, AfrEA and UNICEF, was to bring together different voices and so work toward a consensus. A key session in this approach was a plenary in which experts from different perspectives were asked how they would approach the evaluation of three interventions: a conditional cash
transfer, an infrastructure project and an anti-corruption program. The motivation for the session was that debates get stuck when they remain at the conceptual level, but that a greater degree of consensus can be achieved once we move to the specifics of the design of a particular evaluation. I am very pleased that the four presenters agreed to write up their views so they can be more widely disseminated.

Thanks are due to Hugh Waddington and Rizwana Siddiqui for assistance in the preparation of this collection.

Howard White
Executive Director, 3ie

Assessing aid impact: a review of Norwegian evaluation practice

Authors: Espen Villanger ; Alf Morten Jerve
Published in: Journal of Development Effectiveness, Volume 1, Issue 2

June 2009 , pages 171 – 194

Warning: Unfortunately you have to pay to get access to this article in full

Abstract

This article reviews recent Norwegian aid evaluations with a mandate to study impact, and assesses how the evaluators establish causal effects. The analytical challenges encountered in the seven studies reviewed are: (1) the Terms of Reference ask for evidence of impact where this is not possible to identify, (2) the distinction between impacts of the aid element versus other components is often blurred, and (3) the methodological approaches to identify impact are either poorly developed or applied superficially. A main conclusion is that most of the evaluators did not have the necessary time or budget to conduct a proper impact evaluation given the large number of questions raised in the commissioning agency.

view references (26)

Impact Evaluation of Population, Health and Nutrition Programs

Date: October 5 – 16, 2009
Venue:  Public Health Foundation of India, New Delhi, India

USAID’s MEASURE Evaluation Project is pleased to announce the regional workshop on “Impact Evaluation of Population, Health and Nutrition Programs,” for English speaking professionals. The workshop is sponsored by the Public Health Foundation of India (PHFI), New Delhi, India in collaboration with MEASURE Evaluation. The two-week course will be held October 5 – 16, 2009 in New Delhi, India.
Continue reading “Impact Evaluation of Population, Health and Nutrition Programs”

a one day event designed to explore impact assessment …

Date: Thursday 2nd July 2009
Venue: London

EGO – Empowering Grassroots Organisations would like to invite you to…a one day event designed to explore impact assessment with leading speakers from the Third Sector

On Thursday 2nd July 2009 in London, empACT will take on three key areas; we will:

· explore the outcomes of impact assessment through sector wide funding and grassroots perspectives

· explore methods of assessment

· host workshops around proven methods and tools.

empACT will bring together speakers with wide-ranging expertise in the field to interrogate the real advantages and disadvantages of some of the methods explored.

Confirmed speakers include…

Sarah Alderson, Head of Projects – TimeBank

Isabel Ros Lopez, Inclusion Manager – United Response

Tanya Murphy, Independent Evaluation Practitioner

Samantha Beinhacker, Producer and Developer – Germination

For tickets: www.amiando.com/empACTconference or for further information please email maor@egonetwork.org.uk

3ie news: Working paper series launched

The first two 3ie working papers are now available:

Working Paper No. 1, Reflections on some current debates in impact evaluation, by Howard White reviews some of the criticisms commonly leveled at quantitative approaches to impact evaluation, arguing that many are based on mis-conceptions; and

Working Paper No. 2, Better Evidence for a Better World, edited by Mark Lipsey and Eamonn Noonan (produced jointly with The Campbell Collaboration) reviews the need for, and uses of, evidence in various fields of social policy.

%d bloggers like this: