Evaluation and Assessment of Poverty and Conflict Interventions (EAPC)

[from the MercyCorps website]

“A significant body of knowledge exists on the relationship between poverty and conflict. Research has shown that low per capita income and slow economic growth drastically increase the chances that a country will experience violence. Driven in part by these findings, donors and their partners are implementing increasing numbers of economic development programs in conflict and post-conflict environments, based on the assumption that these will contribute to both poverty reduction and conflict management.”

“To test this assumption, Mercy Corps implemented the USAID-funded Evaluation and Assessment of Poverty and Conflict Interventions (EAPC) research project. Over the 18 month life of the project, Mercy Corps worked with its field teams in Ethiopia, Indonesia, and Uganda to 1) develop indicators and data collection tools, 2) field test these indicators and tools, and 3) begin to assess several theories of change that inform Mercy Corps’ programs.”

“Findings from the research project are shared in three key documents:
Conflict & Economics: Lessons Learned on Measuring Impact, a summary of learning about M&E in conflict-affected environments, including indicator menus and data collection tools.
A case study highlighting findings from Uganda.
A case study highlighting findings from Indonesia.

Please contact Jenny Vaughan at jvaughan@bos.mercycorps.org for further information.”

IMPACT AND AID EFFECTIVENESS: Mapping the Issues and their Consequences

[from the IDS Virtual Bulletin, March 2011]

Introduction
In this virtual Bulletin we bring together ten articles dating from across three decades. They all address Impact. From the outset, we note that there are a number of common threads and ideas that stretch across all the articles:

  • The implicit emphasis of all the articles on complexity
  • The breadth and depth of impact analysis, from the national level to the individual
  • The importance of knowing the audience for any evaluation or impact assessment
  • The virtuous cycle that can be created by using insights into impact to adjust interventions
  • The dependency of that virtuous cycle on participation and engagement of programme staff and clients.

What we notice, however, is how the articles framing these issues vary according to discipline and research site. We also see how some ongoing preoccupations have been shaped by their proximity to other debates or policy concerns. Our hope is that hindsight will provide some perspective for practice and policy going forward.
View Full Introduction

Articles
A Revolution Whose Time Has Come? The Win-Win of Quantitative Participatory Approaches and Methods
IDS Bulletin Volume 41, Issue 6, November 2010
Robert Chambers

Impact of Microfinance on Rural Households in the Philippines
IDS Bulletin Volume 39, Issue 1, March 2008
Toshio Kondo, Aniceto Orbeta, Clarence Dingcong and Christine Infantado

You Can Get It If You Really Want’: Impact Evaluation Experience of the Office of Evaluation and Oversight of the Inter-American Development Bank
IDS Bulletin Volume 39, Issue 1, March 2008
Inder Jit Ruprah

The Role of Evaluation in Accountability in Donor-Funded Projects
IDS Bulletin Volume 31, Issue 1, January 2000
Adebiyi Edun

Micro-Credit Programme Evaluation: A Critical Review†
IDS Bulletin Volume 29, Issue 4, October 1998
Shahidur R. Khandker

Macroeconomic Evaluation of Programme Aid: A Conceptual Framework
IDS Bulletin Volume 27, Issue 4, October 1996
Howard White

Measurement of Poverty and Poverty of Measurement
IDS Bulletin Volume 25, Issue 2, April 1994
Martin Greeley

Developing Effective Study Programmes for Public Administrators
IDS Bulletin Volume 8, Issue 4, May 2009
Ron Goslin

Improving the Effectiveness of Evaluation in Rural Development Projects
IDS Bulletin Volume 8, Issue 1, July 1976
B. H. Kinsey

Managing Rural Development
IDS Bulletin, Volume 6, Issue 1, September 1974
Robert Chambers

Does Research Reduce Poverty? Assessing the welfare impact of policy-orientated research in agriculture

ALINe publication: Sumner, A., Masset, E. and Mulmi, R. (2010) IDS Practice Paper, under review. Available online

EXECUTIVE SUMMARY

In the current context of the global financial crisis and its aftermath, development resources are likely to be getting scarcer. Resources for development research are too. The set of circumstances generating the resource scarcity is also putting pressure on development gains. More than ever before, every dollar spent on development will have to count towards sustainable poverty reduction as will every dollar spent on development research. However, understanding the impacts of development research on policy change and on poverty is weak at best, with agriculture being no different.

The area of research impact is not a new area of enquiry but an emergent one. Our paper seeks to build on the work of others, notably, IFPRI, CGIAR, IDRC, ODI RAPID, GDN, NR International and ECDPM. In our paper we survey the literature and identify different ways of assessing the impact of ‘policy-oriented’ research. We then take the available literature on agriculture as a specific focus to survey.

Our paper surveys the different types of ‘policy-oriented’ research; the literature on the ‘theories of change’ for policy research in international development; methodologies for analysing the impact of policy-oriented research; the relevant agriculture literature and outlines the types indicators that can be used for impact assessment of research with examples.

The key findings are:

  • There is no standard practice for the evaluation of research projects and every evaluation strategy should be designed on a case-by-case basis.
  • Provided we are willing to accept some assumptions, it is possible to test research project impacts along some dimensions of social welfare (agricultural output, income or poverty) by finding the appropriate indicators (and methodology). The overall goal – welfare impacts of research – is highly desirable but not always feasible (especially so due to time-lags).
  • When a welfare assessment of research projects is not feasible, it is recommended that evaluators test intermediate project outcomes. The articulation of the theory of change of the project allows testing critical links in the causal chain running from research to welfare.


Analyzing the Effects of Policy Reforms on the Poor: An Evaluation of the Effectiveness of World Bank Support to Poverty and Social Impact Analyses

World Bank, 2010

“The World Bank introduced the Poverty and Social Impact Analysis (PSIA) approach in fiscal 2002 to help governments and the Bank anticipate and address the possible consequences of proposed policy reforms, especially on the poor and vulnerable, and to contribute to country capacity for  policy analysis. By fiscal 2007 the Bank had undertaken 156 pieces of analytical work using one or more elements of the PSIA approach (hereafter called PSIAs) in 75 countries and 14 sectors. Total donor support to PSIAs over fiscal 2004–06 was $15 million, which came from the Bank’s earmarked Incremental Fund for PSIAs ($5.8 million), earmarked PSIA Trust Funds contributed by various bilateral donors, and non-earmarked Bank budget and other donor funding.”…

“Although  the Bank has  submitted progress reports  to donors  regarding  the  implementation  of  PSIAs,  it  has  not  yet  completed a  comprehensive  self-evaluation  of  the PSIA  experience.  This  evaluation  by  the Independent  Evaluation Group,  requested by  the Bank’s Board of Executive Directors, represents the first independent evaluation of the PSIA experience.”

Full text available online

The material and political bases of lived poverty in Africa: insights from the Afrobarometer

The lived poverty index: strongly related to measurement of political freedoms
Authors: M. Bratton (ed)
Publisher: Afrobarometer, 2008
Full text of document

(Via Eldis Development Reporter)

Abstract: The Afrobarometer has developed an experiential measure of lived poverty called the Lived Poverty Index (LPI). It measures how frequently people go without basic necessities during the course of a year. This is a portion of the central core of the concept of poverty not captured by existing objective or subjective measures.

As an individual measure, the LPI is found to be valid and reliable. However, it exhibits only moderate external validity when compared with absolute measures of national wealth: contrary to what appears to be the consensus among economists, GDP growth is accompanied by increases in lived poverty. Also, there is only a weak relationship between LPI and measures of human development or income poverty.

At the same time, lived poverty is strongly related to country level measures of political freedom. This supports Sen’s (1999) arguments about development as freedom and Halperin et al’s (2005) arguments about the “democracy advantage” in development. This paper concludes that this measure does well at measuring the experiential core of poverty, and capturing it in a way that other widely used international development indicators do not.

Poverty and environment indicators

Produced by: Capability and Sustainability Centre, University of Cambridge (2008)

The aim of this report is to explain technical aspects in using and developing Poverty & Environment indicators (P&E) by providing a toolbox that will enable the readers to use indicators to mainstream environment into poverty reduction strategies. The document targets policy-makers working with poverty and environment issues in Africa.

The main messages of this document include:

  • poverty reduction strategies needs to encompass the environment in order for it to be successful
  • human development can be promoted with moderate increases in countries’ ecological footprint
  • general human well-being and general environment indicators are not particularly focused on the links between poverty and environment
  • the existing P&E indicators can only partially solve the problems of ‘integration’ between their different dimensions

This report recommends the use of ‘adjustment factors’, which can take into account the nature and extent of environmental problems and ‘regression analysis’ to develop P&E indicators. It proposes a new methodology that enables the development of indicators to be: relational, objective and multidimensional.

Available online at: http://www.eldis.org/cf/rdr/?doc=36898

[This notice was copied from the ELDIS POVERTY REPORTER 3 June 2008]

Basic Necessities Survey versus Schreiner’s Simple Poverty Scorecard

This page compares two simple methods of measuring poverty:

The comparison was prompted by an email to the MandE NEWS email list by Atta Ullah in May 2008 asking for a comparison.
Continue reading “Basic Necessities Survey versus Schreiner’s Simple Poverty Scorecard”

Improving poverty measurement in Sri Lanka

(From ELDIS Poverty Reporter)

Authors: Gunewardena,D.
Produced by: Centre for Poverty Analysis, Sri Lanka (2005)

Recently, conceptual advances in poverty measurement have been made:

  • acceptance of the multidimensionality of poverty
  • parallel use of monetary, capability, social exclusion and participatory approaches
  • better measurement of the dynamics of poverty and vulnerability
  • a rudimentary but growing agenda for the measurement of empowerment
  • empirical work comparing the results of different approaches
  • availability of non-traditional instruments of data collection

Reviewing studies that measure poverty in Sri Lanka, this paper finds that poverty measurement in Sri Lanka has also evolved considerably:

  • establishment of an official poverty line
  • adoption of the cost of basic needs (CBN) poverty measurement methodology by the Department of Census and Statistics
  • operationalising the multidimensionality of poverty via the human poverty index and multidimensional composite index
  • ne w survey instruments and methodologies
  • considerable data generation “capacity” of the Department of Census and Statistics (DCS) and the Statistics Department of the Central Bank of Sri Lanka (CBSL)
  • availability of a large amount of administrative data

However, there has not been much progress in measuring the dynamics of poverty, mainly because of a lack of panel data. Measurement of vulnerability and empirical work comparing the results of quantitative and qualitative approaches are also scarce. The existing data still has to be made into an information system and there is a lack of highly disaggregated data as well as data for the North and the East.

Given those weaknesses and strengths, the following steps toimprove poverty measurement, data generation and dissemination in Sri Lanka are proposed:

  • identifying user needs
  • developing appropriate equivalence scales to assign household expenditure to individuals
  • deriving relative and subjective poverty lines
  • constructing baseline datasets for the North and East
  • combining qualitative and quantitative methods of poverty analysis and data collection
  • making surveys consistent and comparable across time and instruments
  • introducing multi-topic surveys
  • generating panel data
  • publication of consistent and comparable poverty statistics
  • maintenance of an internet site with information on data for monitoring poverty
  • construction of public use data files from the Census and relevant household surveys

Available online at: http://www.eldis.org/go/about-eldis&id=36895&type=Document

Experiences of Uganda’s PPA in implementing and monitoring poverty reduction

[from Eldis Poverty Reporter]

Authors: Ssewakiryange,R.
Produced by: International Institute for Environment and Development (2005)

Uganda’s Poverty Eradication Action Plan (PEAP) incorporates poor people’s voices and perspectives. This paper focuses on the lessons learnt in the implementation of the PEAP by using examples from the Ugandan participatory poverty assessment process.

Key recommendations based on lessons learned include:

  • success of a partnership between civil society, government and other actors depends on each actor’s understanding of their role in the partnership
  • mechanisms for sustained engagement of the poor and other actors in the process are still very weak, it is therefore important design ways to ensure continued engagement
  • practitioners should ask themselves if empowerment of the poor is still a central objective of the poverty reduction strategy, and how it can be attained
  • invest in innovative ways of spending more in poverty reduction areas without expanding the debt volume
  • align monitoring frameworks to enhance coordin ation and to allow for more inclusion of actors

Available online at: http://www.eldis.org/go/topics/resource-guides/conflict-and-security&id=39402&type=Document