Tag: corruption

Randomised controlled trial testing the effects of transparency on health care in Uganda

(from the great AidInfo website)

“At aidinfo we conduct research and liaise with aid donors and recipients to build up a case for aid transparency. We want to show that improving and increasing the amount that donors report on their aid contributions can help communities to track aid spending. In turn, donors and governments will be more accountable for their aid spending. It is expected that in this way aid will reach more people on the ground, helping to contribute more in the fight against poverty.

This is all well and good, but it is difficult to prove. Svensson’s work, then, is of great importance to us here.

This Study by Reinikka and Svensson (2005) found that in 1995 only 20 percent of a primary education grant program to rural Uganda actually reached its intended target. This figure rose by a striking 60 percent in 2001 when information was published detailing where this money was going; a full 80 percent of funds reached their intended destination, greatly improving education services in the area.

Björkman and Svensson (2009) followed up on this study with a compelling randomised controlled trial testing the effects of transparency on health care in Uganda. The experiment randomly assigned community health clinics to receive published ‘report cards’ and NGO-organised public meetings on the quality of the clinics’ health care.

The results of this transparency ‘treatment’ rivalled the effects of the best health interventions involving expensive new medicines, equipment, and procedures. Waiting time for care decreased, absenteeism among doctors and nurses plummeted, clinics got cleaner, fewer drugs were stolen, 40-50 percent more children received dietary supplements and vaccines, health services got used more, and, powerfully, 33 percent fewer children died under the age of five. This amounted to 550 saved lives in a small area of Uganda encompassing merely 55,000 households.

This is strong evidence that access to information about services empowers citizens to get better services and saves lives.”

MPs report on Department for International Development Financial Management

The Commons Public Accounts Committee publishes its 52nd report of Session 2010-12, on the basis of evidence from the Department for International Development (DfID).

“The Rt Hon Margaret Hodge MP, Chair of the Committee of Public Accounts, said:

“The amount DfID spends on aid will rise by 35% by 2013, but at the same time the Department has to cut its overall running costs by a third.
Achieving this level of savings at a time of rapid expansion in frontline services involve a substantial challenge if taxpayers’ money is to be properly protected and value for money secured. [emphasis added]

The Department is going to be spending more in fragile and conflict-affected countries and the danger to the taxpayer is that there could be an increase in fraud and corruption. However, the Department could not even give us information as to the expected levels of fraud and corruption and the action they were taking to mitigate it.

Unfortunately, the Department has not always kept its eye on the financial ball, and in 2010 stopped monitoring its finance plan. That must not happen again and DFID should report publicly on its financial management.

The Department’s ability to make informed spending decisions is undermined by its poor understanding of levels of fraud and corruption. Its current approach is too reactive and it needs to develop a sound framework for making sure funds are spent properly on the ground. This will be even more important as the Department channels more of its funding into fragile and conflict-affected states.

The Department’s current plan is to spend more via multilateral organizations and less through bilateral programmes. This poses a risk to value for money because the Department will have less oversight than it does over country-to-country programmes. Indeed, we are concerned that the strategy has more to do with the fact that it is easier to spend through multilaterals than go through the process of assessing value for money of bilateral programmes. [emphasis added]

To maximise the amount of aid that gets through to the frontline, DfID should have clear plans for how it is going to reduce or control running costs – particularly when channelling funding through partner and multilateral organizations with a management overhead at every stage.”[emphasis added]

Margaret Hodge was speaking as the committee published its 52nd Report of this Session which, on the basis of evidence from the Department for International Development, examined its financial management capability, its increasing focus on value for money, and the challenges it faces in managing its increasing programme budget while reducing its overall running costs.”

RD Comment: See Rick on the Road blog posting Thursday, July 24, 2008: An aid bubble? – Interpreting aid trends which raises the same issues as highlighted in bold above.

See also HoC International Development Committee, Committee Room 15 Working effectively in fragile and conflict-affected states: DRC, Rwanda and Burundi

How to monitor and evaluate anti-corruption agencies

Guidelines for agencies, donors and evaluators

By Jesper Johnsøn, Hannes Hechler, Luís De Sousa, Harald Mathisen (2011)  Bergen: Chr. Michelsen Institute (U4 Issue 2011:8) 84 p. Available as pdf

“The number of Anti-corruption agencies (ACAs) around the world has increased dramatically over the past decades. Nevertheless, the value of ACAs is increasingly being questioned by international donors and national governments. Frequently, ACAs are not considered to deliver on the high expectations bestowed upon then.

Evaluations of individual agencies were collected and analysed to assess the evidence underlying the assumptions about the effectiveness of ACAs. Surprisingly, few evaluations had actually been done, and even fewer measured the actual outcomes and impacts of the ACA. Thus, whilst opinions about ACAs are many, the actual evidence about their performance is scarce. To develop this body of evidence, ACAs need to do a better job at establishing results-based indicators for their work, showing how activities lead to impact, and collecting data.

To which extent the perceived failure of ACAs is an issue of measurement or design can therefore not be answered with any certainty. The value of ACAs can only be determined once evidence-based evaluations are conducted.

To this end, the report provides technical, methodological, and practical guidance to assist staff of ACAs in undertaking monitoring and evaluation and shows how the outcomes and impact of the work of ACAs can be evaluated in an objective, evidence-based manner.”

 

Measuring Impact on the Immeasurable? Methodological Challenges in Evaluating Democracy and Governance Aid

by Jennifer Gauck, University of Kent, Canterbury – Department of Politics, 2011. APSA 2011 Annual Meeting Paper. Available as pdf

Abstract:

“Recent debates over the quality, quantity and purpose of development aid has led to a renewed emphasis on whether, and in what circumstances, aid is effective in achieving development outcomes. A central component of determining aid effectiveness is the conduct of impact evaluations, which assess the changes that can be attributed to a particular project or program. While many impact evaluations use a mixed-methods design, there is a perception that randomized control trials (RCTs) are promoted as the “gold standard” in impact evaluation. This is because the randomization process minimizes selection bias, allowing for the key causal variables leading to the outcome to be more clearly identified. However, many development interventions cannot be evaluated via RCTs because the nature of the intervention does not allow for randomization with a control group or groups.”

“This paper will analyze the methodological challenges posed by aid projects whose impacts cannot be evaluated using randomized control trials, such as certain democracy and governance (D&G) interventions. It will begin with a discussion of the merits and drawbacks of cross-sectoral methods and techniques commonly used to assess impact across a variety of aid interventions, including RCTs, and how these methods typically combine in an evaluation to tell a persuasive causal story. This paper will then survey the methods different aid donors are using to evaluate the impact of projects that cannot be randomized, such as governance-strengthening programs aimed at a centralized public-sector institution. Case studies will be drawn from examples in Peru and Indonesia, among others. This paper will conclude by analyzing how current methodological emphases in political science can be applied to impact evaluation processes generally, and to D&G evaluations specifically.”

RD Comment: See also the 3ie webpage on Useful resources for impact evaluations in governance which includes a list of relevant books, reports, papers, impact evaluations, systematic reviews, survey modules/tools and website

Eight lessons from three years working on transparency

Blog posting by Owen Barder
February 22nd, 2011

“I’ve spent the last three years working on aid transparency. As I’m moving on to a very exciting new role (watch this space for more details) this seems a good time to reflect on what I’ve learned in the last three years.

This is a self-indulgently long essay about the importance of aid transparency, and the priorities for how it should be achieved. Busy readers can just read the 8-point summary below. For a very clear and concise introduction to the importance of aid transparency, this video by my (former) colleagues at aidinfo is very good.

I’m going to talk in a separate post about the exciting progress that has been made towards a new system of aid transparency, which I believe builds on many of these lessons, and on the next steps for the transparency movement more generally.

The 8-point summary

There is apparently a law that every document in development must have an “Executive Summary”. (Not just a “summary”, mind. It has to be for executives.) So here are what I think are the eight most important things I’ve learned in the last three years about transparency in general, and aid transparency in particular:

1. To make a difference, transparency has to be citizen-centred not donor-centred. A citizen-centred transparency mechanism would allow citizens of developing countries to combine and use information from many different donor agencies; and provide aid information compatible with the classifications of their own country budget.

2. Today’s ways of publishing information serve the needs of the powerful, not citizens. Existing mechanisms for publishing aid information were designed by the powerful for the powerful. Until the aidinfo team started 3 years ago, nobody had ever done a systematic study of the information needs of all stakeholders, including citizens, parliamentarians and civil society, let alone thought about how those needs could be met. That’s why current systems meet only the needs of donors, and powerful parts of governments.

3. People in developing countries want transparency of execution not just allocation. There are important differences between the information requirements of people in donor countries and people in developing countries. Current systems for aid transparency focus mainly on transparency of aid allocation, because that is what donor country stakeholders are largely interested in, and not enough on transparency of spending execution, which is of primary interest to people in developing countries.

4. Show, don’t tell. The citizens of donor nations are increasingly sceptical of annual reports and press releases. In aid as in other public services they want to be able to see for themselves the detail of how their money is being used and what difference it is making. They increasingly expect to be actively involved in decisions, and they are less willing to delegate the decisions entirely to experts. Donor agencies – whether government agencies, international organisations or NGOs – will have to adapt rapidly to become platforms for citizen engagement.

5. Transparency of aid execution will drive out waste, bureaucracy and corruption. There is, unfortunately, quite a bit of waste, bureaucracy and corruption in the aid system. There is good evidence that this kind of waste is rapidly reduced when the flow of money is made transparent. Corruption and waste prosper in dark places. Transparency of planned future aid spending will also help to increase spending efficiency and value for money.

6. Social accountability could be Development 3.0. The results agenda in aid agencies is currently too top down and pays too little attention to the power of bottom up information from the intended beneficiaries of aid. Increased accountability to citizens may be the key to unlocking better service delivery, improved governance and faster development.

7. The burden of proof should be on those who advocate secrecy. We have published a compelling business case for greater transparency, with all the uncertainties this kind of analysis entails. So where is the business case for secrecy, which would be far harder to quantify or defend? Why is the (inevitable) uncertainty in this kind of analysis allowed to count against the case for transparency, when the same uncertainty would deal a much greater blow against the case for secrecy?

8. Give citizens of developing countries the benefit of the doubt. Transparency is necessary but not sufficient for more effective aid. But the fact that transparency alone will not solve every problem should not be an excuse for aid agencies to shirk their responsibilities to be transparent. Nor should we be too attentive to vested interests in the aid industry telling us that transparency is not enough. Citizens of developing countries will be more innovative and effective than some people give them credit for when we give the information they need to hold the powerful to account.

That’s the summary. If any of that whets your appetite and you want the long version, read on.”

%d bloggers like this: