Tag: AusAID

Impact Evaluation: A Discussion Paper for AusAID Practitioners

“There are diverse views about what impact evaluations are and how they should be conducted. It is not always easy to identify and understand good approaches to impact evaluation for various development situations. This may limit the value that AusAID can obtain from impact evaluation.

This discussion paper aims to support appropriate and effective use of impact evaluations in AusAID by providing AusAID staff with information on impact evaluation. It provides staff who commission impact evaluations with a definition, guidance and minimum standards.

This paper, while authored by ODE, is an initiative of AusAID’s Impact Evaluation Working Group. The working group was formed by a sub-group of the Performance and Quality Network in 2011 to provide better coordination and oversight of impact evaluation in AusAID.”

ODE welcomes feedback on this discussion paper at ODE@ausaid.gov.au

AusAID: Establishment of Independent Evaluation Committee

[from AusAID website, 11 May 2012]

Foreign Minister Bob Carr has announced the establishment of an Independent Evaluation Committee (IEC) to strengthen the independence and credibility of the work of the Office of Development Effectiveness (ODE).

Chaired by Jim Adams—a former Vice President of the World Bank—the Committee will oversee ODE in assessing the effectiveness and evaluating the impact of the Australian aid program.

In its policy statement, An Effective Aid Program for Australia, the Government committed to establishing an IEC in response to a recommendation from last year’s Independent Review of Aid Effectiveness. This is part of the Government’s commitment to improve the aid program’s evaluation function in order to deliver more efficient and effective aid.

The IEC will be an advisory body with a whole of government mandate, providing independent expert evaluation advice to the Development Effectiveness Steering Committee, which provides advice to government on Overseas Development Assistance priorities and effectiveness.

It will also oversee the work of ODE in planning, commissioning, managing and delivering a high quality evaluation program. The IEC will provide advice on ODE’s evaluation strategy and work plan. It will also oversee ODE’s preparation of an annual evaluation summary and quality assurance report.

The IEC will meet four times a year, with the first meeting of the IEC scheduled for June 2012.

Who is on the IEC?

The Independent Evaluation Committee has three external members (including the chair) and one senior AusAID representative. External members are appointed by the Minister for Foreign Affairs, while the Director General of AusAID appoints the AusAID representative. Given the IEC’s whole of government mandate, a representative from the Department of Finance and Deregulation will be invited to attend meetings as an observer.

The external members are Jim Adams (Chair), Professor Patricia Rogers and Dr Wendy Jarvie. They contribute a mix of solid international development and aid effectiveness experience, high-level evaluation expertise and strong public sector experience to the IEC.

Read the member biographies [PDF 330kb]
Read the member biographies [Word 89kb]

How will the IEC work?

The IEC will oversee the work program of ODE and reports to the DESC. The terms of reference for the IEC set out its mandate, roles and responsibilities. The terms of reference were  endorsed by the DESC before being approved by the Minister.

Read the terms of reference [PDF 549kb]
Read the terms of reference [Word 59kb]

 

“Unleashing the potential of AusAID’s performance data”

A posting on the Development Policy Blog by Stephen Howes, on 15 february 2012.

This blog examines AusAID’s Office of Development Effectiveness latest annual report released just before Christmas 2010, which was published in two parts, one providing an international comparative perspective (and summarized in this blog), the other drawing on and assessing internal performance reporting. In this blog the author continues his analysis of the  “internal assessment” report.

He points out how the report data shows that poor performance is a much more significant problem than outright fraud. He also examines the results of ODE’s spotchecks on the quality of the self-assessment ratings. There is much else there in the blog that is also of interest.

Of special interest are the concluding paras: “This systematic collation of project self-ratings and the regular use of spot checks is best practice for any aid agency, and something AusAID should take pride in. The problem is that, as illustrated above, the reporting and analysis of these two rich sources of data is at the current time hardly even scratching the surface of their potential.

One way forward would be for ODE or some other part of AusAID to undertake and publish a more comprehensive report and analysis of this data. That would be a good idea, both to improve aid effectiveness and to enhance accountability.

But I have another suggestion. If the data is made public, we can all do our own analysis. This would tremendously enhance the debate in Australia on aid effectiveness, and take the attention away from red-herrings such as fraud towards real challenges such as  value-for-money.

AusAID’s newly-released Transparency Charter[pdf] commits the organization to releasing publishing “detailed information on AusAID’s work” including “the results of Australian aid activities and our evaluations and research.”  The annual release of both the self-ratings and the spot-checks would be a simple step, but one which would go a long way to fulfilling  the Charter’s commitments.”

PS: Readers may be interested in similar data made available by DFID in recent years. See Do we need a minimum level of failure blog posting

 

Is Australian Aid Fair Dinkum? A Forum On The Independent Review Of Aid Effectiveness

Venue: Old Parliament House, 18 King George Tce, Parkes 8222, Canberra
Date: Tuesday, 13 September 2011 6:00 PM

Summary

“In a world where we have achieved so much, from quantum leaps in medical research to the development of sophisticated technologies, it seems implausible that there are more hungry people in the world today than the populations of the United States, Canada and the European Union combined.

But the picture isn’t all bleak. A recent report released by the United Nations reveals that we have made some significant progress in our bid to alleviate poverty around the world, and the Independant Review of Aid Effectiveness commissioned by the Australian Government has made some assessments and recommendations that could help guide progress in the future.

However, when it comes to the complex issue of poverty alleviation, there are no simple answers.

What are some of the challenges faced when it comes to ensuring that we are taking the smartest and most efficient approach to tackling poverty? What are the timeframes within which we can realistically expect change to happen? And are we doing enough to address structural and behavioural issues that perpetuate gender inequality and other forms of exploitation that continue the vicious cycle of poverty.

How much of a difference are we actually making?”

Speakers include:

  • James Batley – Deputy Director-General, Asia Pacific and Program Enabling Group, AusAID
  • Stephen Howes – Director, Development Policy Centre, ANU and member of Independent Aid Effectiveness Review panel
  • Dr Julia Newton-Howes – Chief Executive, CARE AustraliaNikunj Soni – Board Chair, Pacific Institute of Public Policy, Vanuatu

Registration and other information here

Released: Australian Government’s response to the Independent Review of Aid Effectiveness

The ‘Independent Review of Aid Effectiveness’ and the Government’s response were released on 6 July 2011 by Foreign Minister Kevin Rudd, in an official launch at Parliament House, followed by a Ministerial Statement to Parliament. For an overview, see this page on the AusAID website

Independent Review of Aid Effectiveness:

Commissioned in November 2010, this was the first independent review of the aid program in 15 years. It made 39 recommendations to improve the program

Australian Government response:

The Government  has agreed (or agreed in principle) to 38 of the recommendations. Including that the agency develop a three-tiered results framework for reporting on agency-wide performance.

See also

RD Comment: The following section on Independent Evaluation is of particular interest [underlining added]:

ii) Independent Evaluations

“AusAID’s Independent Completion Reports and Independent Progress Reports are another key part of its Performance Management and Evaluation Policy.

Under current guidelines, a report must be completed for an activity every four years, either during its implementation (a progress report) or at completion (a completion report). Reports are required for projects above $3 million and are meant to be made public. They are independent in that they are done by individuals not involved in the project. Typically, but not always, they are written by non–AusAID staff.

By international standards, this policy is thorough. For example, at the World Bank, independent completion reports are done only for a sample of projects

But a study of AusAID evaluation reports commissioned by the Review Panel found that implementation of AusAID’s evaluation policy is patchy:
• Of 547 projects that should have had a completion or progress report in 2006–10, only 170 were recorded as having been done.
• Of the 170, only 118 could be found.
• About 26 per cent of the completion and progress reports were assessed to be too low quality to publish.
• Only about 20 have been published on the AusAID website.

Clearly, the policy is not being fully followed. Other problems were also evident. None of the 118 completion or progress reports reviewed provided an unsatisfactory rating. This raises questions of credibility. In comparison, 20 per cent of World Bank projects are rated unsatisfactory by its independent evaluation group.

There is also a structural issue with the policy: AusAID program managers must approve the publication of an independent report. This risks conflicts of interest and long delays in publication. The low rate of publication suggests these problems may be occurring.

Independent completion reports, when done and published, can be very useful. For example, the completion report on the first phase of the Indonesia Basic Education Project is in the public domain and helped to inform recent public debate about the second phase of the project (AusAID 2010b). In contrast, several useful completion reports have recently been done for the PNG program, but only one has been released.

Given the problems described above, it is not surprising that the Review Panel has seen little evidence that these reports inform and improve aid delivery.“

AusAID’s Information Publication Scheme: Draft Plan & Consultation

The 12th April 2011 Draft plan is now available in pdf and MS Word

Introduction

“AusAID is the Australian Government’s Agency for International Development, an executive agency within the Department of Foreign Affairs and Trade portfolio. Its primary role is the implementation and oversight of the Australian Government aid program. The aim of the program is to assist
developing countries reduce poverty and achieve sustainable development, in line with Australia’s national interest.

Reforms to the Freedom of Information Act 1982 (FOI Act) have established the Information Publication Scheme (IPS). The purpose of the IPS is to give the Australian community access to information held by the Australian Government and enhance and promote Australia’s representative
democracy by increasing public participation in government processes and increasing scrutiny, discussion, comment and review of government activities and decisions.

AusAID is committed to greater transparency through the implementation of the Information Publication Scheme (IPS) and other initiatives that will introduced. As Australia’s ODA commitment has increased, public interest in the aid program has correspondingly increased and this will
continue. Implementation of the IPS will provide more information to Australians about AusAID’s activities and help increase public participation understanding and scrutiny of Australia’s aid program.

This draft plan has been prepared to assist AusAID implement the IPS, in accordance with section 8(1) of the Freedom of Information Act (FOI) 1982 and to give the Australian public the opportunity to comment and provide feedback on this plan.

As AusAID’s final plan is implemented it will be progressively updated in light of experience and feedback. The list of documents that is a core part of this plan will, in particular, be amended.”

The consultation: Visit this AusAid website to see how to participate and to read the views of others who have already contributed.

 

AusAID-DFID-3ie call for Systematic Reviews

The Australian Agency for International Development (AusAID), the UK’s Department for International Development (DFID) and the International Initiative for Impact Evaluation (3ie) have just launched a joint call for proposals for systematic reviews to strengthen the international community’s capacity for evidence-based policy making. AusAID, DFID and 3ie have identified around 59 priority systematic review questions across several themes: education; health; social protection and social inclusion; governance, fragile states, conflict and disasters; environment; infrastructure and technology; agriculture and rural development; economic development; and aid delivery and effectiveness.

Systematic reviews examine the existing evidence on a particular intervention or program in low and middle income countries, drawing also on evidence from developed countries when pertinent. The studies should be carried out according to recognized international standards and guidelines. All studies will be subject to an external review process and for this purpose teams will be encouraged to register for peer review with a relevant systematic review coordinating body.

Applications have to be submitted using 3ie’s online application system. Deadline for submission of applications is 9am GMT on Monday, November 29, 2010.

For information on how to apply, guidance documents and the call for proposals, go to http://www.3ieimpact.org/systematicreviews/3ie-ausaid-dfid.php

%d bloggers like this: