DRAFT DFID Evaluation Policy – Learning What Works to Improve Lives

RD Comment: The policy document is a draft for consultation at this stage. The document will be revised to accommodate comments received. The aim is to have a finished product by the end of this calendar year. People who are interested to comment should do so directly to Liz Ramage by 16th November.

DRAFT FOR DISCUSSION 24 AUGUST 2012 (Pdf available here)

“This Evaluation Policy sets out the UK Government’s approach to, and standards for, independent evaluation of its Official Development Assistance (ODA).

PREFACE

We are publishing this evaluation policy for Official Development Assistance (ODA) at a time when the UK Government’s (the Government) approach to evaluation of international development programmes is being completely transformed.

This policy covers evaluation of all UK ODA around 87% of which is managed by the Department for International Development (DFID).  Major elements of ODA are also delivered through other Government Departments, including the Foreign and Commonwealth Office, the Department of Energy and Climate Change.

The Government is rapidly scaling up its programmes to deliver on international commitments and the Millennium Development Goals.   In doing so, the Government has made a pact with the taxpayer that this will be accompanied by greater transparency and commitment to results and measurable impact.   Evaluation plays a central part in this undertaking.

In 2011, the Independent Commission for Aid Impact (ICAI) was established, a radical change in the UK’s architecture and adopting a model which sets new standards for independence with a focus on value for money and results.  Reporting directly to Parliament, ICAI sets a new benchmark for independence in scrutiny of development programmes which applies across all UK ODA.

In parallel withICAI’s work, UK Government Departments are placing much greater emphasis on evidence and learning within programmes.

I am excited by the changes we are seeing within DFID on this initiative.  We are rapidly moving towards commissioning rigorous impact evaluations within the programmes, with much stronger links into decision making and to our major investments in policy-relevant research.

Not only has the number of specialist staff working on evaluation more than doubled, but these experts are now located within the operational teams where they can make a real improvement to programme design and delivery.

Finally, I want to note that DFID is working closely with Whitehall partners in building approaches to evaluation.  This fits well with wider changes across government, including the excellent work by the Cross-Government Evaluation Group including the updateof the Guidance for Evaluation (The Magenta book)”

Mark Lowcock, Permanent Secretary, Department for International Development

Contents

KEY MESSAGES.

1       INTRODUCTION.

1.1      Purpose of the Policy and its Audience.

1.2      Why we need independent and high quality evaluation.

2       A TRANSFORMED APPROACH TO EVALUATION.

2.1      The Government’s commitment to independent evaluation.

2.2      The Independent Commission for Aid Impact

2.3      The international context for development evaluation.

3       WHAT IS EVALUATION?.

3.1      Definition of evaluation.

3.2      Distinctions with other aspects of results management

3.3      Evaluation Types.

4       ENSURING EVALUATIONS ARE HIGH QUALITY.

4.1      Quality.

4.2      Principles.

4.3      Standards.

4.4      Criteria.

4.5      Methods.

4.6      How to decide what to evaluate.

4.7      Resources.

5       IMPACT EVALUATION.

5.1      Definitions and quality standards for impact evaluation.

6       USING EVALUATION FINDINGS.

6.1      The importance of communicating and using evaluation findings.

6.2      Timeliness.

6.3      Learning and using evidence.

7       PARTNERSHIPS FOR EVALUATIONS.

7.1      A more inclusive approach to partnership working.

7.2      A stronger role for developing countries.

7.3      Partnerships with multilaterals, global and regional funds and civil society organisations.

8       DFID’s STRATEGY FOR EMBEDDING EVALUATION.

8.1      A transformed approach to evaluation.

8.2      DFID’s co-ordinated approach to results: where evaluation fits in.

8.3      Mandatory quality processes.

8.4      Ensuring there are no evidence gaps in DFID’s portfolio.

8.5      Building capacity internally: evaluation professional skills and accreditation programme.

8.6      Roles and responsibilities for evaluation.

PS: For comparison, the previous policy document: Building the evidence to reduce poverty The UK’s policy on evaluation for international development. Department for International Development (DFID) June 2009, and the March 2009 draft version (for consultation).

 

 

NZAID 2008 Evaluations and Reviews: Annual Report on Quality, 2009

Prepared by Miranda Cahn, Evaluation Advisor, Strategy, Advisory and Evaluation Group, NZAID, Wellington, August 2009. Available online

Executive Summary

Introduction

The New Zealand Agency for International Development (NZAID) is committed to improving evaluative activity1, including evaluations and reviews. Since 2005 NZAID has undertaken annual desk studies of the evaluations and reviews completed by NZAID during the previous calendar year. This 2009 study assesses the quality of 29 NZAID commissioned evaluations and reviews that were submitted to the NZAID Evaluation and Review Committee (ERC) during 2008, and their associated Terms of Reference (TOR). The study identifies areas where quality is of a high standard, and areas where improvement is needed. Recommendations are made on how improvements to NZAID commissioned evaluations and reviews could be facilitated.

The objectives of the study are to:

• assess the quality of the TOR with reference to the NZAID Guidelines on Developing TOR for Reviews and Evaluations

• assess the quality of the NZAID 2008 evaluation and review with reference to the NZAID Evaluation Policy, relevant NZAID Guidelines and Development Assistance Committee of Organisation for Economic Cooperation and Development (DAC) Evaluation Quality Standards

• identify, describe and discuss key quality aspects of the TOR and evaluation and review reports that were of a high standard and those that should be improved in future. Continue reading “NZAID 2008 Evaluations and Reviews: Annual Report on Quality, 2009”

[US] OMB Releases Plan to Elevate Performance Evaluation

Posted on October 14, 2009

The [US]  Office of Management and Budget (OMB) released a memo to federal agencies on Oct. 7 that outlines a new initiative to bring a renewed emphasis and additional resources for program evaluation within agencies. Although this initiative is not a comprehensive plan to reinvigorate performance measurement in the federal government, it will help correct many problems that kept previous performance systems from creating real improvement in government performance.

The memo is entitled “Increased Emphasis on Program Evaluations” and details a three-part plan to help agencies develop better systems to conduct “rigorous, independent program evaluations.” The plan includes giving better access to agency program evaluations on the Internet that are both in progress and planned for the future, re-launching an interagency working group on evaluations, and a voluntary pilot program to provide additional resources to fund rigorous program evaluations or strengthen evaluation capacity within agencies. Each of these three policy changes will help to improve performance evaluations within agencies and encourage better use of performance information.

Article continues here

New DFID policy on Evaluation

“DFID takes very seriously the responsibility to ensure high quality, independent evaluation of its programmes, to provide reliable and robust evidence to improve the value of its global work to reduce poverty.

In December 2007 the Independent Advisory Committee on Development Impact was established to help DFID strengthen its evaluation processes. The Committee is there to work with DFID to:

  • Determine which programmes and areas of UK development assistance will be evaluated and when;
  • Identify any gaps in the planned programme of evaluations and make proposals for new areas or other priorities as required;
  • Determine whether relevant standards (e.g. of the OECD Development Assistance Committee) are being applied; and comment on the overall quality of the programme of evaluation work carried out against these.

DFID and IACDI have therefore been working closely together to define a new policy which will set the course for evaluation in the future. We have also produced a ‘topic list’ of potential areas for evaluation over the coming 3 years. So you will see here two documents on which we would like your feedback, the Draft Evaluation Policy and the Evaluation Topic List.

Central to the policy is the emphasis on greater independence of evaluation, along with stronger partnership working, reflecting global commitments to harmonisation, decentralising evaluation to a greater degree, driving up quality, and ensuring that learning from evaluation contributes to future decision making. We would like you to consider those high level issues when offering your comment and feedback during the time the consultation process is open. This document does not focus on the operational issues; they will be considered in a separate DFID strategy document.

During the consultation period, we would also like to hear your views on which topics you consider to be the greatest priority and why. This will help DFID to make decisions on which are to be given the highest priority.

In summary the issues we are particularly keen for you to focus your feedback on are:

1. The definition of ‘independent evaluation’ – what are your thoughts on the policy approach of DFID, working increasingly with partners, to increase independence in evaluation?

2. What are your views on what’s required to drive up quality across the board in evaluation of international development programmes? What role do you think DFID can most valuably play in this?

3. What are the considerations for DFID strengthening its own evaluation processes, whilst ensuring its commitments to harmonisation remain steadfast?

4. DFID is determined to increase the value of learning from evaluation to inform policy – what are your thoughts on the means to bring this about?

5. DFID is committed to consulting stakeholders during our evaluations, including poor women and men affected by our programmes.   Getting representative stakeholders, especially for evaluations which go beyond specific projects and programmes, can often be challenging (for example evaluations of country assistance plans or thematic evaluations).  Do you have any ideas on how to improve this?

6. DFID is committed to developing evaluation capacity in partner countries and increasing our use of national systems. What are your thoughts on the challenges and ways forward?

Please send your feedback to evaluationfeedback@dfid.gov.uk . The public consultation will officially close on Tuesday 3rd March but we would appreciate comments as early as possible, so that they can be considered as the operational issues are further thought out.”

%d bloggers like this: