The beginnings of a list. Please suggest others by using the Comment facility below

Normative statements:

• DFID Evaluation Quality Assurance templates for Entry level and Exit level. Entry level refers to TORs or evaluation plans, Exit level refers to draft evaluation reports. Circa 2011.
• USAID Evaluation Policy, January 2011. See Section 4 on Evalation practices, which lists 6 quality criteria
• American Evaluation Association’s PROGRAM EVALUATION STANDARDS. Available online in  Summary Form. The full text is Yarbrough, D. B., Shulha, L. M., Hopson, R. K., and Caruthers, F. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Thousand Oaks, CA: Sage
• Quality Standards for Development Evaluation, DAC Guidelines and Reference Series, OECD, 2010
• Quality control in the evaluation reports, EuropeAID Co-operation Office, 2006. (Note: a quality rating is given by the EC to every final evaluation report.)
• Norms for Evaluation in the UN System, UN Evaluation Group, 2005
• Standards for Evaluation in the UN System, UN Evaluation Group, 2005
• UNICEF’s Evaluation Standards, based on those also used by AEA above, developed by the American Joint Committee on Standards for Educational Evaluation (AJCSEE), undated
• UK Evaluation Society Good Practice Guidelines, undated

Standards for specific methods (and fields):

• The CONSORT Statement is intended to improve the reporting of a randomized controlled trial (RCT), enabling readers to understand a trial’s design, conduct, analysis and interpretation, and to assess the validity of its results. It emphasizes that this can only be achieved through complete transparency from authors. 2010
• The TREND statement  “Transparent reporting is crucial for assessing the validity and efficacy of these intervention studies, and, it facilitates synthesis of the findings for evidence-based recommendations. Therefore, the mission of the Transparent Reporting of Evaluations with Nonrandomized Designs (TREND) group is to improve the reporting standards of nonrandomized evaluations of behavioral and public health interventions” 2009
• Systematic Reviews: CRD’s guidance on undertaking reviews in health care. Centre for Reviews and Dissemination, University of York, 2009
• Methodology checklist: Qualitative studies,  National Institute for Health and Clinical Excellence, UK, January 2009
• Quality in Qualitative Evaluation: A framework for assessing research evidence A Quality Framework, by  Liz Spencer, Jane Ritchie, Jane Lewis and Lucy Dillon National Centre for Social Research, UK Cabinet Office, 2003

Meta-evaluations:

• Are Sida Evaluations Good Enough?An Assessment of 34 Evaluation Reports” by Kim Forss, Evert Vedung, Stein Erik Kruse,Agnes Mwaiselage, Anna Nilsdotter, Sida Studies in Evaluation 2008:1  See especially Section 6: Conclusion, 6.1 Revisiting the Quality Questions, 6.2 Why are there Quality Problems with Evaluations?, 6.3 How can the Quality of Evaluations be Improved?, 6.4 Direction of Future Studies. RD Comment:  This study has annexes with empirical data on the quality attributes of  34 evaluation reports published in the Sida Evaluations series between 2003 and 2005. It BEGS a follow up study to see if/how these various quality ratings correlate in any way with the subsequent use of the evaluation reports. Could Sida pursuaded to do something like this?

Ethics focused

• Australasian Evaluation Society

• Guidelines for the Ethical Conduct of Evaluations , 2010
• Code of Ethics, 2000
• Policy on the application of the Code of Ethics, 2000

Journal articles

• Assessing Quality in Applied and Educational Research: A Framework for Discussion, by John Furlong, Alis Oancea, 2008
•  Evaluation Standards in an International Context, Love & Russon,  New Directions for Evaluation, 2005

Checklists:

• Evaluation checklists prepared by the Western Michegan University ,covering Evaluation Management, Evaluation Models, Evaluation Values and Criteria, Metaevaluation, Evaluation Capacity Building / Institutionalization, and Checklist Creation

Other lists:

• See also the European Evaluation Association’s lists of:
  • Evaluation Standards of National and Regional Evaluation Societies and
  • Evaluation Standards of International and Supranational Organisations

Utrecht Conference Report. Irene Guijt, Jan Brouwers, Cecile Kusters, Ester Prins and Bayaz Zeynalova. March 2011. Available as pdf

This report summarises the outline and outputs of the Conference ‘Evaluation Revisited: Improving the Quality of Evaluative Practice by Embracing Complexity’’, which took place on May 20-21, 2010. It also adds additional insights and observations related to the themes of the conference, which emerged in presentations about the conference at specific events.

Contents (109 pages):

1 What is Contested and What is at Stake
1.1 Trends at Loggerheads
1.2 What is at Stake?
1.3 About the May Conference
1.4 About the Report
2 Four Concepts Central to the Conference
2.1 Rigour
2.2 Values
2.3 Standards
2.4 Complexity
3 Three Questions and Three Strategies for Change
3.1 What does ‘evaluative practice that embraces complexity’ mean in practice?
3.2 Trade-offs and their Consequences
3.3 (Re)legitimise Choice for Complexity
4 The Conference Process in a Nutshell

OECD DAC NETWORK ON DEVELOPMENT EVALUATION, February 2010 Download a pdf copy

“The DAC Network on Development Evaluation is a unique international forum that brings together evaluation managers and specialists from development co-operation agencies in OECD member countries and multilateral development institutions. Its goal is to increase the effectiveness of international development programmes by supporting robust, informed and independent evaluation.

A key component of the Network’s mission is to develop internationally agreed norms and standards to strengthen evaluation policy and practice. Shared standards contribute to harmonised approaches in line with the commitments of the Paris Declaration on Aid Effectiveness. The body of norms and standards is based on experience, and evolves over time to fit the changing aid environment. These principles serve as an international reference point, guiding efforts to improve development results through high quality evaluation.

The norms and standards summarised here should be applied discerningly and adapted carefully to fit the purpose, object and context of each evaluation. This summary document is not an exhaustive evaluation manual. Readers are encouraged to refer to the complete texts available on the DAC Network on Development Evaluation’s website: www.oecd.org/dac/evaluationnetwork. Several of the texts are also available in other languages.”

Evaluation Office, UNICEF NYHQ , September 2004

>The UNICEF Evaluation Report Standards< have been created as a transparent tool for quality assessment of evaluation reports.  This document outlines what the Standards are, the rationale for each standard and how they are applied.  The Standards are used by UNICEF Evaluation Office to assess
evaluations for inclusion in the organisation’s Evaluation and Research Database to strengthen the Database as a learning tool.  Application of Standards will also provide feedback to UNICEF Country Offices on how the evaluation is seen by someone outside of the evaluation process.

The Standards are also intended for use by UNICEF offices and partners commissioning evaluations to establish the criteria against which the final report will be assessed.  The UNICEF Evaluation Report Standards draw from and are complementary to key references on standards in evaluation design and process increasingly adopted in the international evaluation community

Date: 7-11 September 2009
Venue: Thailand,

Introduction
In response to growing demands to measure results, the DCED (Donor Committee for Enterprise Development) has been working with PSD (Private Sector Development) programmes in the field to define minimum elements for credible results measurement. These elements include:

• articulation of the results chain(s) or logic;
• definition of appropriate indicators;
• good measurement practice;
• attribution;
• capturing wider changes in the market; and
• relating results to costs.

The elements have been incorporated into a Standard, which has been advertised through the networks of member agencies, and through five Workshops – and which is now being applied by many programmes – particularly those seeking ways to quantify and communicate what they are achieving, with credibility. The approach calls for an independent auditor to review the quality of the measurement methodology, rather than to repeat the measurement process. For more information, please refer to www.enterprise-development.org/page/measuring-and-reporting-results

The DCED is preparing a training course, to cover the skills required to meet the Standard. After the course, participants will be able to improve their organisation’s internal results measurement systems to the level required by the Standard, or to help other organisations to do so. This Announcement is for the first such course.

Who is the course for?
The course is particularly for those who already have experience in being:

• Managers and M & E specialists of development programmes in the field, working in any area of private sector development (PSD); or
• Consultants in results measurement, who are interested either to provide technical support to PSD programmes and/or to become an auditor for the Standard.

There will be a limit of 60 on the number of participants, so there may need to be a maximum allocation per agency; in that case, the DCED will ensure a fair allocation, and each agency HQ will choose whom to nominate within that allocation. Please be aware that this is the first such course, and that further courses will be offered in the future; preference will be given to applicants with experience of results measurement, to staff of programmes and agencies already participating in the initiative, and to consultants wishing to become auditors.

What will the course consist of?
The main course, aimed particularly at programme staff, will run from Monday morning 7th September to Thursday evening 10th September; it will cover all aspects of the current Standard, including the skills needed to meet it. Examples, case studies and exercises will be provided, drawing on actual cases from around the world. If there is sufficient demand, a fast training stream will be offered to those who have already participated in one of the DCED workshops on results measurement.

Consultants and potential auditors will also participate in this course, but will work separately from Thursday lunchtime until Friday evening 11th September; that part of the course will cover auditing skills, and in particular what the acceptable minimum level should be, for programmes to comply with each element of the Standard. There will also be a short exam at the end of the course, for potential auditors, to ensure that understanding meets the minimum level required of auditors.

How much will it cost?
The course fee is US$500 per person, payable in advance; the fee may be waived (at the discretion of DCED) in the case of consultants who do not have institutional sponsorship. Participants must also meet their own travel costs, including flights, accommodation, meals (except lunches during the course) and all insurances; the DCED will cover the costs of the trainers and venue.

Where will it be held?
The course will be held in Thailand, either in Chiang Mai or Bangkok; those accepted for participation will be advised of the final venue, and are encouraged to stay in the hotel where the training course will be held.

How do I express interest?
Please write in the first instance to Results@Enterprise-Development.org indicating your experience to date in results measurement, the work of your programme (if any) and whether you would like to also join the training module for auditors, or to remain with the training for programme staff.
We look forward to hearing from you!

Journal of MultiDisciplinary Evaluation
Vol 5, No 9 (2008)

The OECD/DAC Criteria for International Development Evaluations: An Assessment and Ideas for Improvement PDF Thomaz Chianca 41-51

An Association to Improve Evaluation of Development Aid PDF
Paul Clements 52-62

Commentary on “An Association to Improve Evaluation of Development Aid” PDF
Hellmut Eggers 63-69

Reply to Hellmut Eggers’ Comments on “An Association to Improve Evaluation of Development Aid” PDF
Paul Clements 70-73

(These articles were mentioned today by Ian Patrick, Melbourne, Australia in his post on the MandE NEWS email list)

Date: August 3-8, 2008
Venue: University of Bamako, Mali

Organizer: Association pour la Promotion de l’Evaluation au Mali (APEM) – Bamako – Mali

Mamadou Keita – President (mkeita@delta- c.org)

Workshop Coordinator: Ahmed Ag Aboubacrine, DME Coordinator, CARE International in Sierra Leone,

ahmed1996@yahoo. fr

This event is part of the 5th Mali Symposium on Applied Sciences, to be held at the University of Bamako from August 3-8, 2008.

Abstract

Monitoring and evaluation of development intervention and policies’ implementation has become an absolute necessity after the setting of international norms of evaluation practices by donors as well as private sector, governments and local constituencies.

Evaluation has not become only a cross-cutting academic area (studied and subject of scientific researches) but also a basic requirement in almost all sectors: health, education, finance, infrastructure, social, agriculture, livestock, water and sanitation, urban planning, habitat, HIV/AIDS, transport, gender, corruption, governance.

Besides the standards set by independent organizations such as African Evaluation Association (www.afrea.org) and also IDEAS (http://www. ideas-int. org/ ); there are other specific evaluation mechanisms:

* African Peer Review Mechanism (APRM)

* Governance Index of Mo Ibrahim foundation

* Poverty Reduction Strategies (PRS) National Evaluation Mechanisms supported by World Bank, African Development Bank, UNDP and other UN agencies.

Each of the above mentioned organizations has its norms and standards in monitoring and evaluation which are most of them very similar.

This mini-symposium objective is to promote monitoring and evaluation practices in Mali in order to ensure the effectiveness, efficiency, sustainability and impact of development interventions undertaken by the state, local constituencies, donors, NGOs and private sector.

The specific objectives are three:

* Sharing of existing evaluation standards (state-of-art)

* Exchanging on current practices of evaluation in Mali

* Finding out the strategies for generalization / institutionalization of evaluation practice by state actors

Submission of papers

Papers should be submitted in an A4 format and should not exceed four pages. All papers related to the standards and practices of evaluation in any area could be accepted.

For more details, visit our website:

http://www.msas. maliwatch. org/msas2008/ msas2008_ 018.html