NZAID 2008 Evaluations and Reviews: Annual Report on Quality, 2009

Prepared by Miranda Cahn, Evaluation Advisor, Strategy, Advisory and Evaluation Group, NZAID, Wellington, August 2009. Available online

Executive Summary

Introduction

The New Zealand Agency for International Development (NZAID) is committed to improving evaluative activity1, including evaluations and reviews. Since 2005 NZAID has undertaken annual desk studies of the evaluations and reviews completed by NZAID during the previous calendar year. This 2009 study assesses the quality of 29 NZAID commissioned evaluations and reviews that were submitted to the NZAID Evaluation and Review Committee (ERC) during 2008, and their associated Terms of Reference (TOR). The study identifies areas where quality is of a high standard, and areas where improvement is needed. Recommendations are made on how improvements to NZAID commissioned evaluations and reviews could be facilitated.

The objectives of the study are to:

• assess the quality of the TOR with reference to the NZAID Guidelines on Developing TOR for Reviews and Evaluations

• assess the quality of the NZAID 2008 evaluation and review with reference to the NZAID Evaluation Policy, relevant NZAID Guidelines and Development Assistance Committee of Organisation for Economic Cooperation and Development (DAC) Evaluation Quality Standards

• identify, describe and discuss key quality aspects of the TOR and evaluation and review reports that were of a high standard and those that should be improved in future. Continue reading “NZAID 2008 Evaluations and Reviews: Annual Report on Quality, 2009”

International Summer School on Evaluation Research

Date: 30 August to 3 September 2010
Venue: Lille, France

Call for papers – Deadline: 7 May 2010

The Network of Researchers on Evaluation of the French Evaluation Society (Société Française de l’Evaluation) has the pleasure of:

This Summer School is intended to be an ideal opportunity for those in the scientific community interested in public policy evaluation to compare scientific work on the subject and to discuss the interdisciplinary nature of evaluation research. Continue reading “International Summer School on Evaluation Research”

trainings in Evaluation of Humanitarian Action

Date: 14-16 June, 2010 and 17-18 June, 2010
Venue: near Brussels

Evaluation of Humanitarian Action with ALNAP (Active Learning Network for Accountability and Performance in Humanitarian Action)

14-16 June, 2010

This course is an introductory-to- intermediate level course and has the overall aim of making evaluations of humanitarian action more effective in contributing to the improved performance of interventions and to improve the quality of the evaluation process. This 3-day training course is based on an update of the ALNAP training modules. The course will also introduce some new material, specifically:

  • on joint evaluations: the rationale, experience and learning to date, interwoven throughout the training programme
  • on evaluating policy as well as projects and programmes
  • on innovative learning processes as part of the evaluation process.

Continue reading “trainings in Evaluation of Humanitarian Action”

Addressing accountability in NGO advocacy: Practice, principles and prospects of self-regulation

Michael Hammer, Charlotte Rooney, and Shana Warren
ISSN 2043-7943 Briefing paper number 125, March 2010. One World Trust.

“Global and national non-governmental organisations (NGOs) are the most distinct organisational form of civil society, and as such have become increasingly involved and influential in forming public opinion and policy through targeted and professional campaigning and policy advocacy. Yet their growing power has also raised questions about the basis on which they engage in these activities, including their accountability and legitimacy in view of frequent explicit or implicit claims these organisations make to social representation, the quality of their research work, and the public benefit they provide.

Based on a world-wide survey of civil society self-regulatory initiatives undertaken by the One World Trust this paper examines how NGOs have begun to address the accountability challenges they face in particular when engaging in advocacy and explains some of the strengths and weaknesses of existing self-regulation for NGOs engaged in advocacy.

Research presented in the paper suggests that both normative and instrumental reasons account for the adoption of accountability principles by advocacy organisations through self-regulation, and that lessons learnt from the One World Trust’s parallel work on accountability principles for policy oriented research organisations can be usefully applied also to strengthen accountability of advocacy NGOs.

The briefing identifies for each major dimension of accountability a set of initial good practice principles for advocacy organisations, including on:
• transparency of the evidence basis used in advocacy, of funding and funders for specific campaigns and activities, and around forward looking information such as strategy, and the processes used to determine advocacy priorities;
• opportunities for participation of beneficiaries and other key stakeholders of the organisation in the development of advocacy objectives and their review; and
• the development of criteria for evaluating the impact of advocacy with beneficiaries and other stakeholders, and the establishment of feedback and complaints handling mechanisms to address individual experiences and problematic impacts.

The paper concludes with the identification of remaining challenges for research and self-regulation practice to strengthen accountability in advocacy by NGOs: how to deal with inherent tensions between objectivity and messaging in purpose driven advocacy; how to protect independence, freedoms and role of NGOs in the public policy process, and how to strengthen the connection between ethical practice in fundraising and selfregulation of policy advocacy work”

The Implementation of the Paris Declaration on Aid Effectiveness: Where do We Stand and How to Move Forward?

Date: 15th -16th April 2010
Venue: Martin’s Central Park Hotel, Boulevard Charlemagne 80, Brussels:

The Paris Declaration (PD), adopted in 2005, lays down principles and procedures for enhancing the effectiveness of aid and specifies them in twelve targets supplied with monitorable indicators and to be achieved by 2010. The quantitative assessment by the OECD of the progress towards 2010 reveals considerable delays of donors and partner countries. With the global economic crisis and growing concern about climate change the targets of the PD have become a great challenge.

EADI in cooperation with the Institute of Development Policy and Management (IOB) will hold a 2 days intensive training workshop in Brussels for development professionals and practitioners. This training workshop is part of the EADI Masterclasses. Continue reading “The Implementation of the Paris Declaration on Aid Effectiveness: Where do We Stand and How to Move Forward?”

Making Connections: Using Social Network Analysis for Program Evaluation

by Todd Honeycutt, Issue Brief, November 2009, Number 1,  Mathematica Policy Research, Inc. 4 pages

Abstract: “Social network analysis (SNA) is a methodological approach to measuring and mapping relationships. It can be used to study whole networks, all of the ties within a defined group, or connections that individuals have in their personal communities.The resulting graph-based structures illustrate the composition and effectiveness of networks on a variety of levels. Programs that can benefit from a social network perspective are common in evaluation research. This brief introduces the concepts behind SNA and illustrates how to use this method in process and outcome evaluations.”

NETWORKING: CURRENT MODALITIES AND FUTURE SCENARIOS

Final Report, October 2007, Hanne Lund Madsen, h.lund.madsen@email.dk

For The Danish Child & Youth NGO Network, The Danish NGO Education Network, Thematic Forum, Gendernet, Aidsnet

The overall aim of this study is to:

  • Provide a solid basis for discussion of the future perspectives for NGO Networks including a synthesis and analysis of experiences regarding NGO networks in Denmark and aboard
  • Discuss the networking concept on the basis of the study and create a framework for common positioning and decision among Danish NGO Networks regarding possible scenario for future networking cooperation.

A Reference Group appointed by the Networks involved has been engaged in the study and has provided input and reflection during the process, including participation in a mini-scenario workshop. Continue reading “NETWORKING: CURRENT MODALITIES AND FUTURE SCENARIOS”

International Development Evaluation: Foundations and Practices

Date: 5 – 9 July 2010 (5 full days)
Venue: King’s College London

Course Director: Robert Picciotto, Visiting Professor, King’s College London

International development evaluation is increasingly called upon to support evidence based policy making in the management of development programmes. Growing public pressure to secure better results from poverty reduction initiatives have generated a high demand for quality and independent assessments aimed at improved decision making in different countries, contexts and cultures. Consequently, expectations regarding evaluators’ capabilities are rising.

This summer institute course explores core concepts and emerging issues within the rapidly expanding field of development evaluation. It aims at laying solid foundations for the professional growth of development evaluators, managers and commissioners. It addresses:
– The conceptual foundations and basic principles of development evaluation; its purposes, products and uses
– The analytical and policy implications of development trends for evaluation policies, programmes and projects
– The available tool kits of evaluation methods and techniques
– The emerging guidelines and standards for shaping development evaluation policies and practice
– The governance and management arrangements for development evaluation and their implications

Continue reading “International Development Evaluation: Foundations and Practices”

VIRTUAL CONFERENCE ON METHODOLOGY IN PROGRAMME EVALUATION

7th to 9th April, 2010Date:
Venue: Internet

Overall Conference Theme

International Trends in Programme Evaluation Methodology.

Conference Aims

The School of Human and Community Development at the University of the Witwatersrand (Wits) aims to bring together contributions from programme evaluation theorists and practitioners working in both developed and developing country contexts.

The virtual conference focuses on the following general areas as relevant to the development of common approaches to programme evaluation internationally:

  • Theory and paradigms of programme evaluation
  • Programme evaluation methodology
  • Teaching, training and instruction in programme evaluation
  • Applications of programme evaluation theory and models, and the practice of programme evaluation.

Ways of Participation

The conference is a virtual conference, hosted on the internet. It focuses on contributions in electronic format of three kinds:
* Papers on programme evaluation theory and/or methodology. These should be written and presented in such a way as to be useful to others on both conceptual and/or methodological levels.
* Workshop materials in the form of PowerPoint presentations supported by written handouts. These should be written and presented in such a way as to be useful to others who teach programme evaluation, for teaching, training and/or instructional purposes.
* Short case studies of completed evaluations. These should be written and presented in such a way as to be useful to others in documenting applications and the practice of programme evaluation, as applied in specific local, national or international contexts.

To participate in the conference, draft contributions and abstracts relative to these three types of contribution need to be uploaded to the conference website between April 7th and 9th, 2010.

Information on deadlines relative to the timing of different stages in the conference upload and reviewing process is provided below. Additional information, instructions for authors and reviewers, and templates, are provided on the conference website at the following address: http://wpeg.wits.ac.za

Analyzing the Effects of Policy Reforms on the Poor: An Evaluation of the Effectiveness of World Bank Support to Poverty and Social Impact Analyses

World Bank, 2010

“The World Bank introduced the Poverty and Social Impact Analysis (PSIA) approach in fiscal 2002 to help governments and the Bank anticipate and address the possible consequences of proposed policy reforms, especially on the poor and vulnerable, and to contribute to country capacity for  policy analysis. By fiscal 2007 the Bank had undertaken 156 pieces of analytical work using one or more elements of the PSIA approach (hereafter called PSIAs) in 75 countries and 14 sectors. Total donor support to PSIAs over fiscal 2004–06 was $15 million, which came from the Bank’s earmarked Incremental Fund for PSIAs ($5.8 million), earmarked PSIA Trust Funds contributed by various bilateral donors, and non-earmarked Bank budget and other donor funding.”…

“Although  the Bank has  submitted progress reports  to donors  regarding  the  implementation  of  PSIAs,  it  has  not  yet  completed a  comprehensive  self-evaluation  of  the PSIA  experience.  This  evaluation  by  the Independent  Evaluation Group,  requested by  the Bank’s Board of Executive Directors, represents the first independent evaluation of the PSIA experience.”

Full text available online