>The UNICEF Evaluation Report Standards< have been created as a transparent tool for quality assessment of evaluation reports.  This document outlines what the Standards are, the rationale for each standard and how they are applied.  The Standards are used by UNICEF Evaluation Office to assess
evaluations for inclusion in the organisation’s Evaluation and Research Database to strengthen the Database as a learning tool.  Application of Standards will also provide feedback to UNICEF Country Offices on how the evaluation is seen by someone outside of the evaluation process.

The Standards are also intended for use by UNICEF offices and partners commissioning evaluations to establish the criteria against which the final report will be assessed.  The UNICEF Evaluation Report Standards draw from and are complementary to key references on standards in evaluation design and process increasingly adopted in the international evaluation community

It’s not enough for charity communicators to simply tell the world about the great work their organisation does. Today charities are expected to demonstrate their impact. Knowing how well your charity performs will help it to improve, develop, inspire and influence the right people. Join us at this one day conference to learn about the nuts and bolts of measuring and communicating impact.

On the agenda:

* Practical guides on how to measure impact
* ‘How we did it’ case studies from other charities including The National Trust, RNID, and WRVS
* Advice on bringing results information to life
* Learn how to attach financial value to your impact using SROI
* Hear from City Bridge Trust’s Chief Grants Officer on what results information they look for in grant applications
* NCVO explores how to effectively evaluate campaigns

• “An Accountability Framework for Technological Innovation“, ILAC Initiative (Institutional Learning and Change) Brief In this Brief, we offer recommendations to help an organization conducting technological research and design (R&D) to become more accountable. We provide recommendations based on four principles developed by the One World Trust which capture the dimensions of accountability. (The ILAC Initiative aims to increase the contribution of the CGIAR to poverty alleviation by improving planning, monitoring and evaluation of collaborative agricultural research for development. It’s great that they have been able to publish this Brief applying the APRO framework to agricultural science.)
• “Accountability of innovation: A literature review, framework and guidelines to strengthen accountability of organisations engaged in technological innovation“: This paper synthesises some of the most important lessons learned arising from the evolving understanding of innovation, and provides a framework of accountability for organisations engaged in technological R&D. The guidelines focus on supporting organisations to become more effective, while simultaneously ensuring that they adhere to ethical standards in their innovation.

The Environmental Evaluators Network (EEN) will host its 5th annual Forum in Washington DC at The George Washington University on June 7,8 2010. The purpose of the EEN is to advance the field of environmental program and policy evaluation through more systematic and collective learning among evaluators and evaluation users. The 5th annual EEN Forum will bring together evaluators and users of evaluation to explore and articulate the significance of the emerging era of results-based performance on the quality of environmental evaluations.

The Issue:

Growing awareness of the interdependencies of our social, economic, and ecological systems requires more efficient use of scarce resources to evaluate complex problems. In this heightened era of accountability, recipients and funders of environmental programs want evidence of what works, and what does not, and better mechanisms for using real time information for decision-making. How will the era of results-based performance affect the quality of environmental evaluations? What must be done to improve the quality of environmental evaluations to meet the requirements and desire for better and more accessible evidence of program and policy effectiveness?

Theme:

Environmental Evaluation: Quality in an Era of Results-Based Performance

The American Evaluation Association’s 2010 Annual Conference will address the issue of Evaluation Quality. The EEN 2010 Forum will explore the practical and theoretical issues that affect evaluation quality that are most pertinent to environmental evaluators, including:

Balancing the need for accountability, transparency, evidence and improvement

Integrating measurement and evaluation into the design of programs and policies

The dispersed and disjointed nature of relevant data and information

Participatory evaluation vs. expert driven evaluation

The utility of logic models and other articulations of program theory

Communication in evaluation – social media and data visualization

Demonstrating short-term and long term results

Who for?

Environmental evaluators and users of environmental evaluation, both domestic and international, in government agencies, foundations, consulting firms, non-profit organizations, academia, and international institutions who are well poised to contribute to advancing the field of environmental evaluation.

Where?

The George Washington University, Washington, D.C.

When?

Monday and Tuesday, June 7-8, 2010

Sponsors:

U.S. Environmental Protection Agency

National Fish and Wildlife Foundation

National Oceanic and Atmospheric Administration

The George Washington University Trachtenberg School of Public Policy and Pubic Administration

Format:

One of the keys to the success of the 2010 Forum will be our ability to create a space for even more lively, cutting edge dialogue. Rather than a purely academic discussion, the 2010 EEN Forum will aspire to document, in a format useful to policy makers, the environmental evaluators’ perspective on the primary issues influencing evaluation quality. To achieve this and further improve the networking and access to the experience and knowledge of participants, the 2010 Forum will consider the value of integrating Web 2.0 social media and alternative facilitation approaches, such as Open Space Technology, into sections of the Forum.

For updates, networking and more on the EEN and this year’s Forum:

Join the www.LinkedIn.com Environmental Evaluators Network group!!

Check out www.nfwf.org/een for information on past events and products!!

Registration information soon to follow.

We hope to see you there and look forward to your participation!

Most Sincerely,  Matt Keene & Matt Birnbaum

Matt Keene

Evaluation Support Division

Office of Policy, Economics, and Innovation

U.S. Environmental Protection Agency

Office 202-566-2240

Keene.matt@epa.gov

Matthew Birnbaum

Evaluation Officer

National Fish and Wildlife Foundation

Office: 202-595-2461

Cell: 202-360-3969

Matthew.Birnbaum@nfwf.org

“Whilst few doubt the importance of capacity building, and the need for effective monitoring and evaluation (M&E) to support this work, the M&E of capacity building is as much a challenge now as it was two decades ago. This paper examines both theory and current practice, and aims to promote debate on some of the key barriers to progress.

The paper is primarily concerned with capacity building within civil society organisations (CSOs), although many of the lessons also apply to commercial and state organisations. It is based on a literature review and interviews with capacity building providers in the North and South.

The paper begins by looking at some key concepts in capacity building and M&E. It examines different ways of thinking about M&E, and different tools and approaches used to plan, monitor and evaluate capacity building work. It goes on to discuss M&E in relation to donors and outlines current practices. Finally, it highlights key areas for  discussion, and presents some conclusions based on the research.

The main findings are that where organisations are clear about what they want to achieve through improved capacity (or capacity building) and where there is a clear understanding of the purpose of M&E, it is not difficult to come up with a blend of tools, methodologies and approaches to meet the needs of different stakeholders. But if capacity building providers lack an adequate theory of change; if they do not know what results they want to achieve; or if M&E work is burdened by uncertain or unrealistic demands, then the area can appear to be a minefield.

The paper concludes by presenting some practical guidelines for developing or improving M&E processes. It also highlights the importance of internal commitment to M&E at senior levels within capacity building providers. Finally, it asks whether we need to improve the incentives for those organisations that seriously wish to move the debate forwards.”

Available in hard copy and pdf download

See also ‘Developing M&E Systems for Complex Organisations: A Methodology’ by Nigel Simester

“Almost all development organisations are expected to have systems that enable them to collect, analyse, summarise and use information. However, whilst there is a large and growing range of resources covering monitoring and evaluation (M&E), there is little guidance or support for those wishing to design M&E systems. The guidance that is available tends to focus on systems at project or programme level, rather than at country, regional or international levels. This paper attempts to redress this imbalance by describing a methodology that can be used to design an M&E system within a complex organisation.

The paper is divided into two parts:

• Part One describes a methodology for designing an M&E system. This covers a number of stages from defining the scope and purpose of the system to its eventual roll out. However, it does not cover the implementation of the M&E system.
• Part Two addresses many of the technical issues that need to be considered when designing M&E systems for complex organisations. The information in Part Two may be useful to M&E system designers whether or not the basic methodology in Part One is followed.

Also available as pdf download

Folks,

I’d like to draw your attention to a new book, ‘Evaluating Climate Change and Development‘, edited by Rob van den Berg and Osvaldo Feinstein. This to my knowledge is the first volume that systematically addresses issues relating to climate change from an evaluation point of view. How do we know that the policies, programs and projects targeted towards mitigation, adaptation and vulnerability actually work? The book reviews evidence and approaches and also contains a wealth of case studies from the developing world.
…end of email…

Amazon Abstract: Climate change has become one of the most important global issues of our time, with far-reaching natural, socio- economic, and political effects. To address climate change and development issues from the perspective of evaluation, an international conference was held in Alexandria, Egypt. This book distills the essence of that timely conference, building on the experiences of more than 400 reports and studies presented. Developing countries may be particularly vulnerable to the expected onslaught of higher temperatures, rising sea levels, changing waterfall patterns, and increasing natural disasters. All societies will have to reduce their vulnerability to these changes, and this book describes how vulnerabilities may be addressed in a systematic manner so that governments and local communities may better understand what is happening. Different approaches are also discussed, including the use of human security as a criterion for evaluation as well as ways to deal with risk and uncertainty. “Evaluating Climate Change and Development” presents a rich variety of methods to assess adaptation through monitoring and evaluation. The volume deals with climate change, development, and evaluation; challenges and lessons learned from evaluations; mitigation of climate change; adaptation to climate change; vulnerability, risks and climate change; and, presents a concluding chapter on the road ahead. Collectively the authors offer a set of approaches and techniques for the monitoring and evaluation of climate change.

“The latest post on Aid on the Edge post compiles presentations and reports from meetings on evaluation and complexity (NORAD, Panos, Mokoro) that have taken place in the past few months and years, as well as planned meetings later this year”

The list:

• Presentations given at a 2008 NORAD conference ‘Evaluating the Complex‘ - Michael Quinn Paton’s keynoteand my own seminar presentation
• The full report of the July 2009 meeting in London, hosted by Panos, available on the Meetings page
• Details of a forthcoming 2 day conference on Evaluation and Complexity: Evaluation Revisited: Improving the Quality of Evaluative Practice by Embracing Complexity” – May 20-21, 2010, the Netherlands

I’m writing to ask you to submit any tools you know of, or have developed, which you think could help build the accountability of research organisations. These will be entered into a publicly available online database.   The One World Trust has been working with our partners to formulate an accountability framework for organisations conducting research (whether civil society, universities, private or public sector). The database will make available tools which will help research managers reflect on and improve the accountability of the programs they manage .   Specifically, we are looking for descriptions and accounts of innovative tools and processes which fall within the following broad areas:

• Tools suitable for the monitoring and evaluation of research and advocacy;
• Tools for participatory planning of research;
• Tools to assist organisational learning and change;
• Good practices and policies of transparency in research;
• Good practices for working accountably in partnerships and networks;
• Community engagement strategies with research participants;
• Tools to ensure you ‘close the loop’ and manage feedback from your research participants;
• Examples of where organisations conducting research have integrated external stakeholders into governance structures; and
• Ethics standards for participatory and applied research.

Ideally, the description would take the form of a document or case study.   For those interested in submitting a tool, please get in contact with the One World Trust team at apro@oneworldtrust.org. For more information on the project, please visit our website. All relevant tools will be included in the database, which we propose to put online and publicly available by August 2010. If we think the tool is not relevant, we do promise to get back to you and explain why!   Thanks very much for your time,   Brendan.

–
Brendan Whitty
Principal Researcher
One World Trust
3 Whitehall Court, London, SW1A 2EL, UK
> Now Available!  The APRO toolkit provides guidelines outlining accountability principles for research organisations.
http://www.oneworldtrust.org
Tel +44 (0)20 7766 3463
Charity No. 210180

On May 20-21, 2010, a conference on evaluation for development will be held in The Netherlands: “Evaluation Revisited: Improving the Quality of Evaluative Practice by Embracing Complexity.”

This conference focuses on how evaluative practice can be improved, given the need to view much of development as a process of social transformation and, therefore, complex. Current evaluation practice has not yet embraced the full implications of assessing ‘the complex’ and existing approaches often fall woefully short. During the conference, participants can explore concrete evaluation practices that reconcile an understanding of complex societal change processes with quality standards, including rigorous, ethical concerns, appropriateness and feasibility.

Effective development is being understood in many quarters as revolving around people-centeredness and institutional transformation, thereby inherently complex, i.e. non-linear, emergent, unpredictable. However, discussions on measuring development effectiveness have zoomed in on methodologies suited to measure simpler, more linear interventions. The risk is that development, in order to facilitate its measurement, is reduced to simple interventions. These two mutually influential trends sit side-by-side in increasing discomfort for those who understand development as societal transformation.

The Conference aims to contribute to clarity in the development sector about what constitutes robustness, i.e. core values and quality standard, for evaluative practice in development that recognises the complexity of societal transformation.

The attached flyer provides further details for registration.
This information is also available at: http://tinyurl.com/evaluation-revisited.

The Organising Committee:
Jan Brouwers (Context, international cooperation)
Irene Guijt (Learning by Design)
Cecile Kusters (Wageningen UR Centre for Development Innovation)

Kind regards / Hartelijke groeten,

Cecile Kusters
Participatory Planning, Monitoring & Evaluation
Multi-Stakeholder Processes and Social Learning
Centre for Development Innovation
Wageningen UR
P.O. Box 88, 6700 AB Wageningen, The Netherlands
(As from October 2009, Wageningen International – CD&IC Programme is part of the new
Centre for Development Innovation within the Social Sciences Group of Wageningen UR – read more)
Tel. +31 (0)317 4881407 (direct), +31 (0)317 486800 (reception)
Fax +31 (0)317 486801
e-mail cecile.kusters@wur.nl

Dear colleagues,

On behalf of the EES Board of Directors we are pleased to announce that the next EES biennial conference will take place in Prague, Czech Republic on 6-8 October 2010, preceded by professional development training sessions on October 4-5. We have launched a new website www.ees2010prague.org where you can find all useful information in regards to the conference. The website will be regularly updated, for newest information please visit section Home – News.

Please find attached a Call for abstract brochure with all information regarding the abstract submission and programme strands.
We want to encourage a wide range of contributions to the conference. Therefore we offer different types of presentation formats, which should be indicated while submitting your abstract. All abstracts are welcome and the best ones are likely to be published or otherwise disseminated by EES. But in addition participants can make use of a wide spectrum of possibilities (panels and symposia, round tables, posters, etc). We also invite creative and original vehicles using the performing arts, film, music, etc.

The Conference topic „Evaluation in the Public Interest – Participation, Politics and Policy“  will be discussed in five strands you are welcome to submit the proposals. Submission strands are:

1.    Ethics, capabilities and transparency
2.    Evaluation and politics
3.    Evaluation producers, beneficiaries, users and decision makers
4.    Sector policy evaluation
5.    Evaluation in developing and transition economies

The Conference administration is undertaken by:

CZECH-IN s.r.o.
Professional Event & Congress Organiser
Prague Congress Centre
5. kvetna 65
140 21 Prague 4
Czech Republic

In case of any questions, do not hesitate to contact us at:

Tel: +420 261 174 304
Fax: +420 261 174 307
Email: info@ees2010prague.org, abstracts@ees2010prague.org

Abstract submission is to be open at 18 February 2010

Registration is to be open at 18 February 2010

For more information please visit www.ees2010prague.org

Thank you very much
Looking forward to welcoming you in Prague in October 2010

Conference Secretariat and EES Conference co-chairs Marie Kaufmann & Claudine Voyadzis