Original paper by Steven Teles, Department of Political Science, Johns Hopkins University, and Mark Schmitt, Roosevelt Institute. Published with support provided by The William and Flora Hewlett Foundation. Found courtesy of @alb202

A version of this paper was published in the Stanford Social Innovation Review  in May 2011 and is available as a pdf

“The political process is chaotic and often takes years to unfold, making it difficult to use traditional measures to evaluate the effectiveness of advocacy organizations. There are, however, unconventional methods one can use to evaluate advocacy organizations and make strategic investments in that arena”

HIV-related advocacy evaluation training for civil society organisations.

Produced by the International HIV/AIDS Alliance (Secretariat), International Council of AIDS Service Organizations (ICASO), July 2010, 38 pages. Available as .pdf

“This training pack is published by the Alliance and the International Council of AIDS Service Organizations (ICASO) and consists of two guides designed for advocacy, monitoring and evaluation staff of civil society organisations (including networks) who are involved in designing, implementing and assessing advocacy projects at different levels. The purpose of these guides is to increase users’ capacity to evaluate the progress and results of their advocacy work. The guides aim to:

1. help users to identify and confront the challenges faced by community-based organisations evaluating HIV-related advocacy
2. introduce new thinking for designing advocacy evaluations
3. give users the opportunity to apply some aspects of the evaluation design process to their specific contexts
4. make users aware that advocacy evaluation is a fast-growing and evolving field, with a large number of publications on advocacy evaluation design, approaches and methods available via the Internet and summarised in the resources section of the learner’s guide.”

Michael Hammer, Charlotte Rooney, and Shana Warren
ISSN 2043-7943 Briefing paper number 125, March 2010. One World Trust.

“Global and national non-governmental organisations (NGOs) are the most distinct organisational form of civil society, and as such have become increasingly involved and influential in forming public opinion and policy through targeted and professional campaigning and policy advocacy. Yet their growing power has also raised questions about the basis on which they engage in these activities, including their accountability and legitimacy in view of frequent explicit or implicit claims these organisations make to social representation, the quality of their research work, and the public benefit they provide.

Based on a world-wide survey of civil society self-regulatory initiatives undertaken by the One World Trust this paper examines how NGOs have begun to address the accountability challenges they face in particular when engaging in advocacy and explains some of the strengths and weaknesses of existing self-regulation for NGOs engaged in advocacy.

Research presented in the paper suggests that both normative and instrumental reasons account for the adoption of accountability principles by advocacy organisations through self-regulation, and that lessons learnt from the One World Trust’s parallel work on accountability principles for policy oriented research organisations can be usefully applied also to strengthen accountability of advocacy NGOs.

The briefing identifies for each major dimension of accountability a set of initial good practice principles for advocacy organisations, including on:
• transparency of the evidence basis used in advocacy, of funding and funders for specific campaigns and activities, and around forward looking information such as strategy, and the processes used to determine advocacy priorities;
• opportunities for participation of beneficiaries and other key stakeholders of the organisation in the development of advocacy objectives and their review; and
• the development of criteria for evaluating the impact of advocacy with beneficiaries and other stakeholders, and the establishment of feedback and complaints handling mechanisms to address individual experiences and problematic impacts.

The paper concludes with the identification of remaining challenges for research and self-regulation practice to strengthen accountability in advocacy by NGOs: how to deal with inherent tensions between objectivity and messaging in purpose driven advocacy; how to protect independence, freedoms and role of NGOs in the public policy process, and how to strengthen the connection between ethical practice in fundraising and selfregulation of policy advocacy work”

From the ActionAid website

“Systematization is the reconstruction of and analytical reflection about an experience. Through systematization, events are interpreted in order to understand them… The systematization allows for the experience to be discussed and compared with other similar experiences, and with existing theories and, thus, contributes to an accumulation of knowledge produced from and for practice” (Systematization Permanent Workshop in AAI systematization resource pack, pg 10, 2009).

“The second resource is Advocacy for Change, a systematization of advocacy experiences related to the status of youth (in Guatemala), the right to education (in Brazil) and farming (in the United States). The systematizations allowed the actors involved to consider the evolution of the experiences and to identify lessons and insights for future interventions. The Guatemala systematization product was documented in writing and film, the US experience in writing, and the Brazil experience in film”

Tracking Progress in Advocacy – Why and How to Monitor and Evaluate Advocacy Projects and Programmes looks at the scope of, and rational for, engaging in advocacy work as part of development interventions, then focuses on the monitoring and evaluating of these efforts – offering reasons why and when these processes should be planned and implemented, what’s involved, and who should be engaged in the process. By Janice Griffen, Dec, 2009

The Challenges of Monitoring and Evaluating Programmes offers some clarity in understanding the different uses of the term ‘programme’, and uses the different types of programme to demonstrate the issues that arise for M&E. By Janice Griffen, Dec, 2009

(email from Pathfinder) Hi Rick. We recently published a new resource for the advocacy evaluation field.  Our new guide, Pathfinder : A Practical Guide to Advocacy Evaluation, comes in three editions-one each for advocates, evaluators, and funders. All three editions, plus a bibliography of useful resources, are free to download and share from our website:

• http://www.innonet.org/client_docs/File/advocacy/pathfinder_advocate_web.pdf
• http://www.innonet.org/client_docs/File/advocacy/pathfinder_evaluator_web.pdf
• http://www.innonet.org/client_docs/File/advocacy/pathfinder_funder_web.pdf
• http://www.innonet.org/client_docs/File/advocacy/InnoNet_Pathfinder_Resource_List.pdf

Pathfinder provides a “big picture” view for planning and conducting an advocacy evaluation. Drawn from Innovation Network’s research and consulting experience, the guide encourages the adoption of a “learning-focused evaluation” approach, which prioritizes using knowledge for improvement.

Two papers by Ricardo Wilson-Grau

Evaluating the Effects of International Advocacy Networks, Ricardo Wilson-Grau and Martha Nu. “This “think piece” will first sketch the special challenges of evaluating the effects of the advocacy work of international social change networks. I will then present the approach to evaluating advocacy that I use. This essay is an adaptation of my most recent writing on the broader subject of “Complexity and International Social Change Networks,” which is a chapter in a book by the Global Partnership for the Prevention of Armed Conflict.These networks almost by definition have an advocacy component, which often is their central activity. Furthermore, my belief is that to a greater or lesser extent, the challenges and the general evaluation methodology I outline in this essay are applicable to almost all social change organisations. I leave that judgement, however, to the reader. ”

This is a paper presented at the Advocacy Impact Evaluation Workshop at the Evans School for Public Affairs, University of Washington, 4-6 December 2007, Seattle, WA, USA, sponsored by the Bill and Melinda Gates Foundation. A Spanish version “Evaluación de las redes internacionales de cambio social – Efectos y desafíos de las redes internacionales de incidencia” was published by Futuros 21: http://www.futuros21.infodetalle_articulo.asp?id_articulo=55

Evaluating International Social Change Network: A conceptual framework for a participatory approach”: Ricardo Wilson-Grau, 2007. “International networks for social change are growing in number and infuence.While they need to be able to assess the extent to which they achieve their purpose and determine ways in which to be more effective, conventional evaluation methods are not designed for such complex organisational forms, or for the diverse kinds of activity to which they are characteristically dedicated. Building on an earlier version of their paper, the authors present a set of principles and participatory approaches that are more appropriate to the task of evaluating such networks.”

Published in Development in Practice, Volume 17, Number 2, April 2007

Author: Jane Reisman, Anne Gienapp, and Sarah Stachowiak
Publisher: Organizational Research Services
Publication Date: 2007

Abstract
What are examples of data collection tools for evaluating advocacy?

This handbook of tools is a companion to ORS’ “A Guide To Measuring Advocacy And Policy”. The data collection tools included in the handbook have actually been used to evaluate advocacy or related efforts. The data collection instruments apply to six outcomes areas:

* Shifts in Social Norms;
* Strengthened Organizational Capacity;
* Strengthened Alliances;
* Strengthened Base of Support;
* Improved Policies; and
* Changes in Impact.

Author: Coffman, Julia
Publisher: Harvard Family Research Project
Publication Date: September 2007

Abstract
Are there shared elements-goals, outcomes, indicators-across different types of advocacy work? Can we create a common vocabulary for the advocacy evaluation field?

The Composite Logic Model (“CLM”) and associated materials was developed by Julia Coffman from Harvard Family Research Project; Astrid Hendricks and Barbara Masters from The California Endowment; Jackie Williams Kaye from The Atlantic Philanthropies; and Tom Kelly from the Annie E. Casey Foundation. More than 50 funders, evaluators, and advocates also lent their expertise to refine the Model.

…has just been published by the Campaigning Effectiveness Programme at NCVO

The book, written by Jim Coe and Ruth Mayne, seeks to advance the case that simple and effective monitoring and evaluation can, if implemented well, become a powerful tool for social and political change. It is aimed at campaigners and managers who are thinking of introducing monitoring and evaluation approaches within their campaign, or who are looking at ways of improving an existing approach.

Part 1 (available as a free download) provides an overview of the key elements of monitoring and evaluating campaigning, outlining some simple approaches. Part 2 looks in more detail at approaches to tracking outcomes, impact and ways of working. The final sections of the book consider how to design a coherent approach, with commentary on appropriate frameworks, tool and techniques.

Information available at

www.ncvo-vol.org.uk/publications/publication.asp?id=8612