Tag: accountability

Improving the Evaluability of INGO Empowerment and Accountability Programmes

Shutt, C. and McGee, R. CDI Practice Paper 1 March 2013 Publisher IDS Available as pdf (109kb)

Abstract
This CDI Practice Paper is based on an analysis of international NGO (INGO) evaluation practice in empowerment and accountability (E&A) programmes commissioned by CARE UK, Christian Aid, Plan UK and World Vision UK. It reviews evaluation debates and their implications for INGOs. The authors argue that if INGOs are to successfully ‘measure’ or assess outcomes and impacts of E&A programmes, they need to shift attention from methods to developing more holistic and complexity-informed evaluation strategies during programme design. Final evaluations or impact assessments are no longer discrete activities, but part of longer-term learning processes. Given the weak evaluation capacity within the international development sector, this CDI Practice Paper concludes that institutional donors must have realistic expectations and support INGOs to develop their evaluation capacity in keeping with cost–benefit considerations. Donors might also need to reconsider the merits of trying to evaluate the ‘impact’ of ‘demand-side’ NGO governance programmes independently of potentially complementary ‘supply-side’ governance initiatives.

See also: Tools and Guidelines for Improving the Evaluability of INGO Empowerment and Accountability Programmes Centre for Development Impact, Practice paper. No.1 Annex March 2013

Evaluating Peacebuilding Activities in Settings of Conflict and Fragility: Improving Learning for Results

DAC Guidelines and Reference Series

Publication Date :08 Nov 2012
Pages :88
ISBN :9789264106802 (PDF) ; 9789264106796 (print)
DOI :10.1787/9789264106802-en

Abstract

Recognising a need for better, tailored approaches to learning and accountability in conflict settings, the Development Assistance Committee (DAC) launched an initiative to develop guidance on evaluating conflict prevention and peacebuilding activities.  The objective of this process has been to help improve evaluation practice and thereby support the broader community of experts and implementing organisations to enhance the quality of conflict prevention and peacebuilding interventions. It also seeks to guide policy makers, field and desk officers, and country partners towards a better understanding of the role and utility of evaluations. The guidance  presented in this book provides background on key policy issues affecting donor engagement in settings of conflict and fragility and introduces some of the challenges to evaluation particular to these settings. It then provides step-by-step guidance on the core steps in planning, carrying out and learning from evaluation, as well as some basic principles on programme design and management.

Table of Contents

Foreword
Acknowledgements

Executive summary

Glossary

Introduction: Why guidance on evaluating donor engagement in situations of conflict and fragility?

Chapter 1. Conceptual background and the need for improved approaches in situations of conflict and fragility

Chapter 2. Addressing challenges of evaluation in situations of conflict and fragility

Chapter 3. Preparing an evaluation in situations of conflict and fragility

Chapter 4. Conducting an evaluation in situations of conflict and fragility

Annex A. Conflict analysis and its use in evaluation

Annex B. Understanding and evaluating theories of change

Annex C. Sample terms of reference for a conflict evaluation

Bibliography

 

2012 European Evaluation Society Conference in Helsinki

Date: OCTOBER 1-5, 2012
Venue: HELSINKI, Finland

Conference website

EVALUATION IN THE NETWORKED SOCIETY: NEW CONCEPTS, NEW CHALLENGES, NEW SOLUTIONS

The Tenth Biennial Conference of the European Evaluation Society will be the international evaluation event of the year. It will be held in Helsinki, Finland during 3-5 October 2012 (pre-conference workshops 1- 2 October).

Evaluators are living in times of unprecedented challenge and opportunity. The networked information environment is inducing fundamental changes in culture, politics and society. Whereas the industrial society was reliant on centralised, hierarchical, high cost information systems, the networked society is characterised by decentralised, voluntary and cheap information exchange.

The advent of social networking without borders will have fundamental implications for evaluation agendas and methods. First, it will redefine the value and legitimacy of evaluation in global social accountability networks and accelerate the internationalisation of evaluation. Second, evaluation cultures, structures and processes will have to deal  with the limitless quantity, speed and accessibility of information generated by new technologies, e.g. drawing useful meaning from huge data bases, assessing the validity of an exploding number of rating systems, league tables, etc. in ways consistent with democratic values of freedom of expression and protection of privacy.

The new information technologies offer new ways of making authority responsible and accountable as well as bringing real time citizen involvement and reliable information to bear on public policy making. What are the implications of an information economy that allows instant connectivity to thousands of program beneficiaries suddenly able to make their voices heard? Will the spread of mobile telephony to the weakest and most vulnerable members of society and the rising power of social networks act as evaluative and recuperative mechanisms or will they merely aggravate social instability? What are the risks of network capture by single or special interest groups and cooptation of evaluation?

The rise of the evaluation discipline is inextricably linked to the values central to any democratic society. How will these values be protected in a context where weak links and increasing inequalities have created new fissures in society? How will evaluation independence be protected against the pressures of vested interests intent on retaining control over the commanding heights of the society?

To help explore these and other issues relevant to the prospects of evaluation in Europe and beyond the Conference will stimulate evaluators to share ideas, insights and opinions about a wide range of topics that will throw light on the future roles of evaluation in the networked society. The Conference will help draw evaluation lessons learnt in distinct sectors and regions of the world. It will also examine the potential of alternative and mixed evaluation methods in diverse contexts and probe the challenges of assessing public interest in complex adaptive systems and networks.

To these ends the Conference will offer participants a wide choice of vehicles for the transmission of evaluation experience and knowledge: keynote speeches, paper presentations, panel debates, posters, etc.  As in past years the EES Conference will aim at a pluralistic agenda that respects the legitimacy of different standpoints, illuminates diverse perspectives and promotes principled debate. The Conference will also provide an opportunity for evaluation networks to interact and improve the coherence of their activities.

We look forward to welcoming you in Helsinki. It is one of the world leaders in modern design and it provides Europe with a world class high tech platform. It also boasts a 450 year history and lays claim to being the warmest, friendliest, most “laid back” city of Northern Europe. Its nearby archipelago of islands offers an ideal environment for sea cruises and its neighboring old growth forests provide an idyllic setting for restful nature walks. We promise you an enjoyable as well as a professionally rewarding time!!

Ian Davies, President, European Evaluation Society
Maria Bustelo, Vice President and President Elect, European Evaluation Society

DFID&UKES Workshop on Development and Evaluation: Practical Ways Forward.

 

Date:  WEDNESDAY 12 OCTOBER 2011
Venue: BIS Conference Centre, Victor ia, London

Objectives:

  • To examine the key contributions of evaluation to international development
  • To provide an update on the accountability framework for evaluation in the UK
  • To explore the role of professional development in building evaluation capacity

THIS ONE DAY EVENT will raise important issues in the world of development and evaluation. The workshop will offer the chance to hear from senior practitioners and will cover the theory and reality as experienced in many contexts. It will update the accountability framework with particular reference to HM Treasury Guidance for Evaluation (the Magenta Book).

A major challenge for organisations is to develop their own staff as evaluation professionals. UKES will offer international insights as well as an update on its own guidance. DFID will report on how it is going about building its own community of evaluators. These will be presented alongside those from the NGO and voluntary sector. The day is relevant to all individuals and organisations with an interest and experience of development and evaluation, including: Donors, Consultants, Public and private sector representatives, Academics, A wide range of professionals

Programme
The workshop will commence at 09.00 and close at 17.30.
Highlights will include:

  • Updates on the Independent Commission for Aid Impact (ICAI),  HM Treasury’s Magenta Book and the Cross Government Evaluation Group (CGEG)
  • How to evaluate in fragile states, conlict environments and other challenging situations
  •  Case studies of evaluation at different levels: national and local,  sector specific
  • How to build professional capacity: use of accreditation and adapting to it a range of organisations at government and civil society level

Registration
The workshop will be held at the BIS Conference Centre, 1 Victoria, Street, London SW1H OET.
The registration fees are as follows:
UKES members  £75.00 + VAT
Non-members  £100.00 + VAT
Registration and the full programme for the workshop are available from the website  www.profbriefings.co.uk/depwf
For any further information, contact the workshop administrators:
Professional Brieings
37 Star Street
Ware
Hertfordshire SG12 7AA
Telephone:
01920 487672
Email:  london@profbrieings.co.uk

RCTs for empowerment and accountability programmes

A GSDRC Helpdesk Research Report, Date: 01.04.2011, 14 pages, available as pdf.

Query: To what extent have randomised control trials been used to successfully measure the results of empowerment and accountability processes or programmes?
Enquirer: DFID
Helpdesk response
Key findings: This report examines the extent to which RCTs have been used successfully to measure empowerment and accountability processes and programmes. Field experiments present immense opportunities, but the report cautions that they are more suited to measuring short-term results with short causal chains and less suitable for complex interventions. The studies have also demonstrated divergent results, possibly due to different programme designs. The literature highlights that issues of scale, context, complexity, timeframe, coordination and bias in the selection of programmes also determine the degree of success reported. It argues that researchers using RCTs should make more effort to understand contextual issues, consider how experiments can be scaled up to measure higher-order processes, and focus more on learning. The report suggests strategies such as using qualitative methods, replicating studies in different contexts and using randomised methods with field activities to overcome the limitations in the literature.
Contents
1. Overview
2. General Literature (annotated bibliography)
3. Accountability Studies (annotated bibliography)
4. Empowerment Studies (annotated bibliography)

 

A results take-over of aid effectiveness? How to balance multiple or competing calls for more accountability

Date: 25 July 2011 12:00-13:30 (GMT+01 (BST))
Venue: British Academy, London

This debate will explore possible tensions – and opportunities – when donors seek to reassure domestic publics that aid is being spent well, while also endeavouring to support the needs and priorities of aid recipient countries and their citizens.

The language of results is not new – it is integral to the aid effectiveness agenda. But against the backdrop of growing financial constraints, it is receiving renewed emphasis in many donor countries. This debate will explore possible tensions, as well as opportunities, where donors seek to reassure domestic publics that aid is being spent well while they also endeavour to support the needs and priorities of aid recipient countries and their citizens. How can domestic accountability to both these constituencies be supported more effectively? Are there tensions between these different stakeholders and forms of accountability, and how can they be addressed?

Speakers:
Sarah Cliffe – Special Representative and Director, World Development Report 2011: Conflict, Security, and Development
Sue Unsworth – The Policy Practice, and ODI Board Member
Alan Hudson – Senior Policy Manager, Governance (Transparency & Accountability), ONE
John Morlu – former Auditor General, Liberia
Chair:  Alison Evans – Director, ODI

An ODI and BBC World Service Trust public event in the Busan and beyond: aid effectiveness in a new era series.

Click for more details           Register to attend this event

Promoting Voice and Choice: Exploring Innovations in Australian NGOAccountability for Development Effectiveness

– Exploring innovations in Australian NGO accountability for development effectiveness

by Chris Roche, ACFID research paper, 2010

From the Preface

“This research paper represents the latest chapter in a body of work, led by ACFID’s Development Practice Committee (DPC), focused on Australian NGO program quality and effectiveness. Over the past 10 years DPC has engaged the sector in a series of consultations and discrete research phases to define our effectiveness and identify the principles, program strategies, standards of engagement and organisational management practices which underpin it.

The objective of the current research was to capture and share cutting edge practice in demonstrating Australian NGO effectiveness through innovative forms of accountability and social learning, in which the views of those who are ultimately meant to benefit were central. ACFID member agencies participated through submitting examples of their attempts to improve downward accountability.

The findings presented in this report will contribute to ACFID member agencies’ journey of continual improvement of our collective effectiveness. It will do this through engaging with senior NGO managers and AusAID in the analysis of the findings, as well as contributing to the international work on CSO Development Effectiveness. The next research phase will be in partnership with an academic institution to undertake a more rigorous examination of a sample of the case studies and the organisational enablers and obstacles to improving our effectiveness.”

See also Chris Roche’s new guest posting on the (Australian based) Development Policy Centre’s Development Policy Blog, titled “Changing the rules of the game?” In this blog he follows up on issues raised in the above paper.

What Accountability Pressures do MPs in Africa Face and How Do They Respond? Evidence from Ghana

Source:  Lindberg, S., 2010,  Journal of Modern African Studies, Vol. 48, No. 1, pp. 117-142 VIA Governance and Social Development Resource Centre ]

Summary: What is the role of clientelism in African politics? How are MPs held accountable in Ghana? This article examines the daily accountability pressures and responses of Ghanaian Members of Parliament, the strength of the institution, and the formal and informal aspects of their role. It finds that these MPs devote a significant proportion of their time to producing and distributing private goods to constituents, and to constituent service. Marginal attention is devoted to legislating and executive oversight. Some MPs have been able to counter political clientelism, however, through civic education and by reformulating constituent expectations toward the production of collective, public goods.

Despite the rapid expansion in research on African politics, little is known about the daily behaviour of legislators, their accountability pressures and responses. This case study on Ghana finds that groups that hold MPs accountable include constituents, the local party, extended family, chiefs, religious leaders, civil society organisations (CSOs) and businesses (although these last two appear to exert little pressure). They require MPs to perform five core duties – the provision of private goods, constituency service, constituency representation, legislation and executive oversight:

  • Personal benefits and clientelistic goods: This type of accountability is the most common in MPs’ relationships with their constituents and is the one that puts the most pressure on MPs. Different groups have varied expectations of the form that such benefits should take. They range from monetary assistance (such as school fees or small business start-up costs) to the provision of jobs. There is a clear division between rural and urban constituencies; urban MPs have much greater resistance to constituent demands.
  • Constituency service as community development: This is an area of heavy emphasis for constituents and chiefs, causing MPs to spend a lot of their time lobbying ministers for development projects for their area.
  • Constituency representation: There is a strong expectation of MPs to be heard in debates and to have a media presence. This is anchored in the traditional notion of family heads ‘speaking up’ for their people.
  • Legislation and executive oversight: It is primarily the executive which exerts pressure on MPs regarding legislation, particularly regarding voting conformity (by withholding of seats on lucrative tender boards). Active public debate and scrutiny are compromised due to the strength of the executive over the legislature.

The clientelistic relationship between the MP and constituents stems from traditional notions of ‘head of the family’, one who has a moral obligation to solve problems for followers in need. The hybrid role of MP as family head places enormous pressures on officeholders to be responsive to constituents’ needs and priorities. MPs face the dual sanctions of losing office at election time and the informal shame, harassment and loss of status within the context of family and community. However, some MPs have been successful in translating the informal family head role into pressure for the production of collective goods by engaging in civic education and raising political awareness:

  • MPs that have held regular community meetings to explain legislative business and policy have been successful in developing a strong voice for collective goods.
  • Focusing expectations on collective, public, and national-level goods has significantly reduced pressure on MPs to personally provide private goods.
  • It has also increased constituent perception of the importance of legislative behaviour for chances of re-election. This in turn has reduced clientelistic behaviour and promoted democratic responsiveness.

Access full text: available online

Value for Money: How are other donors approaching ‘value for money’ in their aid programming?

….A question posed to the Research Helpdesk of the Governance and Social Development Resource Centre

“Key findings: DFID appears to have gone the furthest among aid agencies in developing the concept of ‘value for money’ (VFM). It is the only agency that explicitly uses the terminology frequently in its policies and procedures and has a Value for Money department. DFID’s approach to VFM involves assessing whether level of results achieved represent good value for money against the costs incurred. Processes include the use of logframes, economic appraisals and portfolio reviews. Newer initiatives include the adoption of a business case model for project approval and the development of unit cost metrics in key sectors. Other donors, while not explicitly adopting ‘value for money’ terminology, aim to achieve VFM through rigorous economic analysis and results-based management.

The ‘value for money’ agenda has also been linked to efforts to improve accountability and transparency. This requires strengthening audit bodies, parliaments, media, civil societies and independent watchdogs such that they can hold government to account for spending. It also involves greater transparency, in particular publishing information on projects and allocation of funds.”

Full response: http://www.gsdrc.org/docs/open/HD712.pdf

Review of Impact and Effectiveness of Transparency and Accountability Initiatives

[from the IDS website]
McGee,R. and Gaventa, J. – 23-Nov-10
Download this publication free of charge

Transparency and accountability have emerged over the past decade as key ways to address both developmental failures and democratic deficits. In the development context, the argument is that through greater accountability, ‘leaky pipes’ of corruption and inefficiency will be repaired, aid will be channelled more effectively, and in turn development initiatives will produce greater and more visible results. For scholars and practitioners of democracy, a parallel argument holds that following the twentieth-century wave of democratisation, democracy now has to ‘deliver the goods’, especially in terms of material outcomes, and that new forms of democratic accountability can help it do so. While traditional forms of state-led accountability are increasingly found to be inadequate, thousands of multi-stakeholder and citizen-led approaches have come to the fore, to supplement or supplant them.Despite their rapid growth, and the growing donor support they receive, little attention has been paid to the impact and effectiveness of these new transparency and accountability initiatives. Responding to this gap, this report, based on a review of literature and experience across the field with special focus on five sectors of transparency and accountability work, aims to improve understanding among policy-makers and practitioners of the available evidence and identify gaps in knowledge to inform a longer-term research agenda. Commissioned by the Policy Research Fund of the UK Department of International Development (DFID), this project also hopes to inform the Transparency and Accountability Initiative, a new donor collaborative that includes the Ford Foundation, Hivos, the International Budget Partnership, the Omidyar Network, the Open Society Institute, the Revenue Watch Institute, and the William and Flora Hewlett Foundation.

Download Review of Impact and Effectiveness of Transparency and Accountability Initiatives – Executive summary

Review of Impact and Effectiveness of Transparency and Accountability Initiatives – Service delivery

Review of Impact and Effectiveness of Transparency and Accountability Initiatives – Budget processes

Review of Impact and Effectiveness of Transparency and Accountability Initiatives – Freedom of information

Review of Impact and Effectiveness of Transparency and Accountability Initiatives – Natural resource governance

Review of Impact and Effectiveness of Transparency and Accountability Initiatives – Aid transparency

Review of Impact and Effectiveness of Transparency and Accountability Initiatives – Abstracts

%d bloggers like this: