Date: OCTOBER 1-5, 2012
Venue: HELSINKI, Finland

Conference website

EVALUATION IN THE NETWORKED SOCIETY: NEW CONCEPTS, NEW CHALLENGES, NEW SOLUTIONS

The Tenth Biennial Conference of the European Evaluation Society will be the international evaluation event of the year. It will be held in Helsinki, Finland during 3-5 October 2012 (pre-conference workshops 1- 2 October).

Evaluators are living in times of unprecedented challenge and opportunity. The networked information environment is inducing fundamental changes in culture, politics and society. Whereas the industrial society was reliant on centralised, hierarchical, high cost information systems, the networked society is characterised by decentralised, voluntary and cheap information exchange.

The advent of social networking without borders will have fundamental implications for evaluation agendas and methods. First, it will redefine the value and legitimacy of evaluation in global social accountability networks and accelerate the internationalisation of evaluation. Second, evaluation cultures, structures and processes will have to deal  with the limitless quantity, speed and accessibility of information generated by new technologies, e.g. drawing useful meaning from huge data bases, assessing the validity of an exploding number of rating systems, league tables, etc. in ways consistent with democratic values of freedom of expression and protection of privacy.

The new information technologies offer new ways of making authority responsible and accountable as well as bringing real time citizen involvement and reliable information to bear on public policy making. What are the implications of an information economy that allows instant connectivity to thousands of program beneficiaries suddenly able to make their voices heard? Will the spread of mobile telephony to the weakest and most vulnerable members of society and the rising power of social networks act as evaluative and recuperative mechanisms or will they merely aggravate social instability? What are the risks of network capture by single or special interest groups and cooptation of evaluation?

The rise of the evaluation discipline is inextricably linked to the values central to any democratic society. How will these values be protected in a context where weak links and increasing inequalities have created new fissures in society? How will evaluation independence be protected against the pressures of vested interests intent on retaining control over the commanding heights of the society?

To help explore these and other issues relevant to the prospects of evaluation in Europe and beyond the Conference will stimulate evaluators to share ideas, insights and opinions about a wide range of topics that will throw light on the future roles of evaluation in the networked society. The Conference will help draw evaluation lessons learnt in distinct sectors and regions of the world. It will also examine the potential of alternative and mixed evaluation methods in diverse contexts and probe the challenges of assessing public interest in complex adaptive systems and networks.

To these ends the Conference will offer participants a wide choice of vehicles for the transmission of evaluation experience and knowledge: keynote speeches, paper presentations, panel debates, posters, etc.  As in past years the EES Conference will aim at a pluralistic agenda that respects the legitimacy of different standpoints, illuminates diverse perspectives and promotes principled debate. The Conference will also provide an opportunity for evaluation networks to interact and improve the coherence of their activities.

We look forward to welcoming you in Helsinki. It is one of the world leaders in modern design and it provides Europe with a world class high tech platform. It also boasts a 450 year history and lays claim to being the warmest, friendliest, most “laid back” city of Northern Europe. Its nearby archipelago of islands offers an ideal environment for sea cruises and its neighboring old growth forests provide an idyllic setting for restful nature walks. We promise you an enjoyable as well as a professionally rewarding time!!

Ian Davies, President, European Evaluation Society
Maria Bustelo, Vice President and President Elect, European Evaluation Society

Date:  WEDNESDAY 12 OCTOBER 2011
Venue: BIS Conference Centre, Victor ia, London

Objectives:

• To examine the key contributions of evaluation to international development
• To provide an update on the accountability framework for evaluation in the UK
• To explore the role of professional development in building evaluation capacity

THIS ONE DAY EVENT will raise important issues in the world of development and evaluation. The workshop will offer the chance to hear from senior practitioners and will cover the theory and reality as experienced in many contexts. It will update the accountability framework with particular reference to HM Treasury Guidance for Evaluation (the Magenta Book).

A major challenge for organisations is to develop their own staff as evaluation professionals. UKES will offer international insights as well as an update on its own guidance. DFID will report on how it is going about building its own community of evaluators. These will be presented alongside those from the NGO and voluntary sector. The day is relevant to all individuals and organisations with an interest and experience of development and evaluation, including: Donors, Consultants, Public and private sector representatives, Academics, A wide range of professionals

Programme
The workshop will commence at 09.00 and close at 17.30.
Highlights will include:

• Updates on the Independent Commission for Aid Impact (ICAI),  HM Treasury’s Magenta Book and the Cross Government Evaluation Group (CGEG)
• How to evaluate in fragile states, conlict environments and other challenging situations
•  Case studies of evaluation at different levels: national and local,  sector specific
• How to build professional capacity: use of accreditation and adapting to it a range of organisations at government and civil society level

Registration
The workshop will be held at the BIS Conference Centre, 1 Victoria, Street, London SW1H OET.
The registration fees are as follows:
UKES members  £75.00 + VAT
Non-members  £100.00 + VAT
Registration and the full programme for the workshop are available from the website  www.profbriefings.co.uk/depwf
For any further information, contact the workshop administrators:
Professional Brieings
37 Star Street
Ware
Hertfordshire SG12 7AA
Telephone:
01920 487672
Email:  london@profbrieings.co.uk

A GSDRC Helpdesk Research Report, Date: 01.04.2011, 14 pages, available as pdf.

Query: To what extent have randomised control trials been used to successfully measure the results of empowerment and accountability processes or programmes?
Enquirer: DFID
Helpdesk response
Key findings: This report examines the extent to which RCTs have been used successfully to measure empowerment and accountability processes and programmes. Field experiments present immense opportunities, but the report cautions that they are more suited to measuring short-term results with short causal chains and less suitable for complex interventions. The studies have also demonstrated divergent results, possibly due to different programme designs. The literature highlights that issues of scale, context, complexity, timeframe, coordination and bias in the selection of programmes also determine the degree of success reported. It argues that researchers using RCTs should make more effort to understand contextual issues, consider how experiments can be scaled up to measure higher-order processes, and focus more on learning. The report suggests strategies such as using qualitative methods, replicating studies in different contexts and using randomised methods with field activities to overcome the limitations in the literature.
Contents
1. Overview
2. General Literature (annotated bibliography)
3. Accountability Studies (annotated bibliography)
4. Empowerment Studies (annotated bibliography)

Date: 25 July 2011 12:00-13:30 (GMT+01 (BST))
Venue: British Academy, London

This debate will explore possible tensions – and opportunities – when donors seek to reassure domestic publics that aid is being spent well, while also endeavouring to support the needs and priorities of aid recipient countries and their citizens.

The language of results is not new – it is integral to the aid effectiveness agenda. But against the backdrop of growing financial constraints, it is receiving renewed emphasis in many donor countries. This debate will explore possible tensions, as well as opportunities, where donors seek to reassure domestic publics that aid is being spent well while they also endeavour to support the needs and priorities of aid recipient countries and their citizens. How can domestic accountability to both these constituencies be supported more effectively? Are there tensions between these different stakeholders and forms of accountability, and how can they be addressed?

Speakers:
Sarah Cliffe – Special Representative and Director, World Development Report 2011: Conflict, Security, and Development
Sue Unsworth – The Policy Practice, and ODI Board Member
Alan Hudson – Senior Policy Manager, Governance (Transparency & Accountability), ONE
John Morlu – former Auditor General, Liberia
Chair:  Alison Evans – Director, ODI

An ODI and BBC World Service Trust public event in the Busan and beyond: aid effectiveness in a new era series.

Click for more details           Register to attend this event

- Exploring innovations in Australian NGO accountability for development effectiveness

by Chris Roche, ACFID research paper, 2010

From the Preface

“This research paper represents the latest chapter in a body of work, led by ACFID’s Development Practice Committee (DPC), focused on Australian NGO program quality and effectiveness. Over the past 10 years DPC has engaged the sector in a series of consultations and discrete research phases to define our effectiveness and identify the principles, program strategies, standards of engagement and organisational management practices which underpin it.

The objective of the current research was to capture and share cutting edge practice in demonstrating Australian NGO effectiveness through innovative forms of accountability and social learning, in which the views of those who are ultimately meant to benefit were central. ACFID member agencies participated through submitting examples of their attempts to improve downward accountability.

The findings presented in this report will contribute to ACFID member agencies’ journey of continual improvement of our collective effectiveness. It will do this through engaging with senior NGO managers and AusAID in the analysis of the findings, as well as contributing to the international work on CSO Development Effectiveness. The next research phase will be in partnership with an academic institution to undertake a more rigorous examination of a sample of the case studies and the organisational enablers and obstacles to improving our effectiveness.”

See also Chris Roche’s new guest posting on the (Australian based) Development Policy Centre’s Development Policy Blog, titled “Changing the rules of the game?” In this blog he follows up on issues raised in the above paper.

Source:  Lindberg, S., 2010,  Journal of Modern African Studies, Vol. 48, No. 1, pp. 117-142 VIA Governance and Social Development Resource Centre ]

Summary: What is the role of clientelism in African politics? How are MPs held accountable in Ghana? This article examines the daily accountability pressures and responses of Ghanaian Members of Parliament, the strength of the institution, and the formal and informal aspects of their role. It finds that these MPs devote a significant proportion of their time to producing and distributing private goods to constituents, and to constituent service. Marginal attention is devoted to legislating and executive oversight. Some MPs have been able to counter political clientelism, however, through civic education and by reformulating constituent expectations toward the production of collective, public goods.

Despite the rapid expansion in research on African politics, little is known about the daily behaviour of legislators, their accountability pressures and responses. This case study on Ghana finds that groups that hold MPs accountable include constituents, the local party, extended family, chiefs, religious leaders, civil society organisations (CSOs) and businesses (although these last two appear to exert little pressure). They require MPs to perform five core duties – the provision of private goods, constituency service, constituency representation, legislation and executive oversight:

• Personal benefits and clientelistic goods: This type of accountability is the most common in MPs’ relationships with their constituents and is the one that puts the most pressure on MPs. Different groups have varied expectations of the form that such benefits should take. They range from monetary assistance (such as school fees or small business start-up costs) to the provision of jobs. There is a clear division between rural and urban constituencies; urban MPs have much greater resistance to constituent demands.
• Constituency service as community development: This is an area of heavy emphasis for constituents and chiefs, causing MPs to spend a lot of their time lobbying ministers for development projects for their area.
• Constituency representation: There is a strong expectation of MPs to be heard in debates and to have a media presence. This is anchored in the traditional notion of family heads ‘speaking up’ for their people.
• Legislation and executive oversight: It is primarily the executive which exerts pressure on MPs regarding legislation, particularly regarding voting conformity (by withholding of seats on lucrative tender boards). Active public debate and scrutiny are compromised due to the strength of the executive over the legislature.

The clientelistic relationship between the MP and constituents stems from traditional notions of ‘head of the family’, one who has a moral obligation to solve problems for followers in need. The hybrid role of MP as family head places enormous pressures on officeholders to be responsive to constituents’ needs and priorities. MPs face the dual sanctions of losing office at election time and the informal shame, harassment and loss of status within the context of family and community. However, some MPs have been successful in translating the informal family head role into pressure for the production of collective goods by engaging in civic education and raising political awareness:

• MPs that have held regular community meetings to explain legislative business and policy have been successful in developing a strong voice for collective goods.
• Focusing expectations on collective, public, and national-level goods has significantly reduced pressure on MPs to personally provide private goods.
• It has also increased constituent perception of the importance of legislative behaviour for chances of re-election. This in turn has reduced clientelistic behaviour and promoted democratic responsiveness.

Access full text: available online

….A question posed to the Research Helpdesk of the Governance and Social Development Resource Centre

“Key findings: DFID appears to have gone the furthest among aid agencies in developing the concept of ‘value for money’ (VFM). It is the only agency that explicitly uses the terminology frequently in its policies and procedures and has a Value for Money department. DFID’s approach to VFM involves assessing whether level of results achieved represent good value for money against the costs incurred. Processes include the use of logframes, economic appraisals and portfolio reviews. Newer initiatives include the adoption of a business case model for project approval and the development of unit cost metrics in key sectors. Other donors, while not explicitly adopting ‘value for money’ terminology, aim to achieve VFM through rigorous economic analysis and results-based management.

The ‘value for money’ agenda has also been linked to efforts to improve accountability and transparency. This requires strengthening audit bodies, parliaments, media, civil societies and independent watchdogs such that they can hold government to account for spending. It also involves greater transparency, in particular publishing information on projects and allocation of funds.”

Full response: http://www.gsdrc.org/docs/open/HD712.pdf

“In this essay, Andrew Natsios describes what he sees as the most disruptive obstacles to development work in agencies such as USAID: layers and layers of bureaucracy. He gives a first-hand account of how this “counter-bureaucracy” disfigures USAID’s development practice and even compromises U.S. national security objectives. Most of all, he argues, the counter-bureaucracy’s emphasis on easy measurement is at odds with the fact that transformational programs are often the least measurable and involve elements of risk and uncertainty.

To overcome counter-bureaucracy barriers, Natsios suggests implementing a new measurement system, reducing the layers of oversight and regulation, and aligning programmatic goals with organizational incentives. Unless policymakers address the issue, he says, U.S. aid programs will be unable to implement serious development programs while complying with the demands of Washington.”

Revised 07-13-2010

See also “BEYOND SUCCESS STORIES: MONITORING & EVALUATION FOR FOREIGN ASSISTANCE RESULTS Posted on 8 June, 2009

“On the 22nd September, Rosalind Eyben organised a meeting of some seventy development practitioners and researchers worried about the current trend for funding organisations to support only those programmes designed to deliver easily measurable results, although these may not support transformative processes of positive and sustainable changes in people’s lives.

Following on from a major conference in May in the Netherlands about evaluative practices in relation to social transformation (http://evaluationrevisited.wordpress.com/), the meeting took
the first steps in strategizing collectively in support of these practices”  Attached is Rosalind’s brief report of the meeting.

PS: 11 October 2010. See the latest posting by Ros Eyben on this topic here, on the Hauser Centre blog