by Scott Page, 2011. Available on Google Books Princeton University Press, 14/07/2011 – 296 pages

Abstract: This book provides an introduction to the role of diversity in complex adaptive systems. A complex system–such as an economy or a tropical ecosystem–consists of interacting adaptive entities that produce dynamic patterns and structures. Diversity plays a different role in a complex system than it does in an equilibrium system, where it often merely produces variation around the mean for performance measures. In complex adaptive systems, diversity makes fundamental contributions to system performance. Scott Page gives a concise primer on how diversity happens, how it is maintained, and how it affects complex systems. He explains how diversity underpins system level robustness, allowing for multiple responses to external shocks and internal adaptations; how it provides the seeds for large events by creating outliers that fuel tipping points; and how it drives novelty and innovation. Page looks at the different kinds of diversity–variations within and across types, and distinct community compositions and interaction structures–and covers the evolution of diversity within complex systems and the factors that determine the amount of maintained diversity within a system.Provides a concise and accessible introduction. Shows how diversity underpins robustness and fuels tipping points .Covers all types of diversity. The essential primer on diversity in complex adaptive systems.

RD Comment: This book is very useful for thinking about the measurement of diversity. In 2000 I wrote a paper “Does Empowerment Start At Home? And If So, How Will We Recognise It?” in which I argued that…

“At the population level, diversity of behaviour can be seen as a gross indicator of agency (of the ability to make choices), relative to homogenous behaviour by the same set of people. Diversity of behaviour suggests there is a range of possibilities which individuals can pursue. At the other extreme is standardisation of behaviour, which we often associate with limited choice. The most notable example being perhaps that of an army. An army is a highly organised structure where individuality is not encouraged, and where standardised and predictable behaviour is very important. Like the term “NGO” or “non-profit”, diversity is defined by something that it is not -  a condition where there is no common constraint, which would otherwise lead to a homogeneity of response. Homogeneity of behaviour may arise from various sources of constraint. A flood may force all farmers in a large area to move their animals to the high ground. Everybody’s responses are the same, when compared to what they would be doing on normal day. At a certain time of the year all farmers may be planting the same crop. Here homogeneity of practice may reflect common constraints arising from a combination of sources: the nature of the physical environment, and the nature of particular local economies. Constraints on diversity can also arise within the assisting organisation. Credit programs can impose rules on loan use, specific repayment schedules and loan terms, as well as limiting when access to credit is available, or how quickly approval will be give.”

See also…

• Page’s 2006 book “The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies“
• Scott Page on Leveraging Diversity on YouTube, in 2010
• Scott Page on Resilience (part 1), (part 2) (part 3) (part 4) (part 5) on YouTube, in 2009

On 19 October 2011, Oxfam Australia hosted a ‘Big Push Forward‘ event in Melbourne with the co-conveners of this initiative – Rosalind Eyben and Irene Guijt. Sixty development practitioners, including AusAid staff and academics came together to discuss whether the concerns voiced by the Big Push Forward project are relevant in Australia.

 HOW RELEVANT ARE THE ISSUES TO AUSTRALIA?

Following an introduction from Rosalind and Irene, we  had short inputs from three speakers on how these issues resonated in our part of the world.  Dennis Altman, from the Institute of Human Security, at La Trobe University suggested that the neo-liberal language which permeates Western society has been recast in the development world. into an auditing culture, focusing on evaluation, monitoring, and counting beans.  Marc Purcell the CEO of Australia’s International NGO umbrella group ACFID noted that the commitment to international aid in Australia is extremely brittle, and that the public debate about aid in Australia has led to a deep anxiety in government about how the aid programme is being perceived. But he argued that maybe it’s no bad thing for economists to look at the work of ‘pampered NGOs’. Jess Dart, the Managing Director of consulting company Clear Horizon, felt that whilst Australian NGOs do more internal evaluation than most there was a view expressed at this year’s Australasian Evaluation Conference that ‘development is the cowboy of evaluation’.  If we can’t tell the story of what we’ve done, people will ask for results. There are lots of really good methods out there and we can use these to offer solid alternatives to tell more complex stories of transformation.

OZIFYING THE THEMES

We explored the seven themes that the Big Push Forward website is focusing upon and discussed how  to ‘ozify’ them.  Groups worked on specific clusters to generate ideas for ‘pushing forward’.  This included 5 of the original 7 clusters and one new one focused on Program Design and Evaluation. After a process of ritual dissent, facilitated by Irene, final ideas were presented.

1.       Developing different methods of reporting

• “Making sense of it all”. Look at alternative forms of aggregation and sense-making using the cluster to make a call for ideas about what’s going on and how to engage others.  This should include exploring means of more direct citizen-to-citizen dialogue

2.  Reclaiming value for money

• “ Research to Surface Valuing and Values”. Collaborate amongst agencies, to find out what has been the experience of value for money in Australia and AusAID and the piloting of new approaches.  This would allow the main approaches to be understood, and the values that underpin them to be better understood.

3.  Organisational learning and reflective practice

• “A Practitioners’ forum to build commitment to organisational learning and reflective practice” . Set up an inter-agency  forum on M&E to look at: methodologies for reflective practice and organisational learning, build capacities to ensure better reflection, documenting different models for reflective practice, discussions around conceptual model and practical implications.

4.  Communicating to the public

• “Market complexity to the marketers”. Do more internally in our organisations with those that speak to the public and help them communicate the day-to-day practice on the ground.  We could explore the use of community forums to engage the public in terms of complexity of aid.

 5.   Challenging dominant discourses

• “ Unsettling storytelling from Aboriginal Australia”. Use story-telling and narrative methods to try to unsettle some of the debates about Results Based Management in order to ask the difficult questions e.g. what does development actually mean and what does it mean to those being developed? This would highlight the difficulties and create better understanding through creating this connection for Australian people.

6.  Program design

• “ Peer and Public Review for Good Practice”. Develop a forum that creates more space for program designs that hold the qualities of community participation, co-creation, & voice in determining the results. Use a virtual peer process from different agencies and public peer review enabling commentary from others.

 WRAPPING IT UP

In our final session four people were asked to share their reflections on what they heard

Chris Nelson from AusAID and soon to be with the World Bank Justice for the Poor program suggested there were lots of ingredients but we needed to bake the cake. He also noted that there were several opportunities to engage with AusAID in the next six months in particular as there are some gaps and a need for innovation in their current results framework. He also advised of the importance of building internal networks and coalitions within institutions on this agenda.

Rosalind Eyben remarked that there seemed to be several spaces for BPF engagement: internally in our own organisations; in communities of practice between organisations; with AusAID and internationally as Australia engages more strategically within the international aid system and international policy fora. Rosalind in particular liked the idea of story-telling here in Australia in order to communicate what actually happens to people and their small organisations as the results agenda impinges upon them.

Patricia Rogers, Professor in Public Sector Evaluation at RMIT University, noted that many of the ideas suggested are based on sharing good examples of practice.  She identified some common issues that need to be addressed: Who says it’s good practice? How do we distinguish between ‘this sounds like a good idea’ to ‘this has been done and we can show it makes a difference’? How do we create space to do things differently? How might we leverage resources together and with our partners? (see for example BetterEvaluation.org)

Jo Crawford, a Research and Policy Advisor at the International Women’s Development Agency, felt it critical to consider the tension between acting now on this agenda versus taking the time to define what we need to do.  Jo made a plea for process and participation in these conversations. Otherwise values will be narrowly defined. This requires a diversity of views and voices around the table when frameworks are being established

LOTS OF ENERGY

This was a day which demonstrated a lot of energy, ideas and willingness to engage on this agenda. After the meeting one of the participants Helen Sullivan, the new Director of the Centre for Public Policy (CPP) at Melbourne University wrote the following after attending a couple of other events which touched upon the same issues.

“…there is useful work that CPP can do in helping policy makers (politicians, professionals and ‘publics’) navigate the policy world they inhabit, which is not the world as it is often constructed in text books and guides where rational, reasonable actors with the necessary time and space define, design and deliberate policy problems and solutions to a ‘good’ end, but rather comprises actors who are motivated by reason and emotion, driven by evidence and values, and where the boundary between politics and policy is often blurred.  This is not to deny the potential value of evidence-based policymaking but to assert that politics, evidence and democratic institutions co-exist and that each are informed by values and power relationships that can facilitate and limit the quality of policy and policymaking.”

Hear! Hear!

We urge all participants who were at the Melbourne meeting, and others interested in this debate in Australia, to join the Big Push Forward and share your thoughts and experiences.

Chris Roche and Susan Hornbeck

25 October 2011

Michael Quinn Patton, August 2011. Sage publications

Publisher’s description:

“Based on Michael Quinn Patton’s best-selling Utilization-Focused Evaluation, this briefer book provides an overall framework and essential checklist steps for designing and conducting evaluations that actually get used. The new material and innovative graphics present the utilization-focused evaluation process as a complex adaptive system, incorporating current understandings about systems thinking and complexity concepts. The book integrates theory and practice, is based on both research and professional experience, and offers new case examples and cartoons with Patton’s signature humor. ”

Contents

Date: 17-18 October 2011
Venue: ANU, Canberra, Australia

Concept Note, Chris Roche & Linda Kelly, 4 August 2011

The Developmental Leadership Program (DLP)[1] addresses an important gap in international thinking and policy about the critical role played by leaders, elites and coalitions in the politics of development. At the core of DLP thinking is the proposition that political processes shape developmental outcomes at all levels and in all aspects of society: at national and sub-national levels and in all sectors and issue areas.

Initial findings of the DLP research program confirm that development is a political process and that leadership and agency matter. This is of course not new, but the DLP research provides important insights into how, in particular, leadership, elites and formal and informal coalitions can play a particularly important and under-recognized role in institutional formation (or establishing the ‘rules of the game’), policy reform and development processes[2].

International aid therefore needs to engage effectively with political processes. It needs to be flexible and be able to respond when opportunities open up. It needs to avoid the danger of bolstering harmful political settlements.

Furthermore Monitoring & Evaluation (M&E) mechanisms need to be improved and made compatible with flexible programming and recognize the importance of ‘process’ as well as outcomes. Donors should invest in a range of monitoring and evaluation techniques and approaches which are more aligned with the kinds of non-linear and unpredictable processes which characterise the kinds of political processes which drive positive developmental outcomes. This is important because it can be argued that, at best, current approaches are simply not appropriate to monitor the kinds of processes DLP research indicates are important; or, at worst, they offer few incentives to international assistance agencies to support the processes that actually lead to developmental outcomes

At the same time aid donors are increasingly under pressure to adopt results-based management processes and value for money assessments. There is also growing demand for more rigorous monitoring and evaluation, and greater transparency and public accountability[3].

In the light of these issues DLP will host a seminar on the topic of ‘Monitoring and Evaluation as if Politics Matters’.

The purpose of this event is to explore the strengths and weaknesses of current approaches to M&E, and suggest what ‘intelligent mixes’ of different approaches, methods and innovations might be tested by DLP in order to better track and understand how international assistance can best support the emergence and success of developmental leaderships, elites and coalitions.

In particular the workshop will explore what effective mix of different approaches would be required to:

• Capture changes in agency, leadership, elite formation, and coalitions, i.e. the processes that generate developmental leadership
• Evaluate the developmental outcomes (positive and negative) including in particular institutional and policy reform, associated with these changes
• Assess the contribution of international agencies and aid to these processes
• Address the challenges and opportunities that emerge from the results-based, ‘value for money’, and aid-and-development effectiveness agendas which are prominent concerns in donor thinking and policies at present.

The aim of the workshop will be to produce:

• A limited mixture of ‘testable’ methods, which would be trialled by DLP and its partners over the coming period; and
• Some guidance to the international development community about the value of different evaluative approaches for different development processes in the form of a workshop report.

It is our belief that no single method will be suitable to measure the effectiveness of this work.  We will need a range and mix of approaches that draws from the best quantitative, qualitative and participatory approaches, as well as innovations being generated through social network analysis and social media.

We believe that we therefore need a mix of expertise covering a range of M&E approaches.  The workshop will therefore include experts and researchers with different approaches to Monitoring and Evaluation and political analysis, staff responsible for managing quality and M&E processes in DLP partners, including AusAID, and development practitioners working on these issues from developing countries. It would involve between 15 and 20 people.

We envisage a maximum of 4/5 speakers with an appropriate balance between the following areas.

• Theories of Change and Complexity analysis
• Gendered M&E
• Social Network Analysis and M&E of Governance questions
• Quantitative analysis, Experimental and Randomised Control Trials
• Citizen Voice and crowd-sourcing options to generate feedback
• Specialists in assessing institutional and organizational change
• Specialists with expertise in different approaches to analyzing ‘Value for Money’ questions
  • Participatory M&E and Impact Assessment

Currently the confirmed speakers are:

• Patricia Rogers, Professor in Public Sector Evaluation and leader of the research program in Evidence Based Policy and Practice in the Sustainable Health and Well-Being Research Institute at RMIT, Melbourne (www.rmit.edu.au/rd/sustainablehealthandwellbeing) and author of ‘Purposeful Program Theory’ with Sue Funnell. Patricia is also a key player in the Impact Evaluation for Development initiative. ,
• Rosalind Eyben, Fellow at the Institute of Development Studies, University of Sussex and member of the Participation, Power and Social Change team (http://www.ids.ac.uk/index.cfm?objectid=BB000CE0-5056-8171-7B0079D6144E8C25) an ex Chief Social Development Advisor at DFID. Rosalind is the instigator of the Big Push Forward initiative (http://bigpushforward.wordpress.com/),
• Irene Guijt is an international expert in the application of learning-oriented knowledge processes in international development (http://www.cognitive-edge.com/directoryrecord.php?ID=4621).  She has provided research, advisory, and training services on social and organizational learning, in particular being known for her work on innovative thinking on monitoring, evaluation and learning, most recently engaged in experimenting with ‘SenseMaker’ (http://www.globalgiving.org/stories/))
  • Shawn Powers,  Policy Manager at the Abdul Latif Jameel Poverty Action Lab at MIT a research network specializing in randomized evaluations of social programs (http://www.povertyactionlab.org/powers)
  • Dr. Maria Melody Garcia is a researcher at the German Development Institute / Deutsches Institut für Entwicklungspolitik (DIE) working on impact evaluation . She is the author of Micro-Methods in Evaluating Governance Interventions, a recent overview of  impact evaluation methodologies currently used in the field of governance. focusing specifically on experimental and quasi-experimental designs see http://www.bmz.de/en/zentrales_downloadarchiv/erfolg/BMZ_WP_Micro.pdf
  • Esme Gaussen, is an independent monitoring, evaluation and value for money specialist currently based in South Africa.  She has previously worked for the UK National Audit Office on value for money assessments of DFID-supported programmes, and developed M+E tools for the NAO’s International Technical Cooperation Programme.

The format of the workshop will involve an introduction to DLP’s research and findings by Adrian Leftwich, and a short input on the challenges raised by DLP’s partners in the first M&E workshop (by Chris Roche or Linda Kelly). We are then proposing having pairs of experts responding to these challenges suggesting how they might be addressed by DLP, from different perspectives. Speakers would be asked to prepare presnetations in advance. This would be interspersed with Questions and Answers and discussion.

This would lead on to more workshop type sessions exploring testable approaches and methods. A final session might include sharing these findings with a broader range of development practitioners and experts in Canberra.

We envisage a workshop report, that will be part of the DLP research papers series, outlining an overall approach for the M&E of ‘working politically’ as described by DLP.

Chris Roche & Linda Kelly on behalf of the Developmental Leadership Program, 31 August 2011 (they are interested to hear about any research, experience or ideas that may be relevant to the topic)

Kim Forss, Mita Marra and Robert Schwartz, editors. Transaction Publishers, New Brunswick. May 2011. Available via Amazon

“Problem-solving by policy initiative has come to stay. Overarching policy intiatives are now standard modus operandi for governmental and non-governmental organisations. But complex policy initiatives are not only reserved for the big challenges of our times, but are used for matters such as school achievement, regional development, urban planning, public health and safety. As policy and the ensuing implementation tends to be more complex than simple project and programme management, the task of  evaluation has also become more complex.”

“The book begins with a theoretical and conceptual explanation of complexity and how that affects evaluation. The authors make the distinction between, on the hand, the common-sense understanding of complexity  as something that is generally messy, involves many actors and has unclear boundaries and overlapping roles; and on the hand, complexity as a specific term from systems sciences, which implies non-linear relationships between phenomena. It is particularly in the latter sense that an understanding of complexity has a bearing on evaluation design in respect of how evaluators approach the question of impact.”

“The book presents nine case studies that cover a wide variety of policy initiatives, in public health (smoking prevention), homelessness, child labour, regional development, international development cooperation, the HIV/AIDs pandemic, and international development cooperation. The use of case studies sheds light on the conceptual ideas at work in organisations addressing some of the world’s largest and most varied problems.”

“The evaluation processes described here commonly seek a balance between order and chaos. The interaction of four elements – simplicity, inventiveness, flexibility, and specificity – allows complex platterns to emerge. The case studies illustrate this framework and provide a number of examples of practical management of complexity in light of contingency theories of the evaluation process itself. These theories in turn match the complexity of the evaluated policies, strategies and programmes. The case studies do not pretend to illustrate perfect evaluation processes, the focus is on learning and on seeking patterns that have proved satisfactory and where the evaluation findings have been robust an trustworthy.”

“The contingency theory approach of the book underscores a point also made in the Foreword by Professor Elliot Stern: “In a world characterised by interdependence, emergent proerties, unpredictable change, and indeterminate outcomes, how could evaluation be immune?” The answer lies in the choice of methods as much as in the overall strategy and approach of  evaluation.”

Marc Maxson, Irene Guijt, and others, 2010. GlobalGiving Foundation (supported by Rockefeller Foundation). Available as pdf.  See also the related website.

["Real Book" = The Real Book is a central part of the culture of playing music where improvisation is essential. Real books are not for beginners: the reader interprets scant notation, and builds on her own familiarity with chords. The Real Book allows musicians to play an approximate version of hundreds of new songs quickly]

About this book
“This is a collection of narratives that serve to illustrate some not-so-obvious lessons that affected our story pilot project in Kenya. We gathered a large body of community stories that revealed what people in various communities believed they  needed, what services they were getting, and what they would like to see happen in the future. By combining many brief narratives with a few contextual questions we were able to compare and analyze thousands of stories. Taken together, these stories and their meanings provide a perspective with both depth and breadth: Broad enough to inform an organization’s strategic thinking about the root causes of social ailments2, yet deep and real enough to provoke specific and immediate follow-up actions by the local organizations of whom community members speak.

We believe that local people are the “experts” on what they want and know who has (or has not) been helping them. And like democracy, letting them define the problems and solutions that deserve to be discussed is the best method we’ve found for aggregating that knowledge. Professionals working in this field can draw upon the wisdom of this crowd for understanding the local context, and build upon what they know. Community efforts are complex, and our aim is not to predict the future, but help local leaders manage the present. If projects are observed from many angles – especially by those for whom success affects their livelihood – and implementers use these perspectives to mitigate risks and avoid early failure, the probability of future success will be much greater.”

See also:

• Making Sense of Sensemaker®: Evaluating Development Initiatives through Micro-Narrative Capture and Self-Tagging in Kenya. Presentation to the American Evaluation Association Conference. November, 2010. Dr. Irene Guijt, Learning by Design, John Hecklinger, GlobalGiving
• Amplifying Local Voices: GlobalGiving’s storytelling project turns anecdotes into useful data. By Suzie Boss, Summer 2011, Stanford Social Innovation Review

RD comment 1: See also a different perspective on the Global Giving experience: Networks of self-categorised stories, by Rick Davies

RD comment 2: What I like about this doc: 1. Lots of warts and all descriptions of data collection, with all the problems that occur in real life, 2. the imaginative improvement on Cognitive Edge’s use of triads as tools to enabling self-signifier tools, a circular device call the story marbles approach. This enables respondents to choose which of x categories they will use and then indicate to what extent each of these categories apply to their story. It meets the requirement the author described thus: “What we need is a means to let the storyteller define the right question while also constraining the possible questions enough that we will derive useful clusters of stories with similar question frames.”

Iñigo Retolaza Eguren, HIVOS/DD/UNDP, May 2011 Available as pdf.

“This guide has been jointly published by Hivos and UNDP, and is aimed at the rich constellation of actors linked to processes of social development and change: bilateral donors, community leaders, political and social leaders, NGO’s representatives, community-base organizations, social movements, public decision makers, and other actors related to social change processes.

The Theory of Change approach applied to social change processes represents a thinking-action alternative to other more rigid planning approaches and logics. When living in complex and conflictive times, we need to count with more flexible instruments that allow us to plan and monitor our actions in uncertain, emergent, and complex contexts from a flexible and non-rigid logic. As known, this thinking-action approach is also applied to institutional coaching processes and to the design of social development and change programs.

In general terms, the Guide synthesizes the core of the methodological contents and steps that are developed in a Theory of Change design workshop. The first part of the Guide describes some theoretical elements to consider when designing a Theory of Change applied to social change processes. The second part describes the basic methodological steps to develop in every design of a Theory of Change. For reinforcing this practical part, a workshop route is included, illustrating the dynamics in a workshop of this kind.

The approach and contents of the guide emerge from the learning synthesis of the author, Iñigo Retolaza, as facilitator of Theory of Change design processes where social change actors from several Latin American countries have been involved. His two main bodies of experience and knowledge are: (i) the learning space offered by Hivos, where he could facilitate several Theory of Change workshops with Hivos partner organisations in South and Central America, and (ii) his professional relation with the Democratic Dialogue Regional Project of UNDP, from a research-action approach around dialogic processes applied to various areas of the socio-political field: national dialogues on public policy making and adjusting and legislative proposals, facilitation of national and regional dialogue spaces on several issues, capacity building on dialogue for social and political leaders from several countries in the region”

(via Bob Williams on EvalSys)

Systemic Approaches in Evaluation

Documentation of the Conference on 25-26 January 2011

“Development programs promote complex reforms and change processes. Such processes are often characterized by insecurity and unpredictability, posing a big challenge to the evaluation of development projects. In order to understand which projects work, why and under which conditions, evaluations also need to embrace the interaction of various influencing factors and the multi-dimensionality of societal change. However, present evaluation approaches often premise predictability and linearity of event chains.

In order to fill this gap, systemic approaches in evaluation of development programs are increasingly being discussed. A key concept is interdependency instead of linear cause-effect-relations. Systemic approaches in evaluation focus on interrelations and the interaction between various stakeholders with different motivations, interests, perceptions and perspectives.

On January 25 and 26, 2011 the Evaluation and Audit Division of the Federal Ministry of Economic Cooperation and Development (BMZ) and the Evaluation Unit of GIZ offered a forum to discuss systemic approaches to evaluation at an international conference.
More than 200 participants from academia, consulting firms and NGOs discussed, amongst others, the following questions:

• What are systemic approaches in evaluation?
• For which kind of evaluations are systemic approaches (not) useful? Can they be used to enhance accountability, for example?
• Are rigorous impact studies and systemic evaluations antipodes or can we combine elements of both approaches?
• Which concrete methods and tools can be used in systemic evaluation?

On this website you will find the documentation of all sessions, speeches and discussion rounds. The main conclusions of the conference were summarized in the  final panel discussion.”

Utrecht Conference Report. Irene Guijt, Jan Brouwers, Cecile Kusters, Ester Prins and Bayaz Zeynalova. March 2011. Available as pdf

This report summarises the outline and outputs of the Conference ‘Evaluation Revisited: Improving the Quality of Evaluative Practice by Embracing Complexity’’, which took place on May 20-21, 2010. It also adds additional insights and observations related to the themes of the conference, which emerged in presentations about the conference at specific events.

Contents (109 pages):

1 What is Contested and What is at Stake
1.1 Trends at Loggerheads
1.2 What is at Stake?
1.3 About the May Conference
1.4 About the Report
2 Four Concepts Central to the Conference
2.1 Rigour
2.2 Values
2.3 Standards
2.4 Complexity
3 Three Questions and Three Strategies for Change
3.1 What does ‘evaluative practice that embraces complexity’ mean in practice?
3.2 Trade-offs and their Consequences
3.3 (Re)legitimise Choice for Complexity
4 The Conference Process in a Nutshell

[from the IDS Virtual Bulletin, March 2011]

Introduction
In this virtual Bulletin we bring together ten articles dating from across three decades. They all address Impact. From the outset, we note that there are a number of common threads and ideas that stretch across all the articles:

• The implicit emphasis of all the articles on complexity
• The breadth and depth of impact analysis, from the national level to the individual
• The importance of knowing the audience for any evaluation or impact assessment
• The virtuous cycle that can be created by using insights into impact to adjust interventions
• The dependency of that virtuous cycle on participation and engagement of programme staff and clients.

What we notice, however, is how the articles framing these issues vary according to discipline and research site. We also see how some ongoing preoccupations have been shaped by their proximity to other debates or policy concerns. Our hope is that hindsight will provide some perspective for practice and policy going forward.
View Full Introduction

Articles
A Revolution Whose Time Has Come? The Win-Win of Quantitative Participatory Approaches and Methods
IDS Bulletin Volume 41, Issue 6, November 2010
Robert Chambers

Impact of Microfinance on Rural Households in the Philippines
IDS Bulletin Volume 39, Issue 1, March 2008
Toshio Kondo, Aniceto Orbeta, Clarence Dingcong and Christine Infantado

You Can Get It If You Really Want’: Impact Evaluation Experience of the Office of Evaluation and Oversight of the Inter-American Development Bank
IDS Bulletin Volume 39, Issue 1, March 2008
Inder Jit Ruprah

The Role of Evaluation in Accountability in Donor-Funded Projects
IDS Bulletin Volume 31, Issue 1, January 2000
Adebiyi Edun

Micro-Credit Programme Evaluation: A Critical Review†
IDS Bulletin Volume 29, Issue 4, October 1998
Shahidur R. Khandker

Macroeconomic Evaluation of Programme Aid: A Conceptual Framework
IDS Bulletin Volume 27, Issue 4, October 1996
Howard White

Measurement of Poverty and Poverty of Measurement
IDS Bulletin Volume 25, Issue 2, April 1994
Martin Greeley

Developing Effective Study Programmes for Public Administrators
IDS Bulletin Volume 8, Issue 4, May 2009
Ron Goslin

Improving the Effectiveness of Evaluation in Rural Development Projects
IDS Bulletin Volume 8, Issue 1, July 1976
B. H. Kinsey

Managing Rural Development
IDS Bulletin, Volume 6, Issue 1, September 1974
Robert Chambers