Predicting the achievements of the Katine project

September 2010: This post provides information on a revised proposal for a “Predictions Survey” on the achievements of the Katine Community Partnerships Project, a project managed by AMREF and funded by the Guardian and Barclays Bank, between 2007 and 2011.

Background Assumptions

The Guardian coverage of the Katine project has provided an unparalleled level of public transparency to the workings of an aid project. As of August 2010 there have been approximately 530 articles posted on the site, most of which have specifically about Katine. These posts have included copies of project documentation (plans, budgets, progress reports, review reports) that often don’t enter the public realm.

Ideally this level of transparency would have two benefits: (a) improving UK public knowledge about the challenges of providing effective aid, (b) imposing some constructive discipline on the work of the NGO concerned, because they know they are under continuing scrutiny not only locally, but internationally. Whether this has actually been the case is yet to be systematically assessed. However I understand the effects on the project and its local stakeholders  (i.e b above) will be subject to review by Ben Jones later this year, and then open to discussion in a one day event in November, to be organised by the Guardian.

So far there have been two kinds of opportunities for the British, and other publics, to be engaged with the public monitoring of the Katine project. One has been through posting comments on the articles on the Guardian website. About 30% of all articles have provided this opportunity, and these articles have attracted an average of 5 comments . The other option has been by invitation from the Guardian, to make a guest posting on the website. This invitation has been extended to specialists in the UK and elsewhere.  Multiple efforts have also been made to hear different voices from within the Katine community itself

The Predictions Survey would provide another kind of opportunity for participation. It would be an opportunity for a wide range of participants to:

  • to make some judgments about the overall achievements of the project, and
  • to explain those judgments, and
  • to see how those judgments compared to that of others, and
  • to see how those judgments compare to the facts, about what has actually been achieved at the end of the project

In addition a Predictions Survey would provide a means of testing expectations that greater transparency can improve public knowledge about the challenges of providing effective aid.

My proposal is that that the Prediction Survey would consist of five batches of questions, one for each project component, on a separate page. Each question would be a multiple choice question, but associated with an optional Comment field. People could respond on the basis of their existing knowledge of the project (which could vary widely) and/or extra information about the website obtained via component specific links embedded at the head of each page of the online survey e.g. on water and sanitation. Questions at the end of the survey would identify participants’ sources of knowledge about the project (e.g. obtained before and during the survey, from the website and elsewhere).

A 1st rough draft survey form is already available to view. Any responses entered at this stage may be noted, but they will then be deleted and not included in any final analysis.  The final design of the survey will require close consultation with AMREF and the Guardian.

Intended participants in the survey

  • UK public, reached via the Guardian
  • Uganda public, reached via Ugandan newspapers (likely to be more of a challenge)
  • AMREF staff, especially in Uganda, Kenya HQ and UK
  • The Guardian and Barclays, as donors
  • Monitoring and Evaluation specialists, reached via an international email list

Hypotheses (predictions about the predictions)

  1. We might expect that AMREF would be able to make the most accurate predictions, given its central role. But aid agencies are often tempted to put a gloss on their achievements, because of the gap that sometimes emerges between their ambitions and what can actually be done in practice.
  2. We might expect that participants who have been following the Guardian coverage closely since the beginning might be better informed and make better predictions than others who have become interested more recently. But perhaps those participants are still responding on the basis of their original beliefs (aka biases)?
  3. We might expect M&E specialists to make better than average predictions because of their experience in analysing project performance. But perhaps they have become too skeptical about everything they read
  4. We might expect the Guardian and Barclays staff to make better than average predictions because they have been following the project closely since inception and their organisation’s money is  invested in it. But perhaps they only want to see success.
  5. We might expect the highest frequency choices (across all groups) to be more accurate than the choices of any of the above groups, because of a ” wisdom of crowds” effect. The potential of crowdsourcing was of interest to the Guardian at the beginning of the project, and this survey could be seen as a form of crowdsourcing – of judgements.

This list is not final. Other hypotheses  could be identified in the process of consultation over the design of the survey

There may also be other less testable predictions worth identifying. For example, about the effects of this Prediction Survey on the work done by AMREF and its partners in the final year up to October 2011. Might it lead to a focus on what is being measured by the survey, to the detriment of other important aspects of their work?  If AMREF has a comprehensive monitoring framework and the prediction survey addresses the same breadth of performance (and not just one or two performance indicators) this should not be a problem.

Timeframe

The fourth and final year of the project starts in October 2010 and ends in October 2011.

The finalisation of the design of the Predictions Survey will require extensive consultation with AMREF and the Guardian, in order to ensure the fullest possible ownership of the process, and thus the results that are generated. Ideally this process might be completed by late-October 2010

The survey could be open from late October to the end of March 2011 (six months before the end of the project). All responses would be date stamped to take account of any advantages of being a later participant

A process will need to be agreed in 2010 on how objective information can be obtained on which of the multiple choice options have eventuated by October 2011.

A post 2011 follow up survey may be worth considering. This would focus on predictions of what will happen in the post-project period, up to 2014, the year of the vision statement produced by participants in the September 2009 stakeholders workshop in Katine.

“In 2014, Katine will be an active, empowered community taking responsibility for their development with decent health, education, food security and able to sustain it with the local government”

Supporters

The participation of the Guardian and AMREF will be very important, although it is conceivable that the survey could be run independently of their cooperation

Assistance with publicity, to find participants, would be needed from the Guardian and Barclays

Advisory support is being sought from the One World Trust

Advisory support from other other organisations could also be useful

The online survey could be designed and managed by Rick Davies. However responsibility could be given to another party that was agreed to by AMREF, Guardian and Barclays.

Challenges

  • The survey design needs to be short enough to encourage people to complete it, but not so short that important aspects of the project’s performance are left out
  • The description of the objectives used in the survey needs to be as clear and specific as possible, but also keep as close to AMREF’s original words as possible (i.e. as in the 4th year extension proposal, and using the M&E framework, now being updated)
  • Participants will be asked to make a single choice between multiple options, describing what might happen. These options will need to be carefully chosen, so there are no obvious “no brainers”, and to cover a range of plausible possibilities
  • It may be necessary in some cases (e.g. with some broadly defined objectives) to allow multiple choices from multiple options
  • I have heard that AMREF will be conducting a final evaluation in late 2011, using an external consultant. This evaluation could be the source of the final set of data on actual performance, against which participant’s predictions could be compared. But will it be seen as a sufficiently independent source of information?

The Logical Framework: A list of useful documents

Contents: 1. Explanations of the Logical Framework | 2. Wider discussions of Logic Models | 3. Critiques of the Logical Framework |  4. Alternative versions of the Logical Framework |  5. The Editor’s concerns (about uses of the Logical Framework) | 6. Online survey on views and usage of the Logical Framework

Please feel free to suggest additions or corrections to this list, by using the Comment facility at the end of this post

New section: Software

1. Explanations of the Logical Framework

2. Wider discussions of Logic Models

  • Program logic – an introduction, provided by Audience Dialogue (2007)
  • Enhancing Program Performance with Logic Models This course introduces a holistic approach to planning and evaluating education and outreach programs. Module 1 helps program practitioners use and apply logic models. Module 2 applies logic modeling to a national effort to evaluate community nutrition education. Provided by the University of Wisconsin (2007)
  • Online Logic Model training: an audiovisual presentation by Usable Knowledge, USA Twenty minutes long, with a menu that can be used to navigate to the sections of interest (2006)
  • Network Perspectives In The Evaluation Of Development Interventions: More Than A Metaphor.  Rick Davies, for the EDAIS Conference November 24-25, 2003 New Directions in Impact Assessment for Development: Methods and Practice. “In this paper I argue the case for the use of a network perspective in representing and evaluating aid interventions. How we represent the intentions of aid activities has implications for how their progress and impact can be assessed. Because our representations are by necessary selective simplifications of reality they will emphasise some aspects of change and discourage attention to others. The benchmark alternative here is by default the Logical Framework, the single most commonly used device for representing what an aid project or programme is trying to do. Five main arguments are put forward in favour of a network perspective as the better alternative, along with some examples of their use. Firstly, social network analysis is about social relationships, and that is what much of development aid is about. Not abstract and disembodied processes of change. Secondly, there is wide range of methods for measuring and visualising network structures. These provide a similarly wide range of methods of describing expected outcomes of interventions in network terms. Thirdly, there is also a wide range of theories about social and other networks. They can stimulate thinking about the likely effects of development interventions. Fourthly, network representations are very scalable, from very local developments to the very global, and they can include both formal and informal structures. They are relevant to recent developments in the delivery of development aid. Fifthly, network models of change can incorporate mutual and circular processes of influence, as well as simple linear processes of change. This enables them to represent systems of relationships exhibiting varying degrees of order, complexity and chaos. Following this argument I outline some work-in-progress, including ways in which the conference participants may themselves get involved. Finally I link this paper into its own wider web of intellectual influences and history. ” (Posted here 2003)
  • The Temporal Logic Model: A Concept Paper, by Molly den Heyer. On the IDRC website. (2002)
  • A Bibliography for Program Logic Models/Logframe AnalysisDecember 18, 2001 Compiled by: Molly den Heyer Evaluation Unit, International Development Research Centre
  • W K Kellogg Foundation Logic Model Development Guide. (2001) Using Logic Models to Bring Together Planning, Evaluation, and Action. Updated (original was published in 1998) “The program logic model is defined as a picture of how your organization does its work – the theory and assumptions underlying the program.A program logic model links outcomes (both short- and long-term) with program activities/processes and the theoretical assumptions/principles of the program.”
  • Application of Logic Modeling Processes to Explore Theory of Change from Diverse Cultural Perspectives Ricardo Millett, Sharon Dodson, & Cynthia Phillips American Evaluation Association November 4, 2000
  • The state of the art of Logic Modelling. PowerPoint presentation by Gretchen Jordan (1999?)
  • The Logic Model for Program Planning and Evaluation, Paul F McCawley, 1997, University of Idaho Extension.

3. Critiques of the Logical Framework

  • Debunking misconceptions around the Logical Framework Approach through reviewing available literature by Munyaradzi Madziwa, August 2016
  • Critical Study Of The Logical Framework Approach In The Basque Country (2011) by ECODE, Bilbao.
  • THE USE AND ABUSE OF THE LOGICAL FRAMEWORK APPROACH A Review of International Development NGOs’ Experiences. A report for Sida. November 2005. Oliver Bakewell and Anne Garbutt, of INTRAC. “In this review, we have attempted to take stock of the current views of international development NGOs on the LFA and the ways in which they use it. We start in the next section by considering the different meanings and connotations of the term logical framework approach as it is used by different actors. In Section 3 we look at how LFAs are used by INGOs in both planning and project management. The next section reviews some of the debates and critiques around the LFA arising both from practice and the literature. In response to these challenges, different organisations have adapted the LFA and these variations on the LFA theme are outlined in Section 5. We conclude the paper by summarising the findings and reflecting on ways forward. …This review has been commissioned by Sida as part of a larger project which aims to establish new guidelines for measuring results and impact and reporting procedures for Swedish development NGOs receiving support from Sida. “

4. Alternative versions of the Logical Framework

  • Beyond Logframe: Using Systems Concepts in Evaluation, 2010, FASD (Foundation  for Advanced Studies on International Development)
  • The Social Framework, an actor-oriented adaptation of the Logical Framework, developed by Rick Davies. The sequence of rows found in a Logical Framework now represent a sequence of actors, connected to each other by their relationships, and forming a specific pathway through a wider network of actors. Narrative descriptions of expected changes, indicators of those change and means of verification are still found in the columns, but these relate to actors and their relationships. Actors can be individuals, groups, organisations or type of organisations. The assumptions column still exists, but the assumptions refers to important connections to other actors outside the specific pathway.
  • Can OM and LFA share a space? “OM (Outcome Mapping) and LFA may be useful at different levels, for diverse types of interventions or for information and in different contexts. Rather than pitting LFA and OM against each other, we need to understand what kinds of information and uses each has, as well as their advantages and disadvantages, and find ways for them to add value to each other.” See also Logical Framework Approach and Outcome Mapping: A Constructive Attempt of Synthesis. A Discussion Paper by Daniel Roduner and Walter Schläppi, AGRIDEA; Walter Egli, NADEL (ETH Zurich)
  • Logical Framework Approach – with an appreciative approach. April 2006 SIDA Civil Society Centre. “As a part of its effort to realise the intentions of Sweden’s Policy on Global Development, Sida Civil Society Center (SCSC) initiated a development project in 2005 together with PMU Interlife (the Swedish Pentecostal Mission’s development cooperation agency) and consultant Greger Hjelm of Rörelse & Utveckling. The goal was to create a working model which combines the goal hierarchy and systematics from the Logical Framework Approach (LFA)1 with the approach used in the Appreciative Inquiry tool (AI). AI is both a working method and an approach. In analysing strengths and resources, motivation and driving forces, the focus is placed on the things which are working well, and on finding positive action alternatives for resolving a situation. LFA, which is an established planning model in the field of international development, is found by many to be an overly problem-oriented model. Using this approach, one proceeds based on a situation in which something is lacking, formulates the current situation as a “problem tree”, and thus risks failing to perceive resources which are actually present, and a failure to base one’s support efforts on those resources. Working in close cooperation, we have now formulated a new working method for planning using LFA, one which is built on appreciative inquiry and an appreciative approach. The model was tested by PMU Interlife’s programme officers and their cooperating partners in Niger, Nicaragua and Tanzania during the autumn of 2005. Their experiences have been encouraging, and it is our hope that more Swedish organisations and their cooperating partners will try our model and working method.(Posted 01/07/06)
  • No more log frames!! People-Focused Program Logic Two day workshop Monday 19th and Tuesday 20th of September 2005, in Melbourne, Australia. “Purpose of the workshop: • To understand what ‘people-focused’ program logic is and how to use it • To build a people-focused program logic for their own project Who should attend? People with monitoring and evaluation interests who are working on projects with capacity building components. Course description: In this workshop, participants will build their own ‘people-focused’ logic model. To do this they will analyse the key beneficiaries of their project, build their program logic model around this analysis, and consider assumptions made in the logic. The program logic will be built around a generic theory of how capacity building works, that can be modified to include elements of advocacy and working with or through partners. Participants will also learn how this logic can be used to form the spine of their monitoring, evaluation and improvement framework. As participants will be invited to develop their own program logic model, they are encouraged to bring along others from the same project team. Examples of frameworks, and a workbook will be provided to participants” For additional information: Jo Leddy of Clear Horizon Phone: 03 9783 3662 E-mail: Jo@clearhorizon.com.au Website: www.clearhorizon.com.au See rest of the flyer for more information…(Posted 21/06/05)
  • Intertwining Participation, Rights Based Approach and Log-Frame: A way forward in Monitoring and Evaluation for Rights Based Work. Partha Hefaz Shaikh Initial Draft – Circulated for discussion. “Programme implementation through Rights Based Approach (RBA) in ActionAid Bangladesh started in 2000 and it took us quite a while to understand what it meant to implement programmes in a RBA environment. Side by side we were also grappling with issues of monitoring and evaluation of programmes implemented through a rights based approach. In order to develop a more meaningful framework that has all the elements of participation, RBA and log-frame we developed what we call “Planning and Implementation Framework Analysis (PIFA)”. ” (Posted 20/05/05)
  • A MODIFIED LOGFRAME FOR USE IN HUMANITARIAN EMERGENCIES. by Bernard Broughton (I think)
  • Family Planning Logical Framework (with two parallel processes, one feeding back into the other)
  • Build Reach into Your Logic Model. Steve Montague February 1998 “Analysts have frequently noted the importance of constructing logic models (a.k.a. logic charts, causal models, logical frameworks, and most recently performance frameworks – among other names) to explain the causal theory of a program or initiative before attempting to monitor, measure, or assess performance. …A key limitation to the logic models of the 1980s, as well as many of those in current use, has been their tendency to focus predominantly on causal chains without reference to who and where the action was taking place. “
  • Bennett’s Hierarchy (or Targetting Outcomes of Programs (TOP)). This is not a version of the LogFrame, but it is another type of logic model with multiple steps (7 levels). It has been used widely in the evaluation of agricultural extension activities in Australia. It was originally developed by Bennett in 1975.

5. The Editor’s concerns (about uses of the Logical Framework)

  1.  Long, complex, unreadable sentences, in the narrative column of the Logical Framework
    1. Often the result of compromises between many different parties who have been negotiating the contents of the Logical Framework. Net result: an unreadable document
    2. Sometimes the result of people not knowing that the whole story does not need to be told in one sentence. The row below should say what happens before (the cause) and the row above should say what happens next (the effects)
    3. Sometimes the result of people forgetting there is a column for indicators next door, where they can provide lots of interesting detail about what is expected to happen at this stage
  2. Narrative statements without people in them. E.g “Rice productivity increased”
    1. Another reason some many Logical Frameworks are so unreadable, and so boring when they are readable, is that somehow their authors have managed to leave out people. Instead we have lots of abstract and disembodied processes. And then we wonder why some people have difficulty understanding Logical Frameworks
  3. Means of Verification that refer to reports and surveys, but not who is responsible for generating and / or providing this information (and when it will be available)
    1. This problem is similar to the above, reflecting a continuing aversion to making references to real people in Logical Frameworks.
    2. One consequence is lack of clear ownership and responsibility for M&E of the changes being described at that level of the Logical Framework
  4. Insistence on there being only one Purpose level statement in a Logical Framework
    1. I have recent experience of colleagues insisting on this. For reasons I have not yet established, beyond the “it is not allowed” variety. Insisting on one Purpose and One Goal really is pushing a very linear model of reality. It does not even allow for any parallel but convergent events, such as those usually come through problem tree analyses that sometimes precede the design of a Logical framework
  5. Ambitous narrative statements coupled with modest indicators / Overly simple indicators used to describe complex developments.
    1. Such as “number of meetings held” as an indicator for the functioning of stakeholder’s advisory committee. For an alternative, see “Checklists as mini-theories of change
  6. Lists of indicators in no apparent order
    1. “A (unsorted) list is not a strategy” A sorted list can convey relative importance (most important indicator at the top), or a sequence (starting from the bottom), or multiple alternative routes to the objective in the narrative column. If there is a list, the reader should be told what sort of list it is.
  7. Broad generalisations at the Goal level
    1. Sometimes arising from confusion of a temporal hierarchy (A leads to B which leads to C which leads to D) and a nested hierarchy (A is part of B which is part of C which is part of D). The Logical Framework is supposed to be a temporal hierarchy, that tells a story. Not a pile of increasingly broad statements about the same thing
  8. Confusion over the meaning of different levels in a Logical Framework. Between Activities and Outputs, Outputs and Purpose level outcomes, and outcomes at the Purpose and Goal level.
    1. Often cause by leaving people out of the picture, as above.
    2. A workable rule of thumb, for seperating levels of the Logical Framework
      1. Activities are things that “the project” can control. The boundary of a project being defined by the reach of its contracts (with staff, consultants, suppliers and sub-contractors)
      2. Outputs are the activities of the project (if services), or their results (if goods), that people and organisations outside the project can use e.g workshops, publications, trainings, etc. Ask here: What is available to who, and in what form?
      3. Purpose level changes (outcomes), are changes in those people or organisations who have used those goods or services. Normally the project would hope to influence these (and learn about how it can have influence) but it would not be expected to control events at this level
      4. Goal level changes (outcomes), are longer term changes in those same people or organisations, or others they have subsequently interacted with.
  9. Long lists of assumptions
    1. Apparently designed to cover people’s backsides
    2. Including many events that the project should be able to influence
      1. …which therefore should be listed as one of the outputs or outcomes. I.e. brought into the central narrative of the Logical Framework
  10. Things the Logical Framework cant do very well, even in the best of hands
    1. Represent multiple parallel processes, as distinct from a single process
      1. E.g. What people are doing at multiple project locations, within a single national project
        1. Representing their interactions is even more of a challenge
    2. Represent the interactions between multiple events at the same level of a Logical framework.
      1. E.g. How different project outputs (manuals, training events, newsletters, websites, etc) feed into each other
      2. Or, how different health outcomes (at Purpose level) feed into each other, before finally contributing to Goal level changes e.g. reduced mortality
    3. Represent the interactions between multiple outputs and the many users of those outputs
      1. E.g., the range of communications products used by a range of clients of a project . Many people will use multiple products, but their usage patterns will vary. Many products will be used by multiple users, but their user groups will vary.

All these processes can be represented by network models. See the new page on developing network models of development projects. However network models are generally too complex to provide a substitute for the Logical Framework. One proposed alternative is the Social Framework, originally described here and now updated here. The Social Framework can be used to describes a pathway through a network, in a way that capable of being monitored and evaluated. Your comments are welcome.

6. Online survey into the uses of the Logical Framework

Please consider taking part in this survey. You can access the cumulative results to date at the end of the survey form. It is not long.

thanks, rick davies

A digression on complexity and networks…

….a side argument from the Rick on the Road post: Cynefin Framework versus Stacey Matrix versus network perspectives

In that post I said

PS1:Michael Quinn Patton’s book on Developmental Evaluation has a whole chapter on “Distinguishing Simple, Complicated, and Complex”. However, I was surprised to find that despite the book’s focus on complexity, there was not a single reference in the Index to “networks”. There was one example of a network model (Exhibit 5.3) , contrasted with a Linear Program Logic Model…” (Exhibit 5.2), in the chapter on Systems Thinking and Complexity Concepts. [I will elaborate further]

One interpretation: Complexity arises through the interaction of many agents having some degree of autonomy. With no autonomy there is simple order (complete predictability), with complete autonomy there is chaos (no predictablity). How do we define autonomy? One view: Autonomy = The number of possible relationships an actor can have with others. When realised, this can be measured in terms of  network density (a Social Network Analysis (SNA) measure). Two cariacature examples of the extremes: 1. An army, with a hierarchical chain of command,  is highly ordered. Here the network structure is  sparse (i.e.  a tree structure) and low in density. 2. “Economic man” , who is free to interact with anyone, in order to maximise his/her utility. Here all possible relationships can be realised, as everyone interacts with everyone. Complexity is the territory in between where actors have some degree of choice of who they interact with. And where there is some degree of predictability. When realised, those choices can also be described in terms of different kinds of network structures. So if we want to explore complex systems we need to look at the structure of networks of actors, both as “initial conditions” affecting what happens next and as “final states”, reflecting what has happened over a given period of time. I.e. an empirical approach, not mysticism :-)

PS: The concept of autonomy could probably be further differentiated, in terms of relationship choices, as follows : (a) the range of relationships available to an actor, already discussed above (b) the freedom to choose amongst those that are available, (c) the range of behaviors available within a given relationship. But how do you measure freedom (b) ? One measure might be the degree to which any choices made are uncorrellated with other events. The diversity of choices made could also be important. Diversity suggests freedom from constraint (more on this theme here).

Results Based Management Explained (by the ADB)

ADB website

Results Based Management (RBM) can mean different things to different people. A simple explanation is that RBM is the way an organization is motivated and applies processes and resources to achieve targeted results.

Results refer to outcomes that convey benefits to the community (e.g. Education for All (EFA), targets set in both Mongolia and Cambodia). Results also encompass the service outputs that make those outcomes possible (such as trained students and trained teachers). The term ‘results’ can also refer to internal outputs such as services provided by one part of the organization for use by another. The key issue is that results differ from ‘activities’ or ‘functions’. Many people when asked what they produce (services) describe what they do (activities).

RBM encompasses four dimensions, namely:

  • specified results that are measurable, monitorable and relevant
  • resources that are adequate for achieving the targeted results
  • organizational arrangements that ensure authority and responsibilities are aligned with results and resources
  • processes for planning, monitoring, communicating and resource release that enable the organization to convert resources into the desired results.

RBM may use some new words or apply specific meanings to some words in general usage. Check introduction to RBM presentation[PDF | 56 pages].

RBM references that provide more background

  • A diagram showing relationship between goals and outcomes
  • United Nations Development Program RBM overview
  • Canadian International Development Agency RBM overview
  • RBM diagnostic tool for Cambodia and Mongolia

    • Making government budgets more accessible and equitable

    (from ID21)

    Involvement in the budget process in poor countries has traditionally been limited to a select group of political actors. But this has changed over the last decade with legislators, civil society groups and the media playing a more active role. What impact is broader engagement having?

    Research from the Institute of Development Studies, UK, examines the substance and impact of applied budget work undertaken by civil society groups. The research draws on six case studies of independent budget work in Brazil, Croatia, India, Mexico, South Africa and Uganda. One focus of the research is how civil society budget work influences government budget priorities and spending in a way that benefits poor and socially excluded groups.

    Budget work is carried out by various types of organisations including non-government organisations (NGOs), networks and social movements, and research organisations. All the groups examined in the case studies share a commitment to increasing the influence of poor and marginalised groups in the budget process and ensuring that budget priorities reflect the needs of these groups.

    The six organisations all engage in certain core activities centred on data analysis and dissemination, advocacy and capacity building. Most work on national and state-level budgets, though several groups also work at the local government level.

    The research shows that independent budget work has the potential to deepen democracy by strengthening accountability, fostering transparency and encouraging participation. It can also increase financial allocations in areas that contribute to social justice and equity outcomes and ensure that public money is efficiently spent.

    The research also reveals the limits to budget work. Any increases in financial allocations secured as a result of advocacy initiatives are likely to represent a small share of overall government spending. Also, the scope of budget work to influence financial allocations depends on the openness and flexibility of the budget process (spending priorities may not be open to change).

    The impacts of budget work identified by the research include:

    • improving the transparency of budget decisions and budget processes and increasing the accountability of state actors
    • increasing awareness and understanding of budget issues
    • improving budget allocations in a way that benefits poor and socially excluded groups
    • ensuring better use of spending, for example in areas such as health and education, and reducing corruption (by tracking expenditures)
    • diversifying the range of actors engaged in budget processes (for example, legislators, civil society groups and the media)
    • strengthening democracy and deepening participation.

    The research concludes that:

    • Budget work has been successful in a range of areas, including improving equity and social justice outcomes.
    • The technical nature of the budget process limits the scope for broadening citizen participation.
    • The challenge for budget groups is how to scale-up and replicate the successful impacts achieved to date.
    • Influencing budget policies requires a combination of sound technical knowledge, effective communications and strategic alliances.
    • Promoting the voice of poor and socially excluded groups is an important indirect effect of budget work.

    Source(s):
    ‘Budget Analysis and Policy Advocacy: The Role of Non-governmental Public Action’, IDS Working Paper 279, IDS: Brighton, by Mark Robinson, 2006 Full document.

    Funded by: UK Economic and Social Research Council

    id21 Research Highlight: 16 August 2007

    Further Information:
    Mark Robinson
    Policy and Research Division
    UK Department for International Development (DFID)
    1 Palace Street
    London SW1E 5HE
    UK

    Tel: +44 (0)20 70230000
    Fax: +44 (0)20 70230636
    Contact the contributor: mark-robinson@dfid.gov.uk

    Results Based Management (RBM): A list of resources

    CIDA website: Results-based Management

    Results-based Management (RBM) is a comprehensive, life-cycle approach to management that integrates business strategy, people, processes, and measurements to improve decision-making and to drive change.

    The approach focuses on getting the right design early in a process, implementing performance measurement, learning and changing, and reporting on performance.

  • RBM Guides
  • RBM Reports
  • Related Performance Sites

    • ADB website: Results Based Management Explained

    Results Based Management (RBM) can mean different things to different people. A simple explanation is that RBM is the way an organization is motivated and applies processes and resources to achieve targeted results.

    Results refer to outcomes that convey benefits to the community (e.g. Education for All (EFA), targets set in both Mongolia and Cambodia). Results also encompass the service outputs that make those outcomes possible (such as trained students and trained teachers). The term ‘results’ can also refer to internal outputs such as services provided by one part of the organization for use by another. The key issue is that results differ from ‘activities’ or ‘functions’. Many people when asked what they produce (services) describe what they do (activities).

    RBM encompasses four dimensions, namely:

    • specified results that are measurable, monitorable and relevant
    • resources that are adequate for achieving the targeted results
    • organizational arrangements that ensure authority and responsibilities are aligned with results and resources
    • processes for planning, monitoring, communicating and resource release that enable the organization to convert resources into the desired results.

    RBM may use some new words or apply specific meanings to some words in general usage. Check introduction to RBM presentation[PDF | 56 pages].

    RBM references that provide more background

    UNFPA website: Results-Based Management at UNFPA

    There is a broad trend among public sector institutions towards Results-Based Management–RBM. Development agencies, bilateral such as Canada, the Netherlands, UK, and the US as well as multilateral such as UNDP, UNICEF and the World Bank, are adopting RBM with the aim to improve programme and management effectiveness and accountability and achieve results.

    RBM is fundamental to the Fund’s approach and practice in fulfilling its mandate and effectively providing assistance to developing countries. At UNFPA, RBM means:

    • Establishing clear organizational vision, mission and priorities, which are translated into a four-year framework of goals, outputs, indicators, strategies and resources (MYFF);
    • Encouraging an organizational and management culture that promotes innovation, learning, accountability, and transparency;
    • Delegating authority and empowering managers and holding them accountable for results;
    • Focusing on achieving results, through strategic planning, regular monitoring of progress, evaluation of performance, and reporting on performance;
    • Creating supportive mechanisms, policies and procedures, building and improving on what is in place, including the operationalization of the logframe;
    • Sharing information and knowledge, learning lessons, and feeding these back into improving decision-making and performance;
    • Optimizing human resources and building capacity among UNFPA staff and national partners to manage for results;
    • Making the best use of scarce financial resources in an efficient manner to achieve results;
    • Strengthening and diversifying partnerships at all levels towards achieving results;
    • Responding to the realities of country situations and needs, within the organizational mandate.

    OECD report: RESULTS BASED MANAGEMENT IN THE DEVELOPMENT CO-OPERATION AGENCIES: A REVIEW OF EXPERIENCE BACKGROUND REPORT

    In order to respond to the need for an overview of the rapid evolution of RBM, the DAC Working Party on Aid Evaluation initiated a study of performance management systems. The ensuing draft report was
    presented to the February 2000 meeting of the WP-EV and the document was subsequently revised.
    It was written by Ms. Annette Binnendijk, consultant to the DAC WP-EV.

    This review constitutes the first phase of the project; a second phase involving key informant interviews in a number of agencies is due for completion by November 2001.

    158 pages, 12 page conclusion

    this list has a long way to go….!