Category: Lists of sources re …

Monthly summaries of postings on the MandE NEWS email list

Maren Thompson, Course Coordinator at IMA International produces monthly summaries of the postings on the MandE NEWS email list, including links that were mentioned in the postings. This work has previously been done by Nicky David and Ros MacDonald, also of IMA International. The monthly summaries are MS Word documents. They are listed by month below:

————————————-

May 2007 | June 2007 | July 2007 | August 2007 | September 2007 | October 2007 | November 2007 | December 2007 |

January 2008 | February 2008 | April 2008 | May 2008 | June 2008 | July 2008 | August 2008 | September 2008 | October 2008 | November-December 2008 |

January-2009| February-2009| March 2009 | April 2009 | May 2009 | June 2009 |

Social Network Analysis software: A list

A. Software I have some familiarity with:

UCINET & NetDraw ( a combined package)

  • Easy to import data from Excel
  • Has a huge range of abilities to manipulate and edit the raw data
  • Has an online support group (Yahoo Groups)
  • There is a detailed how to use it text
  • Files can be read by many other software packages
  • Not very expensive, and there is a free trial period
  • Undergoing continuous development
  • Widely used
  • Not easy to draw network diagrams on screen
  • Steep learning curve, many more bells and whistles than you may need
  • No easy to use introductory texts
  • Not easy to edit node and link attribute data on the NetDraw screen
  • PS: See Louse Clark’s very useful and detailed guide to working with NetDraw; and “A Brief Guide to Using NetDraw” by Steve Borgatti; and NETDRAW – BASIC A Practical Guide to Visualising Social Networks by ONA Surveys

Visualyzer

  • Perhaps my favorite, beause it is easy to draw and edit networks on screen, which is very useful in workshop settings
  • Attributes of nodes and links can be easily edited and displayed
  • Can import and export UCINET data
  • Very user-friendly manual
  • Free trial period
  • Now available at a more reasonable price!
  • No online support group
  • Does not seem to be undergoing continuous development

yED Graph Editor

  • Very good for network drawing
  • Many options for layouts
  • Can export files to work as web pages
  • Nodes can include weblinks, allowing quick access to much more information about each node
  • Free
  • Latest version (3.5) can now open data from Excel worksheets, in matrix, edgelist (relationships)  and nodelist (actors) forms. Including as many attributes for the actors and relationships as needed. It seems it will import both one and two mode  (adjacency and affiliation) matrices. This is a major improvement.
  • They are working on capacity to export back to Excel, and ability to search actors and relationships by attribute. Both will be very useful
  • yED is rapidly moving up my list of most favored SNA software packages
  • Now also available as an online version: yED Live
  • Limited analysis capacity

Microsoft NodeXL

  • Free, works as a plug-in to Excel 2007
  • Undergoing continuous development
  • Online support group
  • All node and link attribute data is visible and easy to edit in Excel sheets, which is great
  • Nodes can include weblinks, I think
  • There is a useful users guide here
  • You can’t draw the network direct on the screen,
    • But by using the Excel sheet immediately below the screen you can add nodes and links, and edit their attributes, very easily
  • I have had difficulty in importing yEd (GraphML)
    • PS: They report this is being addressed
  • The layout options (different algorithms) seem quite limited
  • I dont yet know as much about it as the other packages above

C-IKNOW

  • I attended a presentation on C-IKNOW at the 2010 INSNA conference and found this package very impressive, for two broad reasons:
    • User-friendliness
    • Sophisticated range of capacities
  • This is an online service that is open to use by anyone, free of charge
  • Data can be imported, exported and generated by an associated online survey mechanisms
  • There are multiple videos showing how different aspects of the package works, along with a detailed downloadable user guide
  • Development is ongoing and led by Noshir Contactor, a very smart person, and co-author of Theories of Communication Networks
  • “As of September 2012, C-IKNOW Survey is no longer under active development. It can still be used, but no support is available. It should be used at your own risk”

Discourse Network Analyzer

  • “software which combines social network analysis and category-based content analysis. After applying categories to text portions, you can automatically extract two-mode networks or one-mode co-occurrence networks in several file formats. There are also some algorithms for longitudinal analysis.”
  • Exports to Excel (in CSV format),  DL files (UCINET), and GraphML files (visone, yEd etc)
  • Free
  • Looks useful but I have yet to try it out on my own data

Visone

  • Patrick Kenis describes this as “very intuitive programme which can be used instantly in consultancy settings”
  • Free
  • Easy to draw networks live on screen
  • Continuous development, but not so often as UCINET

Gephi (last comments added 21 April 2011)

  • Open source (free)
  • Undergoing continuous development, but not so often as UCINET
  • Very sophisticated graphics, the emphasis is on visualisation as a means of exploratory data analysis
  • Capable of visualising very large networks quickly
  • Dynamic views of networks, as they change over time
  • Many filtering options
  • As in NodeXL, has a Data Table view to browse and edit data
  • Drawing networks on the screen is possible, but not so intuitive
  • Imports GraphML files (e.g. as used by yED, NodeXL), vna (as used by Netdraw), csv (used by Excel etc). Exports as csv (for Excel etc) and GraphML.
  • Has Plugins e.g. Social Network Data Import
  • Looks like it could become very good, in time

KUMU

  • Very impressive network diagramming capacity
  • Lots of capacity to annotate network diagrams
  • Online network visualiation tool
  • Free and paid for subscriptions
  • Good tutorial material and technical support

Others not yet examined in any detail

Inflow: [Not yet tested, but looks good]

Social Networks Visualizer (SocNetV) [Not yet tested, but used by Valdis Krebs]

Cytoscape Thomas Delahais says: “I’ve been using consistently Cytoscape, which was designed for neuro-biological analysis but works very well for social sciences! Cytoscape is free, open source and you should complete it with the Max Planck Analyser Plugin, which includes all or most of the usual indicators (diameter, shortest path, etc.) in a unique interface (free for non-commercial use if I remember well). Cytoscape needs some formatting first but then it is very easy to use, very easy to draw on screen too. As a sidenote this is the software I picked when I decided that Ucinet was too complicated for transferring this competency to my colleagues”

SocioWorks “is an innovative set of web tools for the online application of Social Network Analysis (SNA) methods to collect and analyze data regarding social relationships, from individual to institution to national levels.” (posted 2013 05 02)

B. Lists of software most of which I dont know about, maintained by:

  1. KM4DEV list
  2. Wikipedia list
  3. International Network for Social Network Analysis list
  4. Top 10 Open-Source Social network  Development Platforms
  5. Mark Round’s  “SNA Tools and Formats diagram – updated”showing how different software packages are linked by use of the same data formats

The number of social network analysis packages is exploding, a bit like the Cambrian explosion of organic life. No software package has yet achieved dominance because of its ability to meet a wide variety of needs.

C. Online SNA software

  • IdeaTree was not developed as SNA software, but in practice provides many of the same functions, in terms of visualisation. Key features: (a) it supports online collaborative development of network diagrams, (b) it seems quite user friendly, (c) data can be exported in XML, which can be converted elsewhere into graphml, and as pdf documents

PS April 2011: GraphML is a format for storing network data, used by yED, Gephi, and others. The GraphML Primer provides a simple introduction to its use.

Social Network Analysis and Evaluation: A List

Preface to the list

“There are however three challenges in promoting the use of network models. One is to get people thinking in terms of networks as a kind of a base metaphor, in the same way that in the past people may have seen clockworks as a base metaphor for how the world works. The second is to sift through what often appears to be a surfeit of technical capacity to analyse networks, in order to focus in on the simplest and often most useful basics. The third challenge is to develop more participative and interpretative approaches to the description and analysis of networks, in contrast to the number crunching nature of much social network analysis in the academic world. This is all very much a ‘work in progress‘.” Rick Davies, in newsletter article for American Evaluation Association, 2008

The list

  • Using Social Network Analysis in Evaluation: A Report to the Robert Wood Johnson Foundation. By Kimberly Fredericks, PhD, MPA, RD, Associate Professor
    The Sage Colleges School of Management, 2013  “…Using data gathered from telephone interviews with the program officers, email surveys from project directors, and supplemental information from nine project summaries (see Appendix 1 for the summaries), this report: ? Describes how the Foundation took deliberate steps to create an emerging learning
    community to support using social network analysis to understand and evaluate its funded work ? Illustrates how social network analysis can be used in many applications, ranging
    from a one-time study to longitudinal studies, with a summative purpose (to evaluate the success of a particular intervention) or formative purpose (for learning and improvement)
    ? Identifies the challenges associated with using social network analysis, in terms of understanding the methodology’s limits or constraints, the implications presented by
    the size and scope of the projects, and the need to develop the technical expertise and capacity to manage the projects and use the data”

    •  

  • Using Social Network Analysis to Advance Traditional Qualitative Methods in Evaluation and Program Design. Caroline J. Wilson, Anne E. Dougherty, Mary Sutter, Jennifer Mitchell-Jackson,  Pamela Wellner,  Nick Hall, ABSTRACT Social network analysis (SNA) is a technique used to study relationships between actors, such as people or organizations. It has been applied to a wide array of disciplines, on topics ranging from destabilizing Al-Qaeda to explaining campaign fundraising success. While these applications may seem far removed from the efforts of AESP members, this paper will provide insights into the innovative ways social network analysis can be used in energy-related evaluations. We start by providing parameters for the use of this technique for evaluation efforts and articulate a number of researchable issues central to evaluation that may be answered through social network analysis. Our evaluation team then demonstrates how this method was employed to conduct a more thorough process evaluation as a complementary approach to other, more traditional research techniques. We also demonstrate the value of the social network analysis approach by comparing a network map with a more traditional organization and implementation chart. We then outline other ways it was utilized, including discussion of social network analysis measures such as centrality and density. This paper aims to add value to current evaluation methods and introduce social network analysis to the best practices of program evaluation and design.

 

  • Social Network Analysis: A Useful Tool for Visualizing and Evaluating Forestry Research. N.L. Klenk1, G.M. Hickey1, J.I. MacLellan2, R. Gonzales3 and J. Cardille3 International Forestry Review 11(1):134-140. 2009 SUMMARY “One of the foundational studies of social network analysis produced a depiction of scientific collaboration by tracing a network of scientific papers linked by co-authorships and co-citations, which has since spurred numerous studies on the typology, organization and dynamics of scientific research networks. This paper introduces social network analysis and its analytical measures of network structure. It then demonstrates the utility of social network analysis in forestry, in the evaluation of large research networks such as the Sustainable Forest Management Network (SFMN), and suggests other important uses of network visualization to facilitate exploring, discovering and selecting resources in a database.

  •  

  • Social Network Analysis and the Evaluation of Leadership Networks. Bruce Hoppe, Ph.D., Claire Reinelt, Ph.D. April 2008 . Abstract: “Leadership development practitioners have become increasingly interested in the formation of leadership networks as a way to sustain and strengthen relationships among leaders within and across organizations, communities, and systems. This paper offers a framework for conceptualizing different types of leadership networks and identifies the outcomes that are typically associated with each type of network. One of the challenges for the field of leadership development has been how to evaluate leadership networks. Social Network Analysis (SNA) is a promising evaluation approach that uses mathematics and visualization to represent the structure of relationships between people, organizations, sectors, silos, communities and other entities within a larger system. Core social network concepts are introduced and explained to illuminate the value of SNA as an evaluation and program tool.”
  •  

    Conference evaluation and network mapping Glenn O’Neil (2008) “Often we attend conferences where one of the stated objectives is “increase/build/create networking” and I always found it odd that there is never any attempt to measure if networking really took place. A possible solution is to map networks created by participants at conferences – and compare these networks to those that existed before the conferences. This is exactly what I have done recently in a network mapping study that you can view here (pdf – 1 MB) and the above image is from. From the LIFT conference of 2007, we mapped the networks of 28 participants (out of 450 total participants) before and after the conferences. We found some quite surprising results:…”

  • Evaluating Performance of Project-Centred Research Networks: PhD thesis by Camille Ryan, 2007. Abstract” A socio-economic network analyses of federally funded research projects* Managing knowledge in new technological realms, such as genomics and nanotechnology, involves the collaboration of geographically dispersed actors across multiple disciplines from both the public and private sectors. However, evaluating performance of collaborative activity is lagging the adoption of the model. This paper adapts social networks analysis (SNA) to the task of evaluate not only the government-funded collaborative research projects themselves but to the circumstances under which they are funded as well.”

    •  

  • Social Network Analysis and Non-Profit Organizations. 2007 “Non-profits each exist in a social network. While many other sectors have at least the possibility of performing their services in a vacuum, non-profit organizations very rarely operate without a large constituency of donors, volunteers, community partners, and also exists with an unusual central sense of being ‘in the public eye’, operating as they do as guardians of a public trust. With in mind, let us examine some of the major concepts of Social Network Analysis as it relates to non-profits….(16/06/07)

  • Network Mapping as a Diagnostic Tool [A Manual], by Louise Clark. 2006. (Also in Spanish) “This publication was made possible with support from the FIT-DFID Programme, Bolivia to the FIT 3 (RedCampo) Project – Boosting the Production and Marketing of High-Value crops through ICT-enabled Information Networks – A project implemented by InforCom of the Centro Internacional de Agricultura Tropical – CIAT, to support the Bolivian Agricultural Technology System – SIBTA (Sistema Boliviano de Tecnología Agropecuaria). The results of this action-research project have demonstrated the utility of social network analysis as a diagnostic tool to improve understanding of how information flows among the different actors involved in agricultural supply chains.” Contents include: 1. Who this manual is for.,2. Introduction to Social Network Analysis (SNA)., 3. SNA as a diagnostic tool.., 4. Social Network Analysis: Step by step. Step 1: Survey design. Step 2: Data collection. Step 4: Preparing the database to be transferred to Netdraw..Step 5: Using Netdraw., Step 6: Try for yourself.. 5. The 2-mode network. Step 1 Survey structure. Step 2: Preparing the database.. Step 3: Transferring the data to Ucinet, Step 4: Visualising 2 mode networks with Netdraw..6. A flexible tool.” (16/07/07)

    •  

  • Investigating the Potential of Using Social Network Analysis in Educational Evaluation William R. Penuel, Willow Sussex, Christine Korbak, Christopher Hoadley American Journal of Evaluation, Vol. 27, No. 4, 437-451 (2006) This article describes results of a study investigating the potential of using social network analysis to evaluate programs that aim at improving schools by fostering greater collaboration between teachers. The goal of this method is to use data about teacher collaboration within schools to map the distribution of expertise and resources needed to enact reforms. Such maps are of great potential value to school leaders, who are responsible for instructional leadership in schools, but they also include information that could bring harm to individuals and school communities. In this article, the authors describe interview findings about concerns educators have with collecting and sharing social network data. A chief finding is that although the majority of teachers consider collecting social network data to be problematic but feasible, some teachers report concerns about privacy and the effect on their school’s goals to foster community if the data are shared with their schools.

    •  

  • New Directions in the Use of Network Analysis in Research and Development Evaluation Jonathon Mote, Jerald Hage, University of Maryland, Gretchen Jordan, Sandia National Laboratories, PowerPoint presentation at the 2006 American Evaluation Association Conference: The Consequences of Evaluation
  • Does your strategic planning make a difference? Andrew Rixon. 2005. This brief paper outlines how social network analysis (SNA) was used to assess the change resulting from the use of “Open Space Technology” as a strategic planning tool by a research organisation. A SNA survey was sent out to participants before and after the meeting. The focus of the survey was on who the participants thought “felt passionate about the theme under discussion” The paper identifies and analyses the changes that were observed via the follow up-survey.

    •  

  • A network approach for researching partnerships in health. Jenny M Lewis Department of Political Science, University of Melbourne, Parkville, 3010, Australia. Australia and New Zealand Health Policy 2005, 2:22 “The last decade has witnessed a significant move towards new modes of governing that are based on coordination and collaboration. In particular, local level partnerships have been widely introduced around the world. There are few comprehensive approaches for researching the effects of these partnerships. The aim of this paper is to outline a network approach that combines structure and agency based explanations to research partnerships in health. Network research based on two Primary Care Partnerships (PCPs) in Victoria is used to demonstrate the utility of this approach. The paper examines multiple types of ties between people (structure), and the use and value of relationships to partners (agency), using interviews with the people involved in two PCPs – one in metropolitan Melbourne and one in a rural area.”

    •  

  • New Directions for Evaluation Volume 2005, Issue 107 , Special Issue: Social Network Analysis in Program Evaluation Issue Edited by Maryann M. Durland, Kimberly A. Fredericks. Published Online: 2 Feb 2006 Contents: An introduction to social network analysis (p 5-13); The historical evolution and basic concepts of social network analysis (p 15-23); Exploring and understanding relationships (p 25-40); An evaluation of communication among high school faculty using network analysis (p 41-53); Network analysis of a demonstration program for the developmentally disabled (p 55-68); Application of network analysis in evaluating knowledge capacity (p 69-79); A formative evaluation of the integration of two departments (p 81-94); The value of social network analysis in health care delivery (p 95-98); Next steps for nodes, nets, and SNA analysis in evaluation (p 99-101)

    •  

  • Mapping the Distribution of Expertise and Resources in a School: Investigating the Potential of Using Social Network Analysis in Evaluation William R. Penuel Willow Sussex Christine Korbak SRI International Paper presented at the Joint Conference of the Canadian Evaluation Society and theAmerican Evaluation Association, Toronto, Ontario, Canada, October 2005.“This paper describes results of a study investigating the potential of using social network analysis to evaluate the capacity ofa school to undertake a schoolwide educational reform. The goal of this method is to use data about teacher collaboration within schools to map and explain the distribution of expertise and resources needed to enact reforms. Such maps are of great potential value to school leaders, who are responsible for instructional leadership in schools; but they also include information that can potentially bring harm to individuals and school communities. In this paper, we describe interview findings about potential concerns teachers might have for collecting and sharing social network data. In addition, we describe some of the procedures we undertook to protect participants’ rights and minimize potential harm that could arise from sharing information about collegial interactions with evaluation researchers in a subsequent study in our project” (28/04/06)

    •  

  • Network Approaches to Global Civil Society. Helmut Anheier and Hagai Katz. “…our focus is on global civil society as a transnational system of social networks and, methodologically speaking, on analysing global civil society through the lens of network analysis. We are interested in finding out how useful the various approaches and tools of network analysis are for describing, analysing and understanding global civil society.” Being Chapter 4 of Global Civil Society 2004/5 Anheier, Helmut, Marlies Glasius and Mary Kaldor (eds.). London: Sage, 2004 (Posted 05/12/05)

    •  

  • Network Perspectives In The Evaluation Of Development Interventions: More Than A Metaphor. Rick Davies, for the EDAIS Conference November 24-25, 2003 New Directions in Impact Assessment for Development: Methods and Practice. “In this paper I argue the case for the use of a network perspective in representing and evaluating aid interventions. How we represent the intentions of aid activities has implications for how their progress and impact can be assessed. Because our representations are by necessary selective simplifications of reality they will emphasise some aspects of change and discourage attention to others. The benchmark alternative here is by default the Logical Framework, the single most commonly used device for representing what an aid project or programme is trying to do. Five main arguments are put forward in favour of a network perspective as the better alternative, along with some examples of their use. Firstly, social network analysis is about social relationships, and that is what much of development aid is about. Not abstract and disembodied processes of change. Secondly, there is wide range of methods for measuring and visualising network structures. These provide a similarly wide range of methods of describing expected outcomes of interventions in network terms. Thirdly, there is also a wide range of theories about social and other networks. They can stimulate thinking about the likely effects of development interventions. Fourthly, network representations are very scalable, from very local developments to the very global, and they can include both formal and informal structures. They are relevant to recent developments in the delivery of development aid. Fifthly, network models of change can incorporate mutual and circular processes of influence, as well as simple linear processes of change. This enables them to represent systems of relationships exhibiting varying degrees of order, complexity and chaos. Following this argument I outline some work-in-progress, including ways in which the conference participants may themselves get involved. Finally I link this paper into its own wider web of intellectual influences and history. ” Posted 29/10/03)

 

  • Organizational Network Analysis as a Tool for Program Evaluation, Merrill Eisenberg, Nancy Swanson, University of Connecticut Health Center, Evaluation & the Health Professions, Vol. 19, No. 4, 488-506 (1996) Abstract: “Health program evaluation is generally focused on an examination of individual program characteristics and accomplishments, yet many programs are part of a broader service system. Evaluation of the role a program plays in that system is an important evaluation question to address. A network analysis of program referral patterns was used to evaluate Connecticut’s Healthy Start program. Network analysis showed that Healthy Start played a “broker” role in 4 case study communities, sending and receiving referrals of pregnant women to a higher than average number of other programs. Further, in the urban area case study, competing market players providing services to pregnant women resulted in subsets of services with dense referral patterns within the subsets, but little referral between subsets. Healthy Start was found to be instrumental as an integrator of these otherwise disconnected service subsets”

 

  • [Readers: Feel free to propose additions to this list]

 

Evaluation societies, associations and groups: A list

The most comprehensive list available is maintained by the International Organisation for Cooperation in Evaluation (IOCE)

Regional evaluation organisations (9 as of June 2013)

If you have updated contact information for any of these groups, or know of a group not included on these lists, but perhaps should be,  whether it be formally organized as a society or association or an informal network, whose members are professional evaluators, please email Jim Rugh

But please also note that there is a seperate list of M&E email lists kept here on the MandE NEWS website

The Logical Framework: A list of useful documents

Contents: 1. Explanations of the Logical Framework | 2. Wider discussions of Logic Models | 3. Critiques of the Logical Framework |  4. Alternative versions of the Logical Framework |  5. The Editor’s concerns (about uses of the Logical Framework) | 6. Online survey on views and usage of the Logical Framework

Please feel free to suggest additions or corrections to this list, by using the Comment facility at the end of this post

New section: Software

1. Explanations of the Logical Framework

2. Wider discussions of Logic Models

  • Program logic – an introduction, provided by Audience Dialogue (2007)
  • Enhancing Program Performance with Logic Models This course introduces a holistic approach to planning and evaluating education and outreach programs. Module 1 helps program practitioners use and apply logic models. Module 2 applies logic modeling to a national effort to evaluate community nutrition education. Provided by the University of Wisconsin (2007)
  • Online Logic Model training: an audiovisual presentation by Usable Knowledge, USA Twenty minutes long, with a menu that can be used to navigate to the sections of interest (2006)
  • Network Perspectives In The Evaluation Of Development Interventions: More Than A Metaphor.  Rick Davies, for the EDAIS Conference November 24-25, 2003 New Directions in Impact Assessment for Development: Methods and Practice. “In this paper I argue the case for the use of a network perspective in representing and evaluating aid interventions. How we represent the intentions of aid activities has implications for how their progress and impact can be assessed. Because our representations are by necessary selective simplifications of reality they will emphasise some aspects of change and discourage attention to others. The benchmark alternative here is by default the Logical Framework, the single most commonly used device for representing what an aid project or programme is trying to do. Five main arguments are put forward in favour of a network perspective as the better alternative, along with some examples of their use. Firstly, social network analysis is about social relationships, and that is what much of development aid is about. Not abstract and disembodied processes of change. Secondly, there is wide range of methods for measuring and visualising network structures. These provide a similarly wide range of methods of describing expected outcomes of interventions in network terms. Thirdly, there is also a wide range of theories about social and other networks. They can stimulate thinking about the likely effects of development interventions. Fourthly, network representations are very scalable, from very local developments to the very global, and they can include both formal and informal structures. They are relevant to recent developments in the delivery of development aid. Fifthly, network models of change can incorporate mutual and circular processes of influence, as well as simple linear processes of change. This enables them to represent systems of relationships exhibiting varying degrees of order, complexity and chaos. Following this argument I outline some work-in-progress, including ways in which the conference participants may themselves get involved. Finally I link this paper into its own wider web of intellectual influences and history. ” (Posted here 2003)
  • The Temporal Logic Model: A Concept Paper, by Molly den Heyer. On the IDRC website. (2002)
  • A Bibliography for Program Logic Models/Logframe AnalysisDecember 18, 2001 Compiled by: Molly den Heyer Evaluation Unit, International Development Research Centre
  • W K Kellogg Foundation Logic Model Development Guide. (2001) Using Logic Models to Bring Together Planning, Evaluation, and Action. Updated (original was published in 1998) “The program logic model is defined as a picture of how your organization does its work – the theory and assumptions underlying the program.A program logic model links outcomes (both short- and long-term) with program activities/processes and the theoretical assumptions/principles of the program.”
  • Application of Logic Modeling Processes to Explore Theory of Change from Diverse Cultural Perspectives Ricardo Millett, Sharon Dodson, & Cynthia Phillips American Evaluation Association November 4, 2000
  • The state of the art of Logic Modelling. PowerPoint presentation by Gretchen Jordan (1999?)
  • The Logic Model for Program Planning and Evaluation, Paul F McCawley, 1997, University of Idaho Extension.

3. Critiques of the Logical Framework

  • Debunking misconceptions around the Logical Framework Approach through reviewing available literature by Munyaradzi Madziwa, August 2016
  • Critical Study Of The Logical Framework Approach In The Basque Country (2011) by ECODE, Bilbao.
  • THE USE AND ABUSE OF THE LOGICAL FRAMEWORK APPROACH A Review of International Development NGOs’ Experiences. A report for Sida. November 2005. Oliver Bakewell and Anne Garbutt, of INTRAC. “In this review, we have attempted to take stock of the current views of international development NGOs on the LFA and the ways in which they use it. We start in the next section by considering the different meanings and connotations of the term logical framework approach as it is used by different actors. In Section 3 we look at how LFAs are used by INGOs in both planning and project management. The next section reviews some of the debates and critiques around the LFA arising both from practice and the literature. In response to these challenges, different organisations have adapted the LFA and these variations on the LFA theme are outlined in Section 5. We conclude the paper by summarising the findings and reflecting on ways forward. …This review has been commissioned by Sida as part of a larger project which aims to establish new guidelines for measuring results and impact and reporting procedures for Swedish development NGOs receiving support from Sida. “

4. Alternative versions of the Logical Framework

  • Beyond Logframe: Using Systems Concepts in Evaluation, 2010, FASD (Foundation  for Advanced Studies on International Development)
  • The Social Framework, an actor-oriented adaptation of the Logical Framework, developed by Rick Davies. The sequence of rows found in a Logical Framework now represent a sequence of actors, connected to each other by their relationships, and forming a specific pathway through a wider network of actors. Narrative descriptions of expected changes, indicators of those change and means of verification are still found in the columns, but these relate to actors and their relationships. Actors can be individuals, groups, organisations or type of organisations. The assumptions column still exists, but the assumptions refers to important connections to other actors outside the specific pathway.
  • Can OM and LFA share a space? “OM (Outcome Mapping) and LFA may be useful at different levels, for diverse types of interventions or for information and in different contexts. Rather than pitting LFA and OM against each other, we need to understand what kinds of information and uses each has, as well as their advantages and disadvantages, and find ways for them to add value to each other.” See also Logical Framework Approach and Outcome Mapping: A Constructive Attempt of Synthesis. A Discussion Paper by Daniel Roduner and Walter Schläppi, AGRIDEA; Walter Egli, NADEL (ETH Zurich)
  • Logical Framework Approach – with an appreciative approach. April 2006 SIDA Civil Society Centre. “As a part of its effort to realise the intentions of Sweden’s Policy on Global Development, Sida Civil Society Center (SCSC) initiated a development project in 2005 together with PMU Interlife (the Swedish Pentecostal Mission’s development cooperation agency) and consultant Greger Hjelm of Rörelse & Utveckling. The goal was to create a working model which combines the goal hierarchy and systematics from the Logical Framework Approach (LFA)1 with the approach used in the Appreciative Inquiry tool (AI). AI is both a working method and an approach. In analysing strengths and resources, motivation and driving forces, the focus is placed on the things which are working well, and on finding positive action alternatives for resolving a situation. LFA, which is an established planning model in the field of international development, is found by many to be an overly problem-oriented model. Using this approach, one proceeds based on a situation in which something is lacking, formulates the current situation as a “problem tree”, and thus risks failing to perceive resources which are actually present, and a failure to base one’s support efforts on those resources. Working in close cooperation, we have now formulated a new working method for planning using LFA, one which is built on appreciative inquiry and an appreciative approach. The model was tested by PMU Interlife’s programme officers and their cooperating partners in Niger, Nicaragua and Tanzania during the autumn of 2005. Their experiences have been encouraging, and it is our hope that more Swedish organisations and their cooperating partners will try our model and working method.(Posted 01/07/06)
  • No more log frames!! People-Focused Program Logic Two day workshop Monday 19th and Tuesday 20th of September 2005, in Melbourne, Australia. “Purpose of the workshop: • To understand what ‘people-focused’ program logic is and how to use it • To build a people-focused program logic for their own project Who should attend? People with monitoring and evaluation interests who are working on projects with capacity building components. Course description: In this workshop, participants will build their own ‘people-focused’ logic model. To do this they will analyse the key beneficiaries of their project, build their program logic model around this analysis, and consider assumptions made in the logic. The program logic will be built around a generic theory of how capacity building works, that can be modified to include elements of advocacy and working with or through partners. Participants will also learn how this logic can be used to form the spine of their monitoring, evaluation and improvement framework. As participants will be invited to develop their own program logic model, they are encouraged to bring along others from the same project team. Examples of frameworks, and a workbook will be provided to participants” For additional information: Jo Leddy of Clear Horizon Phone: 03 9783 3662 E-mail: Jo@clearhorizon.com.au Website: www.clearhorizon.com.au See rest of the flyer for more information…(Posted 21/06/05)
  • Intertwining Participation, Rights Based Approach and Log-Frame: A way forward in Monitoring and Evaluation for Rights Based Work. Partha Hefaz Shaikh Initial Draft – Circulated for discussion. “Programme implementation through Rights Based Approach (RBA) in ActionAid Bangladesh started in 2000 and it took us quite a while to understand what it meant to implement programmes in a RBA environment. Side by side we were also grappling with issues of monitoring and evaluation of programmes implemented through a rights based approach. In order to develop a more meaningful framework that has all the elements of participation, RBA and log-frame we developed what we call “Planning and Implementation Framework Analysis (PIFA)”. ” (Posted 20/05/05)
  • A MODIFIED LOGFRAME FOR USE IN HUMANITARIAN EMERGENCIES. by Bernard Broughton (I think)
  • Family Planning Logical Framework (with two parallel processes, one feeding back into the other)
  • Build Reach into Your Logic Model. Steve Montague February 1998 “Analysts have frequently noted the importance of constructing logic models (a.k.a. logic charts, causal models, logical frameworks, and most recently performance frameworks – among other names) to explain the causal theory of a program or initiative before attempting to monitor, measure, or assess performance. …A key limitation to the logic models of the 1980s, as well as many of those in current use, has been their tendency to focus predominantly on causal chains without reference to who and where the action was taking place. “
  • Bennett’s Hierarchy (or Targetting Outcomes of Programs (TOP)). This is not a version of the LogFrame, but it is another type of logic model with multiple steps (7 levels). It has been used widely in the evaluation of agricultural extension activities in Australia. It was originally developed by Bennett in 1975.

5. The Editor’s concerns (about uses of the Logical Framework)

  1.  Long, complex, unreadable sentences, in the narrative column of the Logical Framework
    1. Often the result of compromises between many different parties who have been negotiating the contents of the Logical Framework. Net result: an unreadable document
    2. Sometimes the result of people not knowing that the whole story does not need to be told in one sentence. The row below should say what happens before (the cause) and the row above should say what happens next (the effects)
    3. Sometimes the result of people forgetting there is a column for indicators next door, where they can provide lots of interesting detail about what is expected to happen at this stage
  2. Narrative statements without people in them. E.g “Rice productivity increased”
    1. Another reason some many Logical Frameworks are so unreadable, and so boring when they are readable, is that somehow their authors have managed to leave out people. Instead we have lots of abstract and disembodied processes. And then we wonder why some people have difficulty understanding Logical Frameworks
  3. Means of Verification that refer to reports and surveys, but not who is responsible for generating and / or providing this information (and when it will be available)
    1. This problem is similar to the above, reflecting a continuing aversion to making references to real people in Logical Frameworks.
    2. One consequence is lack of clear ownership and responsibility for M&E of the changes being described at that level of the Logical Framework
  4. Insistence on there being only one Purpose level statement in a Logical Framework
    1. I have recent experience of colleagues insisting on this. For reasons I have not yet established, beyond the “it is not allowed” variety. Insisting on one Purpose and One Goal really is pushing a very linear model of reality. It does not even allow for any parallel but convergent events, such as those usually come through problem tree analyses that sometimes precede the design of a Logical framework
  5. Ambitous narrative statements coupled with modest indicators / Overly simple indicators used to describe complex developments.
    1. Such as “number of meetings held” as an indicator for the functioning of stakeholder’s advisory committee. For an alternative, see “Checklists as mini-theories of change
  6. Lists of indicators in no apparent order
    1. “A (unsorted) list is not a strategy” A sorted list can convey relative importance (most important indicator at the top), or a sequence (starting from the bottom), or multiple alternative routes to the objective in the narrative column. If there is a list, the reader should be told what sort of list it is.
  7. Broad generalisations at the Goal level
    1. Sometimes arising from confusion of a temporal hierarchy (A leads to B which leads to C which leads to D) and a nested hierarchy (A is part of B which is part of C which is part of D). The Logical Framework is supposed to be a temporal hierarchy, that tells a story. Not a pile of increasingly broad statements about the same thing
  8. Confusion over the meaning of different levels in a Logical Framework. Between Activities and Outputs, Outputs and Purpose level outcomes, and outcomes at the Purpose and Goal level.
    1. Often cause by leaving people out of the picture, as above.
    2. A workable rule of thumb, for seperating levels of the Logical Framework
      1. Activities are things that “the project” can control. The boundary of a project being defined by the reach of its contracts (with staff, consultants, suppliers and sub-contractors)
      2. Outputs are the activities of the project (if services), or their results (if goods), that people and organisations outside the project can use e.g workshops, publications, trainings, etc. Ask here: What is available to who, and in what form?
      3. Purpose level changes (outcomes), are changes in those people or organisations who have used those goods or services. Normally the project would hope to influence these (and learn about how it can have influence) but it would not be expected to control events at this level
      4. Goal level changes (outcomes), are longer term changes in those same people or organisations, or others they have subsequently interacted with.
  9. Long lists of assumptions
    1. Apparently designed to cover people’s backsides
    2. Including many events that the project should be able to influence
      1. …which therefore should be listed as one of the outputs or outcomes. I.e. brought into the central narrative of the Logical Framework
  10. Things the Logical Framework cant do very well, even in the best of hands
    1. Represent multiple parallel processes, as distinct from a single process
      1. E.g. What people are doing at multiple project locations, within a single national project
        1. Representing their interactions is even more of a challenge
    2. Represent the interactions between multiple events at the same level of a Logical framework.
      1. E.g. How different project outputs (manuals, training events, newsletters, websites, etc) feed into each other
      2. Or, how different health outcomes (at Purpose level) feed into each other, before finally contributing to Goal level changes e.g. reduced mortality
    3. Represent the interactions between multiple outputs and the many users of those outputs
      1. E.g., the range of communications products used by a range of clients of a project . Many people will use multiple products, but their usage patterns will vary. Many products will be used by multiple users, but their user groups will vary.

All these processes can be represented by network models. See the new page on developing network models of development projects. However network models are generally too complex to provide a substitute for the Logical Framework. One proposed alternative is the Social Framework, originally described here and now updated here. The Social Framework can be used to describes a pathway through a network, in a way that capable of being monitored and evaluated. Your comments are welcome.

6. Online survey into the uses of the Logical Framework

Please consider taking part in this survey. You can access the cumulative results to date at the end of the survey form. It is not long.

thanks, rick davies

M&E blogs: A List

  • EvalThoughts, by Amy Germuth, Durham, NC, United States, President of EvalWorks, LLC a woman-owned small evaluation and survey research consulting business in Durham, NC.
  • Evaluation and Benchmarking. “This weblog is an on-line workspace for the whole of Victorian government Benchmarking Community of Practice.”
  • M&E Blog, by…?
  • Aid on the Edge of Chaos, by Ben Ramalingam
  • Design, Monitoring and Evaluation, by LARRY DERSHEM – Tbilisi, Georgia
  • Managing for Impact: About “Strengthening Management for Impact” for MFIs
  • Genuine Evaluation: “Patricia J Rogers and E Jane Davidson blog about real, genuine, authentic, practical evaluation”
  • Practical Evaluation, by Samuel Norgah
  • AID/IT M&E Blog: “…is written by Paul Crawford, and is part of a wider AID/IT website”
  • Blog: Evaluateca: Spanish language evaluation blog maintained by Rafael Monterde Diaz. Information, news, views and critical comments on Evaluation
  • Empowerment Evaluation Blog “This is a place for exchanges and discussions about empowerment evaluation practice, theory, and current debates in the literature” Run by  Dr. David Fetterman”
  • E-valuation: “constructing a good life through the exploration of value and valuing” by Sandra Mathison,Professor, Faculty of Education, University of British Columbia
  • Intelligent Measurement. This blog is created by Richard Gaunt in London and Glenn O’Neil in Geneva and focuses on evaluation and measurement in communications, training, management and other fields.
  • Managing for Impact: Let’s talk about MandE! “Welcome to the dedicated SMIP ERIL blog on M&E for managing for impact!An IFAD funded Regional Programme, SMIP (Strengthening Management for Impact) is working with pro-poor initiatives in eastern & southern Africa to build capacities to better manage towards impact. It does so through training courses for individuals, technical support to projects & programmes, generating knowledge, providing opportunities for on-the-job-training, and policy dialogue.”
  • MCA Monitor Blog “…is a part of CGD’s MCA Monitor Initiative, which tracks the effectiveness of the US Millennium Challenge Account. Sheila Herrling, Steve Radelet and Amy Crone, key members of CGD’s MCA Monitor team, contribute regularly to the blog. We encourage you to join the discussion by commenting on any post”
  • OutcomesBlog.Org “Dr Paul Duignan on real world strategy, outcomes, evaluation & monitoring Dr Paul Duignan is a specialist in outcomes, performance management, strategic decision making, evaluation and assessing research and evidence as the basis for decision making. He has developed the area of outcomes theory and its application in Systematic Outcomes Analysis, the outcomes software DoView and the simplified approach to his work Easy Outcomes. He works at an individual, organizational and societal level to develop ways of identifying and measuring outcomes which facilitate effective action. For a bio see here.
  • Rick on the Road: “Reflections on the monitoring and evaluation of development aid projects, programmes and policies, and development of organisation’s capacity to do the same. This blog also functions as the Editorial section of the MandE NEWS website“
  • The Usable Blog “A blog on “Thoughts, ideas and resources for non-profit organizations and funders about the independent sector in general and program evaluation in particular” By Eric Graig “
  • The MSC Translations blog is maintained by Rick Davies, and is part of the MandE NEWS website. The purpose of this blog is:1. To make available translations of the MSC Guide in languages other than English. 2. To solicit and share comments on the quality of these translations, so they can be improved.The original English version can be found here The ‘Most Significant Change’ (MSC) Technique: A Guide to Its Use
  • Zen and the art of monitoring & evaluation “This blog is some of the rambling thoughts of Paul Crawford, a monitoring & evaluation (M&E) consultant for international aid organisations” Paul is based in Australia.

And other lists of M&E blogs

%d bloggers like this: