Social Network Analysis and Evaluation: A List

Preface to the list

“There are however three challenges in promoting the use of network models. One is to get people thinking in terms of networks as a kind of a base metaphor, in the same way that in the past people may have seen clockworks as a base metaphor for how the world works. The second is to sift through what often appears to be a surfeit of technical capacity to analyse networks, in order to focus in on the simplest and often most useful basics. The third challenge is to develop more participative and interpretative approaches to the description and analysis of networks, in contrast to the number crunching nature of much social network analysis in the academic world. This is all very much a ‘work in progress‘.” Rick Davies, in newsletter article for American Evaluation Association, 2008

The list

• Using Social Network Analysis in Evaluation: A Report to the Robert Wood Johnson Foundation. By Kimberly Fredericks, PhD, MPA, RD, Associate Professor
  The Sage Colleges School of Management, 2013  “…Using data gathered from telephone interviews with the program officers, email surveys from project directors, and supplemental information from nine project summaries (see Appendix 1 for the summaries), this report: ? Describes how the Foundation took deliberate steps to create an emerging learning
  community to support using social network analysis to understand and evaluate its funded work ? Illustrates how social network analysis can be used in many applications, ranging
  from a one-time study to longitudinal studies, with a summative purpose (to evaluate the success of a particular intervention) or formative purpose (for learning and improvement)
  ? Identifies the challenges associated with using social network analysis, in terms of understanding the methodology’s limits or constraints, the implications presented by
  the size and scope of the projects, and the need to develop the technical expertise and capacity to manage the projects and use the data”

 

• Using Social Network Analysis to Advance Traditional Qualitative Methods in Evaluation and Program Design. Caroline J. Wilson, Anne E. Dougherty, Mary Sutter, Jennifer Mitchell-Jackson,  Pamela Wellner,  Nick Hall, ABSTRACT Social network analysis (SNA) is a technique used to study relationships between actors, such as people or organizations. It has been applied to a wide array of disciplines, on topics ranging from destabilizing Al-Qaeda to explaining campaign fundraising success. While these applications may seem far removed from the efforts of AESP members, this paper will provide insights into the innovative ways social network analysis can be used in energy-related evaluations. We start by providing parameters for the use of this technique for evaluation efforts and articulate a number of researchable issues central to evaluation that may be answered through social network analysis. Our evaluation team then demonstrates how this method was employed to conduct a more thorough process evaluation as a complementary approach to other, more traditional research techniques. We also demonstrate the value of the social network analysis approach by comparing a network map with a more traditional organization and implementation chart. We then outline other ways it was utilized, including discussion of social network analysis measures such as centrality and density. This paper aims to add value to current evaluation methods and introduce social network analysis to the best practices of program evaluation and design.

• Social Network Analysis: A Useful Tool for Visualizing and Evaluating Forestry Research. N.L. Klenk1, G.M. Hickey1, J.I. MacLellan2, R. Gonzales3 and J. Cardille3 International Forestry Review 11(1):134-140. 2009 SUMMARY “One of the foundational studies of social network analysis produced a depiction of scientific collaboration by tracing a network of scientific papers linked by co-authorships and co-citations, which has since spurred numerous studies on the typology, organization and dynamics of scientific research networks. This paper introduces social network analysis and its analytical measures of network structure. It then demonstrates the utility of social network analysis in forestry, in the evaluation of large research networks such as the Sustainable Forest Management Network (SFMN), and suggests other important uses of network visualization to facilitate exploring, discovering and selecting resources in a database.

 

• Social Network Analysis and the Evaluation of Leadership Networks. Bruce Hoppe, Ph.D., Claire Reinelt, Ph.D. April 2008 . Abstract: “Leadership development practitioners have become increasingly interested in the formation of leadership networks as a way to sustain and strengthen relationships among leaders within and across organizations, communities, and systems. This paper offers a framework for conceptualizing different types of leadership networks and identifies the outcomes that are typically associated with each type of network. One of the challenges for the field of leadership development has been how to evaluate leadership networks. Social Network Analysis (SNA) is a promising evaluation approach that uses mathematics and visualization to represent the structure of relationships between people, organizations, sectors, silos, communities and other entities within a larger system. Core social network concepts are introduced and explained to illuminate the value of SNA as an evaluation and program tool.”
•  

  Conference evaluation and network mapping Glenn O’Neil (2008) “Often we attend conferences where one of the stated objectives is “increase/build/create networking” and I always found it odd that there is never any attempt to measure if networking really took place. A possible solution is to map networks created by participants at conferences – and compare these networks to those that existed before the conferences. This is exactly what I have done recently in a network mapping study that you can view here (pdf – 1 MB) and the above image is from. From the LIFT conference of 2007, we mapped the networks of 28 participants (out of 450 total participants) before and after the conferences. We found some quite surprising results:…”

  • Editor’s note: Look here for a similar (but perhaps more ad hoc) network analysis of a workshop held in the Netherlands, May 2008

• Evaluating Performance of Project-Centred Research Networks: PhD thesis by Camille Ryan, 2007. Abstract” A socio-economic network analyses of federally funded research projects* Managing knowledge in new technological realms, such as genomics and nanotechnology, involves the collaboration of geographically dispersed actors across multiple disciplines from both the public and private sectors. However, evaluating performance of collaborative activity is lagging the adoption of the model. This paper adapts social networks analysis (SNA) to the task of evaluate not only the government-funded collaborative research projects themselves but to the circumstances under which they are funded as well.”

 

• Social Network Analysis and Non-Profit Organizations. 2007 “Non-profits each exist in a social network. While many other sectors have at least the possibility of performing their services in a vacuum, non-profit organizations very rarely operate without a large constituency of donors, volunteers, community partners, and also exists with an unusual central sense of being ‘in the public eye’, operating as they do as guardians of a public trust. With in mind, let us examine some of the major concepts of Social Network Analysis as it relates to non-profits….(16/06/07)

• Network Mapping as a Diagnostic Tool [A Manual], by Louise Clark. 2006. (Also in Spanish) “This publication was made possible with support from the FIT-DFID Programme, Bolivia to the FIT 3 (RedCampo) Project – Boosting the Production and Marketing of High-Value crops through ICT-enabled Information Networks – A project implemented by InforCom of the Centro Internacional de Agricultura Tropical – CIAT, to support the Bolivian Agricultural Technology System – SIBTA (Sistema Boliviano de Tecnología Agropecuaria). The results of this action-research project have demonstrated the utility of social network analysis as a diagnostic tool to improve understanding of how information flows among the different actors involved in agricultural supply chains.” Contents include: 1. Who this manual is for.,2. Introduction to Social Network Analysis (SNA)., 3. SNA as a diagnostic tool.., 4. Social Network Analysis: Step by step. Step 1: Survey design. Step 2: Data collection. Step 4: Preparing the database to be transferred to Netdraw..Step 5: Using Netdraw., Step 6: Try for yourself.. 5. The 2-mode network. Step 1 Survey structure. Step 2: Preparing the database.. Step 3: Transferring the data to Ucinet, Step 4: Visualising 2 mode networks with Netdraw..6. A flexible tool.” (16/07/07)

 

• Investigating the Potential of Using Social Network Analysis in Educational Evaluation William R. Penuel, Willow Sussex, Christine Korbak, Christopher Hoadley American Journal of Evaluation, Vol. 27, No. 4, 437-451 (2006) This article describes results of a study investigating the potential of using social network analysis to evaluate programs that aim at improving schools by fostering greater collaboration between teachers. The goal of this method is to use data about teacher collaboration within schools to map the distribution of expertise and resources needed to enact reforms. Such maps are of great potential value to school leaders, who are responsible for instructional leadership in schools, but they also include information that could bring harm to individuals and school communities. In this article, the authors describe interview findings about concerns educators have with collecting and sharing social network data. A chief finding is that although the majority of teachers consider collecting social network data to be problematic but feasible, some teachers report concerns about privacy and the effect on their school’s goals to foster community if the data are shared with their schools.

 

• New Directions in the Use of Network Analysis in Research and Development Evaluation Jonathon Mote, Jerald Hage, University of Maryland, Gretchen Jordan, Sandia National Laboratories, PowerPoint presentation at the 2006 American Evaluation Association Conference: The Consequences of Evaluation

• Does your strategic planning make a difference? Andrew Rixon. 2005. This brief paper outlines how social network analysis (SNA) was used to assess the change resulting from the use of “Open Space Technology” as a strategic planning tool by a research organisation. A SNA survey was sent out to participants before and after the meeting. The focus of the survey was on who the participants thought “felt passionate about the theme under discussion” The paper identifies and analyses the changes that were observed via the follow up-survey.

 

• A network approach for researching partnerships in health. Jenny M Lewis Department of Political Science, University of Melbourne, Parkville, 3010, Australia. Australia and New Zealand Health Policy 2005, 2:22 “The last decade has witnessed a significant move towards new modes of governing that are based on coordination and collaboration. In particular, local level partnerships have been widely introduced around the world. There are few comprehensive approaches for researching the effects of these partnerships. The aim of this paper is to outline a network approach that combines structure and agency based explanations to research partnerships in health. Network research based on two Primary Care Partnerships (PCPs) in Victoria is used to demonstrate the utility of this approach. The paper examines multiple types of ties between people (structure), and the use and value of relationships to partners (agency), using interviews with the people involved in two PCPs – one in metropolitan Melbourne and one in a rural area.”

 

• New Directions for Evaluation Volume 2005, Issue 107 , Special Issue: Social Network Analysis in Program Evaluation Issue Edited by Maryann M. Durland, Kimberly A. Fredericks. Published Online: 2 Feb 2006 Contents: An introduction to social network analysis (p 5-13); The historical evolution and basic concepts of social network analysis (p 15-23); Exploring and understanding relationships (p 25-40); An evaluation of communication among high school faculty using network analysis (p 41-53); Network analysis of a demonstration program for the developmentally disabled (p 55-68); Application of network analysis in evaluating knowledge capacity (p 69-79); A formative evaluation of the integration of two departments (p 81-94); The value of social network analysis in health care delivery (p 95-98); Next steps for nodes, nets, and SNA analysis in evaluation (p 99-101)

 

• Mapping the Distribution of Expertise and Resources in a School: Investigating the Potential of Using Social Network Analysis in Evaluation William R. Penuel Willow Sussex Christine Korbak SRI International Paper presented at the Joint Conference of the Canadian Evaluation Society and theAmerican Evaluation Association, Toronto, Ontario, Canada, October 2005.“This paper describes results of a study investigating the potential of using social network analysis to evaluate the capacity ofa school to undertake a schoolwide educational reform. The goal of this method is to use data about teacher collaboration within schools to map and explain the distribution of expertise and resources needed to enact reforms. Such maps are of great potential value to school leaders, who are responsible for instructional leadership in schools; but they also include information that can potentially bring harm to individuals and school communities. In this paper, we describe interview findings about potential concerns teachers might have for collecting and sharing social network data. In addition, we describe some of the procedures we undertook to protect participants’ rights and minimize potential harm that could arise from sharing information about collegial interactions with evaluation researchers in a subsequent study in our project” (28/04/06)

 

• Network Approaches to Global Civil Society. Helmut Anheier and Hagai Katz. “…our focus is on global civil society as a transnational system of social networks and, methodologically speaking, on analysing global civil society through the lens of network analysis. We are interested in finding out how useful the various approaches and tools of network analysis are for describing, analysing and understanding global civil society.” Being Chapter 4 of Global Civil Society 2004/5 Anheier, Helmut, Marlies Glasius and Mary Kaldor (eds.). London: Sage, 2004 (Posted 05/12/05)

 

• Network Perspectives In The Evaluation Of Development Interventions: More Than A Metaphor. Rick Davies, for the EDAIS Conference November 24-25, 2003 New Directions in Impact Assessment for Development: Methods and Practice. “In this paper I argue the case for the use of a network perspective in representing and evaluating aid interventions. How we represent the intentions of aid activities has implications for how their progress and impact can be assessed. Because our representations are by necessary selective simplifications of reality they will emphasise some aspects of change and discourage attention to others. The benchmark alternative here is by default the Logical Framework, the single most commonly used device for representing what an aid project or programme is trying to do. Five main arguments are put forward in favour of a network perspective as the better alternative, along with some examples of their use. Firstly, social network analysis is about social relationships, and that is what much of development aid is about. Not abstract and disembodied processes of change. Secondly, there is wide range of methods for measuring and visualising network structures. These provide a similarly wide range of methods of describing expected outcomes of interventions in network terms. Thirdly, there is also a wide range of theories about social and other networks. They can stimulate thinking about the likely effects of development interventions. Fourthly, network representations are very scalable, from very local developments to the very global, and they can include both formal and informal structures. They are relevant to recent developments in the delivery of development aid. Fifthly, network models of change can incorporate mutual and circular processes of influence, as well as simple linear processes of change. This enables them to represent systems of relationships exhibiting varying degrees of order, complexity and chaos. Following this argument I outline some work-in-progress, including ways in which the conference participants may themselves get involved. Finally I link this paper into its own wider web of intellectual influences and history. ” Posted 29/10/03)

• Organizational Network Analysis as a Tool for Program Evaluation, Merrill Eisenberg, Nancy Swanson, University of Connecticut Health Center, Evaluation & the Health Professions, Vol. 19, No. 4, 488-506 (1996) Abstract: “Health program evaluation is generally focused on an examination of individual program characteristics and accomplishments, yet many programs are part of a broader service system. Evaluation of the role a program plays in that system is an important evaluation question to address. A network analysis of program referral patterns was used to evaluate Connecticut’s Healthy Start program. Network analysis showed that Healthy Start played a “broker” role in 4 case study communities, sending and receiving referrals of pregnant women to a higher than average number of other programs. Further, in the urban area case study, competing market players providing services to pregnant women resulted in subsets of services with dense referral patterns within the subsets, but little referral between subsets. Healthy Start was found to be instrumental as an integrator of these otherwise disconnected service subsets”

• [Readers: Feel free to propose additions to this list]