Tag: systems

M&E Software: A List

Well, the beginnings of a list…

PLEASE NOTE: No guarantee can be given about the accuracy of information provided on the linked websites about the M&E software concerned, and its providers. Please proceed with due caution when downloading any executable programs.

Contents on this page: Stand alone systems | Online systems | Survey supporting software | Sector specific tools | Qualitative data analysis | Data mining / Predictive Modelling | Program Logic / Theory of Change modeling | Dynamic models | Excel-based tools | Uncategorised and misc other

If you have any advice or opinions on any of the applications below, please tell us more via this survey.

Stand-alone systems

  • AidProject M+E for Donor-funded aid projects
  • Flamingo and Monitoring Organiser: “In order to implement FLAMINGO, it is crucial to first define the inputs (or resources available), activities, outputs and outcomes”
  • HIV/AIDS  Data Capturing And Reporting Platform[Monitoring and Evaluation System]
  • PacPlan: “Results-Based Planning, Monitoring and Evaluation Software and Process Solution”
  • Prome Web: A project management, monitoring and evaluation software. Adapted for aid projects in developing countries
  • Sigmah: “humanitarian project management open source software”

Online systems

  • Activity Info: “an online humanitarian project monitoring tool, which helps humanitarian organizations to collect, manage, map and analyze indicators. ActivityInfo has been developed to simplify reporting and allow for real-time monitoring”
  • AKVO: “a paid-for platform that covers data collection, analysis, visualisation and reporting”
  • Canva Mind Maps: “Create a mind map with Canva and bring your thoughts to life. Easy to use, completely online and completely free mind mapping software”
  • DevResults: “web-based project management tool specially designed for the international development community.” Including M&E, mapping, budgeting, checklists, forms, and collaboration facilities.
  • Granity: “Management and reporting software for Not-for-profits Making transparency easy”
  • IndiKit: Guidance on SMART indicators for relief and development programmes
  • Kashana: An open sourced, web-based Monitoring, Evaluation & Learning (MEL) product for development projects and organisations
  • Kinaki: “Kinaki is a unique and intuitive project design, data collection, analysis, reporting and sharing tool”
  • KI-PROJECTS™ MONITORING AND EVALUATION SOFTWARE:
  • Kobo Toolbox: “a free, more user-friendly way to deploy Open Data Kit surveys. It was developed with humanitarian purposes in mind, but could be used in various contexts (and not just for surveys). There is an Android data collection app that works offline”
  • Logalto:”Collaborative Web-Based Software for Monitoring and Evaluation of International Development Projects”
  • M&E Online: “Web-based monitoring and evaluation software tool”
  • Monitoring and Evaluation Online: Online Monitoring and Evaluation Software Tool
  • SmartME: “SmartME is a tried and tested comprehensive Fund Management and M&E software platform to manage funds better”
  • Systmapp: “cloud-based software that uses a patent-pending methodology to connect monitoring, planning, and knowledge management for international development organisations”
  • TolaData “is a program management and M&E platform that helps organisations create data-driven impact through the adaptive and timely management of projects”
  • WebMo: Web-based project monitoring for development cooperation

Survey supporting software

  • CommCare: a mobile data collection platform.
  • EthnoCorder is mobile multimedia survey software for your iPhone
  • HarvestYourData: iPad & Android Survey App for Mobile Offline Data Collection
  • KoBoToolbox is a suite of tools for field data collection for use in challenging environments. Free and open source
  • Magpi (formerly EpiSurvey)  – provides tools for mobile data collection, messaging and visualisation, lets anyone create an account, design forms, download them to phones, and start collecting data in minutes, for free.
  • Open Data Kit (ODK) is a free and open-source set of tools which help organizations author, field, and manage mobile data collection solution
  • REDCap,a secure web application for building and managing online surveys and databases… specifically geared to support online or offline data capture for research studies and operations
  • Sensemaker(c) “links micro-narratives with human sense-making to create advanced decision support, research and monitoring capability in both large and small organisations.”
  • Comparisons

Sector-specific tools

  • Mwater for WASH, which explicitly aims to make the data (in this case water quality). Free and open source
  • Adaptive Management Software for Conservation projects. https://www.miradi.org/

Qualitative data analysis

  • Dedooose, A cross-platform app for analyzing qualitative and mixed methods research with text, photos, audio, videos, spreadsheet data and more
  • Nvivo, powerful software for qualitative data analysis.
  • HyperRESEARCH “…gives you complete access and control, with keyword coding, mind-mapping tools, theory building and much more”.
  • Impact Mapper: “A new online software tool to track trends in stories and data related to social change”

Data mining / predictive modeling

  • RapidMiner Studio. Free and paid for versions. Data Access (Connect to any data source, any format, at any scale), Data Exploration (Quickly discover patterns or data quality issues). Data Blending (Create the optimal data set for predictive analysis), Data Cleansing (Expertly cleanse data for advanced algorithms), Modeling (Efficiently build and delivers better models faster), Validation (Confidently & accurately estimate model performance)
  • BigML. Free and paid for versions. Online service. “Machine learning made easy”
  • EvalC3: Tools for exploring and evaluating complex causal configurations, developed by Rick Davies (Editor of MandE NEWS). Free and available with Skype video support

Program Logic / Theory of Change modeling / Diagramming

  • Changeroo: “Changeroo assists organisations, programs and projects with a social mission to develop and manage high-quality Theories of Change”
  • Coggle:The clear way to share complex information
  • DAGitty: ” a browser-based environment for creating, editing, and analyzing causal models (also known as directed acyclic graphs or causal Bayesian networks)”
  • Decision Explorer: a  tool for managing “soft” issues – the qualitative information that surrounds complex or uncertain situations.
  • DCED’s Evidence Framework – more a way of using a website than software as such, but definitely an approach that is replicable by others.
  • DoView – Visual outcomes and results planning
  • Draw.io:
  • Dylomo: ” a free* web-based tool that you can use to build and present program logic models that you can interact with”
  • IdeaTree – Simultaneous Collaboration & Brainstorming Using Mind Maps
  • Insight Maker: “…express your thoughts using rich pictures and causal loop diagrams. … turn these diagrams into powerful simulation models.”
  • Kumu: a powerful data visualization platform that helps you organize complex information into interactive relationship maps.
  • Logframer 1.0 “a free project management application for projects based on the logical framework method”
  • LucidChart: Diagrams done right. Diagram and collaborate anytime on any device
  • Netway: a cyberinfrastructure designed to support collaboration on the development of program models and evaluation plans, provide connection to a virtual community of related programs, outcomes, measures and practitioners, and to provide quick access to resources on evaluation planning
  • Omnigraffle: for creating precise, beautiful graphics: website wireframes, electrical systems, family trees and maps of software classes
  • Theory maker: a free web app by Steve Powell for making any kind of causal diagram, i.e. a diagram which uses arrows to say what contributes to what.
  • TOCO – Theory of Change Online. A free version is available.
  • Visual Understanding Environment (VUE): open source ‘mind mapping’ freeware from Tufts Univ.
  • yEd – diagram editor that can be used to generate drawings of diagrams.  FREE. PS: There is now a web-based version of this excellent network drawing application

Dynamic models

  • CCTools: Map and steer complex systems, using Fuzzy Cognitive Maps and others [ This site is currently under reconstruction]
  • Loopy: A tool for thinking in systems
  • Mental Modeller: FCM modeling software that helps individuals and communities capture their knowledge in a standardized format that can be used for scenario analysis.
  • FCM Expert: Experimenting tools for Fuzzy Cognitive Maps
  • FCMapper: the first available FCM analysis tool based on MS Excel and FREE for non-commercial use.
  • FSDM: Fuzzy Systems Dynamics Model Implemented with a Graphical User Interface

Mind-Mapping software (tree diagrams)

  • MindView: “a professional mind mapping software that allows you to visually brainstorm, organize and present ideas.”
  • XMind: “mind mapping and brainstorming tool, designed to generate ideas, inspire creativity, brings you efficiency both in work and life.”
  • MindManager: “
  • Plectica: “Diagram your thinking in real time, together”

Collaboration software

  • Miro:  which can be used to make a collaborative ToC.

Excel-based tools

  • EvalC3: …tools for developing, exploring and evaluating predictive models of expected outcomes, developed by Rick Davies (Editor of MandE NEWS). Free and available with Skype video support

Uncategorised yet

  • OpenRefine: Formerly called Google Refine is a powerful tool for working with messy data: cleaning it; transforming it from one format into another; and extending it with web services and external data.
  • “Overview is an open-source tool originally designed to help journalists find stories in large numbers of documents, by automatically sorting them according to topic and providing a fast visualization and reading interface. It’s also used for qualitative research, social media conversation analysis, legal document review, digital humanities, and more. Overview does at least three things really well.
Other lists
Other other

Essentials of Utilization-Focused Evaluation

Michael Quinn Patton, August 2011. Sage publications

Publisher’s description:

“Based on Michael Quinn Patton’s best-selling Utilization-Focused Evaluation, this briefer book provides an overall framework and essential checklist steps for designing and conducting evaluations that actually get used. The new material and innovative graphics present the utilization-focused evaluation process as a complex adaptive system, incorporating current understandings about systems thinking and complexity concepts. The book integrates theory and practice, is based on both research and professional experience, and offers new case examples and cartoons with Patton’s signature humor. ”
Continue reading “Essentials of Utilization-Focused Evaluation”

Evaluating the Complex: Attribution, Contribution and Beyond.

Kim Forss, Mita Marra and Robert Schwartz, editors. Transaction Publishers, New Brunswick. May 2011. Available via Amazon

“Problem-solving by policy initiative has come to stay. Overarching policy intiatives are now standard modus operandi for governmental and non-governmental organisations. But complex policy initiatives are not only reserved for the big challenges of our times, but are used for matters such as school achievement, regional development, urban planning, public health and safety. As policy and the ensuing implementation tends to be more complex than simple project and programme management, the task of  evaluation has also become more complex.”

“The book begins with a theoretical and conceptual explanation of complexity and how that affects evaluation. The authors make the distinction between, on the hand, the common-sense understanding of complexity  as something that is generally messy, involves many actors and has unclear boundaries and overlapping roles; and on the hand, complexity as a specific term from systems sciences, which implies non-linear relationships between phenomena. It is particularly in the latter sense that an understanding of complexity has a bearing on evaluation design in respect of how evaluators approach the question of impact.”

“The book presents nine case studies that cover a wide variety of policy initiatives, in public health (smoking prevention), homelessness, child labour, regional development, international development cooperation, the HIV/AIDs pandemic, and international development cooperation. The use of case studies sheds light on the conceptual ideas at work in organisations addressing some of the world’s largest and most varied problems.”

“The evaluation processes described here commonly seek a balance between order and chaos. The interaction of four elements – simplicity, inventiveness, flexibility, and specificity – allows complex platterns to emerge. The case studies illustrate this framework and provide a number of examples of practical management of complexity in light of contingency theories of the evaluation process itself. These theories in turn match the complexity of the evaluated policies, strategies and programmes. The case studies do not pretend to illustrate perfect evaluation processes, the focus is on learning and on seeking patterns that have proved satisfactory and where the evaluation findings have been robust an trustworthy.”

“The contingency theory approach of the book underscores a point also made in the Foreword by Professor Elliot Stern: “In a world characterised by interdependence, emergent proerties, unpredictable change, and indeterminate outcomes, how could evaluation be immune?” The answer lies in the choice of methods as much as in the overall strategy and approach of  evaluation.”

GTZ/BMZ Evaluation and Systems Conference papers

(via Bob Williams on EvalSys)

Systemic Approaches in Evaluation

Documentation of the Conference on 25-26 January 2011

“Development programs promote complex reforms and change processes. Such processes are often characterized by insecurity and unpredictability, posing a big challenge to the evaluation of development projects. In order to understand which projects work, why and under which conditions, evaluations also need to embrace the interaction of various influencing factors and the multi-dimensionality of societal change. However, present evaluation approaches often premise predictability and linearity of event chains.

In order to fill this gap, systemic approaches in evaluation of development programs are increasingly being discussed. A key concept is interdependency instead of linear cause-effect-relations. Systemic approaches in evaluation focus on interrelations and the interaction between various stakeholders with different motivations, interests, perceptions and perspectives.

On January 25 and 26, 2011 the Evaluation and Audit Division of the Federal Ministry of Economic Cooperation and Development (BMZ) and the Evaluation Unit of GIZ offered a forum to discuss systemic approaches to evaluation at an international conference.
More than 200 participants from academia, consulting firms and NGOs discussed, amongst others, the following questions:

  • What are systemic approaches in evaluation?
  • For which kind of evaluations are systemic approaches (not) useful? Can they be used to enhance accountability, for example?
  • Are rigorous impact studies and systemic evaluations antipodes or can we combine elements of both approaches?
  • Which concrete methods and tools can be used in systemic evaluation?

On this website you will find the documentation of all sessions, speeches and discussion rounds. The main conclusions of the conference were summarized in the  final panel discussion.”

 

Designing Initiative Evaluation A Systems-oriented Framework for Evaluating Social Change Efforts

W. K. Kellogg Foundation, 2007. 48 pages. Available as pdf.

“Purpose”

“This document is designed for use by external evaluators who conduct initiative evaluations for theW.K. Kellogg Foundation (WKKF) – and, hopefully, other foundations and government agencies. It presents a systems-oriented framework and four general designs for initiative and cluster evaluation. The designs are based on systems concepts related to change and the dynamics of systems. The focus is not on considering all ideas about systems that could be applied to initiative evaluation, rather on how different dynamics within systems can serve as the basis for initiative evaluation designs.” Continue reading “Designing Initiative Evaluation A Systems-oriented Framework for Evaluating Social Change Efforts”

Beyond Logframe: Using Systems Concepts in Evaluation

March 2010. Nobuko Fujita (Ed)  Foundation for Advanced Studies on International Development (FASID) Available as pdf

“Editor’s Note: The 2010 Issues and Prospects of Evaluations for International Development employs systems concepts as clues to re-assess the conventional ways of conducting evaluations and to explore how development evaluation can potentially be made more useful.

In Japan, development evaluation predominantly relies on the Logical Framework (logframe) when conducting evaluations. Evaluations based on a logframe often face difficulties. One such difficulty arises from the futile attempt to develop an evaluation framework based on a logframe, which, in many cases, was prepared as part of the early-stage planning of the project and which then does not necessarily reflect a project’s real situation at the time of evaluation. Although a logframe can be utilised initially as a tentative project plan, logframes are rarely revised even when the situation has changed. By the end of the project, the original logframe may not be an accurate embodiment of what the project is about and therefore logframes do not particularly help in terminal or ex-post evaluations.

Still, having been institutionalized by clients, logframe-based evaluations are common practice and in extreme cases, evaluators face the danger of evaluating the logframe instead of the actual project. Although widely used for its simplicity, logframes can end up becoming a cumbersome tool, or even a hindrance to evaluation.

Various attempts have been made to overcome the limitations of the logframe and some aid organizations such as USAID, UNDP, CIDA and the World Bank have shifted from the logframe to Results-Based Management (RBM). Now GTZ  is in the process of shifting to a new project management approach designed on RBM and systems ideas.

In the first article, “Beyond logframe: Critique, Variations and Alternatives,” Richard Hummelbrunner, an evaluator/consultant from Austria, sums up the critique of logframe and the Logical Framework Approach (LFA), and explores some variations employed to overcome specific shortcomings of LFA. He then outlines a systemic alternative to logframe  and introduces the new GTZ management model for sustainable development called “Capacity WORKS.” Richard has dealt with LFA and possible alternatives to LFA at various points along his career, and he is currently involved in GTZ’s rollout of Capacity WORKS as it becomes the standard management model for all BMZ 5 projects and programmes.

What does he mean by “systemic alternative”? In the second article, “Systems Thinking and Capacity Development in the International Arena,” Bob Williams, a consultant and an expert in systems concepts, explains what “thinking systemically” is about and how it might help evaluation. He boils down systems ideas into three core concepts (inter-relationships, perspectives, and boundaries), and relates these concepts to various systems methods.

In December 2009, FASID offered a training course and a seminar on this topic in Tokyo. Through the exchange of numerous e-mails with the instructors prior to the seminar, it occurred to me that the concepts might be more easily understood presented as a conversation. That is what we tried to do in the third article, “Using Systems Concepts in Evaluation – A Dialogue with Patricia Rogers and Bob Williams –.” These two instructors of the FASID training course and workshop explain in simple conversational style where and how we can start applying systems concepts in development evaluation.

This issue also carries a report of two collaborative evaluations of Japanese Official Development Assistance (ODA) projects. The first case presents an innovative joint evaluation conducted collaboratively with Vietnamese stakeholders. The evaluation took place in 2009 – 2010 as the last year of a three-year evaluation capacity development project coordinated by the Japan International Cooperation Agency. The second case covers a joint evaluation study of another Japanese ODA project in Lao PDR with a local Lao administration for which neither logframe nor OECD DAC five criteria was used. Instead, an evaluation framework was developed from scratch, based entirely on the beneficiaries’ interests and perspectives. In both cases, a partner country’s participation in the evaluation necessitated considerable changes in perspectives of evaluation practice. I hope they provide examples of how boundaries and perspectives, as discussed theoretically in the first three articles, relate to development evaluation in practice.”

SYSTEMS CONCEPTS IN EVALUATION: AN EXPERT ANTHOLOGY

Bob Williams and Iraj Imam (eds.)
EdgePress/American Evaluation Association (2007)

Systems Concepts in Evaluation: An Expert Anthology brings
together a wide range of systems concepts, methodologies and
methods and applies them to evaluation settings. This book
addresses the questions:

• What is a systems approach?
• What makes it different from other approaches?
• Why is it relevant to evaluation?

The 14 chapters cover a wide range of systems concepts and methods. Most chapters are case study
based and describe the use of systems concepts in real life evaluations. The approaches and methods
covered include:

• System Dynamics (both quantitative and qualitative)
• Cybernetics and the Viable System Model
• Soft Systems Methodology
• Critical Systems Thinking
• Complex Adaptive Systems

There are also overview chapters that explore the history and diversity of systems approaches and their
potential within the evaluation field. There is a substantial introduction by Gerald Midgley to the key
developments in systems concepts and methods over the past 50 years, and this explores the
implications for evaluation of each of those developments.

Although focused on evaluation, the book is a valuable source for anyone interested in systems concepts,
action research and reflective inquiry. It is useful for both teaching and practice.

Chapters :
Introduction, Iraj Imam, Amy LaGoy, Bob Williams and authors
Systems Thinking for Evaluation, Gerald Midgley
A Systemic Evaluation of an Agricultural Development: A Focus on the Worldview Challenge,
Richard Bawden
System Dynamics-based Computer Simulations and Evaluation, Daniel D Burke
A Cybernetic Evaluation of Organizational Information Systems, Dale Fitch, Ph.D.
Soft Systems in a Hardening World: Evaluating Urban Regeneration, Kate Attenborough
Using Dialectic Soft Systems Methodology as an Ongoing Self-evaluation Process for a
Singapore Railway Service Provider, Dr Boon Hou Tay & Mr Bobby, Kee Pong Lim
Evaluation Based on Critical Systems Heuristics, Martin Reynolds
Human Systems Dynamics: Complexity-based Approach to a Complex Evaluation, Glenda H
Eoyang, Ph.D.
Evaluating Farm and Food Systems in the US, Kenneth A Meter
Systemic Evaluation in the Field of Regional Development, Richard Hummelbrunner
Evaluation in Complex Governance Arenas: the Potential of Large System Action Research,
Danny Burns
Evolutionary and Behavioral Characteristics of Systems, Jay Forrest
Concluding Comments, Iraj Imam, Amy LaGoy, Bob Williams and authors

PUBLICATION AND PURCHASE DETAILS

NAME : Systems Concepts in Evaluation : An Expert Reader
EDITORS : Bob Williams and Iraj Imam
PAGES : 222pp

ISBN 978-0-918528-22-3 paperback
ISBN 978-0-918528-21-6 hardbound

PUBLISHER :

EdgePress/American Evaluation Association (2007)

PURCHASE

Available via Amazon : Hardback only. $US36 plus postage

%d bloggers like this: