Card sorting methods: A List

Card / pile sorting is a simple and useful means of eliciting and aggregating qualitative data, in a participatory manner. In anthropology, it is described as pile sorting, and is used for domain analysis, in the field of cognitive anthropology. In website design it is known as card sorting.

Anthropology
Website design
Software
  • OptimalSort: Online card sorting software (free and paid for use):
  • SynCapsV2:  For the analysis of the results of physical card sorts, which can be downloaded and used on a desktop/laptop
  • XSort: is a free card sorting application for Mac, aimed at user experience professionals and social scientists.
  • KardSort : Perform Web-Based Cardsort Study for free.
  • Miro Card Sorting template

 

Open Aid: Public Online Monitoring for Better Aid

A new website, launched in December 2009

What is Public Online Monitoring

Public Online Monitoring is an approach to bring development projects into the public realm. Public Online Monitoring improves access to project information and to facilitate communication among project stakeholders using an online platform.

OpenAid is a German NGO based in Hannover created in order to make Public Online Monitoring reality.

Public Online Monitoring has four key elements:

1) Detailed information about individual development projects, online, easy to find and easy to understand. This element is the realm of the International Aid Transparency Initiative (IATI), of the AidDATA project and of donors. OpenAid lobbies for more transparent aid information in German aid agencies.

2) Sector specific online guidance for civil society stakeholders on issues critical to monitoring (e.g. common sector specific forms of corruption).This guidance should support non-experts to ask critical questions and encourage stakeholders to monitor projects in their home area.

3) An online forum for stakeholders to meet, to exchange information, to voice concerns, to defend interests, to discuss policy. Such a forum can should involve local project management, donors, local government and politicians, local media representatives, local NGOs and citizens. It can also involve interested citizens from other countries, development experts and taxpayers in donor countries.

4) Linking up the online forum to people without access to the internet through an involvement of local radio stations and newspapers, creative use of mobile phones and through local NGO activities.

THE PITFALLS OF MONITORING & EVALUATION OF WOMEN’S RIGHTS, WOMEN’S EMPOWERMENT, GENDER EQUALITY: DO CURRENT FRAMEWORKS REALLY SERVE US?

(From The Association for Women’s Rights in Development (AWID))

A summary of Part I “Capturing change in women’s realities: The challenges of monitoring and evaluating our work” a paper by Srilatha Batliwala* and Alexandra Pittman.** [Link yet to be found] PS:  Authors are currently inviting feedback and will publish a final version of this paper, together with additional research later this year.

By Kathambi Kinoti and Sanushka Mudaliar – AWID

Monitoring and evaluation now form an integral part of women’s rights and gender equality programmes as we attempt to measure how effectively we work. But are the frameworks we use able to perform this ambitious task?

In their paper Capturing change in women’s realities: The challenges of monitoring and evaluating our work Srilatha Batliwala and Alexandra Pittman assess the “ifs,” the “whys” and the “hows” of monitoring and evaluation (M&E) in women’s rights, women’s empowerment and gender equality. They observe that “over the past few decades, important strides have been made in developing ways of capturing a whole range of abstract but vital social realities, and particularly in trying to quantify them. These efforts have been the result of the realization that when policies, resources, and strategies are applied towards building more equitable, sustainable, rights-affirming, inclusive and peaceful societies, we have to devise ways of checking whether they are working effectively or not – whether they are producing the changes we wish to see.”
Continue reading “THE PITFALLS OF MONITORING & EVALUATION OF WOMEN’S RIGHTS, WOMEN’S EMPOWERMENT, GENDER EQUALITY: DO CURRENT FRAMEWORKS REALLY SERVE US?”

Training: “Impact Oriented Project Planning and Monitoring”

Date: 8th to 10th of March, 2010
Venue: Karl Kübel Institute for Development Education – India

Dear Friends,

We would like to inform you about the above mentioned workshop which we offer at the Karl Kübel Institute for Development Education, Coimbatore (www.kkid.org).

Target group:

Participants from organisations dealing with projects and programmes of development cooperation in all fields being responsible or involved in activities of project cycle management. Knowledge of formalized planning procedures like logframe-planning and the use of project cycle management are appreciated.

Objectives:

“Research Methods for Guiding Policy and Evaluation with Special Application to Population and Health”

Date: May 2 – July 22, 2010
Venue: Social Research Center of the American University in Cairo.

The Social Research Center of the American University in Cairo is pleased to announce the 9th round of the training course “Research Methods for Guiding Policy and Evaluation with Special Application to Population and Health”. The three-month training course is directed to Arab nationals and will be held during May 2 – July 22, 2010 at the Social Research Center of the American University in Cairo.

The training course aims to acquaint participants with principles of research methods and its application in the field of population and health.
Continue reading ““Research Methods for Guiding Policy and Evaluation with Special Application to Population and Health””

Against Transparency: The perils of openness in government.

Being a keen advocate of greater transparency by aid agencies and programmes this article interested me.

See it on the New Replublic website, published on October 9, 2009

The author, Lawrence Lessig, “is professor of law and director of the Edmond J. Safra Center for Ethics at Harvard Law School, and the author most recently of Remix: Making Art and Commerce Thrive in the Hybrid Economy (Penguin). He is on the advisory board of the Sunlight Foundation and on the board of Maplight.org“

Training: MOST SIGNIFICANT CHANGE TECHNIQUE (Indonesia)

Date: 25-26th November 2009
Venue: Bogor, Indonesia

The Most Significant Change technique is a dynamic qualitative and participatory monitoring and evaluation method.  Through collecting stories of the impact of our program with the program stakeholders we analyse our work.  Through this process of selecting the stories, change management and organizational learning is facilitated. MSC has a particular ability to capture impact in complex cross cultural social change contexts.

With the shift in monitoring and evaluation to a mixed methods approach MSC is a method that fills the gap in data collection. MSC gathers rich qualitative data that captures intangible impact and unexpected outcomes of the program.

This two day training presents MSC through practical examples and exercises.  The training is designed for active learning.
At the end of the training the participants will:
•  Understand how MSC works
•  Understand how MSC fits into a Monitoring and Evaluation Framework,
•  Know the best situation to use MSC and when not to use MSC
•  Understand how MSC facilitates development of stakeholder relationships and organizational learning & development
•  Have learnt through practical exercises, skills to undertake MSC (story collection, story selection & feedback)
•  Know the specific adaptation of MSC for an Indonesian context
•  Have designed an MSC system for your program.

Who Should Attend These Trainings?
Organisations, teams and individuals seeking to:
•  Develop a program’s direction or clarify a shared vision
•  Try something new in an organisational learning context
•  Capture intangible and unexpected outcomes

Download flyers in English and Indonesian Details of the trainers, the location, the costs and contact names and email addresses are all in the flyer

Most Significant Change Training in Melbourne 1-2 December

Date: 1-2 December
Venue: Melbourne

MSC is a powerful tool for monitoring, evaluation and organisational learning. MSC goes beyond merely capturing and documenting participants’ stories of impact, to offering a means of engaging in effective dialogue. Each story represents the storyteller’s interpretation of impact, which is then reviewed and discussed. The process offers an opportunity for a diverse range of stakeholders to enter into a dialogue about program intention, impact and ultimately future direction. This two day workshop provides an introduction to MSC which includes designing your own MSC process. Participants will be provided with experiential learning opportunities and examples of real applications of the technique throughout the workshop.

This two day course includes the Evaluation Summit training. Evaluation Summit will take place on the second half of day two.

You can register online here

As well as our regular courses we are also seeking expressions of interest for two brand new courses in management: Management Core Concepts and Change Management Toolkit facilitated by Carolynne Wilson our newly appointed senior consultant!

[US] OMB Releases Plan to Elevate Performance Evaluation

Posted on October 14, 2009

The [US]  Office of Management and Budget (OMB) released a memo to federal agencies on Oct. 7 that outlines a new initiative to bring a renewed emphasis and additional resources for program evaluation within agencies. Although this initiative is not a comprehensive plan to reinvigorate performance measurement in the federal government, it will help correct many problems that kept previous performance systems from creating real improvement in government performance.

The memo is entitled “Increased Emphasis on Program Evaluations” and details a three-part plan to help agencies develop better systems to conduct “rigorous, independent program evaluations.” The plan includes giving better access to agency program evaluations on the Internet that are both in progress and planned for the future, re-launching an interagency working group on evaluations, and a voluntary pilot program to provide additional resources to fund rigorous program evaluations or strengthen evaluation capacity within agencies. Each of these three policy changes will help to improve performance evaluations within agencies and encourage better use of performance information.

Article continues here

The UK Evaluation Society Conference 2009

Date: 24-25 November 2009
Venue: The Holiday Inn London Bloomsbury

Evaluating Impact in the Current Crisis

The theme of this year’s annual conference takes us back to basics and relates to the perpetual issue taxing all evaluators: how to evaluate impact. The current financial crisis brings this into even sharper focus with many evaluators asking themselves why evaluation does not seem to be answering the key strategic questions facing us today. This environment also brings further uncertainty to evaluators in terms of the complex nature of the economic environment, the stress on evaluation funding and a renewed challenge to demonstrate value for money and the benefits of evaluation.

The conference hopes to look at methods and issues around evaluating impact in key spheres: economic, social, health, cultural and environmental. It will provide evaluators with an opportunity to develop their understanding of methodologies and to air their views and work through issues around the way in which we look at impact.

The conference will address the key issues facing evaluators today and will provide a forum for all evaluators from across government, academia and the voluntary and private sectors. The aim of the conference is the same as that of UKES: to advance and improve the theory and practice of evaluation.

The keynote speakers at the conference are:

  • Will Hutton , Executive Vice-Chair, The Work Foundation
  • Anna Coote, Head of Public Sector Policy, New Economics Foundation and Health Commissioner, Sustainable Development Commission
  • Sandra Nutley, Professor of Public Management, University of Edinburgh
  • Howard White, Head of New Impact Evaluation Entity, 3IE
  • Beatriz Garcia, Director, Impact 08
  • Peter Tyler, Professor of Land Economy, Cambridge University

Conference website