Hierarchical Card Sorting (HCS)
A simple tool for qualitative research
This is a tool I developed in 1993, while doing PhD field work with the Christian Development Commission of Bangladesh (CCDB), in Dhaka, Bangladesh. I subsequently wrote two how-to-do-it papers describing the method (both of which have been on the web for some time):
- Hierarchical Card Sorting: A Tool for Qualitative Research (1996)
- Tree Maps: A Tool for Structuring, Exploring and Summarising Qualitative Information (1998)
This page will now try to integrate the contents of those two papers. An initial clarification may help. Hierarchical Card Sorting refers to the process. Treemaps refer to the product of that process. It is also useful to note that HCS is related to the Most Significant Change technique, but different. Central to the HCS is a question about the “most significant [static] difference”, whereas MSC asks about the “most significant change”. Both ask respondents to make observations and interpretations.
Late note: HSC is a sibling of MSC, because it asks about the “most significant difference” versus the most significant change. The focus is on static/enduring differences, rather than changes (differences over time).
What is Hierarchical Card Sorting (HCS)?
HCS is one type of card sorting method. Card sorting has been used in many contexts, from traditional ethnography to the modern day business of designing usable websites (Ref 1, Ref 2, Ref 3). In these contexts card sorting is typically used to elicit people’s mental models: the categories they use, what belongs to these categories, and how the categories relate to each other.
What use is it?
In many organisations people accumulate a lot of knowledge, but often it is tacit and informal in nature. It is not easily shared. Yet sharing that knowledge can make a difference, other people can make use of it, and they can help correct it.
How do you do it?
The HCS method asks people about significant differences. About differences which are important to them and which have (or had) consequences. It has some similarities in origin and approach with the Most Significant Change technique. The design of both tools was influenced by Gregory Bateson, especially his book “Mind and Nature: A Necessary Unity” (1979), in which he argues that information is “a difference that makes a difference”. In turn, many people would argue that knowledge is structured information. The HCS is about eliciting and representing people’s knowledge (i.e a structured set of differences that make a difference).
Normally the HCS is used with one respondent. However, the process outlined below could easily be used with a small group. The steps:
- Identify the respondent’s area of expertise or knowledge that you want to explore. For example, knowledge of animal diseases held by a paravet, or knowledge of local NGOs held by a INGO staff member working on NGO capacity building.
- Generate a list of actual cases which will be sorted. If possible, try to select cases that represent a wide variety of examples. In many cases you will want to select cases that the respondent is responsible for in some way, and thus should be expected to know about them. For example, a list of cases treated by the paravet in the last month, or a list of NGO grantees managed by an INGO desk officer. Don’t be too ambitious, especially to start with. Large numbers of cases (25+) will make the process more time consuming and will run the risk of boring the respondent and interviewer. Write the name of each case down on a separate card. Cases might be events (treatment provided) or entities (clients).
- Place all the cards in one pile (see this as the trunk of a tree) and ask the respondent to tell you about some of the differences between all these cases. The purpose of this question is simply to generate a shared awareness of the large number of differences that (inevitably) exist. It is a warm up exercise.
- Ask the respondent to sort all the cards into two piles of any size (see these as the first two branches above the trunk), according to what they think is the most important difference between all the cases represented on the cards. Emphasise that it is their opinion of “importance” which is important. If you want to direct their attention in a particular direction then use a prefix to the question, such as “In your roles as…what do you think is the most significant difference between…?. Or “Considering the objectives of this organisation…what do you think is the most significant difference between…?
- Emphasise that a distinction is important if it makes a difference. Because respondents may casually offer a difference simply to oblige the interviewer it is important to check its significance by asking “What difference does this difference make ?” If one can’t be identified then suggest to the respondent that they consider if there are other differences which might be more important.
- Keep a record of which cards are in which pile. This is easier if all the cards have all been numbered beforehand. And then write down a description of the reported difference between the two piles. And what difference that difference makes.
- Take one of the two piles at at time and repeat stages 4 to 6 above. Then repeat this process with second pile. There should now be four piles. Repeat the same process with these piles until there is only one card left in each pile. One way of keeping track of this process is to use a large piece of paper, to draw a tree whose branches spilt into smaller and smaller branches.
- In some cases there may be more than one example left in a pile but the respondent may not be able to identify an important difference between them. Don’t force them to do so, but simply note that no further difference could be identified. It may be useful to do steps 4 to 6 steps with the groups of cards that the respondent feels they know most about first, and the others later on.
Treemaps can be shown in table or diagram form. Here is a treemap in table form (using Excel), produced in 1993. Click on the image to read the text, and click again to enlarge. Read the results from left to right. This example is incomplete because it shows differences, but not the differences they make.
The same results can be shown as a tree structure (shown below without the text notes). Red lines = preferred types of partnerships in future (discussed below).
Problems and limitations
1. Some caution may be appropriate. Like all participatory methods it requires some trust and confidence in the relationship between yourself and the person whose views you are seeking. Secondly, there is no guarantee that the views that are expressed will be stable over time. Peoples views of the world change, and the expression of their views is often very context dependent.
2. Some people react at some stage to the exercise by saying “There is no difference between these“. Here I have cautiously tried to give many examples of possible differences, while being careful not to lead in any particular direction. I have emphasised that differences can be found even between objects that look identical, the question is which of these is most important from their point of view. I also emphasise that we are looking for relative rather than absolute significance.
3. Another problem is almost the opposite in nature. People can approach the task in what appears to be an un-engaged manner, blithely tossing off distinctions which don’t seem too significant. When this happens I have tried asking “In what way is that significant, what difference does that make ?, checking to see that the respondent can articulate the significance, and if not checking to see if they really understand the exercise.
4. Another problem relates to respondents who are almost too helpful. As can be seen from the tables below it is common for some respondents to report more than one difference. When well organised I have dutifully noted these down and then asked the respondent, after reading them back, “and which of these……are the most significant ?” Failing to do this has meant I have been the one that ends up speculating on their relative importance to the respondent.
5. Many respondents find it easier to identify differences, than it is to identify the differences these differences will make. Often both respondent and interviewer assume that this is self-evident and fail to document this part of the exercise. Yet these beliefs can contain important assumptions or hypotheses about the way things work, which can benefit by being scrutinised and tested.
Making use of HCS results
Value can be obtained from treemaps at two stages:(a) during the creation of the treemaps, and (b) through comparisons made between parts of the structure once it has been created.
1. During creation of a treemaps the main use is as a enthographic tool: understanding people’s view of the world. There are two forms of usage:
- Identifying the distinctions that people see as important. This is evident in the contents of the differences cited. It is also evident in how early in the exercise they are cited, and how often they are repeated (on different branches). The use of binary classifications (only two sub-categories can be created at any one time) forces the respondent to prioritise, to choose between a number of potentially important distinctions.
- Identifying the limits to people’s knowledge: When respondents cannot identify differences between two or more entities the limits to their knowledge seem to have been reached. Not knowing what people know about can be important, especially when they might be expected to, or claim to have, expertise in that area.
2. After creation there are two other types of usage. One is based on proceeding from trunk to leaf, the other is based on proceeding from leaf to trunk.
- Planning based discussions: Starting from the trunk, respondents can be asked questions such as “How will your work in the next six months with this group be different, compared to this group (the two groups labelled at the first junction)?” Then move to the next junction on each branch, one a time, and re-iterate the same question. Lack of a difference suggests lack of a strategy, and points to areas where it may need to be more articulated. Other related planning questions which can be asked, from trunk to branch, are “Which of these will be the most immediate priority?”, “Which of these will you be spending the most time with ?”, “Which of these will present the most problems?”, Which of these do you want to scale up / expand / fund more of?”, etc. The answer, and associated rationale, can be added as a further annotation to the tree map, at the appropriate branching point.
- Evaluation based discussions: These proceed in the opposite direction, from leaf to trunk. Starting from two adjacent “leaves” respondents can be asked questions such as ” Which of these two groups was most successful in the last six months?”. Then move to the next two most adjacent leaves and re-iterate the same question. Then compare the two leaves which were identified as the most successful and which share the same larger branch. Reiterate the process again with the other leaves. In each case “move” the most successful case down the tree to the next junction, to meet the most successful case moved down from the next most adjacent branch. As above, the choice, and associated rationale, can be added as an extra annotation to the tree map, at the appropriate junction.
Note: Overall “success” can be defined in two ways. At any one junction it will be in terms of which of the two types were most successful (as represented by the descriptions of the two groups of cases). When viewed in terms of particular cases success could be seen in terms similar to a tennis tournament. The number of grades of success would equal the number of junctions between a leaf and the trunk. In the diagram above there are up to three five junctions linking a leaf with the trunk. Comparisons would need to take into account the fact that some branches may be longer than others.
1. Assessing the impact of capacity building activities: When supporting capacity building work with individuals or whole organisations, we might expect that this assistance, either in the short or long term, would make a difference to the person or organisations relationships with their clients. One attribute of that relationship is responsiveness. The service provider might be more sensitive to the differences between client’s needs. They may also be more up to date in their knowledge about their various clients’ needs. The important differences they see between their clients (that they think are important) may be more reflective of their clients concerns, and not just their own. Much of this information is available, in the first instance, in the form of knowledge the service provider has (or does not have) about its clients. This knowledge is in effect a proxy indicator. Treemaps are a way of mapping a service provider’s knowledge of their clients. How differentiated is their map of the differences between clients, how up to date is their knowledge of each “leaf” and branch? How much do the descriptions of difference reflect client versus provider concerns? This information can be verified by independent observation and follow-up contacts with clients. Are services visibly differentiated (rather than homogenous)? How frequently have they been modified?
2. It should be possible to use treemaps as a means of doing a stakeholder analysis in a development project. This could initially be from the perspective from one observer, possibly an individual stakeholder. Firstly, a list of cases reflecting the maximum possible variety of stakeholders would be identified. The process would then start at the trunk, with the respondent being asked to identify “the most important distinction between all the stakeholders in the project”. Then each of the two initial categories of stakeholder could be progressively differentiated until all cases were located as a leaf of their own.
4. Scenario planning: This is a practice widely associated with Shell. It involves developing a number of alternate views of the future and then identifying how the organisation concerned would react differently to each different scenario. Treemaps of different scenarios could be developed by starting off with a question, such as: “What is the most important difference in views within this organisation/group about what is likely to happen to the oil market over the next 10 years ?”. Within each new branch created by the answer, the question could be repeated, but pre-phrased in terms of “Within this view of the future…”. Once the treemap is constructed a second set of planning oriented questions could then be pursued, asking respondents how they will respond differently to each branch in the “scenario tree”. As above, both individual and group based versions of this exercise could be designed.
5. Qualitative monitoring: An NGO may be using the Most Significant Change technique to identify and document stories of change. One common challenge is how to organise a summary-by-selection process when there are a large number of stories involved. The treemap structure provides a way of dealing with the problem. The NGO may be working with many groups or villages. The NGO’s categorisation of these groups or villages can be documented using the treemap method. This structure then provides a framework for comparing MSC stories, two at a time. Pair comparisons are the easiest way of comparing complex events
Two accounts of the use the HCS
In 1994 staff of the Research Unit of the Christian Commission for Development in Bangladesh (CCDB) used the cardless version of the same method in the process of exploring poor peoples conceptions of health, disease and medicine. The following quote describes the results of two applications of the method:
“In two somities [beneficiary groups] we also asked the members to come up with different medicaments they use when they are ill. We asked them to divide these medicaments into two groups. We asked for an explanation after each division. After dividing the group into two other groups we asked them to do the same for the two other sub-groups. In the first somity they divided the medicaments into allopathic and herbal medicines. The allopathic medicines were divided into tablets and syrup. The herbal medicine into medicines used for cough and influenza, and medicines for influenza.
In the second somity they divided the medicaments into medicines given by the kobiraj and medicines given by the allopathic doctor. The medicines given by the kobiraj were divided into medicines for weakness and medicines for influenza and headache. The medicines of the allopathic doctor were divided into medicines for pain and medicines for gastric burns.
We used this exercise to get more information about people’s concepts of medicines. When we asked about [the consequences of] the difference between herbal and allopathic medicines it was mentioned that allopathic medicines cure better and sooner but they are costly and can give weakness. Herbs are “softer” to the body, they don’t harm the body, they are cheap , and although they work slowly they keep the body healthy.” (People’s Health and Nutrition, January 1995, CCDB)
“I thought I would let you know that I have tried your treemap method for the first time. Despite thinking it sounded rather unlikely to be useful when I first read it, it has provided a very nice alternative to classic wealth ranking for investigating the structure of villages. The people I am working with had the same unease as I do about launching into wealth analysis. We therefore tried your method as it seemed to offer a way round the problem. We ask the number of families in the village, then ask about differences between them with respect to livestock and well-being. Following the process as you describe, we rapidly get a good idea of the wealth structure of villages, which is far more disparate than I had imagined. From this exercise it is possible to find people to interview in more detail from each category. Repeating the exercise with paravets, we then asked them to indicate what proportion of their work was with which group. This worked well and provided an indication of who was benefiting most from the services of the programme.”
Afterword (July 2008)
It is possible to analyse the results of card sorting exercises using social network analysis software. When two cards are placed in the same pile they can be seen as “linked” by their membership of the same category. The more often two cards are found in the same piles, created during card sorting exercises by different respondents, the stronger that links is. These relationships can be documented in a simple matrix, where all the card names are listed down the left column and again across the top row. Cell values tell us how many times that row card was placed in the same group as that column card. This data can be imported into social network analysis software, such as UCINET, and then converted into a network diagram, using NetDraw. Network diagrams can show two structural features of interest: (a) clusters of cards most commonly associated with each other, (b) linkages between clusters of cards.
Afterword (October 2009)
Courtesy of Joachim Harloff I have come across “Sorting Data: Collection and Analysis” by A.P.M. Coxon, published by Sage in 1999. In this useful book Coxon refers to “divisive hierarchical card sorting” and the fact that it was first documented by James S Boster in his 1986 journal article ” Can Individuals Recapitulate the Evolutionary Development of Color Lexicons? Ethnology 25(1):61-74. The question Boster asked respondents was different to the HCS question described above, it focused on similarities rather than differences: “I would like you to do is sort these colors into two groups on the basis of which colors you think are most similar to each other” (page 64).
Afterword (June 2012)
Treemaps can be seen as Decision Trees, a representation device I have discussed elsewhere. They can be seen as both classifications of existing cases but also as predictions of outcomes associated with new cases. Unlike some Decision Trees, which classify cases into binary categories according to their type of outcomes, Treemaps from a HCS process can be used to classify cases in to more ordinal categories (more than/less than) by asking respondents to ask questions like “Which of these two categories of projects would you like to fund more/less in the future?” Starting from the trunk of the tree and moving up each branch, this type of question, when re-iterated at each branch, can produce a complete ranking of all categories of cases (i.e. each of the leaves on the tree). Like other Decision Tree models, this predicted ranking is testable, it can be compared to results observed by others e.g. independent evaluations
Afterword (October 2013)
Some extra resources on card/pile sorting:
- Susan Lowes on Pile Sorts
- (Online) Card Sorting Tools by Tom Tullis, a list of providers and a discussion