Hierarchical Card Sorting: A Tool for Qualitative Research

by  Rick Davies, Social Development Consultant and Research Fellow, Centre for Development Studies, Swansea, UK. 1996 

E-mail: rick.davies@gmail.com

PS: Please see the new (2008) page on Hierarchical Card Sorting at http://mande.co.uk/special-issues/hierarchical-card-sorting-hcs/

This paper describes how to use a device I have called, for want of a simpler term, hierarchical card sorting or HCS. It was developed in the course of research into the ways in which the staff of non-government organisations (NGOs) see the world around them, including their intended beneficiaries and other NGOs. It has been used with NGO staff in Bangladesh, Kenya, Malawi. Nigeria and the United Kingdom.

Some caution may be appropriate. Like all participatory methods it requires some trust and confidence in the relationship between yourself and the person whose views you are seeking. Secondly, there is no guarantee that the views that are expressed will be stable over time. Peoples views of the world change, and the expression of their views is often very context dependent.

I have outlined below the method as I have used it. Three examples are given of the results produced by a HCS, along with some commentary on these. These are followed by an examination of some common problems with the HCS method and how I have dealt with them, as well as an exploration of some possible adaptations of the method. Finally, I have spelled out some of what I think are the key differences between the HCS, the PRA technique known as Matrix Ranking (IIED, 1992) and another related method used by psychologists, known as Kelly's Repertory Grid(1) (Kelly, 1955)(2).

The Method

The starting point was a set of objects that were of concern to myself, and which were known to the person I was doing the exercise with. In two cases these were the names of the other NGOs the respondents(3) NGO was funding. The names of these were put on cards, one NGO per card. For my own recording purposes I scribbled a consecutive number on the back of each card. The cards were then spread out in front of the respondent in a haphazard way, and shuffled around to make sure there was no apparent order. Then two cards were picked out at random and the respondent was asked "What are the differences between these two NGOs" The aim at this stage was to establish that there were all sorts of differences, and if necessary more than one random comparison of cards was used to emphasise the point.

The whole set of cards was then mixed up again, but in such a way that all of them were visible. I then asked the respondent "From your point of view, what is the most important difference between all of these NGOs ?" After he told me, that it was between those that were x and those that were y, I asked him to sort all the NGO cards into two piles representing these differences. I quickly noted down the distinction reported, and the numbers on the back of the cards in each pile and then pushed one pile aside temporarily and focused on the other. "Looking at this group of NGOs, what do you think is the most important difference between them ?, I asked. After being told, and noting down the answer, I asked the respondent to sort the cards into two piles, based on this distinction. When this was done I again scribbled down the numbers on the back of the cards in each of the two groups, and the distinction between them, and again focused on one of the two groups. As can be imagined, I proceeded until the group of NGO cards I was working on only had two cards left, and I had identified the important difference between these. I then went back to each group I had pushed aside and worked through that group again until only two cards were left. At the end all cards were standing on their own, having been differentiated from others.

Using my notes on the distinctions, and the numbers of the cards belonging to each group created by each distinction, I was able to then create a classification which had a tree structure. The root category was "All the NGOs in the group". This then branched into two, which in turn developed sub-branches, and further sub-sub-branches, and so on. This structure is shown in the tables below, and was created using the Table function in Word Perfect(4). The distinctions provide the names of each branch, and at the end of each branch is the name of the NGO, the object that was categorised by that distinction(5). The results was a nested category structure in which an object such as an NGO may belong to up to 5 or 6 different categories at the same time. The result was a much richer NGO categorisation than normally found in published articles about NGOs. Since they were produced by controllers of resources used by these NGOs, it seemed these categorisations were also likely to be more immediately relevant.

As can be seen from Table 1 below, there are some distinctions which are used more than once. With some caution (explained below), their prevalence can be viewed as an indication of their importance. The researcher can summarise the contents of the table by counting the number of distinctions that fall into categories that he/she thinks are of interest. For example, in the case of a HCS with one NGO CEO, which involved looking at differences between the various programmes operated by that NGO, I counted the number of inward looking or outward looking distinctions (e.g those focusing on staff versus beneficiaries of those programmes) made by the CEO, and found that more than 80% were inward oriented.

The Examples

Table 1 presents the distinctions made by the Country Director for Community Aid Abroad (Australia) between the different NGOs that CAA funds in Bangladesh. He is a Bangladeshi national and was resident in Dhaka. In almost all cases he had been responsible for identifying the NGO that his office now funds.

Table 2 presents the distinctions made by the Christian Aid desk officer for Burkina Faso. He was resident in London, and was responsible for some but not all of the initial decisions to fund the NGOs listed.

Table 3 presents the distinctions made by the Welfare Officer responsible for maintaining a register of more than 60 associations(6) in Oju Local Government Area, Benue State, Nigeria. This exercise was done without the use of cards at all, just using a sequence of questions based on that described above. Because of the number of associations involved it is all inclusive but not fully differentiated. The Oju Local Government Authority is not a major donor to these associations but can be influential in effecting their access to government funds.

Commentary on the Examples

I can offer a set of distinctions between these sets of distinctions (a meta-analysis), from my own third party perspective outside these organisations. These are of course influenced by my own contextual knowledge and the dialogue that took place during the exercise. As already indicated, the first two structures differentiate NGOs from the point of view of a donor. In the case of CAA there is a strong focus on the origins of the NGOs, and their subsequent growth and scale. In the case of Christian Aid, there seems to be a stronger emphasis on the current status of the NGOs and more specifically the degree to which these organisations are oriented primarily towards the poor versus other organisations.

In the case of the Oju structure the initial structure is unrelated to funding, but the over- arching distinctions noted underneath bottom of the table were related. When asked why registration was important the Welfare Officer pointed out that it enabled the organisations to avoid prosecutions for holding illegal meetings and it enabled the associations to gain access to loans from government, or coming through government. Discussions had been in process over the past twelve month about the Oju area being included in an impending IFAD project.

More generally, there is extensive use of distinctions in these tables based on age and size of organisations, which are often closely related. Such distinctions are not common in the categorisations of NGO presented in the literature on NGOs (e.g Korten, Carol, Edwards and Hume), which tend to represent static rather than dynamic perspectives. But they can be very relevant both in terms of problems and possibilities. Growth can mean alienation from the organisations original vision or from the views and needs of its base membership. It can also mean more accumulated expertise, capacity for wider impact, capacity to use funds on a larger scale, and greater claims to legitimacy(7).

Distinctions focusing on gender, or which are explicitly gendered, were given a different prominence in each table. In Table 3 they are non-existent. In Table 2 they are present underneath in the form of a brief comment on the significance of non-registration. In Table 1 they are present on a modest scale but inquiries showed a clear preference for funding organisations differentiated by gender in the future. This difference is relevant to the caution above about counting the prevalence of various types of distinctions. Structure of distinctions can be seen as a reflection of the past, but they are not necessarily predictive of the future.

Problems

1. Some people react at some stage to the exercise by saying "There is no difference between these". Here I have cautiously tried to give many examples of possible differences, while being careful not to lead in any particular direction. I have emphasised that differences can be found even between objects that look identical, the question is which of these is most important from their point of view. I have emphasised that we are looking for relative rather than absolute significance.
2. Another problem is almost the opposite in nature. People can approach the task in what appears to be an unengaged manner, blithely tossing off distinctions which don't seem too significant. When this happens I have tried asking "In what was is that significant, what difference does that make ?, checking to see that the respondent can articulate the significance, and if not checking to see if they really understand the exercise.
3. This means of checking up is a procedure that in retrospect I feel I should have used more systematically. The focus on differences was inspired by Gregory Bateson's book "Mind and Nature: A Necessary Unity", where he refers to information as "difference(s) that make a difference". I had obtained differences, but had not usually asked "what difference does that make" I had taken distinctions and had assumed I understood their significance, whereas being dumb and asking what difference they made may have been more enlightening. One example of such a follow-up inquiry is the inquiry about the importance of registration (above), another from a different field is given in the Afterword.
4. Another problem relates to respondents who are almost too helpful. As can be seen from the tables below it is common for some respondents to report more than one difference. When well organised I have dutifully noted these down and then asked the CEO, after reading them back, "and which of these......are the most significant ?" Failing to do this has meant I have been the one that ends up speculating on their relative importance to the respondent.

Three larger sets of distinctions were also made:

1. Between the above and others that were registered with the Ministry of Cooperatives and Industry.

2. Between all those registered at the LGA and Ministry level, and those not registered at all.

3. Between those who have applied for registration but who have not yet been processed and those (mainly women's associations), who have not so far requested registration.

Potential

1. Once the structure of distinctions is developed it can be used as a means of structuring other lines of inquiry. In the case of the CAA HCS I proceeded to ask the Country Director which type of NGOs he would most like to fund in the future, starting at the root distinction and working up the branches following the most preferred lines first and least preferred afterwards. The types preferred at each level are marked in Table 1 by a #(8). It should be possible to do other exercises which involve the same form of bi-polar ranking, such as by asking, "which of these... [two groups]... have been the most/least successful"...."are the most/least sustainable"...etc.

2. The two dimensional format of the table is under utilised at present. For example, in the case of Table 3, the height of each row on the far right could correspond to the number of associations in that group. In Tables 1 and 2 they could represent the scale of funding made available to that type of NGO by CAA and Christian Aid.

3. As explained in the Nigerian example above, it is possible to do the HCS without any actual cards. It is conceivable that the same exercise could be applied even where there are no specific members to a set of objects or events. In this case the process could start out with a questions such as "What sort of x are there ?" , where x is a basic category which is of concern to the person making the inquiries, and is one presumably known to the respondent. After generating some different objects within category x the respondent could be asked to identifying the most significant difference between them until all objects had been sorted by the HCS process. Again with each of the objects sorted the question "What sort of x are there ?" could be re-applied and so on. A simple example relating to peoples views of health services in Bangladesh is given below in the Afterword.

4. In this paper the differences and similarities between sets of constructs used by three different people have been interpreted through the eyes of a third party who has some shared interests with the respondents (myself). Gaines and Shaw (1996) have developed a more systematic approach to comparing peoples constructs elicited by the Repertory Grid, by making use of two types of differences between people's construct systems. These are differences in the terminology used to describe a particular distinction, and differences in the type of distinction made using a given terminology(9). As Gaines and Shaw have shown, it is possible for one persons terms to be given to another person who is then asked to make distinctions between the same set of objects, and then both peoples' sets of distinctions can be compared. It is also possible that one person's set of distinctions to be given to another and that other person is then asked to put their own terms on each branch in the hierarchy of distinctions, and again the results of the two people can be compared. This more systematic and more participatory approach to the analysis of peoples constructs has not yet been attempted with the use of the HCS method, but it should be possible.

Comparisons with Other Methods

Matrix Ranking is a method which has been well described in PLA/PRA Notes (IIED 1992) and other papers, manuals and guides to PRA. It is very similar in structure to what is called the Repertory Grid, a tool developed by the psychologist George Kelly (Kelly, 1955), and shares some of its problems.

1. A limitation of the Matrix Ranking approach, as used by psychologists in the form of the Repertory Grid, is that the process of generating the distinctions between the objects being compared is random and unordered, and as such is only partially an expression of the respondents world view(10). Many distinctions may be of only marginal relevance to the respondents life. However with the HCS method the distinctions are purposively selected because of their significance to the respondent.

2. In the case of both Matrix Ranking and the Repertory Grid the results of the interaction between interviewer and respondent are summarised as a set of figures in table form, and as such they are not so easy to interpret on their own. In the case of the Repertory Grid the figures are analysed statistically, to extract structures from them(11). With Matrix Ranking the emphasis is usually much more on the process of obtaining the figures. With the HCS meaning can be obtained during the process, but the product is also more user-friendly, being summarised immediately in a structured diagram rather than an array of numbers. The results can be read from an introductory level of high generality through progressive levels of greater detail. Because of the nested structure many more criteria of concern can be displayed at the same time.

3. Because the HCS does not uses rating values it avoids a problem that is shared by Matrix Ranking and the Repertory Grid. When middle range values are generated in the course of using Matrix Ranking or the Repertory Grid they can vary in meaning from "irrelevant", to "half and half", to "I don't know".
   
4. Finally, the HCS method is simpler to use.

--o0o--

References

Bateson, G. (1979) Mind and Nature: A Necessary Unity. Bantam, USA.

Gaines, M.L.G., Shaw, B.R. (1996) Comparing Constructions through the Web.

http://ksi.cpsc.ucalgary.ca/articles/CSCL95WG

International Institute for Environment and Development (1992) "Special Issue on Applications of Wealth Ranking", RRA Notes. No. 15. IIED. London.

Kelly, G (1955) The Psychology of Personal Construct Theory. Norton. New York.

Zadek, S (1996) Value-Based Organisations: Organising NGOs for Value Based Effectiveness. New Economics Foundation. London.

Afterword:

In 1994 staff of the Research Unit of the Christian Commission for Development in Bangladesh (CCDB) used the cardless version of the same method in the process of exploring poor peoples conceptions of health, disease and medicine. The following quote describes the results of two applications of the method:

"In two somities [beneficiary groups] we also asked the members to come up with different medicaments they use when they are ill. We asked them to divide these medicaments into two groups. We asked for an explanation after each division. After dividing the group into two other groups we asked them to do the same for the two other sub-groups. In the first somity they divided the medicaments into allopathic and herbal medicines. The allopathic medicines were divided into tablets and syrup. The herbal medicine into medicines used for cough and influenza, and medicines for influenza.

In the second somity they divided the medicaments into medicines given by the kobiraj and medicines given by the allopathic doctor. The medicines given by the kobiraj were divided into medicines for weakness and medicines for influenza and headache. The medicines of the allopathic doctor were divided into medicines for pain and medicines for gastric burns.

We used this exercise to get more information about people's concepts of medicines. When we asked about [the consequences of] the difference between herbal and allopathic medicines it was mentioned that allopathic medicines cure better and sooner but they are costly and can give weakness. Herbs are "softer" to the body, they don't harm the body, they are cheap , and although they work slowly they keep the body healthy." (People's Health and Nutrition, January 1995, CCDB)

Endnotes:

(1)There is a large amount of information available about the Kelly Grid, and the associated theory known as Personal Construct Psychology, on the World Wide Web. See http://ksi.cpsc.ucalgary.ca/articles/CSCL95WG/

(2) The New Economic Foundation has developed their own variation of these two methods, called an "Organisation Ranking Grid" (Zadek,1996).

(3)All three examples presented in this paper were produced by men.

(4) The Line Draw function in Word Perfect 5.1, along with the Outline function, can also be used to draw a tree structure.

(5) In this paper the real names of the NGOs have been omitted.

(6) Membership based organisations rather than service providing NGOs

(7) In Bangladesh major differences in views between NGOs that belong to the Association for Development Agencies in Bangladesh (ADAB) have taken the form of splits between the younger and smaller NGOs and the older larger NGOs.

(8) A # next to both branches of the tree indicates there was no preference for either.

(9) They describe the four possibilities generated as Consensus (same term, same distinction), Conflict (same term, different distinction), Correspondence (different term, same distinction) and Contrast (different term, different distinction).

(10) This generates an additional problem, that of maintaining the persons interest in the exercise. The same problems apply to NEF's Organisation Ranking Grid (Zadek, 1996:41).

(11) For example, by cluster analysis. Hierarchical structures that are developed by cluster analysis methods locate branches according to the degree to which ratings of objects on different attributes differ from each other, not by the consequences of those differences in ratings.. Many of the important differences between objects or events in our world are not necessarily clearly defined. Mathematical analyses exclude meaning.

Last updated: 31 October 1996

This page has had visitors since May 1st 1997

Counter re-srated: Jan 2005