Laura Rodriguez Takeuchi, ODI Project Note 04, June 2014. Available as pdf [2]
“Key messages:
- In the measurement of multidimensional well-being, weights aim to capture the relative importance of each componentto a person’s overall well-being. The choice of weights needsto be explicit and could be used to incorporate people’sperspectives into a final metric.
- Stated preferences approaches aim to obtain weights from individuals’ responses to hypothetical scenarios. We outline six of these approaches. Understanding their design and limitations is vital to make sense of potentially dissimilar result.
- It is important to select and test an appropriate method for specific contexts, considering the challenges of relying on people’s answers. Two methodologies, DCE and PTO, are put forward for testing in a pilot project.”
See also:Laura Rodriguez Takeuchi blog posting on Better Evaluation: Week 26: Weighing people’s values in evaluation [3]
Rick Davies comment: Although this was a very interesting and useful paper overall, I was fascinated by this part of Laura’s paper ”
Reflecting on the psychology literature, Kahneman and Krueger (2006) argue that it is difficult to deduce preferences from people’s actual choices because of limited rationality:
“[People] make inconsistent choices, fail to learn from experience, exhibit reluctance to trade, base their own satisfaction on how their situation compares with the satisfaction of others and depart from the standard model of the rational economic agent in other ways.” (Kahneman and Krueger 2006: 3)
Rather than using these ‘real’ choices, stated preferences approaches rely on surveys to obtain weights from individuals’ responses to hypothetical scenarios.
This seems totally bizarre. What would happen if we insisted on all respondents’ survey responses being rational, and applied various other remedial measures to make them so! Would we end up with a perfectely rational set of responses that have no actual fit with how people or behave in the world? How useful would that be? Perhaps this is what happens when you spend too much time in the company of economists? :-))
On another matter…. Table 1 usefully lists eight different weighting methods, which are explained in the text. However this list does not include one of the simplest methods that exists, and which is touched upon tangentially in the reference to the South African study on social perceptions of material needs (Wright, 2008). This is the use of weighted checklists, where respondents choose both items on a checklist and the weights to be given to each item, in a series of binary (yes/no) choices. This method was used in a series of household poverty surveys in Vietnam in 1997 and 2006 using an instrument called a Basic Necessities Survey [4]. The wider potential uses of this participatory and democratic method are are discussed in a related blog on weighted checklists [5].
Postscript: Julie Newton has point out this useful related website:
- Measuring National Well-being. “ [6]ONS [UK Office for National Statistics] is developing new measures of national well-being. The aim is to provide a fuller picture of how society is doing by supplementing existing economic, social and environmental measures. Developing better measures of well-being is a long term programme. ONS are committed to sharing ideas and proposals widely to ensure that the measures are relevant and founded on what matters to people.” Their home page lists a number of new publications on this subject