Measuring National Well-being: Measuring What Matters

[UK] National Statistician’s Reflections on the National Debate on Measuring National Well-being July 2011

Chapter 1: What is national well-being? … 4
Chapter 2: Why measure national well-being and who will use the measures? . 9
Chapter 3: Measuring national well-being…..15
Chapter 4: Partnerships and next steps..20
References. 24
Notes. 25

“On 25 November 2010, I accepted an invitation from the Prime Minister, David Cameron, to develop measures of national well-being and progress. I am convinced that this is something that can only be done with an understanding of what matters most to people in this country.

In response to this invitation, the Office for National Statistics (ONS) undertook a national debate on ‘what matters to you?’ between 26 November 2010 and 15 April 2011. I was impressed by the number of people who were willing to take part in discussions and also by the depth of responses. In total, ONS held 175 events, involving around 7,250 people. In total the debate generated 34,000 responses, some of which were from organisations and groups representing thousands more. The quotes on each page of this report were taken from online contributions, where permission was given to reproduce the participant’s words anonymously. I am grateful to everyone who took the time to take part in the debate, and to those who organised and hosted events.

The debate has helped us identify the key areas that matter most and will help to ensure that the measures we use will be relevant not only to government but also to the wider public. This is crucial to allow for effective development and appraisal of policy for individuals to use information to identify ways of improving well-being, and to allow for assessment of how society is doing overall.

The term ‘well-being’ is often taken to mean ‘happiness’. Happiness is one aspect of the well-being of individuals and can be measured by asking them about their feelings – subjective well-being. As we define it, well-being includes both subjective and objective measures. It includes feelings of happiness and other aspects of subjective well-being, such as feeling that one’s activities are worthwhile, or being satisfied with family relationships. It also includes aspects of well-being which can be measured by more objective approaches, such as life expectancy and educational achievements. These issues can also be looked at for population groups – within a local area, or region, or the UK as a whole.
Developing better measures of well-being and progress is a common international goal and the UK is working with international partners to develop measures that will paint a fuller picture of our societies. This is a long-term programme and I am committed to sharing our ideas and proposals widely. This will help to ensure that UK well-being measures are relevant and founded on what matters to people, both as individuals and for the UK as a whole as well as being reliable and impartial and serving to improve our understanding of UK society.

This report summarises the contributions made to the debate and explains how ONS is using the findings to develop measures of national well-being. I look forward to your further comments and advice in response to this report. These should be sent to”
Jil Matheson
National Statistician

See more on the ONS website

UK Independent Commission for Aid Impact (ICAI) – online consultation

ICAI website text:

“The Independent Commission for Aid Impact (ICAI) is the independent body responsible for the scrutiny of UK aid, focusing on delivery of value for money for the UK taxpayer, maximising the impact for recipients and ensuring effectiveness of the UK aid budget. ICAI reports to Parliament through the International Development Select Committee.

ICAI is currently running a consultation calling for members of the public to have their say on which areas of UK overseas aid they would like to see looked at. Responses to the consultation will assist ICAI to develop its work plan for the next three years. To respond to the consultation please visit” [where you will find an online survey  with suppporting background information on DFID]

“The deadline for the consultation is the 7th April 2011.

For enquiries about the ICAI consultation please contact Clare Robathan, Communications and Research Officer on 020 7023 6734, or

RD comment:  Re the online survey used for the consultation, this is by no means the best designed online survey I have ever seen, but please make use of it. The survey is also available as a downloadable pdf.

The ICAI website has some basic problems. While there is a Contact Us page there is no comment facility at on any of the pages, as far as I can see. Nor is there a no disclosure/transparency policy. You can ask for the results of the survey via the enquiries email address, but they could be immediately available right now, because the website is using Referring to the three newly appointed commissioners, the website says “The three Commissioners, Mark Foster, John Githongo and Diana Good are acknowledged leaders in their fields. Together they contribute a wealth of international experience in the private sector, in combating corruption and in development.” Yet, as far as I can see, none of the commisioners has any significant evaluation experience. Yet they are responsible for contracting an organisation (or group of organisations) to do evaluation work on behalf of the ICAI. In doing so they will need to secure value for money, which requires assessing both value as well as money spent.  I think we should watch the performance of this commission quite carefully.

PS 15th February 2011: Visitors may be interested to read the ICAI Terms of Reference 2010 for the evaluation functions being contracted out by the ICAI, and the supporting documentation, the Independent Commission for Aid Impact-Presentation-for-pre-bid-meeting, made by DFID on 22 November 2010

Predicting the achievements of the Katine project

September 2010: This post provides information on a revised proposal for a “Predictions Survey” on the achievements of the Katine Community Partnerships Project, a project managed by AMREF and funded by the Guardian and Barclays Bank, between 2007 and 2011.

Background Assumptions

The Guardian coverage of the Katine project has provided an unparalleled level of public transparency to the workings of an aid project. As of August 2010 there have been approximately 530 articles posted on the site, most of which have specifically about Katine. These posts have included copies of project documentation (plans, budgets, progress reports, review reports) that often don’t enter the public realm.

Ideally this level of transparency would have two benefits: (a) improving UK public knowledge about the challenges of providing effective aid, (b) imposing some constructive discipline on the work of the NGO concerned, because they know they are under continuing scrutiny not only locally, but internationally. Whether this has actually been the case is yet to be systematically assessed. However I understand the effects on the project and its local stakeholders  (i.e b above) will be subject to review by Ben Jones later this year, and then open to discussion in a one day event in November, to be organised by the Guardian.

So far there have been two kinds of opportunities for the British, and other publics, to be engaged with the public monitoring of the Katine project. One has been through posting comments on the articles on the Guardian website. About 30% of all articles have provided this opportunity, and these articles have attracted an average of 5 comments . The other option has been by invitation from the Guardian, to make a guest posting on the website. This invitation has been extended to specialists in the UK and elsewhere.  Multiple efforts have also been made to hear different voices from within the Katine community itself

The Predictions Survey would provide another kind of opportunity for participation. It would be an opportunity for a wide range of participants to:

  • to make some judgments about the overall achievements of the project, and
  • to explain those judgments, and
  • to see how those judgments compared to that of others, and
  • to see how those judgments compare to the facts, about what has actually been achieved at the end of the project

In addition a Predictions Survey would provide a means of testing expectations that greater transparency can improve public knowledge about the challenges of providing effective aid.

My proposal is that that the Prediction Survey would consist of five batches of questions, one for each project component, on a separate page. Each question would be a multiple choice question, but associated with an optional Comment field. People could respond on the basis of their existing knowledge of the project (which could vary widely) and/or extra information about the website obtained via component specific links embedded at the head of each page of the online survey e.g. on water and sanitation. Questions at the end of the survey would identify participants’ sources of knowledge about the project (e.g. obtained before and during the survey, from the website and elsewhere).

A 1st rough draft survey form is already available to view. Any responses entered at this stage may be noted, but they will then be deleted and not included in any final analysis.  The final design of the survey will require close consultation with AMREF and the Guardian.

Intended participants in the survey

  • UK public, reached via the Guardian
  • Uganda public, reached via Ugandan newspapers (likely to be more of a challenge)
  • AMREF staff, especially in Uganda, Kenya HQ and UK
  • The Guardian and Barclays, as donors
  • Monitoring and Evaluation specialists, reached via an international email list

Hypotheses (predictions about the predictions)

  1. We might expect that AMREF would be able to make the most accurate predictions, given its central role. But aid agencies are often tempted to put a gloss on their achievements, because of the gap that sometimes emerges between their ambitions and what can actually be done in practice.
  2. We might expect that participants who have been following the Guardian coverage closely since the beginning might be better informed and make better predictions than others who have become interested more recently. But perhaps those participants are still responding on the basis of their original beliefs (aka biases)?
  3. We might expect M&E specialists to make better than average predictions because of their experience in analysing project performance. But perhaps they have become too skeptical about everything they read
  4. We might expect the Guardian and Barclays staff to make better than average predictions because they have been following the project closely since inception and their organisation’s money is  invested in it. But perhaps they only want to see success.
  5. We might expect the highest frequency choices (across all groups) to be more accurate than the choices of any of the above groups, because of a ” wisdom of crowds” effect. The potential of crowdsourcing was of interest to the Guardian at the beginning of the project, and this survey could be seen as a form of crowdsourcing – of judgements.

This list is not final. Other hypotheses  could be identified in the process of consultation over the design of the survey

There may also be other less testable predictions worth identifying. For example, about the effects of this Prediction Survey on the work done by AMREF and its partners in the final year up to October 2011. Might it lead to a focus on what is being measured by the survey, to the detriment of other important aspects of their work?  If AMREF has a comprehensive monitoring framework and the prediction survey addresses the same breadth of performance (and not just one or two performance indicators) this should not be a problem.


The fourth and final year of the project starts in October 2010 and ends in October 2011.

The finalisation of the design of the Predictions Survey will require extensive consultation with AMREF and the Guardian, in order to ensure the fullest possible ownership of the process, and thus the results that are generated. Ideally this process might be completed by late-October 2010

The survey could be open from late October to the end of March 2011 (six months before the end of the project). All responses would be date stamped to take account of any advantages of being a later participant

A process will need to be agreed in 2010 on how objective information can be obtained on which of the multiple choice options have eventuated by October 2011.

A post 2011 follow up survey may be worth considering. This would focus on predictions of what will happen in the post-project period, up to 2014, the year of the vision statement produced by participants in the September 2009 stakeholders workshop in Katine.

“In 2014, Katine will be an active, empowered community taking responsibility for their development with decent health, education, food security and able to sustain it with the local government”


The participation of the Guardian and AMREF will be very important, although it is conceivable that the survey could be run independently of their cooperation

Assistance with publicity, to find participants, would be needed from the Guardian and Barclays

Advisory support is being sought from the One World Trust

Advisory support from other other organisations could also be useful

The online survey could be designed and managed by Rick Davies. However responsibility could be given to another party that was agreed to by AMREF, Guardian and Barclays.


  • The survey design needs to be short enough to encourage people to complete it, but not so short that important aspects of the project’s performance are left out
  • The description of the objectives used in the survey needs to be as clear and specific as possible, but also keep as close to AMREF’s original words as possible (i.e. as in the 4th year extension proposal, and using the M&E framework, now being updated)
  • Participants will be asked to make a single choice between multiple options, describing what might happen. These options will need to be carefully chosen, so there are no obvious “no brainers”, and to cover a range of plausible possibilities
  • It may be necessary in some cases (e.g. with some broadly defined objectives) to allow multiple choices from multiple options
  • I have heard that AMREF will be conducting a final evaluation in late 2011, using an external consultant. This evaluation could be the source of the final set of data on actual performance, against which participant’s predictions could be compared. But will it be seen as a sufficiently independent source of information?