by Sandra Nutley, Alison Powelland Huw Davies, Research Unit for Research Utilisation (RURU), School of Management, University of St Andrews, www.ruru.ac.uk November 2012
Available as pdf. This is a paper for discussion. The authors would welcome comments, which should be emailed to smn@st-andrews.ac.uk or Jonathan.Breckon@nesta.org.uk
In brief
Making better use of evidence is essential if public services are to deliver more for less. Central to this challenge is the need for a clearer understanding about standards of evidence that can be applied to the research informing social policy. This paper reviews the extent to which it is possible to reach a workable consensus on ways of identifying and labelling evidence. It does this b y exploring the efforts made to date and the debates that have ensued . Throughout, the focus is on evidence that is underpinned by research, rather than other sources of evidence such as expert opinion or stakeholder views .
After setting the scene, the review and arguments are presented in five main sections:
We begin by exploring practice recommendations: many bodies provide practice recommendations, but concerns remain as to what kinds of research evidence can or should underpin such labelling schemas.
T his leads us to examine hierarchies of evidence: study design has long been used as a key marker for evidence quality, but such ‘hierarchies of evidence ’ raise many issues and have remained contested. Extending the hierarchies so that they also consider the quality of study conduct or the use of underpinning theory have enhanced their usefulness but have also exposed new fault – lines of debate.
More broadly, in beyond hierarchies, we recognise that hierarchies of evidence have seen most use in addressing the evidence for what works . As a consequence,several agencies and authors have developed more complex matrix approaches for identifying evidence quality in ways that are more closely linked to the wider range of policy o r practice questions being addressed.
Strong evidence, or just good enough? A further pragmatic twist is seen by the recognition that evaluative evidence is always under development. Thus it may be more helpful to think of an ‘evidence journey’ from promising early findings to substantive bodies of knowledge.
Finally, we turn to the uses and impacts of standards of evidence and endorsing practices . In this section we raise many questions as to the use, uptake and impacts of evidence labelling schemes, bu t are able to provide few definitive answers as the research here is very patchy.
We conclude that there is no simple answer to the question of what counts as good evidence. It depends on what we want to know, for what purposes, and in what contexts we en visage that evidence being used. Thus while there is a need to debate standards of evidence we should be realistic about the extent to which such standard – setting will shape complex , politicised, decision – making by policy makers, service managers and local practitioners.
