by John Mayne. ILAC Brief, October 2012 Available as pdf
“An ongoing challenge in evaluation is the need to make credible causal claims linking observed results to the actions of interventions. In the very common situation where the intervention is only one of a number of causal factors at play, the problem is compounded – no one factor ’caused’ the result. The intervention on its own is neither necessary nor sufficient to bring about the result. The Brief argues the need for a different perspective on causality. One can still speak of the intervention making a difference in the sense that the intervention was a necessary element of a package of causal factors that together were sufficient to bring about the results. It was a contributory cause. The Brief further argues that theories of change are models showing how an intervention operates as a contributory cause. Using theories of change, approaches such as contribution analysis can be used to demonstrate that the intervention made a difference – that it was a contributory cause – and to explain how and why.”
See also Making Causal Claims by John Mayne at IPDET 2012, Ottawa
RD Comments:
What I like in this paper: The definition of a contributory cause as something neither necessary or sufficient, but a necessary part of a package of causes that is sufficient for an outcome to occur
I also like the view that “theories of change are models of causal sufficiency”
But I query the usefullness of distinguishing between contributory causes that are triggering causes, sustaining causes and enabling causes, mainly on the grounds of the difficulty of reliably identifying them
I am more concerned with the introduction of probablistic statements about “likely” necessity and “likely” sufficiency, because it increases the ease with which claims of casual contribution can be made, perhaps way too much. Michael Patton recently expressed a related anxiety: “There is a danger that as stakeholders learn about the non-linear dynamics of complex systems and come to value contribution analysis , they will be inclined to always find some kind of linkage between implemented activities and desired outcomes ….In essence, the concern is that treating contribution as the as the criterion (rather than direct attribution) is so weak that a finding of no contribution is extremely unlikely”
John Mayne’s paper distinguishes between four approaches to demonstrating causality (adapted from Stern et al., 2012:16-17):
- “Regularity frameworks that depend on the frequency of association between cause and effect – the basis for statistical approaches making causal claims
- Counterfactual frameworks that depend on the difference between two otherwise identical cases – the basis for experimental and quasiexperimental approaches to making causal claims
- Comparative frameworks that depend on combinations of causes that lead to an effect – the basis for ‘configurational’ approaches to making causal claims, such as qualitative comparative analysis
- Generative frameworks that depend on identifying the causal links and mechanisms that explain effects – the basis for theory-based and realist approaches to making causal claims .”
- Claims can be made about the co-variance of events
- Counterfactual approaches: describing the effects of the absence and presence of an intervention on an outcome of interest, when all other conditions being kept the same
- Configurational approaches, describing the effects of the presence and absence of multiple conditions (relating to both context and intervention)
- Statistical approaches, describing the effects of more complex mixes of variables
- Claims can be made about causal mechanisms underlying each co-variance that is found
Good causal claims contain both 1 and 2: evidence of co-variance and plausable or testable explanations of why each co-variance exists. One without the other is insufficient. You can start with theory (a proposed mechanism) and look for supporting co-variance, or start with a co-variance and look for a supporting mechanism. Currently theory led approaches are in vogue.
