The “Rick on the Road” blog is where I write about specific issues M&E issues that are of interest to me. Here is the list of the most recent “Rick on the Road” editorial postings:
2023
- Finding useful distinctions between different futures
- How can evaluators practically think about multiple Theories of Change in a particular context?
2022
- Four types of futures that should be covered by a Theories of Change
- We need more doubt and uncertainty!
- Using ParEvo to conduct thought experiments
- Alternative futures as “search strategies”
- Budgets as theories
- Making small samples of large populations useful
2021
- Choosing between simpler and more complex versions of a Theory of Change
- Exploring counterfactual histories of an intervention
- Reconciling the need for both horizontal and vertical dimensions in a Theory of Change diagram
- Diversity and complexity? Where should we focus our attention?
- Paired case comparisons as an alternative to a configurational analysis (QCA or otherwise)
- The potential use of Scenario Planning methods to help articulate a Theory of Change
- Mapping the “structure of cooperation”: Adding the time dimension and thinking about further analyses
- Connecting Scenario Planning and Theories of Change
2020:
- The implications of complex program designs: Six proposals worth exploring?
- “If you want to think outside of the box, you first need to find the box” – some practical evaluative thinking about Futures Literacy
- Has the meaning of impact evaluation been hijacked?
- Quality of Evidence criteria that can be applied to Most Significant Change (MSC) stories
- Mapping the structure of cooperation
- EvalC3 versus QCA – compared by a re-analysis of one data set
- Converting a continuous variable into a binary variable i.e. dichotomising
- Rubrics? Yes, but…
- Temporal networks: Useful static representations of dynamic events
2019:
- Combining the use of the Confusion Matrix as a visualisation tool with a Bayesian view of probability
- On finding the weakest link…
- Participatory design of network models: Some implications for analysis
- Extracting additional value from the analysis of QuIP data
- On evaluating innovation
- Where there is no (decent / usable) Theory of Change…
- On using clustering algorithms to help with sampling decisions
2018:
2017
2016
- …and then a miracle happens (or two or three)
- Three ways of thinking about linearity
- EvalC3 – an Excel-based package of tools for exploring and evaluating complex causal configurations
- Why we should also pay attention to “what does not work”
- Why I am sick of Evaluation Questions!
2015
- False Positives – why we should pay more attention to them
- Macro versus meta Theories of Change
- Clustering projects according to similarities in outcomes they achieve
- Evolving better performing hypotheses, using Excel
- Is QCA its own worst enemy?
- Characterising purposive samples
- Evaluating the performance of binary predictions
- How to select which hypotheses to test?
- In defense of the (careful) use of algorithms and the need for dialogue between tacit (expertise) and explicit (rules) forms of knowledge
- A mistaken criticism of the value of binary data
2014
- Comparing QCA and Decision Tree models – an ongoing discussion
- Pair comparisons: For where there is no common outcome measure?
- The challenges of using QCA
- Thinking about set relationships within monitoring data
2013
- Complex Theories of Change: Recipes for failure or for learning?
- Measuring the impact of ideas: Some testable propositions
- A reverse QCA?
- Another perspective on the uses of control groups
- An example application of Decision Tree models
- My problem with RCTs
2012
- Evolutionary strategies for complex environments
- AusAID’s ‘Revitalising Indonesia’s Knowledge Sector for Development Policy’ program
- Open source evaluation – the way forward?
- Representing different combinations of causal conditions
- A perspective on “Value for Money” relationships
- Data mining algorithms as evaluation tools
- Criteria for assessing the evaluability of Theories of Change
- Can we evolve explanations of observed outcomes?
- Modular Theories of Change: A means of coping with diversity and change?
- Evaluation questions: Managing agency, bias and scale
2011
- Evaluation quality standards: Theories in need of testing?
- Relative rather than absolute counterfactuals: A more useful alternative?
- Evaluation methods looking for projects or projects seeking appropriate evaluation methods?
- Models and reality: Dialogue through simulation
- A submission to the UK Independent Commission for Aid Impact (ICAI)
2010
-
- Counter-factuals and counter-theoreticals: What to do when random assignment is not an option….
- Do we need a Minimal Level of Failure (MLF)?
- Meta-narratives, evaluation and complexity
- Cynefin Framework versus Stacey Matrix versus network perspectives
- Evaluating a composite Theory of Change (ToC)
2009
- Reflections on Dave Snowden’s presentations on sense-making and complexity
- On the poverty of baselines and targets…
- Why we should make economists work harder
- Constructing longer term perspectives
- Bibliographic Timelines
2008
- Comments on the draft DFID evaluation policy
- An aid bubble? – Interpreting aid trends
- Aid organisations as self-interested businesses?
- Social Frameworks: An improvement on the Logical Framework?
- Assessing achievements in Katine, Uganda
- A network approach to the selection of “Most Significant Change” stories
2007
- Managing expectations about monitoring and evaluation in Katine
- Katine: an experiment in more publicly transparent aid processes
- Checklists as mini theories-of-change
- Evolving storylines: A participatory design process?
- Prediction markets as a source of independent and continuous evaluation for development projects?
2006
- Assumptions, evidence and multiple stakeholders
- Evidence that the (development) world is getting better
- Integrating funding applications and baseline surveys
- The risks of big increases in aid flows to poor countries
- The “attribution problem” problem
2005
- Impact pathways and genealogies
- Networks of Indicators
- Fight institutional Alzheimers
- Using “modular matrices” to describe programme intentions and achievements
- Constructing “an auditable trail of intentions….”
- Identifying the impact of evaluations: Follow the money?
- Learning circles and loops: Time for some more sophisticated representations
2004
- No more paradigm changes please!
- Where have all the evaluations gone?
- Projects versus Project Funding Mechanisms
- Treating organisations as though they were machines
- Is moving the goal posts a good thing?
- Where are the partners?
- Monitoring empowerment: A contradiction in terms?
- Why did the chicken cross the road?
- Thinking about networks of policies
- Question: How do you assess a country’s ownership of a PRSP?
- Hypothesis-led Surveys of Influence – on KAP
- PRSP Monitoring: Target fixation and mission creep
Readers may also be interested in
- The Rick’s Methods page on this site
- Rick Davies’ comments on other websites and blogs [in process]
- My YouTube presentations from some training events and conferences
- See also Evaluating Katine
Dear Rick Davies, I found your website, find this highly interesting and would like to join the email list. But do I really need a yahoo account? (I have so many email accounts already…)
Kind regards, Marion
Marion Karmann, PhD
Monitoring and Evaluation Program
FSC International Center GmbH
Charles de Gaulle Straße 5
53113 Bonn Deutschland/ Germany
Phone: +49 (0)228 367 66-77
Fax: +49 (0)228 367 66-30
E-Mail: m.karmann@fsc.org
FOREST STEWARDSHIP COUNCIL (FSC)
http://www.fsc.org
Geschäftsführer/ Director: Andre de Freitas
Handelsregister/ Commercial register: Bonn HRB12589 USt.-Id Nr./ VAT No: DE 814618194
® FSC, A.C. All rights reserved. FSC-SECR-0002
The information in this email and any attachments is confidential and is intended for the addressee only. If you are not the intended recipient, please delete this message and any attachments and advise the sender.
Unfortunately yes, you do need to sign up to a Yahoo account.
regards, rick
hello mr ricks davies, you are doing a marvellous job there
but how can l subscribe to the site.
thank
You can use the Google Alert service. See the link on the top right side of the webpage, under the Search button
I want to receive latest news on M&E how can these be directed to my e-mail address provided.
Thanks your site is very insightful.
You can join the MandE NEWS email list, by clicking on the M&E Email list tab at the top of this page
Greetings from the U4 Anti-Corruption Resource Centre
Evidence on what works and why is sparse in anti-corruption. Credible, unbiased evidence of outcomes and impact of anti-corruption interventions is in great demand.
Therefore, the U4 Anti-Corruption Resource Centre has launched a new thematic area which will research and distil lessons for donors on how to improve evaluations and measurement tools to better inform donor policies and programme designs relating to anti-corruption. I would be glad to discuss with any experienced practitioners in the field on how to achieve this goal, just email jesper.johnson@cmi.no.
The first publication we have produced is on “How to monitor and evaluate anti-corruption agencies”. The report provides technical, methodological, and practical guidance to assist staff of ACAs and donors in undertaking monitoring and evaluation and shows how the outcomes and impact of the work of ACAs can be evaluated in an objective, evidence-based manner.
Access here: http://www.cmi.no/publications/file/4171-how-to-monitor-and-evaluate-anti-corruption.pdf
Kind regards,
Jesper Johnson