Results of the CONFERENCE ON IMPACT EVALUATION HELD IN CAIRO 29 MARCH – 2 APRIL

Further contributions  to this list are welcome (linked documents, or the documents themselves). Please use the Comment facility below, or email rick at mande.co.uk

2 thoughts on “Results of the CONFERENCE ON IMPACT EVALUATION HELD IN CAIRO 29 MARCH – 2 APRIL”

  1. (Email copied here with permission from Irene Guijt)

    Hi ….

    It seems that protest at and since Cairo may have some effect as the door to revision of the Nonie guidance notes is not as 100% shut as I had heard it was. But this will only be effective if people put pressure on the core writing team to move from a very problematic set of guidance notes to a more acceptable product. The immediate ‘project’ is for as many people as possible to provide comments on the Nonie guidance notes so the core team realise they can’t just ignore the noise and wait for it to go away. http://www.worldbank.org/ieg/nonie/docs/Guidance_IE.pdf.

    As you can see, there are around 9 pages on ‘methodological approaches addressing the attribution problem’ (this is the Nonie definition of ‘mixed methods’) and two pages on ‘other’ including one page on ‘participatory approaches’.

    Some of the points of concern that connected with me, drawn mainly from the inspiring thoughts of Patricia Rogers and the audience in general:

    1. Provide guidance on how to understand the distribution of costs and effects, not just the average effect.

    2. Make participatory approaches an option for each task in IE, not just a one-page clause (4.4.1) in the guidance notes.

    3. Address the implications of simple, complicated, complex (aspects of) problems – reductionism works fine for a bicycle but is a bit tough on a fish (paraphrasing a great comparison Patricia made) and the same goes for interventions you are evaluating that have different ‘natures’.

    4. Provide more insight on knowledge uptake of IE findings.

    5. Why focus so much on addressing the attribution problem? Include solid discussion on contribution analysis, which is not in the current list of methods.

    6. Edit out the erroneous statements about the supposed certainty provided by RCTs that are dotted in the document.

    7. Where is the discussion on implications for IE on aspects such as gender equality and rights-based thinking?

    8. Does the focus on IE as a process of statistical analysis mean it is not doable for the ‘average’ evaluator?

    9. It is seriously unappealing to read. Edit it to make it accessible for the intended users (not evaluation academics, which is how the document seems to be currently oriented).

    PS: The contact address for the NONIE Secretariat is nonie@worldbank.org

    Greetings, irene

    Learning by Design

    Bredeweg 31, 6668 AR Randwijk, The Netherlands

    Tel: (+31) 0488-491880 Fax: (+31) 0488-491844

    Skype: ireneguijt

Leave a Reply