Category: Books

USAID, September 2018. 98 pages. Available as PDF

Rick Davies comment: A very good overview, balanced, informative, with examples. Worth reading from beginning to end.

Contents

Introduction
Roadmap: How to use this document
Machine learning: Where we are and where we might be going
• ML and AI: What are they?
• How ML works: The basics
• Applications in development
• Case study: Data-driven agronomy and machine learning
at the International Center for Tropical Agriculture
• Case study: Harambee Youth Employment Accelerator
Machine learning: What can go wrong?
• Invisible minorities
• Predicting the wrong thing
• Bundling assistance and surveillance
• Malicious use
• Uneven failures and why they matter
How people influence the design and use of ML tools
• Reviewing data: How it can make all the difference
• Model-building: Why the details matter
• Integrating into practice: It’s not just “Plug and Play”
Action suggestions: What development practitioners can do today
• Advocate for your problem
• Bring context to the fore
• Invest in relationships
• Critically assess ML tools
Looking forward: How to cultivate fair & inclusive ML for the future
Quick reference: Guiding questions
Appendix: Peering under the hood [ gives more details on specific machine learning algorithms]

See also the associated USAID blog posting and maybe also  How can machine learning and artificial intelligence be used in development interventions and impact evaluations?

 

 

by Judea Pearl, Allen Lane, May 2018

Publisher blurb: “‘Correlation does not imply causation.’ This mantra was invoked by scientists for decades in order to avoid taking positions as to whether one thing caused another, such as smoking and cancer and carbon dioxide and global warming. But today, that taboo is dead. The causal revolution, sparked by world-renowned computer scientist Judea Pearl and his colleagues, has cut through a century of confusion and placed cause and effect on a firm scientific basis. Now, Pearl and science journalist Dana Mackenzie explain causal thinking to general readers for the first time, showing how it allows us to explore the world that is and the worlds that could have been. It is the essence of human and artificial intelligence. And just as Pearl’s discoveries have enabled machines to think better, The Book of Why explains how we can think better.

Introduction: Mind over data (pdf copy)

Chapter 1: The Ladder of Causation (pdf copy)

Reviews: None found yet, but they will be listed here when found

2020 05 20: New material from Judea Pearl – “On this page I plan to post viewgraphs and homeworks that I currently use in teaching CAUSALITY. So far, we have covered the following chapters”:

http://bayes.cs.ucla.edu/BOOK-2K/viewgraphs.html

by Justin Parkhurst © 2017 – Routledge,
Available as pdf and readable online, and hardback or paperback

“There has been an enormous increase in interest in the use of evidence for public policymaking, but the vast majority of work on the subject has failed to engage with the political nature of decision making and how this influences the ways in which evidence will be used (or misused) within political areas. This book provides new insights into the nature of political bias with regards to evidence and critically considers what an ‘improved’ use of evidence would look like from a policymaking perspective”

“Part I describes the great potential for evidence to help achieve social goals, as well as the challenges raised by the political nature of policymaking. It explores the concern of evidence advocates that political interests drive the misuse or manipulation of evidence, as well as counter-concerns of critical policy scholars about how appeals to ‘evidence-based policy’ can depoliticise political debates. Both concerns reflect forms of bias – the first representing technical bias, whereby evidence use violates principles of scientific best practice, and the second representing issue bias in how appeals to evidence can shift political debates to particular questions or marginalise policy-relevant social concerns”

“Part II then draws on the fields of policy studies and cognitive psychology to understand the origins and mechanisms of both forms of bias in relation to political interests and values. It illustrates how such biases are not only common, but can be much more predictable once we recognise their origins and manifestations in policy arenas”

“Finally, Part III discusses ways to move forward for those seeking to improve the use of evidence in public policymaking. It explores what constitutes ‘good evidence for policy’, as well as the ‘good use of evidence’ within policy processes, and considers how to build evidence-advisory institutions that embed key principles of both scientific good practice and democratic representation. Taken as a whole, the approach promoted is termed the ‘good governance of evidence’ – a concept that represents the use of rigorous, systematic and technically valid pieces of evidence within decision-making processes that are representative of, and accountable to, populations served”

Contents
Part I: Evidence-based policymaking – opportunities and challenges
Chapter 1. Introduction
Chapter 2. Evidence-based policymaking – an important first step, and the need to take the next
Part II: The politics of evidence
Chapter 3. Bias and the politics of evidence
Chapter 4. The overt politics of evidence – bias and the pursuit of political interests
Chapter 5. The subtle politics of evidence – the cognitive-political origins of bias
Part III: Towards the good governance of evidence
Chapter 6. What is ‘good evidence for policy’? From hierarchies to appropriate evidence.
Chapter 7. What is the ‘good use of evidence’ for policy?
Chapter 8. From evidence-based policy to the good governance of evidence

by Julian King, Oxford Policy Management. January 2018. Available as pdf

Excerpt from Foreword:

In 2016, Oxford Policy Management (OPM) teamed up with Julian King, an evaluation specialist, who worked with staff from across the company to develop the basis of a robust and distinct OPM approach to assessing VfM. The methodology was successfully piloted during the annual reviews of the Department for International Development’s (DFID) Sub-National Governance programme in Pakistan and MUVA, a women’s economic empowerment programme in Mozambique. The approach involves making transparent, evidence-based judgements about how well resources are being used, and whether the value derived is good enough to justify the investment.

To date, we have applied this approach on upwards of a dozen different development projects and programmes, spanning a range of clients, countries, sectors, and budgets. It has been well received by our clients (both funding agencies and partner governments) and project teams alike, who in particular appreciate the use of explicit evaluative reasoning. This involves developing definitions of what acceptable / good / excellent VfM looks like, in the context of each specific project. Critically, these definitions are co-developed and endorsed upfront, in advance of implementation and before the evidence is gathered, which provides an agreed, objective, and transparent basis for making judgements.

Table of contents
Foreword 1
Acknowledgements 3
Executive summary 4
1 Background 7
1.1 What is VfM? 7
1.2 Why evaluate VfM? 7
1.3 Context 8
1.4 Overview of this guide 8
2 Conceptual framework for VfM assessment 9
2.1 Explicit evaluative reasoning 9
2.2 VfM criteria 10
2.3 DFID’s VfM criteria: the Four E’s 10
2.4 Limitations of the Four E’s 12
2.5 Defining criteria and standards for the Four E’s 13
2.6 Mixed methods evidence 15
2.7 Economic analysis 15
2.8 Complexity and emergent strategy 17
2.9 Perspective matters in VfM 19
2.10 Integration with M&E frameworks 19
2.11 Timing of VfM assessments 20
2.12 Level of effort required 20
3 Designing and implementing a VfM framework 21
3.1 Step 1: theory of change 21
3.2 Steps 2 and 3: VfM criteria and standards 22
3.3 Step 4: identifying evidence required 26
3.4 Step 5: gathering evidence 26
3.5 Steps 6–7: analysis, synthesis, and judgements 27
3.6 Step 8: reporting 29
Bibliography 30
Contact us

Review by Dr E. Jane Davidson, author of Evaluation Methodology Basics (Sage, 2005) and Director of Real Evaluation LLC, Seattle

Finally, an approach to Value for Money that breaks free of the “here’s the formula” approach and instead emphasises the importance of thoughtful and well-evidenced evaluative reasoning. Combining an equity lens with insights and perspectives from diverse stakeholders helps us understand the value of different constellations of outcomes relative to the efforts and investments required to achieve them. This step-by-step guide helps decision makers figure out how to answer the VfM question in an intelligent way when some of the most valuable outcomes may be the hardest to measure – as they so often are.

 

by Matthew J. Salganik, Princeton University Press, 2017

Very positive reviews by…

• Times Higher Education, Farida Vis, 11 January 2011
• Science Magazine, David Lazar, 2 January 2018
• Princeton University Press Blog, author interview, 7 December 2017

Selected quotes:

“Overall, the book relies on a repeated narrative device, imagining how a social scientist and a data scientist might approach the same research opportunity. Salganik suggests that where data scientists are glass-half-full people and see opportunities, social scientists are quicker to highlight problems (the glass-half-empty camp). He is also upfront about how he has chosen to write the book, adopting the more optimistic view of the data scientist, while holding on to the caution expressed by social scientists”

“Salganik argues that data scientists most often work with “readymades”, social scientists with “custommades”, illustrating the point through art: data scientists are more like Marcel Duchamp, using existing objects to make art; meanwhile, social scientists operate in the custom-made style of Michelangelo, which offers a neat fit between research questions and data, but does not scale well. The book is thus a call to arms, to encourage more interdisciplinary research and for both sides to see the potential merits and drawbacks of each approach. It will be particularly welcome to researchers who have already started to think along similar lines, of which I suspect there are many”

Illustrates important ideas with examples of outstanding research

Combines ideas from social science and data science in an accessible style and without jargon

Goes beyond the analysis of “found” data to discuss the collection of “designed” data such as surveys, experiments, and mass collaboration

Features an entire chapter on ethics

Includes extensive suggestions for further reading and activities for the classroom or self-study

Matthew J. Salganik is professor of sociology at Princeton University, where he is also affiliated with the Center for Information Technology Policy and the Center for Statistics and Machine Learning. His research has been funded by Microsoft, Facebook, and Google, and has been featured on NPR and in such publications as the New Yorker, the New York Times, and the Wall Street Journal.

Contents

Preface
1 Introduction
2 Observing Behavior
3 Asking Questions
4 Running Experiments
5 Creating Mass Collaboration
6 Ethics
7 The Future
Acknowledgments
References
Index

More detailed contents page available via Amazon Look Inside

PS: See also this Vimeo video presentation by Salganik: Wiki Surveys – Open and Quantifiable Social Data Collection plus this PLOS paper on the same topic.

 

Virginia Eubanks, (2018), New York, NY: St. Martin’s Press

Unfortunately, a contents list does not seem to be available online.  But here is a  lengthy excerpt from the book.

And here is a YouTube interview with the author, a University of Albany political scientist Virginia Eubanks discusses her new book “Automating Inequality: How High Tech Tools Profile, Police, and Punish the Poor.” (Taped: 12/05/2017)

 

by  Howard White David A. Raitzer. Published by Asian Development Bank. 2017. Available as a pdf (3.12Mb)

The publisher says “This book offers guidance on the principles, methods, and practice of impact evaluation. It contains material for a range of audiences, from those who may use or manage impact evaluations to applied researchers”

“Impact evaluation is an empirical approach to estimating the causal effects of interventions, in terms of both magnitude and statistical significance. Expanded use of impact evaluation techniques is critical to rigorously derive knowledge from development operations and for development investments and policies to become more evidence-based and effective. To help backstop more use of impact evaluation approaches, this book introduces core concepts, methods, and considerations for planning, designing, managing, and implementing impact evaluation, supplemented by examples. The topics covered range from impact evaluation purposes to basic principles, specific methodologies, and guidance on field implementation. It has materials for a range of audiences, from those who are interested in understanding evidence on “what works” in development, to those who will contribute to expanding the evidence base as applied researchers.”

The Tyranny of Metrics, by Jerry Z Muller, Princeton University Press, RRP£19.95/ $24.95, 240 pages

See Tim Harford’s review of this book in the Financial Times, 24, January 2018

Some quotes: Muller shows that metrics are often used as a substitute for relevant experience, by managers with generic rather than specific expertise. Muller does not claim that metrics are always useless, but that we expect too much from them as a tool of management. ….

The Tyranny of Metrics does us a service in briskly pulling together parallel arguments from economics, management science, philosophy and psychology along with examples from education, policing, medicine, business and the military.

 In an excellent final chapter, Muller summarises his argument thus: “measurement is not an alternative to judgement: measurement demands judgement: judgement about whether to measure, what to measure, how to evaluate the significance of what’s been measured, whether rewards and penalties will be attached to the results, and to whom to make the measurements available”. 

 The book does not engage seriously enough with the possibility that the advantages of metric-driven accountability might outweigh the undoubted downsides. Tellingly, Muller complains of a university ratings metric that rewards high graduation rates, access for disadvantaged students, and low costs. He says these requirements are “mutually exclusive”, but they are not. They are in tension with each other,

Nor does this book reckon with evidence that mechanical statistical predictions often beat the subjective judgment of experts.

…and perhaps most curiously, there is no discussion of computers, cheap sensors, or big data. In this respect, at least, the book could have been written in the 1980s.

To be published in Jan 2018. SECOND EDITION. Published by Sage
Stephen P Borgatti – University of Kentucky, USA
Martin G Everett – Manchester University, UK
Jeffrey C Johnson – University of Florida, USA

Publishers blurb: “Designed to walk beginners through core aspects of collecting, visualizing, analyzing, and interpreting social network data, this book will get you up-to-speed on the theory and skills you need to conduct social network analysis. Using simple language and equations, the authors provide expert, clear insight into every step of the research process—including basic maths principles—without making assumptions about what you know. With a particular focus on NetDraw and UCINET, the book introduces relevant software tools step-by-step in an easy to follow way.

In addition to the fundamentals of network analysis and the research process, this new Second Edition focuses on:

• Digital data and social networks like Twitter
• Statistical models to use in SNA, like QAP and ERGM
• The structure and centrality of networks
• Methods for cohesive subgroups/community detection
• Supported by new chapter exercises, a glossary, and a fully updated companion website, this text is the perfect student-friendly introduction to social network analysis.”

Detailed contents list here

 

by Cass R. Sunstein, Cambridge University Press, 2016

Contents:

1. The age of behavioral science;
2. Choice and its architecture;
3. ‘As judged by themselves’;
4. Values;
5. Fifty shades of manipulation;
6. Do people like nudges? Empirical findings;
7. Green by default? Ethical challenges for environmental protection;
8. Mandates – a very brief recapitulation;
Appendix A. American attitudes toward thirty-four nudges;
Appendix B. Survey questions;
Appendix C. Executive Order 13707: using behavioral science insights to better serve the American people;

Amazon blurb: “In recent years, ‘nudge units’ or ‘behavioral insights teams’ have been created in the United States, the United Kingdom, Germany, and other nations. All over the world, public officials are using the behavioral sciences to protect the environment, promote employment and economic growth, reduce poverty, and increase national security. In this book, Cass R. Sunstein, the eminent legal scholar and best-selling co-author of Nudge (2008), breaks new ground with a deep yet highly readable investigation into the ethical issues surrounding nudges, choice architecture, and mandates, addressing such issues as welfare, autonomy, self-government, dignity, manipulation, and the constraints and responsibilities of an ethical state. Complementing the ethical discussion, The Ethics of Influence: Government in the Age of Behavioral Science contains a wealth of new data on people’s attitudes towards a broad range of nudges, choice architecture, and mandates.

Book Review by Roger Frantz (pdf)