Theme: Evaluation + Design
Website: http://www.eval.org/p/cm/ld/fid=451
Everything we evaluate is designed. Every evaluation we conduct is designed. Every report, graph, or figure we present is designed. In our profession, design and evaluation are woven together to support the same purpose—making the world a better place. By considering both as parts of a whole, we can advance that purpose.
This year, we will consider the integration of design and evaluation in three areas.
Program Design: We call the intentional actions that organizations take to improve the social or natural world programs. Today,organizations of every kind are designing and implementing programs. They include:
- nonprofit organizations serving the most critical needs of communities with education, human services, and health programs;
- government agencies piloting innovative solutions to longstanding social problems;
- philanthropists supporting collaborative programs designed to create collective impact;
- corporations implementing social responsibility programs that promote social equity and environmental sustainability;
- and public-private initiatives, such as pay-for-success funding, social enterprises, and impact investing, that leverage private capital for social good.
Every program is designed, yet the field of evaluation has not developed a systematic approach to designing programs. Should we? Can we? What would it look like? What role should evaluators play? How can evaluation be built into a program from the start? Can we design for sustainability? What does it mean for a program to have an exemplary design?
Evaluation Design: An evaluation design integrates evaluation theories, approaches, and methods to achieve a set of intended purposes in a specific context. An evaluation design encompasses more than research design. It also includes:
- the use of evaluation as a direct means of creating change, for example by working with stakeholders in ways that promote equity and empower communities;
- strategies for understanding, working with, and building consensus among stakeholders;
- the thoughtful combination of quantitative and qualitative methods;
- the application of evaluation for formative, summative, and developmental purposes;
- and the development of evaluative criteria that establish what matters, to whom, and why.
Evaluation design is dynamic, changing as programs develop and real-world challenges emerge. Given this complexity, what constitutes an exemplary evaluation design? Does flexibility and responsiveness come at the expense of quality, credibility, or usefulness? Can we learn to design evaluations that are faster, cheaper, and better? How do we balance the competing interests of stakeholders?
Information Design: Evaluators must communicate to diverse audiences about the complexities of programs and their impacts. They strive to be accurate, compelling, and clear. At the same time, they are constrained by the time, attention, and training of audiences. To be successful, evaluators must develop strong information design skills, such as:
- data visualization techniques that transform mind-boggling complexity into clear, meaningful images;
- real-time data displays that help managers make decisions better and more quickly;
- storytelling that speaks to the part of our brains hardwired by evolution to learn from narrative;
- and multi-faceted communication strategies that leverage social psychology and social media to promote appropriate and timely use.
Information design plays a central role in evaluation. But what is good information design? How should it be taught and learned? Can we be persuasive and accurate at the same time? Have evaluation reports become obsolete? What roles can online and interactive technologies play?
In 2016, professiona