• Boele, Richard, and Christine Crispin. 2013. “What Direction for Human Rights Impact Assessments?” Impact Assessment and Project Appraisal 31 (2): 128–134. doi:10.1080/14615517.2013.771005.

• Collins, Nina, and Alan Woodley. 2013. “Social Water Assessment Protocol: a Step Towards Connecting Mining, Water and Human Rights.” Impact Assessment and Project Appraisal 31 (2): 158–167. doi:10.1080/14615517.2013.774717.

• Graetz, Geordan, and Daniel M. Franks. 2013. “Incorporating Human Rights into the Corporate Domain: Due Diligence, Impact Assessment and Integrated Risk Management.” Impact Assessment and Project Appraisal 31 (2): 97–106. doi:10.1080/14615517.2013.771006.

• Hanna, Philippe, and Frank Vanclay. 2013. “Human Rights, Indigenous Peoples and the Concept of Free, Prior and Informed Consent.” Impact Assessment and Project Appraisal 31 (2): 146–157. doi:10.1080/14615517.2013.780373.

• Harrison, James. 2013. “Establishing a Meaningful Human Rights Due Diligence Process for Corporations: Learning from Experience of Human Rights Impact Assessment.” Impact Assessment and Project Appraisal 31 (2): 107–117. doi:10.1080/14615517.2013.774718.

• Kemp, Deanna, and Frank Vanclay. 2013a. “Human Rights and Impact Assessment: Clarifying the Connections in Practice.” Impact Assessment and Project Appraisal 31 (2): 86–96. doi:10.1080/14615517.2013.782978.

• ———. 2013b. “Human Rights and Impact Assessment.” Impact Assessment and Project Appraisal 31 (2): 85–85. doi:10.1080/14615517.2013.791507.

• Sauer, Arn Thorben, and Aranka Podhora. 2013. “Sexual Orientation and Gender Identity in Human Rights Impact Assessment.” Impact Assessment and Project Appraisal 31 (2): 135–145. doi:10.1080/14615517.2013.791416.

• Watson, Gabrielle, Irit Tamir, and Brianna Kemp. 2013. “Human Rights Impact Assessment in Practice: Oxfam’s Application of a Community-based Approach.” Impact Assessment and Project Appraisal 31 (2): 118–127. doi:10.1080/14615517.2013.771007.

See also Gabrielle Watson’s related blog posting: Trust but verify: Companies assessing their own impacts on human rights? Oxfam’s experience supporting communities to conduct human rights impact assessments

And docs mentioned in her post:

• the United Nations Guiding Principles on Business and Human Rights in 2011
• Oxfam’s community-based Human Rights Impact Assessment (HRIA) tool, Getting it Right,The tool was first tested in the Philippines, Tibet, the Democratic Republic of Congo, Argentina and Peru, and then improved. In 2010 and 2011, Oxfam supported local partner organizations to conduct community-based HRIAs with tobacco farmworkers in North Carolina and with mining-affected communities in Bolivia. In our experience, community-based HRIAs have: (1) built human rights awareness among community members, (2) helped initiate constructive engagement when companies have previously ignored community concerns, and (3) led to concrete actions by companies to address concerns.

“The Evaluating Humanitarian Action Guide supports evaluation specialists and non-specialists in every stage of an evaluation, from initial decision to final dissemination.

Here are six reasons we think it’s time for a comprehensive EHA guide:
1. Official donor assistance for humanitarian action has increased  nearly six times in real terms from 1990 to 2011.
2. More interest and investment in evaluations as concerns are raised about effectiveness of development aid and humanitarian relief.
3. A critical mass of collective knowledge now exists to build on – ALNAP’s evaluation database alone contains over 500 covering the last decade.
4. Commissioning of evaluations has shifted from agency headquarters  to field-based staff as agencies decentralise – yet field-based managers often have little experience in planning and managing evaluations, especially EHA.
5. Little evidence that evaluation results lead to change of, or reflection on, policy and practice – better designed evaluations could provide more compelling evidence for policy change and promote utilisation.
6. The demand for guidance on EHA is growing – a Humanitarian Practice  Network member survey in 2009 found that the number one guidance material requests were for EHA.

This ALNAP guide provides practical and comprehensive guidance and good practice examples to those planning, designing, carrying  out, and using evaluations of humanitarian action.
The focus is on utilisation: to encourage you to consider how to ensure from the outset that an evaluation will be used.
This guide attempts to support high-quality evaluations that contribute to improved performance by providing the best evidence possible of what is working well, what is not, and why. The ultimate goal is to better meet the needs of people affected by humanitarian crises, who will be referred to throughout this guide as the affected population.”

These notes speak to “one case – one record” data situations, which are typical of most surveys and assessments. For the special challenges that “one case – many records” datasets offer, see an example further down on the same page.”

[from the IFAD website] “The Independent Office of Evaluation of IFAD (IOE) is making publicly available all the ratings on the performance of IFAD-supported operations evaluated since 2002.  As such, IOE joins the few development organizations that currently make such data available to the public at large. The broader aim of disclosing such evaluation data is to further strengthen organizational accountability and transparency (in line with IFAD’s Disclosure and Evaluation Policies), as well as enable others interested (including researches and academics) to conduct their own analysis based on IOE data.

All evaluation ratings may be seen in the Excel database. At the moment, the database contains ratings of 170 projects evaluated by IOE. These ratings also provide the foundation for preparing IOE’s flagship report, the Annual Report on Results and Impact of IFAD operations (ARRI).

As in the past, IOE will continue to update the database annually by including ratings from new independent evaluations conducted each year based on the methodology captured in the IFAD Evaluation Manual. It might be useful to underline that IOE uses a six-point rating scale (where 6 is the highest score and 1 the lowest) to assess the performance of IFAD-funded operations across a series of internationally recognised evaluation criteria (e.g., relevance, effectiveness, efficiency, rural poverty impact, sustainability, gender, and others).

Moreover, in 2006, IOE’s project evaluation ratings criteria were harmonized with those of IFAD’s operations, to ensure greater consistency between independent and self-evaluation data (Agreement between PMD and IOE on the Harmonization of Self-Evaluation and Independent Evaluation Systems of IFAD). The Harmonization agreement was further enhanced in 2011, following the Peer Review of IFAD’s Office of Evaluation and Evaluation Function. The aforementioned agreements also allow to determine any ‘disconnect’ in the reporting of project performance respectively by IOE and IFAD management.”

Question : What recent work has been done on assessing the quality and limitations of using perception surveys in fragile and conflict affected states?
Contents (10 pages in all)
1. Overview
2. Strengths of perception surveys
3. Limitations of perception surveys
4. Methodological approaches to ensure quality in perception surveys
5. References

Last year Lant Pritchett, Salimah Samji and Jeffrey Hammer wrote this interesting (if at times difficult to read) paper “It’s All About MeE: Using Structured Experiential Learning (‘e’) to Crawl the Design Space“(the abstract is reproduced below). In the course of that paper they argued that CCT programs are not as simple as they might seem. Looking at three real life examples they identified at least 10 different characteristics of CCTs that need to be specified correctly in order for them to work as expected. Some of these involve binary choices (whether to do x or y) and some involve tuning of a numerical variable. This means there were at least 2 to the power of 10 i.e. 1024 different possible designs. They also pointed out that while changes to some of these characteristics make only a small difference to the results achieved others, including some binary choices, can make quite major differences. In other words, overall it may well be a rugged rather than a smooth design space. The question then occurs, how well are RCTs suited to exploring such spaces?

Today the World Bank Development Blog posted an interesting confirmation of the point made in Pritchett et al paper, in a blog posting titled:  Defining Conditional Cash Transfer Programs: An Unconditional Mess. Basically they are in effect pointing out that the design space is even way more complicated than Princhett et al describe!. They conclude

“So, if you’re a donor or a policymaker, it is important not to frame your question to be about the relative effectiveness of “conditional” vs. “unconditional” cash transfer programs: the line between these concepts is too blurry. It turns out that your question needs to be much more precise than that. It is better to define the feasible range of options available to you first (politically, ethically, etc.), and then go after evidence of relative effectiveness of design options along the continuum from a pure UCT to a heavy-handed CCT. Alas, that evidence is the subject of another post…“

So stay tuned fore their next installment. Of course you could quibble with the fact that even this conclusion is a bit optimistic, in that it talks about a a continuum of design options, when in fact it is multi-dimensional space  with both smooth and rugged bits

PS: Here is the abstract for the Printchett paper:

Executive Order — Making Open and Machine Readable the New Default for Government Information

EXECUTIVE ORDER

- – - – - – -

MAKING OPEN AND MACHINE READABLE THE NEW DEFAULT
FOR GOVERNMENT INFORMATION

By the authority vested in me as President by the Constitution and the laws of the United States of America, it is hereby ordered as follows:

Section 1. General Principles. Openness in government strengthens our democracy, promotes the delivery of efficient and effective services to the public, and contributes to economic growth. As one vital benefit of open government, making information resources easy to find, accessible, and usable can fuel entrepreneurship, innovation, and scientific discovery that improves Americans’ lives and contributes significantly to job creation.

Decades ago, the U.S. Government made both weather data and the Global Positioning System freely available. Since that time, American entrepreneurs and innovators have utilized these resources to create navigation systems, weather newscasts and warning systems, location-based applications, precision farming tools, and much more, improving Americans’ lives in countless ways and leading to economic growth and job creation. In recent years, thousands of Government data resources across fields such as health and medicine, education, energy, public safety, global development, and finance have been posted in machine-readable form for free public use on Data.gov. Entrepreneurs and innovators have continued to develop a vast range of useful new products and businesses using these public information resources, creating good jobs in the process.

To promote continued job growth, Government efficiency, and the social good that can be gained from opening Government data to the public, the default state of new and modernized Government information resources shall be open and machine readable. Government information shall be managed as an asset throughout its life cycle to promote interoperability and openness, and, wherever possible and legally permissible, to ensure that data are released to the public in ways that make the data easy to find, accessible, and usable. In making this the new default state, executive departments and agencies (agencies) shall ensure that they safeguard individual privacy, confidentiality, and national security.

Sec. 2. Open Data Policy. (a) The Director of the Office of Management and Budget (OMB), in consultation with the Chief Information Officer (CIO), Chief Technology Officer (CTO), and Administrator of the Office of Information and Regulatory Affairs (OIRA), shall issue an Open Data Policy to advance the
management of Government information as an asset, consistent with my memorandum of January 21, 2009 (Transparency and Open Government), OMB Memorandum M-10-06 (Open Government Directive), OMB and National Archives and Records Administration Memorandum M-12-18 (Managing Government Records Directive), the Office of Science and Technology Policy Memorandum of February 22, 2013 (Increasing Access to the Results of Federally Funded Scientific Research), and the CIO’s strategy entitled “Digital Government: Building a 21st Century Platform to Better Serve the American People.” The Open Data Policy shall be updated as needed.

(b) Agencies shall implement the requirements of the Open Data Policy and shall adhere to the deadlines for specific actions specified therein. When implementing the Open Data Policy, agencies shall incorporate a full analysis of privacy, confidentiality, and security risks into each stage of the information lifecycle to identify information that should not be released. These review processes should be overseen by the senior agency official for privacy. It is vital that agencies not release information if doing so would violate any law or policy, or jeopardize privacy, confidentiality, or national security.

Sec. 3. Implementation of the Open Data Policy. To facilitate effective Government-wide implementation of the Open Data Policy, I direct the following:

(a) Within 30 days of the issuance of the Open Data Policy, the CIO and CTO shall publish an open online repository of tools and best practices to assist agencies in integrating the Open Data Policy into their operations in furtherance of their missions. The CIO and CTO shall regularly update this online repository as needed to ensure it remains a resource to facilitate the adoption of open data practices.

(b) Within 90 days of the issuance of the Open Data Policy, the Administrator for Federal Procurement Policy, Controller of the Office of Federal Financial Management, CIO, and Administrator of OIRA shall work with the Chief Acquisition Officers Council, Chief Financial Officers Council, Chief Information Officers Council, and Federal Records Council to identify and initiate implementation of measures to support the integration of the Open Data Policy requirements into Federal acquisition and grant-making processes. Such efforts may include developing sample requirements language, grant and contract language, and workforce tools for agency acquisition, grant, and information management and technology professionals.

(c) Within 90 days of the date of this order, the Chief Performance Officer (CPO) shall work with the President’s Management Council to establish a Cross-Agency Priority (CAP) Goal to track implementation of the Open Data Policy. The CPO shall work with agencies to set incremental performance goals, ensuring they have metrics and milestones in place to monitor advancement toward the CAP Goal. Progress on these goals shall be analyzed and reviewed by agency leadership, pursuant to the GPRA Modernization Act of 2010 (Public Law 111-352).

(d) Within 180 days of the date of this order, agencies shall report progress on the implementation of the CAP Goal to the CPO. Thereafter, agencies shall report progress quarterly, and as appropriate.

Sec. 4. General Provisions. (a) Nothing in this order shall be construed to impair or otherwise affect:
(i) the authority granted by law to an executive department, agency, or the head thereof; or

(ii) the functions of the Director of OMB relating to budgetary, administrative, or legislative proposals.

(b) This order shall be implemented consistent with applicable law and subject to the availability of appropriations.

(c) This order is not intended to, and does not, create any right or benefit, substantive or procedural, enforceable at law or in equity by any party against the United States, its departments, agencies, or entities, its officers, employees, or agents, or any other person.

(d) Nothing in this order shall compel or authorize the disclosure of privileged information, law enforcement information, national security information, personal information, or information the disclosure of which is prohibited by law.

(e) Independent agencies are requested to adhere to this order.

BARACK OBAMA

• American Evaluation Association: Coffee Break Demonstrations are 20 minute long webinars designed to introduce audience members to new tools, techniques, and strategies in the field of evaluation.
  • Better Evaluation Series (2013) 8 webinars starting May 2013
  • Archives: 135 webinars!!
• INTERACTION: Impact Evaluation Guidance Note and Webinar Series: 8 webinars covering Introduction to Impact Evaluation, Linking Monitoring and Evaluation to Impact Evaluation, Introduction to Mixed Methods in Impact Evaluation, Use of Impact Evaluation Results
  
• Measure Evaluation webinars:     20 webinars since Jan 2012
• Claremont Evaluation Center Webinar Series  “The Claremont Evaluation Center is pleased to offer a series of webinars on the discipline and profession of evaluation.  This series is free and available to anyone across the globe with an internet connection.”
• MY M&E website: Webinars on Equity-focused evaluations (17 webinars), IOCE webinar series on evaluation associations, Emerging practices in development evaluation (6 webinars), Developing capacities for country M&E systems (16 webinars), Country-led M&E Systems (6 webinars)

Plus some guidance on developing and evaluating webinars

• How to think about evaluating a webinar (PennState Cooperative Extension, 2009)
• Best Practices for Making Your Webinar a Success: Post-Event Evaluation (Frost and Sullivan, 2010)

Independent Commission for Aid Impact, Wednesday 1 May 2013

 ”In our first report, ICAI’s Approach to Effectiveness and Value for Money,we set out an evaluation framework, consisting of 22 questions under 4 guiding criteria (objectives, delivery, impact and learning), to guide our lines of enquiry in reviews. In the light of our experience to date in carrying out our reports, we have reviewed this framework. The revised framework is available at this link: ICAI revised evaluation framework

We are now entering a period of consultation on the revised framework which will run until 24 May 2013. If you have any comments or views, please email enquiries@icai.independent.gov.uk  or post them to: The Secretariat, Independent Commission for Aid Impact, Dover House, 66 Whitehall, London SW1A 2AU”

Inspired by visitors” positive responses to the imaginative use of flow charts I have wondered how else Theories of Change could be described. The following thought came to me early this morning!

(with apologies to South Park)

See 6 Free Sites for Creating Your Own Comics, at Mashable, for links to stripgenerator and others

AN OFFER: I will give a £50 donation to Oxfam UK to the person who can come up with the best comic strip description of the Theory of Change of a real development project. Post your entry using Comment below, with a link to where the comic is and a link to where we can find a factual description of the project it represents. Your comic strip version can be as humourous (slapstick, farce, wit, irony, sarcasm, parody, gallows, juvenile, or…) or as serious as you like. It can be as long as you like and it does not need to be a simple sequence of panels, it could get way more complicated!

I will try set up an opinion poll so visitors can vote for the ones they like the most. The winning entry will definitely be posted as an  item here on MandE NEWS and be publicised via Twitter. The deadline: May 31st might do. One proviso: Nothing obscene or libelous