ParEvo – a web assisted participatory scenario planning process

The purpose of this page

…is to record some ongoing reflections on my experience of running two pre-tests of ParEvo carried out in late 2018 and early 2019.

Participants and others are encouraged to add their own comments, by using the Comment facility at the bottom of this page

Two pre-tests are underway

  • One involves 11 participants developing a scenario involving the establishment of an MSC (Most Significant Change) process in a development programme in Nigeria. These volunteers were found via the MSC email list. They came from 7 countries and 64% were women.
  • The other involves 11 participants developing a Brexit scenario following Britain failing to reach an agreement with the EU by March 2019. These participants were found via the MandE NEWS email list. They came from 9 countries and 46% were women.

For more background (especially if you have not been participating) see this 2008 post on the process design and this 2019 Conference abstract talking about these pre-tests

Reflections so far

Issues arising…

  1. How many participants should there be?
    • In the current pre-tests, I have limited the number to around 10. My concern is that with larger numbers there will be too many story segments (and their storylines) for people to scan and make a single preferred selection. But improved methods of visualising the text contributions may help overcome this limitation. Another option is to allow/encourage individual participants to represent teams of people, e.g. different stakeholder groups
  2. Do the same participants need to be involved in each iteration of the process?
    1. My initial concern is that not doing so would make some of the follow up quantitative analysis more difficult, but I am not so concerned about that now, its a manageable problem. On the other hand, it is likely that some people will have to drop out mid-process, and ideally, they could be replaced by others, thus maintaining the diversity of storylines.
  3. How do you select an appropriate topic for a scenario planning exercise?
    1. Ideally, it would be a topic that was of interest to all the participants and one which they felt some confidence in talking about, even if only in terms of imagined futures. One pre-test topic, the use of MSC in Nigeria, was within these bounds. But the other was more debatable: the fate of the UK after no resolution of BREXIT terms by 29th March 2019
  4. How should you solicit responses from participants?
    1. I started by sending a standard email to all the (MSC scenario) participants, but this has been cumbersome and has risks. it is easy to lose track of who contributed what text, to add to what existing storyline. I am now using two-part single question survey via SurveyMonkey. This enables me to keep a mistake-free record of who contributed what to what, and who has responded and who has not.
  5. How should you represent and share participants responses?
    1. This has been done in two forms. One is a tree diagram, showing all storylines, where participants can mouseover nodes to immediately see each text segment. Or they can click on each node to go to a separate web page and see complete storylines. These are both laborious to construct, but hopefully will soon be simplified and automated via some tech support which is now under discussion
  6. Should all contributions be anonymous?
    1. There are two types of contributions: (a) the storyline segments contributed during each iteration of the process, (b) Comments made on these contributions, that can be enabled on the blog page that hosts each full storyline to date. This second type was an afterthought, whereas the first is central to the process.
    2. The first process of contributing to storylines designed to make authorship anonymous, so people would focus on the contents.  I think this remains a good feature.
    3. The second process of allowing people to comment has pros and cons. The advantage is that it can enrich the discussion process, providing a meta-level to the main discussion which is the storyline development. The risk, however, is that if the comments are not enabled to be anonymous then a careful reader of the comments can sometimes work out who made which storyline contributions. I have tried to make comments anonymous but they still seem to reveal the identity of the person making the comment. This may be resolvable.
  7. How many iterations should be completed?
    1. It has been suggested that participants should know this in advance, so that their story segments don’t leap in the future too quickly, or the reverse, progress the story too slowly. With the Brexit scenario pre-test I am inclined to agree. It might help to saing at the beginning that there will be 5 iterations, ending in the year 2025. With the MSC scenario pre-test I am less certain, it seems to be moving on at a pace I would not have predicted
  8. What limits should there be on the length of the text that participants submit?
    1. I have wobbled on this issue, ranging from 100-word limits to 50-word limits to no voiced limits at all. Perhaps when people select which storyline to continue the length of the previous contributions will be something they take into account? I would like to hear participants views on this issue. Should there be word limits, and if so, what sort of limit?
  9. What sort of editorial intervention should there be by the facilitator, if any?
    1. I have been tempted, more than once, to ask some participants to reword  and revise their contribution. I now limit myself to very basic spelling corrections, checked with the participant, if necessary. I was worried that some participants have a limited grasp of the scenario topic, but now think that just has to be part of the reality, some people have little to go on when anticipating specific the future, and others may have “completely the wrong idea”, according to others. As the facilitator, I now think I need to stand back and let things run.
  10. What other issues are arising and need discussion?
    1. Over to you…

 

5 thoughts on “ParEvo – a web assisted participatory scenario planning process”

  1. To prune or not to prune …
    Tree growers — fruit trees, bonsais — often prune their trees to get a more productive or a more beautiful tree. Should the same be acceptable for a “scenario tree”? Is it best to let the stories develop completely ‘wild’? Or should the gardener have the power to intervene and cut off branches which are less likely to develop nicely, or those which will be less able to carry the fruit that tree is expected to produce in future? It could e.g. be that two branches are shooting out close to each other (i.e. rather similar scenarios), and instead of spending energy growing both of them in parallel, one of the two could be pruned so that the other one can grow more vigorously. A pruning like that is probably best done early on.
    Some pruning could also happen later on, when there is a branch that started to develop, but it did not branch off any further. This latter pruning is perhaps more for aesthetics.

  2. Limit on length of submitted text.
    I prefer the “… up to two sentences” stipulation, rather than a 50- or 100-word limit. I also found it easier to develop a continuation on a longer storyline than a shorter one, probably because a longer one may already have more specificity and detail.

  3. How should storylines be allowed to develop? After the second iteration should the third iteration only continue on the Storylines 1.x that had additional branches? Or could a participant’s third iteration be another possible development on a Storyline 1.x (but a different one s/he had picked before)? The analogy is that in a tree the new shoots are not always at the outer end of the tree; there can be new sprouts on some older branches as well.
    It would result in a “fuller” scenario tree, but it would also mean that some storylines are more extended than others.

  4. Does anyone have sufficient experience of the Delphi Method to add to any of these comments. Delphi isn’t quite so scenario based, and other aspects are radically different, but some of the rule structures might be relevant even if the actual rule is different.

  5. Thanks Bob.

    There are a series of papers that review the literature on scenario planning, which I am accumulating and will make available in the form of a Zotero bibliography online (no subs or sign-ups needed)

Comments?