Last week the Outcome Mapping Learning Community hosted the third OM community webinar: Systems Concepts and Outcome Mapping.
We were fortunate enough to have Richard Hummelbrunner and Bob Williams presenting some of the ideas from their new book: Systems Concepts in Action: A Practitioners Toolkit – which features a chapter on Outcome Mapping (you can read an article they wrote in our previous newsletter). Their presentation introduced three systems concepts – inter-relationships, perspectives and boundaries – and discussed OM as a systems tool in relation to these. They also provided some interesting reflections on OM, some of which I’ll mention here but the rest you can read in their presentation, which can be downloaded here.
Summary
Three systems concepts and how OM deals with them:
- OM deals with inter-relationships very well
- Perspectives take into account three different notions: stakes, stakeholders and framings. OM deals with stakeholders very clearly but doesn’t distinguish the stakes or framings that are held. E.g. different stakeholders (or in the case of OM, BPs) can have the same stakes (motivations, agenda’s or drivers) but they may also have conflicting stakes. Analysing the stakes is an important systems tool. Framings are important because the same situation can be understood in different ways depending on the angle chosen – this will depend on the stakes and stakeholders.
- Boundaries are everywhere; every decision we make defines boundaries and these have important consequences. OM tends to deal with boundaries in terms of influence of stakeholders (cf boundary partners) and could benefit by taking a broader view about what is valued and what is not, who’s in and who’s not.
Advantages of OM:
- good at clarifying stakeholder roles
- especially useful for interventions where capacity building plays a major role
- well suited for interventions involving complex change processes
- integrates learning and reflection in the design of interventions
Disadvantages:
- demands for data collection (journals) can be challenging
- focus on boundary partners neglects relevance of other actors
- difficult and time consuming to accommodate, especially for newcomers
Questions
We were able to take a number of questions in the webinar but there were some that we had to leave unanswered due to time. Here’s a quick summary of some of the questions that we addressed, and below that is a list of unanswered questions that Bob and Richard have agreed to follow up on.
Q (Ricardo Wilson-Grau): How flexible is systems thinking? Can we just apply one of the concepts are still be confident we are working systemically?
A: No, they are a set. It’s the way systems methods deal with these concepts make them distinctive and recognisable. There is not always a balance in the way a method deals with these but they will touch on each concept.
Q (Samuel Sanders): When you say system do you mean LFA (logframes)?
A: No. Logframes are a way of expressing the intentions of a project/programme. When assessing the tool against the three concepts, it deals with interrelationships somewhat, perspectives a little and boundaries not at all.
Q (George de Gooijer): If you limit frames here to stakes and stakeholders, I think it is too limited. Where does ‘history’ come in to this (the change in behaviour sought by OM is very much influenced by inferences/prejudices that are the consequence of the history of a situation/person/society).
A: Stakes can be seen as a product of history. This isn’t explicitly acknowledged- possibly a weakness in this model. There are methods that explore historical issues specifically: cultural historical activity theory – a way of understand how contradictions between the past and present affect motivations towards a goal.
Q (Samuel Sanders): Why hasn’t om yet been widely used in M&E? And are there anything that one must especially keep in mind when using OM to follow up results and coming to conclusions on effectiveness? (in response to a comment by Ricardo that we far less about how OM is used for M&E than for intentional design)
A: OM has been seen as a planning instrument as a substitute for the logframe. There’s less emphasis on M&E, possible for practical reasons – donors fund plans.
Q (Douglass Orr): Are the presenters aware of any documented applications of systems concepts for analysing power relationships ? Could they recommend any ? Thanks.
A: Critical systems heuristics and the work of Martin Reynolds comes to mind. This area of systems thinking reframes power from a position of ethics (read C. West Churchman).
Q (Geir Sundet): Outcome Mapping tends, in my experience, to focus on changes in behaviour. Does this not make it much more suitable for tracking changes in complex systems? Meaning that the conventional LF approach adopts a much more linear input, output, outcome approach, which is less likely to capture unexpected (or partly expected) impacts/outcome.
A: Yes, this is one of the very useful parts of OM. Unexpected outcomes are a challenge for OM as with most other PME approaches – intentional design is about clarifying expected changes, so how are unexpected changes tracked? When we set up an intentional design we are drawing boundaries about what will be monitored and what won’t – this will naturally limit our ability to identify unexpected changes. But this is a work in progress across the systems field as well as the evaluation field.
Gier followed up: I find that the very exercise of tracking and observing behaviour focuses attention on the political and power relations. This has not always been routinely done, and organisations have often shied away from attempting political analysis and interpretations.
Q (Nasim Kung’u): Can Progress Markers be used to identify unexpected behavioural change?
A: Progress Markers are an instrument so it would depend on how the behavioural change is being observed and the kind of information being collected. There are methods (such as SenseMaker) that collect large quantities of narrative data so that early signs of unexpected change can be detected. These are just starting to be explored within international development. In complex situations, it is difficult to write progress markers because it’s impossible to predict the outcomes ahead of time. In these situations it’s much more important to have real time monitoring to record changes in relationships, actions, policies and practices of BPs and others who are unexpectedly influenced. A book by Jonny Morel ‘Evaluation in the Face of Uncertainty: Anticipating Surprise and Responding to the inevitable’ is recommended.
Q (Emebet Ayelign): Can OM stand with other planning tools?
A: Yes, OM can be blended with other methods such as scenario planning or assumption based planning – particularly in situations of uncertainty.
Unanswered questions
1. George de Gooijer asks: Is the second point (in Richard’s list of disadvantages of OM: focus on boundary partners neglects relevance of other actors) in fact a call to include the ‘principal-agent’ problem into the OM approach, where OM has in fact excluded the problem by limiting to boundary partners?
2.Another from George: In addition to principal agent: maybe the pressure coming from pushing for ‘boundary partners’ is 2-fold: pressure to get the right partners during the design-phase, but also pressure to ‘nest’ the method in the rest of the process you want to influence (we could borrow from the Viable Systems Model??)
3. Bob Sutton asks: Are case studies the best way to improve and promote OM to often sceptical or unfamiliar project partners? Or do you have other suggestions for new users to promote the OM approach?
4. Wessel van den Berg asks: Does anyone know of a particular analysis of OM according to the qualities of a complex system a la Morin such as unpredictability, small causes large effects, difficulty with models etc.
I leave these with Bob and Richard to respond to.