From using evidence to improving judgement: lessons from disaster response in Nepal

14 August 2018

Imagine you’re a mid-level policymaker in a government agency or a manager in an NGO. A major incident has just occurred. You have to drop everything you’re doing and shift all your attention and resources into understanding and managing the situation. This is what managers in Nepal were faced with when a powerful earthquake struck in April 2015. They had to work out how many people had been affected, where they were and what help they needed. They also needed to know if there were going to be aftershocks and how this would affect the delivery of relief. So what factors shaped managers’ actions in the immediate aftermath of the earthquake and what role was there for scientific evidence if any? Recent research reveals some useful insights.

Psychological and environmental factors that drive managers’ judgement

Managers were driven by considerable anxiety, a desire to do their job well but crucially a need to act quickly. This was helped by most managers being given a lot of autonomy. They subsequently improvised in uncertain conditions using their judgement and ‘gut instinct’. Judgement was in turn informed by a range of other factors which included those that were 1) psychological and 2) environmental.

Psychological factors included:

  • Values or principles: these included the need to keep people alive, to prioritise vulnerable groups and to work in partnership with government authorities;
  • Assumptions and ideas: including that the worst hit area would likely be at or close to the epicentre of the earthquake, that affected people would primarily require shelter in the event of an earthquake and that secondary effects of the earthquake such as landslides were unlikely to cause a lot of damage;
  • Experience: managers, especially those who had flown in from overseas as part of the ‘surge’, often had considerable experience in managing emergencies in other contexts;
  • Preparedness work: immediately after the earthquake, some managers focused initial relief and search and rescue efforts in the Kathmandu Valley which had been the focus of considerable preparedness work;
  • Estimates: some managers made estimates of the most affected populations and their locations by combining primary information from international sources, such as the USGS ShakeMap with secondary information from local sources, such as food and nutrition vulnerability;
  • Risk: many managers were reluctant to know if roads and trails were likely to be damaged by landslides as this may have prevented urgent supplies from being delivered.

Environmental factors included:

  • Historical: some managers chose to provide relief in areas where their organisation had existing programmes, supplied forms of relief they were familiar with and/or worked with target groups who they had previously worked with;
  • Competitive pressures: managers would often choose to work independently to distinguish themselves from other organisations, enabling them to provide a stronger account of their own work to regional and international headquarters and overseas funders;
  • Social and other forms of media: many managers received information about the earthquake, building damage and casualties through social media and traditional local media outlets. For instance, a lot of airtime was given to news about the destruction of historical sites in Kathmandu in the first few days of the response, which influenced early search and rescue efforts.
  • Formal guidance: this was issued by various authorities and groups including the formal humanitarian cluster system, government, donors and humanitarian agencies. Guidance included identification of the most severely affected areas and on occasion came in the form of standard operating procedures and;
  • Informal relationships: managers were influenced by other, less formal groups, comprising people from a variety of sectors including those from knowledge or science institutions. For instance government officials were able to acquire information about building damage from NGO practitioners who happened to be their friends.

Managers’ judgement wasn’t always sound. Mistakes were made (unintentionally). For instance, initial response efforts were skewed towards the epicentre and areas near Kathmandu. But the worst hit areas were not at or close to the epicentre or in the Kathmandu Valley but concentrated in a band running east to west with the hardest hit areas located to the north of the Kathmandu Valley. And with managers not recognising the potential for landslides to cause a lot of damage, parts of the country which were severely affected by landslides were left isolated, leaving thousands of households vulnerable despite response efforts elsewhere.

Expertise not evidence is more useful

Evidence, including scientific evidence, about the location and intensity of the earthquake; how the earthquake may have evolved over time; and the nature of any secondary hazards, such as landslides, could have improved managers’ judgement, but was not considered during the immediate response. The time needed to produce reasonably robust scientific evidence made promoting uptake in the timeframes impossible. But in any case, managers simply didn’t have the time (and perhaps skills) to pause and consider scientific evidence and its implications for their actions. What was considered useful by some managers, however, was expert advice from trusted and credible scientists in the hours and days after the earthquake had struck. In providing advice, these scientists were able to draw on a considerable experience and evidence, much of which they had acquired and generated during long careers.

How think tankers can help managers to improve their judgement

So, what does this mean for the role of scientists and think tankers during times of crisis? Well, they can play an important role in supporting managers to make better judgements. But this involves building longer term relationships with relevant managers well before disasters strike. For instance, during conversations with managers, scientists and think tankers can help them to reflect on the sorts of assumptions that managers tend to make or the ideas they have, that are likely to inform their actions. Scientists and think tankers can also help managers think about what they might do (differently) and how they can be more improvisational and creative given a specific disaster scenario (through for instance, scenario planning).

Finally, although I am arguing here for scientists and think tanks to help managers improve their judgement rather than their use of evidence, during emergencies or crises, I would argue that this is as relevant during non-crisis situations in policy areas such as health, education and employment. It is not unusual for policymakers’ attention to lurch from issue to issue unpredictably, often at short notice and where there simply isn’t the time to commission and/or consider robust evidence.

To learn more about the role of scientific evidence during the 2015 Nepal earthquake relief efforts, see the ODI research report.