Pilot Performance During Multiple Failures: An Empirical Study of Different Warning Systems

Abstract

Dynamic fault management-that is, dealing with a cascade of failures while maintaining process integrity-is a dominant human task in various transportation modes (e.g., commercial aviation, shipping). The way in which a warning system represents failures and the way in which the system contributes to failure management jointly determine the amount and kind of cognitive work in which the human has to engage to manage multiple failures. In this study pilot performance on 4 different commercial aviation warning systems was tested by measuring time and error rates in identifying root causes in a cascade of failures. All systems tested represent failures in the same basic way (a message list) but differ in the kind of contribution they make to the failure management task; for example, by sorting failures, prioritizing them, selecting only some failures for presentation, guiding the pilot on what to do next, or showing the pilot which systems are still operational. Human performance benefits accrued in systems that (a) provided guidance on what to do next and (b) showed which systems were still operational. These findings are consistent with the cognitive demands of dynamic fault management and carry important messages for how those demands should be supported. The results suggest that rather than automating even more of the human role in fault management to minimize error counts, attention should be paid to the kinds of referents and representations that are most useful in informing the operator of what is going on in the underlying process and how best to cope with it.