Predicting the Validity of Expert Judgments in Assessing the Impact of Risk Mitigation Through Failure Prevention and Correction
Open Access
- 19 June 2020
- journal article
- research article
- Published by Wiley in Risk Analysis
- Vol. 40 (10), 1928-1943
- https://doi.org/10.1111/risa.13539
Abstract
Operational risk management of autonomous vehicles in extreme environments is heavily dependent on expert judgments and, in particular, judgments of the likelihood that a failure mitigation action, via correction and prevention, will annul the consequences of a specific fault. However, extant research has not examined the reliability of experts in estimating the probability of failure mitigation. For systems operations in extreme environments, the probability of failure mitigation is taken as a proxy of the probability of a fault not reoccurring. Using a priori expert judgments for an autonomous underwater vehicle mission in the Arctic and a posteriori mission field data, we subsequently developed a generalized linear model that enabled us to investigate this relationship. We found that the probability of failure mitigation alone cannot be used as a proxy for the probability of fault not reoccurring. We conclude that it is also essential to include the effort to implement the failure mitigation when estimating the probability of fault not reoccurring. The effort is the time taken by a person (measured in person‐months) to execute the task required to implement the fault correction action. We show that once a modicum of operational data is obtained, it is possible to define a generalized linear logistic model to estimate the probability a fault not reoccurring. We discuss how our findings are important to all autonomous vehicle operations and how similar operations can benefit from revising expert judgments of risk mitigation to take account of the effort required to reduce key risks.Keywords
Funding Information
- Natural and Environment Research Council (NE/I015647/1)
This publication has 42 references indexed in Scilit:
- The ‘Heuristics and Biases’ Bias in Expert ElicitationJournal of the Royal Statistical Society Series A: Statistics in Society, 2007
- Techniques for Deep Sea Near Bottom Survey Using an Autonomous Underwater VehicleThe International Journal of Robotics Research, 2007
- Measurement of Sea-ice draft using upward-looking ADCP on an autonomous underwater vehicleAnnals of Glaciology, 2006
- Seabed AUV offers new platform for high‐resolution imagingEos, 2004
- Quantitative risk-based requirements reasoningRequirements Engineering, 2003
- A review of studies on expert estimation of software development effortJournal of Systems and Software, 2003
- Approximate Inference in Generalized Linear Mixed ModelsJournal of the American Statistical Association, 1993
- Expert Judgment in Risk Analysis and Management: Process, Context, and PitfallsRisk Analysis, 1992
- Performance of a composite as a function of the number of judgesOrganizational Behavior and Human Performance, 1978
- Judgment under Uncertainty: Heuristics and BiasesScience, 1974