The art of measuring nothing: The paradox of measuring safety in a changing civil aviation industry using traditional safety metrics

Measuring safety as an outcome variable within the ultra-safe civil aviation industry during periods of deliberate organizational change is a difficult, and often fruitless, task. Anticipating eroding safety processes, based on measuring nothing happening over time, does not adequately capture the true state of an evolving safe system, and this is particularly relevant for leaders and managers in a civil aviation industry responsible for maintaining and improving ultra-safe performance while simultaneously managing demanding strategic business goals.In this paper, I will look at the difficulties of measuring safety as an outcome measure in high reliability organizations (HROs) using the traditional measures of incident and accident reporting during periods of deliberate organizational change inspired by the results from a 3 year longitudinal case study of the Norwegian Air Navigation Services provider – Avinor. I will first review the current safety literature relating to safety management systems (SMSs) used in the civil aviation industry. I will then propose a more holistic model that shifts the focus from the traditional safety monitoring mechanisms of risk analysis and trial and error learning, to the natural interactivity within socio-technical systems as found in high reliability organizations. And finally, I will present a summary of the empirical results of an alternate methodology for measuring perceived changes in safety at the operational level as leading indicators of evolving safety at the organizational level.     

A systems perspective of managing error recovery and tactical re-planning of operating teams in safety critical domains

Introduction

Research in human error has provided useful tools for designing procedures, training, and intelligent interfaces that trap errors at an early stage. However, this “error prevention” policy may not be entirely successful because human errors will inevitably occur. This requires that the error management process (e.g., detection, diagnosis and correction) must also be supported. Research has focused almost exclusively on error detection; little is known about error recovery, especially in the context of safety critical systems. The aim of this paper is to develop a research framework that integrates error recovery strategies employed by experienced practitioners in handling their own errors.

Method and Results

A control theoretic model of human performance was used to integrate error recovery strategies assembled from reviews of the literature, analyses of near misses from aviation and command & control domains, and observations of abnormal situations training at air traffic control facilities. The method of system dynamics has been used to analyze and compare error recovery strategies in terms of patterns of interaction, system affordances, and types of recovery plans. System dynamics offer a promising basis for studying the nature of error recovery management in the context of team interactions and system characteristics.

Impact on industry

The proposed taxonomy of error recovery strategies can help human factors and safety experts to develop resilient system designs and training solutions for managing human errors in unforeseen situations; it may also help incident investigators to explore why people's actions and assessments were not corrected at the time.     

New considerations for SIL verification of functional safety fieldbus communication

Safety integrity level (SIL) verification of functional safety fieldbus communication is an essential part of SIL verification of safety instrumented system (SIS), and it requires quantifying residual error probability (RP) and residual error rate of function safety communication. The present quantification method of residual error rate uses RP of cyclic redundancy check (CRC) to approximately replace the total RP of functional safety communication. Since CRC only detects data integrity-related errors and CRC has intrinsically undetected error, some other residual errors are not being considered. This research found some residual errors of the present quantification method. Then, this research presents an extended new approach, which takes the found residual errors into account to determine more comprehensive and reasonable RP and residual error rate. From perspective of the composition of safety message, this research studies RPs of those controlling segments (sequence number, time expectation, etc.) to cover the found residual errors beyond CRC detection coverage, and the influences of insertion/masquerade errors and time window on RP are investigated. The results turn out these residual errors, especially insertion/masquerade errors, may have a great influence on quantification of residual error rate and SIL verification of functional safety communication, and they should be treated seriously.     

Issues of the Human Reliability Analysis in the Context of Probabilistic Safety Studies

This article addresses methodological issues of the human reliability analysis (HRA) in the context of probabilistic safety studies. Several conventional HRA techniques, more often used for the evaluation of the human error probabilities (HEPs), have been classified. A taxonomy of human actions, failure events, and related factors is outlined in order to distinguish action phases, human behavior types and incorrect outputs (errors of omission or commission), error types (slips, lapses, and mistakes), and performance-shaping factors (PSFs) influencing the human performance. A tree is proposed to facilitate the selection of a specific method for the evaluation of human reliability with regard to attributes of the situation analyzed. A software system based on the expert system technology to facilitate and document PSA and HRA is outlined. At the end of the article some research challenges in the domain are discussed.     

Human error taxonomies applied to driving: A generic driver error taxonomy and its implications for intelligent transport systems

Recent research indicates that driver error contributes to up to 75% of all roadway crashes. Despite this, only relatively little is currently known about the types of errors that drivers make and of the causal factors that contribute to these errors being made. This article presents an overview of the literature on human error in road transport. In particular, the work of three pioneers of human error research, Norman, Reason and Rasmussen, is scrutinised. An overview of the research on driver error follows, to consider the different types of errors that drivers make. It was found that all but one of these does not use a human error taxonomy. A generic driver error taxonomy is therefore proposed based upon the dominant psychological mechanisms thought to be involved. These mechanisms are: perception, attention, situation assessment, planning, and intention, memory and recall, and action execution. In addition, a taxonomy of road transport error causing factors, derived from the review of the driver error literature, is also presented. In conclusion to this article, a range of potential technological solutions that could be used to either prevent, or mitigate, the consequences of the driver errors identified are specified.     

航空事故调查中人的因素安全建议框架研究

对航空事故中人的不安全行为进行了系统分析,将失误划分为感知失误、记忆失误、决策失误、技能失误4个类别,将违章划分为习惯性违章和偶然性违章。将人的因素干预维度确定为组织管理、人/团队、技术、任务和环境。针对不安全行为制定相应的初步改进措施后,还需从措施的可行性、可接受性、经济性和有效性4个方面进行综合评估和取舍。实证研究表明,人的因素干预矩阵是制定安全建议的有效工具,该框架具备良好的适用性。     

Human reliability analysis: A critique and review for managers

In running our increasingly complex business systems, formal risk analyses and risk management techniques are becoming more important part to managers: all managers, not just those charged with risk management. It is also becoming apparent that human behaviour is often a root or significant contributing cause of system failure. This latter observation is not novel; for more than 30 years it has been recognised that the role of human operations in safety critical systems is so important that they should be explicitly modelled as part of the risk assessment of plant operations. This has led to the development of a range of methods under the general heading of human reliability analysis (HRA) to account for the effects of human error in risk and reliability analysis. The modelling approaches used in HRA, however, tend to be focussed on easily describable sequential, generally low-level tasks, which are not the main source of systemic errors. Moreover, they focus on errors rather than the effects of all forms of human behaviour. In this paper we review and discuss HRA methodologies, arguing that there is a need for considerable further research and development before they meet the needs of modern risk and reliability analyses and are able to provide managers with the guidance they need to manage complex systems safely. We provide some suggestions for how work in this area should develop. But above all we seek to make the management community fully aware of assumptions implicit in human reliability analysis and its limitations.     

库存弹药事故人误危险性分析与评估模型研究

针对库存弹药安全的特殊性,提出库存弹药事故人为失误的定义,并从个体角度分析人误事件发生的机理。根据人为失误的不同主体,从管理决策、组织和勤务处理、操作两个方面构建了29个人误危险性的评价指标及相应的评判标准;综合考虑生理、心理、知识与技能、设备、环境、监督与管理等6类危险性抵消因子对人误危险性的影响;进而建立了库存弹药事故人误危险性的理论评估模型。研究结果表明:人为失误属于弹药仓储安全管理中的一类特殊危险源,人误评估指标体系和评估模型的构建过程应充分考虑人自身条件以及内、外部环境等因素的综合影响;同时研讨了人为失误危险性分析和评估模型存在的问题,并从发展人误数据采集技术、完善人误事故报告制度以及建立人误数据库等方面,指出下一步的研究重点。     

An application of data mining tools for the study of shipping safety in restricted waters

The effectiveness of the enforcement of the ISM-Code and the examination of its role in the distribution of causes of shipping accidents between human and non-human error was studied. All accidents involving Greek-flagged ships from 1995 to 2006, a time-scale which spans over the pre- and post-ISM period in navigational regions of restricted waters, were analyzed.The accident data was processed through a classification tree analysis which enabled the classification of various accident factors. The analysis revealed that although the human error maintained its position as the dominant factor in shipping accidents, there is also substantial evidence in support of the ISM-Code effective control over shipping accidents during the post-ISM period. The implementation of the ISM-Code led to an overall reduction of human-induced accidents in total. Furthermore, in terms of location, the ISM-Code improved the human-induced accident record within restricted waters.Conclusively, the ISM-Code constitutes an effective policy measure for shipping safety. The results of the classification tree analysis reported in the present work can be used by decision makers in companies and international organizations to build knowledge-based expert systems and augment their information in the field of safety policy and management.     

Examining the relationship between driver distraction and driving errors: A discussion of theory, studies and methods

Over the past two decades the concept of driver distraction has been the focus of intense research attention. One aspect of distraction for which there has been limited systematic research, however, is its role in driver error causation. This article presents a review of the distraction literature with a view to elucidating what is currently known about the types of driving errors that distraction contributes to and the mechanisms by which distraction induces these errors. The review revealed a number of fundamental gaps in our knowledge, including the number and nature of errors made by drivers when distracted; the mechanisms by which distraction causes errors; whether and how distraction disrupts drivers’ ability to recover from errors; and how system-wide factors moderate the relationship between distraction and error. In closing, we attempt to identify the most appropriate theoretical and methodological approach to drive the integrated study of distraction and error forward. We conclude that it is only through the adoption of a systems approach that integrated countermeasures can be proposed and implemented to mitigate driver errors caused by distraction.     

Assessment of complex sociotechnical systems – Theoretical issues concerning the use of organizational culture and organizational core task concepts

This article studies organizational assessment in complex sociotechnical systems. There is a practical need to monitor, anticipate and manage the safety and effectiveness of these systems. A failure to do so has resulted in various organizational accidents. Many theories of accidents and safety in industrial organizations are either based on a static and rational model of an organization or they are non-contextual. They are thus reactive in their search for errors and analysis of previous accidents and incidents, or they are disconnected from the actual work in the organization by their focus on general safety attitudes and values. A more proactive and predictive approach is needed, that is based on an accurate view on an organization and the demands of the work in question. This article presents and elaborates four statements: (1) the current models of safety management are largely based on either a rational or a non-contextual image of an organization, (2) complex sociotechnical systems are socially constructed and dynamic cultures, (3) in order to be able to assess complex sociotechnical systems an understanding of the organizational core task is required, and (4) effectiveness and safety depend on the cultural conceptions of the organizational core task. Finally, we will discuss the implications of the proposed concepts for safety research and development work in complex sociotechnical systems.     

Human error risk analysis in offshore emergencies

Human factors play an important role in the completion of emergency procedures. Human factors analysis is rooted in the concept that humans make errors, and the frequency and consequences of these errors are related to work environment, work culture, and procedures. This can be accounted for in the design of equipment, structures, processes, and procedures. As stress increases, the likelihood of human error also increases. Offshore installations are among the harshest and most stressful work environments in the world. The consequences of human error in an offshore emergency can be severe.A method has been developed to evaluate the risk of human error during offshore emergency musters. Obtaining empirical data was a difficult process, and often little information could be drawn from it. This was especially an issue in determining the consequences of failure to complete muster steps. Based on consequences from past incidents in the offshore industry and probabilities of human error, the level of risk and its tolerability are determined. Using the ARAMIS (accidental risk assessment methodology for industries) approach to safety barrier analysis, a protocol for choosing and evaluating safety measures to reduce and re-assess the risk was developed. The method is assessed using a case study, the Ocean Odyssey incident, to determine its effectiveness. The results of the methodology agree with the analysis of survivor experiences of the Ocean Odyssey incident.     

基于人因失误分析的医院电气安全管理研究

医院是重点用电单位,用电的安全性和供电可靠性都比较高,但因不可抗力、供电系统故障、医院管理问题、人为失误等方面的原因,仍存在发生各类电气突发事件的风险。其中,由人的误操作或不安全行为因素而诱发的电气方面的突发事件已成为医院非医疗事故的主要原因。文章针对医院电气安全操作方面的人因失误,从个人和组织两个角度进行了失误原因的分析,认为人的失误既受个体因素的影响,也受环境、制度和管理水平的影响。在此基础上,提出完善相关规章制度建设、加强教育与培训等,从组织制度建设、人员技术素质提高等方面,提出预防与减少人因失误的措施与方法,提高供配电质量,为医院医疗工作提供有效的电气安全后勤保障。     

Economic assessment of human errors in manufacturing environment

Traditional human reliability assessment techniques and accounting system cannot directly provide loss information for assessing the impacts of human errors. This obstacles force industrial managers to justify the proper accident and injury prevention process through their experiences. The efficiency and effectiveness of the system safety barriers are in doubt and the smooth operation of manufacturing activities are insecure. In this study, a human error cost estimation model is introduced to facilitate line managers with a proper tool to collect and calculate the total losses of its impact. Experts’ judgments and pair wise comparison technique are incorporated to interrogate managers’ knowledge of human errors and correspondent costs. This approach can overcome the problem of insufficient cost information caused by current accounting system and compensate the influence of safety and health department due to the low organizational status in quo. Although the cost figures may not represent exact amount of losses, the percentage of each cost factor in terms of department operation budget gives the managers a practical way for justifying how the resources should be allocated.     

基于RBF的起重作业岗位人因可靠性预测

为提高起重作业可靠性,防止人因失误酿成事故,针对人因失误的随机性、模糊性和不确定性特点,提出运用具有非线性映射能力和容错能力的径向基函数(RBF)神经网络,分析人因失误非线性动力学过程。以起重机操作岗位作为人因可靠性分析(HRA)实例,首先,建立基于"作业人员、交流界面、作业环境、作业特性、作业组织"的人因可靠性预测指标体系,并对指标进行量化;其次,根据人因可靠性原理,统计出人因失误次数,给出人因失误率;最后,通过对"人的疲劳和情绪、交流通道、作业复杂程度和时间裕度、照明环境和风力影响、工作强度和安全监管"等因素的分析,构建基于RBF的起重机操作岗位人因可靠性预测分析神经网络模型。分析结果表明,RBF预测分析同时包含人的操作可靠性与认知可靠性,预测结果同现场实际观测结果的符合度达到92.0%。     

人为差错成因分析方法研究

控制甚至消除人为差错成因能从本质上降低人为差错的发生概率,从而提高系统安全性。因此,辨识和分析人为差错成因是一件非常必要而又十分重要的工作。针对该问题,在总结和分析现有的人为差错成因分析方法的基础上,提出了一种新的人为差错成因分析框架。在该框架中,将差错成因分为"操作者-系统-任务-环境-组织因素"5个方面进行分析,并提出了"从大类到小类"的层次化分析原则。为了尽量降低不同差错成因之间的关联,提出了以任务为中心,将差错成因分为任务相关和任务无关两大类因子的分类方式。最终共分析得出了34种不同的差错成因。该差错成因分类框架的提出,为人为差错概率量化以及差错规避措施的设计提供了依据。     

Risk assessment: an approach to control hazards in mines

Accidents and injuries related to work are major occupational health problems in most of the industrialized countries.Traditional approaches to manage workplace safety in mines have mainly focused on job redesign and technical aspects of engineering systems.It is being realized that compliance to rules and regulations of mines is a prerequisite;however,it is not sufficient to achieve further reduction in accident and injury rates in mines.Proactive approaches are necessary to further improve the safety standards in mines.Unsafe conditions and practices in mines lead to a number of accidents,which in turn may cause loss and injury to human lives,damages to property,and loss of production.Hazard identification and risk assessment is an important task for the mining industry which needs to consider all the risk factors at workplaces.Applications of risk management approaches in mines are necessary to identify and quantify potential hazards and to suggest effective solutions.In this paper,the following risk estimation techniques were discussed:(i)DGMS(Directorate General of Mines Safety,India)risk rating criterion,and(ii)a matrix based approach.The proposed tools were demonstrated through an application in an opencast coal mine in India.It was inferred that the risk assessment approach can be used as an effective tool to indentify and control hazards in mines.     

基于信息加工模型的管制员差错分类与分析

在Wickens的人类信息加工模型的基础上,加入注意功能、情景意识、内部和外部操作成形因素,建立了管制员信息处理模型。按照该模型,空中交通管制人为差错可按照认知领域分为感知与警觉性差错、短时记忆差错、长时记忆差错、判断与计划差错、响应选择差错和响应执行差错;影响管制员操作的情境条件可分为外部操作成形因素和内部操作成形因素。对以往空管人为差错的分析表明,在信息加工层面分类的人为差错更宜于确定差错的心理致因。基于该理论模型的差错分类系统可以改进人为差错分析的有效性和一致性,从而提高差错管理的有效性。