The collection of data relating to incidents and near-miss events has become an entrenched and critical component of safety management across high-risk industries worldwide.1 According to the quality and safety axiom that “every defect should lead to improvement”, reporting systems exist to provide the raw data for continual improvement processes,2 as well as serving critical functions with respect to the local management of incidents.
In the quest towards enhancing patient safety, health care has also embraced the collection of near-miss and incident data, with all health services in Australia collecting some form of data. Across most health systems, voluntary reporting systems exist for near-miss and minor incidents, with the primary aim of collecting information about vulnerabilities in the health care system, so that remedies can be applied before an actual adverse event takes place.3 According to the World Health Organization, the primary role of such reporting systems is to enable learning across large health systems:
The fundamental role of patient safety reporting systems is to enhance patient safety by learning from failures of the health care system. We know that most problems are not just a series of random, unconnected one-off events. We know that health-care errors are provoked by weak systems and often have common root causes that can be generalized and corrected. Although each event is unique, there are likely to be similarities and patterns in sources of risk which may otherwise go unnoticed if incidents are not reported and analysed.4
Collecting data is only the first step in a process of organisational learning, through which lessons can be drawn from incidents, modifications made to practice, and the risk of adverse events reduced. At the organisational level, aggregate data are available and more sophisticated monitoring and analysis processes can take place, using data from multiple facilities or multiple units within the organisation. This process of using safety-related data has been termed quadruple-loop learning (personal, local, national and international), emphasising the wide-reaching potential for harnessing lessons from incident report data.5
Recently, a disparity between the number of reports being made and the rate of meaningful evidence-based change to practice has been identified.6 This in turn has suggested that the ineffective use of data, and a lack of published learnings from such systems, may be responsible for the deficit of evidence-based change.
Our study was part of a project funded by the Australian Commission on Safety and Quality in Health Care to investigate ways to improve learning from patient safety incident data.
We undertook a retrospective analysis of incident reports relating to patient misidentification, submitted by health professionals to a number of Australian health services’ incident reporting and management systems. Using a text-based search function for “patient identification”, we extracted incidents from the incident reporting system. From these, we selected incidents that met the inclusion criteria of having patient misidentification as the primary incident cause; occurring within a recent 5-year window (2004–2008); and having complete entry of data in the primary database fields.
The selected incidents were subjected to classification and analysis according to principal natural categories7 and system safety classification schemes.8
Natural categories were first defined in terms of incident types. These categories simply describe what went wrong in the clinical context, and map the error phenotype, which describes the observable manifestation of the error in the context of its occurrence.8
To understand the underlying aetiology of each incident, we classified error type using a 10-category scale developed for use in the context of patient safety.9 The classification describes the genotype of errors made by clinicians and enables an understanding of the reasons behind why an incident occurred from a human factors perspective.8
To classify the ways in which the clinical team identified and recovered from an error, we used an error detection mechanisms measure. This classification reflects the current focus on the design of resilient systems rather than simply attempting to eradicate error.10-12
Data were exported from the health services incident reporting systems and imported into a custom-built FileMaker database (FileMaker Inc, Santa Clara, Calif, USA) to preserve the original fields. Classification was undertaken within this database into relevant natural categories. These data were subsequently exported to Microsoft Excel (Microsoft Corp, Redmond, Wash, USA) and PASW Statistics version 18.0 (SPSS Inc, Chicago, Ill, USA) for analysis. Due to the exploratory nature of this study, analyses were primarily restricted to descriptive analyses.
Of more than 1000 incidents extracted from the health services’ incident reporting systems, 487 incidents met our inclusion criteria. There were 43 unique incident types present within the dataset, giving a ratio of incident types to incidents of about 1 : 11. However, most of these categories were represented by few instances, with about half the categories (22/43) represented by three or fewer instances of that particular incident type. The five most prevalent categories accounted for 60% of all 487 incidents (Box 1).
The process of category saturation and the rate at which the unique incident types were identified in the dataset are shown in Box 2. The shape of the curve indicates a rapid initial classification of different incident types, with half the total number of categories identified in the first 13.5% of the incidents. All 43 incident types were classified within 76.2% of the dataset.
Of the 487 incidents analysed, 52 (10.7%) had sufficient information to identify and classify a specific error type. Of these, 67.3% (35/52) were skill-based errors (slips or lapses), which is an intuitive finding given the prevalence of errors in automated tasks such as labelling specimens or labelling request forms. Of the remainder, 11.5% (6/52) were deliberative errors, 7.7% (4/52) involved choice of rule, and 5.8% (3/52) involved both matching and perception error types.
Two hundred and eighty-eight incident reports (59.1%) had sufficient detailed information to enable classification of the error-detection mechanism. The most prevalent error-detection mechanism was routine check (44.1%, 127/288). Mismatch with the patient’s clinical presentation was the next most prevalent mechanism (15.6%, 45/288), followed by mismatch with the clinician’s expectation (12.5%, 36/288).
The principal role of incident reporting systems is to ensure a consistent and coordinated approach to the identification and analysis of incidents so that lessons can be learned and shared across the whole health system.13,14 Reporting of near-miss events has been shown to offer numerous benefits compared with retrospective investigation of adverse events. Perhaps the most important of these benefits are the greater frequency of near-miss events, thus allowing quantitative analysis; and the ability to identify and analyse the recovery functions that enabled the accident trajectory to be stopped before an actual adverse event took place.15
These results echo a report of the Institute of Medicine in the United States, which suggested that the value of near-miss reporting required further investigation.16 Although there is general agreement that near-miss reports contain relevant information about identifiable hazards that cannot be collected by other means,15 most organisations only take action on serious adverse events, which diminishes the value of reporting large numbers of near misses.14,17 Indeed, the sheer volume of incidents reported means that health care organisations tend to investigate most events superficially. For the few incidents that receive thorough investigation, the principal method is root cause analysis, which is seen as the gold standard for gaining deeper insights into the causal features of an adverse event.14,18
Incident reporting systems are also subject to a number of other limitations, including international suggestions of significant underreporting of incidents,19 and of medical error, as doctors rarely report.20 Any reporting is highly affected by hindsight bias (based on the degree of harm the patient has suffered, especially considering that doctors tend to disproportionally report the more severe events). Incident reports are thus “a nonrandom sample of identified hazards from a larger unknown universe of hazards”.14 In light of these limitations, it is critical that we explore alternative ways to learn from incidents in health care.15-18
Human factors classification — measuring the resilience of the system. Early studies on adverse events in health care focused on the origins of errors and on the mechanisms involved in their production. Accordingly, most effort was dedicated to the field of error prevention and focused on addressing the contextual and organisational issues in the production of errors.1 However, in a more enlightened world where we accept error to be both inevitable and ubiquitous, current safety science now focuses on the processes of error detection and error recovery.21-23
A novel approach to more effective use of incident report data in industries such as health care would first involve risk-based sampling of incidents from traditional incident reporting systems. The aim of this sampling would be the proactive identification of emergent high-risk safety concerns, rather than the analysis of sentinel events after they have occurred. Risk-based sampling would involve introducing risk assessment into the initial data-collection process within the incident reporting system. This approach already exists within other health care safety and quality methods. Health care failure mode and effects analysis, for example, risk assesses each failure event in terms of potential severity and probability of occurrence, producing a hazard score that in turn enables prioritisation of safety management activities.18
Provenance: Not commissioned; externally peer reviewed.
Abstract
Objectives: To assess the utility of Australian health care incident reporting systems and determine the depth of information available within a typical system.
Design and setting: Incidents relating to patient misidentification occurring between 2004 and 2008 were selected from a sample extracted from a number of Australian health services’ incident reporting systems using a manual search function.
Main outcome measures: Incident type, aetiology (error type) and recovery (error-detection mechanism). Analyses were performed to determine category saturation.
Results: All 487 selected incidents could be classified according to incident type. The most prevalent incident type was medication being administered to the wrong patient (25.7%, 125), followed by incidents where a procedure was performed on the wrong patient (15.2%, 74) and incidents where an order for pathology or medical imaging was mislabelled (7.0%, 34). Category saturation was achieved quickly, with about half the total number of incident types identified in the first 13.5% of the incidents. All 43 incident types were classified within 76.2% of the dataset. Fifty-two incident reports (10.7%) included sufficient information to classify specific incident aetiology, and 288 reports (59.1%) had sufficient detailed information to classify a specific incident recovery mechanism.
Conclusions: Incident reporting systems enable the classification of the surface features of an incident and identify common incident types. However, current systems provide little useful information on the underlying aetiology or incident recovery functions. Our study highlights several limitations of incident reporting systems, and provides guidance for improving the use of such systems in quality and safety improvement.