Descriptions of medical errors are essential to understanding the types of mistakes occurring in general practice and to develop strategies to improve patient safety. There is no reason to suppose that a taxonomy peculiar to errors in one country can be applied in another. Indeed, an international taxonomy is needed if direct comparisons are to be made between countries.
In this article, we describe the Australian results of the 2001 international pilot study, the Primary Care International Study of Medical Errors (PCISME), drawing comparisons and contrasts with findings from all other participating countries combined.
PCISME is the first international study of medical errors in general practice, and involved six countries with similar primary healthcare standards. It aimed to classify the types of errors recognised by primary medical care providers, and develop a taxonomy of the errors reported. It also tested an electronic method of data collection in general practice that could be applied to the six participating countries.
The Robert Graham Center of the American Academy of Family Physicians invited investigators from Australia, Canada, the Netherlands, New Zealand, the United Kingdom and the United States to participate in PCISME. Countries were chosen because they had a similar concept of primary healthcare and their healthcare systems all delivered a First World style of medicine.
In essence, PCISME was a survey of the errors noted by a non-random sample of GPs in the six participating countries. All participants were allowed a minimum of three months to contribute reports. Data collection occurred from June to October 2001 in Australia, and in other countries between June and December 2001.
The protocol was approved by the University of Sydney Human Research Ethics Committee.
In each country, selection of GPs was conducted through invitation by local investigators using established GP research networks. Each country aimed to include about 20 doctors. Eligibility criteria included working in clinical general practice for at least 20 hours per week (excluding a main work activity of teaching or research), with an absence from work during the study period not exceeding two weeks. Participants had to be computer literate and have access to a personal computer with a CD-ROM drive, running Windows 95 or higher, and connected to the Internet.
The protocol regarding payments varied among participating countries. Australian doctors were invited to claim an honorarium: if they reported at least 10 errors, they could claim the equivalent of the general practice fee recommended by the Australian Medical Association for 10 standard consultations. They were also eligible for Quality Assurance Audit points from the Royal Australian College of General Practitioners (RACGP). It was not possible to pay doctors per report, as this would have impinged upon the anonymity of the participants.
Characteristics of the participating Australian doctors were collected during enrolment to compare the sample with the population of Australian GPs.
The definition of "medical error" used was tested in a pilot study at the American Academy of Family Physicians in 2000: 1
"Errors are events in your practice that make you conclude: 'that was a threat to patient well-being and should not happen. I don't want it to happen again'. Such an event affects or could affect the quality of the care you give your patients. Errors may be large or small, administrative or clinical, or actions taken or not taken. Errors may or may not have discernable effects. Errors in this study are anything that you identify as something wrong, to be avoided in the future."
This definition attempts to incorporate the broadest possible range of problems. It allows reports not only on events attributable to the reporter's actions, but also on unwanted occurrences noted concerning patient care.
Error reports could be completed on any computer with an Internet connection. The World Health Network's medical information software "Healix" was used to access the study's electronic reporting form and to transmit data. Each report was identified with a self-chosen personal identification number (PIN). Doctors indicated their country, and then proceeded through the questionnaire (Box 1). They were asked not to use any identifying information in their reports.
High-level encryption was used when transmitting the data to the server in London, UK. Doctors were advised not to disclose their PIN, and not to retain paper copies of their error reports. The PINs were not known by the researchers, and were used solely to identify how many reports a participant had contributed. Access to the database was limited to the chief investigators in each country.
As the reports were submitted, the Australian investigators, in consultation with the International Principal Investigator, categorised them. Error reports from the other participating countries were simultaneously categorised in this manner. The taxonomy was initially based on an earlier US pilot study.1 It was further developed and refined during the trial in order to capture the full extent of error types reported from all countries.
The proportion of reports attributable to each category was calculated for Australia and for all other countries combined.
Australian participants were drawn from the 400 GPs listed on the research databases of the Department of General Practice, University of Sydney, and the NSW Projects, Research and Development Unit of the RACGP. In total, 101 doctors received telephone invitations and 23 were enrolled. Doctors outside the Sydney metropolitan area were excluded because of cost constraints. Of those declining, 32 were too busy and 23 offered no reason. Reasons for non-participation given by the remaining 23 doctors were uncomfortable with an electronic study (6), no Internet connection (4), wished not to participate in further research (4), concerned with potential bad press from the study (3), incompatible operating system (2), on holidays (2), and concerned about the accuracy of the results (2).
The total number of doctors invited to participate in the study in the other countries is unknown.
The 23 participating Australian doctors were similar to Australian GPs with respect to sex and computer use (Box 2).
Australian doctors on average contributed the highest number of reports (Box 3). Although more participants were enrolled in the study in Australia than in other countries, the US had the highest number of different PINs submitted, implying more individuals contributed reports.
The final taxonomy was a five-level system encompassing 171 error types. Errors were classed primarily as due to a problem with processes involved in carrying out general practice, or associated with a problem with the knowledge and skills involved in general practice.
Two reports from Australia and four from other countries were excluded as non-errors. In all countries, more reports were of "process" than "knowledge and skills" errors. Box 4 shows the taxonomy, to the first three levels.
Patient harm was reported in 32% of Australian errors, compared with 31% in other countries. This harm was considered "very serious" or "extremely serious" (4 or 5 on the Likert scale) in 9% of Australian cases, compared with 3% in other countries. Consequences included hospital admission in 4.5% of the Australian cases, and 3.7% of others, and death in one (0.8%) Australian case, compared with four (1.3%) in other countries.
Errors that were classified with the same code could have different levels of seriousness. Examples of such errors are the two Australian reports classified as 1.1.3, a "process error", due to office administration, concerning patient flow through the healthcare system. In one of these reports, patients were incorrectly called into another doctor's room in a group practice, and the doctor who was supposed to see them could not provide the intended follow-up. The other report described a situation in which an elderly patient was triaged in a local emergency department as low priority. During a long, unattended wait, a fall during a bathroom visit resulted in a broken hip.
The principal findings of the PCISME pilot study support the theory that general practice in Australia shares many of the types of problems encountered by GPs in other, similar developed countries. This is demonstrated by the taxonomic descriptions at the second order of classification: all categories of errors that occurred in Australia occurred in one or more of the other countries. We have also been able to demonstrate a successful method of electronically reporting mistakes in an Australian general practice setting.
Our aim has been to try to capture the breadth of different mistakes occurring. The design of the study did not allow quantification of the prevalence of different error types. The limitation with all research of this nature is that a doctor must be aware that an error has occurred, and then must be willing to report the error.
It is possible that the payment of the Australian doctors contributed to the greater numbers of reports from this group. Previous primary care research has suggested that recruitment and quality might be improved through payments,5 and that, for research that is otherwise ethically permissible, it is also ethically permissible for researchers to pay participants for their out-of-pocket expenses, participation, inconvenience and risk-taking.6 Primary care funding varies significantly among the participating countries, and other countries chose not to offer an honorarium.
A high level of computer literacy was required to participate in PCISME, and this might have affected the types of errors that were described. All error reports were submitted electronically from participating countries, except for Canada, where software problems resulted in their doctors mainly reverting to paper reports. In Australia, there is a high level of computer use in general practice, with around 89% of practices now using computers.3
In relation to other studies, PCISME built upon earlier patient safety research conducted in Australia, and extended it into an international context. Australia has been a leader in carrying patient safety research into primary care settings. In 1993, the Commonwealth Government provided funding to test incident monitoring in six specialties, including general practice. A pilot incident-monitoring study was conducted by the RACGP and the Family Medicine Research Unit of the University of Sydney. An analysis of the first 805 incidents reported by GPs between October 1993 and June 1995 demonstrated that incident monitoring can be successfully applied in general practice,7,8 and is useful for identifying sources of misdiagnosis and for implementation and assessment of quality improvement strategies.9
PCISME differs from previous Australian work with respect to the methodology of error reporting and the classification system used to describe these errors. As the two studies used non-random samples of Australian GPs taken from different study populations, there is limited comparability between the results of PCISME and the earlier incident-monitoring study.7,8,10 Furthermore, PCISME used a different definition of error from previous Australian studies, which defined an incident as "an unintended event, no matter how seemingly trivial or commonplace, that could have harmed or did harm a patient".7 The definition of error used in our study was broader, being concerned with mistakes rather than patient harm, and so could have encouraged more reports that posed less threat to patient safety, such as administrative problems. The definition we used was generally well understood by PCISME participants, with only six reports from all countries (1.4%) assessed as not involving an error.
Doctors and other healthcare professionals have been reluctant to admit and address the problem of errors, both because of feelings of guilt and from the desire to avoid colleagues' disapproval or punishment.11,12 Studies of this nature might increase the acceptability of discussing mistakes in general practice.
Our study has commenced the development of an international taxonomy of errors in primary care which can be used to plan future studies examining the prevalence of mistakes in general practice. The strength of an international collaboration will become apparent when meaningful differences between countries are defined in the prevalence of different error types. This information can then be used to design interventions or alter existing systems to reduce errors in primary care.
1: Questionnaire used in PCISME 2001
Question |
Answer format |
||||||||||
a) Is the problem related to a specific patient? |
YES/NO |
||||||||||
b) If yes, how well do you know the patient? |
5 point Likert scale |
||||||||||
c) Patient age |
Free text |
||||||||||
d) Patient sex |
Male/Female |
||||||||||
e) Is the patient a member of a group designated an ethnic minority by your country? |
YES/NO |
||||||||||
f) Does patient have a complex health problem? |
YES/NO |
||||||||||
g) Does patient have a chronic health problem? |
YES/NO |
||||||||||
h) What happened? Please consider what, where and who was involved. |
Free text |
||||||||||
i) What was the result? Please think about actual and potential consequences. |
Free text |
||||||||||
j) What may have contributed to this error? Please consider any special circumstances. |
Free text |
||||||||||
k) What could have prevented it? Please consider what should change to prevent recurrence. |
Free text |
||||||||||
l) Where did the error happen? Choose all that apply from Office or surgery, Nursing home, Hospital, Patient's home, Telephone contact, Emergency Room, Laboratory, Pharmacy, Radiology |
Check a box or boxes: 9 choices |
||||||||||
m) To your knowledge, was any patient harmed by this error? |
YES/NO |
||||||||||
n) If yes, how would you rate the seriousness of this harm? |
5 point Likert scale |
||||||||||
o) How often does this error occur in your practice? First time, Seldom 1–2 per year, Sometimes 3–11 per year, Frequently >1/month |
Check a box: 4 choices |
||||||||||
p) Other comments? |
Free text |
||||||||||
2: Characteristics of PCISME Australian participants compared with recent Australian general practice statistics
Characteristic |
PCISME participants |
Australian general practice |
P* |
||||||||
Male |
18/23 (78%) |
13 929/20 852 (66.8%)2 |
0.24 |
||||||||
Doctor works in metropolitan area |
23/23 (100%) |
16 186/20 852 (77.6%)2† |
0.01 |
||||||||
Computers used in practice |
21/22 (95%) |
1070/1202 (89%)3 |
0.34 |
||||||||
Computer used by practice for prescribing |
19/22 (86%) |
4113/5273 (78%)4 |
0.34 |
||||||||
|
* P-value by χ2 test. † Derived from Australian Medical Workforce Advisory Committee total GP population of 20 852, with 75.6% of men, and 81.7% of women practising in metropolitan areas. PCISME = Primary Care International Study of Medical Errors. |
|||||||||||
3: PCISME results from participating countries
Australia |
Canada |
United Kingdom |
Netherlands |
New Zealand |
United States |
||||||
Number of doctors enrolled |
23 |
15 |
20 |
8 |
20 |
18 |
|||||
Number of PINs submitted |
17 |
15 |
14 |
4 |
11 |
18 |
|||||
Number of reports received in total |
134 |
84 |
63 |
14 |
66 |
74 |
|||||
Median number of reports per PIN (range) |
9 (1–17) |
5 (1–10) |
2 (1–18) |
3 (1–7) |
5 (1–26) |
2 (1–27) |
|||||
PIN = personal identification number (self-chosen by participants). PCISME = Primary Care International Study of Medical Errors. |
|||||||||||
4: The first three levels of the five-level taxonomy of errors, with number (%) of reports in the first two levels
Australia |
Other countries |
||||||||||
1. Process Errors |
104 (79%) |
236 (79%) |
|||||||||
1.1. Errors in office administration |
26 (20%) |
55 (19%) |
|||||||||
1.1.1. Filing system errors |
|||||||||||
1.1.2. Chart completeness errors |
|||||||||||
1.1.3. Patient flow (through the healthcare system) |
|||||||||||
1.1.4. Message handling errors |
|||||||||||
1.1.5. Appointments errors |
|||||||||||
1.1.6. Errors in maintenance of a safe physical environment |
|||||||||||
1.2. Investigation errors |
17 (13%) |
55 (19%) |
|||||||||
1.2.1. Laboratory errors |
|||||||||||
1.2.2. Diagnostic imaging errors |
|||||||||||
1.2.3. Errors in the processes of other investigations |
|||||||||||
1.3. Treatment errors |
38 (29%) |
72 (24%) |
|||||||||
1.3.1. Medication errors |
|||||||||||
1.3.2. Errors in other treatments |
|||||||||||
1.4. Communication errors |
20 (15%) |
42 (14%) |
|||||||||
1.4.1. Errors in communication with patients |
|||||||||||
1.4.2. Errors in communication with other healthcare providers (non-medical) |
|||||||||||
1.4.3. Errors in communication with other doctors |
|||||||||||
1.4.4. Errors in communication amongst the whole healthcare team |
|||||||||||
1.5. Payment errors |
1 (1%) |
4 (1%) |
|||||||||
1.5.1. Errors in processing insurance claims |
|||||||||||
1.5.2. Errors in electronic payments |
|||||||||||
1.5.3. Wrongly charged for care not received |
|||||||||||
1.6. Errors in healthcare workforce management |
2 (2%) |
8 (3%) |
|||||||||
1.6.1. Absent staff not covered |
|||||||||||
1.6.2. Dysfunctional referral procedures |
|||||||||||
1.6.3. Errors in appointing after-hours workforce |
|||||||||||
2. Knowledge and Skills Errors |
28 (21%) |
61 (21%) |
|||||||||
2.1. Errors in the execution of a clinical task |
7 (5%) |
7 (2%) |
|||||||||
2.1.1. Non-clinical staff made the wrong clinical decision |
|||||||||||
2.1.2. Failed to follow standard practice |
|||||||||||
2.1.3. Lacked needed experience or expertise in a clinical task |
|||||||||||
2.2. Errors in diagnosis |
18 (14%) |
36 (12%) |
|||||||||
2.2.1. Error in diagnosis by a nurse |
|||||||||||
2.2.2. Delay in diagnosis |
|||||||||||
2.2.3. Wrong or delayed diagnosis attributable to misinterpretation of investigations |
|||||||||||
2.2.4. Wrong or delayed diagnosis attributable to misinterpretation of examination |
|||||||||||
2.2.5. Wrong diagnosis by a pharmacist |
|||||||||||
2.2.6. Wrong diagnosis by a hospital-based doctor |
|||||||||||
2.3. Wrong treatment decision with right diagnosis |
3 (2%) |
18 (6%) |
|||||||||
2.3.1. Wrong treatment decision, influenced by patient preferences |
|||||||||||
2.3.2. Wrong treatment decision by doctor |
|||||||||||
Received 19 April 2002, accepted 3 June 2002
- Meredith A B Makeham1
- Mary County2
- Michael R Kidd3
- Susan M Dovey4
- 1 Department of General Practice, University of Sydney, Sydney, NSW.
- 2 The Robert Graham Center: Policy Studies in Family Practice and Primary Care, American Academy of Family Physicians, Washington, DC, USA.
We gratefully acknowledge the GP participants from the six countries. We would like to thank Walter Rosser, Aneez Esmail, Katherine Hall, Chris Van Weel, Anton Kuzel, Steven Woolf, and the other research members of the LINNAEUS collaboration. We also thank the World Health Network, Professor Charles Bridges-Webb AO of the NSW Projects, Research and Development Unit, RACGP, and Dr Jonathon Craig, Department of Public Health, University of Sydney, for their assistance.
None identified.
- 1. Dovey SM, Meyers DS, Phillips RL, et al. A preliminary taxonomy of medical errors in family practice. J Qual Safety Health Care 2002. In press.
- 2. Australian Medical Workforce Advisory Committee. The general practice workforce in Australia. AMWAC Report 2000.2. Sydney: AMWAC, 2000. Available at <http://amwac.health.nsw.gov.au/amwac/amwac/pdf/gp_2002.pdf>.
- 3. Western M, Dwan K, Makkai T, et al. Measuring IT use in Australian general practice 2001. Brisbane: University of Queensland, 2001.
- 4. General Practice Computing Group. Practice Incentive Payment Statistics: electronic data connectivity. <http://www.gpcg.org/topics/pip.html#connectivity>. Accessed 3 June 2002.
- 5. Foy R, Parry J, McAvoy B. Clinical trials in primary care: targeted payments for trials might help improve recruitment and quality. BMJ 1998; 317: 1168-1169.
- 6. Moore A. The ethics of payment for research participants. Monash Bioethics Rev 1996; 15(3): 10-13.
- 7. Bhasale AL, Miller GC, Reid SE, Britt HC. Analysing potential harm in Australian general practice: an incident monitoring study. Med J Aust. 1998; 169: 73-76. <MJA full text>
- 8. Britt H, Miller GC, Steven ID, et al. Collecting data on potentially harmful events: a method for monitoring incidents in general practice. Fam Pract 1997; 14: 101-106.
- 9. Bhasale A. The wrong diagnosis: identifying causes of potentially adverse events in general practice using incident monitoring. Fam Pract 1998; 15: 308-318.
- 10. Miller G, Britt H, Steven I, et al. Clinical Incident Monitoring. Aust Fam Physician 1996; 25: 821.
- 11. Leape LL. A systems analysis approach to medical error. J Eval Clin Pract 1997; 3: 213-222.
- 12. Kidd M, Veale B. How safe is Australian general practice and how can it be made safer? [editorial]. Med J Aust 1998; 169: 67-68. <MJA full text>
Abstract
Objectives: To develop an international taxonomy describing errors reported by general practitioners in Australia and five other countries.
Design and setting: GPs in Australia, Canada, the Netherlands, New Zealand, the United Kingdom and the United States reported errors in an observational pilot study. Anonymous reports were electronically transferred to a central database. Data were analysed by Australian and international investigators.
Participants: Non-randomly selected GPs: 23 in Australia, and between 8 and 20 in the other participating countries.
Main outcome measures: Error categories, and consequences.
Results: In Australia, 17 doctors reported 134 errors, compared with 301 reports by 63 doctors in the other five countries. The final taxonomy was a five-level system encompassing 171 error types. The first-level classification was "process errors" and "knowledge and skills errors". The proportion of errors in each of these primary groups was similar in Australia (79% process; 21% knowledge and skills) and the other countries (80% process; 20% knowledge and skills). Patient harm was reported in 32% of reports from Australia and 30% from other countries. Participants considered the harm "very serious" in 9% of Australian reports and 3% of other countries' reports.
Conclusions: This pilot study indicates that errors are likely to affect primary care patients in similar ways in countries with similar primary healthcare systems. Further comparative studies are required to improve our understanding of general practice error differences between Australia and other countries.