The importance of better understanding error and safety in primary care is widely accepted, with recent calls to promote efforts and improve resources in this area of research.1 A few studies have described errors in general practice using the incident monitoring technique,2-10 but none have quantified the problem.
A study of family medicine physicians in Cincinatti, USA, showed errors were identified in almost a quarter of clinical encounters.2 An error rate of 75.6 per 1000 general practice appointments (7.6%) was reported in one UK study.3 Reviewing incident reports from an outpatient primary care setting, a US study reported a prevalence of adverse events of 3.7 per 100 000 clinic visits.11
We adopted the definition of “error” used in previous work.5,6,12,13 Errors may have been attributable to the reporter’s actions or other unwanted occurrences:
The General Practice Programs Branch of the Commonwealth Department of Health and Ageing (GP Branch) provided a stratified random sample of 320 names from the population of 4666 vocationally registered GPs in NSW whose Medicare billings in the previous quarter indicated that they were engaged in full-time work.14
A sample size of 80 was calculated from pilot study results of 134 reports made from up to 23 GPs over 4 months, which had a 25% participation rate.5 Allowing 1 week of absence each and estimating 132 consultations per week,15 we hypothesised a reported error rate of 0.27% per consultation. To estimate this with a 95% confidence interval of 0.25%–0.29%, we required 205 465 patient encounters. We anticipated that 80 GP participants would have about 480 000 patient encounters, with the excess allowing for any effect of clustered samples.
The sample was drawn from Rural, Remote and Metropolitan Area (RRMA) groupings of RRMA 1, RRMA 2–3, and RRMA 4–7.14 The GP Branch stratified the sample by gender and age, dichotomised around 45 years. One hundred and sixty (50%) of the sample were drawn from RRMA 1, 80 (25%) from RRMA 2–3 and 80 (25%) from RRMA 4–7 regions. The proportions of the source population in these regional groupings were 68%, 16% and 16%, respectively. The smaller groups were over-sampled to ensure adequate counts of Medicare items to allow future subgroup analysis.
A secure website and reporting process were developed to house the error questionnaire, shown in Box 1, which was piloted in 2001.5 Data transmission was protected by 128-bit encryption and the website was hosted by a secure server behind a firewall. The website allowed participants to contact investigators and view linked medical resources. Participants were asked about any website problems in monthly contact via email, telephone or facsimile.
Box 2 shows RRMA grouping comparisons of the number of reports submitted, PINs submitted, GPs who reported at interview sending a report, average number of reports per GP, and the median number of reports per PIN over the study duration.
The distribution of reporting frequencies among RRMA groupings was assessed by looking at the number of reports per PIN, shown in Box 3. This varied from one to 25, with a median of three for all participants combined. There was no significant difference between RRMA groupings in the median number of reports submitted per PIN (P = 0.40) (for this analysis, values that were equal to the median were evenly split between the above and below groups).
The total numbers of Medicare items relating to a patient encounter and total numbers of individual patients seen by RRMA grouping are shown in Box 4. Each report within an RRMA grouping was matched with the group’s average number of Medicare items and average number of patients seen, and assigned a sampling weight to adjust for the larger representation of RRMA 2–3 and RRMA 4–7 GPs in the sample design.
Original research on errors and adverse events in primary care consists of relatively limited numbers of studies, focusing on qualitative descriptions.16 The TAPS study provides the first calculated incidence of reported error from a representative random sample of GPs. Despite the small sample size and participation rate of 26%, these findings are likely to be generalisable to GPs in NSW, as we detected no statistically significant differences between the age, sex, and Medicare billings when comparing participants to the source population of 4666 GPs. Comparative data were not available from the GP Branch for the non-participants due to consent issues.
It has been suggested previously that an error database in general practice should be established.17,18 Our results indicate that using a secure website with an anonymous reporting method is a practical way of collecting error information from GPs, as 94% of doctors who enrolled in the study reported at interview that they sent an error report.
Our findings provide an estimation of the incidence of errors in general practice that would be reported if an appropriate reporting system were in place, but this should not be interpreted as an estimate of the total number of errors that participants noticed, or of the underlying number of errors occurring in the GP community. It is very difficult to assess the proportion of errors that would go unreported even when a reporting system such as TAPS is available. A GP may not be aware that an error has occurred. GPs have been found to under-report adverse drug events,19 and so other patient safety threats may also be under-reported.
Anonymity was of major importance in our methodological design, to encourage reporting. There has previously been a reluctance of health professionals to address the problem of errors due to feelings of guilt and a desire to avoid disapproval from colleagues.20 However, using a PIN that was unknown to investigators led to some difficulties in interpreting reports per GP. Researchers were unable to assist if a PIN was forgotten, and there was no way of knowing how many GPs changed PINs during the study. Seventy-nine GPs gave feedback that they submitted an error report, but 85 PINs were used for error reports, which suggests that on at least six occasions GPs changed their PINs. Our estimate of report numbers per participant, which is based on PINs, could therefore be an underestimate.
To our knowledge, there are no studies directly comparable with our incidence results, which are lower than hypothesised. The authors of a US study that found an error rate close to 25% state that their results were not generalisable due to limitations of the study.2 Only a small group of 15 doctors in seven practices made reports, completing a review form after every consultation during three half-day sessions. The UK finding of a 7.6% error rate was based on a 2-week collection of errors from 10 practices in a single city.3 However, a major difference is that this result was based upon a collection of reports from a large cross-section of staff working in general practice, with 163 people able to submit reports, and a high proportion of participants who had administrative rather than clinical roles.
Our method of prospective self-reporting appears to have returned a higher rate of error detection than that in a US study using retrospective record review in eight primary health care clinics.11 Again, direct comparison is difficult, as medical adverse events were counted in this work, rather than errors in the sense that we defined.
1 Error questionnaire used in the Threats to Australian Patient Safety (TAPS) study
- Meredith A B Makeham1
- Michael R Kidd1
- Deborah C Saltman1
- Michael Mira1
- Charles Bridges-Webb2
- Chris Cooper1
- Simone Stromer1
- 1 Discipline of General Practice, The University of Sydney, Sydney, NSW.
- 2 RACGP NSW Projects, Research and Evaluation Unit, The Royal Australian College of General Practitioners, Sydney, NSW.
Meredith Makeham is a National Health and Medical Research Council scholarship holder. We gratefully acknowledge the contribution of the 84 NSW GPs who provided the data, and the funding from the National Health and Medical Research Council and the Primary Health Care Research, Evaluation and Development program. We also thank Mr Tony Lawrence and Ms Angela Mikalauskas of the General Practice Branch, Commonwealth Department of Health and Ageing, for assistance with sampling and Medicare data provision, and Ms Geraldine Card for her valuable contribution in managing the study and collating results.
None declared.
- 1. Jacobson L, Elwyn G, Robling M, Jones RT. Error and safety in primary care: no clear boundaries. Fam Pract 2003; 20: 237-241.
- 2. Elder NC, Vonder Meulen M, Cassedy A. The identification of medical errors by family physicians during outpatient visits. Ann Fam Med 2004; 2: 125-129.
- 3. Rubin G, George A, Chinn DJ, Richardson C. Errors in general practice: development of an error classification and pilot study of a method for detecting errors. Qual Saf Health Care 2003; 12: 443-447.
- 4. Dovey SM, Phillips RL, Green LA, Fryer GE. Types of medical errors commonly reported by family physicians. Am Fam Physician 2003; 67: 697.
- 5. Makeham MA, Dovey SM, County M, Kidd MR. An international taxonomy for errors in general practice: a pilot study. Med J Aust 2002; 177: 68-72.
- 6. Dovey SM, Meyers DS, Phillips RL Jr, et al. A preliminary taxonomy of medical errors in family practice. Qual Saf Health Care 2002; 11: 233-238.
- 7. Steven ID, Malpass A, Moller J, et al. Towards safer drug use in general practice. J Qual Clin Pract 1999; 19: 47-50.
- 8. Bhasale A. The wrong diagnosis: identifying causes of potentially adverse events in general practice using incident monitoring. Fam Pract 1998; 15: 308-318.
- 9. Bhasale AL, Miller GC, Reid SE, Britt HC. Analysing potential harm in Australian general practice: an incident-monitoring study. Med J Aust 1998; 169: 73-76.
- 10. Britt H, Miller GC, Steven ID, et al. Collecting data on potentially harmful events: a method for monitoring incidents in general practice. Fam Pract 1997; 14: 101-106.
- 11. Fischer G, Fetters MD, Munro AP, Goldman EB. Adverse events in primary care identified from a risk-management database. J Fam Pract 1997; 45: 40-46.
- 12. Woolf SH, Kuzel AJ, Dovey SM, Phillips RL Jr. A string of mistakes: the importance of cascade analysis in describing, counting, and preventing medical errors. Ann Fam Med 2004; 2: 317-326.
- 13. Tilyard M, Dovey S, Hall K. Avoiding and fixing medical errors in general practice: prevention strategies reported in the Linnaeus Collaboration’s Primary Care International Study of Medical Errors. N Z Med J 2005; 118: U1264.
- 14. Budget and Performance Branch, Primary Care Division, Department of Health and Ageing. General Practice in Australia: 2004. 1st ed. Canberra: Commonwealth of Australia, 2005.
- 15. General Practice Branch, Department of Health and Aged Care. General Practice in Australia: 2000. 1st ed. Canberra: Commonwealth of Australia, 2000.
- 16. Elder NC, Dovey SM. Classification of medical errors and preventable adverse events in primary care: a synthesis of the literature. J Fam Pract 2002; 51: 927-932. [Erratum appears in J Fam Pract 2002; 51: 1079.]
- 17. Sheikh A, Hurwitz B. Setting up a database of medical error in general practice: conceptual and methodological considerations. Br J Gen Pract 2001; 51: 57-60.
- 18. Runciman WB. Lessons from the Australian Patient Safety Foundation: setting up a national patient safety surveillance system — is this the right model? Qual Saf Health Care 2002; 11: 246-251.
- 19. Moride Y, Haramburu F, Requejo AA, Begaud B. Under-reporting of adverse drug reactions in general practice. Br J Clin Pharmacol 1997; 43: 177-181.
- 20. Kidd MR, Veale B. How safe is Australian general practice and how can it be made safer [editorial]? Med J Aust 1998; 169: 67-68.
Abstract
Objective: To determine the incidence of errors anonymously reported by general practitioners in NSW.
Design: The Threats to Australian Patient Safety (TAPS) study used anonymous reporting of errors by GPs via a secure web-based questionnaire for 12 months from October 2003.
Setting: General practices in NSW from three groupings: major urban centres (RRMA 1), large regional areas (RRMA 2–3), and rural and remote areas (RRMA 4–7).
Participants: 84 GPs from a stratified random sample of the population of 4666 NSW GPs — 41 (49%) from RRMA 1, 22 (26%) from RRMA 2–3, and 21 (25%) from RRMA 4–7. Participants were representative of the GP source population of 4666 doctors in NSW (Medicare items billed, participant age and sex).
Main outcome measures: Total number of error reports and incidence of reported errors per Medicare patient encounter item and per patient seen per year.
Results: 84 GPs submitted 418 error reports, claimed 490 864 Medicare patient encounter items, and saw 166 569 individual patients over 12 months. The incidence of reported error per Medicare patient encounter item per year was 0.078% (95% CI, 0.076%–0.080%). The incidence of reported errors per patient seen per year was 0.240% (95% CI, 0.235%–0.245%). No significant difference was seen in error reporting frequency between RRMA groupings.
Conclusions: This is the first study describing the incidence of GP-reported errors in a representative sample. When an anonymous reporting system is provided, about one error is reported for every 1000 Medicare items related to patient encounters billed, and about two errors are reported for every 1000 individual patients seen by a GP.