Investigations are essential to the clinical practice of medicine, fulfilling roles in screening, diagnosis and monitoring of disease. The wide variation in test-ordering, particularly when tests are used for diagnostic purposes, suggests that some tests are unnecessary or ordered inappropriately.1-4 As investigations account for a significant proportion of total healthcare costs,5,6 unnecessary tests are a waste of resources and an opportunity cost to the community.
The checking and follow-up of a test result is an important part of patient management and critical to effective risk management.7 The failure to follow up such results can cause patient harm and poses a significant medicolegal risk for the responsible clinician or institution.8 The large number of diagnostic tests ordered in the emergency department and the difficulties involved in the continuity of patient care mandate the need for an effective follow-up system.
We developed and evaluated an intervention aimed at improving the utilisation and follow-up of diagnostic investigations in the emergency department of a public teaching hospital.
A qualitative approach based on standard behavioural theory was used in developing the intervention. The PRECEDE (Predisposing, Reinforcing, Enabling, Causes in Educational Diagnosis and Evaluation) model was selected because it provides a practical framework for understanding behaviour modification in the context of test ordering by clinicians.9 The model classifies factors that assist or inhibit behaviour change into three groups: predisposing, enabling and reinforcing. Predisposing factors consist of pre-existing attitudes or knowledge that support test-ordering practice. Enabling factors comprise individual skills, available resources and structural barriers that assist or inhibit appropriate test-ordering behaviour. Reinforcing factors relate to feedback that can positively or negatively influence practice.
The intervention reflected the core elements of the PRECEDE model and comprised three integrated components:
an education program for medical staff (addressing predisposing factors);
implementation of a protocol for test ordering (addressing enabling factors); and
an audit/feedback process (addressing reinforcing factors).
The education program raised awareness of the issue of overutilisation of tests. A training program introduced staff to the test-ordering protocol developed for the study and facilitated discussion and comment. The program took 30–45 minutes to complete and was delivered to groups of three or four medical officers by the emergency department consultant responsible for the project. All emergency department medical staff received the training. An abbreviated version of the program was included in the orientation for new medical staff.
A structured protocol for ordering tests was developed. The protocol divided clinical investigations into three categories (Box 1).
Category 1 tests included commonly used tests for which the result was available within two hours. No restriction was placed on ordering these tests, but the medical officer was obliged to ensure that the results were checked and documented in the case notes before the patient was discharged.
Category 2 tests were investigations that were commonly overused (eg, coagulation studies, lumbosacral spine x-rays), could not be checked before the patient was discharged (eg, blood cultures, HIV serology, routine drug levels), or had limited indications (eg, urgent computed tomography scan or ultrasound). Authorisation was required from the emergency department registrar or consultant for these tests. An evidence-based list of clinical indications for ordering these tests was developed, largely based on a published list.10
Category 3 tests were tests for which the result was not immediately available and a delay of 24–48 hours would not affect clinical care. These tests could not be ordered for emergency patients.
Liaison with laboratory and imaging departments enabled the development of a feedback process that ensured breaches of the protocol could be readily identified. Each day, a random sample of case notes was audited to ensure the documentation of test results by medical staff.
The intervention was studied prospectively from May 1999 to October 2000, and the results were compared with a six-month pre-intervention period (November 1998 to April 1999).
Test utilisation was the primary outcome measure for the study. Cost (calculated by multiplying the number of tests by the relevant MBS item fee) was chosen as the indicator for test utilisation, as this facilitated comparisons between groups and allowed the changes to be related to the overall laboratory or imaging budgets. To account for possible variation in patient loads and facilitate meaningful comparison, all primary outcome measures were adjusted, dividing the monthly cost by the (monthly) patient load, resulting in a cost (in dollars) per patient for each variable.
Secondary outcomes were documentation of test results by treating medical officers, time for the daily audit and follow-up of test results, adverse patient outcomes due to test-ordering practice, and cost of investigations in the other hospital divisions (to detect possible cost-shifting).
Analysis compared the cost of investigations for six-month periods before and after implementation. To avoid seasonal bias, the six-month periods were taken at identical times during the year (November to April). The trends over the 18-month post-intervention period (May 1999 to October 2000) were examined to identify any long term impact of the study.
Primary outcome variables for the study were compared using Student's t test. Mean values, percentage difference, P value and 95% confidence intervals for the pre- and post-intervention cost/patient were calculated. Statistical analysis used the STATA 6.0 statistical package.11
Following the intervention, there was a 40% decrease in the ordering of tests in the emergency department, with test utilisation falling from a mean of $39.32/patient to $23.72/patient (Box 2). The decrease was similar for both laboratory and imaging tests. The decrease was sustained over the post-intervention period (Box 3) and represented a budget saving over the 18 months of $1 008 197 ($15.60/patient).
Category 3 tests showed greatest change. Reductions of more than 80% were noted for many of the Category 3 tests, including thyroid function tests (89%), faecal cultures (90%) and serology (94%). Decreases in excess of 50% occurred for most of the Category 2 investigations, and most of the Category 1 tests showed significant reductions.
For several radiology investigations, including cervical spine x-ray, limb x-ray, plain CT scans and upper-abdominal ultrasound, the decrease was not significant. One test, cardiac enzymes, showed a non-significant increase during the study period (Box 2).
Following the intervention, the time taken to complete the daily checking and follow-up of the test results decreased by 75%, from an average of 3.9 h/day to 57.5 min/day.
A documentation audit of 4000 case notes identified one case in which the test results had not been documented. No adverse patient outcomes relating to underutilisation of investigations attributable to the protocol were identified.
There was a post-intervention decrease in the adjusted costs of investigations for the Division of Surgery (0.6% decrease) and the Division of Medicine (12% decrease).
We developed and implemented an intervention for reducing test ordering in a public hospital emergency department. A sharp decline in test ordering occurred immediately following the implementation of the program and was sustained throughout the 18-month follow-up. The reduction was similar for laboratory tests and imaging studies, with each showing an overall decrease of 40%. Microbiology and "other" imaging (CT scan, ultrasound) subcategories showed the largest decline.
A few of the tests we examined did not show significant changes. Most of these were already subject to clinical protocols (eg, cervical spine x-ray, selected limb x-ray, CT head). One test category, cardiac enzymes, showed a non-significant increase. This increase is attributed to the introduction of troponin I isoenzyme testing shortly after the implementation of the protocol (with a period in which both creatine kinase and troponin I were being ordered) and to decreased coronary care unit bed availability, which led to longer stays in the emergency department (and repeat troponin I requests).
Studies of interventions to modify test ordering frequently report marked reductions in the ordering of tests immediately following the intervention. Maintaining the changes has, however, proved problematic, with test ordering often returning to pre-intervention levels after a short period.12,13 In our study, the change was sustained for the 18-month follow-up period, suggesting that behaviour might have been structurally modified as a result of the intervention. Previous research has suggested that the most effective programs for facilitating long term changes involve interventions that target a range of behavioural factors and incorporate an environmental or administrative intervention.14 Our study exemplifies the success of this approach — we identified key factors for assisting or inhibiting behaviour modification and developed strategies for addressing each of the factors.
Many suggested interventions for modifying test ordering require significant funding (eg, recruitment of a project officer, delivery of training programs, development of educational material, hosting of consensus meetings or provision of feedback). These funding issues are an obstacle for implementation and are likely to place at risk the long term sustainability of a program, despite the potential for savings from reduced test ordering. Our program was developed as part of the routine quality assurance process for the emergency department. Although requiring enthusiasm from the staff in the emergency department and support from the wider hospital, the program imposed no additional costs on the hospital.
The major limitation to our study relates to whether patient outcomes are improved by having fewer inappropriate or unnecessary tests. It seems reasonable that a greater awareness by medical staff of the role of investigations and the improvements to test documentation and follow-up are likely to result in improvements to patient care. It could also be argued that, as there were no identified adverse outcomes due to the intervention, ordering fewer tests did not influence patient outcome. However, we did not formally assess test underutilisation, a major concern with any intervention for reducing test ordering. Few studies have attempted to address this highly complex issue, and further research is required before firm conclusions can be drawn.15
Our model could be applied across a range of departments in the hospital and could be integrated into software for electronic ordering of investigations.16-18 Of particular interest to administrators is the potential of the model not only to improve the follow-up of test results (reducing potential medicolegal exposure), but to deliver significant savings, enabling resources to be redirected to other critical areas of service delivery.
1: Summary of the test-ordering protocol
Category 1 tests: No restrictions to ordering All results must be checked before discharge |
|||
Blood tests
|
Imaging: Plain x-rays
Miscellaneous
|
||
Category 2 tests: Ordering criteria must be met and follow-up arranged Emergency department senior medical officer approval required |
|||
Blood tests
Miscellaneous
|
Microbiology
Imaging
|
||
Category 3 tests: May not be ordered for emergency patients. Refer patient to GP or arrange tests as an outpatient |
|||
Blood tests
|
Microbiology and cytology
Imaging
|
||
DIC = disseminated intravascular coagulation. EUC = electrolyte, urea, creatinine. FSH = follicle-stimulating hormone. βhCG = β-human chorionic gonadotropin. LFT = liver function tests. LH = luteinising hormone. MC&S = microscopy, culture and sensitivity. PTH = parathyroid hormone. V/Q = ventilation/perfusion. IVP = intravenous pyelogram. |
|||
2: Comparison of the mean costs per patient for emergency department investigations pre-intervention (November 1998 to April 1999) and post-intervention (November 1999 to April 2000).
Pre ($/patient) |
Post ($/patient) |
% Difference (95% CL) |
P |
||||||||
Total: all investigations |
39.32 |
23.72 |
–40% |
(–29% –50%) |
<0.001 |
||||||
Total laboratory costs |
23.100 |
13.935 |
–40% |
(–30%, –50%) |
<0.001 |
||||||
Haematology |
9.015 |
6.072 |
–33% |
(–25%, –40%) |
<0.001 |
||||||
Complete blood picture |
4.84 |
3.92 |
–19% |
(–11%, –27%) |
<0.001 |
||||||
Coagulation study |
1.71 |
0.84 |
–51% |
(–39%, –63%) |
<0.001 |
||||||
Blood typing/screening |
2.32 |
1.17 |
–50% |
(–34%, –65%) |
<0.001 |
||||||
Chemistry |
8.428 |
5.743 |
–32% |
(–24%, –40%) |
<0.001 |
||||||
Biochemistry (EUC/LFT) |
5.93 |
4.49 |
–24% |
(–16%, –24%) |
<0.001 |
||||||
Drug level |
0.245 |
0.077 |
–69% |
(–53%, –85%) |
<0.001 |
||||||
Thyroid function |
0.426 |
0.047 |
–89% |
(–55%, –123%) |
0.001 |
||||||
Cardiac enzymes |
0.093 |
0.19 |
+114% |
(–20%, +237%) |
0.084 |
||||||
Iron studies |
0.079 |
0.016 |
–80% |
(–47%, –113%) |
<0.001 |
||||||
Microbiology |
5.125 |
1.221 |
–76% |
(–66%, –86%) |
<0.001 |
||||||
Blood culture |
1.26 |
0.246 |
–80% |
(–63%, –98%) |
<0.001 |
||||||
Urine MC&S |
1.03 |
0.461 |
–55% |
(–37%, –73% |
<0.001 |
||||||
Serology |
1.04 |
0.062 |
–94% |
(–71%, –118%) |
<0.001 |
||||||
Faeces MC&S |
0.372 |
0.038 |
–90% |
(–65%, –115%) |
<0.001 |
||||||
Vaginal MC&S |
0.298 |
0.0817 |
–73% |
(–32%, –113%) |
0.006 |
||||||
Wound/skin/eye swab |
0.115 |
0.033 |
–71% |
(–55%, –88%) |
<0.001 |
||||||
Total imaging costs |
16.22 |
9.78 |
–40% |
(–21% –58%) |
0.002 |
||||||
Plain x-rays |
10.47 |
8.51 |
–19% |
(–1.5% –36%) |
0.037 |
||||||
Chest |
4.09 |
3.31 |
–19% |
(–11%, –37%) |
0.040 |
||||||
Abdomen |
1.47 |
0.967 |
–34% |
(–14%, –55%) |
0.004 |
||||||
Skull |
0.257 |
0.0558 |
–78% |
(–52%, –104%) |
<0.001 |
||||||
Lumbosacral spine |
0.199 |
0.0928 |
–53% |
(–22%, –84%) |
0.004 |
||||||
Cervical spine |
0.482 |
0.445 |
–8% |
(–35%, +20%) |
0.547 |
||||||
Limb |
2.98 |
2.83 |
–5% |
(–23% +13%) |
0.563 |
||||||
Other imaging |
5.7 |
1.27 |
–78% |
(–55%, –101%) |
<0.001 |
||||||
Computed tomography head (plain) |
1.90 |
1.55 |
–19% |
(–55%, +18%) |
0.276 |
||||||
Intravenous pyelogram |
0.38 |
0.176 |
–54% |
(–25%, –82%) |
0.002 |
||||||
Ultrasound (upper abdomen) |
0.281 |
0.273 |
–3% |
(–38%, + 32%) |
0.854 |
||||||
Ultrasound (female pelvis) |
0.553 |
0.358 |
–35% |
(–22%, –84%) |
0.045 |
||||||
Received 20 August 2001, accepted 28 February 2002
- Peter J Stuart1
- Shelley Crooks2
- Mark Porton3
- Lyell McEwin Health Service, Elizabeth Vale, SA.
None identified.
- 1. Wennberg JE. Dealing with medical practice variations: a proposal for action. Health Aff (Millwood) 1984; 3: 6-32.
- 2. Daniels M, Schroeder SA. Variation among physicians in use of laboratory tests II: relation to clinical productivity and outcomes of care. Med Care 1977; 15: 482-487.
- 3. Grantham P, Weinstein S. Reducing pathology test misuse. Aust Health Rev 1993; 16: 16-23.
- 4. Dickinson JC. From process to policy: a generic prescription for test over-utilisation in the emergency department. Fam Pract Res J 1987; 7: 12-21.
- 5. Australian Institute of Health and Welfare. Australia's Health 2000. Canberra: AIHW, 2000; 298-300. Available at <http://www.aihw.gov.au/publications/index.cfm/title/5742>. (Link updated Feb 2007)
- 6. Conyers R. Modifying use of pathology services. Med J Aust 1999; 170: 8-9. <MJA full text>
- 7. Wilson LL, Fulton M. Risk management: how doctors, hospitals and MDOs can limit the costs of malpractice litigation. Med J Aust 2000; 172: 77-80. <MJA full text>
- 8. Henry GL. Dr. Risk management and diagnostic testing. In: Cantrill S, Karas S, editors. Cost-effective diagnostic testing in emergency medicine. 2nd ed. Dallas: American College of Emergency Physicians, 2000; 26.
- 9. Solomon D, Hashimoto H, Daltroy L, Liang M. Techniques to improve physicians' use of diagnostic tests: a new conceptual framework. JAMA 1998; 280: 2020-2027.
- 10. Cantrill S, Karas S, editors. Cost-effective diagnostic testing in emergency medicine. 2nd ed. Dallas: American College of Emergency Physicians, 2000.
- 11. STATA statistical package [computer program]. Version 6.0. College Station, Tx: Stata Corporation, 1999.
- 12. Isouard G. A quality management intervention to improve clinical laboratory use in acute myocardial infarction. Med J Aust 1999; 170: 11-14. <MJA full text>
- 13. Eisenberg JM. An educational program to modify laboratory use by house staff. J Med Educ 1977; 52: 578-581.
- 14. Lundberg GD. Changing physician behaviour in ordering diagnostic tests [editorial]. JAMA 1998; 280: 2036.
- 15. Kroenke K, Hanley JF, Copely JB, et al. Improving house staff ordering of three common laboratory tests. Reductions in test ordering need not result in underutilisation. Med Care 1987; 25: 928-935.
- 16. Nightingale PG, Peters M, Mutimer D, Neuberger JM. Effects of a computerised protocol management system on the ordering of clinical tests. Qual Health Care 1994; 3: 23-28.
- 17. Chin HL, Wallace P. Embedding guidelines into direct physician order entry: simple methods, powerful results. Proc AMIA Symp 1999; 221-225.
- 18. Tjahjono D, Kahn CE Jr. Promoting the online use of radiology appropriateness criteria. Radiographics 1999; 19: 1673-1681.
Abstract
Objective: To evaluate an intervention developed to improve test-ordering practice.
Setting: Public hospital emergency department with an annual census of 42 500. The study comprised a six-month pre-intervention stage (November 1998 to April 1999), which was compared with a similar post-intervention period (November 1999 to April 2000), and trends were examined over an 18-month post-intervention period (May 1999 to October 2000).
Intervention: The intervention comprised three integrated components: implementation of a protocol for test ordering; education program for medical staff; and audit/feedback process.
Main outcome measure: Test utilisation (assessed as cost per patient).
Results: There was a 40% decrease in the ordering of investigations in the emergency department (95% CI, 29%–50%), with test utilisation falling from a mean of $39.32/patient to $23.72/patient. The decrease was similar for both laboratory and imaging tests and was sustained for the duration of the 18-month follow-up.
Conclusions: Our intervention appears to have produced long term modification of test ordering in the emergency department of a public teaching hospital.