The effectiveness of clinical practice guidelines continues to be debated, with an editorial in the Journal suggesting that "guidelines have only modest effects in practice".1 However, there is good evidence that guidelines can be effective if they are developed and implemented appropriately.2 For example, involving the clinicians who will use the guidelines in their development is more likely to promote a sense of ownership and has the greatest chance of changing clinical behaviour.3
Two-thirds of patients presenting to emergency departments (EDs) with chest pain are admitted to hospital, with only 15% confirmed to have had an acute myocardial infarction (AMI).4 Of the patients not admitted, it is estimated that up to 5% have a missed AMI, and that these patients have four times greater mortality than admitted patients.5 To reduce the risk of missing AMIs while minimising unnecessary hospital admissions, the National Health and Medical Research Council (NHMRC) developed clinical practice guidelines that recommend using risk-stratification algorithms to triage patients with chest pain.6 The risk-stratification principle is consistent with an emerging focus on the safety of the management strategy rather than establishing a diagnosis.7,8
The NHMRC unstable angina guidelines were adapted to local circumstances in southwestern Sydney, New South Wales, by the Coronary Heart Disease Advisory Committee of the South Western Sydney Area Health Service, which included cardiologists, emergency medicine physicians and cardiac nurses. This study aimed to evaluate the effects of introducing this chest-pain guideline on clinical decision-making and on the medium-term outcomes of patients presenting to EDs with chest pain.
All ED staff (doctors, nurses and clerks), cardiology staff (junior and senior doctors) and other hospital doctors who might work in the ED were informed about the objectives of the study by the medical ED and cardiology directors.
To assess decision-making by ED doctors, all doctors working in the ED were asked to complete a one-page chest-pain assessment form for all patients who presented to the ED with non-traumatic chest pain, beginning in September 2000. The form was attached to the ED medical record and was to be completed by the ED doctor who initially saw the patient. It asked about features of the chest pain, cardiac risk factors, physical findings, electrocardiogram changes, diagnostic tests performed, diagnosis made (non-cardiac pain, possibly cardiac pain, or cardiac pain, subdivided into angina, unstable angina or acute myocardial infarction), admission destination, and seniority of the doctor who made the management decisions. Interns and resident medical officers were classified as "junior staff", and registrars, career medical officers and specialists as "senior staff".
The chest-pain guideline introduced into the ED and cardiology departments comprised a one-page risk-stratification flow chart defining groups at "high", "medium" and "low" risk for death or myocardial infarction within six months, with admission/discharge recommendations (Box 1). During the post-implementation phase (mid-February to August 2001), this flow chart was printed on the reverse of the chest-pain assessment form. A range of strategies were used to ensure guideline uptake: 9,10
The clinical directors of cardiology and EDs ("opinion leaders") involved in development of the local guidelines promoted their implementation.
The guidelines and risk-stratification flow chart were formally presented at hospital medical grand rounds and teaching sessions for junior medical officers.
Multidisciplinary educational sessions about the guideline and the appropriateness of chest-pain risk-stratification were held throughout the implementation phase for ED, cardiology and other hospital staff involved in management of chest pain in the ED.
Monthly feedback on the percentage of patients who had the guideline flow chart used was provided on noticeboards and at regular clinical meetings.
Post-implementation, ED medical staff were asked by the ED and cardiology directors to use the chest-pain guideline flow-chart to inform their admission/discharge decisions, as well as to complete the chest-pain assessment form.
To determine proportions of patients with non-traumatic chest pain who had a completed chest-pain assessment form, ED electronic records were searched for all patients with chest pain as a symptom. Those with a diagnosis of injury recorded in the ED database were excluded (ICD-10-AM diagnosis codes S00–T9811). To validate the accuracy of the derived denominators, the study research officer manually reviewed 100 consecutive medical records of patients with non-traumatic chest pain. High agreement between the electronic data and medical records was found for the main symptom (89%), principal diagnosis (79%) and destination of the patient (93%).
Admission/discharge decisions were compared before and after guideline implementation for patients assessed as low, medium and high risk by the cardiologist.
Statistical analyses used SAS12 or StatsDirect13 statistical software. Comparisons before and after guideline implementation were made by χ2 tests (for categorical outcome variables and proportions) and t tests (for continuous and normally distributed variables, such as age and length of stay). Logistic regression analysis was used to determine the predictors of appropriate admission/discharge decisions.
Chest-pain assessment forms were completed for 422 of 768 patients (54.9%) who presented to the ED with non-traumatic chest pain before implementation and for 461 of 691 patients (66.7%) who presented after implementation.
Characteristics of patients with complete assessment forms are shown in Box 2. There were no significant differences between these patients and those who presented with non-traumatic chest pain but did not have completed forms in mean age, hospital admissions, hospital bed destination or mean length of stay in either study phase (data not shown).
Among patients with complete assessment forms, there were no significant differences pre- and post-implementation in age (59.9 years pre-implementation v 60.8 years post-implementation) or sex distribution (60% men pre-implementation v 56% post-implementation). Most patients were assessed by the senior cardiologist as being at moderate or high risk of acute myocardial infarction and death, with no significant difference between study phases.
However, the proportion of patients who were diagnosed with cardiac or possibly cardiac pain was significantly higher post-implementation (increasing from 63% to 70%), as was the proportion of patients assessed by junior medical staff (increasing from 42% to 53%) and the proportion admitted to hospital (increasing from 71% to 79%). There were no significant differences for monitored/non-monitored bed destinations or overall mean length of stay between the two study phases.
Appropriateness of admission/discharge decisions for patients with cardiac or possibly cardiac pain is shown in Box 3. Appropriate decisions increased significantly from 68% of decisions pre-implementation to 81% post-implementation. Most of the change was attributable to a significant decrease in underadmission decisions, from 20% to 10%.
Significant increases in appropriate decisions were seen for both junior and senior staff. Although appropriate decisions increased for patients in all risk categories, the change was statistically significant only for patients in the moderate-risk group.
The effects of patient and doctor characteristics on the appropriateness of decisions were examined simultaneously with the effects of guideline implementation using logistic regression. Patient age, sex and risk factors for cardiovascular disease were not significant predictors, leaving guideline implementation, patient history of coronary disease and seniority of the assessing doctor as the only independent predictors of appropriate decisions in the final model (Box 4).
Telephone follow-up was successful in 319 of 422 patients pre-implementation and 349 of 461 patients post-implementation (both 76%). The remaining patients either refused follow-up (21 pre-implementation and 29 post-implementation, respectively) or were unable to be contacted because of wrong or missing telephone numbers or no answer after five attempts. There were no significant differences between those who were followed up and those who were not in mean age, sex distribution, and percentage with cardiac or possibly cardiac pain in both study phases (not shown).
Outcomes at three-month follow-up are shown in Box 5. There were 11 deaths pre-implementation, six from a cardiovascular cause (five in the cardiac/possibly cardiac pain group and one in the non-cardiac pain group). Post-implementation, there were seven deaths, two from a cardiovascular cause (both in the cardiac/possibly cardiac pain group).
While a similar proportion of patients reported experiencing chest pain in both study phases, re-presentations to the ED with recurrent chest pain in patients with cardiac and possibly cardiac pain declined significantly, from 23% pre- to 13% post-implementation. The rate of ED re-presentations for patients with non-cardiac pain was low in both study phases (3% and 2%, respectively). Most hospital readmissions were in the cardiac/possibly cardiac chest pain group, with no significant difference between study phases.
Implementation of our guideline resulted in significant improvement in appropriate admission/discharge decisions for patients with chest pain, most improvement being in patients at moderate risk of death or myocardial infarction within six months. Independent predictors of appropriate decisions were implementation of the guideline, degree of seniority of the assessing doctor and patient history of coronary disease. Guideline implementation resulted in an 8% increase in hospital admissions, counterbalanced by a 10% decrease in re-presentations to the ED with cardiac and possibly cardiac pain.
The strengths of our study were that it was prospective, used blinded "gold standard" assessment of appropriateness of decisions, assessed secondary endpoints in follow-up, and used logistic regression to define factors related to appropriate decisions.
However, study limitations need to be considered in evaluating the results. The chest-pain assessment form was our main data source, rather than patient examination and interview. The relatively low completion rates for this form and loss of about a third of patients with completed forms to follow-up raise questions about the validity of the results. However, there were no significant differences in patient demographics, hospital admission rates, bed destination or hospital length of stay between patients with or without completed forms in either study phase. Nor were there significant differences in patient demographics or prevalence of cardiac or possibly cardiac pain between patients with and without follow-up. This suggests that the low response and follow-up rates were unlikely to have affected our findings.
We used a before-and-after study design rather than a randomised controlled trial for logistical reasons, owing to the difficulty in blinding our medical workforce to patient status, and to allow in-depth analysis of our intervention. Every effort was made to minimise biases. In particular, the senior cardiologist was blinded to all patient data, study phase and outcomes, had minimal direct patient contact, and undertook his assessment at the end of the study using a consistent method.
ED doctor decisions disagreed with those of the cardiologist in 45% of patients assessed at moderate risk after guideline implementation. Most high-risk patients are easily recognised on the basis of history, resting electrocardiogram, and cardiac enzyme levels.14 The situation is less clear for moderate-risk patients. Our results may be due to the failure of ED staff to recognise that patients with multiple coronary risk factors and possibly cardiac chest pain have an intermediate risk of adverse events. For accurate risk assessment, they need observation over eight to 12 hours with serial ECGs and cardiac enzyme testing, as highlighted in a previous study.15
It has been suggested that the introduction of chest-pain risk-stratification guidelines might not decrease hospital admissions.16 However, other studies have shown ED chest pain protocols to be cost effective, with shorter hospital stays.17,18 The rise in hospital admissions in our study did not result in longer hospital stay and was counterbalanced by a significant decrease in the rate of re-presentation to the ED with chest pain.
Our study shows that adapting a national chest-pain guideline to local needs can improve clinical decision-making for patients with chest pain, with improved medium-term patient outcomes. To sustain these gains, we plan to review and refine the guidelines, with particular emphasis on the contentious moderate-risk group.
1: South Western Sydney Area Health Service chest pain risk-stratification guideline
LVF = left ventricular failure. ECG = electrocardiogram. LBBB = left bundle branch block. CKMB = creatine kinase MB isoenzyme. IHD = ischaemic heart disease. LMO = local medical officer. IV = intravenous. * Of death or myocardial infarction within six months.
2: Characteristics and admission destinations of patients with complete chest-pain forms before and after guideline implementation
Characteristic |
Pre-
implementation |
Post-
implementation |
Difference* (95% CI) |
||||||||
All patients |
|
|
|
||||||||
Mean age, in years (SD) |
59.9 (17.3) |
60.8 (17.8) |
0.92 (-1.42 to 3.22) |
||||||||
Men |
254 (60%) |
258 (56%) |
-4% (-11% to 2%) |
||||||||
Patients with cardiac/possibly cardiac pain |
265/422 (63%) |
324/461 (70%) |
8% (1% to 14%) |
||||||||
Mean age, in years (SD) |
63.3 (15.7) |
64.3 (16.0) |
1.0 (-1.4 to 3.6) |
||||||||
Men |
169 (64%) |
185 (57%) |
-7% (-15% to 1%) |
||||||||
Assessed by junior medical staff |
110 (42%) |
173 (53%) |
12% (4% to 20%)† |
||||||||
Risk stratification (by cardiologist) |
|
|
|
||||||||
Low risk |
27 (10%) |
42 (13%) |
3% (-3% to 8%) |
||||||||
Medium risk |
96 (36%) |
103 (32%) |
-4% (-12% to 3%) |
||||||||
High risk |
142 (54%) |
179 (55%) |
2% (-6% to 10%) |
||||||||
Admitted to |
|
|
|
||||||||
Hospital |
189 (71%) |
256 (79%) |
8% (0.7% to 15%) |
||||||||
Coronary care unit |
40 (21%) |
51 (20%) |
-1% (-9% to 6%) |
||||||||
Subacute bed |
124 (66%) |
157 (61%) |
-4% (-13% to 5%) |
||||||||
Non-monitored bed |
25 (13%) |
48 (19%) |
6% (-2% to 12%) |
||||||||
Mean days of stay (SD) |
5.1 (4.5) |
5.0 (4.6) |
0.1 (-0.8 to 1.0) |
||||||||
* Post-implementation value – pre-implementation value. † Difference is significantly different from zero. |
|||||||||||
3: Appropriateness of admission/discharge decisions for patients with cardiac or possibly cardiac pain compared with the "gold standard"*
Decision |
Pre- implementation (n = 265) |
Post- implementation (n = 324) |
Difference in %† (95% CI) |
||||||||
Appropriate decision |
180 (68%) |
261 (81%) |
13% (6% to 20%)‡ |
||||||||
Underadmission |
54 (20%) |
32 (10%) |
-10% (-16% to -5%)‡ |
||||||||
Overadmission |
31 (12%) |
31 (10%) |
-2% (-7% to 3%) |
||||||||
Appropriate decision, by staff seniority |
|
|
|
||||||||
Junior staff |
63/110 (57%) |
131/173 (76%) |
18% (7% to 30%)‡ |
||||||||
Senior staff |
117/155 (75%) |
130/151 (86%) |
11% (2% to 19%)‡ |
||||||||
Appropriate decision, by patient risk |
|
|
|||||||||
Low risk |
22/27 (82%) |
40/42 (95%) |
14% (-1% to 33%) |
||||||||
Medium risk |
36/96 (38%) |
57/103 (55%) |
18% (4% to 31%)‡ |
||||||||
High risk |
122/142 (86%) |
164/179 (92%) |
6% (-1% to 13%) |
||||||||
* Decision made by senior cardiologist based on risk stratification using the chest-pain clinical guideline.
|
|||||||||||
4: Predictors of appropriate admission/discharge decisions
Category |
Adjusted odds ratio (95% CI) |
||||||||||
Guideline-implementation |
|||||||||||
No* |
1 |
||||||||||
Yes |
2.12 (1.43–3.13) |
||||||||||
Assessing doctor |
|
||||||||||
Junior doctors* |
1 |
||||||||||
Senior doctors |
2.12 (1.43–3.14) |
||||||||||
History of coronary heart disease |
|
||||||||||
No* |
1 |
||||||||||
Yes |
1.54 (1.03–2.29) |
||||||||||
* Reference category. |
|||||||||||
5: Outcomes at three-month follow-up for patients with different categories of chest pain
|
Cardiac/possibly cardiac pain |
Non-cardiac pain |
|||||||||
Outcome |
Pre- implementation (n = 201) |
Post- implementation (n = 247) |
Difference in %*
|
Pre- implementation (n = 118) |
Post- implementation (n =102) |
Difference in %* |
|||||
Further chest pain |
64 |
73 |
-2% |
17 |
20 |
5% |
|||||
Re-presentation to ED |
46 |
32 |
-10% |
4 |
2 |
-1% |
|||||
Readmission to hospital |
36 |
31 |
-5% |
3 |
2 |
-0.6% |
|||||
All deaths |
8 |
5 |
-2% |
3 |
2 |
-0.6% |
|||||
Cardiovascular deaths |
6 |
2 |
-2% |
1 |
0 |
0.8% |
|||||
ED = emergency department. * Post-implementation value – pre-implementation value. † P = 0.006 for fewer re-presentations to the ED for patients who originally presented with cardiac/possibly cardiac pain post-implementation. † Difference is significantly different from zero. |
|||||||||||
Received 26 July 2002, accepted 13 November 2002
- Soufiane Boufous1
- Bin B Jalaludin2
- Charles H Pain3
- Susan Ieraci4
- Anne-Louise Gray5
- Susan E Harris6
- Craig P Juergens7
- Peter W Kelleher8
- Linda M Dann9
- 1 South Western Sydney Area Health Service, Sydney, NSW.
- 2 Bankstown–Lidcombe Hospital, Bankstown, NSW.
We acknowledge the financial support for the study project officer from the Health Research Foundation Sydney South West and the South Western Sydney Area Health Service. We thank Professor Jeanette Ward (Division of Population Health, South Western Sydney Area Health Service, NSW) for helpful comments on earlier drafts of this manuscript.
None identified.
- 1. Thompson PL. Time to move beyond clinical practice guidelines [editorial]? Med J Aust 2001; 174: 211-212.
- 2. Pain C. Time to move beyond clinical practice guidelines [letter]? Med J Aust 2001; 175: 287.
- 3. Grimshaw JM, Russel IT. Implementing clinical practice guidelines: can guidelines be used to improve clinical practice? Effective Health Care 1994; 8: 1-12.
- 4. Lee TH, Rouan GW, Weisberg M, et al. Sensitivity of routine clinical criteria for diagnosing myocardial infarction within 24 hours of hospitalisation. Ann Intern Med 1987; 106: 181-186.
- 5. Herren KR, Mackway-Jones K, Richards CR, et al. Diagnostic cohort study of an emergency department based 6 hour rule-out protocol for myocardial damage. BMJ 2001; 323: 1-4.
- 6. Diagnosis and management of unstable angina. Canberra: National Health and Medical Research Council, 1998.
- 7. Nichol G, Walls R, Goldman L, et al. a critical pathway for management of patients with acute chest pain who are at low risk for myocardial ischemia: recommendations and potential impact. Ann Intern Med 1997; 127: 996-1005.
- 8. Lee TH, Cook EF, Weisberg M, et al. Acute chest pain in the emergency room. Identification and examination of low-risk patients. Arch Intern Med 1985; 145: 65-69.
- 9. A guide to the development, implementation and evaluation of clinical practice guidelines. Canberra: National Health and Medical Research Council; 1998.
- 10. Jankowski RF. Implementing national guidelines at local level. BMJ 2001; 322: 1258-1259.
- 11. National Centre for Classification in Health. The international statistical classification of diseases and related health problems, 10th revision, Australian modification (ICD-10-AM). Sydney: National Centre for Classification in Health,1998.
- 12. SAS Institute. SAS statistical software. Version 8.02. Cary (NC): SAS Institute Inc, 2000.
- 13. Stats Direct Statistical Software, Version 2.0. Cambridge: Stats Direct Ltd, 2001.
- 14. Capewell S, McMurray J. "Chest pain — please admit": is there an alternative? BMJ 2000; 320: 951-952.
- 15. Fitzpatrick MA, Dodd M, Shoevers D, Tracy E. Do management algorithms improve chest pain triage? Med J Aust 1999; 171: 402-406.
- 16. Katz DA, Griffith JL, Beshansky JR, Selker HP. The use of empiric clinical data in the evaluation of practice guidelines for unstable angina. JAMA 1996; 276: 1568-1574.
- 17. Roberts RR, Zalenski RJ, Menash EK, et al. Costs of an emergency department-based accelerated diagnostic protocol vs hospitalisation in patients with chest pain: a randomised controlled trial. JAMA 1997; 278: 1670-1676.
- 18. Gomez MA, Anderson JL, Karagounis LA, et al. An emergency department-based protocol for rapidly ruling out myocardial ischemia reduces hospital time and expense: results of a randomized study (ROMIO). J Am Coll Cardiol 1996; 28: 25-33.
Abstract
Objective: To evaluate the impact of a chest-pain guideline on clinical decision-making and medium-term outcomes of patients presenting to a hospital emergency department (ED) with non-traumatic chest pain.
Design: Before-and-after guideline implementation study.
Setting: Bankstown–Lidcombe Hospital, Sydney, NSW (454-bed metropolitan teaching hospital), in the six-month periods before and after guideline implementation in February 2001.
Participants: Patients presenting to the ED with non-traumatic chest pain who had chest-pain assessment forms completed by ED doctors, comprising 422/768 (54.9%) of those presenting before and 461/691 (66.7%) after guideline implementation.
Main outcome measures: Appropriateness of admission/discharge decisions compared with decision of senior cardiologist based on guideline; death, recurrent chest pain, ED re-presentation and hospital readmission in the ensuing three months.
Results: After guideline implementation, appropriate admission/discharge decisions increased significantly from 180/265 (68%) to 261/324 (81%) (difference, 13%; 95% CI, 6%–20%). The largest increase was for patients at moderate risk of death or acute myocardial infarction within six months, from 39/96 (38%) to 57/103 (55%) (difference, 18%; 95% CI, 4%–31%). Increases were seen for both junior doctors (interns and resident medical officers) (18%; 95% CI, 7%–30%) and senior doctors (11%; 95% CI, 2%–19%). Logistic regression showed that implementation of the guideline, seniority of assessing doctor and patient history of coronary disease were independent predictors of appropriate decisions. There was a significant decline in re-presentations to ED with recurrent chest pain in patients previously presenting with cardiac or possibly cardiac pain, from 46/201 (23%) before implementation to 32/247 (13%) after (difference, 210%; 95% CI, 217% to 23%).
Conclusions: The chest-pain guideline resulted in a significant improvement in clinical decision-making in the ED and reduced re-presentations with cardiac/possibly cardiac chest pain.