Implementing change: evaluating the Accelerated Chest pain Risk Evaluation (ACRE) project

Chest pain is among the most common primary problems with which patients present to emergency departments (EDs) in Australia.1 Causes of acute chest pain include pulmonary embolus and aortic dissection, but by far the most common life-threatening causes are acute coronary syndromes (ACS), which must accordingly be investigated in all patients presenting with chest pain. Minimising the risk of missing a serious condition while recognising that most patients do not have serious underlying causes for their symptoms constitutes a management dilemma for clinicians.2 Consensus guidelines have traditionally adopted a conservative approach, recommending clinical pathways that include comprehensive diagnostic assessment at the time of the initial presentation, and this often requires admitting the patient to hospital.3

The 2006 guidelines of the National Heart Foundation and the Cardiac Society of Australia and New Zealand (NHF/CSANZ) recommend stratifying patients presenting with possible ACS into low, intermediate and high risk categories.3 We have previously reported that adherence to these guidelines results in a small proportion of patients satisfying the criteria for low risk (1%), while the incidence of ACS in the intermediate risk group (62.6% of patients) is less than 2%.2 These guidelines recommend that intermediate risk patients undergo evaluation for 6–8 hours when sensitive troponin assays and other early objective testing for coronary artery disease are employed. The cost of identifying one ACS event in intermediate risk patients has recently been estimated to be $174 191, compared with $31 895 in patients with high risk features.4

Accelerated diagnostic protocols (ADPs) that can improve efficiency while maintaining patient safety have been described. The ADP based on the 2-hour Accelerated Diagnostic protocol to Assess Patients with chest pain symptoms using contemporary Troponins as the only biomarker (ADAPT) study allows identification of patients at very low short term risk (30 days) of a major adverse cardiac event who could be redefined as “low risk” after clinical risk stratification, serial troponin tests, and electrocardiography (ECG) 0 and 2 hours after presentation to the ED (Box 1).5 These patients can be discharged from the ED earlier, with further care undertaken in an ambulatory setting. The reported outcomes of implementing the ADAPT ADP at a single site indicated that the strategy translated well into clinical practice.6 Other studies have reported varying proportions of patients identified with the ADAPT ADP as being at low risk.7,8

The Accelerated Chest pain Risk Evaluation (ACRE) Project was a large scale initiative that aimed to translate the ADAPT ADP into clinical practice across Queensland public hospitals. Our aims were to assess the feasibility of applying the ADP in a broad range of acute hospital settings, to quantify changes in hospital length of stay (LOS) and admission rates, and to estimate its net cost.

Methods

All public hospitals with EDs treating adult patients and with 24-hour access to laboratory-based troponin testing were eligible for inclusion in the project. Access to pathology testing was required to ensure that clinical pathways were aligned with the core elements of the ADAPT ADP. Clinical pathways were applicable to patients with possible cardiac chest pain once other serious causes had been considered and excluded. The protocol was implemented incrementally across sites between October 2013 and September 2015, as previously described in detail.9

Data collection

Data were collected from the Emergency Department Information System (EDIS; CSC Healthcare) and the Hospital Based Corporate Information System (HBCIS) by extracting reports from the Queensland Health Enterprise Reporting Service (QHERS), stratified by study site. Pre-implementation data were collected for the 12 months prior to implementation at each hospital, using a pre-existing QHERS report to obtain all ED presentations by adults for the relevant diagnosis-related groups (online Appendix, table 1). Diagnosis codes were inclusive, to maximise the probability of capturing patients with possible cardiac chest pain while excluding patients with clearly non-cardiac causes (eg, trauma).

Post-implementation EDIS data were collected between May 2014 and November 2015. Following implementation at each site, EDIS was reprogrammed to prompt clinicians to identify patients in relevant diagnosis-related groups with chest pain who were at low risk for ACS according to the ADP (using a binary code: 1 = yes, 2 = no). After implementation, a customised QHERS report containing the same parameters as the pre-implementation report was prepared, stratified by risk group (low risk v not low risk). EDIS data were linked with HBCIS inpatient records by the Queensland Health Research Linkage Group when relevant.

Outcome measures

The primary outcome was the total hospital LOS for patients presenting to an ED with chest pain. Secondary outcome measures included hospital admission rate, ED LOS, and the proportion of patients identified by the ADP as being at low risk of an ACS within 30 days.

Data analysis

Data were analysed in Stata 14 (StataCorp). For all analyses, the pre-implementation period comprised the 12 months before implementation; the post-implementation period included all available data for 12 months after implementation. Data for two weeks immediately before and after implementation were excluded to allow for transition education and implementation. Baseline demographic data for the pre- and post-implementation groups were compared. The proportion of patients assessed with the ADP was computed. We compared mean and median LOS, and the proportions of patients admitted during the pre- and post-implementation periods. The LOS variables were overdispersed, so a negative binomial distribution was applied when calculating standard errors of the mean. Robust standard errors were calculated for all 95% confidence intervals (CIs) to account for clustering of patients within hospitals.

Interrupted time series analyses ensured that any differences in outcome variables before and after implementation of ACRE were not attributable to downward trends across the entire study period. Trends in hospital LOS were estimated by Prais–Winsten regression, which accounted for serial correlation across time, while also adjusting for clustering within hospitals. The median LOS by week was regressed on time (weeks from the beginning of data collection) to assess the linear trend prior to implementing ACRE. Time after implementation (weeks) was also entered to assess any change in the trend in LOS after implementing ACRE. Study period (pre- v post-implementation) was entered as a dichotomous variable to assess whether there was a quantitative change in LOS immediately after implementing ACRE. Clustered robust standard errors accounted for clustering within hospitals. A second regression compared trends in hospital admission rates; hospital admission was regressed on time, weeks after implementation, and study period.

Economic analysis

The cost of evaluating patients with chest pain was calculated on the basis of a previous economic analysis of the Brisbane cohort of the ADAPT study.4 Change in the cost of ED care was calculated from the number of patients, the change in mean ED LOS, and the hourly cost of ED care. Change in the cost of inpatient care was calculated from the change in the number of admissions, the mean inpatient LOS before implementation, and the hourly cost of inpatient care for chest pain.

Ethics approval

The Queensland Health Human Research Ethics Committee waived review of the study (reference, HREC/13/QGC/142) as the project was a quality activity and not recognised as research according to the definition in the National Statement on Ethical Conduct in Human Research of the National Health and Medical Research Council.

Results

The ADP was implemented in 16 public hospitals with ED presentation numbers of 28 000–100 000 patients per year, including all major regional and large metropolitan hospitals and most major metropolitan hospitals in Queensland (online Appendix, table 2).

Data were collected from 30 770 patients prior to implementation and 23 702 patients after implementation. One patient was removed from the pre-implementation and three patients from the post-implementation period for implausible LOS figures, leaving 30 769 and 23 699 patients respectively. Baseline characteristics for the pre- and post-intervention cohorts were similar (Box 2). Post-implementation, 5041 patients (21.3%; 95% CI, 16.5–27.0%) presenting with possible cardiac chest pain were classified by the ADP as being at low risk for an ACS.

Total mean hospital LOS fell from 57.7 hours to 47.3 hours (rate ratio [RR], 0.82; 95% CI, 0.74–0.91). The mean ED LOS fell from 292 minutes to 256 minutes (RR, 0.88; 95% CI, 0.82–0.94). Median total hospital LOS fell by 6.1 hours (95% CI, 5.5–6.6 h), while median ED LOS fell by 20 minutes (95% CI, 17.9–22.1 min). The hospital admission rate fell from 68.3% pre-implementation to 54.9% post-implementation (RR, 0.80; 95% CI, 0.72–0.89) (Box 3). The outcomes for individual hospitals are listed in Box 4.

Box 5 depicts the median hospital LOS by week and Box 6 the hospital admission rates for the 12 months before and after the intervention. The regression of LOS on time and study period indicated that LOS was stable during the 12 months before implementation of ACRE (trend, –0.02 hours/week; 95% CI; –0.08 to 0.03 hours/week; P = 0.35). The median LOS declined after implementation (by 3.86 h; 95% CI, 0.27–7.45 h; P = 0.04). LOS stabilised 12 months after the start of the intervention (trend, –0.05 hours per week; 95% CI, –0.16 to 0.06 hours/week; P = 0.35). Inspection of the residuals identified one outlier, removal of which caused only minor changes to the regression coefficients; it was therefore retained in the analysis. Regression of admission on time and study period indicated that admission rates were stable prior to implementation (trend, –0.04% per week; 95% CI; –0.17% to 0.09% per week; P = 0.51). The proportion of patients admitted after ACRE was implemented declined (by 13.3 percentage points; 95% CI, 2.8–23.7 percentage points; P = 0.02), and remained stable during the 12 months after the intervention (trend, –0.04 percentage points per week; 95% CI; –0.28 to 0.36 percentage points per week; P = 0.80).

Economic analysis

The estimated released capacity resulting from the reduction in hospital admissions during the post-implementation period was $11.2 million; the estimated released capacity resulting from reducing ED LOS was $2.3 million.

Discussion

We report several important findings. Firstly, implementing an ADP to identify patients at very low 30-day risk of an ACS was associated with significant reductions in ED LOS, total hospital LOS, and hospital admissions. Secondly, the implementation of the ADP was feasible across a variety of hospital types, including large metropolitan centres and smaller regional hospitals; the proportion of patients defined as low risk was comparable with that in the original observational study in which the ADP was developed. Thirdly, a widespread, sustainable change in clinical practice was achieved within 4 years of the publication of the original evidence by applying a structured program of clinician-led knowledge transfer (clinical re-design). Finally, using contemporary estimates of the cost of evaluating patients with possible cardiac chest pain, we found that a substantial release of health service capacity was achieved within 12 months of implementing the program.

Timely and safe evaluation of patients with possible cardiac chest pain is an essential service in acute care hospitals. However, a large majority of patients presenting with chest pain are at low risk of an ACS, and earlier clinical guidelines have been inefficient in identifying these individuals. We found that an efficient ADP, such as the ADAPT ADP, can substantially improve the efficiency of health service delivery without compromising patient safety. The need to increase the value of care delivered by improving or maintaining outcomes relative to expenditure is widely recognised. Health service providers face a major challenge from increased demand in an environment of constrained funding. As the total cost for implementing the ADP in all 16 hospitals was about $1.6 million, the return on investment was better than 8:1 during the first year.

Successful knowledge transfer in health service provision is difficult. Non-adherence to clinical pathways is a common problem, and the full impact of novel strategies is often diminished when translated from the research setting to clinical care.10 Reassuringly, we found that the proportion of patients defined as being at low risk was similar to that in the pilot study; two randomised controlled trials employing the ADAPT ADP have also reported that 20–30% of patients were eligible for early discharge.8,11 Further, reductions in hospital admission rates and LOS were sustained for up to 12 months after implementing the program.

When introducing a simple innovation into a complex health care environment at several sites, each with its own barriers and enablers, some heterogeneity of response is expected. A detailed consideration of the causes of such heterogeneity is beyond the scope of this article, but has been discussed previously.9 The ADP aimed to facilitate reductions in LOS and admission rates; it achieved lower admission rates at all sites, and reduced LOS at all but one of 16 sites (Box 4).

Translation of the ADAPT ADP into practice was achieved more rapidly than is often the case with more traditional approaches to change. These rely on incorporating research evidence into consensus clinical guidelines that are not always supported by local or broader health policies and programs. This time lag in translating health research into practice has been estimated to average 17 years.12 The time between publishing the ADAPT study and completing and evaluating its implementation was 4 years. Our findings should encourage those considering evidence-based ADPs, such as those described in the recently revised national clinical guidelines for managing ACS.13

One limitation of our study was that tracking clinical outcomes for individual patients was beyond the ambit of our analysis. However, the clinical pathway at all sites complied with the ADP derived from the original ADAPT study, at the time of its publication the largest observational study of chest pain evaluation in Australia and New Zealand. The sensitivity and negative predictive value of its approach to risk stratification has been validated overseas.8,14 Further, the ADAPT principle (Box 1) has subsequently been endorsed in recent revisions of clinical practice guidelines for the management of ACS in Australia, New Zealand, and Europe.13,15 Secondly, we relied on administrative datasets for hospital admissions and LOS data, which may have introduced some error that might have been excluded by a study-specific research dataset. Our results are nevertheless likely to be representative, given the importance of measures of patient flow in EDs following the introduction of National Emergency Access Targets in 2011. Thirdly, the before-and-after design of our study, without control data, means that other changes in the health care system may have influenced our findings. Finally, we relied on local clinicians to identify patients who were at low risk of ACS according to the ADP, and central adjudication of this risk stratification was not undertaken. However, implementing the ADP at each site was supported by education and site-specific documentation. The similarity of the proportions of low risk patients in the ADAPT study (20%) and this study (22%) suggest that this process was robust.

Conclusion

Implementing an evidence-based ADP for evaluating patients with possible cardiac chest pain through a structured program of clinical re-design was associated with a substantial release of health service capacity. This approach to reform could be applied to other high volume clinical services in order to increase the value of health care delivery.

Box 5 – Median hospital length of stay before and after implementation of the accelerated diagnostic protocol

Box 6 – Hospital admission rates for possible cardiac chest pain before and after implementation of the accelerated diagnostic protocol