Prior to 2007, Queensland Health clinicians selected a suite of clinical indicators from a range of sources and refined these to ensure their relevance to the Queensland context for inclusion in hospital-specific annual reports. Annual reports were initially disseminated to hospital executives only, presenting cross-sectional analysis of 12-month risk-adjusted outcome rates and highlighting the statistical significance of variation between hospital and state rates. Although the intention of these reports was to prompt local review to determine the cause of variation, it was recognised that the reports were not used as intended because the data were at least 1 year old at publication and the methodology summarised results over 1 year, hiding trends.
Since 2007, Queensland Health has employed a statistical process control methodology that has been adapted for clinical utility, the variable life-adjusted display (VLAD), to systematically monitor patient outcomes (Box).1-3 VLAD methodology based on available administrative data has been found to provide results equivalent to more resource-intensive clinical audit methodology that uses individual chart review.4 Queensland Health hospitals have undertaken numerous quality initiatives as a result of the use of VLADs, leading to improvements in discharge processes, clinician documentation, patient education, implementation of and adherence to standard clinical pathways, and allocation of resources to areas of most need.
One of the key elements that led to these initiatives is having well defined indicators that are clinically relevant, clinically significant in terms of burden of disease, and clinically responsive to changes in practice. Queensland Health has used the quality improvement cycle (plan, do, check and act cycle)5 to improve or remove indicators, suspending two indicators, developing two new indicators and refining three indicators. A review of the balance of indicators is currently underway. This process of development and refinement has led to a closer working relationship with clinicians, and resulted in improved indicator definitions to more effectively monitor safety and quality.
To overcome issues of timeliness and sensitivity, the VLAD methodology was applied to the 31 existing clinical indicator definitions and disseminated monthly to Queensland public and private hospitals. VLADs graphically display patient outcomes, “flagging” if predetermined levels of variation in patient outcomes are exceeded. Flags require a VLAD review to examine data and clinical practice to detect potential patient safety and quality issues and identify action to resolve issues.6-8 Formal documentation of the VLAD review is required within 30 days of notification and is evaluated within a governance structure that was introduced concurrently with the VLADs.9 Indicator definition issues identified by hospital staff are also captured in the VLAD review documentation. This feedback, and issues captured through an online feedback form,10 is then used to improve or remove clinical indicator definitions.
Two indicators for which the quality improvement cycle was recently applied within Queensland were laparoscopic cholecystectomy complications of surgery and third- and fourth-degree perineal tears during childbirth. The “plan” and “act” parts of the cycle had been in place for at least 12 months, with hospitals using the VLAD indicators to examine data and review practice. During this time, feedback from clinicians, quality coordinators and coders suggested these indicator definitions were problematic, triggering a review of the indicator definition — the “check” part of the quality improvement cycle.
Clinical improvement opportunities were identified as a result of clinical reviews of laparoscopic cholecystectomy undertaken by hospitals following a VLAD flag. One hospital instituted a strict policy of single use when it identified that a laparoscopic clip applicator that had been reused a number of times failed to secure the cystic duct. Another hospital, as a result of a significant increase in bleeding complications, changed practice to ensure locum surgeons were supervised by a senior surgeon and given less complex cases to manage.
The Australian Council on Healthcare Standards, Women’s Hospitals Australasia (WHA) and the WHA’s Core Maternity Indicators Project11 have used third- and fourth-degree perineal tears as clinical indicators for some years. The usefulness of the perineal tear indicator was challenged by some Queensland Health clinicians, who argued that there was no evidence that intervention could reduce the perineal tear rate.
Following the literature review and analysis of the coding definitions, the perineal tears VLAD indicator was refined and reinstated,12 induction of labour and caesarean section indicators were refined, and episiotomy and instrumental delivery VLAD indicators were developed and introduced, completing the “act” part of the quality improvement cycle. Within 3 months of reintroducing the perineal tear indicator, a VLAD review raised a clinical practice issue identifying the use of midline episiotomies causing third-degree tears.
Interestingly, issues with the indicator definitions only became apparent when the VLAD methodology was introduced, perhaps confirming a more comprehensive use of the monthly disseminated VLAD monitoring tool, compared with the annual outcome rates. In this sense, a lack of feedback regarding a given indicator should not be interpreted as the sign of an effective indicator. A lack of feedback may be a result of the statistical methodology used or the frequency or timeliness of dissemination.
Above all, the most important message is that quality improvement indicators should not to be used as performance measures.13-15 Indicators (eg, VLADs, annual rates) are tools that help identify variation. Review of the variation is required to determine its cause. Indicator variation (without the benefit of review) cannot be equated with either high or low hospital performance. The notion that hospitals with higher risk-adjusted mortality have poorer-quality care is neither consistent nor reliable.16
Abstract
The variable life-adjusted display is a graphical, statistical methodology used in Queensland to monitor patient outcomes of clinical indicators.
The quality improvement cycle is a systematic approach employed by patient safety and quality programs worldwide to improve patient care.
The quality improvement cycle is beneficial to the review and refinement of indicator definitions.
Indicators with definitional issues that are not subject to the quality improvement cycle may initially prompt quality improvement opportunities, but are more likely to potentially lead to unnecessary chart and clinical reviews, which will disengage coders and clinicians.
Queensland recently used the quality improvement cycle to refine the laparoscopic cholecystectomy complications of surgery indicator definition and several maternity definitions.