Sea change: public reporting and the safety and quality of the Australian health care system

Ariel’s vision of the sea change in the drowned mariner has been transliterated in Australia to a purposeful life change. For many, a sea change conjures up a dream — safety and security in an idyllic setting by a bay on Australia’s rugged coastline. But for many of those who attempt such a change, the reality is different: water supplies, secure power, sanitation and, above all, health services are not quite as plentiful or as immediate as they once dreamed.

Similarly, quality and safety of health care have proved elusive. It was once considered simple to report on the safety of the system, to weed out improper practices, and to render the system perfectly safe. But the complexity of modern medicine, the increasing age of patients, our greater understanding of human factors and system errors, the huge impact of communication issues, and the frequency with which the baton of care is passed from practitioner to practitioner have rekindled anxiety for many.

We review some of the important steps in the journey towards health care that is characterised by safety, efficiency, appropriateness, effectiveness, equity of access and consumer participation.

The most startling system report was the Quality in Australian Health Care Study (QAHCS) published in 1995.2 Wilson and colleagues clearly identified that adverse events occured in Australian health care, that their frequency was significantly underestimated, and that much was needed to restore faith in the system. These findings were reinforced by the US Institute of Medicine report, To err is human, which demonstrated the magnitude of adverse events in the United States.3

Anaesthetic reports: QAHCS was not the first attempt to measure adverse events. In 1960, a Special Committee Investigating Deaths Under Anaesthesia (SCIDUA) was sponsored by the New South Wales Government. Over the next two decades, all other Australian states followed suit. Two publications on anaesthesia mortality in New South Wales4,5 attracted worldwide interest and prompted regular publications and reports by each of the state anaesthesia mortality committees. In 1976, the Victorian Consultative Council on Anaesthetic Mortality and Morbidity was established, with its terms of reference expanded to include anaesthesia morbidity. Over the ensuing 30 years, the focus has changed from analysis of anaesthetic and perioperative mortality to detailed consideration of morbidity, with provision of feedback to practitioners both directly and via a web-based information service.

In 1990, jurisdictional anaesthesia mortality data were pooled in a national report compiled by the National Health and Medical Research Council and, from 1995, by the Australian and New Zealand College of Anaesthetists (ANZCA).

Despite a high participation rate by anaesthetists, these reports were limited by varying state legislation, their voluntary nature, and limited information on the numerators (incidence of death) and denominators (population at risk). Even so, the reports clearly demonstrate a substantial reduction in the incidence of anaesthesia-related mortality. This success prompted other parts of the health care system to follow suit.

Perinatal reports: In addition, since 1964, reports on perinatal mortality have been published from around Australia. Victoria, Western Australia, the Northern Territory and NSW each publish annual reports on maternal and perinatal deaths, and the Australian Institute of Health and Welfare provides national statistics.

Surgical reports: In 1994, the NSW Minister for Health initiated a Special Committee Investigating Deaths Associated With Surgery (SCIDAWS). Its purpose was to collate surgical mortality data submitted voluntarily, particularly for the purpose of educating surgeons and health care providers. This committee shared resources with SCIDUA; both relied on the commitment of a small group of practitioners to review reported deaths, to provide feedback to the reporting practitioners, and to aggregate and analyse the data.

The surgical committee was much less successful than its anaesthetic counterpart. This was in part due to the wide variety of surgical specialties, compared with the relative homogeneity of anaesthesia practice, and to a very inconsistent commitment to reporting systems between surgical units.

The first attempt to move from individual processes to an integrated system was in January 2000, with the appointment by Australian health ministers of a 25-member Australian Council for Safety and Quality in Health Care (ACSQHC). This was a serious and integrated approach by governments and clinicians to address the complex issues of safety and quality throughout the system. The Council was supported by a federal secretariat, with “in kind” support from each jurisdiction and a supporting forum of state quality officials to assist in implementing a defined national agenda. The Council was able to identify other key bodies of work and to promulgate information about their effectiveness throughout the system. Significantly, the Council included major contributors to the QAHCS study and to surgical reporting and incident monitoring, allowing individual areas of work to coalesce systematically.

In 1981, the Australasian Faculty of Anaesthetists (now ANZCA) developed guidelines for safe practice in a range of conditions. These guidelines identified not only models of care, but also technical considerations, professional standards, and training and education requirements. In close liaison with the Australian Society of Anaesthetists, this college became one of the first to issue specific guidelines on quality assurance and to maintain and update them.

In 1988, a national anaesthesia incident monitoring system (Advanced Incident Management System [AIMS]) was developed, with its base in South Australia. The system was designed to implement effective procedures for analysis of critical incidents and development of crisis management.

In 1993, the first 2000 incidents were reported,6 and numerous subsequent publications accelerated the introduction of mandatory monitoring standards. These have significantly reduced perioperative hypoxic deaths. In addition, management algorithms were developed and regularly improved, and are now widely used. AIMS has made a valuable contribution to reducing morbidity and mortality, along with education efforts by ANZCA, and the introduction of new agents and techniques.

Nevertheless, there was significant disquiet. Occasional catastrophic events, both in Australia (King Edward Memorial Hospital Inquiry8) and internationally (Bristol Royal Infirmary Inquest9 and the Manitoba Pediatric Cardiac Surgery Inquest9) indicated that the health care system was reacting to “disasters”, rather than developing systematic approaches to quality and safety.

In the 1990s, evidence-based medicine became a catchcry of academia and scientific research. Anaesthesia, surgery and other specialties began to adopt these principles, but the process of developing a significant evidence base was slow; alternatives were required to improve safety and quality more rapidly than is possible with large randomised trials. Nevertheless, the MASTER anaesthesia trial established the value of postoperative review of pain relief, particularly in patients with severe respiratory illness.10 The B-Aware trial evaluated the use of bispectral index monitoring to detect awareness during anaesthesia,11 significantly contributing to the wider adoption of depth-of-anaesthesia monitors.

Subsequently, ANZCA established a clinical trials group to collaborate with other specialties in designing trials of mutual interest. ANZCA has gone on to publish guidelines on the best evidence currently available for acute pain management.12 These have allowed clinicians to develop toolkits for use in specific situations, and the Victorian Quality Council is currently establishing a toolkit for the safe management and evaluation of acute pain, applicable throughout the state.

Systemic reporting remained elusive, but isolated examples of reporting began to appear. In cardiac surgery, the Victorian committee of the Australasian Society of Cardiac and Thoracic Surgeons developed a state-based report on mortality and morbidity after cardiac surgery. De-identified data on individual hospital cardiac surgery units have been published annually since 2001–2002.13 This report was one of the first partially risk-adjusted mortality reviews in a specialty in Australasia. It built on the work of the National Heart Foundation and the Australian Institute of Health and Welfare, which had published annual reports on mortality in cardiac surgery for many years. Other state committees within the specialty of cardiac surgery are following.

Incidents related to specific devices are also monitored. The first device-specific register was established by the Australia and New Zealand Heart Valve Registry in 1998 and has followed up almost 1000 patients who received Bjork–Shiley CC valves between 1979 and 1984. The 20-year survival of the Australian cohort is about to be reported (A Callaway, Australia and New Zealand Heart Valve Registry; C F Hughes, unpublished observation).

In addition, the Australian Orthopaedic Association has developed a register of joint replacements.14 The centralisation of data in this register and liaison with similar registers in other countries has enabled surgeons to identify devices with unacceptably high failure rates and to remove them from clinical use much more rapidly.15

In 2003, the ACSQHC commissioned a major study to identify eight major sentinel events, agreed by all jurisdictions, for compulsory annual reporting. This prompted state jurisdictions to begin developing their own process to identify adverse events.

The Victorian State Government produced its first annual report on adverse events in 2003, based on the nationally defined sentinel events. More recently, the Victorian Auditor General released the report Managing patient safety in public hospitals.16

In January 2005, NSW Health issued its first adverse event report, as part of its Patient Safety and Clinical Quality Program.17 This went far beyond the nationally defined sentinel events, including all serious adverse events identified. The reporting process builds on software provided by AIMS to create an incident information management system available electronically across the health care system. Any of the 108 000 clinician and non-clinician employees can report (anonymously if they wish) any incident — clinical or corporate — to a database jointly managed by NSW Health and the NSW Clinical Excellence Commission. All incidents are coded by severity and likelihood of recurrence (Severity Assessment Code [SAC], 1–4). NSW Health SAC1 events (the most severe) must be notified, and a root cause analysis completed within a statutory time frame. This process is protected by privilege, but a publicly available causal statement is issued on its completion. A second report on SAC1 events was released in January 2006.

The reporting rates with this electronic system have been staggering, increasing from an initial 2000 to about 10 000 reports per month. Trend analysis of system-wide data is provided to each area health service and, for the first time, information based on these data is being returned to clinicians and managers alike.

In addition to ANZCA and the Perinatal Society of Australia and New Zealand, other clinical colleges have had an active interest in audits as part of their commitment to professional development. The Royal Australasian College of Surgeons (RACS) monitors compliance with its recommendations for surgical audits for all fellows.

More recently, the RACS has embraced the concept of a binational audit of surgical mortality based on the Western Australian Audit of Surgical Mortality,18 now in its third year of reporting surgical outcomes. South Australia and Tasmania already participate in this audit, and others are expected to follow, including New Zealand.

In Western Australia, 73% of surgeons indicated that they had changed their practice, and 11% knew of surgeons who had done so, as a result of participation in the audit.18 Practice modification depended not on the public reports produced annually, but on the regular individual feedback provided to surgeons, and the illustrative information provided to the entire profession.

NSW will develop a virtually identical model, coordinated by the Clinical Excellence Commission, to replace the SCIDAWS program. The new program will begin by selectively targeting specialty groups, to avoid duplication of reporting from specialties such as cardiac surgery and neurosurgery, which are developing their own specialty-based audits.

The sea change has now reached maturity. We live in a health environment in which data can become information, information applied can become effective knowledge, and clinical pathways and guidelines can be developed on the available evidence and tested against predetermined performance measures. Many accreditation systems have evolved. For instance, the Australian Council on Healthcare Standards is now contracted to accredit 94% of hospitals Australia-wide. Much of this surveillance is based on outcome data regularly supplied by clinicians and institutions.

The debate has now turned to public reporting of individual clinician performance and is by no means limited to Australia.19-23 The fact remains that performance data are already available and will be made public from time to time in circumstances which may be both helpful and harmful to practitioners and patients alike. This debate should also include the issue of a unique patient identifier, and whether it will be part of a clinical and secure medical information program or a de facto national identity card.

It is also important that we continue to develop evidence about the use of public reports. In an article comparing public reporting in the United States and the United Kingdom, Marshall and colleagues stated that “the trend towards increased disclosure of quality information is, in our view, irresistible and, therefore, determining what style of reporting works best and in what circumstances is a major policy task”.19 The impact that public reports have on patient confidence, on the behaviour of policy makers, and on clinicians needs to be evaluated in a scientific manner.

Marshall and colleagues also reported that a consistent finding in both the US and the UK is the lack of public interest in quality reports.19 While public reporting has the potential to bring about change, its purpose needs to be clarified and debated by the entire community.

An interesting area of research will be whether the changes that follow public reporting relate to better delivery of health care or better risk management (avoidance), as high-risk cases are shifted to other practitioners, institutions or even modes of care. Marshall et al demonstrated clearly that the most significant benefits occur at the local level, when individual hospitals or surgeons review their own performance in their own environment and make decisions to improve outcomes in comparison with reliable benchmarks.19

From the comfort of our beach chairs, we observe the relentless waves. The demands of clinical activity, strained resources and a limited workforce will continue to roll towards us. While redesign of the system may focus our attention on better ways of coping with these demands, we can relax somewhat in knowing that — together with all the players we have described — the seascape has changed inevitably and forever. Safety and quality, indeed effectiveness, appropriateness, equity and ease of access are now firmly in focus.