Gaps between best evidence and practice: causes for concern

How much mainstream medical care is based on the best available scientific knowledge about what does or doesn’t work? Claims that only 10%–20% of care was based on good evidence were made by a few influential people such as Kerr White (a US physician who pioneered the discipline of health services research) and Archie Cochrane (the UK epidemiologist who provided inspiration for the Cochrane Collaboration) in 1976,1 and by David Eddy (a leading US health policy expert) in 1991.2 These statements were repeated in widely read publications such as the US Office of Technology Assessment reports3,4 and the BMJ.2

The estimates themselves had a weak evidence base. White’s assessment relied on prescribing data gathered in 1961 from 19 British general practitioners. Eddy’s comments were apparently based on research in selected areas, such as treatment of glaucoma. The most substantial body of data came from a 1991 study of the evidence used by the US National Institutes of Health to assess 126 therapeutic and diagnostic technologies.5 The study concluded that only 21% of the technologies were supported by controlled research evidence, and took no account of how often they were actually used in patient care.

Spurred by these dismal pronouncements, clinicians in several specialties have studied the evidence base of their practice. In 1995, an Oxford general medical unit reported that 82% of primary treatments provided to 109 patients were based on evidence, with 53% supported by randomised controlled trials (RCTs).6 Further studies in internal medicine,7 general practice,8 psychiatry,9 surgery,10-13 anaesthesia,14 paediatrics,15 and other areas16,17 have concluded that between 11% and 65% of treatment is supported by findings from RCTs. The percentages are higher when other forms of evidence are included, although there are issues about the quality of the evidence used to derive these higher estimates.

While these figures are substantially higher than earlier estimates, critical evaluation of the reported studies highlights the potential for biased results. Most studies examine treatment decisions made during a series of patient encounters over a few weeks in a particular setting. Some have low numbers of participants, are confined to the practice of a small group of doctors, or are undertaken in atypical settings. Moreover, they often concern only one aspect of care, assume that all relevant information was recorded, and do not adequately note possible underprovision of care. The main value of these studies may be their potential to highlight areas in which the lack of good quality research data is failing doctors and patients who need to make practical decisions about healthcare.

A different picture of the extent to which evidence is used in clinical practice emerges from studies of the care that patients receive for specific illnesses. Recently, researchers from the RAND Corporation interviewed a random sample of over 6000 adults living in 12 metropolitan areas in the United States and examined their healthcare records.18 They compared the care received with evidence-based quality indicators for common acute and chronic conditions. Most indicators were process measures (eg, the provision of counselling for smoking cessation and other lifestyle changes), because these reflect the activities that clinicians control most directly and, unlike outcome measures, do not generally require risk adjustment. Overall, participants in the survey received about half of the recommended care processes. Some specific areas of underprovision included failure to treat with aspirin following myocardial infarction, failure to conduct routine monitoring of glycosylated haemoglobin levels in diabetes, and failure to provide pneumococcal vaccination for elderly people.

Although there have been no similar studies in Australia, we know that, in a number of important areas, good research evidence is not being applied universally in practice. For example:

• Heart failure affects over 300 000 Australians and is a common reason for hospitalisation.19 Although angiotensin-converting enzyme inhibitors reduce heart failure symptoms and hospital admissions and lengthen life,20 they are underprescribed by Australian GPs19 and hospital doctors.21

• Smoking in pregnancy adversely affects mothers and babies, and smoking-cessation programs can reduce smoking rates among pregnant women.22 However, a study of the protocols and policies used by antenatal care providers in Australia found that 90% did not include written information and advice about smoking cessation.23

• There is strong evidence for the benefit of measures to prevent deep-vein thrombosis in hospitalised patients,24 but studies of high-risk patients in Australian hospitals suggest that some do not receive appropriate preventive care.25

These Australian examples of opportunities to improve care come from a recent National Institute of Clinical Studies (NICS) report that aims to raise awareness of the potential impact of better use of research findings in practice.25 The NICS was established 3 years ago by the Australian Government to help close gaps between evidence and practice in healthcare. The NICS’ evidence practice gaps report does not provide a comprehensive overview of the most important gaps, mainly because we lack good quality routine data that describe the processes of care in Australian hospitals and primary care settings. Mostly, we depend on one-off surveys and studies to describe whether the care that is delivered is consistent with the best available evidence. While systems for exploring the cost of healthcare become increasingly sophisticated, the capacity to monitor and improve the value of what we do remains primitive.

The lack of documentation of important gaps is one challenge that faces the Institute. A second is that the reasons for the gaps between what is known and what actually happens in clinical practice are complex, and efforts to improve uptake are unlikely to be successful if they are unidimensional or focus only on individual health professionals. The primary role of the Institute is not to answer research questions about whether specific methods work in particular settings for specific problems (although better-quality evidence in these areas would be enormously helpful). Rather, its role is to identify and test approaches that might improve evidence uptake across the system and that are feasible and affordable in Australia.

Concerted effort by motivated groups to improve uptake can improve care. For example, the incidence of pulmonary embolism at the Canberra Hospital fell 25% after clinicians reviewed and changed their practice to ensure that patients actually received care that was in line with the current best available evidence. This description of use of evidence made the Clinical Practice Improvement Unit at the Canberra Hospital the overall winner of the inaugural NICS Cochrane Users’ Awards.26 Identifying ways to achieve these kinds of results in multiple settings for a broad array of problems was the focus of the November 2003 workshop in Hobart on effective implementation strategies.

Better use of research knowledge in practice will require partnerships between a range of organisations and the capacity to combine scientific rigour with a recognition of real world needs and constraints. In 1991 Trevor Baylis, a British inventor, heard of the difficulties of healthcare workers in Africa who were trying to inform the population about ways to prevent AIDS. Radio broadcasts could not reach a wide audience because the cost of batteries was prohibitive for many Africans. Baylis developed clockwork-powered radios that are now distributed throughout the world by agencies like the United Nations and the Red Cross.27 Improving research implementation will require a similar mix of ingenuity, science and partnership to develop affordable approaches and products that are tailored to the environments where they are most needed.