Improving the effectiveness of clinical medicine: the need for better science

Internationally, there is a drive to render clinical medicine more effective, evidence-based, cost-efficient and accountable. In Australia, the National Health and Hospitals Reform Commission recommends a strong focus on continuous learning and evidence-based improvements to the delivery of health care.1 In the United States, the Institute of Medicine is forging the concept of a value- and science-driven learning health care system that is effective and efficient.2 In the United Kingdom, guidance from the National Institute for Health and Clinical Excellence, combined with quality and outcome frameworks that include financial incentives, seeks to align clinical practice with best available evidence.3

But translating medical research into practice is not the only challenge — medical research itself has problems. These include poor reporting and non-publication of research, and questionable relevance and value of research to clinical practice. Respected former and current editors of high-impact medical journals feel their publications have become marketing vehicles for commercial interests, profiling interventions of marginal clinical value.4 If the effectiveness of clinical medicine is to improve (defined as the consistent delivery of appropriate, evidence-based care to individual patients designed to maximise clinical benefit), we need to address the failings in clinical science. In this article, we define several problems and propose remedial recommendations.

The clinical research agenda is becoming increasingly distorted by commercial and academic priorities and pressures that do not match the concerns and needs of patients and clinicians.5 In Australia, funding for trials targeting cardiovascular disease and cancer is considerably greater than that for trials related to conditions with similar or greater disease burden, such as musculoskeletal disorders, diabetes, injury, asthma and mental illness.6 Research preference is also given to manufactured products (drugs and devices), while potentially efficacious non-drug, non-device interventions are largely ignored.7 Trial entry criteria tend to exclude the elderly, children, patients with multiple comorbidities, or less adherent patients (using run-in phases). Such exclusions may be selected by investigators or mandated by research ethics committees. As a result, on average, less than 15% of subjects screened for trial entry are ever subject to randomisation, thus limiting the generalisability of trial results.8 This problem fosters off-label use of interventions in patient groups in whom efficacy and potential for harm remain uncertain.9

There is also an urgent need for more research on the comparative effectiveness of different interventions, as many clinical dilemmas centre on choosing the most clinically effective and cost-effective forms of condition-specific care among multiple options within the constraints of limited resources. In this regard, more head-to-head randomised trials incorporating economic evaluations are required, with a close watch on the increasing use of non-inferiority study designs that use non-inferiority margins that are far too liberal.10 More attention should be directed to evaluating the benefits and harms of implanted devices, which attract far less rigorous research than pharmaceuticals.11 At the level of health services, more research is needed into how they can be redesigned in order to achieve more efficient, reliable delivery of safe, high-quality care.

Many trials could potentially yield more useful information if, in the design phase, investigators made themselves more aware of what is already known about the issues under study. Less than a quarter of reports of clinical trials incorporate a systematic synthesis of pre-existing evidence to guide researchers in formulating research questions and study design, and only a third cite such a synthesis when discussing their results.24 On average, more than 75% of trials predating an index trial are not cited, including many that involved large sample sizes.25

In addition, 30%–50% of randomised trials are seriously limited by one or more methodological flaws (Box 2).26,27 Of greater concern are highly publicised observational studies claiming intervention effects that fail to adequately account for possible confounders and other forms of bias.28 Such claims tend to persist in published literature, despite strong contradictory evidence from subsequent randomised trials or more rigorous analyses of the original studies.29

Many of these research flaws underpin false claims of efficacy appearing in commercial promotions and advertisements. Unfortunately, no more than a third of these flaws are detected in the peer-review process before journal publication,30 and different reviewers, chosen for their content knowledge rather than their methodological expertise, can assign different quality ratings to the same article.31 In particular, many reviewers show bias in recommending publication of studies that report a positive outcome in preference to those with null or negative results, despite no difference in methodological quality.32

Finally, in assessing real-world relevance, users of research often want to know much more about the logistics and processes of implementation of interventions (who, what, how, when, how much) than is commonly reported in trial reports.33

It is estimated that less than 7% of all reported clinical trials are valid and highly relevant to clinical practice.38 In developed countries, this figure is likely to decrease over time as greater affluence, better overall health care and longer life expectancies reduce the potential magnitude of clinical benefit achievable with any new therapy.39 In response, industry-funded trials increasingly resort to inflating event rates by using inappropriate composite end points or overinclusive, subclinical diagnostic criteria or biomarkers.40 Many contemporary trials involve “me too” drugs, report trivial gains in patient benefit, are intended simply to obtain marketing approval, and fail to live up to exaggerated claims of “breakthrough” or “miracle” advances.41

Non-publication or delayed publication of negative or no-effect trials remains a challenge for both industry and non-industry trials, more so for the former.42 Recent exposés of internal drug company documents demonstrate how commercially inconvenient results of industry-sponsored trials continue to be concealed from independent review by regulatory authorities or other researchers.43 Even if released, most are buried within dense and misleading approval applications and never see the light of day in product labels and patient information leaflets.44

Systematic reviews, if done well according to the standards of the Cochrane Collaboration, greatly assist clinicians in evidence-based decision making, by providing a convenient and unbiased summary of the evidence on a particular topic. They generate more precise overall estimates of intervention effects and can highlight variations in effects between patient subgroups. Although more than 6000 Cochrane reviews now exist, at least another 20 000 are needed to ensure adequate coverage of all areas of clinical practice; at the same time, many existing reviews need updating.49

Funding for systematic reviews and technology appraisals (which are themselves forms of research) is scant compared with the money spent on basic research and clinical trials. In the UK, it accounts for no more than 3% of all biomedical research spending, and even less in Australia.

In many respects, clinical research stands at the crossroads. The reputation of researchers and the science that they generate has been tainted by methodological shortcomings, commercial influences, neglect of societal needs and notorious cases of outright fraud. Researchers need to be more proactive in evaluating clinical interventions in terms of patient-important benefit, wide applicability and comparative effectiveness. Funders need to be more supportive of applied clinical research that rigorously evaluates effectiveness of new treatments and synthesises existing knowledge into clinically useful systematic reviews. Advancing the effectiveness of clinical practice is predicated on producing more good science.