Hard lessons from a randomised controlled trial

The combination of hazardous consumption of alcohol, Aboriginal people, primary care and a randomised controlled trial (RCT) of interventions sets a daunting challenge as a research project. In this issue of the Journal, Sibthorpe et al (page 273) describe, with disarming candour, how they took on this challenge and failed.1 After two false starts, and having recruited only one participant per fortnight (when they had originally aimed to enrol two per working day), the research team felt they had no option but to abandon the project and return the funds to the National Health and Medical Research Council (NHMRC).

There is little that is new for clinical practice here, but there are important lessons about the design, execution and funding of research studies. The project was motivated by concerns about the limited external validity of existing evidence about the impact of simple interventions on hazardous drinking in an Indigenous primary care setting. The attempt to conduct a new RCT in such a setting was laudable, but the NHMRC process of reviewing grant applications apparently did not detect that failure was predictable. The inability of the team to conduct the study as conceived should not compound any negative perceptions about Aboriginal Medical Services and Aboriginal patients. Rather, the NHMRC might have served them better by advising the researchers about simplifying recruitment, need for consent and statistical power, as well as taking a more critical view of the underlying rationale for the project. It is not clear whether the original application to the NHMRC was supported by a feasibility study, but, given the challenge faced by the investigators, funding should not have been granted without one.

As the project was initially designed, the complexity of the screening and recruitment processes flew in the face of well established principles.2 Simple requirements for enrolment make participation in RCTs easy for both patients and providers of healthcare services. By contrast, the combination of a detailed interview about a taboo subject, extensive paperwork and the need for blood all act as disincentives to participation by members of a community whose standard of education and reading ability are often poor, that sees paperwork as the hallmark of an officialdom that too often has been oppressive, and that attaches special significance to body fluids. All of these should have been identified by the NHMRC's assessors as likely to be prejudicial to success.

A requirement to seek informed consent to participation runs contrary to the stated aim of the study to assess "effectiveness" (as opposed to "efficacy"3) of brief advice about drinking in a primary care setting. Alerting potential participants to the existence of a trial of this kind is likely to have a Hawthorne effect, thereby eroding statistical power. It is not clear from the account whether gaining consent was originally proposed by the investigators or imposed by an ethics committee — both would be conscious of the special nature of the target population — but it is another neat example of ethics getting in the way of good science.4 Trials of effectiveness do need ethical oversight, but clear thinking about ethical requirements is required when the control group is to receive "usual care".

The counterargument that consent was needed because the study required additional blood tests that did not form part of routine care would not have any bearing when screening and recruitment were simplified. The investigators may have wanted to use γ-glutamyltransferase as an endpoint, but this is "medicalising" a social problem long before it becomes a biochemical one, and, in any case, represents a further departure from a study of "effectiveness". Part of the challenge of working in primary care and Aboriginal health is to devise and apply measures of impact and outcome that are relevant and robust, but also simple and credible.

The trial as conceived was almost certainly underpowered statistically through overestimating the likely net effect of a brief intervention. In the study on general practitioner intervention in excessive alcohol consumption by Wallace et al,5 the initial prevalence of imprudent drinking was 35%, and the trial was designed on the assumption that 30% of men drinking excessively would respond to the intervention compared with 20% in the control group, the corresponding figures for women being 40% and 20%. Sibthorpe et al were aiming for an absolute difference in response between intervention and control groups of 20%, a bigger average change. They calculated correctly that 200 participants per group would be required to have a 90% chance of detecting such a difference and declaring it significant at P < 0.01. However, it is easy to overestimate the likely impacts of treatments, and changing personal and social behaviour can be even more difficult. Rather than anticipating that anywhere between 5% and 50% of members of the control group might stop drinking hazardously during the course of the study, a pilot study would have been particularly useful for clarifying the likely absolute prevalences and between-group difference in heavy drinking at follow-up.

Finally, the whole concept of this study again throws into sharp focus the tension between high-risk and population-wide approaches to prevention, intervention and control of common health problems so eloquently described by Geoffrey Rose.6 Notwithstanding the fact that the prevalence of drinking alcohol is actually lower in the Aboriginal than in the mainstream population, the damage done by hazardous drinking affects a very great proportion of many Aboriginal communities and social factors play a significant role in the behaviour. Rose has clearly identified the futility of trying to get individuals to change their own health-threatening behaviour in such unsupportive circumstances. In Sibthorpe's project, the principles enunciated by Rose would have dictated that, from the outset, one should simply have attempted to ascertain which of the patients drank alcohol at all; given all of these a simple and unambiguous message about the NHMRC's recommendations about patterns of drinking consistent with best health; told them all about the Aboriginal Sobriety Group; and offered to provide extra help, probably at a separate consultation, to those who then felt that they needed it to achieve change. Eventually, Sibthorpe and colleagues began to stumble down something like this path, but the lesson is a salutary one. Too often, clinicians identify the screening/high-risk/selective medical intervention sequence as a first response to problems that are actually present on a mass scale. There is also an important lesson about both investigators and the NHMRC having available good epidemiological, biostatistical and public health advice at all phases of development and assessment of applications for research funding.