Left in the dark: the importance of publicly available clinical trial protocols

Full disclosure of research plans and their subsequent modifications facilitates transparency in medical research

Prospective registration of a randomised controlled trial (RCT) based on a protocol with formal ethics approval is a benchmark for transparent medical research. The reporting of the primary results of the study should correspond to the design, analysis, and reporting specified in the protocol and trial registration. However, modifications to various aspects of the trial are often made after registration, ranging from administrative updates to substantial protocol amendments. To track the history of revisions, the protocol and registry entry should be updated, and the documentation trail should support an independent appraisal of whether any biases have been introduced that could affect interpretation of trial results.

In this issue of the MJA, Coskinas and colleagues report their investigation of changes to 181 phase 3 RCTs registered with the Australian New Zealand Clinical Trials Registry (ANZCTR) during 1 September 2007 – 31 December 2013.1 The authors compared protocol documents (including ANZCTR registration information) with subsequent journal publications for any changes to the primary outcome, treatment comparisons, analysis set definition, eligibility criteria, sample size, or primary analysis method. They found that protocols were available for only 124 trials (69%); it could be determined that no major changes had been made to eleven of these trials (9%), while 78 had definitely been modified (63%). By comparing publications with trial registration information, it was found that no changes were made to five of the 57 trials without available protocols (9%), and it could not be determined whether changes had been made to a further ten (18%).

The protocol is an essential document required by ethics committees before the trial commences, but, in contrast to trial registration, making the protocol publicly available is not mandatory.2 It is disappointing that protocols were not available for almost one‐third of the clinical trials reviewed by Coskinas and her colleagues. This situation may have since been improved by incentives and requirements to increase accountability and transparency, including some journals accepting study protocols as stand‐alone publications or requesting that protocols accompany submitted research manuscripts.

The SPIRIT 2013 statement recommends documenting protocol changes, and that major changes, including to the primary outcome or sample size, be reported in formal protocol amendments.2,3 It is notable that Coskinas and colleagues identified differences between the primary outcome in the protocol and in the subsequent publication for 52 of 118 trials with available protocols (44%). Other investigations of such modifications have found proportions ranging between 31% (46 of 147) and 62% (51 of 82).4,5,6 Unfortunately, Coskinas and her co‐authors did not provide details about whether these changes were acknowledged or justified by trial investigators, or whether the changes were foreseen by the protocols. Of particular concern were changes informed by unplanned assessment of interim data; that is, with knowledge of trial group allocation. Although Coskinas and colleagues reported that the changes were usually made with appropriate blinding, establishing this on the basis of the available documentation was difficult for most trials.1

Not all changes to trial protocols are inappropriate. For example, it is increasingly common to specify potential adjustments in the protocol, such as changes to the sample size or the removal of treatment arms after an interim analysis as part of an adaptive study design. Although not ideal, some unplanned changes may also be acceptable, including changes made blinded to treatment assignment and fully documented and justified in the protocol or study report. Despite careful planning, events outside the control of the investigators may necessitate unforeseen protocol modifications, including changes to the standard of care or COVID‐19‐related modifications of practice. In such cases, it is important to acknowledge and justify the changes to the protocol, statistical analysis plan, and study report.7,8

We wholeheartedly agree with the recommendations by Coskinas and colleagues1 that investigators adhere to the principles of the CONSERVE statement for implementing and documenting modifications, include contingency plans in the protocol for dealing with challenges, and make all versions of the protocol and the analysis plan publicly available.7 We would add that it should also be clear whether modifications were made blinded to treatment assignment and that they be fully justified, in line with the global move toward open access for publishing medical research, strongly supported by the Australian Chief Scientist.9 Although some journals now encourage transparency by publishing study protocols, this is not a complete solution, as these publications are often condensed versions of the full protocol and not updated when modifications are made.10 Ideally, journal editors and referees would review a manuscript together with its registration and protocol documents to determine whether the reported investigation was consistent with the planned study.

The importance of detailed tracking of protocol modifications is growing, given the increasing complexity and flexibility of trial designs that anticipate potential adjustments that can lead to multiple protocol amendments.