Evaluating recruitment strategies for AUSPICE, a large Australian community‐based randomised controlled trial

Recruitment is a critical element of any research study, and its success depends on the sampling frame, appropriate methods for contact and follow‐up, and the participation rate in the target group. Sampling frames for research studies in Australia have traditionally relied on the Australian Electoral Commission roll and telephone listings.1,2,3,4 However, access to the electoral roll has become increasingly problematic, and the shift to unlisted mobile phones means that large proportions of target populations are not included in telephone directories.2,5 Some population‐based studies have used the Medicare database of the Australian Department of Human Services (DHS), as it includes recent and reliable contact information for most people in Australia.4,6 This database also has limitations that can lead to sampling bias; for example, prospective participants may be inappropriate for recruitment (terminally ill, cognitively impaired, or deceased individuals) and people may be excluded from recruitment because their details have not been updated.2

Willingness to participate in research has decreased in recent times, and recruitment is often the rate‐limiting step for randomised controlled trials (RCTs).7,8,9,10 Typical participation rates for health surveys, for example, declined around the world from 80% in the 1980s to 40–50% in 2015.7 Trials with minimally restrictive inclusion criteria achieve better response rates than those with more restrictive conditions,9 and response rates for therapeutic trials are higher than for prevention trials.10 Several systematic reviews have sought to identify methods for enhancing recruitment for RCTs,8,11,12,13,14 one identifying 137 different strategies.13 Providing monetary incentives was the most prominent strategy, and was also a good strategy for recruiting participants for cardiovascular disease prevention trials.15 However, this approach is not always successful, and is expensive for large scale clinical trials. A review of publicly funded clinical trials in the United Kingdom found that only 55% achieved their target sample sizes.8 Newer approaches, including recruitment via social media, can increase message reach and response rates.16,17,18

The Australian Study for the Prevention through Immunisation of Cardiovascular Events (AUSPICE), a community‐based, randomised, placebo‐controlled clinical trial of the value of the 23‐valent pneumococcal polysaccharide vaccine (23vPPV) for preventing cardiovascular events,19 recruited participants in six cities. In this article, we describe the challenges of recruiting participants for this large scale Australian study and discuss potential lessons for future studies.

Methods

Sampling frame

To minimise volunteer bias, we used the Medicare database for establishing our sampling frame. The DHS randomly selected people aged 55–60 years, stratified by sex (equal numbers of men and women) and site (six locations, based on the data collection clinics). To maximise the likelihood of participation, only people with addresses within 25 km of the participating centres were included, and their data were cross‐linked with the National Death Index to avoid contacting people who had recently died.

Mailing protocol

The DHS employed a mail house to prepare the mail‐out to potential participants (Box 1). Each person received cover letters from the DHS and the investigators, an invitation letter, a screening questionnaire for assessing study eligibility, and a reply‐paid envelope. Potential participants could complete the questionnaire on paper or on the AUSPICE website (https://auspice.apps.hmri.com.au), which includes a short video on the scientific background to the study. Cover and reminder letters were posted 2 weeks after the initial mail‐out by the DHS; reminder phone calls were not made.

Screening questionnaire

To be eligible for the study, an individual was required to have at least two of three risk factors for cardiovascular disease (CVD):

• hypertension: self‐reported (based on a physician's diagnosis), or receiving one or more antihypertensive medications;
• hypercholesterolaemia: self‐reported (based on a physician's diagnosis), or receiving lipid‐lowering medication;
• overweight/obesity: body mass index exceeding 27 kg/m² or a waist circumference exceeding 88 cm (women) or 102 cm (men).

Exclusion criteria included a history of CVD, an immediate indication for receiving or a history of having received the pneumococcal vaccine, and being over 65 years of age (as pneumococcal vaccination is part of the national immunisation schedule for people in this age group, administering a placebo would be unethical).19

Recruitment

After their questionnaire responses were entered into the database, eligible participants were notified of their eligibility by text message or email. They also received an information statement and consent form electronically (or on paper, for people without email addresses) before they were contacted to organise a clinic appointment. A reminder text message or email was sent the week before and the day before their appointment. Recruitment began in February 2016 and was completed by December 2017 at six sites: Newcastle (Hunter Medical Research Institute), Central Coast (Gosford Hospital), Canberra (Canberra Hospital), Melbourne (Caulfield Clinical Trials Centre), Adelaide (South Australian Health and Medical Research Institute), and Perth (Institute of Respiratory Health).

Expected participation rate

The proportion of eligible persons in the catchment areas was estimated to be 29% of people aged 55–60 years, based on the results of a large cohort study in Newcastle.3 We anticipated a 33% participation rate by eligible people, based on the minimal study requirements (one postal questionnaire, one clinic visit) and the potential benefits to those at high risk of CVD. The overall anticipated participation rate was thus 33% × 29%, or about 10%, and we accordingly planned to mail 60 000 invitation letters to obtain the sample size of 6000 participants required for 80% power to detect a hazard ratio of 0.82 (α = 0.05).19

Responses

Eligible participants were sent a short questionnaire on basic demographic characteristics (age, sex, self‐reported blood pressure, cholesterol and weight status). We investigated whether the characteristics of participants who responded to the first invitation (early responders) differed from those of people who responded only after receiving the reminder (late responders). We also investigated response rate according to level of socio‐economic disadvantage; postcodes of residence were allocated to Index of Relative Socio‐economic Disadvantage (IRSD) quintiles.20 We assessed the reach and impact of our recruitment strategy in Google Analytics (https://marketingplatform.google.com/about/analytics).

Statistical analyses

Participation rates are reported overall and by study site and IRSD quintile. Characteristics of early and late responders are presented as means with standard deviations (SDs) for continuous variables and frequencies and percentages for categorical variables, and compared in Student t tests and χ² tests respectively. All statistical analyses were conducted in SAS 9.4 (SAS Institute).

Ethics approval

Approval for the clinical trial and for data linkage was granted by the Human Research Ethics Committees (HRECs) governing each trial centre: the University of Newcastle HREC (reference, H‐2014‐0064), the Hunter New England Local Health District (also for Central Coast Local Health District; references, 15/08/19/3.01; HREC/15/HNE/298), the ACT Health HREC (reference, ETH.7.14.177), the Australian National University HREC (reference, Human Ethics Protocol 2015/523), the SA Health HREC (reference, HREC/16/SAH/45), the Monash University HREC (reference, CF14/3016‐2014001638), the Department of Health WA HREC (reference, 2015/54), and the University of Western Australia HREC (reference, RA/4/1/7101).

Results

Recruitment: response rates

In the first of four waves of recruitment planned for February–December 2016, 2500 invitations were sent for each of the six participating centres; a total of 254 participants were recruited during this wave (1.7% participation rate). The mail‐out was therefore increased to 5000 per site per wave, and we also encouraged people who received invitations to participate by advertising in local media (radio, television, newspapers), primarily at the start of the recruitment wave. In June–July 2016, we began snowball sampling; that is, asking people who responded to invitations, regardless of whether they were eligible for the trial or not, to invite family and friends to participate.

By December 2016, 2800 participants had been recruited, or fewer than 50% of our recruitment target. The study steering committee extended recruitment for a further year and sought strategies for increasing recruitment. Two of the six sites (Central Coast, Perth) did not continue recruitment into the second year because of other commitments. The sample size calculation for the trial was reviewed; by extending the follow‐up period of our study from 4 to 6 years, we could reduce the required sample size from 6000 to 4500 participants without reducing its statistical power, after taking into account the unblinding and censoring of participants who reached age 65.

In the second year of recruitment, there were two mail‐outs with reminders for each of the four sites, and one final mail‐out for two sites (Melbourne, Newcastle). Facebook advertising commenced for all sites, targeting people in selected postcodes within 25 km of each centre. Recruitment was completed by December 2017, by which time 4725 participants had been recruited.

Recruitment: participation rates

A total of 154 992 invitation letters (and an equal number of reminders) were sent by the DHS. Overall, 21 526 invited people (14%) responded by completing the screening questionnaire (in whole or in part), while 4725 people who responded to the screening questionnaire (22%) completed clinic visits and were randomised for participation in the trial, an overall participation rate of 3% (Box 2, Box 3).

As this participation rate included many people who did not meet the CVD risk factor inclusion criteria, we also calculated the participation rate for respondents eligible for the trial. We estimated that 47% of eligible people completed screening, and that 10% attended a clinic and were randomised for participation in the trial (Box 2).

Of the 4697 participants for whom complete baseline data were available, 3951 (84%) responded to the invitation only after receiving a reminder. Early and late responders who were eligible for the trial were similar in most demographic and clinical characteristics. Equal proportions of early responders replied online or on paper, whereas 65% of late responders replied on paper (Box 4).

There were several postcodes within 25 km of each study site. The participation rate in areas of IRSD quintile 1 (greatest disadvantage) was 2.0%, whereas in quintiles 2–5 participation rates were 3.0–3.1% (Box 5).

Of the 4725 study participants who were randomised for the study, 31 (0.7%) had no email address or mobile phone number and had to be contacted by mail.

Social media

The number of page views for the AUSPICE website consistently increased sharply following mail‐outs and corresponded to peaks in invitee responses. The initial DHS mail‐out generated almost immediate increases in the numbers of page views for the AUSPICE website and of online responders. There was a lull in all activity during the 2016–17 Christmas and summer holidays. Free and paid Facebook advertisements placed at various points during 2017 to recruit people in areas close to the participating sites did not influence the numbers of either hard copy or online responses (Box 6). We did not systematically collect the source of referral (direct or snowball) for recruited participants.

Costs of recruitment

The mail‐outs of invitations with questionnaires and of reminder letters (309 984 letters and reply‐paid envelopes in total) cost $357 775, including printing and postage costs, or $75.72 per participant. Facebook advertising ($14 162.45 in year 2) and start‐up funds for five of the six clinics (5 × $44 390.40 in year 1) increased total recruitment costs to $593 890, or $125.69 per participant.

Discussion

The response rate to invitations to participate in an RCT of an established vaccine, despite the modest burden on prospective participants during screening, was about 3% overall, or an estimated 10% of the target recruitment population. Low response rates have been reported for many epidemiological studies in recent years.7,21,22,23 Proposed explanations have included lack of interest by potential participants in information about their own health and in volunteering, their preference for less intrusive and time‐demanding studies, the lack of incentives to participate, mistrust of the scientific community, the increasing volume of junk mail received by many people, and not clearly identifying the organisers of the RCT in invitations with institutional logos and academic signatures.17,21,22,23 For vaccine trials, distrust of vaccines and fear of vaccine‐induced disease may also reduce participation.24 Oversampling of target groups itself depresses the participation rate.21

We accordingly provided prospective participants with information about the trial, including a video on the science underlying the study and what was required of participants, available both on the study website and on YouTube (https://youtu.be/BScWxe3aAIE). We attempted to minimise the burden on participants, including that of data collection; the screening questionnaire took about 3 minutes to complete, the postal questionnaire 20 minutes, and the clinic visit about 45 minutes. The invitation letters included colourful institutional logos on the letterhead and information about the researchers involved in the study. We advertised on television and radio, as well as in print and social media, to encourage participation by invitees. We did not offer any financial incentives, both to reduce costs and because the evidence on their effectiveness is equivocal.12,25 However, we acknowledged that participants in the active intervention group would receive a vaccine they would normally not receive for free, under the National Immunisation Program, until age 65.

Site‐specific overall response rates ranged between 2.0% (Perth) and 4.5% (Canberra). At the Canberra site, the private rooms of a cardiologist were adapted for clinical research, providing convenient parking and a non‐hospital setting. Further, a sub‐study examining the mechanisms of any protective effects of the vaccine was also undertaken here; 1000 participants received free blood tests (for serum anti‐oxidised low‐density lipoprotein antibody), pulse wave velocity tests, electrocardiography, and carotid intima media thickness measurements at baseline and follow‐up. Despite the greater burden on participants, the sub‐study may have been regarded positively, as a free health check. It is difficult to know whether any of these factors explained the higher participation rate in Canberra, which may also have been related to the higher socio‐economic status of the catchment area, despite the general lack of influence of socio‐economic status (measured by postcode IRSD quintiles) on response rates.

More than 80% of eventual participants responded to the invitation only after receiving the reminder letter, indicating that this was a worthwhile, cost‐effective measure. To reduce postage costs, we did not re‐send the hard copy questionnaire. Despite including the weblink for the questionnaire in the reminder letter, nearly two‐thirds of late responders responded in hard copy, indicating they had saved the questionnaire from the initial mail‐out for more than 2 weeks. The demographic and health characteristics of early and late responders were similar.

The cost of the mail‐out, $76 per participant, was similar to that in a pilot study which assessed the feasibility of recruiting participants with the same method for a research study in women's health ($101 per participant).26 Facebook advertising has been reported to be effective in smaller studies,16,17,27 and was relatively inexpensive in our large study, but it did not increase recruitment of people in the age group we targeted (55–60 years).

Limitations

The main barriers to increasing the number of study sites were financial in nature, but the number of experienced cardiovascular clinical trial sites interested in participating was also limited. We needed to balance the costs of each mail‐out against the time required to process the responses. We could not extend the recruitment period by another year, mainly for financial reasons. The decision at the end of 2016 to increase recruitment at the remaining four sites, rather than to recruit more sites, was again based on financial considerations, as each participating site received nearly $50 000 for training and start‐up costs.

Conclusions

On the basis of our experience with recruiting participants for AUSPICE, we recommend employing a mixture of recruitment methods when possible, providing both online and printed form options for responses, and sending reminder letters to boost response rates. The demographic and relevant clinical features of early and late responders eventually included in the study were similar. Social media advertising was not effective in increasing recruitment from our target age group of older adults.

Data sharing

The authors have full access to all de‐identified data (including statistical reports and tables) from the study. We do not have permission to supply de‐identified data for individual participants. Further data may be obtained by application to the study investigators. Additional study‐related information is available from the Australian New Zealand Clinical Trials Registry (ACTRN12615000536561).

Box 1 – Sampling process for the Australian Study for the Prevention through Immunisation of Cardiovascular Events (AUSPICE)

---

 

Box 3 – Flow chart of the Australian Study for the Prevention through Immunisation of Cardiovascular Events (AUSPICE)

---

* All participants received the allocated intervention. ◆

Box 6 – Timeline of recruitment for Australian Study for the Prevention through Immunisation of Cardiovascular Events (AUSPICE)*

---

* Red arrows: initial mail‐outs; blue arrows: reminder letters; solid bars: periods of Facebook advertising. ◆