Influenza vaccine effectiveness in general practice and in hospital patients in Victoria, 2011–2013

The 9th edition of the Immunisation Handbook sponsored by the National Health and Medical Research Council maintained that influenza vaccines were 70%–90% effective in preventing influenza when the match between vaccine strains and circulating strains was good.1 Even when published in 2008, this was probably a generous assessment of the evidence. The 10th edition, published in 2013, maintained that influenza vaccines were 59% effective in preventing influenza in healthy adults and at least as effective in children, although in some years there was no evidence of any benefit.2

Although not explicitly stated in the handbooks, these estimates referred to efficacy in protecting against influenza infections managed in the community, the majority of which are relatively mild. While protection against the mild disease seen in primary care might be modest, it is nevertheless possible that the protection provided against more serious disease, including confirmed influenza infections requiring admission to hospital, might be greater.

In Victoria, two surveillance schemes make it possible to investigate whether there was any major difference in vaccine effectiveness estimates in community and hospital patients. The Victorian Sentinel Practice Influenza Network (VicSPIN) is a group of sentinel general practitioners in Melbourne and regional Victoria, operating since 1997, that has provided estimates of influenza vaccine effectiveness for protection against laboratory-confirmed influenza since 2003.3 The Influenza Complications and Alert Network (FluCAN) is a national hospital-based sentinel surveillance scheme that has provided estimates of influenza vaccine effectiveness since 2010.4 About 40% of patients registered by this scheme were reported by Victorian hospitals.

We compared influenza vaccine effectiveness estimates for 3 years in Victoria, basing our analysis on data from these two sentinel surveillance systems. Each scheme has published separate vaccine effectiveness estimates for the three study years.5,6,7,8,9,10

Methods

We reviewed data for the influenza seasons of 2011–2013 from the general practice and hospital-based schemes. Vaccine effectiveness was estimated by comparing the vaccination status of influenza cases (patients with laboratory-confirmed influenza) with that of non-cases (patients for whom influenza test results were negative). The use of test-negative controls is an established variation of the case–control study design.11

VicSPIN uses a community-based, test-negative design. Sentinel GPs were located in metropolitan Melbourne, Geelong and regional Victoria, and patients were recruited by sentinel GPs when they presented with symptoms consistent with influenza infection. At presentation and before their case status was known, patients were swabbed at the discretion of the GP. Patients with positive influenza test results were defined as cases, and those with negative results as non-cases or controls. Vaccine effectiveness was calculated as 1 − odds ratio (OR), and expressed as a percentage, where the OR compared the odds of vaccination for cases with the odds for controls. Logistic regression was used to adjust estimates for age group (0–17 years, 18–64 years, ≥ 65 years), comorbidity (yes v no), and time within influenza season (number of weeks from peak). Estimates were restricted to patients vaccinated at least 14 days before the onset of symptoms, accepted as the time needed to produce protective antibodies, and to those presenting within 7 days of symptom onset, as data from shedding studies suggest that influenza virus detection declines after 7 days.12 On the assumption that vaccine effectiveness would not be detectable when influenza virus was not circulating, we also restricted our analyses to patients who presented during the influenza season, as defined by positive case ascertainment and sentinel surveillance of influenza-like illness.13 Year was added as a covariate to the regression analysis for the combined 3-year estimate. Although the number of sentinel practitioners participating in the scheme in different years varied slightly, the approach to surveillance remained constant. Vaccination status was determined by patient or GP report, with vaccine date requested as a proxy for a register record. All samples were tested by polymerase chain reaction (PCR) assays at the Victorian Infectious Diseases Reference Laboratory, a designated National Influenza Centre of the World Health Organization. The assay detected influenza A(H3N2), influenza A(H1N1), influenza B and influenza C viruses. Two patients with influenza C were excluded from the analysis.

FluCAN is a sentinel surveillance system that receives data from 17 Australian hospitals. It provides data on the number of cases admitted with severe influenza A or B confirmed by PCR nucleic acid assays in the reporting hospitals’ laboratories. We based our analysis on the four Victorian hospitals reporting to FluCAN (the Alfred Hospital, Monash Medical Centre, University Hospital Geelong, the Royal Melbourne Hospital). Only adults (over 18 years of age) were included in the study. The vaccination status of cases was compared with that of controls (1:1); each selected control was the next patient after each case who presented with an acute respiratory infection and a negative influenza test result. Vaccination was defined as the patient having received the inactivated influenza vaccine at least 14 days before presentation, and the patient’s status was based on patient report and medical record. Vaccine effectiveness was estimated in the same way as for the community patients, but adjusted for potential confounders using conditional logistic regression to account for the matched design. Binary covariates included in the model were: being over 65 years of age, chronic comorbidities, Indigenous Australian status, and pregnancy. Because the control group was frequency-matched by the date of admission using the incidence density control selection strategy, we did not adjust or stratify estimates for time. We conditioned the analysis on the basis of hospital site. For the pooled analysis of all three seasons, the analysis accounted for year by adjusting standard errors with the Huber–White robust sandwich estimator.

During the years included in our study, FluCAN did not collect control data for people aged 0–17 years. We estimated vaccine effectiveness for all ages from the VicSPIN data, but, to improve the comparability of results, we also calculated vaccine effectiveness for the VicSPIN data after excluding the 0–17-year-old age group. All vaccines used in Australia during the study period were trivalent inactivated vaccines. We did not collect information on vaccine manufacturer, and assumed that all vaccines performed equivalently.

Ethics approval for FluCAN data collection and reporting was obtained from the Human Research Ethics Committees of all participating hospitals and the Australian National University. VicSPIN data were collected, analysed and reported under the legislative authorisation of the Victorian Public Health and Wellbeing Act 2008 and the Public Health and Wellbeing Regulations 2009, and therefore did not require formal Human Research Ethics Committee approval.

Results

In the VicSPIN surveillance system, before exclusion of patients vaccinated within 14 days of symptom onset or presenting outside the influenza season, 1680 patients for whom vaccine status by age group was known were available for the three seasons 2011–2013. Their number varied from 354 in 2013 to more than 600 in each of the two earlier years (Box 1). Eighty-five per cent of swabbed patients were from Melbourne or Geelong, the locations of the Victorian sentinel hospitals. Most patients consulting a sentinel GP were aged between 18 and 64 years. Older people were under-represented, but more than 70% were vaccinated each year (Box 1). For the 3 years combined, 5% of patients aged 0–17 years reported a comorbidity, compared with 16% of those aged 18–64 years and 48% of those aged 65 years or more. Comorbidity status was not recorded for 12% of patients.

In the FluCAN surveillance system, 1289 patients were enrolled in Victorian hospitals for whom vaccine effectiveness estimates could be made for the three seasons 2011–2013. The number of participants varied from 271 in 2011 to 737 in 2012. The majority of patients admitted to hospital were at least 65 years old. For the 3 years combined, 76% of adults aged 18–64 years had a comorbidity, compared with 91% of those aged 65 years or more. The estimated vaccine coverage varied during the 3 years between 76% and 82% in those aged at least 65 years, and between 38% and 44% in adults aged 18–64 years.

Information on Indigenous status was not recorded in the VicSPIN data. Fifteen Indigenous patients (nine influenza-positive) were recorded in the FluCAN data. VicSPIN included six pregnant patients, while FluCAN recorded 26 (including 19 who were influenza-positive).

Estimates of protection afforded by influenza vaccines were similar in both schemes. On the basis of the VicSPIN data, vaccine effectiveness against influenza for all age groups managed in general practice varied between 37% in 2011 (when the confidence interval included zero) and 61% in 2013 (Box 2). Vaccine effectiveness estimates changed by no more than four percentage points when the 0–17 years age group was omitted from analysis of the VicSPIN dataset. The pooled estimate for the 3 years was 50% (95% CI, 26%–66%). When the youngest age group was excluded in order to improve comparability with the data for hospitalised patients, the pooled vaccine effectiveness was 51% (95% CI, 27%–67%) (Box 2).

The estimates based on the data from the Victorian sentinel hospitals reporting to FluCAN varied between 35% in 2012 and 52% in 2013, with a pooled estimate of 39% (95% CI, 28%–47%). The crude and adjusted VicSPIN estimates were very similar to those of FluCAN, apart from in 2012. Point estimates were highest in both settings for 2013, and confidence intervals for each estimate included zero in 2011. The point estimates were higher in the general practice than the hospital setting in two of the three years when a significant protection could be demonstrated, but the confidence intervals for the two schemes overlapped in each year. The difference between the pooled vaccine effectiveness in general practice and hospital settings was 12% (P = 0.23).

Discussion

We found that the estimated protection provided by inactivated influenza virus vaccines, after adjustment for important confounders, was slightly higher in general practice than in hospital-based studies, but with overlapping confidence intervals. All FluCAN and most VicSPIN patients were recruited from metropolitan Melbourne and Geelong; very ill patients from regional Victoria can be transferred to any of the FluCAN hospitals.

Our estimates are similar to those found by other studies of similar design that have used PCR-confirmed influenza as the outcome,14 and suggest that older vaccine effectiveness estimates based on serology data15 or non-specific endpoints16 may have overestimated protection.

By targeting populations at risk of severe outcomes, the national immunisation program has assumed that the vaccine protects against severe outcomes associated with laboratory-confirmed influenza, such as hospitalisation, as well as influenza infections managed in the community. Our data support this view, while the small differences in the point estimates of vaccine effectiveness in community and hospital patients suggest that the influenza vaccine prevents hospital admission by preventing symptomatic infection rather than by attenuating the severity of illness. The difference in vaccine effectiveness may reflect the population at risk of hospitalisation, which includes people more likely to be elderly and to have comorbidities, characteristics that may be associated with impaired vaccine-induced immunity.17

A limitation of this study was the potential for selection bias, given that clinicians (GPs or hospital doctors) had discretion as to which patients were swabbed. However, we have shown there was no association between swabbing and vaccine status in VicSPIN patients during 2011–2014. In an unpublished study of 3649 patients with influenza-like illness who presented to VicSPIN GPs during the influenza seasons of 2011–2014, 2224 samples (64%) were submitted for testing. In the crude analysis, age, sex and year were associated with testing, but vaccination status was not. After adjustment, none of the variables were statistically associated with testing (Lisa McCallum, epidemiologist, Hunter New England Health; personal communication, 20 October 2015). We have not explored this association in the FluCAN dataset.

The selection of patients for inclusion in both VicSPIN and FluCAN distinguish them from surveillance schemes reporting the same outcomes, such as those in the United States18 and New Zealand,19 although the vaccine effectiveness estimates were broadly similar.

Another limitation of our study was that the two surveillance systems collected different covariate data, limiting combined data reporting. Details about covariates have been reported elsewhere.5,6,7,8,9,10 Influenza subtyping was incomplete in the FluCAN data, but the match between circulating and vaccine strains in the VicSPIN data has been previously explored.20 In all 3 years of our study, circulating influenza A(H1N1) and influenza B strains were matched to the vaccine strains. The influenza A(H3N2) subtype was matched in 2011, but was partially mismatched in the following two years.

A further substantial limitation of this study was that it was underpowered to detect small differences, if they existed, in the two clinical settings.

Vaccine status was incompletely reported in the FluCAN system, but ascertainment has improved in recent seasons. In particular, previous sensitivity analyses using multiple imputation have found similar estimates for vaccine effectiveness when comparing collected and imputed missing data; this suggests that missing data are unlikely to significantly bias estimates of vaccine effectiveness.8 A date of vaccination was provided for at least 85% of VicSPIN patients during this period.

Despite these shortcomings, published results from the VicSPIN studies are consistent with estimates of protection based on meta-analyses of community trial data.21,22 No reviews of efficacy in preventing hospital admission have been published because there have been no trials examining this outcome. However, in schemes that recruit patients from the same defined population in the same year, such as those conducted in Navarra (Spain) and Auckland (New Zealand), vaccine effectiveness estimates have been reported to be similar for hospital and community patients. For instance, vaccine effectiveness in Navarra during 2010–2011 was 75% (95% CI, 61%–84%) in preventing outpatient influenza cases, and 60% (95% CI, 37%–75%) in preventing influenza-associated hospitalisations.23 In Auckland, interim estimates of vaccine effectiveness against laboratory-confirmed influenza for 2014 were 67% (95% CI, 48%–79%) for presentation to a sentinel GP and 54% (95% CI, 19%–74%) for hospitalisation.24 While point estimates of protection in this and in two other studies where patients in community and hospital settings were recruited from the same population were higher for community than for hospital patients, the comparisons are illustrative rather than exhaustive; further, the confidence intervals for vaccine effectiveness estimates overlapped.

A randomised controlled trial of vaccine efficacy in averting hospital admission for laboratory-confirmed influenza might help to resolve the question. To account for annual variation in influenza circulation and vaccine effectiveness, however, this would require a study including tens of thousands patients conducted over more than one influenza season. As well as being extremely expensive, such a trial would not be ethical in view of current recommendations that all people aged 65 years and over, the age group most commonly admitted to hospital for influenza infection, receive the influenza vaccine.4,8,9,10 This emphasises the importance of observational studies in this context.

The Australian and international surveillance systems provide continuous data that support the effectiveness of the national influenza immunisation program. While the magnitude of benefit may not be as great as earlier studies had suggested, and while variation from year to year is acknowledged, influenza vaccination remains an important intervention for protecting vulnerable patients, as shown by our pooled analyses; this is especially true in the FluCAN setting, where a large majority of patients reported a comorbidity. Further evidence that protection against confirmed influenza infection managed in the community is similar to protection against hospitalisation will require additional studies in which patients in both clinical settings are drawn from exactly the same population.