After the surge in adverse events following immunisation (AEFI) among children receiving seasonal influenza vaccine in April 2010, the limitations of existing passive vaccine safety surveillance in Australia have been widely acknowledged.1-3
In an effort to remedy the inherent constraints of passive AEFI reporting, the Illawarra Medical Centre (IMC) in Perth, Western Australia, began exploring opportunities to implement active surveillance of vaccine safety. In 2011, IMC collaborated with a software developer (Datavation) to create a tool that extracts vaccination data from the clinic’s existing, commercially available practice management software. This new tool, called SmartVax, can send a short message service (SMS) text message to patients who have been vaccinated, then automatically collate any replies received via SMS.
In this study, we aimed to assess SmartVax’s performance in terms of response rates and timeliness.
During the assessment period, 2134 individuals contributed a total of 3281 vaccination visits for analysis. Of the patients contributing the 3281 vaccination visits, 3226 (98.3%) had a mobile number in their record and were sent an SMS enquiring about any reaction to the vaccine(s) (Box 1). Of the patients who were sent an SMS, 2342 (72.6%; 95% CI, 70.0%–75.1%) responded with a reply SMS. A further 374 patients (11.6%; 95% CI, 10.3%–12.9%) did not respond by SMS but were subsequently contacted by telephone. The remaining 510 patients (15.8%; 95% CI, 13.3%–18.3%) were uncontactable.
There was no significant difference between paediatric and adult patients in regards to the overall proportion who replied by SMS (73.9%; 95% CI, 71.4%–76.3% v 69.8%; 95% CI, 67.5%–72.1%) (Box 2). In the adult group, women were significantly more likely than men to reply by SMS (73.1% [427/584] v 65.3% [269/412]; P < 0.01).
Between patients who replied by SMS and those who did not but were subsequently contacted through a phone call, there was no significant difference in the proportion reporting a reaction (11.3%; 95% CI, 9.9%–12.7% v 11.8%; 95% CI, 9.2%–14.4%; P = 0.99).
The SMS replies were very timely. Half the patients (1212/2342, 51.8%) responded within 10 minutes of receiving the SMS, and more than 80% of responses (1944/2342) were received within 2 hours.
The analysis restricted to data from the first vaccination visit for each individual produced very similar results: the response rate remained 70% or greater for both paediatric and adult patients, and more than 80% of SMS responses were received within 2 hours.
In Australia and other countries, post-licensure monitoring of vaccine safety relies largely on passive surveillance. The constraints of passive AEFI reporting systems are well recognised and include underreporting, biased reporting and the inability to establish rates.3-5 These limitations may result in delayed detection of potential safety signals.6
The SmartVax tool permits active monitoring of vaccine safety by extracting clinical data from practice management software and using SMS technology to query vaccinated patients. Our evaluation identified several potential advantages with this approach. First, the response rate to the outgoing SMS was ≥ 70% for both paediatric and adult patients. Second, the information received was very timely, with more than 80% of all SMS replies received within 2 hours after the outgoing message was sent. Such timeliness could be valuable when an urgent investigation into potential vaccine safety issues is necessary, and it would permit ongoing safety monitoring in near real-time. Last, we found that the proportion of patients who reported an adverse reaction was similar between those who replied by SMS and those who did not but were subsequently contacted by telephone. This finding suggests that, during routine monitoring of vaccine safety, it may not be necessary to telephone people who do not respond to the SMS, substantially reducing the staff resources required to maintain the system.
We know of one other study that investigated the use of SMS text messages to monitor vaccine safety. In 2012, researchers in Cambodia testing an open-source SMS-based tool to monitor adverse events among 184 adult patients receiving vaccinations found a response rate (72%) similar to ours.7 This suggests SMS may be a viable method of conducting AEFI surveillance in a variety of settings worldwide.8
One of the strengths of the SmartVax system is that it seamlessly interfaces with the existing practice management software in our practice, reducing the amount of effort required to identify vaccinated patients for inclusion in active surveillance. The medical practice software in use at IMC is popular in Australia,9 and it is likely that SmartVax could be modified to work with other commonly used software packages. Also, because the SMS replies are linked to data extracted from the patient record, this system enables the responses to be assessed in the context of the type and number of vaccines administered, with ready access to brand and batch number. Another strength is that when more detailed information for suspected vaccine reactions is obtained through subsequent telephone interviews, this information can be directly entered into SmartVax through a data entry screen and is immediately available for aggregation and review.
The major limitation of the system, as currently configured, is that it requires clinic personnel to telephone patients who respond affirmatively to experiencing suspected AEFI, to ascertain the nature and severity of the alleged reaction. In our experience, the process of personally contacting the small subset of respondents with suspected AEFI is not particularly onerous and is certainly more efficient than attempting to interview all vaccinated patients as part of an active AEFI surveillance system. Even so, this approach necessarily offsets some of the benefits of using an otherwise very timely SMS-based reporting system. A possible solution to gathering additional clinical information on patients responding affirmatively to the initial SMS, while minimising the staff resources involved, might be to send the patient a series of sequential text message questions or a link to a web-based survey; we are exploring these options.
The ability of an AEFI monitoring system to detect potential safety signals early is predicated, in part, on having large numbers of patients included in the surveillance program. By developing an AEFI monitoring system that works with a widely used practice management software application, and that could likely be modified to work with others, there is potential to expand the number of participating practices to achieve a large representative sentinel population. If this type of AEFI surveillance were to be implemented in other practices throughout Australia, de-identified results could be aggregated so that meaningful sample sizes for individual vaccines could be attained in a timely manner. Such a system could make a valuable contribution to assuring vaccine safety when new vaccine formulations are introduced, including any future rollout of pandemic influenza vaccines.
In summary, the experiences with seasonal influenza vaccine in Australia in 2010 highlight the limitations of relying solely on passive surveillance for early detection of vaccine safety problems.3 Active postmarketing surveillance of vaccine safety using SMS technology has the capacity to complement passive reporting systems, potentially enabling more rapid identification of emerging safety signals. Further development and evaluation of systems like SmartVax are warranted.
1 Follow-up of short message service (SMS) query to patients about adverse events following immunisation, Illawarra Medical Centre, Western Australia, 11 November 2011 – 10 June 2013
2 Response rate to short message service (SMS) query about adverse events following immunisation, Illawarra Medical Centre, Western Australia, 11 November 2011 – 10 June 2013, by age group and sex of vaccinated patients*
* Response rate was defined as the percentage of patients sent an SMS who responded with a reply SMS. Bars indicate 95% confidence intervals. Paediatric patients were those aged ≤ 12 years who received one or more routine childhood vaccinations, as appropriate for their age. Adult patients were those aged ≥ 18 years who received seasonal influenza vaccine.
Received 9 September 2013, accepted 28 November 2013
- Alan Leeb1
- Annette K Regan2
- Ian J Peters3
- Candice Leeb1
- Gregory Leeb4
- Paul V Effler5,2
- 1 Illawarra Medical Centre, Perth, WA.
- 2 School of Pathology and Laboratory Medicine, University of Western Australia, Perth, WA.
- 3 Datavation, Perth, WA.
- 4 Bond University, Brisbane, QLD.
- 5 Communicable Disease Control Directorate, Department of Health, Western Australia, Perth, WA.
Alan Leeb and Ian Peters are co-creators of the prototypic software system described here. At present, the system is not under copyright and is not commercially available. However, if in the future this software were to be licensed and sold commercially, they might derive financial benefit.
- 1. Armstrong PK, Dowse GK, Effler PV, et al. Epidemiological study of severe febrile reactions in young children in Western Australia caused by a 2010 trivalent inactivated influenza vaccine. BMJ Open 2011; 1: e000016.
- 2. Horvath J. Review of the management of adverse events associated with Panvax and Fluvax: final report 10 March 2011. Canberra: Australian Government Department of Health and Ageing, 2011. http://www.health.gov.au/internet/immunise/publishing.nsf/Content/11DFBB4 FD968D072CA25789400172DA1/$File/adverse-event-march-2011.pdf (accessed Oct 2013).
- 3. Gold MS, Effler P, Kelly H, et al. Febrile convulsions after 2010 seasonal trivalent influenza vaccine: implications for vaccine safety surveillance in Australia [editorial]. Med J Aust 2010; 193: 492-493. <MJA full text>
- 4. Mahajan D, Roomiani I, Gold MS, et al. Annual report: surveillance of adverse events following immunisation in Australia, 2009. Commun Dis Intell Q Rep 2010; 34: 259-276.
- 5. Zhou W, Pool V, Iskander JK, et al. Surveillance for safety after immunization: Vaccine Adverse Event Reporting System (VAERS) — United States, 1991–2001. MMWR Surveill Summ 2003; 52: 1-24.
- 6. Rosenthal S, Chen R. The reporting sensitivities of two passive surveillance systems for vaccine adverse events. Am J Public Health 1995; 85: 1706-1709.
- 7. Baron S, Goutard F, Nguon K, Tarantola A. Use of a text message-based pharmacovigilance tool in Cambodia: pilot study. J Med Internet Res 2013; 15: e68.
- 8. Kaplan WA. Can the ubiquitous power of mobile phones be used to improve health outcomes in developing countries? Global Health 2006; 2: 9.
- 9. Best Practice Software. Media statement. 15 Apr 2013. MIMS Matters 2013; Winter Edition: 7. http://www.mims.com.au/content/MimsMatters/WinterMMJul13.pdf (accessed Mar 2014).
Abstract
Objective: To assess the performance of SmartVax, a prototypic active monitoring system for adverse events following immunisation (AEFI) using short message service (SMS) text messages and clinical data extracted from commercially available medical practice management software.
Design, setting and participants: Between 11 November 2011 and 10 June 2013, adult patients and parents of paediatric patients receiving routine vaccinations in general practice were sent an SMS by SmartVax enquiring if they had experienced any AEFI and requesting a reply by SMS. Attempts were made to telephone patients who did not reply by SMS.
Main outcome measures: The proportion of patients sent an SMS who replied by SMS, and the proportion of respondents indicating possible AEFI.
Results: Of 3281 vaccinated patients, 3226 (98.3%) had a mobile telephone number on record and were sent an SMS. Of 2342 patients (72.6%; 95% CI, 70.0%–75.1%) who responded by SMS, 264 (11.3%; 95 CI, 9.9%–12.7%) reported possible AEFI. The response rate was ≥ 70% for both paediatric and adult patients. Eighty-per cent of SMS replies were received within 2 hours of transmission of the query SMS. There was no significant difference in the proportion reporting possible AEFI between patients who replied by SMS and those who did not respond by SMS but were subsequently contacted by a telephone call (P = 0.99).
Conclusions: More than 70% of patients responded by SMS to an SMS query about whether they had any vaccine reactions, with the data received in near real-time. Active surveillance of AEFI using SMS has the capacity to complement existing passive reporting systems, potentially permitting more rapid identification of emerging safety signals.