Coronavirus (COVID-19) vaccination information must pay attention to health literacy: analysis of readability of official COVID-19 public health information

Introduction

Throughout the coronavirus (COVID-19) pandemic there has been limited attention to the information needs of people with lower health literacy and people from culturally and linguistically diverse backgrounds. In April 2020, research showed that vast majority of government COVID-19 information developed in Australia and internationally exceeded the recommended 8^th grade reading level, rendering it too difficult for general audiences, notwithstanding those with lower health literacy.¹ Since then, mass global vaccination has begun bringing new information to communicate.  This study compares Australian and international COVID-19 vaccination information to other COVID-19 information (physical distancing and mask wearing) and provides an updated and local assessment of understandability and actionability including  ‘easy-read’ resources developed in Australia and New Zealand.

Methods

Between March and April 2021, we purposively selected consumer-facing information about vaccination, physical distancing, and face masks for COVID-19 from government websites of Australia (Federal and three states), New Zealand and the United Kingdom and three international Public Health Agencies (Centers for Disease Control and Prevention (CDC), European Centre for Disease Prevention and Control (ECDC) and World Health Organization (WHO). We also searched each site for resources labelled as ‘easy-read’. Readability was assessed using the Simple Measure of Gobbledygook (SMOG) which estimates grade reading level (range 5 to 18). Grade 8 is recommended for general audiences. To complement readability, we used the widely established Patient Education Materials Assessment Tool (PEMAT)². The PEMAT uses 24 items to measure ‘understandability’, including elements such as word choice, use of visual aids, layout and organisation; and ‘actionability’, which assesses the clarity of actions or steps the user can take to follow the health advice. Two researchers independently scored each PEMAT item (n=24; 1-Agree, 0-Disagree or NA-Not Applicable), with discrepancies resolved by discussion. Total score is a proportion of all Agree responses.³  A score of ³70% is considered adequate³. Agreement was assessed using Cohen’s kappa coefficient. Additional methodological details are provided in Appendix 1 (available in PDF) where we have also provided some exemplar websites differing by readability, understandability and actionability.

Statistical analyses used Microsoft Excel and SPSS version 26.0 (IBM). Kruskal-Wallis tests compared medians across topics (vaccination, physical distancing and mask wearing); significance defined as P<0.05. Analyses are presented separately for ‘standard’ and ‘easy read’ content.

Results

All standard content exceeded an 8^th grade reading level. The median grade reading level was 12.0 (interquartile range [IQR], 2.0; Table 1 - available in PDF) The median grade reading level of vaccination information (13.0; IQR, 2.0) was significantly higher than physical distancing (11.5; IQR, 2.0; p=0.007), and mask information (11.0; IQR, 2.3; p<0.001). Twelve of the 21 easy-read content (55%) exceeded an 8^th grade reading level. The median grade reading level of easy-read content about vaccination (10.0; IQR, 1.0) was also significantly higher than information about physical distancing, (7.0; p=0.03) and masks (7.5; IQR, 1.5; p=0.036).

The median understandability score of standard content was 87.7% (IQR 16.4). Most standard content (n=20/24, 83%) met the recommended understandability threshold (>70%). Median understandability scores for vaccination were 83.3% (IQR, 17.9) ,  92.8% (IQR, 21.7) for physical distancing, and 92.8% (IQR, 7.0) for masks. These differences were not significant (p=0.074). The median actionability score of standard content was 80% (IQR, 40) however,  less than half (n=14/24, 46%) met the recommended actionability threshold. All easy-read content (n=12, 100%) met the threshold for understandability (median, 87.5%; IQR, 0.6), while only 3 (25%) met the actionability threshold (median, 60.0%; IQR, 20.0). Kappa agreements were rated as substantial (>0.70).

Discussion

Twelve months since the onset of the pandemic, much of the COVID-19 public health information in Australia and internationally still performs poorly on readability metrics, and on measures of ‘actionability’. Of concern, vaccination information was the hardest to read, understand, and act upon according to widely used metrics, compared to information about physical distancing and face masks. The existence of easy-read content (with better readability and understandability) demonstrates it is possible to explain COVID-19 topics in plain language, however, they were few in number and difficult to locate on the website.

Public health information should be easily accessible to the general population. Despite plain language being endorsed by WHO and the Institute of Medicine for over a decade, there remains a disconnect between guidelines and practice internationally³.

The high health literacy demands of vaccination information is a pertinent issue in Australia and has important implications for vaccine uptake .^4,5 Successful management of COVID-19 requires a whole of community response. This necessitates effective public communication and a strong commitment to health literacy.

References

1. Mishra V, Dexter JP. Comparison of readability of official public health information about COVID-19 on websites of international agencies and the governments of 15 countries. JAMA Netw Open 2020; 3: e2018033.
2. Shoemaker SJ, Wolf MS, Brach C. Development of the Patient Education Materials Assessment Tool (PEMAT): A new measure of understandability and actionability for print and audiovisual information. Patient Educ Couns 2014; 96: 395-403
3. Agency for Healthcare Research and Quality. Health Literacy Universal Precautions Toolkit, 2^nd edition. Jan 2015. https://www.ahrq.gov/health-literacy/improve/precautions/toolkit.html (Viewed May 2021).
4. Leask J, Carlson S, Attwell K, et al. Communicating with patients and the public about COVID-19 safety: recommendations from the Collaboration on Social Science and Immunisation. Med J Aust 2021; 215: 9-12.e1
5. Abdi I, Murphy B, Seale H. Evaluating the health literacy demand and cultural appropriateness of online immunisation information available to refugee and migrant communities in Australia. Vaccine 2020; 38: 6410-6417