Connect
MJA
MJA

The readability of official public health information on COVID‐19

Olivia A Mac, Danielle M Muscat, Julie Ayre, Pinika Patel and Kirsten J McCaffery
Med J Aust 2021; 215 (8): 373-375. || doi: 10.5694/mja2.51282
Published online: 18 October 2021

The information needs of people with lower health literacy or from culturally and linguistically diverse backgrounds have received limited attention during the coronavirus disease 2019 (COVID‐19) pandemic. In one study, the complexity of most government‐produced COVID‐19 information in Australia and overseas exceeded the recommended grade 8 reading level, making it too difficult for general audiences, let alone people with lower health literacy.1 We therefore compared the complexity (readability, understandability, actionability) of Australian and overseas COVID‐19 vaccination information with that of other COVID‐19‐related information (physical distancing, mask wearing), including “easy read” resources developed in Australia and New Zealand.

During March and April 2021, we selected public sources of information on vaccination, physical distancing, and face masks on government websites in Australia, New Zealand, and the United Kingdom, and by three overseas public health agencies: the Centers for Disease Control and Prevention, the European Centre for Disease Prevention and Control, and the World Health Organization (WHO). We also searched each site for resources labelled “easy read”. Readability was assessed with the Simple Measure of Gobbledygook (SMOG) index,2 which estimates the grade reading level (range, grade 5‒18); grade 8 is recommended for general audiences.3

We also used the widely employed Patient Education Materials Assessment Tool (PEMAT)4 to assess “understandability” (elements include word choice, use of visual aids, layout, and organisation) and “actionability” (the clarity of the recommended actions or steps). Two researchers independently scored each of the 24 PEMAT items (0 = disagree, 1 = agree, NA = not applicable), with disagreements resolved by discussion; the total score is the proportion of “Agree” responses, and a score of 70% is deemed “adequate”.4 We report PEMAT scores for “standard” content on a website by topic, as the linked pages were designed to be used together; we assessed the easy read items individually, as they are designed as standalone resources (further details: online Supporting Information).

The statistical significance of differences between medians for information topics (vaccination, physical distancing, mask wearing) was assessed separately for “standard” and “easy read” content in Kruskal–Wallis tests; P < 0.05 was deemed statistically significant. Analyses were conducted in Excel (Microsoft) and SPSS 26.0 (IBM).

All “standard” content exceeded the grade 8 reading level (median grade reading level, 12; range, 9–16); the median grade reading level was significantly higher for vaccination information (13; interquartile range [IQR], 12–14) than for physical distancing (12; IQR, 11–13; P = 0.007) or mask information (11; IQR, 10–12; P < 0.001). The complexity of twelve of the 21 “easy read” items exceeded the grade 8 reading level (range, 7‒10). The median reading level of “easy read” vaccination information (grade 10; IQR, 9–10) was significantly higher than for information about physical distancing (grade 7; IQR, 7–8; P = 0.030) or masks (grade 8; IQR, 7–8; P = 0.036) (Box).

The median understandability score of “standard” content was 88% (IQR, 79–93%; range, 25‒100%); 20 of 26 sources met the recommended understandability threshold of 70%. Median understandability scores were similar for vaccination (83%; IQR, 75–86%) physical distancing (93%, IQR, 79–93%), and mask‐related information (93%, IQR, 87–93%). The median actionability score of “standard” content was 80% (IQR, 60–100%; range, 0‒100%), but 13 of 26 sources did not meet the recommended actionability threshold (70%). All 21 “easy read” items met the threshold for understandability (median, 88%); five met the recommended actionability threshold. Inter‐rater assessment agreements were rated as substantial (Cohen κ > 0.70).

Twelve months into the pandemic, a considerable amount of COVID‐19 public health information in Australia and overseas rates poorly on measures of readability and actionability, particularly information about vaccination. “Easy read” content (with better readability and understandability) showed that it is possible to convey information about COVID‐19 in plain language, but such items were few in number and difficult to locate on websites.

Our study was limited by our exclusion of multimedia formats, and our findings reflect information available in April 2021.

Public health information should be easily accessible to the general public. Despite the WHO5 and the National Academy of Medicine6 endorsing plain language, practice has not been aligned with guidelines. The high health literacy demands of vaccination information in Australia has important implications for vaccination uptake.7,8 Successfully managing the COVID‐19 pandemic requires a whole‐of‐community response based on effective public communication and a commitment to health literacy.

 

Box – Readability, understandability, and actionability of official COVID‐19 public health information*

Information type and source

Number of webpages

Readability (SMOG): median (range)


Patient Education Assessment Tool (PEMAT):
median (range)


Understandability

Actionability


Vaccination (standard)

 

 

 

 

Australia

 

 

 

 

 Department of Health

11

13 (12–14)

80%

100%

 New South Wales

1

12

75%

60%

 Queensland

1

12

93%

60%

 Victoria

3

14 (13–15)

86%

60%

New Zealand

3

13 (11–14)

86%

60%

United Kingdom

1

12

83%

80%

Centers for Disease Control and Prevention

16

13 (11–16)

87%

100%

European Centre for Disease Prevention and Control

1

18

25%

0%

World Health Organization

1

15

62%

40%

Physical distancing (standard)

 

 

 

 

Australia

 

 

 

 

 Department of Health

1

12

93%

100%

 New South Wales

1

13

100%

80%

 Queensland

1

11

93%

60%

 Victoria

1

11

92%

60%

New Zealand

1

11

85%

60%

United Kingdom

1

11

77%

60%

Centers for Disease Control and Prevention

1

13

93%

100%

European Centre for Disease Prevention and Control

2

14 (12–15)

63%

60%

World Health Organization

1

11

93%

100%

Masks (standard)

 

 

 

 

Australia

 

 

 

 

 Department of Health

1

11

87%

100%

 New South Wales

1

10

100%

80%

 Queensland

1

10

93%

60%

 Victoria

7

11 (9–12)

94%

100%

New Zealand

3

9 (9–12)

93%

100%

United Kingdom

1

14

77%

60%

Centers for Disease Control and Prevention

7

10 (9–12)

87%

100%

World Health Organization

1

12

92%

100%

Vaccination (easy read)

 

 

 

 

Australia (Department of Health)

11

10 (8–10)

88% (87–88%)

60% (60–60%)

New Zealand

1

10

88%

60%

Physical distancing (easy read)

 

 

 

 

Australia

 

 

 

 

 Department of Health

1

7

88%

80%

 Victoria

1

7

88%

80%

New Zealand

1

8

88%

60%

Masks (easy read)

 

 

 

 

Australia

 

 

 

 

 Department of Health

1

7

88%

80%

 Victoria

3

7 (7–8)

88% (88‒88%)

60 % (60‒60%)

New Zealand

2

9 (8–10)

88% (88‒88%)

80% (80‒80%)


COVID‐19 = coronavirus disease 2019; SMOG = Simple Measure of Gobbledygook.  * The materials evaluated are listed in the online Supporting Information. Readability scores and “easy read” PEMAT scores were calculated separately for each webpage; PEMAT scores of “standard content” were calculated by topic.

 

Received 25 May 2021, accepted 28 July 2021

  • Olivia A Mac1
  • Danielle M Muscat1
  • Julie Ayre1
  • Pinika Patel1
  • Kirsten J McCaffery1

  • The University of Sydney, Sydney, NSW



Acknowledgements: 

We thank Erin Cvejic (University of Sydney) for his statistical advice.

Competing interests:

No relevant disclosures.

Author

remove_circle_outline Delete Author
add_circle_outline Add Author

Comment
Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

Online responses are no longer available. Please refer to our instructions for authors page for more information.