Cancer is a major cause of death and disability and, as such, attracts strong media attention.1,2 The implication is clear: everyone is at risk of cancer;3 therefore, everyone has a vested interest in finding out about the latest preventive measures, diagnostic tests, and treatments for cancer. This is especially true for people who are directly affected by the disease who actively seek new information in the hope of improving their prognoses.4
There has been extensive study of how cancer is portrayed in the media. However, studies have tended to focus on one or two types of cancer (breast or skin), one source of news (online), or one type of therapy (medication).5 Analyses have also been qualitative in nature, examining the content of the reports for tone or themes.6 To our knowledge, no previous analyses have used quantitative assessments of the quality of information contained in news reports about cancer.
We describe an in-depth analysis of the content, context and quality of cancer reporting in Australian media during the 5-year period from June 2004 to June 2009 using data collected by the media-monitoring website Media Doctor Australia (http://www.mediadoctor.org.au). Media Doctor Australia posts reviews of health news stories, published in Australian media, including newspaper articles, online news stories, and television and radio broadcast transcripts, and aims to provide an objective analysis of the strengths and weaknesses of news stories about treatments and procedures intended for use in humans that the articles describe as new. The methods used for data collection and analysis have been described previously.7-10 Similar websites have been launched in Canada, the United States and Hong Kong using identical methods for appraising health news stories.11,12
A trained Media Doctor researcher collects news stories from individual media outlet websites. Each story, with any relevant material such as media releases and journal articles, is sent for evaluation to two of the 15 Media Doctor volunteer reviewers, who are clinicians, medical writers, journalists and public health specialists.
The two reviewers rate stories independently using a validated 10-item rating instrument (Box 1).9 For each news story, the 10 items are rated as “satisfactory”, “not satisfactory” or “not applicable”. The total score is the percentage of all completed items that are rated satisfactory. A higher score reflects a more informative and complete story. The current mean score for all health news articles on the Media Doctor website is 51%.9 Stories rating higher than this can be considered superior, but may still have significant weaknesses; it is therefore important to look at individual item scores when assessing each article. In our experience, good articles score in the 80%–100% range, but less than 10% of reports fall into this category.9 A short commentary from the reviewers is also posted. This is used to highlight strengths or weaknesses and areas not covered in the rating instrument, such as sensationalist language or inappropriate headlines. Any discrepancies in reviewers’ ratings are resolved by consensus or, if necessary, by using a third reviewer.
We conducted a review of cancer-related reports (“cancer stories”) critically reviewed by Media Doctor Australia from June 2004 to June 2009. Although multiple reports may be published by different media outlets on the same topic, all the cancer and other stories we analysed were unique. We included only stories about newly reported interventions under four broad categories — pharmaceutical interventions, diagnostic tests, surgical interventions and “other” (including complementary therapies, physiotherapy, and dietetic interventions). We searched the Media Doctor database for all news stories about cancer within these four categories. The search terms included, for example, “cancer”, “leukaemia”, “melanoma” and “tumour”. All other stories served as a comparison group.
A mixed-methods approach was used for analysis of data and story content.
We calculated the total scores for cancer stories and other stories by category and by type of media outlet. Data were normally distributed; we used analysis of variance (ANOVA) to compare unweighted scores between story categories and media outlets. All statistical calculations were made using StatsDirect, version 2.3.6 (Stats Direct, Sale, Cheshire, UK).
For analysis of the content and tone of cancer stories, we entered text into NVivo, version 8.0.148.0 (QSR International, Melbourne, Vic). Our qualitative analysis followed the grounded theory method.13 Coding was done by text analysis and concepts that we expected to emerge, based on our previous experience. Stories were coded for concepts, and coded segments of text were then analysed and compared for similarities and differences, and categorised into themes.
A third of the reports were coded by a single coder on two separate occasions to ensure consistency in coding. A further subset of 62 reports (23%) was re-coded by a second coder, with any coding discrepancies resolved by discussion between the coders.
For qualitative analysis of emotive language, text coded as “emotive language” was linked to that coded under “sources”. Sources were divided into subcategories including government departments, support groups, researchers, independent sources, and patients.
A search of the Media Doctor website identified a total of 1323 stories about new treatments and procedures between June 2004 and June 2009, of which 272 (20.6%) dealt with interventions for cancer. Of these, the text of four older stories could not be located, and these were not included in the content analysis. However, their scores were included in the quantitative analysis.
Overall, across three of the intervention categories (pharmaceutical, diagnostic and other), cancer stories rated higher than other stories (Box 2). Typically, the differences between mean scores were three to four percentage points. The exception was reports about surgical procedures, where cancer stories rated poorly, although the sample size was very small (nine cancer stories). Overall, the difference between mean scores for cancer stories and other stories across all four categories combined was statistically significant (F = 7.1; df = 1; P = 0.008). However, differences in pairwise comparison of cancer stories and others for the individual categories were not statistically significant (data not shown). Many cancer stories described new medicines, and cancer was the primary focus of around 20% of all stories about new drugs. About 40% of all stories about diagnostic tests were related to cancer (Box 2).
Most cancer stories concerned disease affecting the breast or prostate gland. Comparisons of the rates of different types of cancer in Australia and the proportion of reports covering these diseases demonstrated that breast cancer was over-represented as a topic (Box 3).
Broadsheet newspapers scored higher than online news services or tabloid newspapers, and the overall difference between these outlets was statistically significant (F = 6.1; df = 2; P = 0.003) (Box 4). Television had the lowest scores. Broadsheet newspapers scored between 6 and 21 percentage points higher than other outlets, and pairwise comparisons were statistically significant for broadsheet versus online scores (F = 12.7; df = 1; P < 0.001) and versus television (F = 10.7; df = 1; P = 0.001).
Most cancer stories (222/268; 83%) referred to their sources of information. The predominant source (160/222; 72%) was the researcher or doctor who had tested or administered the intervention; the next most common source (71/222; 32%) was an independent expert (someone recognised as having specialised knowledge but not connected with the research or funding body). Industry sources comprised 11% of sources (24/222). Of 148 stories citing the researcher, only four (3%) reported ties between the expert and a company. In each of these cases, the researcher was employed by or owned the company.
Many stories (98/222; 44%) referred to a medical journal research paper as a source, and 36 different journals were identified. The five most commonly mentioned journals comprised 57% of all journals cited, and all had high impact factors (Box 5).
More than a third (105/268; 39%) of the cancer stories featured estimates of the disease burden of a particular type of cancer. Morbidity and mortality were variously expressed relative to other cancers, in whole numbers, in subgroups (eg, age or sex) and in time periods (eg, every day or annually). National and international figures were quoted, sometimes without making clear which were being used. The wide variety of descriptions of the impact cancers have at the population level was confusing and sometimes meaningless (Box 6).
There were numerous examples of highly emotive language and use of adjectives and literary devices such as hyperbole, analogy and metaphor, as well as extensive use of personal narrative. However, a “war” or “military” metaphor was not commonly used in these reports, with words such as “strike”, “battle”, “weapon”, “victim”, “war” and “desperate” occurring less than 10 times each.
A small number of reports (39/268; 15%) used the narrative technique, which usually took the form of a patient relating his or her experience of cancer. In 10 reports, this technique comprised 20% or more (54% in one story) of the text. About half (19/39) of these narratives contained highly emotive language or related personal stories with moving or disturbing themes, as in the following example.
[E J] had a lot of living to do when she was told she had an aggressive form of bone cancer. At 18, doctors discovered she had a “galaxy of tumours”. She lost her leg and underwent intensive chemo treatment to try and stop it spreading. Her cancer was so severe that she relapsed soon after. (A Current Affair [television program] 9 Aug 2004.)
At least five stories used narratives as the primary source of information. Most were positive stories about cure or improved prognosis; however, none of these referred to any evidence beyond personal experience.
“The [positron emission tomography] scan was immediately conclusive so it was a tremendous relief ... I am just very fortunate that the machine is here and that I had access to it and I truly hope that it is more widely accessible to more women”, Ms [M] said. (The Age [Melbourne] 13 Sep 2006.)
Only a few reports contained narratives that were negative, and all were about a new intervention being proposed for government funding, such as Herceptin (Roche):
Herceptin is subsidised only for women with advanced breast cancer — small comfort for [the patient], who was diagnosed with HER2-positive breast cancer in March ... [The patient]’s doctor recommended she take Herceptin, but she cannot afford it. (The Australian 20 Oct 2006.)
Most reports (22/33) coded as containing narrative text contained emotive language or hyperbole, compared with less than half (43/104) of the reports coded as containing evidence-based information (ie, results derived from high-quality studies).
One hundred and eleven of 269 reports (41%) contained overtly emotive language in a direct quote from a source. Most of these (60/111; 54%) were attributed to researchers talking about their work. For example:
This is fantastically significant for the 2800 Australian men who die of the disease every year ... I think it’s going to wipe out a hell of a lot of these deaths. (Courier Mail [Brisbane] 22 Jul 2008.)
Some researchers took part in what appeared to be overt self-promotion.
Other researchers promoted the intervention they had been studying.
Our analysis of the content, context and quality of Australian news stories over a recent 5-year period has shown that reporting of cancer in the mainstream media continues to be of poor quality, particularly stories carried by television and tabloid newspapers. This is troubling considering the distress caused by cancer and its treatment.
Individual narrative or testimonial is a device widely used in the media and seen to add the human dimension of a story. However, it provides anecdotal information, which is the lowest form of evidence, and this can be misleading if it is not balanced by strong evidence.
Although broadsheet newspapers performed better than other types of media outlets, the differences were small, and major weaknesses in reporting the cost of the intervention, potential harms of treatment, the magnitude of treatment benefits and the quality of the clinical evidence remained.9 Specialist health journalists, employed by most broadsheet newspapers, produce high-quality stories compared with other authors, which may explain the higher scores achieved by the broadsheets.10
A key finding from our study was the frequent use of emotive words and metaphor. Media Doctor does not specifically criticise (or score down) the use of a dramatic narrative. However, we believe journalists and researchers have a particular responsibility not to raise patients’ hopes unreasonably. This view is reflected in guidelines for journalists, including those of the Australian Press Council.14 Although use of the fight or war metaphor to describe diseases such as cancer and AIDS has been used for many decades,15-17 we did not find substantial evidence of it. The term “aggressive” was widely used, and this may be because it has become an everyday part of the medical vernacular — “aggressive therapy”, “aggressive cancer” — and, as such, has lost its emotional impact for health professionals.
The medical literature often contends that the “blame” for poor-quality news coverage of cancer lies with the media.3,18 However, we found otherwise, in that most of the hyperbole and emotive statements were attributable to the researchers. By raising their profiles and awareness of their research through media coverage, researchers, their institutions and industry alike stand to benefit in terms of grant funding and profits.19
Likewise, public relations employees working for research institutions are compromised in their selection of research to promote. Their positions are based on achieving media coverage for their institutions. As such, they will choose stories they know will appeal to the general public.20 Media officers write or edit research press releases, often supplying quotes from researchers and narratives by patients that may be more sensational than the research warrants or the researchers would wish. In the interest of promoting good health literacy, researchers should ensure they have the final say on what information is released; however, this does not always happen.
The findings of our study are limited to media reporting on new cancer interventions, not all types of cancer reporting. Also, some categories included in our analysis (eg, stories about surgical interventions) involved low numbers of news stories. While a consistent and comprehensive strategy was used to collect all eligible stories over the 5-year period, it is probable that some stories were missed, and these may have included stories about cancer. However, any incomplete sampling was random and the study provides a broad and representative sample of cancer stories in Australian media.
We propose that researchers have a responsibility to present their findings to the media in a manner that increases the probability of accurate reporting by the mainstream media. The host institutions, research institutes, universities and hospitals share this responsibility, as do journals. In this context, we endorse previous calls for journals to do more.21-23 Journals take a great deal of interest in authors — the accuracy of their work, their academic freedom and competing interests — to ensure that information on new medical therapies is presented accurately to the public. We believe the responsibilities of journals should extend to monitoring the post-publication coverage of the research, including researchers’ behaviour, by assessing the accuracy of media coverage of the work they publish. The Media Doctor rating instrument provides a useful checklist for this purpose.
1 Items in the Media Doctor rating instrument
* For each news story, the 10 items are rated as “satisfactory”, “not satisfactory” or “not applicable”. |
|||||||||||||||
2 Mean Media Doctor scores for cancer-related stories and other reports* about new health interventions and procedures in Australian media, June 2004 – June 2009
3 Proportion of cancer-related stories reported in Australian media, June 2004 – June 2009, and 2006 annual Australian incidence and mortality,* by cancer type
* Per 100 000 population; calculated using the Australian Standard Population, 2001; data retrieved from <http://www.aihw.gov.au/cancer/data/acim_books/index.cfm>. † Percentages do not add to 100% as only main cancer types are shown. |
|||||||||||||||
4 Mean Media Doctor scores for cancer-related stories reported in Australian media, June 2004 – June 2009, by type of media outlet
5 Five most frequently cited medical journals in news stories about cancer reported in Australian media, June 2004 – June 2009
6 Variation in descriptions of the disease burden attributable to prostate cancer in Australian media reports, June 2004 – June 2009
- Amanda J Wilson1
- Billie Bonevski2,3
- Alison L Jones4
- David A Henry5
- 1 School of Medicine and Public Health, University of Newcastle, Newcastle, NSW.
- 2 Centre for Health Research and Psycho-oncology, University of Newcastle, Newcastle, NSW.
- 3 Cancer Council New South Wales, Sydney, NSW.
- 4 University of Western Sydney, Sydney, NSW.
- 5 Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada.
We thank the Media Doctor reviewers for their time and expertise.
None identified.
- 1. Atkin CK, Smith SW, McFeters C, Ferguson V. A comprehensive analysis of breast cancer news coverage in leading media outlets focusing on environmental risks and prevention. J Health Commun 2008; 13: 3-19.
- 2. Australian Bureau of Statistics. Cancer in Australia: a snapshot, 2004–05. Canberra: ABS, 2006. (ABS Cat. No. 4822.0.55.001.)
- 3. Clarke JN, Everest MM. Cancer in the mass print media: fear, uncertainty and the medical model. Soc Sci Med 2006; 62: 2591-2600.
- 4. Chen X, Siu LL. Impact of the media and the internet on oncology: survey of cancer patients and oncologists in Canada. J Clin Oncol 2001; 19: 4291-4297.
- 5. Wilson PM, Booth AM, Eastwood A, Watt IS. Deconstructing media coverage of trastuzumab (Herceptin): an analysis of national newspaper coverage. J R Soc Med 2008; 101: 125-132.
- 6. Lewison G, Tootell S, Roe P, Sullivan R. How do the media report cancer research? A study of the UK’s BBC website. Br J Cancer 2008; 99: 569-576.
- 7. Smith DE, Wilson AJ, Henry DA, media doctor study group. Monitoring the quality of medical news reporting: early experience with media doctor. Med J Aust 2005; 183: 190-193. <MJA full text>
- 8. Bonevski B, Wilson A, Henry DA; media coverage of complementary and alternative medicine. PLoS One 2008; 3: e2406.
- 9. Wilson A, Bonevski B, Jones A, Henry D. Media reporting of health interventions: signs of improvement, but major problems persist. PLoS One 2009; 4: e4831.
- 10. Wilson A, Roberston J, McElduff P, et al. Does it matter who writes medical news stories? PloS Med 2010; 7: e1000323.
- 11. Cassels A, Lexchin J. How well do Canadian media outlets convey medical treatment information? Initial findings from a year and a half of media monitoring by Media Doctor Canada. Open Medicine 2008; 2 (2): 20-23.
- 12. Schwitzer G. How do US journalists cover treatments, tests, products, and procedures? An evaluation of 500 stories. PLoS Med 2008; 5: e95.
- 13. Strauss A, Corbin J. Basics of qualitative research: techniques and procedures for developing grounded theory. 2nd ed. London: Sage Publications, 1998.
- 14. Australian Press Council. Reporting guidelines. (General press release No. 245.) Sydney: APC, April 2001.
- 15. Lupton D. Medicine as culture: illness, disease and the body in Western societies. 2nd ed. London: Sage Publications, 2003.
- 16. Wallis P, Nerlich B. Disease metaphors in new epidemics: the UK media framing of the 2003 SARS epidemic. Soc Sci Med 2005; 60: 2629-2639.
- 17. Sontag S. Illness as metaphor and AIDs and its metaphors. 1st Anchor Books ed. New York: Doubleday, 1990.
- 18. MacKenzie R, Chapman S, Barratt A, Holding S. “The news is [not] all good”: misrepresentations and inaccuracies in Australian news media reports on prostate cancer screening. Med J Aust 2007; 187: 507-510. <MJA full text>
- 19. Chapman S, Nguyen TN, White C. Press-released papers are more downloaded and cited. Tob Control 2007; 16: 71.
- 20. Avery EJ, Lariscy RW, Sohn Y. Public information officers’ and journalists’ perceived barriers to providing quality health information. Health Commun 2009; 24: 327-336.
- 21. Woloshin S, Schwartz LM. Press releases: translating research into news. JAMA 2002; 287: 2856-2858.
- 22. Woloshin S, Schwartz LM, Casella SL, et al. Press releases by academic medical centers: not so academic? Ann Intern Med 2009; 150: 613-618.
- 23. Woloshin S, Schwartz LM, Kramer BS. Promoting healthy skepticism in the news: helping journalists get it right. J Natl Cancer Inst 2009; 101: 1596-1599.
Abstract
Objective: To describe an in-depth analysis of the content and quality of stories about new cancer interventions in Australian media.
Design and setting: Search of the Media Doctor Australia media-monitoring website for stories about newly reported cancer interventions, including drugs, diagnostic tests, surgery and complementary therapies, that had been collected from June 2004 to June 2009 and rated for quality using a validated rating instrument. A mixed-methods approach was used to analyse data and story content. Data from the website on stories about other new health interventions and procedures were compared.
Main outcome measures: Differences in quality scores between cancer-related news stories (“cancer stories”) and other stories, and between types of media outlet; differences in how cancer was reported in terms of cancer type, morbidity, mortality, and in the use of hyperbole and emotive language.
Results: 272 unique cancer stories were critically reviewed by Media Doctor Australia. Cancer stories had significantly higher scores for quality than other stories (F = 7.1; df = 1; P = 0.008). Most cancer stories concerned disease affecting the breast or prostate gland, with breast cancer appearing to be over-represented as a topic relative to its incidence. Pairwise comparisons showed statistically significant superiority for broadsheet newspaper stories over online stories (F = 12.7; df = 1; P < 0.001) and television stories (F = 10.7; df = 1; P = 0.001). Descriptions of morbidity and mortality were variable and often confusing in terms of numbers, time periods and locations. Literary devices including hyperbole and emotive language were used extensively, mostly by the researchers.
Conclusions: While reporting of cancer in the general media is of low quality, many of the poorer aspects of content are directly attributable to the researchers. Researchers and journals need to do more to ensure that a higher standard of information about cancer is presented to the media.