Content analysis of Australian direct-to-consumer websites for emerging breast cancer imaging devices

Despite evidence from multiple randomised controlled trials (RCTs),1 there is continued controversy surrounding the effectiveness of mammographic screening at reducing breast cancer mortality relative to its potential harms.2,3 Amid this ongoing debate, a number of emerging breast imaging devices have entered international consumer markets.4,5 These devices operate outside of publicly funded screening and diagnostic programs, are advertised directly to consumers and can be accessed without a referral from a health care practitioner. Three such devices currently gaining market traction include digital infrared thermal imaging (DITI), electronic palpation imaging (EPI) and electrical impedance scanning (EIS).

Online direct-to-consumer advertisements that compare these devices with established imaging practices have recently gained the attention of regulators, researchers, and the National Health and Medical Research Council (NHMRC).4,6,7 On 9 June 2011, the Australian Competition and Consumer Commission, Cancer Council Australia and the Therapeutic Goods Administration issued a joint media release urging Australian women not to rely on unproven commercial breast imaging devices to detect breast cancer.8

Prompted by a series of public complaints, several Australian companies have been faced with sanctions requiring the removal or amendment of misleading advertising claims of performance,9-11 ultimately resulting in the Federal Court finding in March 2014 that two Perth companies had engaged in misleading conduct.12

While these isolated instances demonstrate some degree of action taken on behalf of Australian regulators, the quality of information presented on websites for medical devices in the broader industry remains largely reliant on self-regulation and consumer complaints.

Due to the current regulatory environment for medical device advertising in Australia, the quality of content found in advertisements outside of the complaints process is currently unknown. However, a number of recent systematic reviews have indicated that the evidence base to support these devices for breast cancer imaging is highly variable, and therefore do not recommend their use for either breast cancer screening or diagnosis.13-15 In light of this development, a review of the advertising material for these devices is required to ensure that their promotion in Australia is both appropriate and ethical. In the present study, we aimed to describe the nature and frequency of information presented on Australian direct-to-consumer websites for three emerging breast cancer imaging devices.

Methods

Search strategy

Websites were chosen for analysis because they have been the primary medium used to promote emerging breast imaging tests in Australia. A systematic search aimed at capturing all relevant websites was conducted on the three largest internet search engines by market share (Google, Yahoo, Bing) on 1 March 2011. We searched for synonyms of “electrical impedance”, “electronic palpation” and “thermography” in combination with the key terms “breast AND Australia”. Websites were included if they were located in Australia and advertised either DITI, EPI or EIS directly to consumers for breast imaging. Included websites were monitored monthly for changes from 2 March 2011 to 30 March 2012, and a repeat search was conducted in March 2012 to capture any new websites. Information provided in a direct link to an external website was considered an extension of the companies' promotional material, and was also included in the analysis.

Codebook development

The coding scheme was developed both deductively and inductively (Appendix).16,17 In the first instance, the main code categories (eg, comparator, safety) were derived from diagnostic technology assessment literature so that advertising claims could be compared with peer-reviewed evidence for each device.18 Subcode categories (eg, direct comparison) were developed inductively from a sample of included websites, as the primary goal of the study was to describe the overall content in advertisements.16,17 These emergent variables were initially drafted by two of us (TV and CL) during the development of the coding scheme, and revised through a process of consensus with the coders.

Coder training

Coding was carried out by two independent coders in combination with the principal author (TV). Coders were trained using a sample of international websites that were not included in the final analysis, and a pilot coding exercise was conducted to assess the reliability and validity of the coding scheme. Intercoder reliability was tested on a subset of included websites before the formal coding process,16 and was measured using the proportional reduction in loss (PRL) approach.19 A baseline reliability score of at least 0.80 was established for all code categories during three training sessions, after which the remaining websites were divided evenly among the coders.

Data analysis

Data were coded with NVivo version 9.2 (QSR International), and matrices were then imported into Stata version 12 (StataCorp) for analysis. The number of websites that referenced a code was the primary unit of analysis, in lieu of the total number of codes, so that each website contributed equally to the analysis regardless of size. This prevented larger websites and linked external websites from misrepresenting the extent to which codes were reported overall. For example, while the benefits of a comparator were reported in 15 individual instances, half of these instances were from one website.

Results

The search strategy identified 39 relevant Australian websites; 28 during the initial search, and an additional 11 after a repeat search at the end of the 12-month monitoring period. Twenty-four websites promoted DITI, 10 promoted EPI, and five promoted EIS. Six Australian websites promoted links to an external website, and one provided downloadable resources from external authors, including journal articles and conference posters. However, the addition of external material had minimal impact on the study results, contributing between one and three additional codes for 24 of 38 code subcategories. The final coding scheme comprised six main categories: performance, application, safety, target population, comparison with conventional imaging and evidence, as well as 38 subcategories. The PRL reliability scores for each main category are shown in Box 1.

Device performance

The advertised ranges of diagnostic accuracy (DITI, 88%–90%; EPI, 94%; EIS, 85%–90%), sensitivity (DITI, 78%–99%, EPI, 80%–92%; EIS, not applicable) and specificity (DITI, 44%–91%; EPI, 72%–88%; EIS, not applicable) were relatively high compared with published literature for each technology. As shown in Box 2, the degree to which the device could determine the risk of developing disease was advertised by four DITI websites and one EIS website, and was reported on two DITI websites as being 10 times more significant than a family history of the disease20,21 and on one DITI website as “5–8 years ahead of mammography as a risk indicator”.22 Of the 18 websites that advertised measures of diagnostic performance (accuracy, sensitivity or specificity), only eight provided references to peer-reviewed journal articles to support their claims. Three DITI websites claimed over 30 years of research compiling over 600 peer-reviewed studies on breast thermography,23-25 but did not provide references to any of the studies mentioned.

Application of device

Devices were advertised for a wide range of applications, including diagnosis (22 websites), adjunct diagnosis (19), screening (20), adjunct screening (14), prevention of disease (13) and identifying breast cancer risk factors (13). Websites most commonly used broad terms such as early detection (33) and monitoring breast health (31) to promote the device (Box 2); however, only a small number of sites advertised the devices for these broad indications exclusively (five). Interestingly, while nine websites stated that their device could not provide a clinical diagnosis, five of the same websites also advertised their device for the diagnosis of breast cancer. Peer-reviewed evidence to support using an advertised device for the diagnosis of breast cancer was presented by only 10 websites (Box 2). Further, the studies referenced on DITI websites often included studies published before 1990, which tested the application of an older device that is no longer used in practice.

Device safety

Claims relating to the safety of devices focused on their non-invasiveness (30 websites), lack of physical pain (28), lack of breast compression (28) and lack of exposure to radiation (31) (Box 2). In addition to the physical safety risks associated with a device, a small number of websites mentioned risks due to technical limitations. Specifically, nine websites indicated that the device being promoted was not capable of offering a clinical diagnosis while a further five websites suggested that a negative test result was not necessarily a guarantee of good health. Peer-reviewed literature was not presented as evidence for safety by any of the included websites. In lieu of such evidence, three websites offered testimonials from either patients or health professionals, while a further 16 websites stated that either their staff members or imaging device had received regulatory approval as an endorsement that the device was safe (Box 3).

Target population

Twenty-five websites promoted breast imaging devices primarily for use on all women, followed by women with high breast density (22 websites), young women (21), women with breast implants (13) and men (12). The age range implied by the term “young women” was not always clearly indicated; however, sites that did report an age range targeted women aged 18 to 40 years. Three DITI websites stated that the device was suitable for use on children under the age of 18 years. Further, 12 websites suggested that the device was suitable for women who do not participate in mammographic screening or are not eligible for a screening mammogram (Box 2).

Comparison with conventional breast imaging technology

Direct comparisons between the device and a conventional imaging technology were frequently made (31 websites). References to the benefits of a conventional imaging technology were identified in seven websites, such as being able to detect tumours in the pre-invasive stage,21 while 18 websites made mention of their limitations, eg, “Mammograms expose your breasts to harmful radiation”.34 Sixteen websites also used indirect comparisons (Box 2), by emphasising the characteristics of their device that appear preferable to mammography, eg, “no radiation”, or “no discomfort”. In addition to these comparisons, 12 websites explicitly stated that their device could be used as an alternative to conventional breast imaging, while 11 suggested that their device should not be used as an alternative to conventional imaging techniques. However, these codes were not mutually exclusive, as four websites that claimed their device was not an alternative to a comparator device also stated that it was a suitable option for women who do not want to have a mammogram. RCT evidence to support mammography was not presented on any observed websites.

Change over time

Of the 28 websites that were monitored during the 12-month period between March 2011 and March 2012, nine altered their advertising material, five discontinued the breast imaging service, and a further seven websites were closed. Two of the websites that changed their advertising material added a statement indicating that their device was not an alternative to a conventional screening mammogram; however, the remaining changes were minor and did not affect the overall content of advertisements.

Discussion

Problems were identified relating to the manner in which information was presented about the performance, application, safety and target population of the devices. While the results of this study are presented within a local context, these devices are also available under similar arrangements and have faced similar criticisms internationally.4 DITI is currently available in 20 countries worldwide,35 EPI in 15 countries36 and EIS in three countries.37 The findings of this study therefore raise important considerations about the manner in which these devices and their advertising material are regulated around the world.

The advertised accuracy of devices differed from that reported in several recent systematic reviews.13-15 Three systematic reviews of DITI reported broad ranges of sensitivity (25%–97%) and specificity (12%–85%) for detecting breast cancer in symptomatic populations,13-15 and two reviews reported sensitivity ranges between 25% and 70% and specificity between 62% and 79% in asymptomatic populations.14,15 Similarly, a systematic review of EIS reported wide ranges of sensitivity (26%–98%) and specificity (8%–81%) in symptomatic populations, and only identified one study of screening in asymptomatic populations that showed poor results (sensitivity, 26%; specificity, 81%).13 The same review identified one low-quality study for EPI (sensitivity, 88%; specificity, 84%) in symptomatic populations, and no evidence for use in asymptomatic screening.38 This large degree of variation in the supporting evidence base for each class of device was not evident in the online advertising content for these devices, which instead presented very high, narrow ranges of diagnostic and screening performance.

In light of the reviews' findings, there is currently insufficient evidence to support the indications for which these devices were often advertised, namely breast screening and diagnosis and identifying risk factors for breast cancer.13-15 Quality appraisal conducted in the reviews noted significant methodological deficiencies in the peer-reviewed literature, specifically, the lack of blinding of investigators to the results of prior tests, inappropriate selection criteria, and the removal of technical errors from the results.13-15 The same studies identified in the reviews were reported on websites without mention of study design limitations, an example of which is provided in Box 3. In addition, no evidence from RCTs for these devices was identified, contrasting with the current level of evidence supporting conventional imaging devices, specifically mammographic screening.1,13-15 As such, advertising claims that promote their use as suitable methods for breast cancer screening and diagnosis are not evidence-based, and therefore may present safety risks to women, particularly if they choose the device over an established imaging tool.4 Despite this lack of evidence, 31 of 39 websites advertised their device in direct comparison with mammography screening.

All diagnostic and screening devices have safety implications beyond their risk of inflicting physical harm, which relate to their ability to correctly differentiate between malignant and benign lesions.39 Despite these implications, few websites provided information about diagnostic sensitivity and specificity. It has also been argued that consumers can find it difficult to interpret reported measures of diagnostic accuracy, and instead need this information translated into a form that is easily understandable.40 Only five websites addressed this issue, by advising consumers not to accept a test result as a guarantee. Safety issues relating to the performance of a device should be made explicit and understandable to consumers, especially in a direct-to-consumer market where there is no prerequisite involvement from a health care practitioner.

Our trained and experienced coders sometimes had difficulty discerning between technical and diagnostic sensitivity in advertising claims. As such, it seems less likely that members of the public will be able to easily distinguish the difference. This was a key issue in instances where companies referenced the term “sensitivity” in a general manner, eg, “ultra-sensitive, high resolution digital infrared cameras”,41 without describing what it was sensitive to. Advertising information presented in this way is difficult to interpret and can be misleading, as a claim meant to represent the high technical sensitivity of the device for measuring a test characteristic may be misinterpreted as a claim of high diagnostic sensitivity for detecting breast cancer.

The results of this study should be interpreted with regard to its limitations. First, coding definitions for sensitivity were deliberately restricted to account for the differentiation between diagnostic and technical measures, and therefore broad claims of sensitivity, such as “a highly sensitive device” were not coded or accounted for in the analysis. Second, as Australian websites were the primary source of information, the generalisability of these results to other countries may be limited. However, similar patient safety issues relating to online advertising for medical devices, including these breast imaging devices, have been noted in the United States and United Kingdom.4,42,43 Third, links to external websites may not be regarded as direct advertising by Australian websites. However, it is argued that this material would not have been otherwise affiliated with the Australian website if it had not been actively promoted, and was therefore relevant to the analysis. Finally, while our search strategy aimed to identify all relevant websites offering breast imaging devices in Australia, it is possible that some clinics without websites may have been missed.

Contrary to published evidence, digital infrared thermal imaging, electrical impedance and electronic palpation devices are advertised on Australian websites as safe and effective solutions for breast cancer screening and diagnosis. These devices are advertised towards populations, and for uses that are not supported by evidence. As these devices are accessible without a referral from a registered doctor, women who are informed solely by advertising information material face serious potential safety risks. It is clear that reforms to the current model of regulation for medical device advertising in Australia are needed.