Current challenges in appraising complementary medicine evidence

In Australia, the use of complementary and alternative medicine (CAM) has grown,1 in concert with increased global research in specific areas such as herbal medicine.2 Although research in the field is essential to assess the safety and potential efficacy of CAMs, there are vital considerations in appraising the current state of evidence. A recent article by Dwyer in the Journal pointed out examples of current areas that lack evidence.3 However, certain CAM interventions do have evidence, and the entire industry should not be tarred with the same brush. For example, a snake oil spray for miracle weight loss should not be lumped in with a standardised St John’s wort formulation for depression.4 A major overarching issue that currently prohibits confidence in the CAM industry that needs to be addressed is the potential for selective publication of results (publication bias). This may result in clinicians and the public being misinformed about the level of evidence for certain CAM products, as results from negative studies may be swept under the carpet. This experience applies to both Australian-based and international companies. Although some companies are ethical and accept publication of negative results, negative results are sometimes obfuscated or buried.

Selective publication of CAM study results has been raised previously. In 2005, Vannacci and colleagues5 commented that there is potential for CAM companies, like their pharmaceutical counterparts, to selectively report studies, as their ultimate goal is financial profit. It could be further argued that there is additional incentive for the CAM industry to publish selectively, due to a limited budget for research (thus replicated studies are rare) and because CAM products (in most countries) are not required to demonstrate proof of efficacy — if it’s already selling well, why rock the boat. Sponsor bias relating to published studies also exists. One critical analysis of CAM homotoxicology product trials noted common issues concerning a lack of declaration of conflicts of interest, with some coauthors of publications affiliated with the company sponsoring the study.6 Although it is not imperative for academics with related company positions to abstain from coauthoring research publications, such relationships must obviously be declared. Beyond the unlikely awakening of corporate conscience, the obvious solution to address publication bias is to increase study registration, and to improve accountability of the registered studies through mandatory publication of results. The creation of trial registration sites such as the Australian New Zealand Clinical Trial Registry (www.anzctr.org.au) is vital to ensure that studies can be traced and monitored, and that the results can be accessed. Regardless, even study registration does not guarantee that the results will be published. Another solution that may not be entirely palatable, but may appease CAM companies, is the acceptance by journals of negative results de-identifying the product brand. A caveat of this approach would be that the natural product’s chemical profile and pharmacokinetics must be detailed, to differentiate it from other brands that may have supportive evidence.

Increased government funding for conducting objective and unbiased clinical trials would also be of great benefit; however, this is unlikely to occur in the near future, and large robust trials are expensive. This situation is highlighted by the CAM industry’s peak research body, the National Institute of Complementary Medicine, which, after receiving government support in 2007, has had to cease official operation due to a lack of funding.7 Thus, the current situation lies in the hands of academics, clinicians and the public, who must raise their voices to demand from the CAM industry better quality research, with greater transparency in reporting results.

Another important issue is that although high-quality CAM studies exist, poorly designed studies still persist. These potentially elicit positive results and provide companies with good marketing fodder, but limit conclusions about the true efficacy of the intervention tested. Therefore, a challenge exists in educating the public about current CAM evidence. One element of this involves communication about the differences between evidence-based complementary medicines and therapies and slick marketing campaigns involving unsubstantiated gimmicky products. Public education on the difference between the relative quality of products is also needed as, quite simply, “oils ain’t oils”. A suggested (and somewhat controversial) method to convey this information to the public could involve grading a product’s evidence using a “traffic-light” system or a “tick of approval”.8

Another concern is that some companies may selectively borrow positive evidence from studies of standardised high-quality products to promote their own inferior products (which may not be standardised or have any similar chemical equivalence, and may have little or no active constituents). One way of encouraging the industry to conduct more research is for their intellectual property to be protected by only allowing the researched product to hold these claims of efficacy (or, potentially, other chemically equivalent products). Company tax breaks to promote research and development would also encourage more investment; however, this incentive must be tethered to full disclosure of results to the public, to better inform their health care decisions.

Although increased research in the field of CAM is important, many considerations exist, such as the need to increase the quality of research, minimise publication bias, protect intellectual property, and separate gimmicky products from bona fide complementary therapies and medicines. In the long term, this will ultimately enhance the standing of the CAM industry and increase consumer confidence.