Clinical practice guidelines: the need for greater transparency in formulating recommendations

The quality of CPG bears little relation to the level of seniority or expertise of guideline authors.9 Guideline panellists often harbour COI that may not be fully evident, even to the panellists themselves, but which can potentially bias their recommendations.10 These conflicts include not only financial ties with industry but also practice reimbursement incentives, professional affiliations and practice specialisation, intellectual attachment to their own studies, ideas and innovations, and desire for academic recognition and career advancement.11 The most entrenched conflict can be a disinclination to challenge or reverse strongly held beliefs. Using research evidence to make recommendations requires subjective interpretations, which will be influenced by the value structure of panel members.12 Vulnerability to preconceptions is greatest for recommendations based on low-quality evidence — an increasingly frequent occurrence in contemporary CPG13 — although recommendations based on high-quality evidence are far from invulnerable.

Most current guidelines remain susceptible to COI, which can impinge on all stages of the CPG development process (Box 1). Many are published without peer review or, if contained in journal supplements, escape the standard of peer review applied to articles published in the parent journal.14 Moreover, many guidelines (79% in a recent survey of Australian guidelines15) fail to mention possible competing interests of guideline panellists. Even if COI are disclosed, guideline users may not adjust their perceptions of recommendations in response to such disclosures.16

Strategies for dealing with COI are outlined in Box 1,17-19 with key strategies being:

• Nominated panellists must disclose all industry-related professional activities, including research grants and speaker support, and, for the duration of guideline development, divest themselves of direct financial interests (stock ownership, board positions, consultancy agreements) in commercial companies with an interest in any guideline recommendation.

• Panellists are required to identify all sections of the draft guidelines for which they have COI. These conflicts are recorded in a COI grid maintained by the guideline chairperson.

• Methodologists free of financial or intellectual conflicts of interest share responsibility with content experts for collecting and interpreting evidence.

• Explicit processes must be used to assess evidence quality and link this directly with strength of recommendations.

• Only conflict-free panellists (both methodological and content experts) are involved in determining the direction (for or against a specific clinical action) and strength of recommendations.

• Lack of consensus around evidence quality or recommendations is resolved by explicit democratic processes (such as Delphi rounds and nominal group techniques) involving conflict-free panellists who have thoroughly reviewed the related evidence.

• Individuals should be invited to join guideline panels through an open, transparent application process centred on selection criteria that ensure an appropriate balance of content and methodological expertise. Such criteria may comprise extent of clinical experience with the topic in question, prior participation in undertaking critically appraised literature reviews, intended commitment in time and intellectual input into the guideline development process, and referee reports. For guidelines that deal with common conditions and are aimed at large, multidisciplinary audiences, panel composition should reflect the spectrum of end users and avoid being dominated by a narrow spectrum of specialists.

The impact on guideline content if such policies were implemented and enforced is yet to be empirically determined,20 but many organisations involved in guideline development have now adopted at least some of them as best practice for reducing the probability of conflicted panellists having undue influence.17-19

Clinicians lose confidence in CPG when separate guidelines on the same clinical topic from seemingly authoritative sources produce conflicting recommendations. For example, United States and European CPGs differ in their recommendations for use of anticoagulants in acute coronary syndromes.21 The ways in which guideline panellists have interpreted and weighted the evidence and used it to formulate recommendations of different strength must be clearly communicated.

While various systems exist for rating evidence according to hierarchies of study design, with randomised controlled trials (RCTs) at the top, most contain no explicit processes for assessing evidence quality or linking it with recommendations.22 The Grading of Recommendations Assessment, Development and Evaluation (GRADE) system (Box 2) attempts to meet this need23 and has advantages over other grading systems in that it:

• clearly separates quality of evidence and strength of recommendations;

• explicitly evaluates alternative management strategies;

• provides clear-cut, detailed criteria for downgrading and upgrading quality of evidence ratings related to different outcomes that are of importance to patients and which exclude surrogates;

• provides a transparent process for moving from evidence to recommendations and grading recommendations as strong or weak on the basis of clearly defined, pragmatic interpretive criteria;

• explicitly acknowledges patient preferences; and

• details potential resource use.

Adopting the GRADE system may assist in exposing and mitigating bias arising from COI, thus augmenting COI policies. Applying the GRADE quality of evidence classification to contemporary CPG suggests that many (in one study almost 50%24) RCT-derived recommendations fail to meet a-priori definitions of high-quality evidence. Several examples now exist of how the GRADE system promotes the development of CPG recommendations that are more aligned with evidence quality.25,26

More than 50 organisations worldwide have adopted the GRADE system, including the World Health Organization, the American College of Physicians, the Cochrane Collaboration, the Scottish Intercollegiate Guidelines Network (SIGN) and UpToDate (online clinical decision support system). In Australia, the National Health and Medical Research Council (NHMRC) has recently produced a revised draft schema for more explicit, structured grading of evidence quality and strength of recommendations which has several similarities — and differences — to the GRADE system.27

Guidelines commonly base their recommendations on trials involving selected populations and standardised interventions. These may not be applicable to unselected populations receiving care from clinicians working under real-world constraints.3 Benefits reported in trials may not be reproducible in patient groups, such as older patients with multiple comorbidities, that are underrepresented in such studies or in clinical settings very different to those used in trials.28

Guidelines tend to focus on single clinical conditions in isolation (such as heart failure or acute coronary syndromes) and may not adequately address situations where the management of commonly encountered comorbidities (such as asthma, diabetes or dementia) may conflict with, or override, recommendations for the index condition.

Guideline panellists should assess the extent to which evidence of treatment benefit is consistent, or even exists, across different populations with different comorbidity spectra, in different settings and with different modes of treatment administration. The circumstances under which the magnitude of treatment benefit (and harm) is significantly enhanced or attenuated should be highlighted in the way recommendations are presented. Recommendations should, where appropriate, stratify populations according to disease risk and target treatments to those who will experience greatest net benefit.29

Ideally, in developing the guideline, panels should seek feedback from a separate reference group of front-line clinicians who are likely to use the guidelines (also chosen by an open application process and subject to the same disclosure policies as panellists) regarding the impacts and feasibility of guideline recommendations and the extent to which proposed guidelines meet their care needs.

Guideline authors must avoid exercising power without responsibility in obliging clinicians and health services to enact recommendations and satisfy guideline-based performance measures with little regard to the added problems and pressures these may engender in terms of professional interactions, team functioning, organisational predispositions, resource availability and medicolegal considerations.3,30

Following guideline release, a process of public consultation should exist (as it does for NHMRC guidelines) that allows a representative cross-section of health managers, quality improvement experts, and patient support groups to provide feedback on the wider environmental implications of specific CPG recommendations. In future iterations of the guideline, authors should respond to comments by reviewing and, where appropriate, modifying recommendations that have been identified as particularly problematic. Guideline authors may also consider asking other agencies to undertake formal cost-effectiveness analyses or modelling exercises related to recommendations that raise concerns about the availability and cost of resources.