Clinicians' attitudes to clinical practice guidelines:  a systematic review

Farquhar, Cynthia M; Kofa, Emma W; Slutsky, Jean R

doi:10.5694/j.1326-5377.2002.tb04920.x

Topics

Abstract

Objective: To systematically review surveys of clinicians' attitudes to clinical practice guidelines.

Data sources: MEDLINE, HealthStar, Embase and CINAHL were searched electronically for English-only surveys published from 1990 to 2000.

Study selection: We included surveys with responses to one or more of seven propositions (see below). Studies were excluded if they had fewer than 100 respondents or if the response rate was less than 60%.

Results: Thirty studies included responses to one or more of the seven items, giving a total of 11 611 responses. The response rate for the included studies was 72% (95% confidence interval [CI], 69%–75%). Clinicians agreed that guidelines were helpful sources of advice (weighted mean, 75%; 66%–83%), good educational tools (71%; 63%–79%) and intended to improve quality (70%; 60%–80%). However, clinicians also considered guidelines impractical and too rigid to apply to individual patients (30%; 23%–36%), that they reduced physician autonomy and oversimplified medicine (34%; 22%–47%), would increase litigation (41%; 32%–49%) and were intended to cut healthcare costs (52.8%; 39%–66%).

Conclusions: Surveys of healthcare providers consistently report high satisfaction with clinical practice guidelines and a belief that they will improve quality, but there are concerns about the practicality of guidelines, their role in cost-cutting and their potential for increasing litigation.

Clinical practice guidelines (CPGs) are defined as "systematically developed statements to assist practitioner and patient decisions about appropriate healthcare for specific clinical circumstances."1 They are tools used by healthcare professionals to assist in clinical decision-making and to improve healthcare for patients. Medical clinicians have used treatment recommendations, immunisation schedules, algorithms, textbooks and practice bulletins to guide practice for many years,2 but the difference over the past decade has been the increasing focus on summarising research to develop evidence-based recommendations. This more rigorous approach to the development of guidelines involves a multidisciplinary team representing various stakeholders and working with explicitly described methods.3-5

The motivation behind CPGs is to improve the quality of healthcare and outcomes for patients.5 One simple definition of quality in healthcare is providing the right care, at the right time, for the right person in the right way.6 CPGs should improve healthcare quality both at a clinical level and by influencing policies that promote efficient allocation of resources and better delivery systems.7

Producers of clinical practice guidelines are encouraged to base guidelines on research evidence, validity, reliability, and clinical applicability, as these characteristics are associated with better adherence to the recommendations within the guidelines.8,9 However, in spite of the considerable efforts of many healthcare organisations over the past 20 years, there is limited evidence of the impact of CPGs on clinical practice or health outcomes. Although some studies have shown that guidelines have improved the quality and consistency of healthcare,10-12 there is concern that guidelines have not always delivered the predicted improvements in clinical care.13-15

It is widely perceived that CPGs are not popular with clinicians. CPGs have been variously described as anti-intellectual, standardising practice around the average, preventing discretion in individual cases, cost-cutting, limiting innovation and clinical freedom and encouraging litigation.16,17 These attitudinal barriers have the potential to limit implementation.

In this study, we systematically reviewed surveys of clinicians' attitudes to CPGs to find out just how high these attitudinal barriers might be.

Methods

Data sources

Articles published in English from 1990 to 2000 were sought using Medline, HealthSTAR, Embase and CINAHL electronic databases.

The keywords located in the title, abstract, or subject used to select articles, were "clinical practice guidelines," "clinical practice guideline survey," "clinical guideline surveys," "guideline adherence," "CPG — knowledge, attitudes, practice, evidence-based medicine — survey — physician," "medical practice guidelines — survey," and "attitudes — practice guideline recommendations." In addition, the references listed in articles found in the database search were searched by hand. Consensus statements were considered to be guidelines.

In an attempt to access the unpublished literature, members of an electronic mailing list on evidence-based medicine (evidence-based-healthAT jiscmail.ac.uk) were sent the list of identified studies and asked about unpublished surveys or other surveys that had not been identified by our search strategy.

Articles were included if they included survey data on medical or allied health clinicians' attitudes to CPGs in general. Exclusion criteria were articles written before 1990 (the definition of clinical practice guidelines was published by the Institute of Medicine in 1990 1); editorials about CPGs; published CPGs; surveys that dealt with attitudes to specific guidelines only; studies that did not provide suitable data for abstraction; surveys of managers or representatives of professional organisations; surveys with a response rate below 60%; and surveys with fewer than 100 respondents.

Each abstract was reviewed by two authors (E K, C F) and, where uncertainty existed about including particular articles, the full-text article was requested, and a consensus about inclusion reached. Full-text copies of all included articles were obtained. A list of the excluded studies is available from the authors.

Data extraction and analysis

The following data were collected from each report: year, author, organisation affiliation, country of origin, guideline focus, health practitioner surveyed, type of survey, number of surveys (sent and returned), rate of responses to items on attitudes. The percentage of positive responses to seven propositions was sought from each survey. These seven propositions were common to many of the surveys and evolved from one of the earliest surveys.18 These were that guidelines:

were a helpful source of advice
were good educational tools
were intended to improve quality of care
were impractical and too rigid to apply to individual patients
reduced physician autonomy and were an oversimplification of medicine or "cookbook" medicine
will increase litigation or disciplinary action
were intended to cut healthcare costs.

In some surveys only the mean Likert scale was given and therefore the data were not suitable for extracting. Attitudes to many other propositions about CPGs were surveyed, but not in a consistent way that could be reliably extracted from each survey.

We calculated means (weighted by the number of respondents for each survey), and pooled confidence intervals. The influence of response rate was explored by calculating weighted mean responses for surveys with a ≥ 75% response rate for the seven items and comparing these with the weighted means for the total sample (≥ 60% response rate). Responses from the United States were compared with responses from the other countries. Differences in the response rates were explored by comparing weighted mean responses between groups. To assess differences over time in the response to individual items, weighted means were calculated for three time intervals (1990–1994, 1995–1997, 1998–2000). Statistical differences between weighted means were assessed by t-tests. All analyses and statistical tests were performed with SAS software (Cory, NC).

Results

We found 153 surveys of attitudes toward CPGs, and 30 (20%) met the inclusion criteria. Eighteen surveys were excluded because of a response rate less than 60% and seven because they had fewer than 100 respondents. The total number of respondents in the included surveys was 11 611.

Box 1 summarises the surveys included in the study.18-47 Nine were surveys of primary care doctors, general practitioners or family physicians, 14 of hospital doctors, three of health maintenance organisation doctors and four of a mixed group of healthcare professionals. Ten surveys were from the United States, five from Canada, five from the United Kingdom, four from Australia, two from Italy and one each from Israel, Denmark, the Netherlands and Ireland.

The weighted mean response rate of the surveys included in the study was 72% (95% confidence interval [CI], 69%–75%). Sixty-seven percent of the surveys had two or more follow-ups, either by telephone or additional mailings. Financial incentives were only offered in three surveys.

The responses to the seven propositions about CPGs are presented in Box 2. Most respondents agreed that guidelines were helpful sources of advice (weighted mean, 75%), good educational tools (weighted mean, 71%) and intended to improve quality (weighted mean, 70%). Just over half (weighted mean, 53%) thought that guidelines were intended to cut healthcare costs. Sizeable minorities thought that guidelines were impractical and too rigid to apply to individual patients (weighted mean, 30%); reduced physician autonomy and were oversimplified or "cookbook" medicine (weighted mean, 34%); or would increase litigation (weighted mean, 41%).

There were no significant differences in the responses for any of the seven items between those surveys with a response rate of ≥ 60% and those with a response rate of ≥ 75%; between United States surveys and surveys elsewhere; or between the three time categories (1990–1994, 1995–1997, 1998–2000).

Discussion

We sought to review clinician attitudes to clinical practice guidelines as shown in studies from diverse settings. Most clinicians in the surveys included in this report were supportive of CPGs, finding them to be useful, educational and likely to improve quality. Less frequent responses were that clinical practice guidelines were impractical, unable to be used for individual patients, limited clinician autonomy, increased the likelihood of litigation or disciplinary action and were used to cut costs.

Our approach of reviewing surveys from multiple sources has obvious limitations. Few of the surveys were planned to be compared with others. Because of differences in the context and wording of the surveys, our combined data should be interpreted cautiously. However, as the number of surveys and the total number of respondents was large, some insight into clinician attitudes is provided.

Many of the surveys initially identified had a poor response rate, which might raise doubts about bias in the response. Eighteen studies were excluded from the review for this reason. Only three surveys used incentives to increase response rate. Consideration could be given in future to offering incentives to increase the response rate. A recent systematic review has suggested that continuing medical education credits can significantly improve survey response rates.48

The strength of this review is the number of surveys with respondents from a variety of backgrounds, and the consistency of responses across these differing settings and countries. Although only four of the 36 surveys were Australian, this consistency suggests that the results of this review could be generalised to the Australian context.

Our chief finding is that, contrary to popular belief, negative clinician attitudes are not the major barrier to implementation of CPGs. There has been some concern that guidelines have failed to change practice and to reduce variations in practice.10-14,16 Clinician intransigence is often suggested as the cause,17,49-52 but this review does not bear the theory out.

There are some areas of concern for clinicians that should not be overlooked. For example, nearly half of all respondents considered that CPGs increased the chances of litigation or disciplinary action, although there is little evidence that guidelines have actually been used in this way.53,54 In the US, where CPGs have perhaps proliferated the most, guidelines played "a relevant or pivotal role in the proof of negligence" in less than 7% of medicolegal cases.55

Underlying the medicolegal concerns may be a concern that CPGs will be used to set standards or develop regulations. In the UK, the National Health Service has agreed that, even when CPGs are produced by a professional body or endorsed by the NHS, they are only aids to decision making, and cannot be used in a regulatory fashion.56 In the US, the standard of care against which malpractice liability is measured is defined informally as "what a similar doctor would do under similar circumstances". If there is evidence that most clinicians agree with a CPG, then it will serve as a standard against which conduct will be measured. However, if a guideline is ignored then it is unlikely to be used as a standard in the courts.57 As "good medicine is good law" there may in fact be some protection for clinicians who adhere to guidelines.

Another major concern among survey respondents was that CPGs are aimed at reducing healthcare costs. In fact, although healthcare savings are an often-expressed hope when introducing CPGs, there is only limited evidence that guidelines have reduced costs, except where misuse of a procedure or medication was widespread.58,59 If an effective treatment or intervention has been underused, the implementation of CPGs can actually increase costs.

Does it matter that clinicians actually like CPGs? The finding that most respondents agreed that CPGs are useful, educational and likely to improve quality does not necessarily translate into practice changes. The nature of modern healthcare makes it improbable that individual clinicians could make significant changes without some educational, organisational and structural changes in the healthcare system at either local or regional level.60 There have been several initiatives, both in the US and Australia, to support efforts to implement evidence from research into everyday clinical settings.61,62 Focusing on what we can do better in healthcare is the way forward for guideline development and implementation and has the potential for success, given that practitioner attitudes to guidelines are, for the most part, positive. Guideline development and implementation programs should be encouraged by these findings, but should also recognise that addressing clinicians' concerns is necessary if their programs are to succeed.

1: Surveys of attitudes to guidelines included in this systematic review

Year	First author	Affiliation	Clinicians	Country	Guideline focus	Respondents	Response rate

1990	Grol19	Nederlands Huisartsen Genootschap	Primary care doctors	Netherlands	Diabetes care	453	70%
1990	ACS20*	American Cancer Society	Primary care doctors	US	Cancer preventive services	1029	74%
1992	Nowells21	HMO	HMO doctors	UK	No specific guideline	194	86%
1993	Madhok22	None	Senior house officers	UK	Head injuries	140	80%
1994	Tunis18	American College of Physicians	Internists	US	No specific guideline	1513	60%
1995	Shye23	HMO	HMO primary care doctors	US	No specific guideline	168	80%
1995	Mansfield24	None	Hospital specialists	US	No specific guideline	268	66%
1996	Hornbrey25	None	Senior house officers	UK	No specific guideline	164	71%
1996	Grilli26	None	Physicians	Italy	No specific guideline	216	72%
1996	Ferrier27	None	Family physicians	Canada	No specific guideline	395	70%
1996	Pathman28	None	Family physicians/ paediatricians	US	Vaccine recommendations	1421	66%
1997	Hayward29	None	Primary care doctors/specialists	Canada	No specific guideline	1513	60%
1997	Gupta30	None	Primary care doctors	Australia	General practice	286	77%
1997	Salem-schatz31	HMO	HMO physicians	US	HMO guidelines	203	77%
1997	Alston32	None	Anaesthetists	UK	No specific guideline	144	64%
1997	Dodek33	None	Healthcare professionals	Canada	Arterial blood gas measurement	107	90%
1997	Olesen34	College of General Practice	Primary care doctors	Denmark	Diabetes	293	79%
1998	Wolff35	None	Family physicians	US	No specific guideline	205	65%
1998	Carrick36	National Breast Cancer Centre	Surgeons	Australia	Early breast cancer	150	64%
1998	Browman37	Cancer Care Ontario	Oncologists	Canada	Cancer practice guidelines	304	72%
1999	Halm38	None	Hospital specialists	US	Pneumonia	140	89%
1999	Girgis39	None	Primary care doctors	Australia	Prostate cancer	145	65%
1999	Grilli40	AIOM/SIOG^†	Gynaecological oncologists	Italy	No specific guideline	120	70%
1999	Cotton41	None	Nurses/general practitioners	UK	Scottish Intercollegiate Guideline Network guidelines	437	74%
1999	Shah42	None	Cardiologists/ cardiac surgeons	Australia	NHMRC‡ cardiac guidelines	110	63%
1999	Constantini43	None	Hospital staff	US	No specific guideline	254	67%
2000	Harris44	ACOEM§	Mixed healthcare providers	US	Occupational and environmental medicine	238	63%
2000	Dye45	None	Obstetricians/ gynaecologists	Ireland	No specific guideline	114	67%
2000	Graham46	Cancer Care Ontario	Oncologists	Canada	Cancer Practice Guidelines	584	73%
2000	Vinker47	None	Family physicians	Israel	Diabetes guidelines	293	83%
				Total responses		11611
				Weighted mean response rate (95% CI)			72% (69%–75%)

* American Cancer Society. †Associazione Italiana di Oncologia Medica and Societa Italiana di Oncologia Ginecologica. ‡National Health and Medical Research Council. §American College of Occupational and Environmental Medicine.

2: Aggregated survey responses to clinical practice guidelines

Proposition	Number of surveys	Weighted mean positive response	Weighted 95% confidence interval	Weighted median positive response	Interquartile range

Guidelines are a helpful source of advice	24	75%	66%–83%	80%	69%–88%
Guidelines are good educational tools	14	71%	63%–79%	72%	60%–80%
Guidelines are intended to improve quality of care	16	70%	60%–80%	75%	59%–84%
Guidelines are intended to cut health care costs	13	53%	39%–66%	46%	39%–53%
Guidelines will increase litigation or disciplinary action	15	41%	32%–49%	40%	26%–52%
Guidelines reduce physician autonomy and are oversimplified or "cookbook" medicine	12	34%	22%–47%	32%	21%–42%
Guidelines are impractical and too rigid to apply to individual patients	19	30%	23%–36%	26%	20%–35%

Received 22 January 2002, accepted 29 August 2002

View this article on Wiley Online Library

Cynthia M Farquhar¹
Emma W Kofa²
Jean R Slutsky³

Centre for Practice Technology Assessment, Agency for Healthcare Research and Quality, Rockville, MD, USA.

Correspondence: c.farquhar@auckland.ac.nz

Acknowledgements:

Dr C M Farquhar was supported by the Commonwealth Fund of New York. Ms Kofa was sponsored as a Summer Intern by the Department of Health and Human Services, Washington DC.

Competing interests:

None identified.

1. Field M, Lohr KN. Attributes of good practice guidelines. In: Field M, Lohr KN, editors. Clinical practice guidelines: directions for a new program. Washington, DC: National Academy Press, 1990: 53-77.
2. Carnett WG. Clinical practice guidelines: a tool to improve care. Qual Manag Health Care 1999; 8: 13-21.
3. Handley MR, Stuart ME, Kirz HL. An evidence-based approach to evaluating and improving clinical practice: implementing practice guidelines. HMO Pract 1994; 8: 75-83.
4. Shekelle PG, Woolf SH, Eccles M, Grimshaw J. Clinical guidelines: developing guidelines. BMJ 1999; 318: 593-596.
5. Woolf S. Practice guidelines: a new reality in medicine II. Methods of developing guidelines. Arch Intern Med 1992; 152: 946-952.
6. Eisenberg J, Power E. Transforming insurance coverage into quality health care: voltage drops from potential to delivered quality. JAMA 2000; 284: 2100-2107.
7. Woolf SH, Grol R, Hutchinson A, et al. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ 1999; 318: 527-530.
8. Grol R, Dalhuijsen J, Thomas S, et al. Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. BMJ 1998; 317: 858-861.
9. Appraisal of Guidelines Research and Evaluation in Europe. The AGREE Collaboration. <http://www.agreecollaboration.org/> Accessed 24 September 2002.
10. McLaughlin TJ, Soumerai SB, Wilson DJ, et al. Adherence to national guideline for drug treatment of suspected acute myocardial infarction: evidence for undertreatment in women and the elderly. Arch Intern Med 1996; 156: 799-805.
11. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet 1993; 342: 1317-1322.
12. Thomas L, Cullum N, McColl E, et al. Guidelines in professions allied to medicine [Cochrane Review]. In: The Cochrane Library, Issue 2, 2002. Oxford: Update Software, 2002.
13. Soumerai SB, McLaughlin TJ, Spiegelman D, et al. Adverse outcomes of underuse of beta-blockers in elderly survivors of acute myocardial infarction. JAMA 1997; 277: 115-121.
14. Armstrong D, Fry J, Armstrong P. General practitioners' views of clinical guidelines for the management of asthma. Int J Qual Health Care 1994; 6: 199-202.
15. Audet AM, Greenfield S, Field M. Medical practice guidelines: current activities and future directions. Ann Intern Med 1990; 113: 709-714.
16. Deutsch SC, Denton M, Borenstein J. Clinical practice guidelines: a tool to help provide quality care. Geriatrics 1998; 53: 57, 61-64, 70, 73-74.
17. Delmothe T. Wanted: guidelines that doctors will follow. BMJ 1998; 307: 218.
18. Tunis SR, Hayward RS, Wilson MC, et al. Internists' attitudes about clinical practice guidelines [see comments]. Ann Intern Med 1994; 120: 956-963.
19. Grol R. National standard setting for quality of care in general practice: attitudes of general practitioners and response to a set of standards. Br J Gen Pract 1990; 40: 361-364.
20. American Cancer Society. 1989 Survey of physicians' attitudes and practices in early cancer detection. Cancer 1990; 40: 77-101.
21. Nowells D. Features of clinical practice guidelines that influence physician perceptions of credibility. Conducted 1992. MPH thesis 1997. Washington, DC: University of Washington.
22. Madhok R, Thomson RG, Mordue A, et al. An audit of distribution and use of guidelines for management of head injury. Qual Health Care 1993; 2: 27-30.
23. Shye D, Brown JB. Primary care HMO clinicians' opinions about clinical practice guidelines. HMO Pract 1995; 9: 111-115.
24. Mansfield CD. Attitudes and behaviours towards clinical guidelines: the clinicians' perspective. Qual Health Care 1995; 4: 250-255.
25. Hormbrey P, Todd BS, Mansfield CD, et al. A survey of teaching and the use of clinical guidelines in accident and emergency departments. J Accid Emerg Med 1996; 13: 129-133.
26. Grilli R, Penna A, Zola P, et al. Physicians' view of practice guidelines. A survey of Italian physicians. Soc Sci Med 1996; 43: 1283-1287.
27. Ferrier BM, Woodward CA, Cohen M, et al. Clinical practice guidelines. New-to-practice family physicians' attitudes. Can Fam Physician 1996; 42: 463-468.
28. Pathman DE, Konrad TR, Freed GL, et al. The awareness-to-adherence model of the steps to clinical guideline compliance. The case of pediatric vaccine recommendations. Med Care 1996; 34: 873-889.
29. Hayward RS, Guyatt GH, Moore KA, et al. Canadian physicians' attitudes about and preferences regarding clinical practice guidelines [see comments]. CMAJ 1997; 156: 1715-1723.
30. Gupta L, Ward J, Hayward RS. Clinical practice guidelines in general practice: a national survey of recall, attitudes and impact. Med J Aust 1997; 166: 69-72.
31. Salem-Schatz SR, Gottlieb LK, Karp MA, et al. Attitudes about clinical practice guidelines in a mixed model HMO: the influence of physician and organizational characteristics. HMO Pract 1997; 11: 111-117.
32. Alston RP. Guidelines and cardiac anaesthesia. Anaesthesia 1997; 52: 328-331.
33. Dodek P. Attitudes to guidelines for arterial blood gas measurements. Presentation to staff of Division of Critical Care Medicine, St Paul's Hospital and University of British Columbia, BC, Canada. 1997.
34. Olesen F, Lauritzen T. Do general practitioners want guidelines? Attitudes toward a county-based and a national college-based approach. Scand J Prim Health Care 1997; 15: 141-145.
35. Woolf M, Bower DJ, Marbella AM, Casanova JE. US Family Physicians' experiences with practice guidelines. Fam Med 1998; 30: 117-121.
36. Carrick SE, Bonevski B, Redman S, et al. Surgeons' opinions about the NHMRC clinical practice guidelines for the management of early breast cancer. Med J Aust 1998; 169: 300-305.
37. Browman GP, Newman TE, Mohide EA, et al. Progress of clinical oncology guidelines development using the practice guidelines development cycle: the role of practitioner feedback. J Clin Oncol 1998; 16: 1226-1231.
38. Halm EA, Atlas SJ, Borowsky LH, et al. Understanding physician adherence with a pneumonia practice guideline: effects of patient, system, and physician factors. Arch Intern Med 2000; 160: 98-104.
39. Girgis S, Ward JE, Thomson CJH. General practitioners' perception of medicolegal risk. Med J Aust 1999: 171: 362-366. http://www.mja.com.au/public/issues/171_7_041099/girgis/girgis.html
40. Grilli R, Trisolini R, Labianca R, et al. Evolution of physicians' attitudes towards practice guidelines. J Health Serv Res Policy 1999; 4: 215-219.
41. Cotton P, Sullivan F. Perceptions of guidelines in primary and secondary care: implications for implementation. J Integrated Care 1999; 3: 79-83.
42. Shah S, Tonkin A, Ward J, Harris P. Implementation of nationally developed guidelines in cardiology: a survey of NSW cardiologists and cardiothoracic surgeons. Aust N Z J Med 1999; 29: 678-683.
43. Costantini O, Papp KK, Como J, et al. Attitudes of faculty, housestaff, and medical students toward clinical practice guidelines [published erratum appears in Acad Med 1999; 74: 1322]. Acad Med 1999; 74: 1138-1143.
44. Harris JS, Mueller KL, Low P, et al. Beliefs about and use of occupational medicine practice guidelines by case managers and insurance adjusters. J Occup Environ Med 2000; 42: 370-376.
45. Dye TD, Alderdice F, Roberge E, et al. Attitudes toward clinical guidelines among obstetricians in Northern Ireland. BJOG 2000; 107: 101-107.
46. Graham ID, Evans WK, Logan D, et al. Canadian oncologists and clinical practice guidelines: a national survey of attitudes and reported use. Oncology 2000; 59: 283-290.
47. Vinker S, Nakar S, Rosenberg E, et al. Attitudes of Israeli family physicians towards clinical guidelines. Arch Fam Med 2000; 9: 835-840.
48. Edwards P, Roberts I, Clarke M, et al. Increasing response rates to postal questionnaires: systematic review. BMJ 2002; 324: 1183-1192.
49. Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States? Milbank Q 1998; 76: 509, 517-563.
50. Poses RM. One size does not fit all: questions to answer before intervening to change physician behavior. Jt Comm J Qual Improv 1999; 25: 486-495.
51. Merritt TA, Gold M, Holland J. A critical evaluation of clinical practice guidelines in neonatal medicine: does their use improve quality and lower costs? Eval Clin Pract 1999; 5: 169-177.
52. Rappolt SG. Clinical guidelines and the fate of medical autonomy in Ontario. Soc Sci Med 1997; 44: 977-987.
53. Hyams AL, Brandenbrug JA, Lipsitz SR. Practice guidelines and malpractice litigation: a two way street. Ann Intern Med 1995; 122: 450-455.
54. Hurwitz B. Legal and political considerations of clinical practice guidelines. BMJ 1999; 318: 661-664.
55. Brushwood DB. Clinical practice guidelines and the standard of care. Am J Health Syst Pharm 2000; 57: 159-161.
56. National Health Service Executive. Clinical guidelines. Leeds: NHSE, 1996: 10.
57. Miller FH. The legal ramifications of the NCCN practice guidelines. Oncology 1996; 10: 35-39.
58. Vollman K, Sprung P, Posa S. Strategies for reducing material costs through implementation of clinical guidelines. J Soc Health Syst 1998; 5: 69-73.
59. Pitimana-aree S, Forrest D, Brown G. Implementation of a clinical practice guideline for stress ulcer prophylaxis increases appropriateness and decreases cost of care. Intensive Care Med 1998; 24: 217-223.
60. Greco PJ, Eisenberg JM. Changing physicians' practice. N Engl J Med 1993; 329: 1271-1273.
61. Farquhar CM, Stryer D, Slutsky J. Translating research into practice: the future ahead. Int J Qual Health Care 2002. 14: 233-249.
62. National Health and Medical Research Council. Guidelines for the development and implementation of clinical practice guidelines. Canberra: NHMRC, 2001.

Online responses are no longer available. Please refer to our instructions for authors page for more information.