Drug therapy often does not provide a benefit, for several reasons: many patients are not prescribed the appropriate medicine or dosage; doctors may not properly monitor for efficacy or adverse events; patients may misunderstand even simple instructions;1 and patients taking long term medication often fail to adhere to the prescribed regimen.2
Medication error plays a role in morbidity and death.3,4 In one study, more than half the iatrogenic adverse events at a teaching hospital were related to medication.5 A meta-analysis of drug-related hospital admissions found 5.1% (95% CI, 4.4%–5.8%) of all hospital admissions were related to adverse drug reactions.6
Computerised decision support systems (DSSs) can help reduce these problems. Reminder systems have improved both preventive practices7,8 and compliance with clinical guidelines.9 Feedback from peer comparison results in a small effect on rates of clinical procedures (eg, prescribing rates, laboratory tests).10
We developed a categorisation of DSSs based on reviews of randomised controlled trials (RCTs) of their clinical effectiveness.11,12 Reminders and feedback differ in terms of their timing, information content and intended scope of effect. Reminders are delivered at the time of or before decision-making and are specific to a single patient; they prompt healthcare providers or patients to act (eg, order tests, pick up a repeat prescription). Feedback aggregates information from multiple patients (and sometimes providers) with the intention of altering future decisions.
We aimed to systematically review the benefits of computerised systems that support medication management through reminders or feedback to healthcare providers, or reminders to patients.
When an article seemed relevant based on the title and abstract, the full article was reviewed by both of us. We included any system that used computers to assist in identifying patients and generating reminders or feedback. We also included studies that tried to influence medication use but measured non-clinical outcomes (eg, rate of generic prescribing or costs). Such studies provide evidence of the capacity of DSSs to alter the behaviour of healthcare providers. We excluded systems that calculate drug doses, for which there is a Cochrane review.13
Studies were critically appraised using an instrument developed specifically for evaluating the quality of RCTs of reminder or feedback DSSs. Each RCT was assessed on randomisation method (3 points), parity of baseline data (2), objectivity of primary outcome measure (4), loss to follow-up (4), clarity of inclusion criteria (1), unit of allocation (1), ease of implementation of reminder system (1), and statistical analysis (1), to give a score out of 17. Although there is little empirical evidence to support our choices,14 the scale contains the elements present in many similar instruments.15,16
We independently assessed quality and eligibility, and resolved disagreements by consensus. Where possible, data were extracted using an intention-to-treat principle. We report the rate of compliance with (potential) reminders for the control and intervention groups, or, where this is not possible, we indicate how the values were derived.
The studies and their outcome measures were too heterogeneous (eg, system, outcome measures, settings, participants) for meta-analysis. We consider them in five categories (numbers of comparisons in parentheses): reminders to providers in outpatient settings (12);17-28 feedback to providers in outpatient settings (7);20,29-34 combined reminders and feedback to providers in outpatient settings (1);20 reminders to providers in inpatient settings (5);35-39 and reminders to patients (4).40-42
From 76 potentially relevant trials, we agreed on including 26. The quality scores of these 26 RCTs ranged from 9 to 17 (mean, 13.6; κ = 0.54, indicating fair agreement between raters' scores). The methodological quality was high, with 25 (96%) either describing a valid method of randomisation or stating that random allocation was used. Eighteen studies (69%) presented adequate baseline data or made appropriate adjustments. Twenty-four studies (92%) were judged to have objective outcome measures or appropriately blinded subjective outcome measures. Follow-up was adequate (greater than 95%) in 18 studies (69%). The inclusion criteria were clear in 23 of the studies (88%). The unit of allocation was judged to be appropriate for the type of intervention in 24 (92%). The reminder system was considered to be easy to implement elsewhere in 18 (69%). Statistical analysis was judged appropriate in 17 studies (65%). The flaw in the remainder was that allocation was by healthcare provider or group (eg, general practice, resident team), but the analysis was by patient, which will underestimate the standard error,, although the effect size is unbiased.
Box 1 to Box 5 present the extracted data. Two studies20,41 provided more than one comparison. A L'Abbé plot43 (Box 6) compares effect sizes for RCTs containing sufficient data. This shows significant heterogeneity of studies.
Twenty-two trials evaluated the effects of a medication management DSS on either provider (19) or patient adherence to medication (3).40-42 Three studies assessed cost as their main outcome measure.30,31,35 No studies were able to demonstrate an effect on patient outcomes.
DSSs generally improved medication management in this setting, with relative rates from 1.0 to 42 (Box 1). Poor user interface24 and requiring healthcare providers to enter redundant data22,25 were cited as reasons for poor system utilisation.
Physician feedback systems in outpatient settings (Box 2) generally had smaller effects on clinician behaviour than reminder systems. Relative rates were from 1.0 to 2.5. One group29 influenced rates of median weighted generic prescribing in a family medicine clinic by 35% through monthly feedback of prescription patterns. There were mixed results from feedback aimed at containing prescribing costs. One group decreased the average charge per prescription,30 but another group was unable to reduce prescribing costs.31 Feedback was ineffective in reducing polypharmacy,32 in increasing the use of aspirin in coronary artery disease,33 and in increasing the use of antidepressants in depression.34
Only one study directly compared feedback and reminders (Box 3).20 Feedback and reminders (10.2%) compared with reminders alone (10.4%) did not increase compliance with preventive medication recommendations. This is consistent with results from the previous two sections showing that reminders are more effective than feedback.
There were five studies in this category (Box 4).35-39 In one, medical staff using computerised workstations to order investigations and medications were offered reminders aimed at decreasing costs; this resulted in a 15% reduction in drug costs per admission.35 Extending upon their work in outpatients, Overhage et al38 more than doubled the ordering of additional tests or treatment to monitor or ameliorate effects of the primary treatments or tests. In the final study in this category, provision of electronic guidelines decreased vancomycin prescribing.39
The three RCTs of patient-directed reminders in outpatient settings used computer-generated reminders (timetable, mailed and phone) to improve medication compliance (Box 5). Two studies40,41 conducted entirely in outpatient settings used compliance with refills (ie, repeat prescriptions) as their outcome measurement. Overall, the effect of these interventions was disappointing. One study group showed a 7% increase (not statistically significant) in refill compliance.41 The third study commenced while the patients were medical inpatients in a British district hospital. It demonstrated a 25% increase in the mean compliance scores (measured by pill counts) for patients in the timetable group, but the follow-up period was only 10 days.42
Computerised DSSs providing reminders and feedback to healthcare providers and patients can make modest improvements in medication management. They have successfully changed the class of medication prescribed, increased generic prescribing, improved activities related to medication management (eg, diagnostic testing), and enhanced patient adherence to medication regimens. It appears that reminders are more effective than feedback. However, other factors, such as the study population, the nature of "usual care", or the outcome measures (eg, compliance measures), may explain these differences. Mugford et al44 found information was most effective if presented close to the time of decision making. Our results support this, as, compared with feedback, reminders are presented closer to the time of decision-making.
The current evidence should encourage wider use of DSS for medication management, based on careful consideration of local factors. One academic medical centre produced eight studies with some of the larger effect sizes.17-20,35-38 Whether commercial systems in non-academic settings can produce the same magnitude of effect is an open question. US policy makers are calling for mandatory use of computerised physician order entry as a means to increasing use of DSSs for medication management.45
Four studies17-20 of reminders in outpatient settings were conducted by the same group using a sophisticated electronic health record (EHR) that has operated since 1973.17 Very few EHRs have such richness of patient data and completeness of history. Clinician-authored rules detected critical changes in clinical variables (eg, serum potassium for patients on diuretics) that might need correction and would trigger a recommendation from the DSS.17-20 A DSS coupled to an EHR removes reliance upon duplicate data entry by healthcare providers,22,25 but there are many barriers to the implementation of EHRs.46
Some authors20,26 commented on how little improvement was achieved, despite the healthcare providers agreeing with the importance of the actions requested by the reminders. Further research should focus on what features of DSSs — such as multiple rather than single options, "help" and explanation functions, or speed — might enhance the effects seen. For example, by requiring a response to computer-generated reminders Litzelman et al47 improved compliance with preventive care protocols by internal medicine residents.
Using computers to improve medication use is worthwhile, but care is needed in choosing the most appropriate means of delivering messages. Reminders and feedback can improve various behaviours related to medication management. However, their implementation requires consideration of factors that are likely to bring success.
1: Provider reminders in outpatient settings
First author (year; country) |
Setting; participants |
Nature of reminders |
Control complied/ no. of potential reminders (rate) |
Intervention complied/no. of actual reminders (rate) |
Relative rate |
||||||
McDonald17 |
Diabetes clinic (primary care); 257 patients |
Follow and assess medication use to assure investigations were undertaken at proper intervals and that clinicians reacted to results |
54/470 (12%) |
175/500 (35%) |
3.0 |
||||||
McDonald18 |
General medicine clinic; 26 physicians, 5 nurse practitioners |
Reminders regarding "usage and follow-up of medications" |
236/1158 (20%) |
956/2533 (38%) |
1.9 |
||||||
McDonald19 |
General medicine clinic; 126 physicians, 4 nurse clinicians |
Monitoring of serum K+ or K+ supplements for diuretics; digitalis trial if evidence of CCF on chest x-ray or echocardiogram; monitoring of liver enzyme levels for hepatotoxic drugs; antacid prophylaxis for aspirin, NSAIDs or corticosteroids |
(54%) |
(67%) |
1.3 |
||||||
We were unable to extract exact values. We averaged the percentage responses of the four medication-related actions, assuming even distribution of values. |
|||||||||||
Tierney20
|
General medicine clinic; 135 internal medicine house staff |
Antacids, antidepressants, aspirin, β-blockers, calcium supplements, digitalis, long-acting nitrates, metronidazole. Specific reminders at patient visit or monthly feedback reports |
(8%) |
(10%) |
1.3 |
||||||
2×2 factorial design of feedback and reminders. House staff were their own controls. |
|||||||||||
Rossi21
|
General medicine clinic; 59 physicians, 12 nurse practitioners |
Reminder to switch from calcium-channel blockers to other antihypertensive agents |
1/373 (0.3%) |
39/346 (11%) |
42.0 |
||||||
Hetlevik22 |
29 Health centres; 53 GPs, 2239 patients |
Computerised clinical guidelines for hypertension |
No usable data for medications, and no effect on medication management. |
1.0 |
|||||||
Demakis23 |
12 Ambulatory care clinics; 275 resident physicians |
Standards-of-care reminders for atrial fibrillation, myocardial infarction and gastrointestinal bleeding (+ other preventive care reminders). Computerised and paper reminders |
621/1001 (62%) |
657/1059 (62%) |
1.0 |
||||||
Montgomery24 |
27 General practices; 74 GPs; 11 practice nurses |
Computer-generated cardiovascular risk assessment |
No usable data, and no difference in overall prescribing rates between groups. |
1.0 |
|||||||
McCowan25 |
17 General practices; 477 patients |
Computerised asthma management clinical guidelines |
No usable data for medications, and no difference in maintenance prescribing. |
1.0 |
|||||||
Christakis26 |
Paediatric clinic; 36 physicians, 2 nurse practitioners |
Point-of-care evidence-based messages on otitis media antibiotic prescribing |
(10%) |
(44%) |
4.2 |
||||||
Used a primary study outcome of change in frequency of antibiotics prescribed for <10 days (before v after). |
|||||||||||
Frances27 |
3 General medicine clinics; 66 physicians |
Combination of computer-generated and written reminders for aspirin, β-blockers and cholesterol lowering agents in coronary artery disease |
(30%) |
(29%) |
1.0 |
||||||
These are averages for the 3 medication groups. |
|||||||||||
Rollman28 (2001; US) |
Primary medical care practice; 227 patients; 16 internists |
Computer-generated patient-specific advisory messages on management of depression |
No usable data and no difference in overall prescribing rates between groups. |
1.0 |
|||||||
2: Provider feedback
First author (year; country) |
Setting; participants |
Nature of feedback |
Control complied/ no. of potential reminders (rate) |
Intervention complied/no. of actual reminders (rate) |
Relative rate |
||||||
Gehlbach29 |
Family medicine residency; 32 physicians |
Monthly feedback reports regarding brand versus generic prescribing |
(23%) |
(58%) |
2.5 |
||||||
The article contained no raw data to derive rates. Reported "median weighted rates of generic prescribing". |
|||||||||||
Hershey30 (1986; US) |
General medicine clinic; 48 residents |
Feedback regarding average charges per prescription |
US$8.79 |
US$8.22 |
0.9 |
||||||
The article contained no raw data to derive rates. We used their "resident's average charge per prescription". |
|||||||||||
Tierney20 |
General medicine clinic; 135 internal medicine house staff |
Antacids, antidepressants, aspirin, β-blockers, calcium supplements, digitalis, long-acting nitrates, metronidazole. Specific reminders at patient visit or monthly feedback reports |
(8%) |
(9%) |
1.2 |
||||||
2×2 factorial design of feedback and reminders. House staff were their own controls. |
|||||||||||
Steele31 |
General medicine clinic; 20 physicians (residents and fellows) |
Peer-comparison feedback of prescribing costs (+ another arm with educational intervention) |
No usable data, and no difference in prescribing costs between peer-comparison feedback and controls. |
1.0 |
|||||||
Meyer32 |
General medicine clinic; 292 patients |
Two types of feedback: letter identifying patients on 10 or more long term medications; similar letter including personalised review of medications |
No usable data, and no difference reported between either intervention group and control in reducing polypharmacy. |
1.0 |
|||||||
McCartney33 |
28 General practices |
Baseline prescribing of prophylactic aspirin in coronary artery disease. Facilitated support through medical audit advisory groups |
610/1220 (50%) |
1004/1725 (58%) |
1.2 |
||||||
Simon34 |
5 Primary medical care clinics; 613 patients |
Two detailed reports about depression management and treatment recommendations |
(18%) |
(21%) |
1.2 |
||||||
Data for adequate antidepressant use at least 90 days after initial prescription. |
|||||||||||
3: Combined reminders and feedback
First author (year; country) |
Setting; participants |
Nature of reminders and feedback |
Control complied/ no. of potential reminders (rate) |
Intervention complied/no. of actual reminders (rate) |
Relative rate |
||||||
Tierney20 |
General medicine clinic; 135 internal medicine house staff |
Antacids, antidepressants, aspirin, β-blockers, calcium supplements, digitalis, long-acting nitrates, metronidazole. Specific reminders at patient visit or monthly feedback reports |
(10%) |
(10%) |
|
1.0 |
|
||||
2×2 factorial design of feedback and reminders. House staff were their own controls. |
|
||||||||||
4: Inpatient reminders
First author (year; country) |
Setting; participants |
Nature of reminders |
Control complied/ no. of potential reminders (rate) |
Intervention complied/no. of actual reminders (rate) |
Relative rate |
||||||
Tierney35 |
General medicine; 68 resident teams (5219 patients) |
All inpatient orders (including medications) on computer, linked to a comprehensive electronic medical record |
US$1181 |
US$1001 |
0.85 |
||||||
The article contained no raw data we could use to derive rates. We used "total drug charges per admission". |
|||||||||||
Overhage36 |
General medicine; 68 physicians (1622 patients) |
Aspirin, oestrogen, calcium, ACE inhibitor, heparin prophylaxis, β-blocker, and 18 other preventive care measures |
71/794 (9%) |
63/811 (8%) |
0.9 |
||||||
Smith37
|
General medicine; 12 medicine ward teams (348 patients) |
Computer-generated list of all active outpatient prescriptions at discharge and the capacity to alter discharge prescriptions. |
(3%) |
(3%) |
1.0 |
||||||
The article contained no raw data to derive rates. We used their "change in active prescriptions from admission to discharge". |
1.0 |
||||||||||
Overhage38 (1997; US) |
General medicine service; 89 physicians (2181 patients) |
Reminders of corollary orders to prevent errors of omission. Computer suggested orders to detect or ameliorate adverse reactions to the trigger orders (76 drugs) |
(22%) |
(46%) |
2.1 |
||||||
We were unable to extract exact values. We used percentages as reported in the article. |
|||||||||||
Shojania39 (1998; US) |
Tertiary care teaching hospital; 396 physicians (1798 patients) |
Electronic guideline for vancomycin use |
17 |
11 |
0.7 |
||||||
The article contained no raw data to derive rates We used their "Total numbers of vancomycin orders per prescriber". |
|||||||||||
5: Patient-directed reminders
First author (year; country) |
Setting; participants |
Nature of reminders |
Control complied/ no. of potential reminders (rate) |
Intervention complied/no. of actual reminders (rate) |
Relative rate |
||||||
Baird40 (1984; US) |
Urban outpatient pharmacy; 324 patients |
Mailed reminders for prescription refills |
32/163 (20%) |
29/161 (18%) |
0.9 |
||||||
Simkins41 (1986; US) |
Primary care and speciality clinics; 207 patients |
Postcard refill reminders for cardiovascular medications due in 2 days |
(58%) |
(65%) |
1.1 |
||||||
The article contained no raw data. We used their "mean compliant events per group". |
|||||||||||
Simkins41 (1986; US) |
Primary care and speciality clinics; 208 patients |
Telephone refill reminders for cardiovascular medications due in 2 days |
(58%) |
(42%) |
0.7 |
||||||
We used the "mean compliant events per group", but with an intention-to-treat analysis, assuming the worst case. The authors' original analysis had excluded 35 of 104 patients who could not be contacted by phone. |
|||||||||||
Raynor42 (1993; UK) |
3 Medical wards, district hospital; 197 patients |
Medication timetable for patients on discharge |
60/96 (62%) |
83/95 (87%) |
1.4 |
||||||
Received 29 May 2002, accepted 13 December 2002
- John W Bennett1
- Paul P Glasziou2
- 1 University Health Service, University of Queensland, Brisbane, QLD.
- 2 School of Population Health, University of Queensland Medical School, Herston, QLD.
We thank the anonymous referees for their most thoughtful and constructive comments.
None identified.
- 1. Hanchak NA, Patel MB, Berlin JA, Strom BL. Patient misunderstanding of dosing instructions. J Gen Intern Med 1996; 11: 325-328.
- 2. Rich M, Gray D, Beckham V, et al. Effect of a multidisciplinary intervention on medication compliance in elderly patients with congestive heart failure. Am J Med 1996; 101: 270-276.
- 3. Leape LL. Error in medicine. JAMA 1994; 272: 1851-1857.
- 4. Kohn LT, Corrigan JM, Donaldson MS. To err is human: building a safer health system. Washington, DC: National Academy Press, 1999.
- 5. Steel K, Gertman PM, Crescenzi C, Anderson J. Iatrogenic illness on a general medical service at a university hospital. N Engl J Med 1981; 304: 638-642.
- 6. Einarson TR. Drug-related hospital admissions. Ann Pharmacother 1993; 27: 832-840.
- 7. Austin SM, Balas EA, Mitchell JA, Ewigman BG. Effect of physician reminders on preventive care: meta-analysis of randomized clinical trials. In: Ozbolt JG, editor. Proceedings for 18th Annual Symposium on Computer Applications in Medical Care; 1994 November 5–9; Washington, DC. Philadelphia: Hanley & Belfus, 1994: 121-124.
- 8. Shea S, DuMouchel W, Bahsmonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. J Am Med Inform Assoc 1996; 3: 399-409.
- 9. Shiffman RN, Liaw Y, Brandt CA, Corb GJ. Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. J Am Med Inform Assoc 1999; 6: 104-114.
- 10. Balas EA, Boren SA, Brown GD, et al. Effect of physician profiling on utilization. Meta-analysis of randomized clinical trials. J Gen Intern Med 1996; 11: 584-590.
- 11. Bennett JW, Glasziou PP. A review of the usefulness of computerised systems in clinical medicine. In: Royal Australian College of General Practitioners 7th Computer Conference; 1993 June 3–5. Melbourne: RACGP, 1993; 127-130.
- 12. Bennett JW, Glasziou PP. A review of the usefulness of advice and therapy systems in clinical medicine. In: Hovenga, EJS, Whymark GK, editors. Proceedings of the Inaugural Health Informatics Conference; Brisbane, Australia; 1993 August 2–3. Melbourne: HISA, 1993; 228-233.
- 13. Walton RT, Harvey EL, Dovey S, Freemantle N. Computerised advice on drug dosage to improve prescribing practice. In: Bero L, Grilli R, Grimshaw J, Oxman A, editors. Cochrane Collaboration on Effective Professional Practice Module of The Cochrane Database of Systematic Reviews [updated March 2001]. Available in The Cochrane Library. The Cochrane Collaboration; Issue 2. Oxford: Update Software, 2001.
- 14. Moher D, Jadad AR, Tugwell P. Assessing the quality of randomized controlled trials. Current issues and future directions. Int J Technol Assess Health Care 1996; 12: 195-208.
- 15. Guyatt GH, Sackett DL, Cook DJ, for the Evidence-Based Medicine Working Group. Users' guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? JAMA 1993; 270: 2598-2601.
- 16. Sackett DL, Richardson WS, Rosenberg W, Haynes RB. Evidence-based medicine: how to practice and teach EBM. 1st ed. New York: Churchill Livingstone, 1997.
- 17. McDonald CJ. Use of computer to detect and respond to clinical events: its effect on clinician behavior. Ann Intern Med 1976; 84: 162-167.
- 18. McDonald CJ, Wilson GA, McCabe GP. Physician response to computer reminders. JAMA 1980; 244: 1579-1581.
- 19. McDonald CJ, Hui SL, Smith DM, et al. Reminders to physicians from an introspective computer medical record. A two-year randomized trial. Ann Intern Med 1984; 100: 130-138.
- 20. Tierney WM, Hui SL, McDonald CJ. Delayed feedback of physician performance versus immediate reminders to perform preventive care. Effects on physician compliance. Med Care 1986; 24: 659-666.
- 21. Rossi RA, Every NR. A computerized intervention to decrease the use of calcium channel blockers in hypertension. J Gen Intern Med 1997; 12: 672-678.
- 22. Hetlevik I, Holmen J, Kruger O. Implementing clinical guidelines in the treatment of hypertension in general practice. Evaluation of patient outcome related to implementation of a computer-based clinical decision support system. Scand J Prim Health Care 1999; 17: 35-40.
- 23. Demakis JG, Beauchamp C, Cull WL, et al. Improving residents' compliance with standards of ambulatory care: results from the VA Cooperative Study on Computerized Reminders. JAMA 2000; 284: 1411-1416.
- 24. Montgomery AA, Fahey T, Peters TJ, et al. Evaluation of computer based clinical decision support system and risk chart of management of hypertension in primary care: randomised controlled trial. BMJ 2000; 320: 686-690.
- 25. McCowan C, Neville RG, Ricketts IW, et al. Lessons from a randomized controlled trial designed to evaluate computer decision support software to improve the management of asthma. Med Inform Internet Med 2001; 26: 191-201.
- 26. Christakis DA, Zimmerman FJ, Wright JA, et al. A randomized controlled trial of point-of-care evidence to improve the antibiotic prescribing practices for otitis media in children. Pediatrics 2001; 107: E15-E18.
- 27. Frances CD, Alperin P, Adler JS, Grady D. Does a fixed physician reminder system improve the care of patients with coronary artery disease? A randomized controlled trial. West J Med 2001; 175: 165-166.
- 28. Rollman BL, Hanusa BH, Gilbert T, et al. The electronic medical record. Arch Intern Med 2001; 161: 189-197.
- 29. Gehlbach SH, Wilkinson WE, Hammond WE, et al. Improving drug prescribing in a primary care practice. Med Care 1984; 22: 193-201.
- 30. Hershey CO, Porter DK, Breslau D, Cohen DI. Influence of simple computerized feedback on prescription charges in an ambulatory clinic. A randomized clinical trial. Med Care 1986; 24: 472-481.
- 31. Steele MA, Bess DT, Franse VL, Graber SE. Cost effectiveness of two interventions for reducing outpatient prescribing costs. Drug Intell Clin Pharm 1989; 23: 497-500.
- 32. Meyer TJ, Van Kooten D, Marsh S, Prochazka AV. Reduction of polypharmacy by feedback to clinicians. J Gen Intern Med 1991; 6: 133-135.
- 33. McCartney P, Macdowall W, Thorogood M. A randomised controlled trial of feedback to general practitioners of their prophylactic aspirin prescribing. BMJ 1997; 315: 35-36.
- 34. Simon GE, VonKorff M, Rutter C, Wagner E. Randomised trial of monitoring, feedback, and management of care by telephone to improve treatment of depression in primary care. BMJ 2000; 320: 550-554.
- 35. Tierney WM, Miller ME, Overhage JM, McDonald CJ. Physician inpatient order writing on microcomputer workstations. Effects on resource utilization. JAMA 1993; 269: 379-383.
- 36. Overhage JM, Tierney WM, McDonald CJ. Computer reminders to implement preventive care guidelines for hospitalized patients. Arch Intern Med 1996; 156: 1551-1556.
- 37. Smith DM, Cox MR, Brizendine EJ, et al. An intervention on discharge polypharmacy. J Am Geriatr Soc 1996; 44: 416-419.
- 38. Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A randomized trial of "corollary orders" to prevent errors of omission. J Am Med Inform Assoc 1997; 4: 364-375.
- 39. Shojania KG, Yokoe D, Platt R, et al. Reducing vancomycin use utilizing a computer guideline: results of a randomized controlled trial. J Am Med Inform Assoc 1998; 5: 554-562.
- 40. Baird TK, Broekemeier RL, Anderson MW. Effectiveness of a computer-supported refill reminder system. Am J Hosp Pharm 1984; 41: 2395-2397.
- 41. Simkins CV, Wenzioff NJ. Evaluation of a computerized reminder system in the enhancement of patient medication refill compliance. Drug Intell Clin Pharm 1986; 20: 799-802.
- 42. Raynor DK, Booth TG, Blenkinsopp A. Effects of computer generated reminder charts on patients' compliance with drug regimens. BMJ 1993; 306: 1158-1161.
- 43. L'Abbé KA, Detsky AS, O'Rourke K. Meta-analysis in clinical research. Ann Intern Med 1987; 107: 224-233.
- 44. Mugford M, Banfield P, O'Hanlon M. Effects of feedback of information on clinical practice: a review. BMJ 1991; 303: 398-402.
- 45. Overhage JM, Middleton B, Miller RA, et al. Does national regulatory mandate of provider order entry portend greater benefit than risk for health care delivery? J Am Med Inform Assoc 2002; 9: 199-208.
- 46. McDonald CJ. The barriers to electronic medical record systems and how to overcome them. J Am Med Inform Assoc 1997; 4: 213-221.
- 47. Litzelman DK, Dittus RS, Miller ME, Tierney WM. Requiring physicians to respond to computerized reminders improves their compliance with preventive care protocols. J Gen Intern Med 1993; 8: 311-317.
Abstract
Objective: To systematically review randomised controlled trials (RCTs) of computer-generated medication reminders or feedback directed to healthcare providers or patients.
Data sources: Extensive computerised and manual literature searches identified 76 English-language reports of RCTs reported before 1 January 2002. Searches were conducted between June 1998 and April 2002.
Study selection: 26 papers making 29 comparisons (two papers reported on multiple interventions) of computer-supported medication management to a control group.
Data extraction: The quality of the RCTs was systematically assessed and scored independently by two reviewers. Rates of compliance with (potential) reminders for the control and intervention groups were extracted.
Data synthesis: Heterogeneity of studies prevented a meta-analysis. Where possible, rates were calculated using the intention-to-treat principle. The comparisons were grouped into five areas. Reminders to providers in outpatient settings: six of 12 comparisons demonstrated positive effects (relative rates [RRs: intervention rates/control rates], 1.0 to 42.0). Provider feedback in outpatient settings: five of seven comparisons showed improved clinician behaviour (RRs, 1.0 to 2.5). Combined reminders and feedback in outpatient settings: the single comparison found no improvement. Reminders to providers in inpatient settings: three of five comparisons showed improvements (RRs, 1.0 to 2.1). Patient-directed reminders: two of four comparisons showed improvements in patient compliance.
Conclusion: Reminders are more effective than feedback in modifying physician behaviour related to medication management. Patient-directed reminders can improve medication adherence.