MJA
MJA

Computerised reminders and feedback in medication management: a systematic review of randomised controlled trials

John W Bennett and Paul P Glasziou
Med J Aust 2003; 178 (5): 217-222. || doi: 10.5694/j.1326-5377.2003.tb05166.x
Published online: 3 March 2003

Abstract

Objective: To systematically review randomised controlled trials (RCTs) of computer-generated medication reminders or feedback directed to healthcare providers or patients.

Data sources: Extensive computerised and manual literature searches identified 76 English-language reports of RCTs reported before 1 January 2002. Searches were conducted between June 1998 and April 2002.

Study selection: 26 papers making 29 comparisons (two papers reported on multiple interventions) of computer-supported medication management to a control group.

Data extraction: The quality of the RCTs was systematically assessed and scored independently by two reviewers. Rates of compliance with (potential) reminders for the control and intervention groups were extracted.

Data synthesis: Heterogeneity of studies prevented a meta-analysis. Where possible, rates were calculated using the intention-to-treat principle. The comparisons were grouped into five areas. Reminders to providers in outpatient settings: six of 12 comparisons demonstrated positive effects (relative rates [RRs: intervention rates/control rates], 1.0 to 42.0). Provider feedback in outpatient settings: five of seven comparisons showed improved clinician behaviour (RRs, 1.0 to 2.5). Combined reminders and feedback in outpatient settings: the single comparison found no improvement. Reminders to providers in inpatient settings: three of five comparisons showed improvements (RRs, 1.0 to 2.1). Patient-directed reminders: two of four comparisons showed improvements in patient compliance.

Conclusion: Reminders are more effective than feedback in modifying physician behaviour related to medication management. Patient-directed reminders can improve medication adherence.

Drug therapy often does not provide a benefit, for several reasons: many patients are not prescribed the appropriate medicine or dosage; doctors may not properly monitor for efficacy or adverse events; patients may misunderstand even simple instructions;1 and patients taking long term medication often fail to adhere to the prescribed regimen.2

Medication error plays a role in morbidity and death.3,4 In one study, more than half the iatrogenic adverse events at a teaching hospital were related to medication.5 A meta-analysis of drug-related hospital admissions found 5.1% (95% CI, 4.4%–5.8%) of all hospital admissions were related to adverse drug reactions.6

Computerised decision support systems (DSSs) can help reduce these problems. Reminder systems have improved both preventive practices7,8 and compliance with clinical guidelines.9 Feedback from peer comparison results in a small effect on rates of clinical procedures (eg, prescribing rates, laboratory tests).10

We developed a categorisation of DSSs based on reviews of randomised controlled trials (RCTs) of their clinical effectiveness.11,12 Reminders and feedback differ in terms of their timing, information content and intended scope of effect. Reminders are delivered at the time of or before decision-making and are specific to a single patient; they prompt healthcare providers or patients to act (eg, order tests, pick up a repeat prescription). Feedback aggregates information from multiple patients (and sometimes providers) with the intention of altering future decisions.

We aimed to systematically review the benefits of computerised systems that support medication management through reminders or feedback to healthcare providers, or reminders to patients.

Methods
Results

From 76 potentially relevant trials, we agreed on including 26. The quality scores of these 26 RCTs ranged from 9 to 17 (mean, 13.6; κ = 0.54, indicating fair agreement between raters' scores). The methodological quality was high, with 25 (96%) either describing a valid method of randomisation or stating that random allocation was used. Eighteen studies (69%) presented adequate baseline data or made appropriate adjustments. Twenty-four studies (92%) were judged to have objective outcome measures or appropriately blinded subjective outcome measures. Follow-up was adequate (greater than 95%) in 18 studies (69%). The inclusion criteria were clear in 23 of the studies (88%). The unit of allocation was judged to be appropriate for the type of intervention in 24 (92%). The reminder system was considered to be easy to implement elsewhere in 18 (69%). Statistical analysis was judged appropriate in 17 studies (65%). The flaw in the remainder was that allocation was by healthcare provider or group (eg, general practice, resident team), but the analysis was by patient, which will underestimate the standard error,, although the effect size is unbiased.

Box 1 to Box 5 present the extracted data. Two studies20,41 provided more than one comparison. A L'Abbé plot43 (Box 6) compares effect sizes for RCTs containing sufficient data. This shows significant heterogeneity of studies.

Twenty-two trials evaluated the effects of a medication management DSS on either provider (19) or patient adherence to medication (3).40-42 Three studies assessed cost as their main outcome measure.30,31,35 No studies were able to demonstrate an effect on patient outcomes.

Reminders to providers in outpatient settings

DSSs generally improved medication management in this setting, with relative rates from 1.0 to 42 (Box 1). Poor user interface24 and requiring healthcare providers to enter redundant data22,25 were cited as reasons for poor system utilisation.

Provider feedback systems in outpatient settings

Physician feedback systems in outpatient settings (Box 2) generally had smaller effects on clinician behaviour than reminder systems. Relative rates were from 1.0 to 2.5. One group29 influenced rates of median weighted generic prescribing in a family medicine clinic by 35% through monthly feedback of prescription patterns. There were mixed results from feedback aimed at containing prescribing costs. One group decreased the average charge per prescription,30 but another group was unable to reduce prescribing costs.31 Feedback was ineffective in reducing polypharmacy,32 in increasing the use of aspirin in coronary artery disease,33 and in increasing the use of antidepressants in depression.34

Combined reminders and feedback in outpatient settings

Only one study directly compared feedback and reminders (Box 3).20 Feedback and reminders (10.2%) compared with reminders alone (10.4%) did not increase compliance with preventive medication recommendations. This is consistent with results from the previous two sections showing that reminders are more effective than feedback.

Reminders to providers in inpatient settings

There were five studies in this category (Box 4).35-39 In one, medical staff using computerised workstations to order investigations and medications were offered reminders aimed at decreasing costs; this resulted in a 15% reduction in drug costs per admission.35 Extending upon their work in outpatients, Overhage et al38 more than doubled the ordering of additional tests or treatment to monitor or ameliorate effects of the primary treatments or tests. In the final study in this category, provision of electronic guidelines decreased vancomycin prescribing.39

Patient-directed reminders

The three RCTs of patient-directed reminders in outpatient settings used computer-generated reminders (timetable, mailed and phone) to improve medication compliance (Box 5). Two studies40,41 conducted entirely in outpatient settings used compliance with refills (ie, repeat prescriptions) as their outcome measurement. Overall, the effect of these interventions was disappointing. One study group showed a 7% increase (not statistically significant) in refill compliance.41 The third study commenced while the patients were medical inpatients in a British district hospital. It demonstrated a 25% increase in the mean compliance scores (measured by pill counts) for patients in the timetable group, but the follow-up period was only 10 days.42

Discussion

Computerised DSSs providing reminders and feedback to healthcare providers and patients can make modest improvements in medication management. They have successfully changed the class of medication prescribed, increased generic prescribing, improved activities related to medication management (eg, diagnostic testing), and enhanced patient adherence to medication regimens. It appears that reminders are more effective than feedback. However, other factors, such as the study population, the nature of "usual care", or the outcome measures (eg, compliance measures), may explain these differences. Mugford et al44 found information was most effective if presented close to the time of decision making. Our results support this, as, compared with feedback, reminders are presented closer to the time of decision-making.

The current evidence should encourage wider use of DSS for medication management, based on careful consideration of local factors. One academic medical centre produced eight studies with some of the larger effect sizes.17-20,35-38 Whether commercial systems in non-academic settings can produce the same magnitude of effect is an open question. US policy makers are calling for mandatory use of computerised physician order entry as a means to increasing use of DSSs for medication management.45

Four studies17-20 of reminders in outpatient settings were conducted by the same group using a sophisticated electronic health record (EHR) that has operated since 1973.17 Very few EHRs have such richness of patient data and completeness of history. Clinician-authored rules detected critical changes in clinical variables (eg, serum potassium for patients on diuretics) that might need correction and would trigger a recommendation from the DSS.17-20 A DSS coupled to an EHR removes reliance upon duplicate data entry by healthcare providers,22,25 but there are many barriers to the implementation of EHRs.46

Some authors20,26 commented on how little improvement was achieved, despite the healthcare providers agreeing with the importance of the actions requested by the reminders. Further research should focus on what features of DSSs — such as multiple rather than single options, "help" and explanation functions, or speed — might enhance the effects seen. For example, by requiring a response to computer-generated reminders Litzelman et al47 improved compliance with preventive care protocols by internal medicine residents.

Conclusion

Using computers to improve medication use is worthwhile, but care is needed in choosing the most appropriate means of delivering messages. Reminders and feedback can improve various behaviours related to medication management. However, their implementation requires consideration of factors that are likely to bring success.

1: Provider reminders in outpatient settings

First author (year; country)

Setting; participants

Nature of reminders

Control complied/ no. of potential reminders (rate)

Intervention complied/no. of actual reminders (rate)

Relative rate


McDonald17
(1976; US)

Diabetes clinic (primary care); 257 patients

Follow and assess medication use to assure investigations were undertaken at proper intervals and that clinicians reacted to results

54/470 (12%)

175/500 (35%)

3.0

McDonald18
(1980; US)

General medicine clinic; 26 physicians, 5 nurse practitioners

Reminders regarding "usage and follow-up of medications"

236/1158 (20%)

956/2533 (38%)

1.9

McDonald19
(1984; US)

General medicine clinic; 126 physicians, 4 nurse clinicians

Monitoring of serum K+ or K+ supplements for diuretics; digitalis trial if evidence of CCF on chest x-ray or echocardiogram; monitoring of liver enzyme levels for hepatotoxic drugs; antacid prophylaxis for aspirin, NSAIDs or corticosteroids

(54%)

(67%)

1.3

We were unable to extract exact values. We averaged the percentage responses of the four medication-related actions, assuming even distribution of values.

Tierney20
(1986; US)

General medicine clinic; 135 internal medicine house staff

Antacids, antidepressants, aspirin, β-blockers, calcium supplements, digitalis, long-acting nitrates, metronidazole. Specific reminders at patient visit or monthly feedback reports

(8%)

(10%)

1.3

2×2 factorial design of feedback and reminders. House staff were their own controls.

Rossi21
(1997; US)

General medicine clinic; 59 physicians, 12 nurse practitioners

Reminder to switch from calcium-channel blockers to other antihypertensive agents

1/373 (0.3%)

39/346 (11%)

42.0

Hetlevik22
(1999; Norway)

29 Health centres; 53 GPs, 2239 patients

Computerised clinical guidelines for hypertension

No usable data for medications, and no effect on medication management.

1.0

Demakis23
(2000; US)

12 Ambulatory care clinics; 275 resident physicians

Standards-of-care reminders for atrial fibrillation, myocardial infarction and gastrointestinal bleeding (+ other preventive care reminders). Computerised and paper reminders

621/1001 (62%)

657/1059 (62%)

1.0

Montgomery24
(2000; UK)

27 General practices; 74 GPs; 11 practice nurses

Computer-generated cardiovascular risk assessment

No usable data, and no difference in overall prescribing rates between groups.

1.0

McCowan25
(2001; UK)

17 General practices; 477 patients

Computerised asthma management clinical guidelines

No usable data for medications, and no difference in maintenance prescribing.

1.0

Christakis26
(2001; US)

Paediatric clinic; 36 physicians, 2 nurse practitioners

Point-of-care evidence-based messages on otitis media antibiotic prescribing

(10%)

(44%)

4.2

Used a primary study outcome of change in frequency of antibiotics prescribed for <10 days (before v after).

Frances27
(2001; US)

3 General medicine clinics; 66 physicians

Combination of computer-generated and written reminders for aspirin, β-blockers and cholesterol lowering agents in coronary artery disease

(30%)

(29%)

1.0

These are averages for the 3 medication groups.

Rollman28 (2001; US)

Primary medical care practice; 227 patients; 16 internists

Computer-generated patient-specific advisory messages on management of depression

No usable data and no difference in overall prescribing rates between groups.

1.0

2: Provider feedback

First author (year; country)

Setting; participants

Nature of feedback

Control complied/ no. of potential reminders (rate)

Intervention complied/no. of actual reminders (rate)

Relative rate


Gehlbach29
(1984; US)

Family medicine residency; 32 physicians

Monthly feedback reports regarding brand versus generic prescribing

(23%)

(58%)

2.5

The article contained no raw data to derive rates. Reported "median weighted rates of generic prescribing".

Hershey30 (1986; US)

General medicine clinic; 48 residents

Feedback regarding average charges per prescription

US$8.79

US$8.22

0.9

The article contained no raw data to derive rates. We used their "resident's average charge per prescription".

Tierney20
(1986; US)

General medicine clinic; 135 internal medicine house staff

Antacids, antidepressants, aspirin, β-blockers, calcium supplements, digitalis, long-acting nitrates, metronidazole. Specific reminders at patient visit or monthly feedback reports

(8%)

(9%)

1.2

2×2 factorial design of feedback and reminders. House staff were their own controls.

Steele31
(1989; US)

General medicine clinic; 20 physicians (residents and fellows)

Peer-comparison feedback of prescribing costs (+ another arm with educational intervention)

No usable data, and no difference in prescribing costs between peer-comparison feedback and controls.

1.0

Meyer32
(1991; US)

General medicine clinic; 292 patients

Two types of feedback: letter identifying patients on 10 or more long term medications; similar letter including personalised review of medications

No usable data, and no difference reported between either intervention group and control in reducing polypharmacy.

1.0

McCartney33
(1997; UK)

28 General practices

Baseline prescribing of prophylactic aspirin in coronary artery disease. Facilitated support through medical audit advisory groups

610/1220 (50%)

1004/1725 (58%)

1.2

Simon34
(2000; US)

5 Primary medical care clinics; 613 patients

Two detailed reports about depression management and treatment recommendations

(18%)

(21%)

1.2

Data for adequate antidepressant use at least 90 days after initial prescription.

4: Inpatient reminders

First author (year; country)

Setting; participants

Nature of reminders

Control complied/ no. of potential reminders (rate)

Intervention complied/no. of actual reminders (rate)

Relative rate


Tierney35
(1993; US)

General medicine; 68 resident teams (5219 patients)

All inpatient orders (including medications) on computer, linked to a comprehensive electronic medical record

US$1181

US$1001

0.85

The article contained no raw data we could use to derive rates. We used "total drug charges per admission".

Overhage36
(1996; US)

General medicine; 68 physicians (1622 patients)

Aspirin, oestrogen, calcium, ACE inhibitor, heparin prophylaxis, β-blocker, and 18 other preventive care measures

71/794 (9%)

63/811 (8%)

0.9

Smith37
(1996; US)

General medicine; 12 medicine ward teams (348 patients)

Computer-generated list of all active outpatient prescriptions at discharge and the capacity to alter discharge prescriptions.

(3%)

(3%)

1.0

The article contained no raw data to derive rates. We used their "change in active prescriptions from admission to discharge".

1.0

Overhage38 (1997; US)

General medicine service; 89 physicians (2181 patients)

Reminders of corollary orders to prevent errors of omission. Computer suggested orders to detect or ameliorate adverse reactions to the trigger orders (76 drugs)

(22%)

(46%)

2.1

We were unable to extract exact values. We used percentages as reported in the article.

Shojania39 (1998; US)

Tertiary care teaching hospital; 396 physicians (1798 patients)

Electronic guideline for vancomycin use

17

11

0.7

The article contained no raw data to derive rates We used their "Total numbers of vancomycin orders per prescriber".

Received 29 May 2002, accepted 13 December 2002

Online responses are no longer available. Please refer to our instructions for authors page for more information.