Quality of drug interaction alerts in prescribing and dispensing software

Six prescribing systems — Best Practice (Best Practice Software, Bundaberg, Qld), Genie (Genie Solutions, Brisbane, Qld), Medical Director 2 (Health Communication Network, Sydney, NSW), MedTech32 (Medtech Global, Melbourne, Vic), Plexus (IBA Health, Sydney, NSW) and Profile (Intrahealth Systems, Sydney, NSW) — and three dispensing systems — Amfac Dispense (Corum Health Services, Sydney, NSW), Pharmasol LOTS (Corum Health Services, Sydney, NSW) and WiniFRED (PCA NU Systems, Melbourne, Vic) — were selected. It was estimated that 80%–90% of GPs3,13 and 65% of community pharmacists (Mike Farrell, Councillor, Pharmacy Guild of Australia, personal communication) in Australia used one of these systems.

Five electronic reference sources were selected for comparison — three Australian publications commonly used by doctors and pharmacists, and two publications from the United Kingdom. The Australian Medicines Handbook14 contains brief information on drug interactions, which is intended to be used in conjunction with the full drug monographs in the handbook. MIMS DrugAlert Interactions15 is a specialist knowledgebase on drug interactions that is integrated into some software systems. Product information15 is the official source of drug information in Australia, and is approved by the Therapeutic Goods Administration. Stockley’s Drug Interactions,16 published in the UK, is a comprehensive source widely used by drug information pharmacists, with a companion clinical decision support tool, Stockley’s Interaction Alerts.17

Twenty “major” (defined here as clinically significant) and 20 “minor” (clinically unimportant) drug interactions were assessed in each software system (Box 1). The selection criteria for major interactions were: both drugs used in primary care + pair is potentially used in combination + at least one drug in pair is commonly used or is a new drug or the pair must produce a potentially serious adverse outcome + interaction is classified as “moderate” or “severe” in Stockley’s Interaction Alerts17 and “moderate” or “major” in Drug Interaction Facts.18 Selection criteria for minor interactions were: both drugs are used in primary care + pair is potentially used in combination + at least one drug in pair is commonly used or is a new drug + interaction is classified as “minor” in Drug Interaction Facts18 OR one drug is a topical product and topical use is unlikely to result in a clinically significant interaction.

The major interactions represent a range encountered in practice, not necessarily those that are most common or most severe. Potential drug pairs (47 major and 51 minor interactions) were identified on the basis of our clinical experience, reports to the Adverse Drug Reactions Advisory Committee, similar published studies, and whether the drugs are commonly prescribed.19 A modified Delphi process with the expert panel members was used to remove drug pairs to produce the 40 interactions for testing.

Testing was conducted between September and November 2006 with the latest version of each software system. If the system allowed customisation of the severity level of drug interactions to be detected, the minimum setting was selected (to detect the maximum number of interactions).

A new “dummy” patient was entered into each system. Each of the drug pairs was prescribed or dispensed for this patient, and deleted before the next pair was entered. The contents of any drug interaction alerts were copied to a spreadsheet. Any further information that could be obtained by clicking on a link was not assessed, so as to compare like with like, and because it was not known how often users click on links.

Each reference source was searched for information about each drug pair, and any information found was copied to a spreadsheet. If an individual drug was not found, the drug class was checked. As for software systems, if the reference source allowed customisation of the severity level of drug interactions to be shown, the minimum setting was selected. The interaction information from the product information of the first drug in each pair was assessed.

Five of the six prescribing systems had high sensitivity, detecting ≥ 90% of the major drug interactions (Box 2). The systems were more variable and generally performed less well with minor interactions, with specificity ranging from 25% to 85%. For example, in the system with a specificity of 25%, alerts came up for 75% (15/20) of the minor interactions, and the information provided did not indicate that the interaction was not important in clinical practice.

Only one of the three dispensing systems detected ≥ 90% of the major interactions (Box 2). In general, their specificity was better than the prescribing systems (range, 65%–85%).

Overall, the reference sources had higher sensitivity and specificity than the software systems. They all detected ≥ 90% of the major interactions, and specificity ranged from 65% to 90%.

Only two of the nine software systems provided adequate information about clinical effects for more than half of the major drug interactions (Box 3). Little or no information about the timeframe of interactions was provided, even in a comprehensive reference source such as Stockley’s Drug Interactions, which had this information for 11/20 interactions. Only one software system described the pharmacological mechanism for most interactions (15/20). Overall, the software systems did not provide useful management advice, most providing advice considered “somewhat useful” or none at all. Four software systems provided links to additional information: two to the product information, and two to more detailed information from the drug interactions knowledgebase used. This information was not assessed.

In some cases, the language used in alerts lacked clarity and succinctness, and was occasionally incomprehensible. Some systems used unhelpful phrases such as “use with caution”, spelled drug names incorrectly, or named drugs that are not available in Australia.

In general, the reference sources performed better than the software systems in terms of the comprehensiveness of information provided (Box 3).