A national study of the processes and outcomes of paediatric formulary applications in Australia

Medication and its management is central to the quality of care received by most hospitalised children and is dependent on an evidence-based formulary that ensures safe, effective and rational use of medicines. Evaluation of medicines for addition to the hospital formulary is a primary responsibility of drug and therapeutics committees.1 For paediatric hospitals, committee decisions also have a direct impact on access to medicines to treat rare conditions, transition to adult services, outpatient medication supply and hospital drug budgets.

In Australia, medicines for inpatients in public hospitals are primarily funded by the hospital under agreements among the states, territories and the federal government, without direct costs to patients.2 The Pharmaceutical Benefits Scheme (PBS) provides subsidised access to PBS-approved medicines for patients in the community, at hospital discharge and for outpatients of some public and private hospitals. Local drug and therapeutics committees make their own decisions to fund medicines within the hospital's allocated budget.3 Committees are chaired by a medical specialist, with the director of pharmacy acting as committee secretary. Membership includes representation from pharmacy, medical specialties, administration, finance and nursing. While most hospital-based formulary decisions are made independently of other hospitals, four states of Australia currently have state-based formularies4-7 and some committees serve more than one hospital within a campus or Local Hospital Network.

We recently published a review of drug approval processes in Australian paediatric hospitals highlighting considerable variability between institutions, duplication of processes and high approval rates for submissions,8 with similar trends noted internationally;9 however, a formal evaluation of the evidence to support these submissions was not included. Our aims were to describe the submissions for addition to the formulary in all paediatric hospitals in Australia, and evaluate the evidence supporting these submissions.

Methods

Setting

This study was conducted at all eight tertiary paediatric hospitals across Australia for the period 1 July 2010 to 31 December 2011.

Inclusion criteria

We included applications to list new medicines or to make substantial changes to current listings, based on published criteria.10 New presentations or changes to conditions of use of previously approved drugs were excluded as these applications generally did not include sufficient supporting information. Submissions for individual patient use of non-formulary items were analysed previously8 and also excluded.

Data collection and analysis

We undertook a comprehensive review of documents relating to formulary applications, including formulary submissions, supporting literature, meeting minutes and correspondence. One of us (YS) extracted data from committee documents to determine the number of formulary applications at each hospital, the indication and outcome. Applications were stratified by the World Health Organization Anatomic Therapeutic Chemical (ATC) classification system level 1.11

Evaluation of applications and supporting data

Formulary applications were reviewed using a standardised checklist incorporating criteria developed to provide a rigorous, standardised framework for evaluating medicines.10,12-14 Applications were classified as “formal” where they were submitted using a standardised committee template. These templates included domains such as indications for use, regulatory approval status, comparative efficacy, safety and costs.12 “Informal” applications included those written by email to the committee chair or secretary, applications automatically accepted by hospital committees following decisions by statewide medicines advisory services and formulary decisions made internally by committees.

YS reviewed each application, noting the format, documentation of Therapeutic Goods Administration approval status and PBS listing for the indication sought. The supporting data for each application were classified by study type (eg, randomised controlled trial [RCT]),15 with the number and type of studies determined for each application. We also added discrete categories for product information, clinical practice guidelines, narrative reviews and animal data. Applications could be placed in more than one category where the applicant provided more than one type of supporting evidence (eg, two RCTs, a narrative review and the product information would place an application in three categories). Where an application provided data from systematic reviews as well as from individual RCTs, these were counted in both categories, regardless of whether the individual trials were part of the systematic review. Where supporting data for applications were quoted as citations only, we searched MEDLINE to determine the type of study.

Ethics approval

The study was approved by the Human Research Ethics Committees of the Sydney Children's Hospitals Network, Queensland Children's Health Services (Royal Children's Hospital), Women's and Children's Health Network, Adelaide, and the University of Sydney.

Results

All eight paediatric hospitals participated. There was variable infrastructural support for committee processes (Box 1).

Outcomes of formulary decisions

The total number of applications considered during the 18-month period was 120, and ranged from three to 23 per hospital, with most applications being approved (Box 1). Ninety-five applications for new medicines or to request substantial changes to current formulary items were submitted (Appendix). Of these, 80 were approved, five deferred and 10 rejected. Several committees considered identical applications during the period of review and with different outcomes. The most common applications considered were for neurological agents (30/95). Ten applications were rejected because of insufficient evidence, no prescribing protocol provided, cost or need to manufacture an unlicensed product.

Application characteristics

Most applications were formally submitted using a standardised template (77/95), with documentation of regulatory approval (74/95) and PBS listing completed on the application form (65/95) (Box 2). Few drugs were listed on the PBS at the time of application (19/95). Where cost data were included (70/95), this was often brief and incomplete.

Formulary submission templates were generally similar in content, with two hospitals adopting a less formal approach in certain circumstances. For one hospital, many applications were informal because, as the committee secretary commented, prior knowledge of the circumstances surrounding the application and assisting applicants was of greater priority than a formal requirement for documentation. The other hospital with few formal applications automatically approved several applications after they were evaluated by a statewide medicines advisory committee.

Of the total 120 applications considered during the period of review, 87 (73%) were submitted by medical staff, 23 (19%) by pharmacists and 10 (8%) by other health professionals. Forty-five applications (38%) were independently reviewed by a statewide medicines advisory committee or drug committee pharmacist, with processes for review varying considerably between hospitals. For four hospitals, there was no independent review of applications. For one hospital, all applications were routinely evaluated by the drug information pharmacist, before submission to a statewide medicines advisory service. For another hospital, selected medicines not recommended for listing for a statewide formulary were independently reviewed by the drug information pharmacist. A formulary for high-cost drugs operated in one state, with all other applications considered by the hospital committee.

Conflict of interest was declared by 10 applicants. Conflicts included assistance by the manufacturer in the preparation of the submission, applicant membership of advisory boards for the manufacturer, expenses for travel to meetings and drug supply for compassionate use or for use in a clinical trial. In two instances, a conflict of interest was not declared for an application where the supporting data included a manufacturer submission and where an application was submitted for a patient familiarisation program, pending listing on the PBS. For one hospital, a conflict of interest declaration was not included in the formulary submission template.

Evidence

Most applicants provided supporting data for their application (Box 3). Supporting data were generally provided as full-text papers, with some applicants providing a list of citations or abstracts. Applications with little or no supporting data were from hospitals where there had been prior evaluation by a statewide medicines advisory service or internal decisions by the drug and therapeutics committee to change restrictions for the use of a medicine. Few applicants provided a systematic review or meta-analysis as supporting data (14/95). Similarly, where RCTs were provided (37/95), there were five or fewer trials supplied for 33 applications, with only four applications supported by more than five trials. Many applications included case series (36/95) and narrative reviews (29/95). Animal studies were included in four applications (4/95). For applications independently submitted to more than one hospital for the same medicine (LMX4 lidocaine 4% [Perrigo Australia] and bisphosphonates), we found the same references submitted to support all applications. There were four applications for the antiepileptic medication lacosamide submitted independently; however, references to support these applications varied for each hospital, and one application was informal and incomplete.

No application included a formal evaluation of the risk of bias of supporting studies. Safety data were provided for 68/95 applications, mostly in the form of product information, drug monographs or data from RCTs or case series, and 33/95 included a protocol for use for the requested drug.

Discussion

We have found considerable variability, overlap, poor evidence and incomplete documentation supporting formulary decisions across all paediatric institutions in Australia. It is also clear that the lack of resources available to duly appointed committees means insufficient time and technical expertise for the critical evaluation of supporting data presented. Variability may be explained by some differences in processes such as evaluation by statewide advisory services for some hospitals, varying requirements for formal documentation by committees and the local definition of what constitutes a “formulary” application. We found that applications from hospitals that accepted less information had undergone prior evaluation by a statewide process. While our definition of “formulary” included applications to list new medicines or to make substantial changes to current listings, the application of this definition varied between institutions. Variability in quality of applications may be partly explained by the background and prior experience of applicants, with a tendency towards better quality submissions from “academic” clinicians. The variability in the number of applications per hospital may be partly population driven, with hospitals in New South Wales serving a larger population than those in other states.

We found overlap for several applications submitted to different committees, with similar supporting data provided. This suggests that a coordinated formal national process to share evidence and approval decisions would be resource-efficient and facilitate consistency. The National Institute for Health and Care Excellence has assisted in standardising access to medicines across the UK, most recently through the provision of evidence summaries for recently registered medicines and off-label indications.16 In Australia, the collating and sharing of drug and therapeutics committee decisions through state-based therapeutic advisory groups has been a step towards a more unified process;17 however, this may be less advantageous for paediatric hospitals with few or no state-based peers. Currently, with the operation of statewide formulary committees, there is the potential for a shared approach through rigorous evaluation and statewide position statements; however, there is limited formal sharing at a national level. Internationally, the New Zealand Formulary (http://nzformulary.org) has assisted in standardising access to funded medicines, although findings elsewhere are similar to our study.9,18

Few applications were supported by high-level evidence such as systematic reviews, with many applications including case series and narrative reviews. This may represent either failure to submit an existing review, or the absence of an existing relevant review. A review of child-relevant submissions in the Cochrane Database of Systematic Reviews showed that while the number and proportion of child-relevant reviews is increasing, fewer than half of the trials in these reviews included only children.19 Similarly, the number of child-relevant RCTs and systematic reviews in major biomedical and paediatric journals is low compared with those for adults.20

Approval rate for submissions was high and possibly reflects poor outcomes from previous treatments and a lack of therapeutic options. Committees may rely on the experience of the applicant in guiding their decision on therapies for rare conditions where evidence is deficient. They may also consider outcomes for similar submissions for individual patient use, and unpublished evidence, including expert consensus. No application included an evaluation of risk of bias of supporting studies, which may be due to a lack of resources and level of training to provide a thorough evaluation. While some templates requested a level of evidence grading and summary statistical data, this was rarely completed except where requested by a statewide medicines evaluation committee.

Documentation of essential information including cost, and regulatory and PBS listing was incomplete, and this is of concern if uniform evidence-based policy is to be implemented. While evidence-based practice for paediatrics is hindered by small numbers of patients and a lack of clinical trial data, our study shows that few resources are devoted to evaluation compared with the rigorous process for listing drugs for federal subsidy.10 Funders of health services need to consider the importance of resources for hospital-based drug evaluation in ensuring that a valid, unbiased appraisal of evidence is met. It is also uncertain whether pharmacy and medical personnel have the requisite skills to critically appraise the data, particularly as more complex, high-cost drugs are being used. In the absence of adequate data, a more co-ordinated approach to collecting evidence of outcomes to support the knowledge base would also be beneficial.

The strengths of the study were the inclusion of all tertiary paediatric hospitals in Australia and the completeness of the data provided by participating sites. We are not aware of existing studies that have evaluated the supporting data for formulary decisions in paediatric or adult patients, with most studies being surveys of drug committee function and quantitative summaries of committee outcomes.1,9,18 Limitations of the study were the small number of applications per hospital over the 18-month period, making formal comparisons between institutions difficult. We excluded formulary applications from regional hospitals or paediatric wards in general hospitals; however, as most complex conditions are managed in tertiary paediatric hospitals, this is unlikely to have affected the conclusions of the study.

Our findings suggest that standardisation of formulary application processes among Australian paediatric hospitals may improve the quality and consistency of formulary decisions. This could be facilitated through formal agreements for sharing of evidence and approval outcomes. More resources and training for personnel involved in evaluating submissions may improve the quality and efficiency of this process.

2 Characteristics of formulary applications to drug and therapeutics committees at Australian paediatric hospitals between 1 July 2010 and 31 December 2011, by site*

---

PBS = Pharmaceutical Benefits Scheme. * Data collected from October 2010 to April 2012 for Hospital 1.

3 Information supporting formulary applications to drug and therapeutics committees at Australian paediatric hospitals between 1 July 2010 and 31 December 2011, by site