Bringing evidence to bear on policy processes: the challenge of the Australian Primary Health Care Research Institute

The Australian Primary Health Care Research Institute (APHCRI) is part of the Primary Health Care Research Evaluation and Development (PHCRED) strategy.1 The Institute was the most recent element of the PHCRED strategy to be established, beginning research activities following the appointment of the Research Advisory Board (RAB) in November 2003. APHCRI’s mission is to “provide national leadership in improving the quality and effectiveness of primary health care through the conduct of high quality priority-driven research and the support and promotion of best practice”.2 APHCRI has a specific focus on the links between primary health care evidence and policy. Its activities not only fund research programs, but also seek to build capacity within the research community and policy community to facilitate the adoption of evidence into policy.

There are two important interdependent features in the APHCRI model adopted to fulfil this mission. Firstly, it is a “virtual” institute operating a “hub and spoke” model. The Institute comprises staff based at the Australian National University (the “hub”) and, within different streams of research activities, multiple “spokes” undertaking commissioned programs of research. Secondly, the Institute has a specific pool of funds to commission research — it both funds research and undertakes research. The Institute’s RAB sets the research priorities, oversees the independent assessment of applications for funding and determines the successful spokes. Groups compete to be commissioned within a particular stream of research, but, once successful, collaborate as part of the virtual institute.

In this article, we provide an overview of the approach APHCRI has taken to bring research evidence to bear on policy formation. We also reflect on lessons learned through the process of conducting our fourth research stream.

Influencing policy with research evidence is not a simple “linear” proposition. Health policy draws on many information inputs apart from research evidence, including political realities.3 APHCRI has adapted the Canadian Health Services Research Foundation “linkage and exchange” approach4 in order to make its research products more useful to policymakers.

APHCRI links together four groups of participants to exchange knowledge from their different perspectives. These are:

• Policymakers and decisionmakers in both the federal and state/territory spheres;

• Providers of primary health care services and the various organisations with which they are associated;

• Researchers; and

• Users of primary health care services, and the various organisations with which they are associated.

Members of these four groups serve on the RAB. APHCRI’s research priorities are iterated with policy advisers and the RAB to ensure they are relevant to policy. Expert review committees, convened to assess applications within the different streams, include members with expertise across these groups. The assessment criteria for applications within streams reflect the emphasis on policy and provider expertise in addition to more usual academic criteria.

APHCRI organises its research programs in “streams”. Each stream has a particular focus and may have several spokes or individuals working within it (the numbers of the streams denote the chronological order in which they were announced).

The Institute’s Stream 4 program (with $1.8 million funding in total) further sharpened the linkage and exchange focus. It aimed to increase both the capacity of researchers to respond to policy priorities and the capacity of policy advisers to utilise research evidence. A list of policy-relevant topics, approved by the RAB, was identified in consultation with the Australian Government Department of Health and Ageing. A total of 12 spokes were commissioned to address these topics, using a common methodology to address two broad questions:

• What do we know about the topic?; and

• What are the possible options for the Australian context?

The first question focuses the systematic review that is synthesised by the research team. The second question requires the researchers to use the results of the review to develop evidence-based recommendations for ways forward for Australia’s primary health care system. The policy options include consideration of funding arrangements (existing and alternative), delivery arrangements and governance arrangements reflecting system-level perspectives.

An overview of the steps, timelines and major activities that comprised Stream 4 is presented in the Box. Within this supplement, we summarise key findings from each spoke on the topics as follows:

• Chronic disease management (Cranston et al,6 Dennis et al7);

• Integration, coordination and multidisciplinary care (Jackson et al,8 Mitchell et al,9 Powell Davies et al10);

• Innovative models for comprehensive primary health care delivery (McDonald et al,11 Naccarella et al,12 Humphreys et al13);

• Innovative models for the management of mental health in primary health care settings (Griffiths and Christensen14);

• Children and young Australians, health promotion and prevention (McDonald et al,15 Hearn et al16); and

• Workforce (focus on competency-based training) (Glasgow et al17).

The disparate nature of much of the published literature required careful consideration of the appropriate methods for systematically reviewing and synthesising such evidence. Stream 4 drew heavily on the Journal of Health Services Research and Policy supplement, “Synthesizing evidence for management and policy-making”,18 to underpin its approaches, particularly the narrative synthesis approach of Mays and colleagues.19

Additional funding was provided to Stream 4 participants to allow their engagement in structured meetings in Canberra on four separate occasions during the 12 months of the program. These served five main purposes:

1. Facilitating agreement on methodological issues (eg, common approaches to searching for primary health care literature or economic literature, assignment of quality criteria to diverse literature, “stopping” rules to allow a decision to be made that enough material has been obtained);
2. Minimising duplication of effort through sharing of material and, where more than one spoke was working on a topic, agreeing on how the work of one spoke would complement the work of others on that topic;
3. Facilitating interactions with policy advisers to provide provisional results to them and to test emerging options for their policy relevance;
4. Allowing access to international experts to ensure a high standard of review (eg, Nicholas Mays, Professor of Health Policy at the London School of Hygiene and Tropical Medicine, delivered a workshop on systematic review methods); and
5. Ensuring that progress against stated milestones was achieved so that the results were delivered in a timely fashion.

The research teams were asked to present their results (preliminary and final) in different ways through the program. The intent was twofold — addressing the concern of policymakers that research results are often not delivered in a timely fashion, and presenting the final reports in a manner that was easily accessible. The structured research components in Canberra allowed presentation of early results, as did the meetings between the individual spokes and their reference groups and interactions with key stakeholders. Throughout this interaction, the independent nature of the process was maintained by careful attention to the scientific method involved in synthesis and critical internal and external review. The final reports were prepared for web-based presentation using the “1:3:25” approach,5 with one page summarising the key take-home messages, three pages providing an overview, and the longer report containing all the information, including full references and appendices where appropriate.

Most spokes found the systematic review process very demanding. A number of participants had experience with the Cochrane approach to systematic reviews, but did not find this suitable for the kind of literature being surveyed. The volume of potential literature identified in the searches was very large, and making decisions about when to stop searching and how to adjudicate the relevance and weight that should be given to retrieved material was a challenge throughout the process. The result for most spokes was a greater proportion of the 12 months being spent on the review process than had been anticipated at the outset, with a relatively lesser proportion of time iterating potential options with key stakeholders.

Researchers are accustomed to writing for research audiences. Most spokes found the production of the one- and three-page summaries of the options for non-research audiences challenging. Researchers tended to default into research writing mode — for example, qualifying statements in the summary documents to convey the sense of uncertainty around them rather than stating the implications for policy less ambiguously.

While policy advisers played a significant role in priority setting for the research program, participation by policy advisers in the structured sessions in Canberra was more variable. Senior policy advisers usually had unanticipated demands being made on their time and thus were unable to attend. More junior policy advisers were hesitant to offer critical comments from a policy perspective on the material being discussed, and had to balance the commitment of being present for the full day against the other requirements of their roles. Engagement with senior policy advisers was more successful when the separate spokes arranged to meet with individuals outside the structured sessions.

Locating the material on the Internet has made it easily accessible to Australian audiences and, to some extent, international audiences. The number of hits suggested the resources have been useful, and, interestingly, it seems the full reports are most often visited rather than the one- or three-page summaries. Anecdotal accounts suggest that the recent Australian Government intervention in the Northern Territory has resulted in much use being made of the reports by McDonald et al15 and Humphreys et al,13 although this can not be corroborated through analysis of APHCRI website activity. Anticipating future policy challenges in an explicit and timely fashion allows for a repository of relevant research information to be developed.20

Because of the largely positive experiences associated with the conduct of Stream 4, the RAB has continued to support the development of the linkage and exchange approach. Stream 6 is repeating the Stream 4 process, with a single focus on addressing the primary health care workforce shortage. Stream 7 provided opportunities for researchers involved in Stream 4 to compete for new linkage and exchange travelling fellowships, allowing Australian primary health care researchers to visit world-renowned international academic primary health care institutions in relevant comparator countries and consider their Stream 4 work in the context of these international settings. On their return, they will provide written reports of their findings and participate in a briefing to policy advisers in Canberra.

Improving the quality and effectiveness of primary health care requires the adoption of evidence into policy and practice. Has APHCRI’s research been taken up in policy? As Nutley et al report,21 direct or instrumental use of research findings to shape policy is unusual. Research evidence is only one source of information that policymakers draw upon. APHCRI does not expect to demonstrate direct links between its research programs and subsequent policy. However, it does expect to contribute to the policy processes through use of its research to assist with conceptualisation of issues and to mobilise support for key reforms. Conceptual use is illustrated by the provision of succinct summaries of relevant information or provision of new ways of framing issues or gaining further insights into the strengths and weaknesses of different options — all illustrated in the articles of this supplement. APHCRI’s Stream 4 program has contributed to debate and raised public discussion of crucial issues confronting Australia’s health system. An example of this is the contribution APHCRI’s Stream 4 program and related activities have made to mobilising support for discussion of the health system reforms necessary to meet the challenges posed by chronic disease.

APHCRI’s development of the linkage and exchange approach through its Streams 4, 6 and 7 has been positively received by the primary health care research community. The RAB will continue to develop and implement refinements to this approach, with a view to enhancing the uptake of evidence in policy.