Intervention research uses scientific methods to test the effectiveness of health practices and other strategies that affect health at individual and population levels.1 It is particularly useful for evaluating the effectiveness of policy and program delivery, and has the potential to play an important role in achieving health gains for Aboriginal and Torres Strait Islander peoples.
The National Health and Medical Research Council (NHMRC) allocates 5% of its annual research budget to Indigenous health, and strategically directs research through the guiding principles of the Road Map and Road Map II for Aboriginal and Torres Strait Islander research.2,3 One of the six major action areas for 2009–2015 identified in Road Map II is evaluation and intervention research. The Lowitja Institute, Australia’s first National Institute for Aboriginal and Torres Strait Islander Health Research, was launched in March 2010 and has similarly stated an intention to drive a research agenda to “enable a greater focus on implementation research . . . developing the necessary knowhow to embed and scale up interventions with demonstrated efficacy”.4
There is considerable momentum therefore in national policy to direct research in Aboriginal and Torres Strait Islander health towards intervention research that can find real-world solutions to persistent health issues. Yet most Aboriginal and Torres Strait Islander health research to date has been to describe health problems. Several reviews have shown that less than 15% of research reports published between 1987 and 2003 relate to intervention research, with very few meeting basic design criteria for methodological rigor.5-8 Addressing some of the barriers perceived by researchers in undertaking intervention research may assist in achieving national policy goals of increasing high-quality intervention research.
Here, we identify strategies perceived by leading researchers in the field as effective in increasing the amount of high-quality intervention research undertaken in Aboriginal and Torres Strait Islander health. Although researchers potentially have biased views about intervention research, they are also active practitioners in research partnerships with communities and have direct experience regarding the support required to overcome some of the barriers.
Ethics approval was obtained from the University of Newcastle Human Research Ethics Committee.
An advisory panel of leading Aboriginal and Torres Strait Islander (n = 4) and non-Indigenous (n = 3) researchers in Aboriginal health was convened based on our research networks. A Delphi process9,10 was used via email to obtain an agreed list of possible strategies for increasing the amount of methodologically sound intervention research undertaken in Aboriginal health, and a pilot survey tool was tested among 15 of our colleagues.
A MEDLINE search via Ovid — of the string [Aborigin*] OR [Oceanic Ancestry Group] OR [Torres Strait Islander] OR [Indigenous] — was used to identify first authors who had published in Indigenous health in Australia between 1 January 2005 and 1 August 2009. Those who had published more than once were posted a personalised letter on the 21 August 2009, inviting them to complete the survey, and were sent an email 1 week later with a link to a web-based survey instrument. A reminder email was sent 2 weeks later to those who had not responded. The web-based survey instrument used three steps to obtain researchers’ opinions.
Step 1: selection of strategies to increase intervention research. A list of 17 strategies was provided to participants, who were asked to select those which, in their view, would be most effective for increasing relevant, high-quality intervention research in Indigenous health over the next 5 years that would lead to improved health outcomes. Items were categorised under six domains, representing who would be responsible for implementing the strategies (Box).
Step 2: “willingness to pay” for strategies to increase intervention research. Respondents were required to allocate a total of 100 points across the strategies they had selected to represent relative importance. A modified willingness-to-pay technique was used to compel respondents to weight strategies according to perceived value in a way that indicated not only direction of preference, but also the intensity of their preference, to elicit virtual policy decisions.11,12 Respondents also identified additional strategies that they believed were important for increasing intervention research undertaken in Aboriginal health, and rated how well the survey captured their views about how to increase high-quality intervention research using a 5-point Likert scale.
Step 3: demographic information. Respondents’ demographic information regarding professional experience was captured, including what proportion of their work was in Aboriginal health, the broad areas of health research they worked in, and the number of years they had worked in Aboriginal health.
We identified 491 first authors who had published in Indigenous health in Australia between 1 January 2005 and 1 August 2009. Of these authors, 157 had published more than once and were invited to participate in the study. They were based in clinical, public health and other academic institutions. Of the 107 researchers who opened the email about the survey, 75 clicked on the survey link, giving a response rate of (70%); 74 surveys were complete and one survey was incomplete. Overall, respondents had considerable experience in the field, with 26 having worked in Aboriginal health for 11–20 years, and 58 for more than 5 years. Twenty researchers worked for at least 50% of their time in public health — based on NHMRC-defined broad research areas — and 29 said they worked in Aboriginal health more than half of the time.
A median of eight out of 17 strategies to increase effective, high-quality intervention research in Aboriginal health were selected. Each strategy was selected by at least 16 respondents, with a mean of 47 (SD, 19) respondents per strategy; the most selected strategy was chosen by 71 respondents.
Five strategies had a median weighted score of 8% or more when respondents indicated willingness to pay by weighting the relative importance of the strategies they had selected. This is high considering that there were 17 items to select from. The remaining 12 strategies had a median weighted score of 5% or less. The weighting and selection of strategies is presented in the Box. Forty-five respondents provided additional strategies and comments that built on existing items in the survey, including a need for intervention research to focus more on the social determinants of health at the community level and be more grounded in self-determination. The survey instrument was seen as positive by 47 respondents, and 11 responded negatively — largely due to the difficult task of allocating weightings across strategies.
We showed that researchers who are active in Aboriginal health research perceive that improvements in funding, priority setting and research systems are required to increase the amount of high-quality intervention research being conducted in this field.
The strategies developed by our advisory panel were generally perceived by this group of researchers to be important for increasing intervention research. The strength of this study was to compel the respondents to weight selected strategies based on their relative importance, to mimic policy decision making and give a more precise prioritisation than ranking. The strategies that researchers viewed as important for increasing intervention research fell to the responsibility of a range of key stakeholders, indicating that research review panels, ethics committees, peak Aboriginal bodies, health care organisations and research funding bodies all have a role in promoting intervention research in Aboriginal health.
Based on the median weightings, two of the most highly weighted strategies were for research funding bodies to give funding priority to intervention research proposals that specifically target Aboriginal populations and for peak bodies representing Aboriginal communities to clearly specify intervention research priorities in a national Aboriginal health research agenda. The other highly weighted strategies were for research funding bodies to fund research to develop reliable measures of health for Aboriginal people, for health care organisations to participate more in intervention research targeting Aboriginal populations, and for research review panels to accept research designs other than the randomised controlled trial.
Although the NHMRC allocates at least 5% of its total research budget to Indigenous health research, of which a third is for intervention research,12 there are no published intervention research studies in Australian Indigenous health that meet the Cochrane Effective Practice and Organisation of Care Group criteria of methodological rigour.7,8 To successfully undertake intervention research, a considerable amount of time and financial investment is needed, with strong partnerships between researchers, Aboriginal and Torres Strait Islander health organisations and communities keen to undertake the research. A more collaborative approach of multidisciplinary intervention research teams working with communities on identified priorities may encourage more such research. The newly established Lowitja Institute is an example of an organisation with a strong collaborative approach to research, governed by an Aboriginal and Torres Strait Islander board.
Establishing a national intervention research agenda that is led by Aboriginal and Torres Strait Islander peoples could help direct efforts towards relevant intervention research by clearly specifying health priority areas, issues on the ground that need investment, and types of intervention research that are needed. The research agendas of the Road Map, Road Map II and the Lowitja Institute contribute significantly towards this by identifying intervention research as a priority. More specific direction of funding to these priorities through mechanisms that encourage multidisciplinary research teams and community partnerships may assist in increasing the evidence base of effective health strategies.13
Alongside intervention research, measurement research is needed to ensure that Indigenous health statistics are accurate and reflect cultural definitions of health and wellbeing from the perspective of Aboriginal and Torres Strait Islander peoples. Between 2000 and 2008, only 8.7% of funded NHMRC project grants in Indigenous health were classified as measurement studies.14 Research funding bodies should consider specific funding of measurement research studies in identified priority areas where it is likely to contribute most value.
Health care organisations are crucial partners in testing the practical implementation of health interventions at the patient level. In particular, support is needed for the 145 Aboriginal community-controlled health services in Australia which in the 2007–08 financial year provided 1.85 million episodes of primary health care to Aboriginal and Torres Strait Islander people, but which may have had restricted research capacity.15,16
Recommendations could include funding specific research positions within organisations such as the National Aboriginal Community Controlled Health Organisation and its state affiliates, running accredited intervention research training programs for staff of Aboriginal community-controlled health services, and creating formal processes around research engagement. This would increase the agency of health care organisations and health practitioners to be partners and drivers of intervention research. Mainstream services will also need to improve the identification of patients’ Aboriginality to identify potential research study populations.
There was strong interest from our participants in encouraging research review panels to consider alternative research designs to the randomised controlled trial. The stepped wedge (or interrupted time-series) design allows staggered introduction of interventions across sites such as a health service or community. In this design, a repeated pattern of change in measured outcomes in each site following the intervention is used as an indication of the intervention’s effect. These studies are more cost- and time-effective than randomised controlled trials and therefore more attractive for integration into policy roll-out.17-19 Having a single standardised ethics process competent in Aboriginal health ethics review, with shared acknowledgement of approval between committees, may also be useful. In addition, academic research institutions may need to consider performance measures other than publications for estimating researcher productivity, that account for the longer time involved in doing intervention research.
The results of our study are limited as they only reflect the views of published researchers in Aboriginal and Torres Strait Islander health — one of the key groups currently undertaking intervention research. Similar research that elicits views from the Aboriginal community-controlled health and research sectors and Aboriginal and Torres Strait Islander peoples would expand our findings.
Without an evidence-based approach to Aboriginal health there is justifiable concern that ineffective strategies will be implemented, resulting in little improvement in health outcomes.7 There is an urgent need for evidence of effective multilevel strategies capable of addressing health at the community level within a framework of self-determination. A model relevant to the Australian context may be the Population Health Intervention Research Initiative for Canada, which uses a multidisciplinary network to address the social determinants of health through intervention research. This model could be considered for Aboriginal and Torres Strait Islander health research in Australia.1
Selection and weighting of strategies for increasing intervention research in Aboriginal health by leading researchers
Received 28 December 2009, accepted 7 April 2010
- Jessica M Stewart1
- Rob W Sanson-Fisher2
- Sandra J Eades3
- Nicole M Mealing1
- 1 Centre for Aboriginal Health, New South Wales Department of Health, Sydney, NSW.
- 2 Faculty of Medicine and Health Sciences, University of Newcastle, Newcastle, NSW.
- 3 Indigenous Maternal and Child Health Group, Baker IDI Heart and Diabetes Institute, Melbourne, VIC.
We thank Dr Christine Paul for providing critical feedback and input. This research was supported by an Indigenous Training scholarship from the NHMRC. Nicole Mealing was employed by the New South Wales Department of Health on the NSW Biostatistical Officer Training Program at the time this work was conducted. Jessica Stewart is the recipient of a PhD Scholarship from the NHMRC.
None identified.
- 1. Sullivan L. Introduction to the Population Health Intervention Research Initiative for Canada. Can J Public Health 2009; 100 (1 Suppl): I5-I7.
- 2. National Health and Medical Research Council Aboriginal and Torres Strait Islander Research Agenda Working Group. The NHMRC Road Map: a strategic framework for improving Aboriginal and Torres Strait Islander health through research. Canberra: NHMRC, 2002.
- 3. National Health and Medical Research Council. NHMRC Road Map 2009–2015: a strategic framework for improving the health of Aboriginal and Torres Strait Islander people through research. Final consultation draft March 2009. http://www.docstoc.com/docs/22981548/NHMRC-Action-Plan-for (accessed Mar 2010).
- 4. The Lowitja Institute: Australia’s National Institute for Aboriginal and Torres Strait Islander Health Research. http://www.lowitja.org.au/files/crcah_docs/Lowitja-brochure_feb2010.pdf (accessed Mar 2010, link updated May 2010).
- 5. National Health and Medical Research Council Public Health Research Advisory Committee. Report of the review of public health research funding in Australia. Canberra: NHMRC, 2009.
- 6. Sanson-Fisher RW, Campbell EM, Perkins JJ, et al. Indigenous health research: a critical review of outputs over time. Med J Aust 2006; 184: 502-505. <MJA full text>
- 7. Paul CL, Sanson-Fisher RW, Stewart JM, Anderson AE. Being sorry is not enough. The sorry state of the evidence base for improving Indigenous health. Am J Prev Med 2010; 38: 566-568.
- 8. Cochrane Effective Practice and Organisation of Care Group. Data collection checklist. http://www.epoc.cochrane.org/Files/Website/Reviewer%20Resources/Data%20Collection% 20Checklist%20-%20EPOC%20-%202007-Feb-27.doc (accessed Oct 2009, link no longer available).
- 9. Verhagen AP, de Vet HC, de Bie RA, et al. The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus. J Clin Epidemiol 1998; 51: 1235-1241.
- 10. Robson N, Rew D. Collective wisdom and decision making in surgical oncology. Eur J Surg Oncol 2010; 36: 230-236.
- 11. Donaldson C. Willingness to pay for publicly-provided goods. A possible measure of benefit? J Health Econ 1990; 9: 103-118.
- 12. Dixon S, Shackley P. The use of willingness to pay to assess public preferences towards the fortification of foodstuffs with folic acid. Health Expect 2003; 6: 140-148.
- 13. Street J, Baum F, Anderson I. Developing a collaborative research system for Aboriginal health. Aust N Z J Public Health 2007; 31: 372-378.
- 14. Stewart J, Sanson-Fisher RW, Stewart M. Indigenous health research: are current research investments best placed to improve Indigenous health status? A review of NHMRC grants 1996–2008. Unpublished report.
- 15. Dwyer J, O’Donnell K, Lavoie J, et al. The overburden report: contracting for Indigenous health services. Darwin: Cooperative Research Centre for Aboriginal Health, 2009.
- 16. Department of Health and Ageing. Service activity reporting: 2003–04 key results. A national profile of Australian Government funded Aboriginal and Torres Strait Islander primary health care services. Canberra: Commonwealth of Australia, 2006.
- 17. Hawkins NG, Sanson-Fisher RW, Shakeshaft A, et al. The multiple baseline design for evaluating population-based research. Am J Prev Med 2007; 33: 162-168.
- 18. Hawe P, Potvin L. What is population health intervention research? Can J Public Health 2009; 100 (1 Suppl): I8-14.
- 19. Biglan A, Ary D, Wagenaar AC. The value of interrupted time-series experiments for community intervention research. Prev Sci 2000; 1: 31-49.
Abstract
Objective: To identify policy strategies that are perceived by researchers active in Aboriginal and Torres Strait Islander health as effective in increasing the amount of high-quality intervention research undertaken in this field.
Design and setting: A cross-sectional study using a web-based survey was emailed to researchers based in clinical, public health and other academic institutions.
Participants: Researchers who had published more than once in Aboriginal health between 1 January 2005 and 1 August 2009, based on a MEDLINE search.
Main outcome measures: Participants selected and weighted 17 strategies that were, in their opinion, important for increasing the amount of high-quality intervention research being conducted in Aboriginal health.
Results: We invited 157 researchers to complete the survey, and received 74 completed surveys. The most highly weighted strategies were: for research funding bodies to give funding priority to intervention research proposals that target Aboriginal populations (median weighted score,15%); for peak bodies representing Aboriginal communities to clearly specify intervention research priorities in a national Aboriginal health research agenda (median weighted score, 10%); for research funding bodies to fund research to develop reliable measures of health for Aboriginal people (median weighted score, 9.5%); for health care organisations to participate more in intervention research targeting Aboriginal populations (median, 8.5%); and for research review panels to accept intervention research designs other than the randomised controlled trial (median weighted score, 8%).
Conclusions: Researchers who are active in Aboriginal health research perceive that improvements in funding mechanisms, priority setting and research systems are required to increase the amount of high-quality intervention research being conducted in this field. A national intervention research agenda that encourages multidisciplinary research teams and community partnerships may offer a solution.