Traumatic brain and spinal cord injuries are debilitating and have lifelong impacts on the injured person. Aside from the physical impact of these injuries, many aspects of the person’s daily function are affected, including the ability to work and take part in social and community activities. Advances in treatment have led to a reduction in mortality, meaning that an increasing majority of those affected are living with the consequences of traumatic brain injury (TBI) or spinal cord injury (SCI) for decades after injury.
There are an estimated 1493 new cases of moderate TBI and 1000 new cases of severe TBI per year in Australia, and an estimated 137 new cases of paraplegia and 136 new cases of quadriplegia.1 Despite the relatively low incidence of these injuries, they impose a large economic burden, with the total lifetime cost of new cases in 2008 estimated to be $8.6 billion for moderate and severe TBI and $2 billion for traumatic SCI.1 In 2005, the Victorian government established the Victorian Neurotrauma Initiative (VNI), a health and medical research fund to support research in these areas. The VNI was established with an initial 5-year time frame and a budget of $63 million to commit to projects before 30 June 2010.
Health and medical research funding agencies are increasingly concerned with measuring outputs and outcomes from the research they support, in order to demonstrate return on investment and to justify continued or increased investment. Evaluation of research outcomes may also provide valuable insight into the most effective funding mechanisms and allow the value delivered by different funding agencies to be compared.
In this context, research agencies typically attempt to determine the extent to which their activities contribute to gains in knowledge, wealth and health.2,3 Knowledge gains are the easiest to measure (eg, via bibliometric analyses) and have received the most attention. Furthermore, knowledge gains become apparent sooner than health benefits and commercial returns, and provide a relatively simple method of comparing outcomes between research funding agencies.4 A number of Australian funding agencies have published analyses of their funding programs.3-5 These have focused on knowledge gain as measured by research publications and other bibliometric outcomes. Economic and commercial payback from research investment in the Australian Government’s Cooperative Research Centres has also been evaluated.6
In this article, I report the results of an evaluation conducted 3 years after the launch of the VNI. The focus of the evaluation was to determine the state of neurotrauma research activity within Victoria before and 3 years after the establishment of the VNI; it included a focus on knowledge gains and research capacity.
Data was collated from two sources. First, funding applications, progress reports and end-of-grant reports submitted by VNI-funded researchers provided much of the background data required for this analysis. This included data such as the number of applications and amount of approved funding, research categories and areas of research focus. Second, a survey of grant recipients was conducted in August and September 2008. This self-report survey required chief investigators from each of the 29 laboratories who had received a first-round or second-round VNI project grant to rate their laboratory status and output at two time points: the establishment of the VNI (1 July 2005) and its third anniversary (1 July 2008).
Outcome measures were selected to correspond with two major aims of the VNI program — increasing the state of Victoria’s neurotrauma research activity and research capacity. Within these two broad categories, numerous individual measurements were made, including that of workforce capacity, research collaboration, number of published reports and presentations of the research, commercial activity and research translation activity.
Two project grant funding rounds were completed in the first 3 years of the VNI’s operations, and were the focus of this analysis. VNI funding is nationally competitive but must include a substantial Victorian component. During these two funding rounds, the VNI received 147 applications requesting a total of $68.4 million in funding. Of these, 34 successful applications (23%) received funding to the value of $18.6 million (27%). Funding decisions for the first and second project grant rounds were reached in May 2006 and June 2007, respectively. As an indication of scale, the National Health and Medical Research Council (NHMRC) awarded 35 traumatic brain injury and spinal cord injury project grants to the value of $13.8 million in 2005 and 2006 (note that this sum includes funds for all grant subtypes). Of these, 11 grants to the value of $3.8 million were awarded to Victorian institutions. The VNI contribution represents a near fivefold increase in the amount of competitive funding for neurotrauma research in the state of Victoria during this period. Since June 2008, much of the remaining funding has been allocated, with about $13 million unallocated as at September 2009.
Of the projects awarded VNI grants, eight were focused on SCI, 23 on TBI and the remaining three had a joint brain and spinal cord injury focus. Thirty-two percent (11/34) of awarded projects focused on clinical management; 26% (9/34) on rehabilitation and disability management; 32% (11/34) on biomedical science; and 9% (3/34) on technological developments. The average value of approved grants was $547 947. Thirty projects were of 36 months’ duration, with one of the remaining two of 12 months’ duration and the other of 24 months’ duration.
Twenty-seven of 29 laboratories (93%) receiving VNI funding responded to the survey. Of these, 17 were undertaking neurotrauma research at the beginning of the survey period (1 July 2005). The 27 laboratories included in the second sample (ie, undertaking neurotrauma research at the end of the period [1 July 2008]) received 32 of the 34 project grants approved by the VNI during the period.
Substantial self-reported changes in measures of research activity and capacity were recorded between 1 July 2005 and 1 July 2008 (Box 1 and Box 2). These included a 110% increase in the number of individual researchers working in Victoria (Box 1); the attraction of 72 new staff to the area of neurotrauma research, including 27 who moved from interstate to begin work in Victoria during the period (Box 1); and a 186% increase in total collaborative activity (Box 2).
Similar changes in research output were observed over the period. Comparison of outputs in the year 2007–2008 with those in the year 2004–2005 showed a 196% increase in the number of refereed journal articles (Box 1); a near doubling of the number of presentations given by research staff (Box 1); a 193% increase in funding attracted from all sources (Box 2); and a 200% increase in the number of clinical policies and guidelines incorporating Victorian research outcomes (Box 2).
The data also indicated substantial differences in the magnitude of neurotrauma research activity when laboratories in Victoria were compared. For example, the number of staff employed in individual research groups at 1 July 2008 ranged from one to 44 (Box 1), while at the same time point the amount of funding from all sources received in the past 12 months ranged from $12 000 to $3.4 million (Box 2).
Prior analysis of the effects of health and medical research funding has indicated that open competitive grant schemes can make a significant contribution to knowledge creation, conventionally measured by scientific publication rate.3,7-9 However, there is as yet no consensus regarding the most appropriate measures of the impact of research funding activities. Within Australia, nationally competitive grant schemes have used outcome measures such as average grant expenditure per scientific publication,3 number of publications per grant4 and measures of workforce capacity.6 These approaches, while facilitating some measure of comparability between grant schemes, do not address other arguably more important outcomes of research funding, including research capacity and translation of research findings into practice. Recently, the NHMRC published outcomes from analyses of end-of-grant reports, including an increased focus on collaborative activity, research training and commercialisation.5 These studies represent an important benchmark against which future performance can be measured.
The data presented here demonstrate substantial increases in neurotrauma research activity and output within the state of Victoria, occurring concurrently with the first 3 years of the VNI’s operations. These changes were evident on measures of research collaboration, workforce capacity, research output, economic and commercial activity and research translation (Box 1 and Box 2).
While it is not possible to imply a causal relationship between research funding provided through the VNI and the observed outcomes, no other significant brain and spinal cord injury research funding initiatives were undertaken in Victoria during the period of this study. Therefore, it is likely that the observed changes are at least partly a consequence of the activities of the VNI.
Australian experience suggests that gains in health and wealth arising from medical research funding require at least 7 years from time of grant receipt to be realised.3,6 The current results suggest that the Victorian neurotrauma research sector is on track to achieve these important outcomes within this expected time frame.
1 Comparison of measures of research capacity in neurotrauma in Victoria at 1 July 2005 and 1 July 2008
* Mean per research unit surveyed. † Staff in this category include students and postdoctoral fellows who moved from interstate to take up positions on Victorian-based research studies. |
|||||||||||||||
2 Comparison of measures of research activity in neurotrauma in Victoria at 1 July 2005 and 1 July 2008
- Alex Collie1
- Institute for Safety Compensation and Recovery Research, Monash University, Melbourne, VIC.
I would like to acknowledge the Transport Accident Commission (TAC) health research team for their assistance with data collation.
I was employed by the TAC during the period the article was written. The TAC provided most of the funding to the VNI, the health and medical research fund that is the subject of the article. I was also Executive Director of the VNI during the period the article was written.
- 1. Access Economics. The economic cost of spinal cord injury and traumatic brain injury in Australia. Canberra: Access Economics, 2009. http://www.accesseconomics.com.au/publicationsreports/getreport.php? report=209&id=267 (accessed May 2010).
- 2. Wells R, Whitworth JA. Assessing outcomes of health and medical research: do we measure what counts or count what we can measure? Aust New Zealand Health Policy 2007; 4: 14-17. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1929109/ (accessed Apr 2010).
- 3. Kingwell BA, Anderson GP, Duckett SJ, et al. Evaluation of NHMRC funded research completed in 1992, 1997 and 2003: gains in knowledge, health and wealth. Med J Aust 2006; 184: 282-286. <MJA full text>
- 4. Clay MA, Donovan C, Butler L, Oldenburg BF. The returns from cardiovascular research: the impact of the National Heart Foundation of Australia’s investment. Med J Aust 2006; 185: 209-212. <MJA full text>
- 5. National Health and Medical Research Council of Australia. NHMRC final report summary: preliminary results from final reports received from NHMRC funded awards that ended between 2003 and 2005. Canberra: NHMRC, 2007.
- 6. Insight Economics. Economic impact study of the CRC programme. Canberra: Insight Economics, 2006. http://www.universitiesaustralia.edu.au/documents/news/media_releases/2006/CRC_Economic_Impact_Study-Oct06.pdf (accessed Apr 2010)
- 7. Bourke P, Butler L. Mapping Australia’s basic research in the medical and health sciences. Med J Aust 1997; 167: 610-613. <MJA full text>
- 8. Butler L. NHMRC-supported research: the impact of journal publication output 1996–2003. Canberra: National Health and Medical Research Council; 2005. http://www.nhmrc.gov.au/_files_nhmrc/file/publications/synopses/nh75.pdf (accessed Apr 2010).
- 9. Anderson WP. Funding Australia’s health and medical research. Med J Aust 1997; 167: 608-609. <MJA full text>
Abstract
Recognising that brain and spinal cord injuries result in significant health and economic burdens for the affected individual and the community, the Victorian government committed $63 million towards neurotrauma research beginning in 2005.
A survey of Victorian neurotrauma research units conducted in 2008 showed substantial increases in workforce capacity, collaborative activity and research output during the first 3 years of the funding program.
Changes in economic and commercial activity, and research translation activity were also observed.
The activity and output of the Victorian brain and spinal cord injury research sector increased substantially during a period coinciding with increased funding.