Three major health reform initiatives — the National Health and Hospitals Reform Commission (NHHRC),1 National Primary Health Care Strategy (NPHCS) working group2 and Preventative Health Taskforce (PHT)3 — released their final reports during 2009. Their processes were remarkably open. Consistent with the Australian Government’s commitment to evidence-informed policy,4,5 each working group placed discussion papers, commissioned papers, submissions and reports on websites in the public domain.6-9 This gave us a rare opportunity to identify the evidence and publications that are used by such working groups to develop their health reform options. The inclusion of citations in final reports allowed analysis of the forms of evidence that supported their decisions
As part of a broader study of the contribution of primary health care research to the health reform initiatives,10 we identified which journals were most frequently cited in the health reform documents. We extracted and coded all references cited in the publicly available discussion documents (n = 4), submissions (n = 1264), commissioned papers (n = 25), an interim report, and final reports (n = 3) from the websites of the three reform initiatives.6-8 As the variety of referencing styles limited the accuracy of the dataset, we undertook more intensive data cleaning and coding of the journal subset to identify the most commonly cited journals. We used Microsoft Excel to organise and code data, and SPSS Version 17.0 for Windows (SPSS Inc, Chicago, Ill, USA) to obtain frequencies. Ethics approval was not required, as all data were in the public domain.
Discussion documents, commissioned papers, submissions and the interim NHHRC report11 contained 8143 references, with almost half (46%; 3713) in documents associated with the NHHRC, 35% (2882) with the PHT and 19% (1548) with the NPHCS. Journal articles were cited most frequently, followed by reports and other publications by government organisations such as the Australian Institute of Health and Welfare and Australian Bureau of Statistics, and documents from Australian university research groups, non-government organisations and private companies. The pattern varied little among the three initiatives (Box 1).
Overall, the Medical Journal of Australia (MJA) was the most frequently cited journal, with 392 references to its articles (11.8% of the total 3321 journal articles cited). The second most cited journal was the BMJ, with 150 citations (4.5%), followed by Tobacco Control, with 131 citations (3.9%), which was cited almost exclusively in documents relating to the PHT (Box 2). A total of 817 journals were cited, with the top 10 providing 32% of citations.
Of the many sources of evidence used to support arguments for reform, peer-reviewed journals were cited most frequently. The “grey” literature, such as government reports from reputable independent organisations, including the Australian Institute of Health and Welfare and Australian Bureau of Statistics, also played a prominent role. The dominance of MJA citations in consultation documents and final reports demonstrates that its published content was accessible, credible, valued and relevant to the broad health reform agenda of these diverse initiatives.
Abstract
In 2008, the Australian Government established three major health reform initiatives — the National Health and Hospitals Reform Commission, the first National Primary Health Care Strategy working group, and the Preventative Health Taskforce.
We examined which journals were most frequently cited in the publicly available discussion papers, commissioned papers, submissions and final reports of these initiatives.
Journal articles were cited most in discussion papers, commissioned papers and submissions, followed by reports and other publications from Australian organisations and governments.
The Medical Journal of Australia was the most cited journal, with 392 references to its articles (11.8% of all journal articles cited) in discussion papers, commissioned papers, submissions and an interim report, and 58 references to its articles (13.7% of total journal articles) in the three final reports.
Our findings demonstrate the importance of credible, local, accessible, peer-reviewed evidence in reforming the national health system, including hospitals, primary health care and preventive health care.