Mapping Australia's basic research in the medical and health sciences

Introduction

Data source

We also code the sector -- universities, hospitals, medical research institutes, government institutions, and "other" -- of each address. The medical research institutes sector comprises the 25 members of the Australian Association of Medical Research Institutes (AAMRI). Medical research in the Government sector comes primarily from the Commonwealth Scientific and Industrial Research Organisation (CSIRO) and the State and Federal departments of health. The "other" sector includes industry and non-profit organisations.

We acknowledge "grey areas" at the margins of these assignments to sectors. For example, where a research group based in a hospital with a university connection lists the university in the address, we consistently assign the publication to the university. Another example is the John Curtin School of Medical Research (JCSMR), a full-time medical research institute (though not a member of AAMRI), which is also a research school of the ANU funded from that university's operating grant. Its publications are assigned to the ANU and hence fall within the universities sector.

Most publications can be unambiguously assigned, but some addresses are not precise guides to the source where the research was conducted. This arises particularly for researchers who hold conjoint or adjunct appointments in two institutions. If such an author nominates a single institution, we accept that this is the location of the research leading to that publication. If an author specifies two separate addresses, two records are created for that publication showing the two institutions (double-counting created by this procedure is removed for sectoral and national analysis, but remains when the individual institution is the focus of the analysis). If the address itself has multiple components, the publication can not be split into multiple records and has to be assigned to the apparent primary institution. Very few publications fall into this latter category, and their existence has little effect on the map of national, sectoral or institutional sites of research.

We have used the standard bibliometric practice of ascribing an article to a particular field of research on the basis of the classification of the journal in which it appears. This procedure is not without problems, particularly in the case of analysis at the subfield level, but experience from our other studies has shown that the results are accurate if the data are being used to map research in a large field.³  

To what extent is Australian basic research in the medical and health sciences covered by the REPP database?

In this article, we are using the REPP database to answer questions about the map of medical research in Australia, concentrating primarily on publications from 1990 onwards, but also introducing some time-series analysis.  

Where is the research being conducted?

Our data also show that little has changed in the sectoral location of research over the past 15 years: the share of publications from the hospitals sector has remained constant, there has been a small drop in the universities share, and there has been a corresponding increase in the share from the medical research institutes.  

Are patterns of medical authorship changing?

Figure 2 shows how the type of authorship of medical research articles has changed in the past 15 years. The two authorship types showing marked decline are "single author" articles, and "group" articles. Medical and health sciences in Australia appeared slow to exhibit the "internationalisation" of research apparent in other fields since the early 1980s, but the period since 1987 has seen a dramatic change. The proportion of publications involving international collaboration nearly doubled between 1987 and 1995. Collaboration with other Australian institutions has also become more common, increasing from 19% to 26% over the 15-year period. (A detailed analysis of Australia's international collaboration in basic research may be found in a monograph we prepared for the Australian Research Council.⁶ )

How "visible" is Australian medical research?

The most notable feature of this Figure is that Australia's share of publications in SCI medical journals increased by 25% between 1986 and 1995. The average RCI for the whole period was 1, indicating that Australian publications are attracting appropriate notice. Australia's RCI has not changed significantly over time, but remained at or marginally above 1. This is a strong performance as citation rates are influenced principally by publications from the major research centres of America and Europe.  

Where is Australia's most "visible" medical research being conducted?

Some of the differences in cpp rates can be attributed to the varied research profiles of different institutions. Data supplied by ISI enable us to quantify this, as we can calculate cpp rates for sets of journals. For example, articles in immunology journals for the same period attracted citations at the average rate of 6.21, while those in clinical sciences journals averaged 3.81. We would therefore expect hospitals, with their strong clinical focus, to have a lower cpp rate than those AAMRI establishments with a strong presence in immunology. However, the difference in cpp rates apparent in Box 1 (above) cannot be explained fully by differences in field concentrations; we calculate that differing citation rates across fields account for only a third of the gap in cpp rates between the AAMRI institutions and other sectors. The remainder is a measure of the differing visibility and impact of the research.

Not all institutions within a given sector have similar visibility. In Box 2 we list the top five institutions (in terms of cpp rates) in each of the four sectors active in medical research. In the universities sector, we looked specifically at the teaching medical schools and for this reason have excluded JCSMR. The institutions in Box 2 are those with more than 100 SCI publications in the period. In this instance, we count publications in multidisciplinary journals such as Nature, Science and Proceedings of the National Academy of Sciences together with publications in medical and health sciences journals, on the assumption that for the listed institutions and faculties these articles would almost certainly relate to medical research.

Sorting the institutions in Box 2 by cpp rates within sectors has the effect of allowing volume of publications to be moderated by impact. The consequences of the choice of the measure on which to rank institutions can be seen by a closer examination of the medical research institutes. While the Walter and Eliza Hall Institute of Medical Research has the largest number of publications and citations, and would be ranked first if these were the measures used, the Ludwig Institute for Cancer Research is top-ranked on the basis of average cpp rates. All three are measures of impact and visibility, but cpp rates take institutional size into account.

We looked at the research focus of institutions to determine if the higher cpp rates of some institutions resulted from differing fields of concentration. Again, we found that while an institution's cpp rate was affected by the relative impact of the fields in which it was active, this accounted at most for only 30% of the variation in cpp rates between institutions in any given sector.  

Where are Australia's most highly cited medical articles produced?

Discussion

The limitations of bibliometric analysis are well documented.⁹ Publications can attract large numbers of citations because they contain error, or because they report a new technique with wide application. Citation data are also highly skewed. Many publications attract no citations at all, and most of those that do receive only one or two. These and other problems are of little consequence when the focus is at the national or sectoral level, involving large numbers of publications.¹⁰

Our data provide only one approach to constructing a profile of Australian medical research. However, bibliometric or literature-based analysis cannot stand in isolation from historical and other kinds of evaluative judgements, and should not be used in a policy setting apart from those perspectives. That said, we believe that the information reviewed here does allow some interesting points to be made.

The most encouraging inference we draw from our study is that, using the measure of Relative Citation Impact, Australian medical research stands relatively high in terms of international visibility. As Box 3 makes clear, Australian-based researchers publish in international journals and have well established links to collaborative projects in the major centres of work in the field.⁶

The bulk of Australia's basic research in the medical and health sciences comes from the universities and hospitals, but Australia's medical research institutes, the members of AAMRI, have the highest international profiles. Research from these institutions has had a major impact on the international community. However, high visibility is not confined to the AAMRI institutions, and, as we have shown, research achieving very high impact also comes from hospitals and universities.

One of the most interesting policy issues which these data raise is the efficacy of block funding by comparison with direct project funding of research. Medical research in Australia is undertaken in a pluralist system, with major contributions from hospitals, universities, AAMRI and government institutions. While at first glance our data may appear to argue for block funding through the prominence of several AAMRI institutions funded in this way, the situation is more complex. Many of the AAMRI institutions are block-funded, but in some instances this accounts for as little as 35% of their total income. Any conclusions about funding await the completion of detailed bibliometric studies of the relative performance of medical research, in which we will attempt to identify the output of research supported by the varying modes of research funding.  

References

1. Bourke P, Butler L. A Crisis for Australian science? Canberra: Performance Indicators Project, Australian National University, 1993. (Monograph Series No. 1.)
2. < http://coombs.anu.edu.au/Depts/RSSS/REPP/repp.htm >
3. Butler L, Bourke P, Biglia B. CSIRO: profile of basic research. Canberra: Research Evaluation and Policy Project, Australian National University, 1997. (Monograph Series No. 4.)
4. National Board of Employment Education and Training (NBEET). Quantitative indicators of Australian academic research. Canberra: AGPS, 1994. (Commissioned Report No. 27.)
5. Bourke P, Butler L. Monitoring research in the periphery. Canberra: Research Evaluation and Policy Project, Australian National University, 1996. (Monograph Series No. 3.)
6. National Board of Employment Education and Training. International links in higher education research. Canberra: AGPS, 1995. (Commissioned Report No. 37.)
7. Smith R. Top of the pile: the institutes. BMJ 1991; 302: 1006-1010.
8. Narin F. Evaluative bibliometrics. Cherry Hill, NJ: Computer Horizons Inc, 1976.
9. Van Raan AFJ, editor. Handbook of quantitative studies of science and technology. Amsterdam: Elsevier Science Publishers, 1988.
10. Garfield E. In: Evered D, Harnett S, editors. Ciba Foundation Conference: the evaluation of scientific research. Chichester (UK): John Wiley & Sons, 1989.