Peer review: maintaining trust in research funding decisions

Ensuring that the peer-review process adapts as medical research changes

Researchers in Australia have recently criticised peer review of applications for grant funding, arguing that the process involves too much work, is too arbitrary and is too conservative in identifying the best grants.1

These criticisms need to be addressed, but they must also be balanced by other considerations, especially the interests of the wider community.

Public research funding bodies like the National Health and Medical Research Council (NHMRC) and the Australian Research Council rely on peer review when deciding who and what to fund. Because Australians' taxes pay for the funding, funders have responsibilities to ensure that peer review is able to identify the most valuable research to fund, and to do so fairly, free from bias and self-interest.

The NHMRC's funding decisions depend almost entirely on researchers' peer review. It is remarkable that the public trusts us researchers with towards a billion dollars a year of their money, and we have a responsibility to maintain this trust.

Peer reviewing is an essential part of being a scientist, and we can expect to spend a sizeable amount of our time providing reviews of other researchers' applications for funding (and publishing). It is a type of mutual obligation — participation in the peer-review process is our reciprocal duty towards other researchers.

To maintain public and researcher trust, it is essential that processes are fair and provide applicants natural justice. There is never enough money to fund all worthwhile research. Therefore, it is important to ensure that the playing field is level. As far as possible, applicants need to be aware of the assessment criteria, and be given a chance to respond to the views of reviewers, including possible misconceptions or errors. Over a decade ago, the NHMRC introduced specific criteria for the assessment of applications and assigned specific weighting to each of these. Applicants' understanding of what they will be assessed on also allows them to write their applications accordingly — a general principle now generally agreed on in the San Francisco declaration on research assessment (http://www.ascb.org/dora-old/files/SFDeclarationFINAL.pdf)

Peer review is a human activity, so any two researchers are likely to have different views. For funders, it is essential to bring as many experts as possible to the review and to ensure that interests (scientific or material) are transparent and taken into account. Peer review is inherently tied up with the values and knowledge of the individuals involved. It follows that it is important to have knowledgeable and ethical researchers involved. Peer review can never be reduced to a simple number that does not require human judgement.

Humans do err in their judgements, and there are few researchers who would be willing to leave a decision on their application to a “lottery”, a political process or the decision of a single superior (eg, a deputy vice-chancellor or institute director). In the 21st century, few researchers will tolerate “I know a good grant when I see one — just trust me”.

We can always do better. Funders need to constantly assess their processes and the outcomes. More training and mentoring in quality peer review would also help. Most of us learn about peer review informally as early career researchers through departmental and research group seminars, and as we begin to be asked to assess grants and papers submitted for publication.

Can peer review ever be a precise instrument? No, but we should strive to make it as good as possible, in order to identify the best research to fund, fairly and without bias.

Health and medical research grows and changes. It is becoming more diverse, complex, multidisciplinary and collaborative across institutions and around the world. We all need to ensure that the peer-review process adapts and evolves to meet the challenges that these changes are bringing.