Counting on citations: a flawed way to measure quality

Walter, Garry; Fisher, Karen; Bloch, Sidney; Hunt, Glenn

doi:10.5694/j.1326-5377.2003.tb05196.x

Topics

The journal Impact Factor and citation counts are misconstrued and misused as measures of scientific quality. Articles must be read in order to judge their quality. We have introduced a system, which may be easily replicated, to identify the best articles published in a journal.

Gloom or glee? Each September, journal editors and publishers anxiously await news of a particular figure from the Institute of Scientific Information (ISI) in Philadelphia, USA. The figure's value is promptly met with despair or delight. We are referring, of course, to the Impact Factor (IF) and the ritual surrounding its release that has come to dominate the editors' and publishers' calendar.

1 Rivendell Unit, Child Adolescent and Family Psychiatric Services, Thomas Walker Hospital, Concord West, NSW.
2 Department of Psychiatry, University of Melbourne, Fitzroy, VIC.
3 Department of Psychological Medicine, Research Unit, Rozelle Hospital, Rozelle, NSW.

Correspondence:

Competing interests:

None identified.

1. Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ 1997; 314: 498-502.
2. Hecht F, Hecht BK, Sandberg AA. The Journal Impact Factor: a misnamed, misleading, misused measure. Cancer Genet Cytogenet 1998; 104: 77-81.
3. Coleman R. Impact Factors: use and abuse in biomedical research. Anat Rec 1999; 257: 54-57.
4. Bloch S, Walter G. The Impact Factor: time for change. Aust N Z J Psychiatry 2001; 35: 563-568.
5. Garfield E. Journal Impact Factor: a brief review. CMAJ 1999; 161: 979-980.
6. Institute of Scientific Information. ISI Web of Knowledge. Available at: http://www.isinet.com (accessed Jul 2002).
7. Hansson S. Impact factor as a misleading tool in evaluation of medical journals [letter]. Lancet 1995; 346: 906.
8. Seglen PO. Citation and journal impact factors: questionable indicators of research quality. Allergy 1997; 52: 1050-1056.
9. Seglen PO. Citation and journal impact factors are not suitable for evaluation of research. Acta Orthop Scand 1998; 69: 224-229.
10. Opthof T. Sense and nonsense about the Impact Factor. Cardiovasc Res 1997; 33: 1-7.
11. Institute of Scientific Information. Web of Science. Available at: http://wos.isiglobalnet2.com (accessed Dec 2001).
12. Prestige factor. Available at: http://www.prestigefactor.com (accessed Nov 2001).
13. Journal of Medical Ethics online. Available at: http://www.jmedethics.com (accessed Dec 2002).
14. Medical Science Monitor online. Available at: http://www.medscimonit.com (accessed Dec 2002).
15. Citation data: the wrong impact? [editorial]. Nat Neurosci 1998; 1: 641-642.
16. Adam D. The counting house. Nature 2002; 415: 726-729.
17. Deutsche Forschungsgemeinschaft. Recommendations of the Commission on Professional Self Regulation in Science. Proposals for safeguarding good scientific practice. January 1998. Available at: http://www.dfg.de/aktuelles_presse/reden_stellungnahmen/download/self_regulation_98.pdf (p.8) (accessed Sep 2002).
18. Faculty of 1000. Available at: http://www.facultyof1000.com (accessed Jul 2002).
19. Recommended reading? [editorial]. Nat Med 2002; 8: 1.

Online responses are no longer available. Please refer to our instructions for authors page for more information.