Tell me the number; what is the ranking? All of us seem to love ratings. Whether it is the standings in the Rugby World Cup, the box office success of Harry Potter or the melting rate of Arctic ice, we all want numbers. So, why would it be any different for medical journal articles or even medical journals themselves?

Who attaches importance to medical journal ratings? The owners/publishers of the journals, readers, advertisers, librarians and journalists may all be interested in journal ratings to varying degrees. Likewise, authors have a need to discern just how a publication is valued before deciding where to send the products of their labours.

How can we evaluate the quality of an article or a journal? Properties of a medical journal that can be assessed include total circulation; readership numbers and surveys; quality of the editorial board, staff and peer reviewers; number of manuscripts received, percentage accepted, and turnaround; Science Citation Index (SCI) raw numbers, Immediacy Factor and Impact Factor (IF); number of paid subscribers; advertising revenue; listing on Medline; international distribution; cost to the reader; and page or peer-review charges to the author.1

But what do authors most value? Frank and colleagues have surveyed the Stanford University School of Medicine faculty regarding the factors that influenced their decisions about where to send manuscripts. The top attribute selected was "prestige".2

Impact factors are also used to adjudicate on academic performance. Some universities, especially in certain European countries, have decided that the IF of journals in which a faculty member publishes will enter into personnel decisions such as appointment, promotion and rate of pay.3 One would like to think that intelligent deans, chairs of departments and administrators, who work daily with faculty members, would have a better way to ascertain quality of performance than an arbitrary number.

Seglen, of Norway, was an early critic of the IF, drawing attention to its narrow worth, and calling for its application to be reined in3 — but apparently to no avail. My belief is that the IF has one specific meaning: it is a clear measure of the extent to which a given journal functions as a connector of researchers in a specific field. This is one (but only one) critical function of medical journals.

When I began as the editor of JAMA in 1982, JAMA's IF was in the range 3–4. Some considered this an embarrassment, so we set out to raise the IF as part of our efforts to improve the quality of the journal. We succeeded, to the extent that by the time I left the journal in 1999 its IF was in the range 10–11. Strange as it may seem, during the mid-1990s I deliberately tried to slow the growth of JAMA's IF. I was afraid that we were changing the character of the journal away from its fundamental purpose — to be useful to all doctors in their practices — and too far towards a research journal, used by researchers to communicate with each other.

In this issue of the Journal, Walter and colleagues4 (page 280) criticise the IF, clarifying what it is and what it isn't. They describe an alternative way they have devised to judge the quality of articles (and presumably journals, if article scores are aggregated and averaged), using a five-person voting method guided by six criteria. It would have been interesting to see a side-by-side comparison between the article rankings of the selection panel and the SCI IF scores for each article. Walter and colleagues' form of post-publication peer review is now into its second year. The authors invite others to try it, and I hope there will be some who take up the challenge.

In 1982, when I and my colleagues were developing plans to celebrate the JAMA Centennial, we tried an approach to evaluating medical articles somewhat like that of Walter et al. We wished to identify and republish the best 50 articles from the first 100 years of JAMA as "landmark articles". A list of prospective articles for inclusion was compiled from three sources: nominations by JAMA editorial board members and staff, entries in the 1976 edition of A medical bibliography (Garrison and Morton), and the most-cited JAMA articles from the Institute for Scientific Information. A total of 150 articles were nominated. The editorial board and staff then ranked the articles by a Delphi process and the top 50 were named "landmark articles".5 The article publication dates ranged from 1884 to 1968, with representatives from each decade.

A subsequent analysis of the landmark articles by Eugene Garfield, founder of the Institute for Scientific Information (and father of the noted [or notorious] IF), demonstrated that of the 100 JAMA articles most cited by SCI up to 1983 only 13 were among the top 50 landmark articles, garnering from 174 to 506 citations by 1987.6 Thus, 37 landmark articles were not included in the top 100 JAMA articles ranked by total citations alone. Notably, such hugely important articles as those of Salk7 and Sabin et al8 had only received 39 and 90 citations, respectively, by 1987. So, number of citations and the derived IF are connected, but only to a limited degree.

I would hesitate to suggest that the post-publication peer review process described by Walter et al could supplant the IF as the way that academic institutions, or even governments, decide on the merit of a publication or an author. But I can say with conviction that man (and academia) should not live by numbers alone.