Enter the Web: an experiment in electronic research peer review

Editorial

Enter the Web: an experiment in electronic research peer review

See also the Internet peer review study home page for subsequent developments.

The World Wide Web is a system for electronic publishing on the Internet. Electronic documents created for the Web can have many features not possible in printed documents. They include hyperlinks: marked text or images within the document that link to other related documents, wherever they might be stored in the Internet. Click on a link and the linked document is brought to your computer screen. Hence the name, "World Wide Web": documents from all over the world join into a web of information that can be rapidly traversed, across national and disciplinary boundaries, to wherever the reader wishes to go.

Web documents can include animated images, sound recordings and various interactive elements, such as the capacity to send an immediate e-mail response to the author, or search a computer database. The Web has the potential to create a closer communication between authors and readers, or even a communication space in which everybody is both author and reader.

Since its inception in 1990, the Web has rapidly grown in size and function to become the "hottest of the hot" applications for the Internet. Governments, universities, businesses, hospitals, courts, single individuals, newspapers and learned journals have rushed to create "Web pages" announcing their existence to the entire (networked) world.

Publishing on the Web is not technically difficult (which is why many individuals have created their own Web pages) but not free, as both the publisher and the reader must pay the costs of connecting to the Internet. However, the publisher has none of the costs associated with printing and distributing a paper publication, and has an effective worldwide distribution that takes seconds rather than weeks.

So, for reasons of economy and utility, there is an incentive to move information publishing from print to the Web. Against this, there is the obvious objection that most readers are not connected to the Internet, plus a problem for publishers in establishing how they will be paid.

At present, most Web pages are freely available to anyone who is connected to the Internet. For instance, one can browse Web sites for Nature,¹ the British Medical Journal² or the Journal of the American Medical Association³ without paying a subscription to any of these journals. Of course, what is available on the Web is no substitute for the paper journal. At the BMJ site, for instance, there are contents lists and the full text of selected articles, plus details of how to subscribe to the printed publication. Such Web sites function more as advertising for or adjuncts to print ( and how indeed could they be anything else, until such time as large numbers of readers demonstrate a willingness to pay for an electronic journal?

It is technically possible to make Web pages available only to paying subscribers, but as yet this is a rarity, with most publishers testing the market and the technology with freely distributed material. Meanwhile, there are those who hope that the journals will die a natural death and that the Web will provide a new publication system entirely free of publishers.⁴

One such vision describes a "global health information server", a Web site where all medical writers could publish their writings, and where editorial selection and peer review processes would be replaced by an automatic system of scoring articles by the number of readers they had attracted.⁴ Such a system is described as more "democratic" and would not refuse publication to anybody.

The authors of this proposal ask "How will the world of health information look when every original paper, letter of criticism, and review article, as well as every form, chart, and database in the computerised world, is accessible with a couple of dozen clicks of a mouse?" The answer might well be "Hopelessly overloaded". Anyone who currently uses the Web to find information has had the experience of losing a grain of sense in a mountain of chaff; medical journals may have a great future on the Web precisely because they offer a selection of material that is intelligently tailored to the needs of particular readers.

At the MJA, we have been as excited as anyone about the potential of the Web and have been looking for ways to exploit it. How can a journal be improved by using the Web? If the Web is to be used for faster or wider electronic publication of research, can this be done within the framework of reliable editorial control and peer review? Or, to put this question the other way around, can using the Web overcome criticisms of editorial control and peer review (i.e., that these processes may clog the progress of research,⁵ that they themselves are uncontrolled and potentially arbitrary, unscientific or unfair^4-7)?

To address these questions, we are going to conduct an experiment this year in electronic publication and open peer review. The project has been made possible by the cooperation of the University of Sydney Library (USL), which is providing essential expertise and support in Web publishing, and by a grant from the Electronic Publishing Working Group of the Australian Vice Chancellors' Committee.

The purpose of the AVCC grants is to encourage innovative models of electronic publication for the creation and distribution of Australian research. The MJA and the USL are working in partnership on this initiative to develop and evaluate such a model.

The project provides opportunities for both partners to develop technical and management skills, investigate the complex issues around electronic publication and position themselves to take further advantage of these new technologies. The USL is also working with the University's Faculty of Medicine in the use of electronic resources for medical teaching and research.

In brief, a Web site will be created for the MJA where selected research articles that have gone through our traditional peer review process and have been accepted by the MJA will be published, together with comments provided by our peer reviewers. The papers will undergo minimal editing at this stage, and the effort will be to achieve rapid electronic publication, without the delays necessary to print. Readers on the Internet will be able to review the articles and reviewers' comments and, using a response mechanism built into the Web site, e-mail their own comments to the MJA. These comments will be filtered editorially to remove irrelevant material, then passed on to the authors and peer reviewers as feedback. Selected comments may be electronically published with the papers and reviews as additional commentary; authors will be able to respond or revise their paper in response.

After a period on the Web, papers will undergo their "definitive" editing and be published in print in the MJA.

Quantitative data (number of participants, number of Web readers, and so on) will be collected via the computer system, and qualitative assessments will be sought from authors, reviewers, editorial staff and an external Project Review Group.

Participation in the open peer review experiment by authors and reviewers will be voluntary, and one thing we look forward to discovering is how many wish to be involved. Authors will be offered more rapid and more international publication, so we expect that most will be keen ( but will our peer reviewers, who have been used to the cloak of anonymity, be willing to have their comments on papers made public? Will this opening up of the review process to wider scrutiny have an effect on the quality (already excellent) of the reviews we receive? And will the comments posted to our Web site represent a valuable extension of the peer review process, leading to further improvements in papers before their appearance in print?

All these questions and many related questions of detail and method introduce the MJA to a new world of electronic research publishing. We do not yet know the shape of the terrain, but we are set to explore.

Craig Bingham
Publication Coordinator, MJA
Ross Coleman
Collection Management Librarian
University of Sydney Library, Sydney, NSW

(©MJA 1996; 164: 8-9)

1. Nature Web home page. http://www.nature.com/
2. British Medical Journal Web home page. http://www.bmj.com/bmj/
3. Journal of the American Medical Association Web home page. http://www. ama-assn.org/journals/standing/jama/jamahome.htm
4. La Porte RE, Marler E, Akazawa S, et al. The death of biomedical journals. BMJ 1995; 310: 1387-1390.
5. Horrobin DF. The philosophical basis of peer review and the suppression of innovation. JAMA 1990; 263: 1438-1441.
6. Matthews R. Storming the barricades. New Scientist 17 June 1995: 38-41.
7. Lock S. A difficult balance. Editorial peer review in medicine. London: BMJ, 1991: 23-55.