Evaluation of the PAPNET system in a general pathology service

The failure of Pap smears to detect some high-grade abnormalities is the result of either sampling or laboratory error.² Sampling error occurs when the person taking the smear fails to collect the abnormal cervical cells because of either poor technique or the small size or localised nature of the lesion.³ Laboratory error is usually caused by the screening cytologist failing to detect the abnormal cells on the slide.

Screening of Pap smears is a labour-intensive task requiring a high degree of skill and concentration. Even in laboratories using well-recognised quality control measures and staffed by highly trained cytologists, many high-grade abnormalities remain undetected by routine manual screening. These may also be missed on a manual rescreen.³

PAPNET is an automated interactive system for analysis of Pap smears which has been shown to detect abnormalities that were repeatedly missed on manual screening.^4,5 We introduced the PAPNET system into our laboratory in January 1995 as a quality-control measure for rescreening of all negative (normal) smears. Here, we compare the detection of abnormalities by PAPNET with those detected by manual screening in our laboratory, and report on the histological follow-up of the PAPNET cytopredictions and the work practices of cytologists using this system.

Interim reports were generated for smears considered normal and technically unsatisfactory, and, after the slide review was completed in the laboratory, a final report was issued incorporating the PAPNET findings. Reporting terminology was based on the Bethesda system of reporting cervical/vaginal cytological diagnoses.⁶

A record was kept of the number of cases each cytologist was able to review per day with PAPNET. The number of slides that required manual rechecking and complete rescreening was also recorded. This involved the cytologists using the light microscope to recheck individual abnormal cells that were tagged on the PAPNET monitor. If these cells were considered to be significantly atypical, the whole slide was rescreened. If any degree of abnormality was found after rescreening, the slide was submitted to the cytopathologist for checking.

To test the specificity of PAPNET, histological follow-up was performed for all patients who had an additional lesion predicted by PAPNET and were recommended to have colposcopy for this lesion. As follow-up of cytological predictions can take many months, the process was expedited and, where possible, histology results are given. In some cases, follow-up was still pending or the referring doctor had decided to follow-up by repeated observation and Pap smears. These are designated as "follow-up to come". In some cases, follow-up was unavailable; the referring doctor had been unable to contact the patient, the patient had moved or had decided to proceed no further with management. These cases are designated as "lost to follow-up".

PAPNET screening was performed on 54 658 smears classified on manual screening as negative (or within normal limits) and 1819 smears classified as technically unsatisfactory. Of the 3840 smears classified on manual screening as abnormal, 1022 smears (27%) were randomly selected and submitted for PAPNET screening: 807 low-grade squamous intraepithelial lesions, 122 high-grade squamous intraepithelial lesions, 91 atypical squamous cells of uncertain significance, and two cases of squamous cell carcinoma.

After training, cytologists performing PAPNET rescreening were able to screen between 100 and 160 slides (mean, 126; SD, 15) per day. This included loading and unloading the tapes, viewing and assessing the images on the PAPNET screen, documentation and generating patient reports.

The number of cases submitted for limited rechecking varied between 10%-90%, depending on the cytologist. The number requiring complete rescreening by a senior cytologist varied between 0 to 10 per cytologist per day and averaged five per day. The number of cases submitted to a cytopathologist by each cytologist averaged one per day.

Although the detection rate of abnormalities on manual screening was well within normal standards in our study, PAPNET rescreening was able to detect an increased number of abnormal Pap smears. This increase was not associated with a loss of specificity, as often occurs with increased detection by manual screening.³

Many studies have shown the ability of PAPNET rescreening to detect a significant number of abnormal Pap smears when compared with manual screening, but these have been predominantly smaller studies using archived material.^8,9 Our study was undertaken as part of the routine workflow in a laboratory processing a large number of smears and is the first Australian study to compare the two screening systems.

The reported increase in detection of abnormalities in other studies has been as high as 16%,¹⁰ and the company which developed PAPNET (Neuromedical Systems Inc., New York, USA) claims increases of up to 30%. The increase in our study was much smaller (7%), with most detected abnormalities being low-grade lesions. However, we believe this increase was worthwhile given the reasonably high rate of detection on manual screening and the large volume of routine smears processed.

Boon et al. reported the results of a large series of smears that were not double-screened, but had been either manually screened or PAPNET screened. Higher detection rates of abnormalities were found in the PAPNET group. Our study is the first report of a large series in which all negative smears were screened both manually and by PAPNET.

Our study showed that the PAPNET system could be feasibly integrated into a routine working laboratory situation. It was a valuable educational and quality assurance tool. Cytologists found it easy to use and were able to review slides at three to four times their manual-screening rate. From our experience, cytologists did not become complacent in their manual screening, but rather became more cautious knowing that all the negative smears would be examined by a computerised device. The additional work for cytopathologists and senior cytologists was not great.

PAPNET is only one of a number of automated cytology systems currently available as an additional test for routine Pap-smear screening. We did not undertake a comparison of other automated systems or their cost-effectiveness. Further studies on the cost-effectiveness of PAPNET testing and its efficacy for use in public health screening programs are needed. It may be that automated testing technology will replace current screening methods in the future.

1. Mitchell HS, Giles GG. Cancer diagnosis after a report of negative cervical cytology. Med J Aust 1996; 164: 270-273.
2. Koss LG. The Papanicolaou test for cervical cancer detection. A triumph and a tragedy. JAMA 1989; 261: 737-743.
3. Boscha MC, Rietveld-Scheffers PEM, Boon ME. Characteristics of false negative smears tested in the normal screening situation. Acta Cytol 1992; 36: 711-716.
4. Koss LG, Hin E, Schreiber K, et al. Evaluation of the PAPNET cytologic screening system for quality control of cervical smears. Am J Clin Pathol 1994; 101: 220-229.
5. Boon M, Kok CP. Neural network processing can provide means to catch errors that slip through human screening of Pap smears. Diagn Cytopathol 1993; 9: 411-416.
6. The Bethesda system of reporting cervical/vaginal cytologic diagnoses: revised after the second National Cancer Institute Workshop, April 29-30, 1991. Acta Cytol 1993; 37: 115-124.
7. Royal College of Pathologists of Australasia. Performance Standards for Australian Laboratories Reporting Cervical Cytology. Sydney: RCPA Quality Assurance Programs, 1995.
8. Rosenthal DL, Acosta D, Peters RK. Computer-assisted re-screening of clinically important false negative cervical smears using the PAPNET testing system. Acta Cytol 1996; 40: 120-126.
9. Sherman ME, Mango LJ, Kelly D, et al. PAPNET analysis of reportedly negative smears preceding the diagnosis of a high grade squamous intraepithelial lesion or carcinoma. Mod Pathol 1994; 7: 578-581.
10. Elgert PA, Suhrland MJ, Re E, et al. Prospective quality control of cervical smears using the PAPNET system [abstract No. 5]. In: Abstracts of the Scientific Session of the 42nd Annual Scientific Meeting of the American Society of Cytology; 1994 Nov 1-6; Chicago. Acta Cytol 1994; 38: 793-805.
11. Boon ME, Kok LP, Beck S. Histologic validation of neural network assisted cervical screening: comparison with the conventional procedure. Cell Vision 1995; 2: 23-27.

Reprints: Dr A Farnsworth, Douglass Hanly Moir Pathology, 95 Epping Road, North Ryde 2113.

Journalists are welcome to write news stories based on what they read here, but should acknowledge their source as "an article published on the Internet by The Medical Journal of Australia <http://www.mja.com.au>".
<URL: http://www.mja.com.au/>
We appreciate your comments.