The effect of radiographic imaging modalities and the observer's experience on postoperative maxillary cyst assessment.

PURPOSE: The purpose of this study was to compare the accuracy of postoperative maxillary cyst (POMC) diagnosis by panoramic radiographs versus computed tomography (CT) and by oral and maxillofacial radiologists versus non-specialists.
MATERIALS AND METHODS: Sixty-five maxillary sinuses with POMCs and 63 without any lesion were assessed using panoramic radiographs and CT images by five oral and maxillofacial radiologists and five non-specialists on a five-point scale. The areas under receiver operating characteristic (ROC) curves were analyzed to determine the differences in diagnostic accuracy between the two imaging modalities and between the two groups of observers. The intra-observer agreement was determined, too.
RESULTS: The diagnostic accuracy of CT images was higher than that of panoramic radiographs in both groups of observers (p<0.05). The diagnostic accuracy of oral and maxillofacial radiologists for each method was higher than that of non-specialists (p<0.05).
CONCLUSION: The use of CT improves the diagnosis of POMC, and radiological training and experience leads to more accurate evaluation.

Introduction

Postoperative maxillary cyst (POMC) is considered to be a delayed complication that occurs decades after surgical intervention in the maxillary sinus, particularly the Caldwell-Luc operation. This cyst seems to be caused by residual epithelium or regenerated mucosa from the nasal cavity.1,2 It forms soon after surgery and gradually increases in size over the years, until symptoms finally become manifest.1,2,3

Clinically, the patient's chief complaints are swelling, pain, or discomfort in the buccal or maxillary region. Radiographically, it appears as a well-defined unilocular or multilocular cyst in post-operative maxillary sinus and usually causes bony expansion and/or perforation.3 On panoramic and Waters' radiographs, this relatively large lesion shows thinning, loss, or expansion of the lateral wall of the maxillary sinus. On panoramic radiographs, it sometimes shows discontinuity of the inferior wall of the maxillary sinus. The patient's history of maxillary surgical intervention is essential for diagnosis. Preoperative imaging is required for evaluation of and treatment planning for POMC. Exact information as to the extent and location of POMC would help surgeons. Surgical treatment, such as enucleation or marsupialization, is recommended for POMC.2

Many studies on radiologic examinations of POMC by conventional, computerized tomography (CT), and magnetic resonance (MR) images have been reported.4,5,6,7,8 Panoramic and Waters' radiographs are generally used for evaluation of maxillary sinus disease. However, each view has its own limitations due to superimposition of anatomic structures.4,5 CT is the main imaging modality for bony analysis; therefore, CT has become a necessary method for complete evaluation of POMC.6 The border and extension of cysts is better seen on MR images owing to the superior ability of these images to distinguish tissues and substances and to the absence of metallic artifacts from dental restorations that inevitably occur on CT images. However, subtle bony changes cannot be identified on MR images owing to poor visualization of bone anatomy of the sinonasal area.7,8

Research on the influence of an observer's experience and diagnostic accuracy of panoramic radiography and CT on POMC diagnosis is rare. This paper reports a comparison of the diagnostic accuracy of panoramic radiography and CT. Further, we present a comparison of the ability of oral and maxillofacial radiologists to accurately diagnose POMC with that of non-specialists.

Materials and Methods

This study was approved by the Institutional Ethics Review Board of Seoul National University Dental Hospital. The study was based on 128 maxillary sinuses of 64 patients who had undergone examination of their maxillary sinuses by panoramic radiography and CT at this hospital between January 2009 and August 2014 (Fig. 1). Histopathological diagnoses of POMCs were confirmed in 65 sinuses, postoperative change in 54 sinuses, and normal status in 9 sinuses.

Fig. 1

A. Panoramic radiograph shows a postoperative maxillary cyst in the right maxillary sinus involving the inferior and posterolateral walls. B. An axial computed tomography (CT) image shows an expansile lesion with the anterior and posterolateral wall perforation of the maxillary sinus. C. A coronal CT image shows the lateral wall perforation of the right maxillary sinus.

Ten observers - five oral and maxillofacial radiologists and five non-specialists - participated in this study. The oral and maxillofacial radiologists (Group R) each had more than five years of experience in the Department of Oral and Maxillofacial Radiology. The non-specialists (Group N) were dentists who had no previous experience working in the Department and Oral and Maxillofacial Radiology.

Prior to interpreting the radiographs, these observers were told that the radiographic characteristics of a POMC included a unilocular or multilocular round radiolucency with a sclerotic margin and expansion and/or thinning of the walls of the maxillary sinus. The two groups then interpreted the panoramic radiographs and CT images separately. All images were displayed on a high-resolution monochrome TFT-LCD monitor (ME315L, Totoku Electric Co. Ltd., Tokyo, Japan) in the DICOM format with a picture archiving and communications system (PACS). Ambient light was as low as feasible in order to eliminate reflections in the monitor. All observers adapted fully to the dark room condition before the evaluation was begun. Further, all observers were allowed to adjust image brightness and contrast and to magnify the image as per their preference by mouse control. No limitation was imposed on the viewing time. The observers did not have any information on the patient or on the proportion of POMCs. They were asked to state whether a lesion was present or not according to the following five-point scale: 1, definitely or almost definitely negative; 2, probably negative; 3, intermediate; 4, probably positive; and 5, definitely or almost definitely positive. All observers viewed the images twice in two weeks in a blind manner.

Receiver operating characteristic (ROC) analysis with the calculation of the area under the ROC curve (AUC) was performed. The AUC was calculated for four groups: Group R interpreting the panoramic radiographs; Group R, the CT images; Group N, the panoramic radiographs; and Group N, the CT images. Overall, observer performances according to the imaging modalities and observers' experience were determined by averaging the AUC values of all the groups. The AUC value of each group was statistically analyzed using a Mann-Whitney test to determine the differences between Group R and Group N by using the two imaging techniques and within the two groups themselves. Statistical analysis was performed using SPSS for Windows (version 21.0, SPSS Inc., Chicago, USA), and p<0.05 was considered to indicate statistical significance. To assess intra-observer and inter-observer agreements, the kappa (κ) value was calculated.

Results

Table 1 presents the AUC values according to imaging modalities and observers. AUC is a measure of the overall performance of a diagnostic test and is interpreted as the average value of sensitivity for all possible values of specificity. The closer the AUC value is to 1, the better is the overall diagnostic performance of the test, and a test with an AUC value of 1 is considered perfectly accurate.9 In Groups R and N, the AUC value of the CT was significantly higher than that of the panoramic radiographs (p<0.05). The diagnostic accuracy of Group R was better than that of Group N for both the panoramic radiographs and CT (p<0.05).

Table 1

Mean area under the receiver operating characteristic curve of oral and maxillofacial radiologists (Group R) and non-specialists (Group N) with panoramic radiographs (PAN) and computed tomography (CT) images

*,†,‡,§: correlation is significant at the 0.05 level (two-tailed), S.D.: standard deviation

The intra-observer agreement of the observers is presented in Table 2. The kappa (κ) value of κ≥0.75 means excellent agreement; 0.40≤κ<0.75, fair or good agreement; and κ<0.40, poor agreement. The intra-observer agreement of Group R was excellent (κ≥0.75) or fair or good (0.40≤κ<0.75), but that of Group N was poor (κ<0.40) except for one observer (κ=0.486). There was a tendency for kappa values for intra-observer agreement to be higher for the oral and maxillofacial radiologists. The kappa values of the CT images in Group R were higher than those of the panoramic radiographs.

Table 2

Intra-observer agreement of oral and maxillofacial radiologists (Group R) and non-specialists (Group N) with panoramic radiographs (PAN) and CT images (kappa value)

Discussion

Most of the reported POMCs resulted from operations for maxillary sinusitis, particularly the Caldwell-Luc approach. Other POMC cases have been attributed to a variety of causes such as gunshot injuries, mid-facial fractures, maxillary orthognathic surgery, and maxillary sinus augmentation.10,11,12,13,14 One case in our study was a POMC due to maxillary orthognathic surgery (Fig. 2).

Fig. 2

A. Panoramic radiograph shows a postoperative maxillary cyst in the right maxillary sinus due to maxillary orthognathic surgery. B. An axial CT image shows an expansile lesion with the anterior and posterolateral wall perforation in the maxillary sinus. C. A coronal CT image shows the lateral wall perforation and surgical screw with the right maxillary sinus.

Panoramic radiography is generally the preferred imaging modality for detecting maxillary sinus pathoses. It gives a clear view of the boundaries of the medial and posterior walls, and the floor of the maxillary sinus.15 However, in a panoramic radiograph, the central ray is approximately parallel to the medial and posterior wall and the other walls are superimposed.15,16 Ohba et al17 reported that the panoramic radiograph was less effective for a lesion on the anterior and medial walls. Heo et al18 found that observers failed to detect lesions limited to the sinus, for example, those on the floor of the maxillary sinus without any apparent bony expansion or those only on the anterior wall when bony expansion was seen. Our study also showed that it was difficult for all observers to detect POMC limited to the sinus.

CT is useful in evaluating the loculation of the lesion, extent of sinus involvement, position of the lesion, and direction of expansion, which comprise information that is important in deciding upon the surgical approach.6,20,21 Cable et al19 revealed that CT was superior to plain films in the evaluation of the maxillary sinus after a Caldwell-Luc operation. A comparison of panoramic radiographs, Waters' radiographs, tomography, and CT was undertaken by Yoshimura et al,20 who found that CT was the most explicit for detecting POMC. Our results concurred with those of the previous studies.

On the evaluation of observer variation with respect to POMC diagnosis, a few studies have been reported. Heo et al18 reported that the diagnostic accuracy of CT was higher than that of combined panoramic and Waters' radiographs for oral and maxillofacial radiologists, but not for non-specialists. However, in our study, the diagnostic accuracy of CT was superior to that of panoramic radiographs for both groups of observers. The mean AUC of CT of non-specialists in our study was 0.818, whereas that of non-specialists in the study by Heo et al18 was 0.699. This difference was attributed to the fact that non-specialists in our study had been more educated about CT images than those in Heo et al's study because cone-beam CT had come into wide use in local dental clinics recently. The diagnostic accuracy of oral and maxillofacial radiologists was consistently higher than that of non-specialists for the methods that we examined. The differences were due to insufficient training and lack of experience of the latter.

Intra-observer agreement (kappa values) ranged from 0.425 to 1 and was fairly good or excellent for the oral and maxillofacial radiologists, whereas it was poor for the non-specialists except for one observer (κ=0.486). This implied that the oral and maxillofacial radiologists who participated in this study had their own decision criteria, but the non-specialists did not.

The diagnostic accuracy of the oral and maxillofacial radiologists was higher with CT than with the panoramic radiographs, indicating that CT offered more detailed information on the maxillary sinus.

In conclusion, CT is superior to panoramic radiographs for evaluating POMC, and radiological training and experience is required to evaluate POMC accurately.