Evaluation of maxillary sinusitis from panoramic radiographs and cone-beam computed tomographic images using a convolutional neural network.

Purpose: This study developed a convolutional neural network (CNN) model to diagnose maxillary sinusitis on panoramic radiographs (PRs) and cone-beam computed tomographic (CBCT) images and evaluated its performance. Materials and
Methods: A CNN model, which is an artificial intelligence method, was utilized. The model was trained and tested by applying 5-fold cross-validation to a dataset of 148 healthy and 148 inflamed sinus images. The CNN model was implemented using the PyTorch library of the Python programming language. A receiver operating characteristic curve was plotted, and the area under the curve, accuracy, sensitivity, specificity, positive predictive value, and negative predictive values for both imaging techniques were calculated to evaluate the model.
Results: The average accuracy, sensitivity, and specificity of the model in diagnosing sinusitis from PRs were 75.7%, 75.7%, and 75.7%, respectively. The accuracy, sensitivity, and specificity of the deep-learning system in diagnosing sinusitis from CBCT images were 99.7%, 100%, and 99.3%, respectively.
Conclusion: The diagnostic performance of the CNN for maxillary sinusitis from PRs was moderately high, whereas it was clearly higher with CBCT images. Three-dimensional images are accepted as the "gold standard" for diagnosis; therefore, this was not an unexpected result. Based on these results, deep-learning systems could be used as an effective guide in assisting with diagnoses, especially for less experienced practitioners.

Introduction

Sinusitis refers to inflammation of the sinuses, which are normally filled with air. The cause of inflammation may be infectious (bacterial, viral, or fungal factors) or noninfectious (allergic factors).1 Three important factors are involved in the pathogenesis of sinusitis: a narrow sinus ostium, ciliary apparatus dysfunction, and the viscosity of sinus secretions. The narrow diameter of the sinus ostium predisposes it to the formation of an obstruction. Factors that predispose the ostium to obstruction include those that result in mucosal swelling and those that directly cause mechanical obstruction. Of these causes, viral upper respiratory tract infection and allergic inflammation are the most common and most important.2 When obstruction of the sinus ostium occurs, there is a temporary increase in pressure within the sinus cavity. Because oxygen is depleted in this closed area, the pressure in the sinus becomes negative relative to the atmospheric pressure. This negative pressure may allow nasal bacteria to enter the sinuses while sniffing or blowing.3 Mucociliary apparatus dysfunction also contributes to sinusitis pathogenesis. After catching a viral cold, both the structure and function of the mucociliary apparatus are disrupted.4 The quality and features of sinus secretions also play a role in the pathogenesis. Cilia can only beat in fluid media. Changes in the mucous layer that occur in the existence of inflammatory debris, such as in an infected sinus, may further damage ciliary movement.5

Odontogenic maxillary sinusitis usually occurs because of apical and/or periodontal lesions of the maxillary molar teeth.67 Pathologies of the maxillary sinus are frequently observed in patients who have dental problems, such as periapical lesions, cysts, or tumors. Radiographic modalities are commonly used to evaluate the maxillary sinus. Panoramic radiography, the Waters’ view, computed tomography, magnetic resonance imaging, and cone-beam computed tomography (CBCT) can be used to observe the maxillary sinus. Computed tomography is accepted as the “gold standard” for maxillary sinus evaluation.89

Compared to computed tomography, CBCT has some advantages, such as a lower cost, smaller device size, and lower radiation dose. CBCT images have perfect tissue contrast and are not subject to blurring or overlapping of adjacent structures.10 In CBCT images, maxillary sinus opacification can be easily observed. Valuable information about paranasal sinus inflammation can be obtained without an additional radiation dose.11 Although CBCT is an important modality for maxillary sinus evaluation due to these advantages, panoramic radiography maintains its value as a routine diagnostic tool. Panoramic radiographs (PRs) are usually taken when patients visit dental clinics or hospitals for an initial examination. PRs can provide evidence of the main complaint and reveal incidental findings.1213 Therefore, PRs are still more popular than CBCT in general dentistry because of certain advantages, such as a lower radiation dose and cost.14 In addition, PRs are still frequently used in the diagnosis of maxillary sinus disorders. However, there are some situations that might particularly benefit from the application of artificial intelligence (AI). First, pain originating from the sinus and pain originating from pulpitis can be confused. It is known that increased pain when the head is tilted forward helps to distinguish sinusitis from tooth pulpitis.15 However, it is not easy to distinguish between these 2 situations. Second, in a radiological examination, there are some radiopaque landmarks, such as the floor of the palate and nasal cavity, or inferior concha, that overlap the maxillary sinus; therefore, the evaluation of maxillary sinus diseases can be difficult for less experienced dentists, and busy dentists may overlook incidental findings.16 As an AI tool, a convolutional neural network (CNN) was used in this study to address these problems. In dentistry, AI has been used to make the diagnostic process more accurate and efficient, which is of great importance in achieving the best treatment results and providing high-quality patient care. Dentists need to use all the knowledge they have acquired to diagnose and decide on the best treatment choice. They also need to predict the prognosis, which requires accurate clinical decision-making skills. However, in some cases, dentists do not have enough information to make the right clinical decision in a limited time. AI applications can guide them to make better decisions and perform better.17

Classification is a well-studied problem in the deep learning domain, the aim of which is to classify an input with a predefined label. The diagnosis of maxillary sinusitis in PR and CBCT images is a classification problem since the aim is to classify an input image as healthy or unhealthy. CNN is a deep-learning technique that is widely used in the image classification domain.18 A CNN consists of various components: convolution layers, pooling layers, and a classifier layer. A CNN creates abstractions from input images to extract features. The extracted features are then utilized to carry out the classification. Several parameters affect how the CNN carries out abstraction and classification. The training process optimizes these parameters to increase the harmony between the predicted and ground values through the application of back-propagation.1920

The aim of the present study was to assess the performance of AI using a CNN to diagnose maxillary sinusitis in both PR and CBCT images.

Methods and Materials

CBCT images were evaluated retrospectively to diagnose patients with healthy maxillary sinuses or inflamed maxillary sinusitis. PRs of those patients, which were taken at a date close to that of the CBCT images, were also used. In this study, 200×200 regions of interest (ROIs) that would include unilateral maxillary sinusitis were utilized in the PR and CBCT images. The final images comprised the PR and CBCT datasets for the CNN model, and each dataset consisted of 148 healthy and 148 inflamed sinus images.

Patient selection

This study protocol was approved by the Ethics Committee of Trakya University Faculty of Medicine (decision date: 20.01.2020; number: 02/27) and was in accordance with the principles of the Helsinki Declaration.

No radiographs were taken for the purpose of this study. PRs and CBCT images previously taken for any reason (e.g., impacted teeth or jaw cysts and tumors) were included in the study and evaluated retrospectively. Images from patients who had both PRs and CBCT scans were included in this study. Images that contained tumoral and fibro-osseous lesions and carcinomas of the maxillary sinus were excluded from this study. Maxillary sinusitis was diagnosed based on the radiological findings from the CBCT images. Both PRs and CBCT images of the same patients were used to assess the diagnostic performance of the CNN application. The PRs were taken before the CBCT examinations in all individuals. The approximate time interval between the examinations was 1 month.

Each maxillary sinus was examined for the presence of mucosal thickening, mucous retention cysts, and fluid in the maxillary sinus. If these conditions were found, a diagnosis of maxillary sinusitis was made.21222324 In the CBCT images, the maxillary sinus was defined as healthy when less than 4 mm of mucosal thickening was seen covering the entire area of the maxillary sinus.21 Each maxillary sinus was evaluated with the same criteria. Multiplanar reformation images (axial, coronal, and sagittal planes) were evaluated. Patients with maxillary sinusitis were selected as the disease group, while patients without inflammation in the maxillary sinus were selected as the healthy group. In the images diagnosed with sinusitis, the disease was classified as right or left.

All PRs were taken with a PaX-I (Vatech, Hwaseong, Korea) with 70 kVp and 10 mA (5.2 lp/mm resolution). CBCT images were taken with Pax-i 3D (75 kVp, 10 mA, 12×9 cm field of view, 0.2 mm voxels, 24 s exposure time) (Vatech, Hwaseong, Korea) in accordance with the maxillofacial radiography format. PRs and CBCT images were obtained from the imaging software. All evaluations were performed on a 15.6-inch full HD notebook with a resolution of 1,920×1,080 pixels. A single observer with 6 years of experience since completing their specialization in dentomaxillofacial radiology evaluated the CBCT images and determined the presence of inflamed sinuses; the PRs of those patients were used in the study. First, the CBCT images were randomly evaluated on the monitor. The observer evaluated the presence or absence of sinusitis without using the help of the CNN. Second, the PRs and CBCT images were evaluated independently by the author with the CNN (C.Y).

Creation of a dataset for deep learning

The original size of the axial CBCT sections was 850×850 pixels, and that of the PRs was 2,508×1,218. Each image was cropped to a region of interest (ROI) measuring 200×200 pixels that contained unilateral maxillary sinusitis (Fig. 1). The cropping size was guided by the study of Murata et al.21 to exclude non-antrum structures.

Fig. 1

An example of a region of interest measuring 200×200 pixels in a panoramic radiograph and cone-beam computed tomographic image.

While the ROI selection was made to include the lesion in the inflamed sinuses, it was constructed not to include anatomical bone structures with healthy sinuses.

Based on Murata et al.,21 the diagnostic performance for setting a relevant ROI on a single side of the maxillary sinuses was higher than setting an ROI on both sides of the maxillary sinuses. Thus, the PRs and CBCT scans were cropped on 1 side of the maxillary sinuses. Examples of cropped images from the disease and healthy groups are shown in Figures 2 and 3.

Fig. 2

Cropped panoramic radiographs from the test set. A. Healthy sinus with a radiolucent cavity. B. Inflamed sinus with increased opacity.

Fig. 3

Cropped cone-beam computed tomographic images from the test set. A. Healthy sinus with a radiolucent cavity. B. Inflamed sinus with increased opacity.

Deep learning and classification

All the deep-learning processes were carried out on a device using the Ubuntu 18.04.3OS with 8 GB of RAM, an Intel Core i7 CPU, and an NVIDIA GeForce GTX 970. A CNN was used as a deep-learning method and applied to CBCT and PR datasets. The CNN model was implemented using the PyTorch library of the Python programming language.

The training and test operations were carried out with k-fold cross-validation. In k-fold cross-validation, the dataset is shuffled randomly and split into k groups. A group is then selected as the test set, and the data in the remaining groups are used as the training set. A model is trained and tested with the selected training and test set. This operation is repeated until all groups are selected as the test set one by one.

In this study, the CBCT and PR datasets contained 296 images, of which 148 were healthy and 148 were inflamed sinuses. In the k-fold cross-validation experiments, for each dataset 5 groups were created by using a k-value of 5. The groups had 59, 59, 59, 59, and 60 images, respectively. Each group was used as the test set once, and the remaining data were used as the training set. For each test set and training set pair, a separate CNN model was trained and tested.

In the training phase, data augmentation was applied only to the training set to increase the amount of data. Data augmentation was carried out by using the Torchvision library of the Python programming language. Three different augmentations were used: auto contrast, sharpness adjusting, and horizontal flip. Auto contrast maximized the contrast of the original image by remapping its pixels per channel so that the lowest became black and the lightest became white. Secondly, the sharpness of the original images was increased by a factor of 3. Finally, the original images are flipped horizontally. After data augmentation was applied to all images, the training set contained the original images, the auto-contrasted images, the sharpened images, and the horizontally flipped images. Then, the CNN model was trained with the final form of the training set.

The CNN model used in this study had 2 convolutional layers and 3 fully connected layers. Each convolutional layer contained 32 filters with a kernel size of 3, a stride of 2, and a padding of 1. After each convolutional layer, there was a rectified linear unit (ReLU) activation layer and a max-pooling layer with a stride of 2.

The flattened output size of a final convolutional layer is 4608, and the output size of the fully connected layers was 1152, 100, and 2, respectively. In addition, 2 dropout layers (P=0.5) were used to prevent overfitting: one before the first fully connected layer and the other after it. Finally, the LogSoftmax classifier was used for the classification. The adaptive moment estimation (Adam) optimizer was used with a learning rate of 0.001 in the training phase. As the loss function, the cross-entropy loss was used. Because the size of the dataset was limited, no validation data were used. Instead, the number of epochs was used as the stopping criterion, and it was set experimentally at 300. A schematic diagram of the CNN model is shown in Figure 4. The ReLU layer is not shown in the diagram, but it was applied after each convolutional and fully connected layer.

Fig. 4

A schematic diagram shows the convolutional neural network model.

The accuracy, sensitivity, specificity, positive predictive value, and negative predictive value were obtained from the experiments.

Statistical analysis

The statistical software MedCalc Version 19.8 (MedCalc Software, Mariakerke, Belgium) was used for the analysis. A receiver operating characteristic (ROC) curve was plotted, and the area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value, and negative predictive value were calculated. The probability values calculated by the AI were used as operational points in the drawing of the ROC curve. The difference between ROC curves was evaluated with the DeLong test, and a P-value less than 0.05 was accepted as significant.

Results

The diagnostic performance parameters of the deep-learning system are shown in Table 1. When the performance of the deep-learning system in diagnosing sinusitis from PRs was evaluated, the AUC values ranged between 0.679 and 0.809 between folds, with an average of 0.755. The average accuracy, sensitivity and specificity of the deep-learning system in diagnosing sinusitis through PRs were 75.7% (range between folds: 72.88%-79.96%), 75.7% (range between folds: 69.23%-86.21%), and 75.7% (range between folds: 60.00%-84.85%), respectively. The average results indicated in Table 1 were obtained by merging the test data predictions for each fold.

Table 1

Diagnostic performance parameters of the deep-learning system with cross-validation

PR: panoramic radiography, CBCT: cone-beam computed tomography, AUC: area under the curve, PPV: positive predictive value, NPV: negative predictive value

When the performance of the deep-learning system in diagnosing sinusitis from CBCT images was evaluated, the AUC values ranged between 0.990 and 1.000 between folds, with an average of 0.997. The average accuracy, sensitivity, and specificity of the deep-learning system in diagnosing sinusitis through CBCT images were 99.7% (range between folds: 98.33%-100%), 100%, and 99.3% (range between folds: 96.67%-100%), respectively. The AUC values of CBCT were found to be significantly higher than those of PR (P<0.05). The accuracy values for each epoch are shown in Figure 5 for the test and training sets of PR and CBCT. In that figure, the accuracy for a specific epoch corresponds to the average accuracy values of all folds in that epoch.

Fig. 5

Accuracy values of training and test results for cone-beam computed tomography and panoramic radiograph data. PR: panoramic radiograph, CBCT: cone-beam computed tomography.

The ROC curves were generated by including all 5-fold test data. The ROC curves of the sinusitis assessment data from the deep-learning system comparing PRs and CBCT images are shown in Figure 6.

Fig. 6

Receiver operating characteristic curves of cone-beam computed tomography (CBCT) and panoramic radiography (PR).

Discussion

In this study, the effectiveness of the CNN method was investigated using 2 imaging systems. The CBCT evaluation was used as the gold standard. Therefore, the presence of sinusitis was diagnosed with CBCT, and its effectiveness with PRs was also evaluated. A CNN was used to create an AI maxillary sinusitis diagnostic tool. This method was chosen because CBCT scans and PRs are images, and CNNs are capable of classifying images.

The CNN method showed almost perfect performance with CBCT images and moderately high performance with PRs. These findings show that a deep-learning network can be used with CBCT images and has incontrovertible potential for the use of PRs to support the diagnosis of maxillary sinusitis.

As shown in Figure 5, the accuracy of the CNN for CBCT data was similar for the test and training sets. However, for PRs, although the accuracy of the training set was high, the CNN could not achieve the same success in the test set. An accuracy difference between the training and test sets generally occurs due to overfitting in the model. Although dropout layers and data augmentation were used to prevent overfitting, the accuracy difference between the test and training set could not be decreased further. Despite this difference in accuracy, obtaining a relatively high accuracy rate in the test set is a promising result. It would also be possible to obtain better results by using a larger amount of data in the PR dataset.

Murata et al.21 studied a deep-learning network to diagnose maxillary sinusitis from PRs and reported accuracy, sensitivity, specificity, and AUC of 87.5%, 86.7%, 88.3%, and 0.875 respectively. All the values in the current study were lower than in the study by Murata et al.21 This may be due to the larger dataset used in the study of Murata et al.21

Kim et al.25 evaluated the diagnostic performance of a deep-learning network to diagnose maxillary sinusitis on Waters’ view radiographs. With an external geographic test set, it was reported that the sensitivity and specificity of the deep-learning network in diagnosing maxillary sinusitis were 56.3% and 99.2%, respectively. With an external temporal set, the sensitivity and specificity of the deep-learning network were 76.9% and 94.2%, respectively. For the external temporal and geographic test sets, the AUCs of the deep-learning network were 0.93 and 0.88 respectively. In the PR evaluation of this study, the AUC values ranged between 0.679 and 0.809 between folds, with an average of 0.755. The average accuracy, sensitivity, and specificity of the deep-learning system in diagnosing sinusitis through PRs were 75.7%. Ohashi et al.24 studied computer-aided detection systems that used PRs to assist in the diagnosis of maxillary sinusitis with inexperienced dentists and experienced radiologists and reported the accuracy, sensitivity, and specificity as 73.5%, 77.6%, and 69.4%, respectively. The AUCs were 0.780 and 0.897 in the computer-aided detection of the inexperienced and experienced groups, respectively. In our study, the sensitivity was lower, the accuracy and specificity were higher. The AUC was lower in the current study than in the study by Ohashi et al.24

In the study by Kuwana et al.,16 using PRs, the data for healthy maxillary sinuses, inflamed maxillary sinuses, and cysts of the maxillary sinus area were determined for the training, test 1, and test 2 datasets. They reported the accuracy, sensitivity, and specificity in diagnosing maxillary sinusitis for the test 1 dataset (90%, 88%, and 91%, respectively) and test 2 dataset (91%, 85%, and 96%, respectively). All the values in the current study were lower than in the study by Kuwana et al.16 Comparing these studies shows that there have been different results, which may result from differences in the datasets, specific AI methods, and imaging modalities.

The current study had some limitations. First, imaging methods alone may not be sufficient to diagnose maxillary sinusitis because mucosal thickening can also occur in infections of the minor upper respiratory tract.262728 Second, the dataset was relatively small because it was difficult to find patients with both PR and CBCT images from roughly the same time period in accordance with the study requirements.

In conclusion, the performance of the deep-learning network in diagnosing maxillary sinusitis based on the PRs was moderately high, whereas it was clearly higher when using the CBCT images. This result is reasonable because 3-dimensional imaging techniques are considered to be more accurate than 2-dimensional ones. Therefore, with these positive results, deep-learning systems may be useful as a guide in diagnosis, especially for less experienced practitioners. However, to the best of our knowledge, studies using AI for the diagnosis of maxillary sinusitis are rare in the literature. In the diagnosis of maxillary sinusitis, further studies will be useful in evaluating and improving the performance of AI methods.