Estimation of Alveolar Bone Loss in Periodontitis Using Machine Learning.

AIM: The objective of this research was to perform a pilot study to develop an automatic analysis of periapical radiographs from patients with and without periodontitis for the percentage alveolar bone loss (ABL) on the approximal surfaces of teeth using a supervised machine learning model, that is, convolutional neural networks (CNN).
MATERIAL AND METHODS: A total of 1546 approximal sites from 54 participants on mandibular periapical radiographs were manually annotated (MA) for a training set (n = 1308 sites), a validation set (n = 98 sites), and a test set (n = 140 sites). The training and validation sets were used for the development of a CNN algorithm. The algorithm recognised the cemento-enamel junction, the most apical extent of the alveolar crest, the apex, and the surrounding alveolar bone.
RESULTS: For the total of 140 images in the test set, the CNN scored a mean of 23.1 ± 11.8 %ABL, whilst the corresponding value for MA was 27.8 ± 13.8 %ABL. The intraclass correlation (ICC) was 0.601 (P < .001), indicating moderate reliability. Further subanalyses for various tooth types and various bone loss patterns showed that ICCs remained significant, although the algorithm performed with excellent reliability for %ABL on nonmolar teeth (incisors, canines, premolars; ICC = 0.763).
CONCLUSIONS: A CNN trained algorithm on radiographic images showed a diagnostic performance with moderate to good reliability to detect and quantify %ABL in periapical radiographs.

Introduction

Periodontitis is a multifactorial disease, which in its severe form affects more than 10% of the adult population., Its primary symptoms include the loss of periodontal attachment and alveolar bone loss (ABL). ABL assessments on radiographic images are used in conjunction with soft tissue measurements. Despite the improvements brought by digital x-rays over the last decade, interpreting them is primarily and subjectively conducted by the dentist. This may result in misdiagnosis and, in the case of periodontitis, may lead to the wrong estimation of ABL.5, 6, 7 Moreover, in our current world of digital transformation, the potential application is that dental radiographs will be analysed through artificial intelligence (AI), quickly and without subjective interpretations. Obviously, the algorithms for such successful applications need development and testing and validation.

AI and machine learning (ML) are parts of computer science that are related to each other. Convolutional neural networks (CNN) specifically identify patterns, and this makes CNN attractive in many fields of science and also in biology and image analysis. Two recent studies have been published using different CNN algorithms demonstrating the applicability of image analysis on periapical dental radiographs. For example, one study reported on a CNN algorithm that was able to diagnose periodontal compromised teeth with similar accuracy compared to trained periodontists. Another study reported that the performance of a CNN algorithm on tooth detection and numbering was close to the level of a junior dentist. Interestingly, there are no studies reporting on the detection of reference points and the quantification of ABL and using ML on periapical x-rays. Nevertheless, one study published in 2017 reported the preliminary results on an automated system that can effectively estimate ABL in patients with periodontitis on periapical x-rays; the investigators used algorithms to detect the cemento-enamel junction (CEJ), the apex, and the most apical extension of the alveolar crest (AEAC); however, these algorithms were not using ML.

The purpose of the current pilot study was to develop and use a CNN to analyse periapical radiographs from patients with and without periodontitis and to assess %ABL on teeth.

Materials and methods

Study design

The study was approved by the Internal Review Board of the Academic Centre for Dentistry Amsterdam (ACTA) (#202055). The details of recruitment of patients with and without periodontitis from our dental institute and the anonymisation process are detailed in the Supplementary Material.

At the onset of our work, there were no previous studies available on %ABL estimation to determine the number of periapical radiographs needed for the construction of training, validation, and test data sets. We estimated that at least 500 radiographic images per jaw (each maxilla and mandible) would be sufficient to train the algorithm. We started with a total of 1144 periapical radiographs from 54 participants retrieved between June 2019 and December 2019 to construct training, validation, and test data sets (Figure 1). The primary aim of the study was to investigate the performance of the CNN output; thus, to prevent influence of data inconsistency, only radiographs from the mandible were selected. Therefore, 566 radiographs from the maxilla were discarded to reduce the complexity of the pilot study due to the projection of various anatomic features (construction and training of the algorithm). Finally, the training and the validation set consisted of 327 and 49 radiographic images, respectively, whilst the test set contained 70 images (Figure 1).

Fig. 1

Flowchart of recruitment and selection of patients with and without periodontitis. CNN, convolutional neural network.

The primary outcome was the radiographically detectable ABL in percentage of the root length on both the mesial and distal sites. For this purpose, first for each tooth, the following 5 points (single rooted) or 6 points (multi-rooted) were manually annotated using an online data-annotation tool: (i and ii) the mesial and distal CEJ, (iii and iv) the deepest point of each root apex (mesial and distal root for multi-rooted teeth), and (v and vi) the AEAC (see Figure 2). Using these reference points, it was now possible to calculate the %ABL for each mesial and distal site:

Fig. 2

Reference points illustration. The reference points have been enlarged for better illustration. CEJ, cementoenamel junction; AEAC, apical extension of the alveolar crest; APEX, apex/apices; Red, CEJ; Yellow, AEAC; Purple, APEX.

In Supplementary Figure 1, the manual annotation (MA) protocol is outlined.

ML annotation protocols/methods

Implementation details

For the machine-based determination of the ABL, per tooth, a 13-layered CNN has been implemented to estimate the reference points (CEJ, APEX, AEAC) on the mesial and distal sites of each tooth. The CNN consists of 13 convolutional layers with leaky rectified linear units (ReLU) as activation function, batch normalisation after each layer, and 4 MaxPooling layers (Supplementary Figure 2). As the loss function, the mean squared error (MSE) is used, as it puts more emphasis on the larger errors than the absolute error. The input to the network was an image of 128 × 128 pixels with corresponding coordinates to the annotated reference points, and the output of the network was a list of predicted reference point coordinates.

After MA of the different structures and reference points on periapical radiographs of the training set and the validation set, the radiographic image data were preprocessed to obtain the right input format into the CNN for training (see Supplementary Material, Supplementary Table 2, Supplementary Figure 4 for preprocessing, augmentation procedures, and performance).

Statistical analysis

Statistical analyses were carried out with SPSS software (v. 24.0, SPSS; IBM Statistics). Anthropometric and clinical characteristics of the participants from whom x-rays were selected are presented as means and standard deviations (SDs). Data analysis was conducted on the test data set that included 70 radiographic images. From each image, only one tooth was selected for two predicted measurements for each variable. Each measurement represented either the mesial or distal site of the selected tooth and was considered as an independent outcome. Therefore, the final test data set is based on 70 periapical radiographic images and contains a total of 140 predicted measurements for each variable. The remainder of the data/radiographs has been stored for future use and analysis.

For the images, the means (and SDs) for %ABL by MA and by CNN were calculated, and the mean differences (± SD) between these were also reported as means. The normality of these mean differences was checked by Bland-Altman plots. The reliability for the predicted %ABL per site compared to %ABL by MA was analysed using the average intraclass correlation coefficient (ICC). ICC values below 0.4 are indicative of poor reliability, values between 0.4 and 0.75 indicate moderate reliability, and values greater than 0.75 are considered to have excellent reliability. The %ABL is presented as the mean ± SD and the ICC with 95% confidence interval (CI). The ICC was assessed for the total test set and thereafter explorative subanalyses were performed based on tooth type, number of roots, and the presence of an angular defect. An angular defect was defined as a site with at least 3 mm of distance between the most apical and coronal extension of the alveolar crest using the tool provided by the Emago x-ray programme (Oral Diagnostic Systems; Figure 2) and at least 5 mm of probing pocket depth. To investigate the performance of the CNN model for periodontal classification purposes based on the amount of bone loss, we transformed the %ABL into bone loss index values (<33% ABL and ABL ≥33%). P values < .05 were considered statistically significant.

Results

Study population and radiographic distribution in the test data set

Supplementary Table 1 presents anthropometric and clinical characteristics of the participants from whom x-rays were retrieved. The majority of x-ray images were retrieved from patients with periodontitis (n = 51); further x-ray images from 3 participants without periodontitis were retrieved.

The test data set consisted of 49 radiographic images from patients with periodontitis and 21 images from participants without periodontitis (Figure 1), which were annotated by the first author (NT) and compared with the predictions made by CNN.

MA and CNN comparisons in the test data set

Table 1, part 1 presents first the results for the %ABL determined by MA and CNN for the complete test set (n = 140 approximal sites). The mean value for %ABL based on MA was 27.8% ± 13.4%, and the mean value for the CNN predicted %ABL was 23.1% ± 11.8%. We observed an ICC value of 0.601 (95% CI, 0.431-0.720) between the MA and the CNN; this was highly significant (P < .001) and indicated moderate reliability. The mean difference in %ABL between MA and CNN was 4.7% ± 10.7%. These mean differences were judged not to have a consistent bias towards one or the other method (Supplementary Figure 3, Bland-Altman plot).

Table 1

MA and CNN comparisons in the test database and in sites with an angular defect.

Part 1: Results for ABL determined by the MA and the CNN analysis for 70 teeth at the mesial and distal sites in the test set of radiographs and the differences between them (MA-CNN)
Tooth type	No. of sites	MA %ABL (SD)	CNN %ABL (SD)	Mean differences MA-CNN %ABL (SD)	ICC (95% CI)	P values
All teeth	140	27.8 (13.4)	23.1 (11.8)	4.7 (10.7)	0.601** (0.431-0.720)	<.001
NONMOLARS	106	25.7 (12.3)	22.3 (11.3)	3.3 (7.6)	0.763*** (0.619-0.848)	<.001
Incisors	46	27.1 (13.8)	24.2 (13.4)	2.9 (5.8)	0.889*** (0.769-0.943)	<.001
Canines	18	21.5 (6.0)	20.6 (6.3)	0.89 (4.8)	0.701** (0.365-0.876)	<.001
Premolars	42	25.9 (12.3)	21.0 (10.4)	4.9 (9.8)	0.581** (0.299-0.761)	<.001
MOLARS	34	34.2 (14.9)	25.7 (13)	8.5 (16.7)	0.245* (-0.053 to 0.519)	<.048

MA, manual annotation; CNN, convolutional neural network; ABL, alveolar bone loss; SD, standard deviation; ICC, intraclass correlation coefficient; CI, confidence interval.

ICC values <0.4 are indicative of poor reliability (*), values between 0.4 and 0.75 indicate moderate reliability (**), and values >0.75 are considered to have excellent reliability (***).

Table 1, part 1 presents also the results for the %ABL determined by MA and CNN for the various tooth types. In the test set, 106 nonmolar and 34 molar sites were present. The mean value for %ABL based on MA was 25.7% ± 12.3% for nonmolars and the corresponding mean value for the CNN %ABL was 22.3% ± 11.3%. We observed an ICC value of 0.763 (95% CI, 0.619-0.848) between the CNN and the MA; this was highly significant (P < .001) and indicated excellent reliability. The mean difference between MA and CNN was 3.3% ± 7.6%. For molars, the mean value for %ABL based on MA was 34.2% ± 14.9%, and the corresponding mean value for the CNN %ABL was 25.7% ± 13%. We observed an ICC value of 0.245 (95% CI, 0.053-0.519) between the CNN and the MA (P < .05) indicating poor reliability. The difference between MA and CNN was 8.5% ± 16.7%.

To further explore the performance of the CNN algorithm, we subdivided the nonmolar test set into incisors, canines, and premolars. The mean value for %ABL based on MA was 27.1% ± 13.8% for incisors, and the corresponding mean value for the CNN %ABL was 24.2% ± 13.4%. We observed an ICC value of 0.889 (95% CI, 0.769-0.943; P < .001; excellent reliability) between the CNN and the MA. The mean difference between MA and CNN was 2.9% ± 5.8%. For canines, the mean value for %ABL based on MA was 21.5% ± 6.0% and the corresponding mean value for the CNN %ABL was 20.6% ± 6.3%. We observed an ICC value of 0.701 (95% CI, 0.365-0.876; P < .001; moderate reliability) between the CNN and the MA. The mean difference between MA and CNN was 0.089% ± 4.8%. For premolars, the mean value for %ABL based on MA was 25.9% ± 12.3%, and the corresponding mean value for the CNN %ABL was 21.0% ± 10.4%. We observed an ICC value of 0.581 (95% CI, 0.299-0.761; P < .001; moderate reliability) between the CNN and the MA. The mean difference between MA and CNN was 4.9% ± 9.8%.

MA and CNN subanalysis in sites with an angular defect

Table 1, part 2 presents the results for the %ABL determined by MA and CNN for sites with and without an angular defect (n = 18 and n = 122 sites, respectively). For sites with an angular defect, the mean value for %ABL based on MA was 40.4% ± 17.1%, and the corresponding mean value for the CNN %ABL was 30.5% ± 15.7%. We observed a nonsignificant ICC of 0.041 (95% CI, –0.349 to 0.459; P > .05) between the CNN and the MA. The mean difference between MA and CNN was 10% ± 22.7%.

In contrast, for the majority of sites without an angular defect, the mean value for %ABL based on MA was 25.9% ± 11.8% and the corresponding mean value for the CNN %ABL was 22.1% ± 10.7%, with an ICC value of 0.742 (95% CI, 0.557-0.842; P < .001). The mean difference between MA and CNN was 3.8% ± 7.4% for the sites without an angular defect.

MA and CNN subanalysis for sites with <33% and ≥33% bone loss

To explore the ML applicability on the new classification, we analysed sites <33% and ≥33% Table 1. also presents the results for the %ABL determined by MA and CNN for sites with ABL <33% and ≥33% of the root length (n = 95 and n = 17 sites, respectively, based on MA). The mean value for %ABL based on MA was 20.5% ± 3.3% for sites with ABL <33% and the corresponding mean value for the CNN %ABL was 18.3% ± 3.8%, with an ICC value of 0.431 (95% CI, 0.399-0.737; P < .001). The mean difference between MA and CNN was 2.1% ± 6.7%.

The mean value for %ABL based on MA was 50.6% ± 8.3% for sites with ABL ≥33%, and the corresponding mean value for the CNN %ABL was 46.0% ± 11.6% (ICC value of 0.641; 95% CI, 0.227-0.855; P < .05). The mean difference between MA and CNN was 4.5% ± 7.9%.

Table 2 shows the analysis of sensitivity and specificity for “automatic” classification of patients using ML. The %ABL determined by CNN exhibited a high sensitivity (0.96) and a moderate specificity (0.41), compared with %ABL determined by MA. The accuracy of ML to correctly classify the patients was 80%.

Table 2

Sensitivity, specificity, and accuracy of the CNN relative to MA with <33% and ≥33% bone loss.

	MA
CNN	<33% ABL	≥33% ABL	Sensitivity	Specificity	Accuracy
<33% ABL	95	24	0.96	0.41	0.80
≥33% ABL	4	17

MA, manual annotation; CNN, convolutional neural network; ABL, alveolar bone loss

Discussion

In the present pilot study, periapical radiographs from 51 patients with periodontitis and 3 patients without periodontitis were first MA. These annotations were used to train a CNN algorithm to develop an automated tool for %ABL assessments in periapical radiographs. The rationale behind this study was to investigate whether CNN can be implemented and be used for assessment of %ABL on periapical images. Our current findings suggest the high potential of ML for quantification of %ABL on dental radiographs.

Although the ICC values were highly significant, ranging from moderate to excellent reliability for the automated estimation of %ABL on the mesial and distal surfaces of the teeth, our analysis revealed an overall slight underestimation (4.7%, SD = 10.7%) of the %ABL when using a CNN. However, taking into consideration that the average root length is 12 to 17 mm, it can be considered that the clinical significance of the average underestimation in specific subgroup of teeth (incisors, canines, premolars) is very limited. Nevertheless, the underestimation was substantial for molars (8.5%, SD = 16.7%) and for teeth with an angular defect (10%, SD = 22.7%).

There are several studies using AI in which different kinds of radiographic images such as cone beam computed tomography, computed tomography, intra-oral scan, and orthopantomography (OPG) have been used for ABL or caries detection with promising results.14, 15, 16, 17, 18, 19, 20 Specifically, in one study on OPG, the researchers found excellent ICC (0.91) in %ABL quantification and high accuracy in periodontitis classification between a CNN algorithm and trained dentists. Interestingly, the ICC was reduced for incisors and molars.

One limitation of our study is the number of available data. ML technologies require a large amount of data because it uses them to recognise patterns and to self-learn. In our study, we used 327 radiographs (654 sites) to construct the training set and we applied augmentation procedures. The total number of 654 radiographs (1308 sites) after augmentation procedures is considered limited to accurately train the algorithm (Figure 1). Another major limitation would be considered the single annotator (NT). There are not many studies comparing the interexaminer agreement of periodontal findings on x-rays. One study found that the ICC between senior dental students and a periodontist (gold standard) was 0.01 to 0.70. There was a considerable and significant difference between a general dentist and a dentist with a specialisation in periodontology. Additionally, the differences between the periodontists were not significant. To overcome as much as possible the limitation of a single annotator, we used the periodontal status as a reference to relatively compare and place the reference points. Another study reported the underestimation of bone level between radiographic examination (analog 2.7 mm; digital 2.5 mm) and clinical examination (probing pocket depth 1.8 mm; bone sounding 0.6 mm), when compared to the flap elevation (absolute truth). The clinical examination seems to perform better in bone level estimation. In that respect, we should consider our study proof of concept. The current results are promising, and now with these preliminary results we can design a larger study for improving the observed underestimation and decreased specificity, with multiple periodontists to score alveolar bone loss to improve performance for molars and include both maxillary and mandibular teeth. Also, we need to validate the system with other academic centres and clinicians in a multicentre study.

Additionally, limitations regarding the technical characteristics can be clustered into 3 different groups (Figure 3): (1) Radiographic characteristics: the angulation and magnification of the periapical image, resolution, differences in mesial and distal bone densities, phosphor plates or solid-state sensors, size, and different x-ray machine characteristics. To overcome this limitation, we used %ABL and not the absolute measures in mm. (2) Difficulties during annotation of APEX, CEJ, and AEAC: root proximity, tooth overprojection with the adjacent teeth, their roots or the bony structures, restorative material in proximity or extending beyond the CEJ, carious lesions, calculus and external root resorption on the CEJ, multiple roots, rotation, concavities, secondary caries under the restoration, the differences in bone height between the vestibular and lingual sites, and possible discrepancies due to potentially overlapping alveolar bone plates on the buccal and lingual aspects. (3) CNN algorithm: the input resolution, argued upon for its computational manageability, and the “manual” cropping of the image to merely 10% of the tooth and its surroundings, could have had a negative influence on the outcome. Part of the future work is also the automatic cropping of the images.

Fig. 3

CNN-MA illustrations from the test set in which the agreement was low. On the left side are the reference points predicted by the CNN and on the right side by MA. Note: A, the presence of a metallic crown on the adjacent tooth and the bone proximity with the crown margins are influencing the result. B, the presence of a metallic crown, the bone proximity, and the angulation of the teeth are influencing the results. C, the CEJ is more accurately marked by CNN than MA. D, the presence of an angular defect limits the agreement between CNN and MA. MA, manual annotation; CNN, convolutional neural network; CEJ, cemento-enamel junction.

In conclusion, a CNN trained algorithm on a limited amount of radiographic images with approximal sites first manually annotated showed comparable diagnostic performance with moderate to good reliability to detect and quantify %ABL. The application of CNNs seems very promising as a computer-aided detection for %ABL in clinical practice.

Conflict of interest

SP, FC, and DAM are employees of Promaton. Promaton is a company focussing on the development of AI for the dental industry.