Intra- and inter-observer agreements in detecting peri-implant bone defects between periapical radiography and cone beam computed tomography: A clinical study.

BACKGROUND/
PURPOSE: Information regarding agreements between periapical radiograph (PA) and cone beam computed tomography (CBCT) in detecting peri-implant defect is still scarce. The aim of this clinical study was to compare agreements between PA and CBCT in detecting peri-implant bone defect.
MATERIALS AND METHODS: This retrospective clinical study enrolled 32 patients with both PA and CBCT filmed right after implant placement. Four modalities were used for film reading: PA1 (original), PA2 (enhanced brightness/contrast), CBCT1 (selected axial and mesial-distal direction images) and CBCT2 (all data with software). 2 experienced and 2 inexperienced observers scored all films. Intra- and inter-observer agreements were estimated with Cohen's kappa coefficient. Categorized agreements were compared and differences among four modalities were calculated.
RESULTS: Agreements of PA were better than CBCT when detecting peri-implant bone defects in inter-observer agreements (median kappa 0.471 vs. 0.192; p = 0.016). Moreover, agreements in experienced observers were better than inexperienced observers (median kappa 0.883 vs. 0.567; p < 0.001). There was significant difference among four modalities except for experienced observer 2 (p = 0.218).
CONCLUSION: Agreements of PA are better than CBCT when detecting peri-implant bone defects, especially for inter-observer agreements. Experienced observers are more consistent in assessment than inexperienced ones.

Introduction

Radiological examination is crucial in pre- and post-operative assessment of implant surgery. Peri-implant bone defects may be found in implants with surgical trauma, peri-implantitis and osseointegration failure. Among all radiological methods, periapical radiography (PA) and cone beam computed tomography (CBCT) are common ways to assess post-operative peri-implant bone defects.

Periapical radiography is considered as an accurate and reliable tool to assess peri-implant bone status with favorable costs and radiation dose. However, the image of PA is only two-dimensional with superimposed structures. Since CBCT has been introduced to dentistry, it enables clinicians to obtain 3-dimensional images with a low dose of radiation. CBCT has been widely used in implant dentistry, including pre-implant assessment of anatomy and post-implant placement. One potential limitation of CBCT in evaluating post-operative peri-implant status is the presence of metal artifacts, which could mask osseointegration, shallow bony defects and other peri-implant radiolucencies, thus may jeopardize diagnosis.

There are many researches focusing on comparison between PA and CBCT at detecting peri-implant bone defects. One review article stated that there was no difference between PA and CBCT in detecting peri-implant bone. However, agreements between PA and CBCT in detecting peri-implant bone defect are inconsistent. Studies conducted by Dave et al. and Vidor et al. proved that PA was a reliable tool of detecting peri-implant bone defects and performed significantly better than CBCT in agreements., However, other studies showed that agreements of CBCT were better than of PA when assessing bone defect around implants.6, 7, 8 To date, information regarding agreements between PA and CBCT in detecting peri-implant defect is still scarce.

Moreover, most of the previous studies were designed in vitro. Little is known about defect detecting agreement between PA and CBCT in clinical situation. Furthermore, some issues still require an appropriate discussion: different PA film brightness/contrast, different CBCT reading methods and influence of expertise of observer.

Therefore, the aim of the present study was to compare agreements between PA and CBCT in bone defect assessment around dental implants in clinical situation. Our hypothesis was that agreements for PA and CBCT would be comparable.

Materials and methods

The present study was approved by the Research Ethics Committee for clinical study, Shanghai Jiao Tong University, under protocol No. 2016-236-C33. A total 44 patients were recruited and all participants have signed informed consent. This clinical study adheres to the principles described by the Declaration of Helsinki that revised in 2013 for research involving human participants. All methods performed were in accordance with the relevant guidelines and regulations. After consultation, individuals who met the following inclusion criteria were asked to participate in the study:

≥18 years of age

no systematic or local conditions presenting a contraindication to implant placement

requirement of single implant placement with guided bone regeneration

no pathologic conditions of teeth adjacent to the area of surgical procedure

periodontium in the area of the surgical procedure to be free of pathology and artifacts such as pins and wires

The following exclusion criteria were also applied:

presence of complex systematic disease or bone metabolic disorder

history of malignancy, radiotherapy, or chemotherapy in the past 5 years

smoking > 10 cigarettes

pregnancy or lactating woman

no attempt to sign consent form

Surgical procedure

A total of 34 patients were enrolled in this study and all informed consents were obtained prior to surgery. All patients received single implant placement with guided bone regeneration. A two-piece, tapered implant system was used (ETGAR medical implant systems, Nahariya, Israel). Bovine bone matrix composed of hydroxyapatite collagen and collagen membrane (Zhenghai Biotechnology, Shandong, China) were used for guided bone regeneration. CBCT and PA were both filmed right after implant placement.

Image acquisition

The periapical films were taken using a size 31 mm × 41 mm digital photostimulable phosphor plate (PSP) (Digora Optime, Soredex, Tuusula, Finland) with a standard intraoral X-ray machine (Heliodent DS, Sirona, Bensheim, Germany) with exposure settings of 70 kV, 7 mA for 0.08 s.

CBCT was undertaken using i-CAT 17–19 (Imaging Sciences International, Hatfield, PA, USA). The exposure settings selected were: 360°scan, 120 kV, 5 mA and acquisition time 14.7 s. The scanning parameters were FOV height∗width 8 cm × 8 cm and voxel size 0.25 mm.

Image preparation

In total, all PA and CBCT of 34 patients were acquired. After carefully screening, 2 CBCT images with motion artifacts were excluded. Finally, 32 patients were enrolled in this study. No prosthetic crowns or root canal filling materials were seen around implants.

Two researchers, who didn't involve in image analysis, were in charge of preparing all films. For PA, two files were created. The first file, named PA1, contained the original digital films. The second file, named PA2, was films adjusted with increasing brightness/contrast subjectively by agreement between those two researchers. For CBCT, two reading methods were used. The first method was to create one file, named CBCT1, with two images selected from software per patient. An axial image (around apical third of implants) and a mesial-distal direction image (taken nearly through the implant center in the axial views) were selected subjectively by agreement between those two researchers. Thus, there were two images per patient, which were considered to be the most representative of this patient, for observers to evaluate peri-implant bone status. These images were chosen so all observers could carry out evaluation on identical images. The second method CBCT2 was to scroll through all data by using software, and to change the contrast and brightness of each patient when needed. Observers were allowed to perform any section in multi-planar reconstructions (MPR) to evaluate peri-implant bone status. The order of the images in the above four files was scrambled and anonymization of all images was performed. Instructions for reading PA and CBCT were also prepared by the two researchers.

Image evaluation

PowerPoint instructions, files of PA films and CBCT were prepared and stored in identical device. Observers were asked to review the radiographs on the same electronic device under quiet, darkened environmental condition. The sequence of periapical films and CBCT images was randomized using Microsoft Excel (Microsoft Excel for Mac 2011, Microscope Corp, Seattle, WA, USA). The viewing and adjusting software for images was Microsoft PowerPoint (Microsoft PowerPoint for Mac 2011, Microscope Corp) and for CBCT discs was i-CAT Vision (Imaging Sciences International).

Four observers consisting of two dentists with more than 5 years’ experience in implant imaging (Observer 1 and Observer 2) and two postgraduate students majored in implant dentistry (Observer 3 and Observer 4) were asked to evaluate all radiographs. All observers were required to repeat the entire evaluation 2 weeks later to produce data for intra-observer reliability. The image order changed for the second evaluation.

A 5-point scale was used, with the following classification:

Peri-implant bone defect definitely not present

Peri-implant bone defect probably not present

Unsure if peri-implant bone defect present or not

Peri-implant bone defect probably present

Peri-implant bone defect definitely present.

For every patient, both mesial and distal bone status of implant were evaluated. All data were transferred to Excel sheets (Microsoft Excel for Mac 2011, Microscope Corp) for analysis.

Statistical analysis

The data were analyzed using Statistical Analysis System, SAS version 9.4 (SAS Institute, Cary, NC, USA). All acquired scores from all observers were summarized. Then, score 1,2,3 were pooled as “0”, indicating absence of bone defect, while score 4,5 were pooled as result “1”, indicating presence of bone defect. Pooled results for each observer were analyzed for intra-observer and inter-observer agreement. Agreements were analyzed by the pooled results of mesial and distal score, respectively. For example, if mesial and distal scores of a patient were 4 and 3. Then agreement was analyzed by the pooled result of “1” and “0”. Cohen's Kappa analysis used for all comparisons between observers was seen in Table 1.

Table 1

Cohen's Kappa variables used for comparisons between observers.

Cohen's kappa for intra- and inter-observer agreements were calculated for the following comparisons
Intra-observer agreement for all observers	PA1-1 versus PA1-2	The first and second assessments with original periapical film
	PA2-1 versus PA2-2	The first and second assessments with increased brightness/contrast of periapical film
	CBCT 1-1 versus CBCT 1-2	The first and second assessments with CBCT selected images
	CBCT 2–1 versus CBCT 2-2	The first and second assessments with CBCT all data
Inter-observer agreement for averaged PA1, PA2, CBCT1, CBCT2	O1 versus O2 (PA1)O3 versus O4 (PA1)	Experienced and inexperienced observer results for original periapical film
	O1 versus O2 (PA2)O3 versus O4 (PA2)	Experienced and inexperienced observer results for periapical film with increased brightness/contrast
	O1 versus O2 (CBCT1)O3 versus O4 (CBCT1)	Experienced and inexperienced observer results for CBCT with selected images
	O1 versus O2 (CBCT2)O3 versus O4 (CBCT2)	Experienced and inexperienced observer results for CBCT with all data

O, Observer; PA1, original periapical radiography; PA2, enhanced brightness/contrast periapical radiography; CBCT1, selected axial and mesial-distal direction images; CBCT2, all data with software.

After all kappa results acquired, sorted value were categorized. Wilcoxon signed rank test (for paired data) was used to compare differences between intra-observer PA and CBCT, inter-observer PA and CBCT, experienced and inexperienced observers, PA1 and PA2, as well as CBCT1 and CBCT2. In this study, any kappa values under 0.4 were regarded as poor agreement, between 0.40 and 0.59 were regarded as moderate agreement, any over 0.6 were considered as good agreement and any over 0.8 as excellent agreement (Landis & Koch 1977).

Averaged results of two assessments in each patient were used for comparison among different imaging modalities. Cochran's Q-test with a post hoc McNemar test and Benjamini and Hochberg correction were used to test for statistically significant differences between the four imaging modalities. Then, all averaged results from all 4 observers with all 4 imaging modalities for each patient added up. For example, if all results were “1”, the highest score one could get was “16”. Then, total score of each patient was ranked with proportion of scores from CBCT calculated. Spearman correlation between final score and proportion of CBCT was analyzed afterwards.

P value smaller than 0.05 was regarded as statistically significant. As the McNemar test involved four comparisons, a correction for multiple tests was carried out and statistical significance was accepted at p < 0.01.

Results

All four observers completed all radiographic assessments. Ages of 32 patients ranged from 21 to 60, with 14 females and 18 males. 1 anterior tooth and 31 posterior teeth were replaced by implants. Four film-reading modalities were used with twice assessment both at mesial and distal aspects. Thus, each observer had 512 radiographic assessments.

Initial results were outlined in Table 2. Observer 1 and 2 were experienced observers, while observer 3 and 4 were graduate students. The results showed variation among different observers. Obviously, the number of score “3” in experienced observers was much less than in graduate students. After pooling, numbers of “1” were least in observer 1 and most in observer 3. Moreover, in experienced observers, the number of “1” was more in PA than in CBCT, while in graduate students, the number of result “1” was more in CBCT.

Table 2

Summary of initial score answers for each observer.

Observer	Methods	Scores
		1	2	3	4	5
1	PA1	0	63	30	14	21
	PA2	0	66	26	19	17
	CBCT1	0	74	31	7	16
	CBCT2	0	80	25	3	20
2	PA1	2	38	29	31	28
	PA2	2	40	18	39	29
	CBCT1	16	45	13	41	24
	CBCT2	6	40	23	35	25
3	PA1	25	35	0	35	33
	PA2	37	38	0	29	24
	CBCT1	15	30	7	42	34
	CBCT2	3	29	11	42	43
4	PA1	31	46	3	28	20
	PA2	46	48	3	18	13
	CBCT1	33	47	5	22	21
	CBCT2	30	30	3	23	42

PA1, original periapical radiography; PA2, enhanced brightness/contrast periapical radiography; CBCT1, selected axial and mesial-distal direction images; CBCT2, all data with software.

The kappa values for intra- and inter-observer agreements were summarized in Table 3. For experienced observers, intra-observer agreements of PA and CBCT at both levels were mostly “excellent” (range 0.795–1.000). In general, kappa values for PA2 and CBCT2 were slightly better than PA1 and CBCT1, respectively. However, agreements for observer 3 and 4 were mostly “moderate to good” (range 0.410–1.000). Meanwhile, kappa values for PA2 were not always better than PA1, while agreements of CBCT2 were better than CBCT1.

Table 3

Intra- and inter-observer agreements for each observer (95% confidence interval).

Observer	Methods	Level1-mesial	Level1-distal
1	PA1	0.932 (0.802–1.000)	0.795 (0.522–1.000)
	PA2	0.932 (0.802–1.000)	0.904 (0.719–1.000)
	CBCT1	0.796 (0.528–1.000)	0.890 (0.680–1.000)
	CBCT2	0.904 (0.719–1.000)	1.000 (1.000–1.000)
2	PA1	0.937 (0.816–1.000)	0.867 (0.691–1.000)
	PA2	1.000 (1.000–1.000)	1.000 (1.000–1.000)
	CBCT1	0.875 (0.709–1.000)	0.938 (0.817–1.000)
	CBCT2	0.937 (0.816–1.000)	0.934 (0.808–1.000)
3	PA1	0.629 (0.370–0.889)	0.629 (0.370–0.889)
	PA2	0.431 (0.190–0.671)	0.529 (0.259–0.801)
	CBCT1	0.410 (0.087–0.732)	0.552 (0.260–0.844)
	CBCT2	0.503 (0.161–0.844)	0.688 (0.449–0.927)
4	PA1	0.563 (0.290–0.835)	0.649 (0.374–0.924)
	PA2	0.600 (0.286–0.915)	1.000 (1.000–1.000)
	CBCT1	0.570 (0.242–0.898)	0.625 (0.363–0.887)
	CBCT2	0.688 (0.436–0.939)	0.751 (0.524–0.978)
O1 versus O2	PA1	0.459 (0.186–0.732)	0.363 (0.072–0.664)
	PA2	0.582 (0.334–0.831)	0.482 (0.216–0.748)
	CBCT1	0.073 (−0.171–0.316)	0.125 (0.143–0.393)
	CBCT2	0.358 (0.122–0.593)	0.321 (0.011–0.630)
O3 versus O4	PA1	0.625 (0.374–0.876)	0.290 (−0.010–0.591)
	PA2	0.458 (0.184–0.731)	0.739 (0.462–1.000)
	CBCT1	0.189 (0.034–0.412)	0.343 (0.055–0.632)
	CBCT2	−0.030 (−0.321–0.261)	0.194 (0.139–0.527)

O, Observer; PA1, original periapical radiography; PA2, enhanced brightness/contrast periapical radiography; CBCT1, selected axial and mesial-distal direction images; CBCT2, all data with software.

For experienced observers, inter-observer agreements at both levels were mostly “poor to moderate”, with poor for CBCT and moderate for PA. Kappa values for PA2 and CBCT2 were slightly better than PA1 and CBCT1, respectively. However, inter-observer agreements for observer 3 and 4 were not as uniform as for observer 1 and 2. One inter-observer values for observer 3 and 4 were noticed, for result was below poor (mesial CBCT2 -0.0303). All intra- and inter-agreements were summarized in Fig. 1.

Figure 1

Intra- and inter-observer agreements of each observer for PA1 (original periapical radiography), PA2 (enhanced brightness/contrast periapical radiography), CBCT1 (selected axial and mesial-distal direction images) and CBCT2 (all data with software).

Results from Wilcoxon signed rank test was shown in Table 4. Significant differences were seen in inter-observer PA/CBCT and experienced/inexperienced observers. Results of Cochran's test were summarized in Table 5. It showed that there was significant difference among four modalities except for observer 2. Significant difference was seen in observer 1, observer 3 and observer 4. However, greater Q values were only seen in observer 3 and observer 4. Moreover, post-hoc multiple tests showed that PA2CBCT1 and PA2CBCT2 of observer 3 and PA2CBCT2 of observer 4 were the most prominent difference.

Table 4

Wilcoxon paired signed rank test for comparison between different categories.

Category	Median1	Median2	p value
Intra-observer PA versus CBCT	0.831 (PA)	0.774 (CBCT)	0.821
Inter-observer PA versus CBCT	0.471(PA)	0.192 (CBCT)	0.016∗
Experienced observers versus inexperienced observers	0.883 (O1, O2)	0.567 (O3, O4)	<0.001∗
PA1 versus PA2	0.629 (PA1)	0.670 (PA2)	0.278
CBCT1 versus CBCT2	0.561 (CBCT1)	0.688 (CBCT2)	0.176

PA, periapical radiography; CBCT, cone beam computed tomography; PA1, original periapical radiography; PA2, enhanced brightness/contrast periapical radiography; CBCT1, selected axial and mesial-distal direction images; CBCT2, all data with software.

∗p < 0.05.

Table 5

Cochran's Q-test with post hoc McNemar test results.

Observer	Cochran's Q-test		Post hoc McNemar test (p value)
	Q value	p value	PA1PA2	PA1CBCT1	PA1CBCT2	PA2CBCT1	PA2CBCT2	CBCT1CBCT2
1	8.032	0.045∗	0.317	0.157	0.046	0.096	0.025	1.000
2	4.435	0.218	0.564	0.167	1.000	0.285	0.564	0.166
3	16.948	0.001∗	0.034	0.052	0.083	0.005∗∗	0.001∗∗	0.739
4	15.737	0.001∗	0.180	0.257	0.018	0.034	0.002∗∗	0.090

PA1, original periapical radiography; PA2, enhanced brightness/contrast periapical radiography; CBCT1, selected axial and mesial-distal direction images; CBCT2, all data with software.

∗p < 0.05; ∗∗p < 0.01.

The Spearman's correlation was −0.74 (Fig. 2). With total score of each patient decreasing, proportion of scores from CBCT was increasing. Moreover, there were 3 patients whose results were all “1” with total score “16”. The lowest final score was “2” in two patients, which were all contributed from CBCT. Images of these patients were seen in Fig. 3. Three patients with peri-implant bone defect identified by all 4 observers were on the left, while two patients with lowest scores were on the right.

Figure 2

Spearman correlation between final score of each patient and proportion of CBCT.

Figure 3

A, B, and C were the three patients with peri-implant bone defect identified by all 4 observers (red arrows point to the defects). D and E were the two patients with lowest scores (yellow arrows point to the artifacts with decreased gray value, while blue arrows point to the artifacts with increased gray value). Notice the upper incisor in D, artifacts with increased gray value were located at mesial and distal direction, while artifacts with decreased gray value was located at mesial-lingual and distal-lingual direction. Image sequence was PA1 (original periapical radiography), PA2 (enhanced brightness/contrast periapical radiography), CBCT axil view and CBCT mesial-distal view (from left to right).

Discussion

Agreements in detecting peri-implant bone defects of PA were better than CBCT, especially for inter-observer agreements. Thus, our postulated hypothesis, that was, agreements for PA and CBCT would be comparable, was rejected. As we know, this is the first clinical study assessing agreements between PA and CBCT and PA showed a more favorable diagnostic consistency than CBCT in detecting peri-implant bone defects. Clinical variation in diagnosing defects around dental implants should be noted in clinical practice.

In fact, there were only a few in vitro studies comparing agreements between PA and CBCT and results were inconsistent. For example, Schwindling et al. and Hilgenfeld et al. created at least 1 mm diameter 1 wall to 4 walls peri-implant bone defects and results showed that agreements of CBCT were slightly better than PA., With unified and regular defect, particular settings, no soft coverage or other surrounding structures, the results should be interpreted with caution. On the other hand, Vidor et al. found that the agreements of PA were slightly better than CBCT, especially for inter-observer assessment with circumferential 0.125 mm peri-implant gap. Meanwhile, the study with 0.35 mm and 0.675 mm circumferential defects conducted by Dave et al. stated that PA showed significantly better intra-observer and inter-observer agreements than CBCT. When peri-implant bone defects were small, artifact presence in CBCT would shallow bony defects, which could result in agreements of PA better than CBCT. Moreover, Pinheiro et al. reported that when peri-implant defects were chemically created, agreements of CBCT were better than PA. In our study, different sizes of bone defects might be overdrilled. It is indicated that overdrilled defects show a dark band similar to that produced by metal artifacts of CBCT, which could go easily unnoticed compared to chemically created irregular defects. Thus, in this respect, better agreements in PA could be expected in our clinical study.

Periapical films have been widely used in daily clinical practice and recommended as initial method for evaluating peri-implant bone status., Although generally claimed as a standard method, diagnostic accuracy of PA is still under debate and information in agreements is still limited., For example, a recent study suggested that marginal bone level changes in PA below 1 mm were likely due to human variation. Additionally, agreements within and between observers when assessing periapical radiographs were mostly 0.40–0.59, which was in agreement with our study. Moreover, brighter radiographs improved intra-observer agreement in determining peri-implant marginal bone level changes. However, in our study, adjusted brightness/contrast radiographs brought little improvement in agreements. In fact, in our study, bone defects were mostly around apical portion of implants. As we all know, implant apical portion of periapical film is influenced by superimposed structures, variation in the trabecular pattern and defect's size, shape and density and the complexity of the surrounding normal anatomic features. Thus, assessment of apical portion bone of implants in PA is more complex than peri-implant marginal bone, which might explain the little improvement after film brightness/contrast adjustment.

It is somewhat surprising that the agreements of CBCT were poorer than PA, especially for inter-observer agreements. As many studies have proved, the diagnostic accuracy of CBCT was good.,,15, 16, 17, 18 For example, Golubovic et al. compared sagittal views of CBCT to histologic standard when evaluating advanced peri-implantitis defects and results showed CBCT could be regarded as an accurate diagnostic tool. As a matter of fact, sagittal image could be least affected by artifacts with reduced gray value in peri-implant bone assessment. Another study proved that CBCT images correlated highly with physical measurements. Meanwhile, the defect size in the their study was at least 1 mm and multiple sectional images were combined to diagnose defects. As pointed in a recent review, larger defect sizes exhibited a trend for better defect detection by CBCT. On the other hand, Ritter et al. reported that PA and CBCT performed similar in assessing mesial and distal peri-implant marginal bone. Meanwhile, other in vitro studies have stated that assessment of interproximal peri-implant defect width at implants was more accurate in PA in comparison to CBCT.,, Smaller defect sizes were seen in those studies. Furthermore, in Steiger-Ronay's study, with only mesio-distal direction images of CBCT evaluated, PA was more accurate than CBCT. Thus, variation in design of previous studies might explain the inconsistent diagnostic accuracy of CBCT.

As we all know, mesio-distal direction image of CBCT with implant artifacts is always present in the proximity of titanium implants with altered gray value, which could affect judgement prominently. A pattern for the distribution of artifacts around titanium implants in CBCT was detected. Regions with reduced gray values were located mesially and distally at molar, premolar and canine sites and at the mesio-buccal, disto-buccal, mesio-lingual and disto-lingual aspects of incisor sites. On the contrary, regions with increased gray value were mostly located buccally and lingually, which would least affect peri-implant bone assessment. In our study, we were focusing on mesial-distal direction defect, which would be strongly affected by artifacts. Moreover, the presence of anatomical structures such as cranium, vertebral column, and soft tissue influence the gray value of the jawbones made the image in our study more complex to evaluate. Identification of artifact-affected regions is difficult, which could lead to false positive diagnosis of peri-implant bone defects. As the result of Spearman correlation showed, evaluations from CBCT were more likely to be considered as bone defect presence. Therefore, combined with inconsistent diagnostic accuracy and influence of artifacts, the poor agreements of CBCT could be explained.

Two methods were used in our study to assess peri-implant bone defect in CBCT. The first method was to assess only selected images, and the second method was to scroll all database of CBCT in each patient. Both methods were seen in previous studies, with the second method more frequently used. It is understandable that most studies have chosen the second method, which would allow observers to use any tools or any sectional images to view. In our study, we have compared the two methods. Our hypothesis was, by choosing the most representative image in CBCT1, diagnosis of bone defect could be facilitated. However, our results (agreements in CBCT2 were slightly better than CBCT1) have denied this hypothesis. The most possible explanation is that the sectional images in CBCT1 was selected by two researchers subjectively without knowing defect presence or not, thus complicated the assessment. On the contrary, by scrolling through all databases would give observers more information to diagnose.

Four observers participated in our study. Two experienced dentists with more than five years of experience in assessing radiographic image with implants and two graduate students major in implant dentistry. As for intra-observer agreements, results of experienced dentists were much better than graduate students. While for inter-observer agreements, there was small difference between experienced and inexperienced observers. Our result was different from Pelekos's study, which proved that the performance of PA film could be affected by experience, while accurate assessment of CBCT would not. In fact, our study was a clinical study with irregular defect type and size. Moreover, in their study, the defect size was at least 1.7 mm. Other in vitro studies conducted by Pinheiro et al. found that kappa values for more experienced maxillofacial radiologists were better than maxillofacial surgeons,, which was in accordance with our study. Moreover, Cochran's Q value was smaller in observer 1 and 2 than in observer 3 and 4, which indicated smaller variation in diagnosis for experienced observers. It's noteworthy that inexperienced observers scored higher for CBCT and scored much less “3” (unsure) than experienced observers, which showed experienced dentists tend to be more careful with diagnosis in clinical situation.

There were only 3 patients whose diagnosis were always “1”, which indicated confirmation of bone defects by all four observers both in PA and CBCT. As we can see from Fig. 3, the defect sizes in all 3 cases were not small. However, owing to undefined peri-implant defect in our study, the real defect size could not be acquired. With known 10 mm implants length placing in all 3 cases, we could only approximate the defect size subjectively. In fact, there were studies focusing on comparing defect sizes and evaluation results. There is a tendency that with defect size increasing, assessment scores would increase. The results of Kavadella's study showed that averaged score was 2.3 for the defect size 1.2 mm and 4.06 for the size 2.5 mm (same 5-point scale used in our study). Dave et al. found that diagnostic accuracy was better for 0.675 mm defect than 0.35 mm space. A recent study conducted by Kerkfeld et al. found bone defects that extending 400 um around implants could be reliably detected by using gray scale analysis in CBCT. Therefore, in the future, with new technique applied, defect assessment around implants might be facilitated.

Nowadays, CBCT has been widely used in clinical practice. Meanwhile, when CBCT is needed, “As Low As Reasonably Achievable” principle must be applied. In our study, we have chosen the smallest FOV with 0.25 voxel size. However, different settings of CBCT are not only associated with dose reduction, but also important in detecting peri-implant defects. However, effects of those parameters such as voxel size, field of view and even metal artifact reduction appliance are still under debate.,26, 27, 28 In addition, different CBCT systems have also showed variation in detecting defects., Continued research effort is urgently warranted to further explore these factors in diagnosing peri-implant defects with as low dose as possible in CBCT.

Although this study offers insights into radiographic agreements in clinical situation, several limitations should be considered. First, without knowing defect presence or not, we could not evaluate the diagnostic accuracy in PA and CBCT. Second, the geometric complexity of the bone defects is limited in the current study. Thus, the results might not be applicable to other clinical situation such as bone defects in peri-implantitis. Last but not the least, the use of only one imaging device for 2D and 3D imaging with only one parameter settings might reduce transferability of the results.

Within the limits of our study, agreements of PA are better than CBCT when detecting peri-implant bone defects, especially for inter-observer agreements. Experienced observers are more consistent in assessment than inexperienced ones. Individual variation in diagnosing defects around dental implants should be noted in clinical practice.

Declaration of competing interest

The authors have no conflict of interest to declare.