Validity and reliability of a three-dimensional dental cast simulator for arch dimension measurements.

BACKGROUND: The accuracy and reproducibility of measurements in a locally made three dimensional (3D) simulator was assessed and compared with manual caliper measurements.
MATERIALS AND METHODS: A total of 20 casts were scanned by our laser scanner. Software capabilities included dimensional measurements, transformation and rotation of the cast as a whole, separation and rotation of each tooth and clip far. Two orthodontists measured the intercanine width, intermolar width and canine, molar and arch depth on the casts and in 3D simulator. For calculating the reliability coefficient and comparing random and systematic errors between the two methods, intra-class correlation coefficient of reliability (ICC), Dahlberg and paired t-test were used, respectively. The ICC and Dahlberg's formula were also applied to assess intra-examiner and inter-examiner reliability of measurements on the casts and in the simulator (P < 0.05).
RESULTS: Canine and molar depth measurements had low reliability on the casts. Reliability between methods for the remaining three variables was 0.87, 0.98 and 0.98 in the maxilla and 0.92, 0.77 and 0.94 in the mandible, respectively. The method error was between 0.31 and 0.48 mm. The mean intra-observer difference were 0.086 and 0.23 mm in the 3D method and caliper. The inter-observer differences were 0.21 and 0.42 mm, respectively.
CONCLUSION: The maximum average absolute difference between the two methods was <0.5 mm, indicating that the new system is indeed clinically acceptable. The examiner reliability was higher in 3D measurements.

INTRODUCTION

Plaster study models have long been an important part of the orthodontic process, particularly in diagnosis and treatment planning and they have been the “gold standard” in orthodontics.[1] In the meantime, due to some drawbacks,[2] three dimensional (3D) digital models were introduced. 3D digital models can benefit the orthodontists in the following areas: Efficiency of having patient records instantly accessible on the computer screen instead of retrieving plaster models from the storage area, accuracy, efficiency and ease of the tooth and arch size measurements, accurate and simple diagnostic setup of various extraction patterns and also the possibility of showing patients the outcome of treatment to enhance their understanding and compliance, easy consultation and referral for interdisciplinary treatments, objective rather than subjective analysis and application in robotic archwire fabrication and in “clear aligners technique”.[34] When using 3D acquisitions of craniofacial structures, three factors should be kept in mind:

The accuracy of the imaging system;

The scientific basis of software designed for analysis of the images; and

The clinician's operational skills.

For the first step for 3D imaging of a structure, we should choose the best method of 3D imaging for the structure we want to capture. Laser scanning, volumetric imaging, computed tomography (CT), cone beam CT, structured light and stereophotogrammetry are among the methods used for reconstruction of craniofacial structures in the recent years.[15] Of the mentioned methods, laser scanning is among the most popular techniques used for cast scanning. Many of the studies have confirmed the accuracy and reliability of this technique.[6] In this method, the object has to stay still for 1 min or longer during scanning. This method is usually applied for 3D reconstruction of dentition by means of dental models.

Different software programs with various diagnostic and treatment planning capabilities have been introduced for cast analysis in the literature.[789101112] Validity of measurements by these software programs and their reliability are among the factors that should be addressed when developing 3D simulators. Furthermore, in order to analyze the casts, the 3D software accompanying the scanner should be user friendly and scientific. The software's accuracy should be tested in tooth size and arch size measurements and the software should be developed in accordance with the basic principles of dental cast and arch form analysis.[13]

The last consideration is the role of clinician as the operator in these systems. The level of clinician's knowledge about applying information technology and his/her concept about orthodontics are factors affecting the treatment planning objectives in the software. That is the reason for the assessment of inter-and intra-observer reliability when developing the software.

We developed a 3D software for cast analysis in this study. The validity of our laser scanner (Patent No 69383, Iran) was confirmed in our previous study.[14] Therefore, in order to find out whether our new software is efficient for longer distance measurements in the simulator and more importantly for basic measurements to reconstruct the arch form, we conducted a diagnostic study to assess the accuracy and reproducibility of measurements by this method in comparison to conventional manual measurements. Furthermore, we calculated intra- and inter-observer reliability to evaluate the role of operator in these measurements.

MATERIALS AND METHODS

In this in-vitro diagnostic study, a total of 20 dental plaster models were fabricated by using artificial teeth demonstrating various types of malocclusions: CL I malocclusion with crowding, CL II Division 1 and Division 2 malocclusions, CL III malocclusion with crowding and CL I malocclusion with bi-maxillary protrusion.

The diagnostic study models were duplicated by taking alginate (Orthoprint, Zhermack, Badia Polesine [Rovigo], Italy) impressions of the models, pouring them with plaster and then properly trimming the casts (type IV Dental Die Stone, GC FUJIROCK EP Polar White, GC Europe N.V, Belgium). The casts were then scanned by means of a laser scanner. Specifications of the laser scanner have been explained in our earlier study.[14] The same plaster models were also used for direct measurement by digital caliper.

The casts were scanned by our linear laser scanner and data were recorded by CCD cameras in analog format. These data were converted to point clouds by a personal computer (PC). The 3D simulator software was written to integrate the point clouds, filter the noises and produce 3D image of each cast. The designed graphic algorithms stored and processed the raw data by integration and deletion; producing 3D image of the cast. The capabilities of the designed software included dimensional measurements, transformation and rotation of the cast as a whole, separation and rotation of each tooth and clip far (omission of the image layer by layer until a certain point is reached to coordinate each arbitrary point located on the surface of each tooth).

Coordination

In order to determine the coordinates of each pixel on the CCD camera, the line produced by each diode laser (100 μm) was related to the surface where the laser was working on before scanning. Each point that was irradiated by laser and its image on the CCD camera were connected by a line; by relating this line to the surface produced by the two diode lasers, we measured the 3D coordinates of each point on the cast. In other words, by knowing the pixels’ coordinates and the characteristics of the camera lens, a linear equation was written and the coordinates were measured. This process eliminated the need for relating the coordinates at each time of scanning and made continuous scanning possible.

Design of computer graphics in laser scanning system

In order to construct graphical models from scanned points, we used a PC; which was connected to the step motor and laser scanner. The image processing unit received the coordinates of each irradiated point as visual signals in analog format from the CCD camera, converted them to digital information and stored them in the computer's hard disk drive as a real-time procedure. Afterwards, the step motor rotated the cast by 0.009° and this process was repeated until the whole cast was scanned. The information was saved in the computer as 3D coordinates.

This information consisted of overlapped and repeated coordinates of each point. Thus, before the construction of the 3D image, the repeated data were deleted. Accordingly, a 3D-wire frame model was created.

The 3D simulator

The designed graphic algorithms produced point clouds by relating color and shadows to the coordinates after their integration. In this 3D software, we used Delaunay triangulation algorithm to integrate the point cloud of coordinates and re-construct the cast as a whole. This algorithm was introduced by Bourke for the 1st time written in Fortran 77. Distance and surface measuring tools were incorporated in the software. The basic parts of the software were written using Visual Basic and C++ codes and Microsoft Windows application programming interfaces were used for increasing the speed during cast animation.

The validity and reliability of the measurements in the simulator

In order to calculate the validity of the software, five parameters were measured both in the simulator and directly on the cast using a digital caliper (Digital Caliper model no. 550-115, MTC tools, China). The five measured parameters were: Intercanine width, intermolar width, canine depth, molar depth and arch depth. The landmarks used as the reference point for measurements were canine cusp tip and mesiobuccal cusp tip of molars (in all measurements except for maxillary molars’ width; for which, the mesiopalatal cusp tip was used). Canine depth and molar depth were defined as the perpendicular distance of canine cusp tip or mesiobuccal cusp tip for mandibular molars and mesiopalatal cusp tip for maxillary molar from the incisal edge at midline. Features for canine width measurement are shown in Figures 1 and 2. Figure 1 shows distance measurement without Clipfar; the total scanned surface is obvious. Figure 2 shows distance measurement by means of Clipfar; we can omit sections that are not needed in certain measurements. In order to calculate the intra-examiner error, three of the measurements in each jaw (we omitted depth measurements) were repeated by one of the examiners after a 1-week interval. The parameters were also measured by two orthodontists in the software and on the cast for assessing the inter-observer reliability.

Figure 1

Distance measurement without clip far

Figure 2

Distance measurement with clip far

Statistical analysis

In order to calculate the reliability coefficient and random and systematic errors between the two methods, intra-class correlation coefficient of reliability (ICC), Dahlberg and paired t-test were used, respectively. The ICC and Dahlberg's formula were also applied to assess intra-examiner and inter-examiner reliability of measurements on the casts and in the simulator (P < 0.05). Statistical analysis was calculated by SPSS version 16 (SPSS Inc., Chicago, IL, USA).

RESULTS

We measured five variables (intercanine width and depth, intermolar width and depth and arch depth) on 20 dental casts (10 upper/10 lower). No statistically significant difference was detected in measurements between the two methods in the upper arches. Lower arch depth measurements were significantly different in the two methods. These differences were mainly attributed to error in locating the intersect of arch midline and canine or molar width line on the cast; therefore, the base of differences was error in landmark detection especially on the casts. For this reason, we omitted depth measurements for assessing the intra-and inter-observer reliability and defined a new way to calculate canine and molar depth on the cast as the basis for depth measurements in the simulator. The newly defined arch depth was the distance between the canine and mesiopalatal molar cusp tips in the maxilla and the distance between the canine cusp tip and the mesiobuccal molar cusp tip on the dental arch in the mandible. Therefore, the remaining calculations were performed using this parameter.

The intra-class reliability coefficients between the 3D method and digital caliper measurements for the three variables were 0.87, 0.98 and 0.98 in the maxilla and 0.92, 0.77 and 0.94 in the mandible, respectively. The method error between the measurements on dental casts and 3D images for three variables was in the range of 0.31-0.48 mm and the differences in this regard were not statistically significant. Tables 1 and 2 shows statistical indexes and the results of paired t-test for the upper and lower archs. Table 3 indicates the inter-observer and intra-observer reliability coefficients for each method and between the two methods; whereas the amount of differences between the two observations in mm has been calculated by means of Dahlberg formula and is demonstrated in Table 4.

Table 1

Statistical indexes and the results of paired t-test for the upper arches

Table 2

Statistical indexes and the results of paired t-test for the lower arches

Table 3

The inter-observer and intra-observer reliability coefficients for each method and between the two methods

Table 4

Inter-observer and intra-observer errors in each method and between the two methods in mm

DISCUSSION

This study has shown that the designed 3D system and its accompanying software (tooth.3D-X) were efficient for virtual dental cast reconstruction. The software had sufficient accuracy and reproducibility for measurement of intercanine and intermolar distances. Our study sample size was adequate according to Fleiss's suggestions[15] for quantitative variables. Our software helps the clinicians to judge more objectively and enables them to simply perform accurate measurements for reconstruction of patients’ arch form. As we know, the mentioned distances are the basis for reconstruction of the dental arch form by using beta function.[13]

The followings are the features of the designed software:

High speed (it can read more than 100,000 points in <10 s)

Real-time rotation of the cast

Moving the cast by the selected range in the software

Translation and zooming using the keyboard

Cutting and removing different parts of the cast, for example removing a tooth or gingival part of a tooth

Clip far that is the omission of the points until reaching the cusp tip or the most prominent point on any surface

Distance measurement between two points with 0.1 mm accuracy

Ease of application and training

Small size of the program (<400 KB): Since the software was developed by Visual Basic, it can be easily downloaded even with a dial up internet connection.

Creation of a 3D model and its analysis are done through the following phases:

Scanning of dental casts, eliminating extra parts and creating the point clouds

Creation of a 3D model by 3D software

Making changes on dental casts, such as rotating the casts, setting up and measuring the distances.

Usually the first and second stages of the process are more time consuming. The main advantages of this software include its high speed (the program opens in <1 s after clicking on its icon), creating a steady picture from the point clouds in <10 s, possibility of real-time rotation and changing the position of dental cast in 3D space and also the possibility of installing the program on PC without the need for up-grading the hardware. The software is user friendly and a typical user can learn it in a few minutes since when the user hovers the mouse pointer over an icon, pop-up descriptions appear representing its application.

The differences between the two methods may result from two sources: The operator and the device. The role of the examiner in landmark identification on casts was also confirmed in our former study.[16] The reliability of depth measurements between direct, 2D and 3D calculations was minimum for mandibular canine depth (r = 0.45). Another explanation is that the observed differences between the two methods may be due to random method error (i.e., point clarification error by the examiner) and not a systematic error (i.e., errors due to calibration of laser scanner or inaccuracy of the caliper itself). Thus, although the errors are small, their main source is the examiner. Measurement of posterior arch width in the mandible had the highest error in millimeters attributed to the cusp anatomy; which is blunt and convex and defining its exact tip may be difficult. The clip far feature designed in our 3D software seems to be responsible for better results. Using this option, the examiner can omit all the point clouds until reaching the cusp tip. Of course this option is not automated but it is possible to manually omit points until reaching the tip.

The differences between the two methods were clinically insignificant (0.31-0.48 mm) since they were smaller than 0.5 mm. In order to indicate the source of errors, we calculated the intra- and inter-examiner agreements in measuring the distances. When using the laser scanner, the coefficient of reliability was between 0.96 and 0.99 in repeated measures. The coefficient was between 0.90 and 0.99 when two examiners made the same measurements. When comparing the results of caliper measurements, the reliability of measurements ranged from 0.85 to 0.99 in repeated measures and 0.79-0.98 between the two observers. The range of errors was 0.06-0.40 in 3D calculations and 0.17-0.56 in caliper measurements. This indicates that caliper is less accurate for distance measurement. Therefore, we may conclude that laser scanning has higher reproducibility and greater agreement between observers. A study by Quimby et al.[8] reported that when ICC is bigger than 0.90, an excellent reliability is seen between the two methods. In his study, the range of differences between methods was 0.15 and 2.9 mm. Bell et al.[17] reported 0.18 mm intra-operator error. Zilberman et al. in their study[7] reported this error to be 0.17 mm. Therefore, we may claim that our scanner is as accurate as those used in the mentioned two studies.

Our results regarding the differences between the two methods were in accordance with those of Bell et al.,[17] Sohmura et al.[18] and Hirogaki et al.[19] The mean differences between the caliper and digital model measurements in Hirogaki et al.[19] and Bell et al.[17] studies were 0.3 and 0.27 mm, respectively. In both mentioned studies, paired t-test was used for determining the differences between the two methods. Paired t-test only considers the systematic error and random error (actual difference in the outcome) is not recognized by this statistical test. Keating et al.[20] used the same statistical analysis and variables. They obtained the mean difference of 0.14 mm, which was slightly lower than our rate. Keating et al.[20] measured the transverse parameters, intercanine and intermolar widths, similar to our study. Quimby et al.[8] also measured transverse parameters and found statistically significant differences between digital and plaster measurements. Variance of measurements in digital method was significantly greater than the manual method except for mandibular intercanine width. Although the differences were statistically significant, they were smaller than 0.5 mm and therefore clinically insignificant.

Canine and molar depths, based on our definitions for these distances, have not been calculated in any other study. Stevens et al.,[9] Mullen et al.,[10] Redlich et al.[11] and Goonewardene et al.[12] measured arch length as sum of mesiodistal lengths of 6 anterior or 12 posterior teeth. Mullen et al.[10] and Goonewardene et al.[12] detected significant differences between the two methods. The mean differences were 1.47 ± 1.55 mm for maxillary and 1.5 ± 1.36 mm for mandibular arch length in Mullen et al.[10] study and 1 mm and 0.8 mm for maxillary and mandibular arch length, respectively in Goonewardene et al.[12] study. These errors were reported to be clinically acceptable. In contrast, Stevens et al.[9] and Redlich et al.[11] reported no statistically or clinically significant difference between the two methods when crowding was <4 mm. Redlich et al.[11] explained that despite the good correlation between measurements by the two methods, when crowding is more than 4 mm, the mesiodistal width of each tooth is larger and use of 3D method will underestimate the amount of crowding if the tangent plane of the proximal contact is used for width measurement. Therefore, in moderate to severe crowding (>4 mm/arch), it is logical to use the conventional method or alter the method of width measurement in the digital system. This finding was also confirmed in a systematic review by Fleming et al.[6]

CONCLUSION

Since the maximum average absolute difference between the two methods was <0.5 mm, the designed system is indeed clinically acceptable for measuring arch depth and width in canine and molar areas. Software accuracy for distance measurement is comparable to digital caliper; which is the conventional method for the analysis of dental casts. Inter- and intra-examiner reliability is higher in 3D simulator compared with the conventional method. Considering the mentioned advantages, this designed 3D virtual orthodontic model simulator can be used as the standard system for clinical orthodontics.