Evaluation of accuracy of 3-dimensional printed dental models in reproducing intermaxillary relational measurements: Based on inter-operator differences.

OBJECTIVE: Although, digital models have recently been used in orthodontic clinics, physical models are still needed for a multitude of reasons. The purpose of this study was to assess whether the printed models can replace the plaster models by evaluating their accuracy in reproducing intermaxillary relationships and by appraising the clinicians' ability to measure the printed models.
METHODS: Twenty sets of patients' plaster models with well-established occlusal relationships were selected. Models were scanned using an intraoral scanner (Trios 3, 3Shape Dental System) by a single operator. Printed models were made with ZMD-1000B light-curing resin using the stereolithography method 3-dimensional printer. Validity, reliability, and reproducibility were evaluated using measurements obtained by three operators.
RESULTS: In evaluation of validity, all items showed no significant differences between measurements taken from plaster and printed models. In evaluation for reliability, significant differences were found in the distance between the gingival zeniths of #23-#33 (DZL_3) for the plaster models and at #17-#43 (DZCM_1) for the printed models. In evaluation for reproducibility, the plaster models showed significant differences between operators at midline, and printed models showed significant differences at 7 measurements including #17-#47 (DZR_7).
CONCLUSIONS: The validity and reliability of intermaxillary relationships as determined by the printed model were clinically acceptable, but the evaluation of reproducibility revealed significant inter-operator differences. To use printed models as substitutes for plaster models, additional studies on their accuracies in measuring intermaxillary relationship are required.

INTRODUCTION

Oral plaster models (plaster models) including teeth have a long history of use in all areas of dental treatment, for various purposes such as diagnosis, manufacturing of appliance, and recording of treatment results.[1,2] Orthodontics requires the preparation of facial and intraoral photographs, radiographs, and plaster models as data for diagnoses.[2] Considerable information can be obtained from plaster models by hard tissue analysis of number, size, and shape of teeth, symmetry of the jaw, and shape of the dental arch; soft tissue analysis can reveal frenal abnormalities and palatal shape. When determining arch length discrepancy, which is one of the important factors that determines the need for tooth extraction during orthodontic treatment, measurement analysis using plaster models provides easier and more accurate information than direct intraoral measurements.[3,4] In orthodontics, plaster models require to be stored for a long duration of time to enable future evaluations of treatment results and to aid in diagnoses and treatment plans. Unlike medical records or radiographs, medical law does not require the preservation of plaster models,[5,6] but the Korean Association of Orthodontics recommends keeping them for 10 years for various reasons such as medical disputes.[6] Mizrahi[7] argued for the necessity of model storage because they are invaluable aids to defend against any future litigation. Additionally, Charangowda[8] advocated that plaster models of orthodontic patients (before and after treatment) be permanently stored for research, teaching, and forensic reasons. However, a considerably large storage space of 17 m3 is required to store the plaster models of one thousand patients.[3,9] Consequently, plaster models have been replaced by digital models,[8] which not only reduce the requirement of storage space, but also have many other advantages such as enabling the sharing of data between distant doctors.[3,9] Despite the many advantages of digital models, physical models may be necessary for manufacturing orthodontic appliances, diagnosing complex cases,[10] for education purposes, and planning surgery.[8,11] Physical models can be reconstructed from 3-dimensional (3D) data obtained from digital models. The most commonly employed techniques by dental 3D printers are stereolithography (SLA), the triple jetting technique (poly jet), and fusion deposition printing.[12] SLA printing involves the use of an ultraviolet laser to cure resin,[12-14] thus, digital models can be produced as physical models when necessary. Therefore, the accuracy of 3D printed dental models (printed models) is important, and many researchers have addressed this topic.[10-12,15,16] However, these studies evaluated accuracy with respect to either the maxillae or the mandibles, and few have evaluated the accuracy of intermaxillary relationships by comparing various measurements such as overbite and overjet. The purpose of this study was to assess whether the printed models can replace the plaster models by evaluating their accuracy in reproducing intermaxillary relationships and by appraising the clinicians’ ability to measure the printed models. We hypothesized that the intermaxillary relationship of the printed model is sufficiently accurate and there is no difference between the plaster and the printed model in clinicians’ measurement ability.

MATERIALS AND METHODS

Subjects

Data for this study was collected from the orthodontic department of Gachon University Gil Medical Center in Incheon, Korea. In order to ensure occlusal stability of the model, only those plaster models with complete eruption of permanent dentition, with no missing teeth, and with no more than 1 prosthesis per side were studied. To check exact anatomical positions during measurements, models of damaged teeth, severely worn teeth, those with unclear shapes of crown-gingival boundaries, or models that were poorly stored (e.g., contaminated) were excluded. When selecting the model, the existence of anterior crowding was not considered. Using these criteria, 20 plaster model sets that well-reproduced intermaxillary relations were selected for the study.

This study was approved by Institutional Review Board of Gachon University Gil Medical Center (GC IRB 2019-344).

Production of printed models

Plaster models were scanned using an intra-oral scanner (Trios 3, software version: TRIOS 1.4.7.5; 3Shape Dental System, Copenhagen, Denmark) according to the manufacturer’s instructions[17] and stored in standard tessellation language (STL) format. All scans were performed by the 2nd (of three) operator who had extensive experience of utilizing this technique in clinical practice. Printed models were created by sending the STL files to Dentis Co. (Seoul, Korea) via e-mail; they then used a Zenith 3D printer (Dentis Co.), the SLA Zenith slicer program (Dentis Co.), and ZMD-1000B MODEL photopolymerizable resin (Dentis Co.) to fabricate the models. The layer height used was 100 μm,[18] and model bases were of the regular American Board of Orthodontists (ABO) type.[12]

Reproduction and maintenance of occlusal relationships

After scanning the plaster models, three operators occluded them together and fixed them with wax to maintain their occlusal states; next, all three operators performed measurements. Intermaxillary relationships of printed models were then reproduced by each operator by referring to the occlusal relationships of the plaster models.

Measurements

The 1st operator measured the plaster and printed models twice, whereas the 2nd and 3rd operators measured each model once. Measurement values in millimeters were recorded up to the second decimal place, using a digital gauge (Teclok, Nagano, Japan) (Figures 1 and 2). Before collecting measurement data, two sets of plaster models and printed models were randomly selected, and all measurement items were repeatedly measured to set measurement points and unify measurement methods between operators. All operators were well-equipped with the use of plaster models clinically, and thus, had considerable experience in performing the measurements, but none of the three operators had used a printed model clinically. The 1st operator measured printed models for the first time during this study, and the 2nd and 3rd operators had measured them purely for research purposes.

Figure 1

Plaster model measurement using a digital caliper.

Figure 2

Printed model measurement using a digital caliper.

Validity of the printed models with respect to the plaster models, and reliabilities of plaster and printed models were evaluated using measurements obtained by the 1st operator. Reproducibilities of the measurements obtained from each plaster and printed model were evaluated using measurements obtained by the three operators (Figure 3).[19] The second set of measurements recorded by the 1st operator were obtained at least 1 week after the initial measurements. Each operator measured the plaster models first and then measured the printed models at least a week later. Item measurements are shown in Table 1.

Figure 3

Schematic view of validity, reliability, and reproducibility in this study.

ANOVA, analysis of variance.

Adapted from the article of Naidu and Freer (Am J Orthod Dentofacial Orthop 2013;144:304-10).[19]

Table 1

Measurement definitions

Measurement	Definition
DZR_7	Distance between #17 gingival zenith and #47 gingival zenith
DZR_3	Distance between #13 gingival zenith and #43 gingival zenith
DZR_2	Distance between #12 gingival zenith and #42 gingival zenith
DZL_7	Distance between #27 gingival zenith and #37 gingival zenith
DZL_3	Distance between #23 gingival zenith and #33 gingival zenith
DZL_2	Distance between #22 gingival zenith and #32 gingival zenith
DZCM_1	Distance between #17 gingival zenith and #43 gingival zenith
DZCM_2	Distance between #27 gingival zenith and #33 gingival zenith
DZCM_3	Distance between #37 gingival zenith and #23 gingival zenith
DZCM_4	Distance between #47 gingival zenith and #13 gingival zenith
DZC_13	Distance between #13 gingival zenith and #33 gingival zenith
DZC_24	Distance between #23 gingival zenith and #43 gingival zenith
Midline	Horizontal distance between the midline of maxillary and mandibular dental arch

Statistical analysis

Descriptive data analysis was conducted using SPSS version 20.0 (IBM Corp., Armonk, NY, USA). Paired t-test was used to evaluate the validity of printed model and the reliabilities of the printed and plaster models. Repeated measured analysis of variance was used to evaluate inter-operator reproducibility.

RESULTS

Validity of printed models was evaluated using measurements taken by the 1st operator. All items showed no significant differences between measurements taken from the plaster and printed models (Table 2).

Table 2

Validity of printed models versus plaster models for measurements obtained by the 1st operator

Measurement	Plaster model			Printed model			Difference	p-value
	Mean	SD		Mean	SD		Mean
DZR_7	10.61	1.42		10.68	1.43		–0.07	0.76
DZR_3	18.66	3.45		18.83	3.47		–0.17	0.06
DZR_2	15.81	2.45		16.01	2.69		–0.20	0.24
DZL_7	10.48	1.95		10.48	1.91		–0.01	0.94
DZL_3	17.85	2.67		17.84	2.58		0.01	0.97
DZL_2	16.26	2.02		16.18	1.95		0.09	0.40
DZCM_1	40.29	3.06		40.37	3.10		–0.08	0.66
DZCM_2	40.75	2.66		40.71	2.65		0.05	0.74
DZCM_3	34.09	6.32		33.93	6.45		0.16	0.11
DZCM_4	35.18	2.23		35.37	2.18		–0.18	0.36
DZC_13	37.59	2.62		37.54	2.47		0.05	0.59
DZC_24	37.92	1.84		37.87	1.67		0.05	0.59
Midline	1.06	0.89		1.09	0.86		–0.03	0.45

Paired t-test was performed.

SD, standard deviation.

See Table 1 for definitions of each measurement.

Reliability testing using measurement values obtained by the 1st operator showed a significant difference between first and second measurements of the distances between the gingival zeniths of #23–#33 (DZL_3) for plaster models and of #17–#43 (DZCM_1) for printed models (Table 3).

Table 3

Intra-operator reliabilities of measurements obtained from plaster and printed models by the 1st operator

Measurement	Plaster model									Printed model
	1st measure			2nd measure			Difference	p-value		1st measure			2nd measure			Difference	p-value
	Mean	SD		Mean	SD		Mean			Mean	SD		Mean	SD		Mean
DZR_7	10.64	1.40		10.58	1.45		0.06	0.17		10.65	1.42		10.70	1.45		–0.05	0.33
DZR_3	18.69	3.44		18.64	3.46		0.05	0.33		18.80	3.37		18.86	3.37		–0.06	0.15
DZR_2	15.79	2.45		15.83	2.46		–0.04	0.35		15.99	2.69		16.02	2.70		–0.03	0.53
DZL_7	10.44	1.93		10.51	1.98		–0.06	0.17		10.50	1.90		10.47	1.92		0.03	0.56
DZL_3	17.80	2.68		17.90	2.66		–0.10	0.02*		17.86	2.62		17.83	2.54		0.03	0.60
DZL_2	16.29	2.01		16.24	2.02		0.05	0.32		16.17	1.94		16.18	1.97		–0.02	0.63
DZCM_1	40.29	3.07		40.29	3.06		–0.01	0.94		40.29	3.15		40.46	3.05		–0.18	0.02*
DZCM_2	40.75	2.63		40.76	2.70		–0.02	0.82		40.66	2.69		40.75	2.62		–0.09	0.06
DZCM_3	34.08	6.32		34.10	6.33		–0.02	0.71		33.93	6.48		33.93	6.44		0.01	0.92
DZCM_4	35.22	2.26		35.15	2.21		0.07	0.27		35.32	2.20		35.42	2.17		–0.10	0.16
DZC_13	37.60	2.63		37.57	2.62		0.03	0.53		37.54	2.48		37.53	2.47		0.01	0.82
DZC_24	37.92	1.82		37.92	1.86		–0.01	0.89		37.89	1.61		37.85	1.74		0.04	0.60
Midline	1.06	0.93		1.06	0.86		0.00	0.93		1.08	0.86		1.10	0.87		–0.02	0.35

Paired t-test was performed.

SD, standard deviation.

*p < 0.05.

See Table 1 for definitions of each measurement.

Reproducibility was evaluated for the plaster and printed models using measurements obtained by all three operators. For plaster models, a significant inter-operator difference was observed at midline; for printed models, distances between the gingival zeniths of #17–#47 (DZR_7), #13–43 (DZR_3), #12–#42 (DZR_2), #23–#33 (DZL_3), #13–#33 (DZC_13), #23–#43 (DZC_24) and midline showed significant differences (Table 4).

Table 4

Inter-operator reproducibilities of measurements taken by the three operators from plaster and printed models

Measurement	Plaster model										Printed model
	1st operator			2nd operator			3rd operator		p-value		1st operator			2nd operator			3rd operator		p-value
	Mean	SD		Mean	SD		Mean	SD			Mean	SD		Mean	SD		Mean	SD
DZR_7	10.61	1.42		10.93	1.14		10.78	1.31	0.21		10.68	1.43		11.01	1.35		10.77	1.40	0.03*
DZR_3	18.66	3.45		18.72	3.54		18.68	3.48	0.64		18.83	3.37		18.89	3.34		18.77	3.37	0.03*
DZR_2	15.81	2.45		15.91	2.41		15.87	2.43	0.44		16.01	2.70		16.03	2.67		15.93	2.65	0.01*
DZL_7	10.48	1.95		10.68	1.82		10.60	1.86	0.06		10.48	1.91		10.60	1.88		10.57	1.87	0.07
DZL_3	17.85	2.67		17.88	2.61		17.89	2.58	0.73		17.84	2.58		17.98	2.55		17.84	2.61	0.02*
DZL_2	16.26	2.02		16.25	2.02		16.25	2.07	1.00		16.18	1.95		16.22	1.89		16.12	1.94	0.12
DZCM_1	40.29	3.06		39.32	4.05		40.42	2.96	0.09		40.37	3.10		40.24	3.16		40.39	2.91	0.17
DZCM_2	40.75	2.66		40.70	2.52		40.71	2.60	0.75		40.71	2.65		40.53	2.64		40.60	2.65	0.20
DZCM_3	34.09	6.32		35.07	2.41		35.05	2.30	0.80		33.93	6.45		34.88	2.31		35.02	2.62	0.60
DZCM_4	35.18	2.23		34.71	4.05		35.13	2.23	0.38		35.37	2.18		35.20	2.24		35.15	2.16	0.54
DZC_13	37.59	2.62		37.65	2.61		37.62	2.57	0.72		37.54	2.47		37.75	2.59		37.65	2.50	0.02*
DZC_24	37.92	1.84		38.01	1.84		36.40	6.44	0.33		37.87	1.67		37.98	1.70		37.78	1.65	0.00**
Midline	1.06	0.89		1.01	0.93		1.03	0.87	0.02*		1.09	0.86		0.88	0.76		1.01	0.84	0.00**

Repeated measured analysis of variance.

SD, standard deviation.

*p < 0.05, **p < 0.01.

See Table 1 for definitions of each measurement.

DISCUSSION

Intermaxillary relations including overbite, overjet, occlusal contact, and midline deviation are important for diagnosis and treatment planning in orthodontic patients.[20] Many studies have been conducted on the intermaxillary accuracies of digital models, which have recently been rapidly adopted by clinics.[20-23] However, studies on the intermaxillary accuracies of printed models derived using digital models are rare, and there is a need for additional studies;[11,16,24,25] thus, we conducted the present study to assess it. Regarding criteria for determining accuracy, Wesemann et al.[24] reported that a measurement error of 250 μm is acceptable. However, Hayashi et al.[26] concluded that an error of 100 μm is clinically acceptable, and Bell et al.[1] and Wan Hassan et al.[11] found that errors of 0.27 mm and 0.5 mm were clinically acceptable. While printing a model, the layer height affects model precision. Brown et al.[16] noted that it is possible to make more precise printed models by thinner layering and conducted a study with 16 μm layer height. However, Loflin et al.[18] and Sherman et al.[27] reported that a 100 μm layer height did not affect diagnosis or treatment planning, was satisfactory for educational purposes, and advantageous in terms of manufacturing time. Therefore, we used a layer height of 100 μm and a beige colored material to print the models as recommended by the American Board of Orthodontics.[28] The model base was made in a regular (ABO) type according to the reports of Camardella et al.[12] and Camardella et al.[29]

In the evaluation of validity, conducted using the measurements obtained by the 1st operator, all items showed no significant difference, and mean difference (range 0.01 to 0.20 mm)[1,24] between measurements obtained using the plaster and printed models were clinically acceptable (Table 2), which indicated that the printed model was sufficiently accurate to determine intramaxillary[16,24,25] and intermaxillary relationships. Research has been published showing the accuracies of intermaxillary relationships in printed models using overbite and overjet measurements.[25,29-31] Saleh et al.[25] and Camardella et al.[29] reported no significant difference in overbite and overjet measurements between plaster and printed models, but Rebong et al.[30] found a significant difference in overbite. Reuschl et al.[31] reported the ‘striking difference’ in the overjet measurements by different operators of the plaster model. Porter et al.[22] did not use overbite and overjet in the study of the precision of intermaxillary relations. Although many researchers have used overbite and overjet to evaluate the precision of the intermaxillary relationship, they do not agree on the definitions of overbite and overjet.[29-31] So we decided to exclude measurements of overbite and overjet, making direct comparisons with previous studies impossible.

In the evaluation of reliability for each model, performed by comparing measurements obtained by the 1st operator, a significant difference was found for DZL_3 of the plaster model and DZCM_1 of the printed model (Table 3). The errors were 0.1 mm and 0.18 mm, respectively, which were clinically acceptable.[1,24] The reliability of intermaxillary relationships in the plaster and printed models was judged to be acceptable.

The evaluation of reproducibility for the plaster model was conducted by comparing measurements obtained by the three operators, and the only significant difference found was for midline deviation; the difference between measured values was 0.1 mm (Table 4). Thus, the reproducibility of plaster models was judged to be acceptable, which is in concordance with previous studies.[1,11] Reproducibility evaluation of the printed model was performed in the same manner. Seven measured distances, including DZR_7, showed significant differences (Table 4), and the range of measurement error was 0.33–0.1 mm, which was acceptable according to criteria proposed by Wan Hassan et al.[11] and Camardella et al.[21]; but not according to the clinical acceptance criteria of Wesemann et al.,[24] Hayashi et al.,[26] and Bell et al.[1] As both types of models used in this study were physical, the measurement methods were identical. However, operator measurements taken from plaster models only differed significantly for one midline item, whereas measurements taken from the printed model differed significantly for 7 items, including DZR_7. Wan Hassan et al.[11] argued that printed model reproduction was inaccurate in detailed areas, such as cervical margins, fossae, fissures, and cusp tips. In this study, using cervical margin as a measurement point, difficult-to-identify landmarks for measurements (due to the loss of detailed definition of the measurement sites[11]) were assumed as one of the causes of these errors. However, Camadella et al.[29] reported that the interarch relationship did not reveal any clinically relevant difference between printed and plaster models, although there was a clinically relevant reduction in the transverse dimension, which is dissimilar from the results of this study. Porter et al.[22] and Darroudi et al.[20] minimized measurement errors by marking plaster models and then scanning them to accurately reproduce measurement points. We found significant inter-operator differences for many items in the printed model and suggest additional research be performed to determine whether this is only due to differences between the measurement points used by operators[11,21]; or both being physical models, due to the tactile feedback[20] of the printed model, which has different surface characteristics from the plaster model; or due to simple artifacts[11] on the occlusal surface.

In this study, the occlusal relationships of printed models were reproduced by referring to the corresponding plaster models. However, with establishment of digital clinics, plaster models will no longer exist, and thus, reproduction of intermaxillary relationships in printed models should be based on intra-oral scanning. Yoo et al.[23] and Camardella et al.[21] concluded that there were no errors with different digital model measurement programs. But Westerlund et al.[32] recommended training to use digital models, and Camardella et al.[21] commented that experience is necessary. Printed models can be fabricated right after a digital model is produced or a long time later. Therefore, we propose a study to produce a printed model under various conditions and to evaluate its intermaxillary accuracies by reproducing the occlusal relationship based on the digital model.

This study was performed using a digital caliper that has been validated as a reliable method by previous studies,[10,11,15,16,29,31] however, a study using latest technology, such as 3D superimposition software[12,21] will be beneficial. Additionally, as observed by Rebong et al.,[30] further research is required to define error limits for accuracy evaluations.

CONCLUSION

The validity and reliability of intermaxillary relationships as demonstrated by printed models were acceptable at the clinical level. However, significant differences were observed in inter-operator reproducibility for intermaxillary relationships of printed models. To use printed models as substitutes for plaster models, additional studies are required on the accuracies of intermaxillary relationships.