Effect of the haptic 3D virtual reality dental training simulator on assessment of tooth preparation.

BACKGROUND/
PURPOSE: The haptic 3D virtual reality dental training simulator has been drawn attention as a educational strategy in Covid-19 pandemic. The purpose of this study is to investigate the feature of the haptics simulator in comparison with a conventional mannequin simulator by analyzing the assessment of products prepared by dental students using these two kinds of simulator.
MATERIALS AND METHODS: The subjects were 30 students in the sixth-year classes of the faculty of dentistry. Abutments for a full cast crown were prepared by each subject using two kinds of simulator; one is the haptics simulator and the other is a mannequin simulator. For the resulted products, occlusal surface form, margin design, surface smoothness, taper angle, total cut volume and overall impression were rated by 3 evaluators. Score differences between two simulators were statistically analyzed.
RESULTS: The kinds of simulator affected subject performance for margin design and total cut volume. The differences in cutting feeling between the simulators as well as variation of stereoscopic ability in subjects were considerable reasons. Evaluators' rating was affected by difference in simulators for occlusal surface form, total cut volume, and overall impression. This may have been due to variation of stereoscopic ability in evaluators.
CONCLUSION: The unique characteristics of virtual reality, such as the simulated cutting sensation and the simulated three-dimensional images created by stereo viewers, affect operators' performance and evaluators' rating. It was suggested that educational programs need to be constructed taking account of the characteristics of virtual reality to make the best use of the haptics simulator.

Introduction

In the recent Covid-19 pandemic, many dental schools are suffering from limited educational circumstances in which students and teachers must keep social distance. As one of the countermeasures for the pandemic situation, the haptic 3D virtual reality dental training simulator has been drawn attention. In training using the device, trainees cut teeth on a 3D monitor using a special controller that imitates an air-turbine handpiece.2, 3, 4, 5, 6, 7 In the combination with internet technology, such digital device would contribute to realize remote training to minimize the spread out the infection. Additionally, some studies have reported usefulness of the digital educational device from standpoints of objective assessment.8, 9, 10, 11, 12, 13 Thus, the digital simulator would hold the promise of its great potential in dental education, however, it has not yet been widespread in dental schools in the world. Majority is still the conventional simulator training, in which trainees cut artificial teeth attached on phantom head and teachers perform visual inspection to make assessment and instruction. To introduce the novel device into conventional courses and make the best use, revealing difference between two instruction process is important.

Although there have been many studies to investigate educational effect of haptics simulators in the past,14, 15, 16, 17, 18 only one study investigated the assessment difference between a haptic and a conventional analog simulator by analyzing students’ learning curves. Up to now, there has not been a study revealing evaluation difference between a haptics and conventional simulator when the same students were evaluated using the same evaluation items. Therefore, in this study, we compared the scores of products prepared by the same student in haptics and mannequin simulators, evaluated by the same evaluator. We tested the hypothesis that “the difference in the simulators used by the students would affect the evaluation of the products”.

Materials and methods

Subjects

Among the sixth-year students of Faculty of Dentistry in 2019 who had completed one year of clinical training, 30 students (12 males and 18 females) who consented to participate in the experiment were included in the study. No exclusion criteria about gender or age were employed in this study. All the subjects in this study used the haptics simulator for the first time.

Devices

Simodont® (Nissin Dental Products Europe BV, Nieuw-Vennep, Netherlands) was used as the haptics simulator. The object to be prepared was the default dentition in the simulation system, and the cutting bar to be used was selected by the operator as appropriate from FG110, FG001, FG172, FG166, FG856 016 BL, FG856 016 RE, FG856 018 RE, FG856 023 RE, FG257. In this simulator, the operator prepare a 3D virtual tooth on the monitor using stereoscopic vision glasses. The angle of the object appearing on the monitor can be changed by operating the simulator. The subject can use this function to change the observation angle of the object as needed during preparation.

A conventional mannequin simulator, Clinsim (Morita, Osaka, Japan), was used with an artificial tooth A5A-500 (Nissin, Kyoto, Japan) and a jaw model D16FE-500H (GSF)-MF (Nissin). The following sets of cutting bar were provided: 102R, F102R, 201, 117, 204, 440, 212R, 213R (Shofu, Kyoto, Japan), and the subjects freely selected from the set as appropriate. The positioning during the cutting and the angle of the mannequin's neck could be changed by the subject as needed.

Experimental procedure

Subjects received explanations on how to operate the haptics simulator before tooth preparation using the simulator. No explanation was given for the conventional simulator because all of the subjects had experience using it. For 10 min after the explanation, the subjects operated the haptics simulator freely to become familiar with the device. Subsequently, the subjects undertook the tooth preparation for a full cast crown of the right first molar in the mandible using each simulator.

During preparation, a reference model was presented to the subjects to prepare teeth in accordance with the morphology. The specific instructions for the details are as follows: 1) the occlusal surface form of abutment should be similar figure of original occlusal surface, 2) the margin design should be light chamfer, 3) the taper should be 2–5° on one side, and 4) total cut volume of occlusal surface should be 1.2–1.5 mm. For each simulator, the time limit was set at one hour, and any number of teeth could be prepared within the time limit and the final product was the one that the subject judged to be their best.

Three evaluators with more than 10 years of university teaching experience evaluated the products of both haptic and conventional simulators. The evaluation items were those that can be measured with both haptics and conventional simulators, referring to previous reports: (1) Is the occlusal surface form of the abutment similar figure of original occlusal surface (hereafter, “Occlusal surface form”), (2) Is the margin design light chamfer (hereafter, “Margin design”), (3) Is the preparation surface smooth (hereafter, “Surface smoothness”), (4) Is the taper appropriate (hereafter, “Taper angle”), and (5) Is the total cut volume appropriate (hereafter, “Total cut volume”). Each item was scored on a 5-point rating scale (5: good, 4: somewhat good, 3: undecidable, 2: somewhat poor, 1: poor). In addition, the overall impression of the simulator products (hereafter, “Overall impression”) was scored on a 10-point rating scale.

In the haptics simulator, standardized images of buccal and mesial surface of prepared tooth were constructed from the STL data of the products, and the tapers of mesial, distal, buccal, and lingual side were measured on these images (Fig. 1a). In the conventional simulator, standardized images of the buccal and mesial views of the prepared tooth were photographed, and taper was measured in the same procedure as the haptics simulator (Fig. 1b).

Figure 1

Standardized images of a buccal surface of prepared tooth. a: Haptics simulator, b: Conventional simulator. In the image, a tangent line was drawn from the cervical margin to the abutment contour of each side, and the angle between the tangent line and a straight line parallel to the tooth axis was measured as taper (arrows).

After the experiment, we obtained written comments from the subjects about their opinion about the difference between the haptics simulator and the conventional simulator.

Statistical analysis

The differences between the two simulators in the ratings of each item of the product and the overall impression were analyzed using Wilcoxon's signed rank test (P < 0.05). To investigate inter-rater reliability, intraclass correlation coefficients were obtained for the conventional simulator scores of the three evaluators. For the actual taper angle, the differences between the two simulators were analyzed by paired t-test (P < 0.05). A linear mixed model was performed with the simulator and evaluator and their interaction term as explanatory variables, and each item rating as objective variables (P < 0.05). The above statistical analyses were performed using SPSS ver. 23 (IBM, New York, NY, USA) statistical analysis software.

Ethical approval

The study protocol was approved by the ethics committees of Tokyo Medical and Dental University (D2018-083).

Results

Evaluation of both simulator products

The scores of each evaluation item for the products of both haptics and conventional simulators are shown in Table 1. The conventional simulator showed higher values for Margin design, Surface smoothness, and Total cut volume, and the difference between the simulators was significant by Wilcoxon's signed rank test (P < 0.05). Differences in scoring between the simulators for each evaluator are shown in Fig. 2. For all evaluation items, the scores for the conventional simulator were highest for Evaluator 3, followed by Evaluator 1 and then Evaluator 2. The intraclass correlation coefficient was 0.896, indicating that the inter-rater reliability of the three evaluators was high. Looking at the changes in the scores between the haptics simulator and the conventional model, the scores of the three evaluators for Surface smoothness changed almost in parallel while those in other evaluation items not.

Table 1

The scores of each evaluation item of both haptics and conventional simulators and the result of Wilcoxon Signed rank test.

		Evaluation items
		Occlusal surface form	Margin design	Surface smoothness	Taper angle	Total cut volume	Overall impression
Simulator (mean ± SD)	Haptics	2.7 ± 1.0	2.6 ± 1.1	2.7 ± 0.9	2.8 ± 1.1	2.6 ± 1.1	5.7 ± 1.6
	Conventional	2.8 ± 1.0	3.0 ± 1.1	3.1 ± 1.0	3.0 ± 1.0	2.9 ± 0.9	5.5 ± 1.5
Wilcoxon signed rank test	Z-score	−0.636	−2.730	−3.011	−1.143	−2.592	−0.292
	p-value	0.525	0.006	0.003	0.253	0.010	0.770
	Effect size	−0.08	−0.35	−0.39	−0.15	−0.34	−0.04

Figure 2

Score difference of each evaluator between haptics and conventional simulators. A: Occlusal surface form, B: Margin design, C: Surface smoothness, D: Taper angle, E: Total cut volume, F: Overall impression.

The results of the linear mixed model analysis are shown in Table 2. The two items that showed a significant difference (P < 0.05) in the score between haptics and conventional simulators were Margin design and Total cut volume. On the other hand, while a significant difference in scoring between evaluators was observed for all evaluation items (P < 0.05), an interaction between simulator and evaluator was observed for Occlusal surface form, Total cut volume, and Overall impression (P < 0.05).

Table 2

The results of the linear mixed model analysis.

	Evaluation items
		Occlusal surface form	Margin design	Surface smoothness	Taper angle	Total cut volume	Overall impression
Hapticsa	B (95%CI)	0.367 (−0.103 to 0.836)	0.600 (0.057–1.143)	0.366 (−0.124 to 0.857)	0.300 (−0.213 to 0.813)	0.533 (0.082–0.984)	0.233 (−0.470 to 0.936)
	p-value	0.125	0.030	0.142	0.250	0.021	0.514
Evaluator 1b	B (95%CI)	0.333 (−0.136 to 0.803)	0.233 (−0.309 to 0.776)	−0.400 (−0.890 to0.090)	0.433 (−0.080 to 0.946)	0.800 (0.348–1.252)	0.333 (−0.670 to 0.736)
	p-value	0.163	0.397	0.109	0.098	0.001	0.936
Evaluator 2b	B (95%CI)	−0.800 (−1.269 to 0.331)	−0.733 (−1.276 to −0.191)	−0.500 (−0.990 to −0.010)	−0.633 (−0.115 to −0.120)	−0.700 (−1.151 to −0.248)	−1.767 (−2.470 to −1.063)
	p-value	0.001	0.008	0.046	0.016	0.003	0.000
Haptics x Evaluator 1c	B (95%CI)	−0.900 (−1.563 to 0.236)	−0.567 (−1.334 to 0.201)	0.167 (−0.527 to 0.860)	−0.533 (−1.259 to 0.193)	−0.833 (−1.471 to −0.195)	−1.400 (−2.394 to −0.405)
	p-value	0.008	0.147	0.636	0.149	0.011	0.006
Haptics x Evaluator 2c	B (95%CI)	0.667 (−0.597 to 0.730)	0.133 (−0.634 to 0.900)	0.100 (−0.593 to 0.793)	0.167 (−0.559 to 0.892)	0.167 (−0.471 to 0.805)	0.233 (−0.762 to 1.228)
	p-value	0.843	0.732	0.776	0.651	0.607	0.644

B: unstandardized partial regression coefficients.

Reference: Conventional.

Reference: Evaluator 3.

Reference: Haptics x Evaluator 3, Conventional x Evaluator 1, Conventional x Evaluator 2, Conventional x Evaluator 3.

Comparison of measured taper angles (Fig. 3)

Comparing the measured taper angles of the product of the haptics and the conventional simulator, the mesial and distal taper angles were significantly larger for the conventional simulator (P < 0.05).

Taper angle of both haptics and conventional simulators. ∗: significantly different with paired-t test.

Comments from the subjects

The comments from subjects pointed out the following differences between the haptics simulator and the conventional mannequin: “sense of touch in cutting,” “degree of freedom in the direction of observing,” “sense of distance and perspective,” and “presence of finger rest” (Fig. 4).

Figure 4

The numbers of the comments about differences of haptics simulator in comparison with conventional mannequin.

Discussion

In this study, the scores of the haptics simulator were lower than those of the conventional type for several evaluation items. Therefore, the hypothesis of this study that “the difference of the simulator used affects the evaluation of the products” was adopted. We considered the simulator difference would have effect on two processes in this study; one is the effect on the student's performance process during preparation and the other is effect on the evaluation process after preparation. In order to examine these two separately, a linear mixed model was employed as multivariate analysis. As a result, we found that the difference in simulator was significantly affected students' performance in margin design and total cut volume. In the questionnaire, more than half of the subjects pointed out the difference in cutting feeling between the two simulators. In addition, some commented that not being able to place the finger rest on the adjacent tooth during cutting made preparation difficult. These differences in hand manipulation during cutting may have been the reason of difference in students' performance between the simulators. In addition, it has been reported that there are individual differences in the depth perception ability of stereoscopic images in virtual reality; some people is insensitive to binocular retinal disparity resulting defective stereopsis.22, 23, 24 In this study, one-fifth of the subjects pointed out that it was more difficult to grasp the depth in the haptics simulator. Therefore, the difference in depth perception ability of stereoscopic images may have also affected the performance of the subjects.

In conventional simulator where the jaw model is fixed to a mannequin, the direction in which the subject can observe the dentition during cutting is limited. On the other hand, in the haptics simulator, the cutting object is a virtual reality constructed from STL data. Thus, operators can rotate the cutting object up, down, left and right on the monitor and can observe from any direction. The result questionnaire survey indicated that 10 subjects emphasized the degree of freedom in the direction of observing the object. In the result, the taper angles of the mannequin simulator were larger than the reference model, which is identical to the result of the past studys.25, 26, 27, 28 Comparing the result of the haptics simulator with that of conventional, the taper angles were significantly smaller in the mesial–distal plane, while there was no difference in the bucco-lingual plane. Considerable reason is that observing molar from the bucco-lingual directions was easier with the haptics simulator by the rotation. Similarly, the rotating operation may compensate for the disadvantage in depth perception of the haptics simulator by transforming the depth direction into width or height direction on a screen plane. Because, after such rotation, operator can perceive original depth length without stereopsis. The score difference between the haptics and conventional model were not observed in Occlusal surface form, Surface smoothness, Taper angle, and Overall impression. There is a possibility that rotating the object may compensate the difficulty of depth perception in haptics. Further investigation is needed to clarify the relation between kinds of item and individual stereoscopic ability.

In the linear mixed model, the evaluation items that showed significant interaction between the simulator and the evaluators were Occlusal surface form, Total cut volume, and Overall impression. All of the items require evaluation of the entire object from a three-dimensional perspective, while evaluation of the other items, Margin design, Surface smoothness, and Taper angle, needed to focus on narrower area. Therefore, variation of stereoscopic ability in evaluators might also have affected the scoring results as an interaction effect.

This study suggests that the unique characteristics of virtual reality, such as the simulated cutting sensation and the simulated three-dimensional images created by stereo viewers, affect operators' performance and evaluators’ perception. Therefore, it is important to develop an educational program that is conscious of the features of each simulator. The findings in this study were obtained from investigation of a limited number of subjects and evaluators. Therefore, the result should not be overly generalized. However, the findings are meaningful when considering the differences between the haptics and the conventional simulator. Further research is needed to determine how the haptics simulator can be effectively used in the educational curriculum in future.

Within the limitation of the present study, the result demonstrated that using a different simulator affected subject performance for Margin design and Total cut volume. This appeared to be due to differences in cutting feeling as well as individual differences in stereoscopic vision of subjects. Evaluators' scoring was affected by difference in simulators for Occlusal surface form, Total cut volume, and Overall impression. This may have been due to individual differences in stereoscopic vision of evaluators. Thus, the unique characteristics of virtual reality, such as the simulated cutting sensation and simulated three-dimensional images, affect operators' performance and evaluators’ perception. It was suggested that educational programs need to be constructed taking into consideration the characteristics of virtual reality to make the best use of Simodont®.

Conflicts of interest

The authors have no conflicts of interest relevant to this article.