Comparison of Digital and Paper Assessment of Smile Aesthetics Perception.

OBJECTIVES: Despite the widespread of assessment of smile aesthetic perception in many areas, there has yet to be a direct comparison of digital and paper-based photographs for the assessment of smile aesthetics. Here we compared digital and paper-based photographs representing different smile aesthetic features using visual analog scale (VAS) scoring.
MATERIALS AND METHODS: One hundred students were randomly recruited from a university campus. Participants were asked to record their perception of smile aesthetics via paper and digital-based platforms. The minimum clinically important difference between platforms was set at 15 mm. The percentage of participants who rated smile attractiveness worse on digital images was recorded. The paired one-tailed Student's t test was used to determine differences between digital and paper platforms, and Bland-Altman analysis and intraclass correlations (ICCs) were used to test for agreement between paper and digital photographs.
RESULTS: Ninety-nine subjects participated, 55 men (mean age = 22.05, standard deviation [SD] = 1.91) and 44 women (mean age 22.05, SD = 1.84). There were statistically significant differences between paper-based and digital photographs for all images except one (paired t test; P < 0.05). Digital ratings were lower than paper-based ratings for all images, and differences were clinically significant in four out of eight images. A high percentage of participants (50.5%-85.9%) rated smile attractiveness worse on digital images than on paper for all images. There was poor agreement between the two methods as assessed by ICCs and Bland-Altman analysis.
CONCLUSION: Equivalence between paper and digital images for smile aesthetics cannot be assumed, and paper-based photographs may lead to clinically relevant overestimations of perceived attractiveness. As academic dentistry increasingly relies on digital imaging and sharing in the post-COVID-19 world, further validation of digital platforms for smile aesthetics assessment is warranted, and care should be taken when interpreting the results of studies assessing smile perception based on different platforms. Copyright:

INTRODUCTION

Dental aesthetics are important in treatment planning, and several valid measures have been developed to assess the need for treatment based on aesthetics including the Aesthetic Component of the Index of Orthodontic Treatment Need Aesthetic Component (IOTN-AC)[1] and the Dental Aesthetic Index (DAI).[2] Furthermore, smile aesthetics is one of the most important parameters to evaluate when assessing patient-reported outcomes in dentistry, despite it being a subjective construct. The perception of smile aesthetics has been investigated amongst dentists, specialist orthodontists, lay people, and students.[3456] Various rating methods have been used to assess dental aesthetics, including smile aesthetics, which can be broadly categorized into those that apply rank-order scales, where the judgments are not spaced at equal intervals (such as the 5-point ordinal ranking scale used in Mackley[7]), and those based on continuous scales, such as the visual analog scale (VAS)[89] and Q-sort methods.[910] VAS have been particularly extensively used in studies of smile aesthetics,[56,9] but whether the format of the accompanying images being assessed impacts the result is unknown.

Nevertheless, traditionally administered in paper format, smile aesthetics are increasingly being assessed in digital formats, where photographs of the subject are presented on a computer screen or handheld device and aesthetics are scored using VAS. VAS are psychometric instruments used to measure subjective endpoints such as characteristics, feelings, or attitudes, and they have been applied to many medical disorders (e.g., pain, tinnitus), market research, and social science investigations.[11] In a VAS, scores are usually recorded by the subject marking a 100 mm line that represents a continuum between two extremes of feeling denoted by verbal descriptions such as “least attractive” to “most attractive.”[111213] Digital VAS offers a number of advantages including easier, automatic, more accurate scoring, improved record keeping, and universal sharing via electronic medical records. We hypothesized that paper and digital versions of VAS may not be equivalent in terms of reporting the assessment of smile aesthetics, as the fidelity of the images presented in digital format might alter viewer perception.

There has yet to be a direct comparison of digital and paper-based assessment of smile aesthetics. To fill this gap, the VAS scores derived from digital and paper-based assessments of smile aesthetics by dental and medical students were compared.

SUBJECTS AND METHODS

SUBJECTS

This was a cross-sectional study of 100 dental students and non-dental students from the same campus at a public university in Saudi Arabia, over a period of two months. One hundred participants were selected from year 4 and 5 student registration lists using systematic random sampling. The Research Ethics Committee at the Faculty of Dentistry of the University reviewed and approved the study protocol.

DIGITAL AND PAPER-BASED ASSESSMENT OF SMILE AESTHETICS

All participants were asked to complete a web-based questionnaire [Figure 1A] and a paper-based questionnaire [Figure 1B] to rate eight color smile images using VAS. Both paper and digital questionnaires were completed by all participants one after the other. The questionnaires first presented brief information about the study and asked questions on participant gender, age, faculty, and year of study. Participants were then informed that by continuing with the survey they would be considered to have given their consent to take part. The questionnaire included a brief introduction about the length of time needed, who the investigator was, and the purpose of the study. For the paper-based format, each participant was provided with a booklet that included printed photographs, on photographic paper, of the eight smile images and was asked to view each image for twenty seconds without being allowed to return to previously rated images, as suggested in a similar study by Flores-Mir et al.[14] For the web-based format, students were provided with a tablet (Apple iPad) and were asked to rate the smile presented consecutively as digital images on separate webpages with responses captured automatically. In both formats, participants were asked to rate each image on a VAS (100 mm) from 1 (least attractive) to 100 (most attractive). The webpage was developed using a combination of JavaScript and the ASP.NET framework. No incentives were given in exchange for participation in the study. Respondents were not allowed to review or change their answers.

Figure 1

(A) Digital visual analog scale (VAS)-based platform for the assessment of smile aesthetics. (B) Paper-based format

SMILE IMAGES

The smile images were used by kind permission of Dr Hanan Omar and as published in (15). Briefly, a suitable image was first identified that, based on dental aesthetics, proportions, and gingival aspects, was deemed to be “ideal.”[16] The image was then digitally cropped to remove the chin, nose, and cheeks to show only the anterior teeth, gingivae, and lips and eliminate other factors that could affect smile perception. The image was then digitally altered using Adobe Photoshop Software to produce eight images with different smile features to test the individual smile aesthetics of reduced maxillary incisor height (image B); darker crown shade (image C); diminutive lateral incisors (image D); flattened maxillary incisor edges (image E); midline deviation (image F); diastema (image G); and increased gingival display (image H) [Figure 2]. The degree of alteration and modification were similar to a number of previous studies.[1718] The face validity of this type of image development has been tested in several studies.[1719,20]

Figure 2

Modified dental images used in either digital or paper format. (A) Model image. (B) Reduced maxillary incisor height. (C) Darker crown shade. (D) Diminutive lateral incisors. (E) Flattened maxillary incisor edges. (F) Midline deviation. (G) Diastema (H) Increased gingival display

STATISTICAL ANALYSIS

The proportion of participants scoring paper and digital images differently was expressed as percentages. VAS scores were described with means, standard deviation (SD), and 95% confidence intervals (95% CI). Both the paper-based and digital VAS scales were 100 mm long. The minimum clinically important difference was set at 15 mm (15% of total scale length), meaning that differences of ≥15% were considered clinically significant. This threshold, based on minimum clinically important differences in pain research being 9%–13% and a 15% threshold used in a study of attractiveness, has been used in a previous study of smile aesthetics.[21] Differences between groups were analyzed using the paired sample t test. To evaluate the level of agreement and test for reliability, Bland and Altman analysis[22] and intraclass correlations (ICCs) were calculated. Data were analyzed using Statistical Package for the Social Sciences (SPSS) software program, version 24.0 (IBM Statistics, Chicago, Illinois), and the significance level for all tests was set at P < 0.05.

RESULTS

Of 100 invitees, 99 subjects, 55 men (mean age 22.1 years, SD = 1.91, range 18–26) and 44 women (mean age 22.1 years, SD = 1.84, range 18–25), participated. Of these, 37 were dental students. Gender and faculty effect on the platform used were evaluated and found to be non-significant.

A comparison of the VAS scores assigned to digital and paper-based images is shown in Tables 1 and 2. The mean scores for paper-based images (range 46.6–64.9) were consistently higher than the scores assigned to digital images (range 21.5–55.7). In all cases, a few participants scored the paper and digital images the same (7.1%–16.2%), a minority assigned higher scores to digital images (7.1%–34.3%), while a majority assigned higher scores to paper-based images (50.5%–85.9%) [Table 1]. The mean paper and digital VAS scores for the eight images were significantly different for all images except image G [diastema; paired t test, all P < 0.05; Table 2]. The scoring differences of four out of eight images were clinically significant (≥15%). The results were similar according to gender in magnitude and direction.

Table 1

Mean visual analog scale scores assigned to the digital and paper-based images and the proportions of individuals scoring the two formats the same or differently

Image	Mean score digital	SD digital	Mean score paper	SD paper	No scoring difference between formats n (%)	Digital image scored higher than paper image n (%)	Paper image scored higher than digital image n (%)
A	41.72	22.41	58.28	20.00	11 (11.1)	20 (20.2)	68 (68.7)
B	30.10	22.20	53.54	20.82	16 (16.2)	13 (13.1)	70 (70.7)
C	49.90	22.79	61.72	22.09	10 (10.1)	28 (28.3)	61 (61.6)
D	40.20	21.85	46.57	22.77	15 (15.2)	34 (34.3)	50 (50.5)
E	21.52	20.02	49.70	22.34	13 (13.1)	9 (9.1)	77 (77.8)
F	29.29	22.33	64.85	23.40	7 (7.1)	7 (7.1)	85 (85.9)
G	55.66	25.08	58.48	22.74	16 (16.2)	32 (32.3)	51 (51.5)
H	50.61	25.23	58.07	22.88	7 (7.1)	34 (34.3)	58 (58.6)

SD = standard deviation

Table 2

A comparison of digital and paper-based scoring of smile aesthetics scores, including by gender

	Female (n = 44)		Male (n = 55)		Overall (n = 99)
Image	Mean difference ± SD (95% CI)	P Value (t test)	Mean difference ± SD (95% CI)	P Value (t test)	Mean difference ± SD (95% CI)	P Value (t test)
A	–20.90 ± 22.8 (–65.6, 23.8)	<0.001	–13.09 ± 28.21 (–68.4, 42.2)	<0.001	–16.57 ± 26.11 (–67.7, 34.6)	<0.001
B	–20.23 ± 30.0 (–79.0, 38.6)	<0.001	–26.0 ± 30.7 (–86.2, 34.2)	<0.001	–23.43 ± 30.41 (–83.0, 36.2)	<0.001
C	–8.11 ± 27.7 (–17.3, 1.1)	NS	–14.03 ± 27.6 (–21.0, –7.0)	<0.001	–11.8 ± 27.6 (–17.3, –6.3)	<0.001
D	–5.45 ± 26.0 (–56.4, 45.5)	NS	–7.09 ± 28.39 (–62.7, 46.6)	NS	–6.36 ± 27.23 (–59.7, 47.0)	<0.05
E	–27.95 ± 23.3 (–73.6, 17.7)	<0.001	–28.36 ± 30.05 (–87.3, 30.5)	<0.001	–28.18 ± 27.16 (–81.4, 25.1)	<0.001
F	–40.0 ± 27.2 (–92.9, 12.9)	<0.001	–32.0 ± 30.57 (–91.9, 27.9)	<0.001	–35.56 ± 29.28 (–92.9, 21.8)	<0.001
G	–4.32 ± 27.0 (–57.2, 48.6)	NS	–1.64 ± 32.19 (–64.7, –61.2)	NS	–2.83 ± 29.9 (–61.4, 55.8)	NS
H	–11.52 ± 29.5 (–69.3, 46.3)	<0.001	–4.22 ± 29.05 (–61.2, 52.7)	NS	–7.46 ± 29.34 (–65.0, 50.0)	<0.05

SD = standard deviation, CI = confidence interval, NS = not significant

Figures highlighted in bold represent clinically meaningful differences (≥15% difference)

To further assess the reliability of the two methods for assessing smile aesthetics, the ICCs [Table 3] and Bland–Altman plots [Figure 3] were evaluated. ICCs between digital and paper ratings were all less than 0.41 (P < 0.05) for the entire cohort. Bland–Altman analysis further revealed poor inter-method agreement, with image F (midline deviation) showing the greatest difference of –35.56 ± 29.28 and limits of agreement (95% CI) ranging from –92.9 to 21.8 [Table 2].

Table 3

Intraclass correlations between digital and paper-based visual analog scale scores for each smile esthetic image

Image	ICC	Lower bound	Upper bound	P Value
A	0.392	0.95	0.592	<0.01
B	0.003	–0.485	0.330	NS
C	0.281	–0.071	0.517	NS
D	0.407	0.117	0.602	<0.01
E	0.305	–0.034	0.534	<0.05
F	0.305	–0.034	0.534	<0.05
G	0.361	0.048	0.571	<0.05
H	0.410	0.121	0.604	<0.01

ICC = intraclass correlation, NS = not significant

Figure 3

Bland–Altman plots for each image assessed by digital and paper-based visual analog scale (VAS) scoring

DISCUSSION

This study was the first direct comparison of digital and paper-based images using VAS scoring of smile aesthetics. The results show that there was poor agreement between digital and paper-based methods as assessed by quantitative differences between the assigned scores and two reliability measures (ICCs and Bland–Altman analysis). Half of the observed differences between methods were of sufficient magnitude to be deemed clinically significant. Overall, participants always assigned lower VAS scores to digital images than paper-based images, suggesting that the use of paper-based images might lead to overestimation of perceived attractiveness.

Previous comparisons of paper and digital VAS scoring for other subjective assessments not relying on visualizing images, such as pain, have shown good agreement between modalities. In a recent study, Delgado et al.[23] compared pain scores recorded using paper-based VAS and digitally using both a laptop and mobile phone in 100 patients presenting with pain as their main complaint and reported no clinically significant differences between paper-based VAS assessment and VAS scores obtained on laptops or mobile phones. While a difference was detected between paper-based and mobile phone-based VAS scores, this difference was small, not clinically significant, and attributed to the size of the measurement entity (the finger) relative to the platform (mobile phone), which was suggested to have reduced the specificity. In their assessment of 33 patients with upper extremity injuries, Sindhu et al.[24] found excellent and identical test–retest reliabilities (r = 0.96) of digital touchscreen and paper-based VAS, indicating that participants scored their pain almost identically. Jamison et al.[25] performed a randomized, cross-over trial of 24 individuals receiving cognitive (descriptions of pain) and sensory (lifting test weights) stimulation and found extremely high correlations (r = 0.99 and r = 0.98) for both stimuli between palmtop device and paper-based VAS formats. Finally, Kreindler et al.[26] assessed mood in 28 subjects using a VAS-based questionnaire using a handheld computer and differently sized paper-based formats and detected no loss of precision or accuracy of the mood scale data. These data have supported the use of VAS-based pain assessment in clinical practice. While the minor differences between different platforms noted in the previous studies may also apply here, the large differences seen between digital and paper-based scoring of smile aesthetics suggest that other factors contribute to poor platform agreement when the subjective assessment is visual.

Therefore, the poor agreement between the two methods observed here may have been due to several factors related to the VAS itself or the presentation of smile images in digital and paper-based formats. Regarding the VAS, similar to Delgado et al.,[23] the nature of the interaction with a paper VAS and our digital VAS format may have altered the specificity of the VAS, such as through the interaction with the slider or the duration of looking at the image. With respect to differences in perception of smile images presented as hardcopy photographs or as digital images, it has been shown that the intrinsic properties of an image, such as color homogeneity, can alter the visual perception of attractiveness.[27] Therefore, if such intrinsic properties were more or less visible in a particular format, this might have introduced bias to explain the differences between the two formats. For instance, if the hue, tint, tone, and shade differed between digital and paper images, this might have given rise to different perceptions of aesthetics of tooth color, which are known to differ according to age and gender. Younger individuals tend to value whiter shade teeth more than older individuals[2829] and, in our population, lighter teeth were preferred by female subjects compared to male subjects,[30] suggesting that there may also be ethnic biases in smile esthetic perception. Furthermore, it has been shown that image quality can alter the perception of smile aesthetics, with smiles rated as more attractive in higher-resolution images.[31] Other properties of the formats that might have influenced perception include the size and print quality of the paper-based images. Although it is unclear exactly which properties of the paper or digital images may have influenced smile perception in this study, the properties of the images used when assessing aesthetics digitally may make a difference and must be carefully considered, particularly if comparing the results across studies. Given that the factors influencing perception of smile aesthetics across platforms are unknown, further research is warranted in this area.

This study has some limitations. First, this was a relatively small study of medical and dental students at a single institution, so the results may not be generalizable to other populations. The differences between the two image formats (e.g., hue, saturation, intensity) that might have accounted for how participants differentially viewed the digital and paper-based images could not be objectively quantified; future studies should aim to identify the factors that affect the validity of this perceptual test. Finally, we did not adopt a crossover study design, so the participants may have experienced fatigue or changed their scores after seeing a number of images, which may have introduced systematic bias and may account for the poor inter-method agreement observed.

In conclusion, this study shows that equivalence between paper and digital images in the assessment of smile aesthetics cannot be assumed. As academic dentistry is increasingly relying on digital imaging and sharing for assessment purposes, this study has practical implications for how students, faculty, and clinicians perceive the outcomes of clinical work depending on the modality used. The findings also have implications for researchers validating digital tools in outcomes-based research. Further evaluation of digital platforms for smile aesthetics assessment is warranted, and care should be taken when interpreting the results of studies based on different platforms.

ACKNOWLEDGEMENT

The author would like to thank Dr. Mustafa Elhusain for his assistance in data collection.

FINANCIAL SUPPORT AND SPONSORSHIP

Not applicable.

CONFLICTS OF INTEREST

There are no conflicts of interest.

AUTHOR CONTRIBUTIONS

SHA designed, conceived and conducted the study, performed data analysis, and wrote the manuscript.

ETHICAL POLICY AND INSTITUTIONAL REVIEW BOARD STATEMENT

Not applicable.

PATIENT CONSENT STATEMENT

The authors certify that they have obtained all appropriate participants’ consent forms. Data was collected anonymously. No clinical or personal information were obtained from any participants.

DATA AVAILABILITY STATEMENT

Not applicable.