Development of the Submental Nasal Appearance Scale for the Assessment of Repaired Unilateral Complete Cleft Lip: A Pilot Study.

OBJECTIVE: To develop the "Submental Nasal Appearance Scale" (SNAS), which is an easy-to-use objectified tool to represent a cleft surgeon's standard for assessment of the nasal appearance from the submental perspective.
DESIGN: Eighty-five photographs of patients with unilateral complete cleft lip and palate were selected and cropped, displaying the submental view. Sixty-one photographs were used to develop 5 sets of reference photographs. Three cleft surgeons graded 24 photographs with these sets and subjectively graded the overall nasal appearance as well. Internal agreement for both methods was calculated, as well as correlation between them. The SNAS was created, by only using the combination of sets that showed the highest reliability and correlation.
SETTING: Boston Children's Hospital, Boston, Massachusetts. PATIENTS: Six- to 9-year-old patients with unilateral complete cleft lip and palate.
RESULTS: The intrarater and interrater reliability was 0.84 and 0.79, respectively, for the SNAS and 0.76 and 0.62, respectively, for the overall appearance assessment. The correlation was 0.74 between the methods.
CONCLUSIONS: The SNAS is a reliable tool that reflects a cleft surgeon's standard and could be used independently or in combination with existing rating scales using the frontal and/or lateral view, for assessment after cleft lip repair.

Introduction

The goal of unilateral cleft lip repair is to create nasolabial symmetry and to correct the deformed nasal structures (Mulliken and LaBrie, 2012). Traditionally, appearance of the lip and nose is assessed by scoring frontal and lateral 2-dimensional (2D) photographs (Saxby and Palmer, 1986; Asher-McDade et al., 1991; Cussons et al., 1993; Tobiasen and Hiebert, 1994; Mosmuller et al., 2013). Nasal, rather than labial, deformities call more attention to facial asymmetries (Sundine and Phillips, 2004; He et al., 2009), while the nose was scored as the least satisfactory structure when rating facial appearance, both by the patients and by professionals, parents, and laypeople (Gkantidis et al., 2013). Given the importance of the nasal appearance, the evaluation of outcomes following cleft lip repair must be comprehensive. Frontal and/or lateral views are insufficient to grade the nose in all dimensions. A new tool focused on the submental view would enable complete 3-dimensional (3D) evaluation of the nasal appearance, when used in combination with lateral/frontal validated scales, such as the Asher-McDade Scale or the Cleft Aesthetic Rating Scale (Mosmuller et al., 2017).

Many studies emphasize the importance of the submental view, characterizing this view as the most difficult to perfect (Pigott, 1985; Rubin et al., 2015; Mosmuller et al., 2017; Mulder et al., 2018).

Although this view does not express a person’s whole nasolabial attractiveness, since the basal perspective is rarely shown in social circumstances, yet important nasal structures, such as the columella and ala, are optimally visualized in this view and could be important when comparing surgical techniques or surgeons and cleft teams.

As expert opinion is still the standard when grading operative outcome of cleft lip repair, this study aimed to identify nasal characteristics that are predictive for a cleft surgeon’s standard, after which a novel 2D photographic rating method is described for assessment of the submental view.

Methods

All participants in this study were treated in the Craniofacial Center at Boston Children’s Hospital (BCH), United States or at the Amsterdam UMC, location VUmc, Amsterdam, Netherlands.

Patient Selection

In this study, patient photographs were obtained from 2 databases. Sixty-one suitable photographs of participants taken by one surgeon during clinic visit in BCH were drawn out of the database from BCH, and 24 photographs were drawn from the VUmc-affiliated Academic Center for Dentistry Amsterdam (ACTA) database where the patient images were stored. A professional medical photographer imaged these patients. All included patients were 6- to 9-year-old and nonsyndromic. Only patients with a unilateral complete cleft lip and palate (UCCLP) were included to form a uniform group. Patients from BCH underwent UCCLP repair by one surgeon. In infancy, these patients were all treated with dentofacial orthopedics using a Latham device, followed by nasolabial adhesion at the age of 3 to 4 months and modified rotation advancement repair 2 months after the nasolabial adhesion (Mulliken and Martinez-Perez, 1999; Stal et al., 2009). The participants from VUmc were repaired by 3 different cleft surgeons also using a modified Millard rotation advancement procedure only.

The specific age range was chosen, because children visit cleft clinic routinely in this period to evaluate progress and future treatment. In most cases, patient images are documented during these visits. Moreover, in 6- to 9-year-olds, the nasal morphology remains stable (Farkas, 1994). All photographs met the following inclusion criteria: (1) medial canthi perpendicular to the midline, (2) nasal tip positioned between the medial canthi and the medial eye brows in the transverse plane, and (3) neutral facial expression. All ethnicities were included. Exclusion criteria were: syndromic diagnosis, photographs taken after alveolar bone grafting, poor quality images, or patients with prominent labial scarring and nostril debris.

Photographs were displayed as being left-sided clefts, and the computer program SymNose (Pigott and Pigott, 2010) was used to automatically set the photographs in the exact transverse plane. The images were adjusted to reduce lighting differentials and cropped to only include the submental view of the nose and both the canthi.

Scale Development

Phase I

Five nasal features were defined as the key nasal structures for assessment in the submental view. These features were determined by consensus of 3 plastic surgeons with their practice focused on the care of cleft lip and palate. The following features were chosen (Figure 1): (1) nasal outline, (2) alar base position, (3) nostril outline, (4) nostril axis, and (5) columellar angle. The alar base position and the columellar angle were previously highlighted in the literature as being important structures in nasal appearance evaluation (Sandor and Ylikontiola, 2006; Fisher et al., 2008; Mommaerts and Nagy, 2008; He et al., 2015).

Figure 1.

Five nasal features assessed by raters in the development of the Submental Nasal Appearance Scale (SNAS).

Phase II

The 61 submental photographs from the BCH database were scored by 2 plastic surgeons and a cleft care dedicated orthodontist. None of the raters were involved in the care of the patients displayed on the 61 images. The 3 raters scored each nose according to the 5 key features. They assessed the degree of symmetry of the first 4 features by grading the difference between the cleft and the noncleft side. The fifth feature (columellar angle) was assessed by grading the degree of deviation using an imaginary vertical midline representing a perpendicular columella. All features were scored using a Likert scale with the following classification: 1 = excellent, 2 = good, 3 = fair, 4 = poor, and 5 = bad. The scores obtained from the 3 raters were averaged.

After this rating session, 5 photographs with the highest interobserver and intraobserver scores were selected under consensus to represent each Likert category for each of the 5 nasal key features. This resulted in 5 sets of reference photographs (Figure 2).

Figure 2.

Five sets of reference photographs for each nasal feature.

Phase III

This phase describes the comparison between the photographic sets to grade symmetry and a scale to grade the overall appearance. Four series containing the 24 ACTA photographs were made: series A1, A2, B1, and B2. All these series consisted of the same 24 photographs, however all placed in random sequence. The ACTA photographs were used since the greater part of the suitable BCH photographs was already used to create the photographic reference sets. Another advantage of using the ACTA images is that it would exclude the chance of any patient recognition for the raters to occur. All series were assessed by the same 3 cleft surgeons (P1, P2, and P3) who initially defined the 5 nasal key features.

First, series A1 was assessed on overall appearance of the submental view, using the following scale: 1 = excellent, 2 = good, 3 = fair, 4 = poor, and 5 = bad. Next, each nose in series A2 was assessed according to the 5 nasal features using the photographic sets. Series B1 and B2 were assessed following the same protocol after a 2-week interval to reduce memory bias (De Vet et al., 2011).

The mean intrarater reliability (Table 1) for the 3 raters was calculated between occasion 1 (A1: Overall appearance scale) and occasion 2 (B1: Overall appearance scale). Next, the mean intrarater reliability for the 3 raters was calculated between occasion 1 (A2: All possible combinations of photographic sets) and occasion 2 (B2: All possible combinations of photographic sets). Subsequently, the mean interrater (Table 2) reliability between the 3 raters was calculated for both series on occasion 1 (A1: Overall appearance scale and A2: All possible combinations of photographic sets). At last the mean correlation (Table 3) for the 3 raters was calculated between both series on occasion 1 (A1: Overall appearance scale and A2: All possible combinations of photographic sets). The mean values calculated from the used combinations were used to determine the reliability and correlation scores.

Table 1.

Intrarater Reliability.

Mean ICC overall appearance-scalea = 0.76
Mean ICC’s for each possible combination of the photographic setsb
I	0.69	I + IV	0.82	IV + V	0.63	II + III + IV	0.79	I + III + IV + V	0.76
II	0.76	I + V	0.62	I + II + III	0.80	II + III + IV	0.69	II + III + IV + V	0.74
III	0.58	II + III	0.81	I + II + IV	0.84c	II + IV + V	0.69	I + II + III + IV + V	0.80
IV	0.68	II + IV	0.76	I + II + V	0.77	III + IV + V	0.66
V	0.61	II + V	0.51	I + III + IV	0.79	I + II + III + IV	0.83
I + II	0.69	III + IV	0.72	I + III + V	0.69	I + II + III + V	0.77
I + III	0.73	III + V	0.53	I + IV + V	0.76	I + II + IV + V	0.80

Abbreviations: ICC, intraclass correlation coefficient; I, nasal outline symmetry; II, alar base position symmetry; III, nostril outline symmetry; IV, nostril axis symmetry; V, columellar angle.

a Calculated from rater P1, P2, and P3, between occasion 1 (A1) and 2 (B1).

b Calculated from rater P1, P2, and P3, between occasion 1 (A2) and 2 (B2).

c Highest mean intrarater reliability obtained with combination I + II + IV.

Table 2.

Interrater Reliability.

Mean ICC overall appearance scalea = 0.62
Mean ICC’s for each possible combination of the photographic setsb
I	0.66	I + IV	0.80	IV + V	0.46	II + III + IV	0.76	I + III + IV + V	0.73
II	0.56	I + V	0.62	I + II + III	0.77	II + III + IV	0.61	II + III + IV + V	0.67
III	0.66	II + III	0.71	I + II + IV	0.79	II + IV + V	0.62	I + II + III + IV + V	0.77
IV	0.65	II + IV	0.74	I + II + V	0.66	III + IV + V	0.55
V	0.34	II + V	0.49	I + III + IV	0.83c	I + II + III + IV	0.82
I + II	0.43	III + IV	0.70	I + III + V	0.71	I + II + III + V	0.73
I + III	0.81	III + V	0.42	I + IV + V	0.69	I + II + IV + V	0.73

Abbreviations: ICC, intraclass correlation coefficient; I, nasal outline symmetry; II, alar base position symmetry; III, nostril outline symmetry; IV, nostril axis symmetry; V, columellar angle.

a Calculated between raters P1, P2, and P3 on occasion 1 (A1).

b Calculated between raters P1, P2, and P3 on occasion 1 (A2).

c Highest mean interobserver reliability obtained with combination I + III + IV.

Table 3.

Kendall’s Tau Correlation.

Mean Kendall’s Tau correlation between the overall appearance scale and each possible combination of the photographic setsa
I	0.55	I + IV	0.72	IV + V	0.41	II + III + IV	0.70	I + III + IV + V	0.68
II	0.64	I + V	0.41	I + II + III	0.67	II + III + IV	0.62	II + III + IV + V	0.56
III	0.64	II + III	0.66	I + II + IV	0.74	II + IV + V	0.65	I + II + III + IV + V	0.72
IV	0.53	II + IV	0.72	I + II + V	0.59	III + IV + V	0.56
V	0.24	II + V	0.53	I + III + IV	0.73b	I + II + III + IV	0.73
I + II	0.63	III + IV	0.66	I + III + V	0.58	I + II + III + V	0.65
I + III	0.67	III + V	0.48	I + IV + V	0.59	I + II + IV + V	0.71

Abbreviations: I, nasal outline symmetry; II, alar base position symmetry; III, nostril outline symmetry; IV, nostril axis symmetry; V, columellar angle.

a Calculated from raters P1, P2, and P3 on occasion 1 (between A1 and A2).

b Highest correlation between overall appearance scale and combination I + II + IV.

Phase IV

In this last phase, the combination of key features (photographic sets) that showed the highest reliability and correlation scores was selected to generate the new assessment tool.

Statistical Analysis

Intra- and interrater reliability was evaluated for the scale to grade the overall appearance and the new defined sets of reference photographs. Reliability scores were analyzed using the intraclass correlation coefficient (ICC; 2-way random model with absolute agreement). Values approaching the upper limit of ICC = 1.0 indicate a high degree of agreement. A correlation coefficient of 0 to 0.30 is considered a negligible or very poor correlation, from 0.30 to 0.50 a low or poor correlation, from 0.50 to 0.70 a moderate correlation, from 0.70 to 0.90 a high correlation, and above 0.90 is considered a very high correlation (Hinkle et al., 2003).

The correlation between the scale to grade the overall appearance and the photographic sets was determined by using a Kendall’s Tau rank correlation coefficient. Similar classification values are used for this coefficient (Hinkle et al., 2003). This coefficient evaluates the degree in similarity between 2 instruments that measure the same set of objects.

Power analysis was performed with the aid of the computer program G*power to determine all sample sizes of photographs and raters that were required to obtain a minimum intra- and interrater reliability score of 0.70. Statistical program IBM SPSS 23.0 was used for all data analysis.

Results

The BCH photographs used for creating the sets of reference photographs consisted of 40 (65.5%) boys and 21 (34.5%) girls. Twenty-three (37.2%) patients had a right-sided and 38 (62.8%) had a left-sided cleft. The photographs from ACTA consisted of 20 (83.3%) boys and 4 (16.7%) girls, of whom 19 (79.2%) had a left-sided and 5 (20.8%) had a right-sided cleft.

Table 1 illustrates the mean intrarater reliability score obtained with the scale to grade the overall appearance and the mean scores for all possible combinations of the 5 nasal features. High intrarater reliability (ICC: 0.76) was found for the overall appearance scale and the highest intrarater reliability (ICC: 0.84) was found for combination I, II, and IV.

The interrater reliability scores between the 3 raters are shown in Table 2. Moderate interrater reliability (ICC: 0.62) was obtained with the scale to grade the overall appearance and the highest interrater reliability (ICC: 0.83) was obtained with combination I, III, and IV.

The correlation scores between the scale to grade the overall appearance and all possible combinations of nasal features are shown in Table 3. The highest correlation (ICC: 0.74) was obtained with features I, II, and IV combined.

Based on the highest intrarater and correlation scores, the following combination of nasal features was eventually chosen to generate the new assessment tool: (I) nasal outline symmetry, (II) alar base position, and (IV) nostril axis symmetry. These 3 features form the base of the Submental Nasal Appearance and Symmetry (SNAS) scale (Figure 3).

Figure 3.

Submental Nasal Appearance Scale (SNAS) demonstrating the nasal features included and the reference photographs selected.

Discussion

The purpose of this study was to develop a simple and highly reliable method to assess the submental view in patients with repaired UCCLP and to identify the nasal characteristics that predict a cleft surgeon’s standard. Nasal outline-, alar base position-, and nostril axis-symmetry combined were found to be most reliable and most predictive for a cleft surgeon’s view on overall appearance. Although, the highest interrater reliability was found for combination I, III, and IV, still high interrater reliability was found for combination I, II, and IV. Therefore, we chose to use combination I, II, and IV. The high reliability scores obtained with the SNAS indicate that this method is sufficient to reliably grade submental appearance. Furthermore, the SNAS could be considered a more accurate and objective method compared to subjective overall appearance assessment. The high Kendall’s Tau correlation confirms that the SNAS reliably could reflect a cleft surgeon’s standard.

The SNAS is applicable for the assessment of 2D submental photographs of 6- to 9-year-old patients and can be used separately or in combination with existing rating scales that rely on the frontal and/or lateral view such as the Cleft Aesthetic Rating Scale (Mosmuller et al., 2017). Images need to be set in exact transverse plane, and the estimated time to report on one participant is less than 30 seconds.

The reference photographs function as a guide to assess symmetry for each of the 3 nasal features. The numerical mean score calculated from grading the 3 nasal features should be applied on the following classification system: 1.0 to 1.8 = excellent; 1.8 to 2.6 = good; 2.6 to 3.4 = fair; 3.4 to 4.2 = poor; and 4.2 to 5.0 = bad. This classification is proposed, since the 3 nasal features used in the SNAS are also scored according to a 5-point scale. In order to refer to a 5-point scale in similar fashion, yet this time with mean values, this proposed system was subdivided in proportional categories with numerical value “3” acting as the central “fair” category.

It is known that facial attractiveness can reliably be judged by human preference (Tobiasen and Hiebert, 1994; Ritter et al., 2002). Rating facial appearance subjectively is still cumbersome and there are no current guidelines on rating the appearance in a quick and reliable way, particularly for the nose. Adding objective elements, such as additional rules, simplifies the task and result in reliability improvement (Mosmuller et al., 2017). In the current study, the individual nasal features with corresponding reference photographs will act as objective elements to maximize the information gathered by its use and form the overall appearance score. A close resemblance of this principle is described by Fisher et al. (2008) in preoperative cases. The authors investigated which objective anthropometric measurements correlated with subjective assessment from the basal view. They identified that measures of the columellar angle and the nostril width ratio correlated with the subjective ranking. The columellar angle alone was not highly predictive in our study, as it showed poor mean correlation (0.24). This could be addressed to the fact that Fisher et al. (2008) assessed the primary deformity and could not be compared with postoperative cases in our opinion. The approach of using photographic rank evaluation and identifying objective elements to increase reliability was administered in the current study.

Advantages of 3D media are plentiful in the literature (Al-Omari et al., 2005; Sharma et al., 2012; Mosmuller et al., 2013). Nevertheless, this methodology is expensive, time-consuming, limited in availability, and it requires training for assessors (Mosmuller et al., 2013). More importantly, 3D photographs may show the nostril outline unclearly and do not show full nostril depth rendering the submental view difficult to judge (Stebel et al., 2016).

Two-dimensional photographic rating scales have been developed using the submental view for the assessment in patients with repaired UCCLP. He et al. (2009) used a combination of the submental, frontal, and lateral view. The highest reliability value (ICC = 0.73) was obtained using all 3 views and a panel of 7 raters. The authors found higher reliability scores when the upper lip was covered, suggesting ratings are more reliable when focused on the nasal complex only.

Rubin et al. (2015) developed a 2D photographic basal scale with the intention to facilitate use of the original Asher-McDade Scale. They defined a set of reference photographs for each point on the ordinal Asher-McDade Scale for 2 of the 4 nasal features (nasal form and nasal symmetry). Intrarater (mean ICC = 0.94) and interrater (mean ICC = 0.68) reliability showed variation and they did not validate the final defined set of photographs.

In the study of He et al. (2009), left- and right-sided noses were mixed in the set of reference photographs. This might confound the rating task. There should be no confusion in assessment with right-sided clefts with the method presented in this study, since the degree of symmetry between the cleft-side is compared to the noncleft side; therefore, we advise mirroring the rating scale for the assessment of right-sided clefts.

Limitations of the Study

Difficulty in obtaining standardized photographs limits the availability of images for scale development. To mitigate this problem in the development of the SNAS, only BCH photographs were used taken by one surgeon who used a strict protocol when imaging his patients. The authors had the option to choose from multiple potential reference photographs representing the numerical values of the 5-point scale of each category in order to create the new SNAS scale. For simplicity, only reference photographs that represented the most common outcomes in nasal appearance and the degree of severity were included. Outlying deformities were not included in the reference photographs, such as a medially malpositioned alar base that resulted in a smaller nostril or an inverted nostril axis. When the degree of symmetry between both sides of the nose was assessed, a score could still be given to these outliers.

In the selection of the reference photographs, images with a mean score closest to 1 to 5 and the least variance in score were chosen. Therefore, images with 100% interrater agreement were automatically qualified to serve as potential reference photographs. For Likert classifications, 2, 3, and 4 photographs with 100% interrater agreement were available. For classification 1 and 5, photographs with 100% interrater agreement were unavailable. Thus, photographs selected in this study for excellent and bad results may underrepresent the spectrum of deformity.

A successful outcome is not solely defined by nasal symmetry (Springer et al., 2007). Nasal form and proportion relative to the face are important factors influencing the appearance of the nose (He et al., 2009). These factors are not addressed or assessed with the SNAS scale.

According to Kuijpers-Jagtman et al. (2009), a limitation of the submental view is that it is rarely visible in social circumstances and therefore less important in assessing outcomes. We submit that this view is important, since significant nasal structures are visible from this point of view, such as the columellar angle and the nostrils. These elements can effortlessly be assessed by means of symmetry. A frequently heard comment on the Asher-McDade Scale and the Cleft Aesthetic Rating Scale is that the basal view is not included in these methods. Therefore, combining the submental view with existing frontal, and lateral view methods, would enable multidimensional evaluation of the nose.

The SNAS is appropriate for assessment of photographic material that is available in almost every cleft lip and palate center. Since the SNAS is an easy and quick-to-use method, retrospective intercenter analyses of postoperative results can be easily made, as well as comparison of surgical results and different techniques. Another application for the SNAS is that it could be used as an educational tool for both parents and patients, since photographs included in the scale can be used as reference material. Showing patients and parents these reference photographs could give them understanding and an impression of possible future outcomes, especially after secondary corrections. Once, multiple results (scores) are obtained after corrective surgery using the SNAS within a cleft center, the mean score and the corresponding photographs could be shown to patients and family, which is valuable in expectation management.

To further determine its validity and its applicability, the SNAS should be validated externally and comparison to current 2D and even 3D rating and measurement methods should be made. Testing of the tool for its generalizability and application with other professional groups and even less experienced raters involved in cleft care need to be undertaken, since the SNAS scale was developed and tested by highly experienced cleft surgeons. Reliability for direct clinical assessment needs to be investigated, as well as applicability of the tool in different age groups. We suggest developing a SNAS for 18-year old patients separately, since nasal characteristics will significantly change as the majority of this group underwent their last secondary corrective surgical procedure. As a result, comparison over time could be made. Finally, similar future rating scales could be developed for the different forms of cleft phenotypes, such as bilateral cleft lip.

Conclusion

The novel SNAS exhibits high overall reliability scores. This new method can be used for quick assessment to compare nasal outcomes between cleft centers and surgeons. The scale is able to reflect a cleft surgeon’s standard and is appropriate to quickly assess large 2D photographic caseloads. The SNAS would be most useful in combination with other frontal/lateral 2D rating methods to enable a multidimensional assessment of nasal appearance. Future studies are needed to further validate the scale and its applicability.