Development and Validation of the Rhinoplasty Outcomes Evaluation (ROE) Questionnaire: An Analytical Study.

BACKGROUND: Rhinoplasty Outcome Evaluation (ROE) is an easy-to-use questionnaire that allows comprehensive assessment of rhinoplasty-related patient satisfaction. However, the normal values for this questionnaire are not known. Therefore, we aim to validate the ROE questionnaire adapted to Iranian culture.
METHOD: In this cross-sectional descriptive study, the statistical population consisted of applicants for cosmetic surgery referred to Shahid Rajaee Hospital, Shiraz, Iran, in the autumn and winter of 2017. Two hundred individuals participated in this research by a convenience sampling method. The questionnaire (ROE) was translated to Persian and backward translated to English by independent medical extern Persian speakers with complete English proficiency. The data were analyzed using SPSS software version 23 using exploratory factor analysis.
RESULTS: The findings showed that the Cronbach's Alpha of composite reliability (CR) and average variance extracted (AVE); overall, values above 0.4 were favorable in this measure. In addition, the AVE ranged from 0.50 to 0.59, which confirmed convergent validity. The AVEs of each factor was higher than the squared correlations and confirmed discriminant validity within the constructs. In the presence of significant factor loadings and composite reliability greater than 0.70, convergence validity was confirmed. Furthermore, the higher AVEs for each factor were compared to the squared correlations to confirm discriminant validity.
CONCLUSION: The Iranian version of ROE is a valid instrument to assess results in rhinoplasty patients.

INTRODUCTION

All aesthetic procedures, including rhinoplasty, are very complex. However, regardless of the satisfactory results achieved with different surgical techniques, the surgeon’s choice should be determined by the anatomical characteristics of the nasal skeleton, nasal obstruction, and the patient’s skin type. The rhinoplasty surgery (RP) must consider the specific features of each patient’s skin, cartilage, and bones. It is important to examine and address anatomic variations associated with pathological conditions before considering rhinoplasty. The nasal skeleton, including its bones and cartilage, must be evaluated carefully [1]. Nasal obstructions typically occur because of anatomic factors such as external nasal deformity, septal deviation, turbinate hypertrophy, and nasal tip ptosis [2],[3]. In addition, it is essential to identify the exact site of obstruction in the preoperative setting by carefully examining the anatomical areas that involve the extra nasal and internal nasal valves. It has primarily studied rhinoplasty from the standpoint of aim parameters such as techniques, complications, anthropometric measurements, etc. [3]-[4]. It is important to evaluate success with outcomes studies, an aspect of research in this area that is often overlooked [5].

In designing several questionnaires for assessing patient satisfaction with facial aesthetic surgery, Alsarraf used this philosophy as a basis [6]. The researchers considered the following reasons for patient satisfaction: physical factors regarding nasal shape and function; emotional factors regarding confidence and desire to change appearances; and social factors regarding acceptance by colleagues, friends, and family [6]. In general, aesthetic therapies take place rarely in academic settings, so ease of use and comfort in using these questions remained highly considered when designing them, allowing them to work in private practice [7]. Alsarraf’s questionnaire series includes four distinct formats, each of which is customized for a specific facial surgical procedure. Rhinoplasty outcomes are determined by the ROE (Rhinoplasty Outcomes Evaluation) is composed of six questions, two for each of the factors considered vital to patient satisfaction (physical, emotional, and social) [6].

ROE is a questionnaire developed to evaluate the results of rhinoplasty surgery. This questionnaire contains six questions (two for each factor physical, emotional, and social dimension of patient satisfaction). Besides the original English language, the authors have translated several other languages into the ROE, including Brazilian-Portuguese, Arabic, and Turkish [8]-[10]. Comparisons of new versions of questionnaires and those published in the literature require careful adaptation; simple translations do not suffice. Newly developed questionnaires need to be validated. Because of its popularity and utility in this study, we translated the ROE into Iranian and validated the new version for Iranian patients. For this questionnaire, we will consider three main factors to determine whether it satisfies a patient after rhinoplasty: satisfaction with nasal shape and function; the level of social, familial, and professional acceptance; and the level of confidence and desire to change nasal shape [10].

As a general rule, RP patients are less satisfied with their results than those of other facial aesthetic procedures [6],[7], [11]. Besides the social environment, education level, work experience, and expectations, patient satisfaction with the RP are influenced by the patient’s expectations, which may differ significantly from the surgeon’s [8]. In addition, although most patients place more importance on postoperative function, others place equal emphasis on aesthetics and function. It is imperative to understand the patient’s expectations before surgery to achieve satisfactory results. Where the patient is not aware of the limits of RP, satisfaction between the two will be out of alignment. An evaluation of the outcome of surgical success will be difficult [10].

For aesthetics, it can be complicated to determine what makes up normality. To determine whether surgery is needed, surgeons and patients use subjective assessment instead of aim criteria. By using questionnaires, we can measure this personalized assessment more effectively. The questionnaire results should thus be clinically relevant, e.g., they should be interpreted as “sick or not sick”, or “normal or altered”. It is possible to determine clinical relevance by determining a normality parameter - the threshold at which a questionnaire is considered normal or altered. These parameters can help determine whether surgery is necessary or determine how swiftly a patient has recovered.

We aimed to translate, develop, and adapt the ROE questionnaire into Iranian to establish a normality parameter.

METHODS

In this descriptive study, the statistical population included cosmetic surgery applicants referred to Shahid Rajaee Hospital, Shiraz, Iran in the fall and winter of 2017. They had a minimum education degree (Diploma), were under 25, between 25 and 35, and were older than 35 years old. Two hundred individuals who participated in this research were selected through the convenience sampling method. There were no specific exclusion criteria other than the reluctance to participate in the research. For the subjects’ participation in the study, the test was explained and written consent was obtained.

The questionnaire was anonymous, only age, gender, education, occupation, and marital status were compulsory among the demographic variables. All the questionnaires completed were valid based on the questionnaire validity and used to analyze the data. In the first place, the researchers used addresses and telephone numbers to gather the necessary information about the participants. After that, they were asked to refer to the hospital to complete the questionnaires during the determinant’s time after presenting the explanation and purpose of the research to the participants in the research. Finally, 200 people collaborated in this way. The hospital clinic was referred for periodic visits by a number of those who participated. For this reason, it took about 3 months to complete the questionnaires. Then, the data were analyzed with SPSS software version 23 using exploratory factor analysis. Meanwhile, all the moral points were met according to Helsinki rules. Additionally, informed consent and ethical considerations were obtained from the applicants. This article has been ethically approved (license number 3158178/98). It was registered in Shiraz, Iran on February 6, 2020.

The ROE questionnaire has six questions, each one with five answer options, graded from zero to four. Therefore, the questionnaire score may vary between zero and 24. To make understanding easier, the scores got must divide by 24 and multiplied by 100, which leads to a score varying between zero and 100, and the higher the score, the greater the patient’s satisfaction with the nose surgery. Alsarraf test [6], was first used to prepare and test an internal consistency and validity reliability questionnaire for multiple plastic surgeries, including rhinoplasty. In 2014, Izu et al. confirmed the Brazilian ROE questionnaire [12]. The number of questions in this questionnaire seemed to be insufficient to understand patients’ satisfaction or dissatisfaction, according to the experts’ opinion. Therefore, experts decided that besides the number of questions in the questionnaire according to the professors in this field, other cases would add to a better understanding of patient satisfaction. We should note that the total of six questions of the main questionnaire expanded to 13 questions. After extracting the contents of each factor, three psychologists and two rhino-plastic surgeons named the factors.

RESULTS

The construct validity of the ROE was determined in this step. For this, exploratory and confirmatory factor analysis, as well as convergent, divergent, and discriminant validity are used. The authors administered ROE to 200 patients within the age range of under 25 to higher 35 years (Table 1).

Table 1

Demographic variables

Variables رررر	Groups	Frequency	Percentage
Gender	Female	127	63.5
	Male	73	36.5
Marital state	Single	89	44.5
	Married	111	55.5
Educational level	Diploma	78	39
	Bachelor of Art	107	53.5
	MA & PhD	15	7.5
Job state	Unemployed	20	10
	Employed	55	27.5
	Housekeeper	60	30
	self-employment	65	32.5
Age	Under25	57	27.5
	25 to 30	91	42
	30 to 35	38	19
	35 and higher	14	7

Table 2, evaluated the criteria for a factor analysis using the Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO) and Bartlett’s Test of Sphericity. The KMO was 0.739, showing the adequacy of the sample, and Bartlett’s Test of Sphericity (χ2 =449.22, df= 78, P < 0.001) showed that the factor analysis was justified.

Table 2

Kaiser-Meyer-Olkin and Kruit-Bartlett test results

KMO	Chi-square statistics	Degrees of freedom	sig
0.739	449.22	78	0.000

Before the assessment of the structural model, we examined the reliability and validity of the measurement model. Item (construct) reliability was assessed by factor loadings (Cronbach’s Alpha) values. Individual item reliabilities use loadings of the items to their respective constructs, and in their standardized form, loadings should be greater than 0.5. Cronbach’s Alpha, which analyses the consistency of the overall participants in answering the statement items of a particular variable/construct assessed the variable/construct reliability. The value of this indicator should be larger than 0.6.2 For the constructs used in our analysis (satisfaction with the nose surgery, satisfaction with the treatment staff, and satisfaction feedback), the values of Cronbach’s Alpha were 0.722, 0.723, and 0.381 respectively, as presented by Table 3, therefore confirming the reliability of the measurement model.

Table 3

Factor loads of variables of each sampling adequacy test structure

Variables	Row	Question description	Factor load
Satisfaction with the nose surgery ( 1)	1	Satisfaction with the fit of the nose in the face	0.638
	2	Satisfaction with action feedback from friends	0.670
	3	Satisfaction with family action feedback	0.625
	4	Satisfaction with action feedback from relatives	0.686
	5	Satisfaction with action feedback from the workplace or place of study	0.535
Satisfaction with the treatment staff (2)	6	Inner satisfaction with the action	0.895
	7	Satisfaction with your observation in the mirror	0.449
Satisfaction feedback (3)	8	Willing to perform restorative surgery	0.456
	9	Satisfaction of medical staff	0.555
	10	Satisfaction with the physician's preoperative knowledge	0.791
	11	Satisfaction with the length of hospital stay	0.803
	12	Satisfaction with postoperative visits by a physician	0.761
	13	Satisfaction with surgery costs	0.510

The C.R. values for satisfaction with the nose surgery, satisfaction with the treatment staff, and satisfaction feedback constructs were 0.819, 0.769, and 0.644 respectively, confirming convergent validity. The average variance extracted should be higher than the minimum threshold of 0.5 (Table 4).

Table 4

Reliability and convergent validity

Variables	(AVE>0.5)	(Cr>0.7)	(Alpha>0.7)
Satisfaction with the nose surgery (1)	0.583	0.819	0.722
Satisfaction with the treatment staff (2)	0.504	0.769	0.723
Satisfaction feedback(3)	0.543	0.644	0.381

In our analysis, the obtained AVE values for satisfaction with the nose surgery, satisfaction with the treatment staff, and satisfaction feedback constructs were 0.583, 0.504, and 0.644 respectively. When taken together with the values of composite reliability (which were higher than 0.6 for each construct), we can state that convergent validity was established (Table 4).

As presented in Table 4, convergent validity can confirm based on the Cronbach’s Alpha CR and AVE extracted; overall, values above 0.4 are favorable in this measure. In addition, the AVE ranged from 0.50 to 0.59, which confirms convergent validity. The AVE for each factor was higher than the squared correlations, thus confirming discriminant validity between the constructs. It confirmed convergent validity based on the significance of factor loadings and composite reliability of > 0.7. In addition, discriminant validity confirms the higher AVEs for each factor, compared to the squared correlations.

However, even if AVE is less than 0.644, but composite reliability is higher than 0.6, the convergent validity of the construct is still adequate (Table 5).

Table 5

Fornell and Larcker matrix

Variables	Satisfaction with the nose surgery(1)	Satisfaction with the treatment staff(2)	Satisfaction feedback(3)
Satisfaction with the nose surgery (1)	0.811
Satisfaction with the treatment staff (2)	0.736	0.809
Satisfaction feedback (3)	0.518	0.728	0.734

According to Table 6, this index has proven to be unrealistic (i.e., to be significant) in most SEM empirical research, therefore it was considered with other indices of model fit. Although the model yields a x2 value of 81.15 when this value is divided by the degrees of freedom, the result got is lower than the desirable level of 3, recommending that the fit of the data to our model is adequate (Table 6).

Table 6

Fit indices for the study model

Fit Index	Recommended Values	This Research	Source
SRMR	≤0.08	0.058	(Hu & Bentler, 1999)
d-ULS	≤0.95	0.124	(Henseler et al, 2015)
d-G1	≤0.95	0.304	(Henseler et al, 2015)
d-G2	≤0.95	0.154	(Henseler et al, 2015)
Chi-square	≥1.96	81.15	(Hu & Bentler, 1999)
NFI	≥0.25	0.598	(Bonett & Bentler, 1999)
			(Bonett & Bentler, 1999)

  Standardized Root Mean Square Residual

2 Normed Fit Index

The value of the coefficient Q2 (Stone-Geiser) in Table 7 determines the predictive power of the model in endogenous structures. A structural model that is a good fit should be able to predict its endogenous variables. If the relationships between structures in a model are properly defined, the structures interact with one another to confirm the hypotheses. It measured operation satisfaction at 0.496, treatment staff satisfaction at 0.347, and operation feedback satisfaction at 0.357. Based on the averages and the strength of these values, these structures are predictive.

Table 7

Stone-Geisser statistics values of research variables

Variables	Criteria Stone-Geisser	Condition
Satisfaction with the nose surgery (1)	0.496	Strong
Satisfaction with the treatment staff (2)	0.347	medium
Satisfaction feedback (3)	0.357	Strong

DISCUSSION

The present study aimed to explore the development and validation of the Persian version of the ROE questionnaire on the Iranian clinical sample. The ROE -6 demonstrated good internal consistency and construct validity in this sample of Iranian applicants’ rhinoplasty. The results revealed that based on reliability statistics, respondents gave the least reliable rating to the appearance of the surgery at 0.55, and they gave the highest rating to feedback from their work or school at 0.96. In this study, instead of worrying about the appearance of your nose? Do you feel stressed by the appearance of your nose? And does that worry you often? The phrase general satisfaction with the appearance of the action and instead of this concern affects your daily life? (For example, your job) Satisfaction with action feedback from the workplace or place of study and internal satisfaction with the action, and instead, does this concern affect your relationships with others? Satisfaction with surgery feedback from friends Satisfaction with surgery feedback from family and Satisfaction with practice feedback from relatives, plus other factors that can develop a questionnaire in consultation with several rhino surgeons. It can be said that this questionnaire contains factors that cover 6 questions in this questionnaire.

Because rhinoplasty significantly alters the appearance of patients (“type change”), they may require more psychological support than with other surgery. Interestingly, most patients who found benefit from rhinoplasty continue to notice the effects even 5 years after surgery, with reported improvement in social relationships; however, patient dissatisfaction after surgery carries an additional burden, even if the surgeon considered the surgery objectively successful. [13]. Rosa et al. showed that there was high internal consistency (Cronbach’s alpha value: 0.88 pre-operatively, and 0.86 postoperatively). The authors observed a significant improvement in response to all individual questions in the postoperative phase as compared with the preoperative situation [14]. Jahandideh et al. [15] reported a Cronbach’s alpha in ROE score for the test 0.925 among patients who underwent RP, while Çelik et al. [2] noted a mean of the scores for each ROE -T question, as well as the total scores, did not differ significantly concerning test-retest reproducibility (all P>0.05). The internal consistency of the ROE -T was high, as evidenced by Cronbach’s α values of 0.887 pre-operatively and 0.798 postoperatively [10]. According to the ROE scale, the average preoperative score of the patients was 45, while the mean postoperative score was 81.5 [16].

It developed the ROE to assess RP outcomes and comprises six questions (two each for physical, emotional, and social factors) relevant to patient satisfaction. Besides the original English language, many researchers have translated the ROE into many other languages, including German and Brazilian-Portuguese [12],[17]. Literature studies [1],[2],[5],[9] showed significant variation over the follow-up period. In studies by Izu and colleagues [12], improvements in ROE scores were because of a decrease in postoperative edema in the later period. According to Arima et al. [18], there was no difference in the quality of life between follow-up periods of 6 months and ten years. Only the ROE-T scores, pre and postoperative compared among patients in the present study. It is possible to determine whether the RP outcome changes over time-based on the follow-up period, although longer follow-up periods may not associate with different results [18].

The greater postoperative improvement in the ROE score in our study compared with previous studies maybe because of the Approximately 6 months to two years after rhinoplasty. Researchers Hellings and Nolst Trenite conducted their study over a mean follow-up period of 30 months, with a mean ROE improvement of 12. Possibly, their low improvement score was because of the revision process used for their cases. There was no significant difference in ROE improvement based on age, sex, endonasal versus external approaches, and different graft (e.g., auricular cartilage, septum, allogenic rib) [19]. This study has more strengths and fewer weaknesses than other studies. We can extend its strengths for a longer period after surgery. The number of patients who took part in the study. The development of a postoperative satisfaction questionnaire and using the statistical method was factor analysis. However, another shortcoming of this study is the lack of questionnaires for participants before surgery and comparison of results with postoperative. Lack of control over intervention variables such as age, gender, marital status, and its effect on patient satisfaction.

CONCLUSION

The expanded ROE was reliable and valid, and the findings matched those of the original ROE. Therefore, the ROE-E is a valid instrument that can determine whether RP is effective among Iranian patients. The ROE-E can apply to multinational investigations since the ROE is the most reliable metric for measuring RP outcomes.

FUNDING

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

CONFLICT OF INTEREST

The authors declared no conflict of interest.