Cross-cultural adaptation, analysis of psychometric properties and validation of the Spanish version of a perioperative satisfaction questionnaire (EVAN-G).

BACKGROUND: Patient satisfaction is a reliable and measurable indicator of the quality provided by a healthcare service. There are several questionnaires for measuring it, but only a few have shown good psychometric properties, an outstanding one being the EVAN-G (Evaluation du Vécu de l'Anesthésie Générale) questionnaire, which measures patient satisfaction regarding perioperative care and is validated in French. The aim of this study is the validation of a Spanish version of the EVAN-G questionnaire.
METHODS: A translation/back-translation of the questionnaire into Spanish was carried out and the final version obtained was administered to three hundred patients. Its psychometric properties were measured and compared with those of the original EVAN-G questionnaire to verify that they had been maintained after the previous translation process. The questionnaire's content, construct and external validity were measured. To calculate reliability, Cronbach-α coefficient and test-retest method were used. The Global Satisfaction Index was calculated and satisfaction level in our sample was analyzed.
RESULTS: Content, construct and external validity were proven with similar results that in the original EVAN-G. The translated version of the questionnaire showed good reliability: Cronbach-α coefficient was 0.92 and intraclass correlation coefficient measured by test-retest method was 0.9. The acceptability was high. The average Global Satisfaction Index in our sample was 73±12.
CONCLUSIONS: The translation into Spanish and cross-cultural adaptation of the EVAN-G questionnaire has proven its validity, reliability, and acceptability to measure patient satisfaction in interventions performed under general anesthesia.

Introduction

Human satisfaction is an indicator of the patient's perception of the quality of healthcare services.1, 2 As with the results of healthcare, patient satisfaction is a fundamental objective and a useful measure of quality. Many questionnaires throughout the scientific literature try to provide a proper assessment of patient satisfaction in the surgical area.4, 5, 6, 7, 8, 9, 10 In a recently published review, the authors affirmed that many studies still use instruments that are not validated or are poorly developed. In that review, the questionnaire with the highest validation score was the EVAN-G (Evaluation du Vécu de l’Anesthésie Générale).

Developing a new valid and reliable questionnaire is a complex process. It is advisable to use summative scales such as the Likert scale. If validated questionnaires with proven methodological quality already exist, a reliable translation must be carried out and a subsequent transcultural adaptation is also needed.

The first version of the EVAN-G questionnaire, written in French, dates from 1999. It evaluated patient satisfaction regarding the perioperative period.4, 5 In 2005 the same author, Pascal Auquier, went through the whole process from the beginning again, performing a rigorous psychometric validation according to several systematic reviews.11, 14 There is no questionnaire in Spanish that meets these requirements as strictly as the EVAN-G does. Thus, the decision to translate the EVAN-G into Spanish was due to its strong psychometric characteristics, the focus on patients receiving general anesthesia and its development in a country with geographical proximity to Spain, sharing for this reason certain cultural characteristics. The EVAN-G questionnaire has been widely used in different studies.15, 16

The hypothesis of our study is that the final version of the EVAN-G questionnaire, after a process of translation into Spanish and cross-cultural adaptation, preserves the solid psychometric properties of the original questionnaire.

Material and methods

A process of translation/back-translation into the original language was followed, a widely endorsed and used method.17, 18, 19 This is the most thorough method and it guarantees the highest translation quality. It was necessary to assess whether the questionnaire was comprehensible, whether format and vocabulary were adequate, as well as to evaluate whether it was culturally appropriate. The questions and answers choices were analyzed and, after the definitive version was obtained (Appendix 1), it was administered to a representative sample of the study population. Finally, a statistical analysis was conducted to determine the quality of the adapted questionnaire and to establish a comparison with the original version.

The original questionnaire consists of 26 items that are encompassed in six dimensions (attention, privacy, information, pain, discomfort and waiting time). The translated EVAN-G comprises 30 items and the same dimensions, with four additional items collected in their corresponding dimensions. Items 20 and 21 from the original EVAN-G make reference to waiting time for obtaining an appointment with the anesthetist or the surgeon and waiting time at hospital during this appointment, respectively. In order to separately analyze waiting time regarding different professionals, the creation of two new items was necessary. Original items 23 and 24 assess the attentiveness of the anesthetist, the surgeon and other healthcare staff. For the same reason, two more items were created regarding this aspect in the translated version.

Different studies show that between five and ten patients per item are needed in order to test traslations.21, 22 Three hundred patients were recruited (ten patients per item) between November 2015 and December 2016, after providing written informed consent. Inclusion criteria were: consent to participate in the study, age older than 18 years, elective surgery under general anesthesia, ASA I–IV and ability to read and understand a questionnaire in Spanish. The study was carried out at Virgen del Rocio University Hospital in Seville, Spain, a tertiary hospital. General Surgery, Urology, Otorhinolaryngology (ORL), Thoracic Surgery and Ophthalmology procedures under general anesthesia were included. Exclusion criteria were: inability to understand Spanish, psychiatric disorders, known substance abuse or any serious medical condition that prevented the understanding and completion of the questionnaire within 48 hours after surgical intervention. All patients completed the questionnaire by themselves, with no intervention from the research team. Patients who did not complete at least a 75% of the questionnaire were excluded from the analysis. Missing items were substituted for the mean of non-missing items of that dimession.

Psychometric properties (content validity, construct validity, external validity, reliability, and acceptability) were analyzed.

So as to check content validity, patients were asked an open-ended question at the end of the questionnaire, as in the original questionnaire. Construct validity (convergent validity and divergent or discriminant validity) was measured using the principal component analyses with the Varimax rotation. The test was expected to reveal the differences between individuals. Items with a standard deviation greater than 1 and with a mean located around the midpoint of the scale were considered suitable. However, the decision to eliminate or preserve an item was based on a joint assessment of all statistical indices, together with a consideration of the conceptual aspects that led to the item’s creation.

External validity measures the degree to which the results of a study can be generalized to a different population. The relation between specific potential dimensions of the EVAN-G and other validated instruments that measure some of the same dimensions were investigated, such as McGill Pain Questionnaire (MGPQ), State-Trait Anxiety Inventory (STAI) or Visual Analogic Scale (VAS). External validity was analyzed by the Pearson correlation coefficient.

Reliability was estimated by internal consistency and test-retest reliability. Internal consistency make reference to the degree of interrelation and coherence of the items. It was considered reliable if the items measuring the same construct showed homogeneity among themselves. Cronbach-α coefficient was calculated to determine this and it was considered acceptable when it was above 0.7. Test-retest reliability measures the constancy of the answers obtained repeatedly from the same subjects using the Intraclass Correlation Coefficient (ICC). It was conducted on a subsample of 20 patients who were evaluated twice before discharge.

Acceptability refers to the time taken to complete the questionnaire, its simplicity, brevity, ease of correction, coding, and interpretation of results.

The Global Satisfaction Index (GSI) is the average score obtained from the means of each dimension. Its value ranges from 0 to 100 (0 being the lowest level of satisfaction and 100, the highest). The relation between the GSI and categorical variables was analyzed with the Student´s t-test and analysis of variance or the Mann-Whitney U-test and Kruskal-Wallis test, accordingly to the normality of the samples; and its relation with quantitative variables through the Pearson correlation coefficient.

Results

The study sample included 300 patients. Of those, two patients (0.7%) responded to less than 75% of the questions and they were consequently excluded from the analysis. Therefore, response rate was 99.3%. Mean (SD) patient age was 58.6 ± 14.2 years. Surgical procedures were performed by General Surgery (56.4%), ORL (15.8%), Thoracic Surgery (14.4%), Urology (13.1%) and Ophthalmology (0.3%). Other baseline variables are showed in Table 1.

Table 1

Sociodemographic variables.

	%
Sex
Male	55.4
Female	44.6
ASA
I–II	69.4
III–IV	30.6
Premedication
Yes	96.6
No	3.4

Values are presented as percentage.

ASA, American Society of Anesthesiologists (physical status); Premedication, Anxiolytic drugs according to weight and previous pathologies.

The item with the highest non-response rate was item 15 (7.7%). The item with the highest score was item 25, with a mean (SD) of 80.87 ± 18.8. Lowest score was obtained on item 17, with a mean (SD) of 63.83 ± 22.28. All items scored higher than 60. The mean (SD) scores of the dimensions were: attention, 78.82 ± 19.83; privacy, 72.34 ± 19.57; information, 70.85 ± 22.54; pain, 71.24 ± 24.13; discomfort, 76.83 ± 24.24; and waiting time, 64.40 ± 25.45.

Content validity was measured with an open-ended question at the end of the questionnaire regarding any domain that was not discussed on the EVAN-G. Only five patients referred to missing content of the questionnaire in their answers (1.7%).

Construct validity was measured by means of principal component analyses with the Varimax rotation test and is shown in Table 2. A six-factor structure accounted for 68% of the total variance. The correlation between items and their corresponding dimensions (internal consistency of the items) ranged from 0.31 to 0.95. The correlation between items with different dimensions with values lower than 0.3 were not considered acceptable and were, therefore, deleted from the table. The correlation between items within one dimension and those in other dimensions (discriminant or divergent validity) ranged from 0.00 to 0.55. Construct validity was quite high and similar to the original EVAN-G, as shown in the comparison of the results in Table 3. The correlation index of each item with its dimension and of each item with other dimensions was almost identical in both the adapted and the original questionnaires and were found within methodologically accepted ranges.

Table 2

Principal Component Analysis of the 30-Item traduced EVAN-G Questionnaire (Varimax Rotation).

Item nº	Dimensions
	Attention	Information	Waiting	Discomfort	Privacy	Pain
26	0.84
28	0.82
25	0.82
27	0.82
24	0.79
30	0.77
29	0.77
3		0.82
2		0.82
5		0.77
4		0.77
1		0.76
7		0.35		0.31
22			0.95
23			0.94
20			0.92
21			0.91
9				0.83
10				0.78
13				0.64
14				0.55
18					0.77
8					0.73
6					0.68
19					0.55
12				0.37	0.38
16						0.75
17						0.66
15						0.65
11						0.45
% Variance explained (total = 68%)	31.31	11.50	9.23	8.03	4.38	3.46

Factor loadings above 0.3 are not reported in the table. For each column, bold numbers are the factor loadings of the items participating in the computation of the corresponding dimension.

Table 3

Comparisons of the construct validity.

	Original EVAN-G	Traduced EVAN-g
IIC	0.55–0.92	0.55–0.95	0.12
IDV	0.02–0.53	0.00–0.55	0.15

IIC, Item-internal consistency (correlation between items scores and their dimension score). IDV, Item-discriminant validity (correlation between items scores of a given dimension with the other dimension scores). Numbers are lowest-highest.

The mean (SD) GSI was 73.02 ± 12, finding the best mean (SD) dimension score on attention (78.82 ± 19.83) and the worst, on waiting time (64.40 ± 25.45). Mean (SD) MGPQ score was 10.96 ± 11.22 (range 0–47), mean (SD) STAI score was 36.18 ± 18.12 (range 0–93) and mean (SD)VAS score was 3.77 ± 2.9. Table 4 shows the scores obtained in GSI, MGPQ, STAI and VAS in the original and the adapted questionnaires.

Table 4

Comparison of GSI, MGPQ, STAI.

	Original EVAN-G	Traduced EVAN-G
GSI	75 ± 14	73 ± 12
MGPQ	6.3 ± 8.6	10.9 ± 11.2
STAI	51.3 ± 2.9	36.2 ± 18.1

Date are presented as mean ± SD (standard deviation).

GSI, Global Satisfaction Index; MGPQ, McGill Pain Questionnaire; STAI, State-Trait Anxiety Inventory.

The correlation between the scores in the translated EVAN-G questionnaire and in GSI, MGPQ, STAI and VAS was calculated through the Pearson correlation coefficient and compared with the correlation obtained with the original EVAN-G questionnaire (Table 5). The highest correlation between scales was found in the pain dimension: strong-moderate negative correlation between the MGPQ and both, the original (r = -0.52, p <  0.01) and the translated EVAN-G (r = -0.33, p < 0.01); strong-moderate correlation between VAS and both, the original (r = -0.68, p < 0.01) and the translated version (r = -0.48, p < 0.01); and moderate negative correlation between STAI and the translated version (r = -0.46, p < 0.01). Also, a strong positive correlation was found between VAS and MGPQ (r = 0.44, p < 0.01), and moderate positive correlation between VAS and STAI (r = 0.34, p <  0.01).

Table 5

MGPQ and STAI in original and traduced questionnaire.

Dimension	Original EVAN-G		Traduced EVAN-G
	MGPQ	STAI	MGPQ	STAI
Attention	−0.07	0.11a	−0.05	−0.20a
Information	−0.10	0.06	−0,07	−0.23a
Waiting	−0.18a	0.05	−0.02	−0.11
Discomfort	−0.36a	0.06	−0.19a	−0.40a
Privacy	−0.02	0.04	−0.19a	−0.26a
Pain	−0.52a	0.09	−0.33a	−0.46a
GSI	−0.33a	0.11	−0.18a	−0.39a

Comparisons of the correlation between original and traduced EVAN-G scores and MGPQ and STAI.

Numbers are Pearson correlation coefficients.

p <  0.01.

Reliability of the translated EVAN-G questionnaire measured by the Cronbach-α coefficient was 0.92. Values ranged from 0.66 in pain dimension to 0.96 in waiting time dimension. In the original questionnaire, discomfort was the dimension with the worst reliability (0.73); while it was pain dimension (0.66) in the translated questionnaire, being all remaining values above 0.7. The ICC of the translated EVAN-G questionnaire was 0.9 (range from 0.52 to 0.94), higher than the ICC of the translated EVAN-G questionnaire 0.77 (range from 0.72 to 0.81).

The average time taken to complete the translated EVAN-G questionnaire by the patients was 14 ± 10 min.

The level of satisfaction in our reference population was high, with an average GSI of 73 ± 12, ranging from 64 ± 25 (waiting time) to 79 ± 20 (attention). GSI was identical between patients older and younger of 65 years (73 ± 12, p < 0.01). It was slightly higher in men over women (74 ± 12 vs. 71 ± 12, p < 0.01) and in ASA I–II patients over ASA III-IV patients (74 ± 12 vs. 71 ± 12, p < 0.01).

A summary of the mean scores of each dimension, percentage of lost values, convergent validity, discriminant validity, Cronbach-α coefficient, ICC for each dimension and GSI is showed in Table 6.

Table 6

Summary table.

Dimension	Mean±Sd	%MV	IIC	IDV	Cronbach-α	ICCa
Attention	78.82 ± 19.83	2.21	0.77–0.84b	0.00–0.25b	0.94	0.94
Information	70.85 ± 22.54	3.29	0.77–0.82b	0.01–0.27b	0.90	0.86
Waiting	64.40 ± 25.45	2.18	0.91–0.95b	0.01–0.16b	0.96	0.92
Discomfort	76.83 ± 24.24	1.34	0.55–0.83b	0.01–0.33b	0.75	0.73
Privacy	72.34 ± 19.57	2.48	0.55–0.77b	0.03–0.41b	0.79	0.85
Pain	71.24 ± 24.13	3.61	0.45–0.75b	0.01–0.55b	0.66	0.52
GSI	73.02 ± 12	15.11	X	X	0.92	0.90

Data are presented as mean SD.

%MV, Percentage of missing values; IIC (Item-Internal Consistency), correlation between items scores and their dimension score; IDV (Item-Discriminant Validity), correlation between items scores of a given dimension with the other dimension scores; ICC, Intraclass Correlation Coefficient; X, Not computable.

On a subsample of 20 patients.

p < 0.01.

Discussion

Patient’s point of view often differs from that of the physician. Measuring patient´s perception of outcomes of care is essential to know the quality of healthcare process. A high-quality questionnaire is required for a proper assessment of satisfaction, and EVAN-G, originally in French, has showed to meet all the requirements to this purpose. With this study, the Spanish-language version of the EVAN-G questionnaire has proven to preserve the solid psychometric properties of the original questionnaire and has been consequently validated.

The translated EVAN-G questionnaire had a higher response rate than the original questionnaire (99.3% vs. 89.5%). The validation of the original EVAN-G reports response rates that range from 62% to 100%. This heterogeneity could be explained by the greater number of professionals and centers that were involved comparing to our study.

According to the Likert scale, which was used for this study, the variability in the answers to different items shows the discrimination capacity of the test and must be relatively high; for this type of scale, a standard deviation around 1 is considered adequate. The standard deviation was close to 1 for all values (ranging from 0.91 to 1.31), so we can assume a good discriminatory capacity of the scale in all dimensions.

Content validity was considered high in the translated version, as 98.3% of patients considered that the questionnaire included questions about all aspects of their perioperative experience. The high construct validity of the original questionnaire was also preserved in the translated version, as shown in the results.

Regarding external validity, MGPQ score and VAS score were quite equivalent in the original and in the translated questionnaires, since they estimate the same dimension (pain). The patient's state of anxiety (assessed using specific scales such as STAI) might be influenced by multiple aspects, such as pain and discomfort, and have an impact on overall satisfaction with the process. Although the original EVAN-G questionnaire did not find an association with the STAI in any of its dimensions, in our study, as might be expected, a correlation has been proven between anxiety and the dimensions of pain, discomfort and overall satisfaction (measured by GSI) in the translated version.

The reliability of the translated EVAN-G questionnaire was maintained after the process. The dimension with the lowest reliability in our study was pain. Pain is the most dynamic condition of the dimensions and can be modified by different aspects, such as the different analgesic drugs provided in different times after the surgery and the relief obtained with them. This would be related with different responses in the same dimension depending on the moment in which the questionnaire is administered and could explain this result.

Patients did not point out any difficulties completing the translated questionnaire, with an average completion time of 14 minutes. This can be considered as a good acceptability of the Spanish EVAN-G and relate with an easy implementation into routine clinical practice.

We obtained a high reported level of satisfaction in our reference population (average GSI of 73 ± 12) similar to the score obtained in the original questionnaire (75 ± 14). The lowest score was obtained in waiting time, which was, conversely, one of the highest rated in the original EVAN-G. We consider this as an important improvement point in our hospital environment and strategies aimed to reduce waiting times in the perioperative period should be implemented.

In contrast with those findings reported in the and the original EVAN-G questionnaire regarding age and satisfaction, where older patients are usually more satisfied, in the translated EVAN-G, satisfaction between age groups was identical with statistical significance. However, as for sex, slightly higher satisfaction levels were found in men over women, as other authors report too.

In order to measure satisfaction level, other studies have used the original EVAN-G questionnaire in different circumstances. A study comparing satisfaction in patients receiving or not sedative premedication, a similar score was reported in both groups (72.5 vs. 71.5). Statistically significant differences in satisfaction between premedication groups were not found in our study either. A randomized trial of prewarming in outpatient surgery used EVAN-G to asses satisfaction level, and a higher score was obtained in patients that had been prewarmed (88) versus those that had not (84). Outpatient surgery is usually related with less complex and painful procedures and cause less impact on patient´s life and this might imply a higher satisfaction level in outpatients.

The study was carried out in a tertiary hospital in Seville (Andalusian region), so the generalization of the conclusions to other contexts require further validations in smaller hospitals, different regions, and other cultures. Even if the language is the same, cultural differences might have influence on the applicability of the questionnaire, so it must be adapted to the context where it will be administered.

Another obvious limitation, already present in the original EVAN-G questionnaire, is that it refers only to procedures under general anesthesia and related with only certain surgical specialties. To measure satisfaction regarding interventions under locoregional anesthesia or monitored anesthetic care other questionnaires should be used. A more extensive validation of the questionnaire is advisable in order to endorse applicability to surgical specialties other than those referred to in the study.

The relevance of this work lies on being the first validated Spanish-language questionnaire to measure perioperative satisfaction preserving the psychometric properties of the original questionnaire. Spanish-language EVAN-G questionnaire is a valid instrument to measure the satisfaction of our patients. Only by using standardized and validated measuring instruments we shall find improvement points and optimize the care process and the perioperative experience.

Funding

Support was provided solely from institutional and/or departmental sources.

Conflicts of interest

The authors declare no conflicts of interest.