Psychometric properties of the Arabic version of the EUROPEP questionnaire.

BACKGROUND: The evaluation of quality of primary health care from the perspective of refugees is very underdeveloped. It depends mainly on the availability of instruments in the language of the refugees. The aim of this study was to translate, culturally adapt, and examine the psychometric properties especially the internal consistency and convergent construct validity of the Arabic version of the European Project on Patient Evaluation of General Practice Care (EUROPEP) questionnaire.
METHODS: The German version of the EUROPEP questionnaire was translated into Arabic language. In total, 619 Arabic-speaking people were invited to participate in this study. Refugees who lived in collective living quarters in the German federal state of Schleswig-Holstein were recruited. The EUROPEP questionnaire is a multidimensional instrument that comprises 23 items, each with a 5-point Likert-type response. Cronbach's alpha, descriptive statistics, and principal component analysis were used to assess a part of psychometric properties. Convergent construct validity was assessed with the validated questionnaire on satisfaction with ambulatory care - quality from the patient perspective (ZAP questionnaire) by using Spearman rank-order correlation test.
RESULTS: A total of 136 questionnaires of refugees were returned (response rate 22%). Of these respondents, 95 participants who had visited a general practitioner were included in the validation study. The exploratory factor analysis extracted four factors, namely, "medical care," "physician-patient relationship," "coordination of care," and "accessibility to care." The internal consistency ranged between α=0.942 for "medical care" and α=0.869 for "coordination of care." The convergent construct validity is supported by a significant positive correlation between the overall score of the EUROPEP questionnaire and the overall score of the ZAP questionnaire (rrho=0.820; p<0.01).
CONCLUSION: The Arabic version of the EUROPEP questionnaire shows appropriate internal consistency and convergent construct validity. The availability of this instrument in Arabic language encourages further research in the field of outcome quality from refugees' perspective in other health service research projects.

Background

Considering the classification of quality of care developed by Donabedian, outcome quality is beside structure and process quality an integral part of measuring quality of care. Outcome quality can also be assessed by patients.1 Patients’ evaluation of medical care consists of a combination of patient expectations regarding health care providers and the actual patient experience.2 Furthermore, it may be seen as feedback for the quality of care provided by physicians and is helpful for identifying opportunities for improvement.3,4

Patient perspective on quality of care in primary care can be evaluated by the European Project on Patient Evaluation of General Practice Care (EUROPEP) questionnaire. This questionnaire is a multidimensional instrument that comprises 23 items consisting of five dimensions: “relationship and communication,” “medical care,” “information and support,” “continuity and cooperation,” and “availability and accessibility.”5 It is a widely used instrument in primary care which is available in different languages.5–12

The first contact with the health care system is for primary health care of most of the people. In 2015, a large number of people from Syria, Afghanistan, and Iraq immigrated to Europe, especially to Germany.13 However, little is known of the health experience of refugees in the host country.

It was found that the main barrier for participation on health research is difficulties in communication.14 The evaluation of quality of care from the perspective of refugees is very underdeveloped until now which also depends on the availability of instruments in the language of the refugees. Arabic language is a very common language among refugees not only in Germany.15 However, an Arabic version of the EUROPEP questionnaire does not exist until now. The aim of this study was to translate, culturally adapt, and examine the psychometric properties especially the internal consistency and convergent construct validity of the Arabic version of the EUROPEP questionnaire.

Methods

The cross-sectional study was based on Arabic refugees in the German federal state of Schleswig-Holstein to explore psychometric properties especially the internal consistency and convergent construct validity of the Arabic version of the EUROPEP questionnaire.

Translation and cultural adaption

To adapt the German version of the EUROPEP questionnaire into Arabic language, we followed the Principles of Good Practice for the Translation and Cultural Adaptation Process by the ISPOR task force16 as follows: we received permission from the author Michel Wensing, who was responsible for the development of the EUROPEP questionnaire, to translate and adapt a Arabic version of the instrument.5 Two Arabic interpreters independently translated the German version of the EUROPEP questionnaire into Arabic language. After back translation, some significant differences were found from the original version. Therefore, a professional interpreting office was instructed with translation of the German version of the EUROPEP questionnaire. Two professional interpreters translated the questionnaire into Arabic language again. Divergent results were discussed during consensus meetings. After a linguistic adaptation, no item was assumed to be completely inappropriate. After cognitive pretesting with two Arabic-speaking people, similar results were received, and no item was perceived as redundant.

Recruitment and data collection

Participants were recruited in four collective living quarters in the German federal state of Schleswig-Holstein after consultation with the facility manager. Inclusion criteria for the validation study were Arabic-speaking people, aged ≥18 years, and have visited a general practitioner (GP) in the last 12 months. In total, 619 Arabic-speaking people were invited to participate in this study. A self-completion questionnaire was handed out to each participant, and the return of the anonymous paper-based questionnaire was classified as informed consent. Data were collected from May 2017 to September 2017.

Measures

The EUROPEP is an established and internationally validated instrument reflecting a set of indicators for patients to evaluate the quality of primary care.5 This questionnaire is a multidimensional instrument that comprises 23 items ranging from “poor” (1) to “excellent” (5) for each item. The EUROPEP questionnaire was validated for different languages such as Norwegian, Portuguese, and Bulgarian.8,11,12 However, an Arabic version of this instrument is still missing.

The validated “questionnaire on satisfaction with ambulatory care – quality from the patient perspective” (ZAP questionnaire) was used to assess convergent construct validity. The ZAP questionnaire consisted of 23 items and included aspects to physician–patient interaction, information, cooperation and competence, and practice organization.17 Each item was scored on a 4-point Likert scale, ranging from 1 (very unsatisfied) to 4 (very satisfied). The ZAP questionnaire was available in Arabic language.18

Statistical analysis

All analyses were carried out using SPSS 24.0 software (IBM Corporation, Armonk, NY, USA). The aim of this study was to assess the psychometric properties especially the internal consistency of the EUROPEP questionnaire and to examine the convergent construct validity with the ZAP. Items were assessed by mean, SD of mean, missing values on item level, and ceiling effects. A principal factor analysis (eigen value >1, VARIMAX rotation) was performed, and the Kaiser-Meyer-Olkin measure (KMO) of sampling adequacy and the Bartlett’s test of sphericity were determined. The internal consistency was assessed by using Cronbach’s alpha, which indicates whether an item of a scale is appropriate for assessing the underlying concept of the scale.19 Values for Cronbach’s alpha range from 0 to 1; the closer they are to 0 the lesser the items are related to one another. Values >0.60 are generally considered to indicate satisfying internal consistency and values >0.80 indicate a high internal consistency.

Convergent construct validity was assessed in terms of a Spearman rank-order correlation test between the mean of each factor (subscale) of the EUROPEP questionnaire and the mean for the overall score of the ZAP questionnaire as well as between the mean overall score of the EUROPEP questionnaire and the mean overall score of the ZAP questionnaire. In this context, correlations often range between 0.2 and 0.6, rarely above; correlations between 0.40 and 0.60 are regarded as good correlations.20 It was hypothesized that high positive correlation should be found between the two instruments to underline the concept of patient satisfaction. An alpha level of p≤0.05 was used for tests of statistical significance.

Ethical approval

The study was approved by the ethics committees of the University of Luebeck (No 17-082). Completion of the survey was voluntary and anonymous. The return of the anonymous paper-based questionnaire was classified as informed consent. No additional data were evaluated.

Results

Of the 619 questionnaires handed out, 136 questionnaires of refugees were returned (response rate 22%). In these respondents, n=95 had visited a GP in the last 12 months and therefore were included in the validation study. More than 63% of the responded refugees were men. The mean age was 30 years (SD=8.31) and ranged between 18 and 56 years. The main proportion of refugees were from Syria (n=74, 78%) followed by Iraq (n=12, 12.7%). For 9 (9.5%) participants, the country of origin was missing.

Mean, SD, missing rates, and ceiling effects of the EUROPEP questionnaire are shown in Table 1. The highest positive rate was observed for “keeping your records and data confidential” (mean =4.53) and “listening to you” (mean =4.12). The least positive rate was found for “waiting time in the waiting room” (mean =3.15) and “being able to speak to the GP on the telephone” (mean =3.19). The missing rate ranged between “interest in your personal situation” (n=3, 3.2%) and “being able to speak to the GP on the telephone” (n=19, 20%). Furthermore, moderate ceiling effect was observed for the items of the EUROPEP questionnaire.

Table 1

Descriptive statistics and ceiling effects for the EUROPEP questionnaire (n=95)

Items*	Mean (SD)	Missing, n (%)	Ceiling effect, %#
1. Making you feel you had time during consultations	4.08 (1.06)	4 (4.2)	48.4
2. Interest in your personal situation	4.07 (1.04)	3 (3.2)	45.3
3. Making it easy to tell about your problems	3.78 (1.12)	5 (5.3)	33.7
4. Involving you in decisions about medical care	3.81 (1.23)	14 (14.7)	33.7
5. Listening to you	4.12 (0.99)	6 (6.3)	45.3
6. Keeping your records and data confidential	4.53 (0.89)	7 (7.4)	67.4
7. Quick relief of your symptoms	3.78 (1.19)	7 (7.4)	36.8
8. Helping to perform your normal daily activities	3.89 (1.20)	7 (7.4)	41.1
9. Thoroughness	3.66 (1.18)	4 (4.2)	29.5
10. Physical examination	3.92 (1.15)	10 (10.5)	37.9
11. Offering you services for preventing	3.66 (1.30)	13 (13.7)	28.4
12. Explaining the purpose of tests and treatments	3.69 (1.23)	9 (9.5)	30.5
13. Telling about your symptoms and/or illness	3.81 (1.15)	7 (7.4)	31.6
14. Help in dealing with emotional problems	3.85 (1.11)	14 (14.7)	31.6
15. Helping understand importance of following advice	3.89 (1.18)	11 (11.6)	35.8
16. Knowing what has been done during previous contacts	4.02 (1.07)	9 (9.5)	37.9
17. Preparing what to expect from specialists	3.81 (1.15)	14 (14.7)	28.4
18. The helpfulness of the staff	4.09 (1.04)	6 (6.3)	41.1
19. Getting an appointment to suit you	3.50 (1.37)	5 (5.3)	31.6
20. Getting through to the practice on telephone	3.79 (1.23)	10 (10.5)	32.6
21. Being able to speak to the GP on the telephone	3.19 (1.40)	19 (20.0)	16.8
22. Waiting time in the waiting room	3.15 (1.55)	5 (5.3)	26.3
23. Quick services for urgent health problems	3.65 (1.29)	6 (6.3)	31.6

Notes:

Items are scored from 1 (poor) to 5 (excellent);

percentage of respondents ticking excellent as response option.

Abbreviations: GP, general practitioner; EUROPEP questionnaire, European Project on Patient Evaluation of General Practice Care questionnaire.

The values of factor loading for each item, variance, and internal consistency for each factor are presented in Table 2. The exploratory factor analysis extracted four factors (sub-scales) with explained variance of R2=82.1% (KMO 0.85, Barlett’s test of sphericity, p<0.001), medical care (65%, α=0.942), physician–patient relationship (6.89%, α=0.934), coordination of care (5.73%, α=0.869), and accessibility to care (4.45%, α=0.880). For the overall EUROPEP questionnaire, the internal consistency was α=0.973. The item “getting through to the practice on telephone” was assigned to factor 4 (accessibility to care) because of changes in internal consistency and content reflection.

Table 2

Factor structure, internal consistency, and Kaiser-Meyer-Olkin measure of the EUROPEP questionnaire

Items	Factor loadings				Kaiser-Meyer-Olkin
	1	2	3	4
1. Making you feel you had time during consultations		0.843			0.860
2. Interest in your personal situation		0.801			0.849
3. Making it easy to tell about your problems		0.690			0.889
4. Involving you in decisions about medical care	0.615				0.840
5. Listening to you		0.564			0.916
6. Keeping your records and data confidential		0.596			0.818
7. Quick relief of your symptoms	0.730				0.826
8. Helping to perform your normal daily activities	0.810				0.849
9. Thoroughness	0.646				0.873
10. Physical examination		0.631			0.842
11. Offering you services for preventing	0.808				0.915
12. Explaining the purpose of tests and treatments	0.759				0.861
13. Telling about your symptoms and/or illness	0.562				0.904
14. Help in dealing with emotional problems		0.645			0.887
15. Helping understand importance of following advice	0.825				0.813
16. Knowing what has been done during previous contacts	0.528				0.902
17. Preparing what to expect from specialists			0.816		0.898
18. The helpfulness of the staff			0.696		0.803
19. Getting an appointment to suit you				0.579	0.859
20. Getting through to the practice on telephone			0.643	0.497	0.775
21. Being able to speak to the GP on the telephone				0.845	0.809
22. Waiting time in the waiting room				0.838	0.770
23. Quick services for urgent health problems			0.772		0.779
Percentage of variance	65%	6.89%	5.73%	4.45%
Variant a: internal consistency (α)	0.942	0.934	0.853	0.871
Variant b: internal consistency (α)*	0.942	0.934	0.869	0.880

Notes:

Item “Getting through to the practice on telephone” was assigned to factor 4 (before factor 3) because of changes in internal consistency and content reflection. Factor 1: medical care; factor 2: physician–patient relationship; factor 3: coordination of care; factor 4: accessibility to care.

Abbreviations: EUROPEP questionnaire, European Project on Patient Evaluation of General Practice Care questionnaire; GP, general practitioner.

The convergent construct validity was measured using the aggregated ZAP questionnaire, which showed significant positive correlation to the EUROPEP overall score and the different subscales. Detail results are presented in Table 3.

Table 3

Convergent construct validity for each subscale and overall scale of the EUROPEP questionnaire

Subscale of EUROPEP	Mean (SD)	Correlations with aggregated ZAP-questionnaire, rrho
Overall EUROPEP score	3.83 (0.87)	0.820*
Factor 1: medical care	3.79 (1.01)	0.769*
Factor 2: physician–patient relationship	4.06 (0.85)	0.734*
Factor 3: coordination of care	3.84 (1.01)	0.750*
Factor 4: accessibility of care	3.46 (1.15)	0.682*

Note:

Statistical significance at p<0.01.

Abbreviation: EUROPEP questionnaire, European Project on Patient Evaluation of General Practice Care questionnaire.

Discussion

The present study describes appropriate internal consistency of the Arabic version of the EUROPEP questionnaire. In addition, it can be used to determine quality of primary health care in Germany from the perspective of refugees. Moderate missing rates and moderate ceiling effects on the item level of the EUROPEP questionnaire support these results. In contrast to other studies, the ceiling effect of the EUROPEP questionnaire in the presented study is low.6,8,12 However, ceiling effect is a common phenomenon in questionnaires on the experience of patients with health services.21

The EUROPEP questionnaire can be divided into four subscales after performing the exploratory factor analysis: medical care, physician–patient relationship, coordination of care, and accessibility to care. Moreover, high internal consistency for each subscale and the overall score of the EUROPEP questionnaire were observed. The significant positive correlation with the corresponding overall score of the ZAP questionnaire indicates appropriate convergent construct validity and confirms the hypothesis that the EUROPEP reflects the concept of patient satisfaction.

The underlying structure of the division of the items into two subscales “organizational of care” and “clinical behavior” cannot be confirmed with our results.5 However, this result is comparable with other validation studies which also concluded a different factor structure through the analysis.12,22 The original version of EUROPEP can be divided into five domains: “relationship and communication,” “medical care,” “information and support,” “continuity and cooperation,” and “facilities availability and accessibility.”5 This structure cannot confirm within our results, but some overlaps can be identified concerning the domain “relationship and communication” as well as “medical care.”

Similarly, other studies, our results showed that the items “keeping your records and data confidential” and “listening to you” were the most highly rated.8,23 Listening is a relevant competence for a satisfied physician–patient relationship. It was found that listening can support refugees’ confidence in the GP in terms of understanding the specific situation as refugee.24 Two items that showed the lowest rating are “being able to speak to the GP on the telephone” and “waiting time in the waiting room.” It was found that patient with a migration background and poor language skills from the host country were less positive about these two aspects.23 A recently published scoping review show that poor communication skills of refugees could reduce health care access.24 Moreover, for most studies that use EUROPEP questionnaire, it was observed that “waiting time” showed the lowest rating independent from patient background (ethnicity as well as country of origin).

The evaluation of quality of care from the perspective of patients could identify possible areas of improvement and can be used as a feedback instrument for health care providers. However, the availability of instruments that are in the first language of the patients is rare. Especially, against the background of more refugees in the host country it seems to be important to have an instrument in their language for including information about health care process and to know more about the perceived quality of health care.

Strengths and limitations

The validation of an instrument that measures quality of care from the perspective of refugees is an important tool for the integration of refugees in health research project and their experience about their quality of care. We included a convenient sample of refugees in one federal state in Germany. However, our results have to be interpreted against the background of potential response bias because of a low participation rate. The study results should be confirmed in further studies with higher response rate. The Arabic version of the questionnaire benefited from cognitive pretesting. However, a quantitative pretest would have been helpful to evaluate potential barriers and should be considered in further studies. Furthermore, only internal consistency and convergent construct validity were determined, which reflected no complete reliability analysis concerning the “Consensus-based Standards for the selection of health Measurement Instrument” checklist.25 Finally, the results of the study are only explorative and should be confirmed in further studies which included a test–retest design.

Conclusion

Overall, the Arabic version of the EUROPEP questionnaire is a valid measurement with appropriate internal consistency and could be suitable for the evaluation of the quality of primary care from the perspective of refugees. Our study is an important step for the consideration of refugees’ feedback to their treatment process in ambulatory care. Moreover, availability of this instrument in Arabic language encourages further research in the field of outcome quality from refugees’ perspective in other health service research projects.