Psychometric Characteristics of the Croatian and the Serbian Versions of the Oral Health Impact Profile for Edentulous Subjects, with a Pilot Study on the Dimensionality.

INTRODUCTION: The aim was to adapt the Croatian and the Serbian versions of the Oral Health Impact Profile for the edentulous population (OHIP-EDENT-CRO and OHIP-EDENT-SRB).
METHODS: The translation and cross-cultural adaptation were carried out in accordance with accepted international standards. A total of 95 and 177 removable denture wearers were recruited in Croatia and Serbia respectively. The reliability was evaluated by calculating Cronbach's alpha coefficient and by test-retest (30 participants in each country). The concurrent validity was determined by calculating the Spearman's rank coefficient between the OHIP-EDENT summary scores and one question related to removable denture satisfaction. Construct validity was determined by exploratory factor analysis (EFA). Responsiveness was determined by comparison of the OHIP-EDENT summary scores before and after dental implant placement to support mandibular overdentures (23 patients in Croatia, 21 in Serbia).
RESULTS: Cronbach's alpha coefficient was 0.92 in Croatia and 0.87 in Serbia. The intraclass correlation coefficient was 0.98 in Croatia and 0.94 in Serbia. In Croatia the Spearman's correlation coefficient was -0.71 (p<0.001) and in Serbia -0.74 (p<0.001). Both confirmed concurrent validity. Construct validity was tested by EFA, which extracted four factors in each country, accounting for 66.59% of the variance in Croatia and 59.33% in Serbia. Responsiveness was confirmed in both countries by a significant OHIP-EDENT summary score reduction and a high standardised effect size (3.9 in Croatia, 1.53 in Serbia).
CONCLUSION: The results prove that both instruments, the OHIP-EDENT-CRO and the OHIP-EDENT-SRB, have very good psychometric properties for assessing OHRQoL in the edentulous population.

Introduction

Oral health-related quality of life (OHRQoL), as a part of general health and well-being, has become very important in dental epidemiological and clinical studies that measure the extent of teeth loss and the impact of dental intervention (1, 2, 3, 4). A recent study identified 20 Dental Patient-Reported Outcome Measure (dPROM) Questionnaires with multiple items in the English language (5). Some of them were aimed at a specific population, such as the young (6), elderly (7), disease-specific (8, 9) or edentulous population (10). Unidimensional questionnaires were developed to measure one specific dimension of OHRQoL, such as the Orofacial Esthetic Scale (11) or the Chewing Function Questionnaire (12). However, the Oral Health Impact Profile (OHIP), a multidimensional questionnaire, has been one of the most frequently used dPROMs. The original version, with 49 items grouped into seven theoretical domains (functional limitation, physical pain, physical limitation, psychological discomfort, social limitation, and disability (handicap)), has been translated into more than 30 languages and culturally adapted (13). Recent studies reported the existence of only four major dimensions for the OHIP-49 questionnaire, i.e., Oral Function, Orofacial Pain, Orofacial Appearance and Psychosocial Impact (14, 15, 16). The short 14-item OHIP questionnaire was developed soon after the original one (17) and adapted worldwide to reduce the time required and the number of incomplete answers. Owing to the floor effect, it was not sensitive enough for status measurement and treatment improvements in edentulous subjects (10). The 19-item OHIP-EDENT questionnaire has therefore been developed and adapted in several different cultural environments (10, 18, 19, 20).

The OHIP-49 and the OHIP-14 questionnaires have already been validated in Slovenia, while in Serbia only the OHIP-14 has been validated (21, 22, 23). The OHIP-EDENT has not yet been translated or psychometrically validated either in Croatia or Serbia. The aim was to develop Croatian and Serbian versions of the OHIP-EDENT, validate them in a target population, and perform a pilot study of the dimensionality of the questionnaires. The hypothesis was that both versions would show good reliability, validity and responsiveness.

Material and methods

Participants

A sample of 95 participants was recruited at the School of Dental Medicine, University of Zagreb, Croatia. A sample of 177 participants was recruited at the Faculty of Dental Medicine, University of Belgrade, Serbia. The criterion for inclusion was the wearing of maxillary and mandibular removable dentures for at least six months prior to the research. This was to ensure full adaptation. In Croatia, 83 participants had complete dentures (CD) in both jaws, while 12 had one complete and one partial removable denture. In Serbia, all participants were CD wearers. Subjects who reported a history of mental disorders in anamnestic data were excluded. The sample size was determined using a strategy similar to that used in other studies (19, 24). A minimum of 95 participants was required. Participants answered questions from the OHIP-EDENT questionnaire by rating the frequency of a particular problem experienced during the previous week (25). The responses were rated on a Likert-type scale (0 – never, 1 – hardly ever, 2 – occasionally, 3 – fairly often, 4 – very often), where “zero” represented the absence of problems while higher scores represented more problems and worse oral health. The research has been conducted in accordance with the Declaration of Helsinki. Written informed consents were obtained from each participant. The study was approved by the ethics committees of the Croatian (approval no 05-PA-26-6/2015) and Serbian dental schools (approval no 36/18).

Translation

The OHIP-EDENT was translated using the accepted standard of the forward-backward process (26). The original and back-translated versions were compared by two specialists in prosthodontics with fluency in the English language, together with one native English language speaker in both countries. The preliminary versions were additionally pilot-tested in ten CD wearers to check understanding and clarity of the items. The OHIP-EDENT-CRO and the OHIP-EDENT-SRB questions are shown in Figure 1.

Figure 1

Translation of the original English language OHIP-EDENT into the Croatian and Serbian languages.

Reliability

Two types of reliability were assessed: internal consistency and test-retest reliability. The internal consistency was assessed by calculating Cronbach’s alpha coefficient, which captures the extent of agreement between all possible subsets of questions. Guttman’s “Split-Half” coefficient was also calculated. A value of >0.7 was considered acceptable.

The test-retest reliability was assessed in 30 participants willing to make a recall visit in each country. They were not provided with any treatment between the two administrations of the questionnaire. The prediction was that OHRQoL would not change during a 15-day treatment-free period. The test-retest reliability was assessed by intraclass correlation coefficients (ICC) of the OHIP-EDENT summary scores, based on the one-way repeated-measure analysis of variance (ANOVA) from the repeated tests.

Validity

Concurrent validity assessed the degree to which the OHIP-EDENT summary scores correlated with another measure of the same construct at the same time. The association between the OHIP-EDENT summary score and a simple question, “Please rate your satisfaction with dentures”, was assessed in both countries. Satisfaction with the existing dentures was rated from 1 to 5 (1 represented the worst and 5 the best score). It was assumed that patients who rated their dentures with higher scores would have lower OHIP-EDENT summary scores. Spearman’s correlation coefficient was calculated.

Construct validity was assessed using exploratory factor analysis (EFA) to identify interrelationships and groupings among items in the OHIP-EDENT-CRO and the OHIP-EDENT-SRB. Before performing EFA, the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and Bartlett’s Test of Sphericity were conducted. KMO values above 0.6 and a Bartlett’s Test of Sphericity significance of <0.0001 were considered adequate for the performance of EFA. The main factors were extracted using the Principal Component Analysis, and varimax rotation was performed. A minimum eigenvalue of 1 was assigned as the factor extraction criterion. Item loadings ≥0.4 were considered sufficient (27).

Responsiveness

Responsiveness is defined as the ability of the measure to evaluate changes after therapy in patients over time (28). We had two samples: 21 CD wearers in Serbia and 23 CD wearers in Croatia. In Serbia, 21 CD wearers received two standard-size implants in the mandible in approximate sites of previous canines. After osseointegration (three months), the implants were loaded. The Locator system was used and the abutments were screwed at 35 Ncm. Metal housings were mounted using a self-curing acrylic resin into the existing mandibular dentures, which were relined and adjusted when necessary. A sample of 23 CD wearers was recruited in Croatia. All participants had slim alveolar ridges (≤4 mm) in the mandible. They therefore received four slim ball-type mini dental implants (MDIs) in the interforaminal region. Early loading protocol was used. Metal housings with “o-rings” were attached into the existing mandibular CDs using a self-curing resin. The dentures were relined when necessary. The participants completed the questionnaire twice: before the implant placement and one month after the implants had been loaded and the dentures adjusted. The standardised effect size was calculated using the equation: Mean (baseline OHIP-EDENT score – follow-up OHIP-EDENT)/ Standard deviation of the baseline OHIP-EDENT score (29).

Results

A sample of 95 participants was recruited in Croatia and 177 participants in Serbia. Sampling strategies, together with the participants’ age and demographic characteristics, are presented in Table 1.

Table 1

Sampling strategy and participants’ demographic data.

Country	Sample	Sampling	Research purpose	n(% women)	Mean age (years) M±SD	Age range (years)	Duration of edentulism (M±SD)	Range of edentulism (years)
Croatia	Removable denture wearers	Convenience	Internal consistency, Concurrent validity	95 (65%)	69.08±9.2	51–93	7.6±7.9	1–30
	Removable denture wearers	Consecutive	Test-retest reliability	30 (60%)	71.8±10.2	52–91	7.6±6.6	1–30
	Removable denture wearers	Convenience	Responsiveness	23 (61%)	66.6±10.2	51–93	7.6±5.9	2–28
Serbia	Complete denture wearers	Convenience	Internal consistency, Concurrent validity	90 (51%)	63.8±11.5	35–90	3.5±7.6	1–21
	Complete denture wearers	Consecutive	Test-retest reliability	30 (40%)	65.3±10.3	43–90	3.8±6.6	1–18
	Complete denture wearers	Convenience	Responsiveness	21 (43%)	63.8±10.5	42–79	3.5±5.6	1–14

M=mean value; SD=standard deviation

Internal consistency

Cronbach’s alpha coefficients, average inter-item correlations, mean and summary scores with standard deviations of the OHIP-EDENT questionnaires, as well as Guttman’s Split-Half coefficients, are presented in Table 2. In Croatia, higher Cronbach’s alpha and Guttman’s Split-Half Coefficients were obtained, as well as higher mean item score and mean summary score (Table 2).

Table 2

Cronbach’s alpha coefficient, average inter-item correlation, mean and summary scores with standard deviations, and Guttman’s Split-Half Coefficients of the OHIP-EDENT-CRO and the OHIP-EDENT-SRB; n=number of participants.

Questionnaire	n	Cronbach’s α	Average inter-item correlation	Mean Summary Score	SD (Mean Summary Score)	Guttman’s Split-Half Coefficient
OHIP-EDENT-CRO
Participants	95	0.92	0.39	30.21	15.95	0.83
OHIP-EDENT-SRB
Participants	177	0.87	0.25	19.51	7.60	0.74

SD=standard deviation

Table 3 presents mean scores of each item with standard deviations, corrected item-total correlation, Cronbach’s alpha when one item was deleted, and factor loadings (EFA) of both questionnaires. All item-total correlations were above 0.20. Even if one item was deleted, Cronbach’s alpha coefficient remained higher than 0.7.

Table 3

Mean item scores, standard deviations, corrected item-total correlations, Cronbach’s alpha coefficients when the item was deleted, and factor loadings of the OHIP-EDENT-CRO and the OHIP-EDENT-SRB; SD=standard deviation.

Questionnaire	OHIP-EDENT-CRO					OHIP-EDENT-SRB
	Mean	SD	Corrected item–Total correlation	Cronbach’s alpha if item deleted	Factor loadings	Mean	SD	Corrected item–Total correlation	Cronbach’s alpha if item deleted	Factor loadings
Difficulty chewing	2.60	1.24	0.57	0.92	0.56	1.17	0.65	0.50	0.84	0.75
Food catching	1.84	1.24	0.49	0.92	0.50	0.92	0.63	0.23	0.87	0.70
Dentures not fitting	2.43	1.46	0.48	0.92	0.60	0.55	0.64	0.28	0.87	0.52
Painful aching	.99	1.12	0.45	0.92	0.69	0.44	0.58	0.37	0.86	0.63
Uncomfortable to eat	2.32	1.39	0.64	0.92	0.61	1.25	0.55	0.46	0.86	0.56
Sore spots	1.67	1.22	0.56	0.92	0.68	0.64	0.57	0.24	0.87	0.40
Uncomfortable dentures	2.16	1.45	0.63	0.92	0.69	0.71	0.62	0.42	0.86	0.44
Worried	2.03	1.50	0.69	0.92	0.65	1.39	0.94	0.68	0.85	0.60
Self-conscious	1.94	1.62	0.51	0.92	0.62	2.68	0.83	0.37	0.86	0.69
Avoid eating	2.06	1.41	0.72	0.92	0.62	1.15	0.53	0.46	0.86	0.39
Unable to eat	1.72	1.37	0.62	0.92	0.71	1.07	0.48	0.44	0.86	0.58
Interrupt meals	1.35	1.33	0.65	0.92	0.73	0.16	0.43	0.34	0.86	0.42
Upset	1.91	1.54	0.79	0.91	0.78	1.59	0.97	0.63	0.85	0.67
Embarrassed	1.51	1.35	0.74	0.91	0.77	1.62	1.18	0.61	0.86	0.69
Avoid going out	0.87	1.21	0.56	0.92	0.71	0.83	1.00	0.72	0.85	0.69
Less tolerant	0.45	.84	0.57	0.92	0.71	0.03	0.21	0.21	0.87	0.66
Irritable with others	0.58	.95	0.52	0.92	0.64	0.56	0.82	0.56	0.86	0.61
Unable to enjoy company	0.71	1.12	0.51	0.92	0.73	0.90	0.75	0.68	0.85	0.70
Life less satisfying	1.07	1.26	0.61	0.92	0.66	1.93	1.07	0.58	0.86	0.64

SD=standard deviation

Test-retest reliability

Thirty CD wearers in each country answered the same questions of the OHIP-EDENT twice, without any treatment between the two administrations. The results are presented in Table 4. The ICC values were high in both countries (0.99 and 0.97 in Croatia and Serbia respectively).

Table 4

Test-retest reliability measured by intraclass correlation coefficients (ICC) for the OHIP-CRO-EDENT and the OHIP-SRB-EDENT); n=number of participants.

Questionnaire	ICC	95% confidence interval of the ICC	Mean difference	95% confidence interval of the difference	P
OHIP-EDENT-CRO
Complete denture wearers (n=30)	0.99	0.98–0.99	-0.37	-1.42–0.69	0.48 NS
OHIP-EDENT-SRB
Complete denture wearers (n=30)	0.97	0.93–0.98	-1.0	-2.01–0.70	0.11 NS

ICC=intraclass correlation coefficient

Concurrent validity

The concurrent validity was confirmed by a significant negative Spearman’s correlation coefficient between the OHIP-EDENT summary scores and one question in which participants rated satisfaction with their dentures. In Croatia the coefficient was -0.71 (P<0.001) and in Serbia -0.74 (P<0.001).

Construct validity

The KMO measure 0.871 and Bartlett’s Test of Sphericity 1,021.58 (df=171, P<0.001) in Croatia and the KMO measure 0.85 and Bartlett’s Test of Sphericity 1,348.96 (df=171, P<0.001) in Serbia demonstrated sufficient values and significant correlations enabling the performance of EFA. All factor loadings of the OHIP-EDENT-SRB questionnaire were >0.4 (except one, which was 0.39). All factor loadings of the OHIP-EDENT-CRO were >0.5 (Table 3). In both questionnaires, four factors (dimensions or domains) with eigenvalues >1 were extracted. Table 5 shows the eigenvalues and percentages of variance of non-rotated and rotated matrices. The OHIP-EDENT-CRO explains 66.65% and the OHIP-EDENT-SRB 59.35% of the variance. When fixed to one factor, the OHIP-EDENT CRO explained 42.51% of variance, while the OHIP-EDENT-SRB explained 30.66%. In non-rotated matrices, most of the items tended to concentrate on the first factor, while the rotated matrix presented more even distribution (Table 5).

Table 5

Factors with eigenvalue >1.0 before and after varimax rotation (Croatian OHIP-EDENT and Serbian OHIP-EDENT questionnaires).

Factor	OHIP-EDENT-CRO				OHIP-EDENT-SRB
	Unrotated		Rotated		Unrotated		Rotated
	Eigenvalue	% of variance	Eigenvalue	% of variance	Eigenvalue	% of variance	Eigenvalue	% of variance
1	8.08	42.51	3.88	20.41	5.83	30.66	4.75	24.98
2	2.10	11.04	3.38	17.78	2.72	14.33	3.22	16.94
3	1.35	7.08	3.03	15.93	1.50	7.90	1.77	9.33
4	1.13	5.96	2.37	12.47	1.23	6.44	1.54	8.08

Factor loadings after rotation are presented in Table 6.

Table 6

Factor loadings after varimax rotation of the OHIP-EDENT-CRO and the OHIP-EDENT-SRB.

Factor	OHIP-EDENT-CRO				OHIP-EDENT-SRB
	Component				Component
	1	2	3	4	1	2	3	4
Difficulty chewing	0.13	0.61	0.41		0.20	0.83
Food catching		0.35	0.61				0.83	0.11
Dentures not fitting	0.13	0.15	0.75			0.52	0.47	-0.15
Painful aching	0.10	0.13		0.81		0.71	0.35
Uncomfortable to eat		0.54	0.47	0.31	0.25	0.71
Sore spots	0.20	0.33		0.72		0.58	0.21
Uncomfortable dentures	0.16	0.28	0.76	0.13	0.19	0.44	0.45
Worried	0.21	0.67	0.33	0.21	0.73	0.15	0.21
Self-conscious		0.72		0.32	0.52			0.64
Avoid eating	0.28	0.50	0.47	0.27	0.41	0.38	0.11	-0.12
Unable to eat	0.23		0.69	0.42	0.29	0.26	0.40	0.52
Interrupt meals	0.38	0.19	0.25	0.70	0.19	0.60	-0.15
Upset	0.41	0.70	0.30	0.19	0.80			0.16
Embarrassed	0.57	0.64	0.19		0.81		-0.13	0.14
Avoid going out	0.79	0.26		0.13	0.76	0.20	0.19	-0.11
Less tolerant	0.76		0.26	0.26	0.22	0.18		-0.76
Irritable with others	0.74		0.23	0.19	0.60		0.40	-0.29
Unable to enjoy company	0.84	0.16			0.80	0.12	0.14	-0.21
Life less satisfying	0.71	0.33		0.21	0.77	0.10	-0.13	0.16

Responsiveness of the Croatian and Serbian versions of the 19-item OHIP was tested in CD wearers before and after dental implant placement in the mandible. The results are presented in Figure 2. Significant improvement of OHRQoL was achieved in both countries, as expected (t=20.9 in Croatia; t=5.05 in Serbia; p<0.001). The standardised effect size was high for both countries. However, it was higher in Croatia (3.9 in Croatia, 1.53 in Serbia).

Figure 2

Mean baseline and after-treatment scores of the Croatian and Serbian 19-item Oral Health Impact Profiles (before and after implant-supported overdenture treatment in the mandible).

Discussion

Each questionnaire needs to be validated before it can be used in different cultures. This is due to differences in language, education, amenities, culture, social or economic factors, which are especially important in a disease-specific population (30, 31, 32, 33). The need existed in Croatia and Serbia for a disease-specific questionnaire for edentulous subjects. Cross-cultural adaptation of the original OHIP-EDENT instrument was therefore performed (10). Both versions used the original five-point Likert scale instead of the three-point scale of the Brazilian questionnaires (20, 24, 37). Schools in Croatia and Serbia use a five-point scale, so the participants have already become accustomed to it. The higher summary score obtained among Croatian participants was attributed to the longer period of being edentulous and consequent alveolar ridge atrophy, especially in the mandible, which resulted in reduced denture stability (34, 35, 36).

Cronbach’s coefficient alpha for internal consistency was calculated to compare the results with other studies. It was higher in Croatia than in Serbia. It was higher than in the Portuguese (20, 24, 37), Japanese (18) and Nepalese OHIP-EDENT questionnaires (38). However, it was lower than in the Chinese version (19). Cronbach’s alpha coefficient in Serbia was similar to the Portuguese (20, 24, 37) and Japanese (18) versions. The differences between counties can be explained by differences in sampling strategy, duration of edentulism and cultural environments. However, both alpha coefficients were >0.7. When the alpha coefficient is close to one, redundant questions may exist. This was not found in this study. Cronbach’s alpha coefficient was not calculated for each of the originally proposed seven domains. A recent study on the dimensionality of the 49-item OHIP recommended that one summary score should be used in research using the shortened OHIP versions due to high correlations between factors and the small number of indicators for some dimensions (16). As Cronbach’s alpha coefficient has recently been criticised (39), another measure for internal consistency, i.e. Guttman’s Split-Half coefficient, was also calculated in this study. This revealed that the internal consistency of both questionnaires was good.

The test-retest reliability was also calculated for the summary scores. It was considered that a 15-day period between the two administrations of the questionnaire would be long enough for the participants not to remember the questions, but short enough to prevent changes to oral health. The high ICC values confirmed satisfactory test-retest reliability for the both instruments.

As already mentioned, an EFA was conducted for the purposes of a pilot analysis of the latent structure of the OHIP-EDENT questionnaire. The KMO measure, together with the significance of the Bartlett’s Test of Sphericity, confirmed that the data was appropriate for the EFA. All items had adequate loadings, indicating a strong relationship with their factors. The EFA extracted four factors (dimensions) from both questionnaires, similar to the Brazilian study by Souza et al. (24). Another study from Brazil extracted and confirmed three factors: “Physical Impact”, “Psychological Impact” and “Social Impact” (36). Five factors were found in the Chinese and Nepalese versions (18, 33). The four factors extracted in this study could not be interpreted in a clear manner. According to the content of the items, the dimensions could be: Function, Pain, Comfort and Psychosocial Impact. Due to a lack of questions related to orofacial aesthetics in the OHIP-EDENT questionnaire, the four extracted factors are not in line with the recent four-dimensional oral health model derived from the OHIP-49 questionnaires (2, 5, 14, 15, 16, 40, 41, 42, 43). The confirmatory factor analysis (CFA) was not performed in this study because a new sample was needed for the CFA in both Croatia and Serbia. It is possible that the deletion of some factors with cross-loadings or the insertion of some additional questions related to orofacial aesthetics would have led to a different pattern of factor loadings and groupings.

In the CFA study on the dimensionality of the OHRQoL using the 49-item OHIP, some items had been omitted because they had no salient loadings on one of the four proposed dimensions (16). Moreover, the study in question recommended that condition-specific versions, such as the OHIP-aesthetic and the OHIP-EDENT questionnaires (which was adapted in the present study), be used with one summary score, supported by the OHIP’s strong general factor (16). This was done in the present study. When fixed to one factor, the OHIP-EDENT-CRO explained 42.51% of the variance (total variance was 66.6%), and the OHIP-EDENT-SRB explained 30.66% of the variance (total variance was 59.34%).

The confirmatory factor analysis (CFA) was not performed in this study because a new sample in each country must be collected if an adequate CFA is to be performed. The CFA will be the object of further research to yield a better insight into the dimensionality of the OHIP-EDENT questionnaire and the item groupings. An attempt to adapt the OHIP-EDENT to fit into the newly 4-D concept of the OHRQoL through the addition of some items and the deletion of others may also be studied.

The convergent validity of the OHIP-EDENT-CRO and the OHIP-EDENT-SRB was confirmed by significant negative Spearman’s rank correlation coefficients between the summary scores and one question in which participants rated their dentures (which represented a measure of the same construct).

One important characteristic of the questionnaire is its sensitivity to changes elicited by treatment, which has been proved in many studies (44, 45). In Croatia and Serbia, CD therapy is covered by health insurance. However, dental implant costs must be paid by the patient. Two-implant supported overdenture has become a minimum standard of mandibular edentulism therapy (47) and, recently, four mini-implants for edentulous patients with slim mandibular ridges (48). We predicted that dental implants would improve OHRQoL in edentulous patients. The cost of implants in Croatia was covered by a research grant. As expected, both treatments increased OHRQoL and significantly reduced the OHIP-EDENT summary scores. After treatment, the summary scores reached almost equal values in both countries. However, a higher effect size was observed in Croatian participants due to their worse baseline oral health, probably resulting from a longer period of edentulism. The results are in line with other studies (35, 48). The Serbian participants had to pay dental implant costs and therefore might have had higher expectations.

Limitations of this study are the inclusion of participants with different edentulism durations and the relatively small sample size (which allowed only a pilot study to be conducted on the dimensionality). Further research should be directed towards performing the CFA on new samples from each country and towards the possible exclusion or inclusion of some new items in the OHIP-EDENT questionnaire (2, 4, 13, 14, 15).

Conclusion

The OHIP-EDENT-CRO and the OHIP-EDENT-SRB questionnaires demonstrated satisfactory internal consistency, validity and responsiveness. Further dimensionality will be the subject of future studies.