Romanian version of the oral health impact profile-49 questionnaire: validation and preliminary assessment of the psychometrical properties.

BACKGROUND AND AIMS: Oral Health Related Quality of Life (OHRQoL) represents a multidimensional structure, being measured by complex instruments, such as the Oral Health Impact Profile (OHIP). The aim of this present study is to develop and test the psychometric properties of an initial Romanian (OHIP-49Ro) version.
METHODS: The original OHIP-49 version was translated using a forward-backward technique into the Romanian OHIP-49Ro, which was applied in an interview form to 150 patients of the Second Medical Clinique of Internal Medicine, Cluj-Napoca. Confirmatory Factor Analysis (CFA) was applied, in order to evaluate the factor structure and construct validity of the OHIP-49Ro.
RESULTS: The correlations between the OHIP-49Ro subscales were all positive and statistically significant. Cronbach's Alpha coefficients values are above 0.7 for all subscales, providing support for the internal consistency of OHIP-49Ro scale scores. Regarding the CFA, for the seven factor model, the Bentler scaled chi-square (S-Bχ(2)) indicated a value of 2193.74 (df=1091; p=0.001), the CFI a value of 0.740, the TLI a value of 0.72 and RMSEA the value of 0.82.
CONCLUSIONS: The results of this study suggest a high internal consistency of the OHIP-49Ro instrument. Due to the correlations between several sets of factors, and the multiple-factorial load for several items, the OHIP-49Ro's factorial structure requires further research on different samples and in different cultural and educational contexts.

Background and aims

Concepts such as “Oral Health” and “Quality of Life” usually have an ambiguous character and often remain insufficiently defined or are prone to a very large range of interpretations. Moreover, they tend to find themselves under continuous progress and to be influenced by factors like: subjectivity, socio-cultural, economical, geo-political, educational or linguistic contexts [1].

According to WHO, “Health” is defined as “a state of complete physical, mental, and social well-being and not merely the absence of disease or infirmity” [2]. The concept of “Quality of Life” consists of both objective and subjective assessment items of an individual’s physical, material, social, emotional wellbeing and personal evolution and enterprise, all influenced by a personal body of principles [3]. It has been stated that in a modern definition of health, psychological and subjective outcomes have to be considered [4], thus connecting the two approaches and leading to the more complex notion of “Health related Quality of Life” (HRQoL), which is an elaborate approach, including five main directions: resilience, perception of health, functional conditions, impairments or diseases and lifespan [5].

In connection with the HRQoL stands the Oral Health related Quality of Life (OHRQoL), which combines “survival, absence of impairment, disease or symptoms; appropriate physical functioning, emotional functioning, social functioning; perceptions of excellent oral health; satisfaction with oral health and absence of socio-cultural disadvantage due to oral status [6]. Until the late 80’s, there was a relative absence of specific instruments to measure the OHRQoL [7,8], but in time a wide range of such indices, under the form of questionnaires, has been developed [8], such as The General Oral Health Assessment Index [1,9] or The Dental Impacts in Daily Living [1,10].

One of the most used, complex and comprehensive indices is the Oral Health Impact Profile (OHIP-49) [11,12], developed by Slade and Spencer [1,13], based on Locker’s conceptual model of oral health [4], which is derived from the World Health Organization’s classification of the impacts of disease [14]. Built with the purpose to assess the socio-psychological impacts of oral pathology [13], OHIP-49 has been successfully used in clinical studies and has proven good psychometric properties [8,15,16]. The OHIP-49 questionnaire has also been translated, validated and applied in various countries: Germany [17], Hungary [18] or China [19]. Also a short form of the index, OHIP-14, has been developed, for easier use in clinical studies [7]. To our knowledge, in Romania only the short form OHIP-14 has been translated, cross-culturally adapted and validated [20] so far, with reliable clinical implement. The validation of an index follows the stages of forward-backward translation, cross cultural adaptation, application of the index on patient samples and testing of the psychometric properties [17,18].

The aim of this present study is to develop and test the psychometric properties of the initial Romanian (OHIP-49Ro) version of the original English OHIP-49 form.

Methods

The Oral Health Impact Profile

The original OHIP-49 consists of 49 questions, divided after Locker’s seven conceptual dimensions of impact (or subscales): functional limitation, physical pain, psychological discomfort, physical disability, psychological disability, social disability and handicap. Using Thurstone’s method of paired comparisons, each statement within a subscale received a specific weight. The interviewed patients are asked to indicate how often they experienced a specific impact in the last year. Responses for each question are structured on a five-point Likert-Scale and encoded as following: never (0), hardly ever (1), occasionally (2), fairly often (3) and “very often” (4). Also a “Don’t know” option is included, which, together with any blank entries, are encoded as missing values [1].

Translation and back-translation of the English-language Oral Health Impact Profile

The development of the initial form of the OHIP-49Ro followed two steps: first, the index was translated and cross-culturally adapted; the second stage referred to the assessment of the psychometric properties, meaning the reliability and validity of the OHIP-49Ro in a conventional target population.

After obtaining the permission from the author to perform the translation, the original English version was forward translated by a Romanian-English bilingual dentist and a bilingual layperson,. The two variants were merged and verified by a third bilingual dentist. The obtained version was back-translated by two bilingual dentists, distinct from the ones involved in the forward translation and sharing the same high level of English, and a different linguist. The three back-translated versions were all merged in one final version, which was compared with the original-English questionnaire. After a discussion, the consent upon a final version to be used in the study was achieved. The same number of 49 questions and response structure as in the original index was kept, in order to allow an international resemblance testing.

Design and Participants

The present study is a cross sectional survey study type. The questionnaire was administered to a group of 150 patients from the Second Medical Clinic of Internal Medicine, Cluj-Napoca, Romania. The group was chosen on the convenience sample criteria and was homogenous regarding gender and age (60% F, 40% M; 18–88 years). Approval of the Ethics Committee of the Iuliu Hattieganu University of Medicine and Pharmacy, Cluj-Napoca, was obtained. Before completing the questionnaire, a form containing the brief description of the purpose of the study, as well as the patients’ rights, was read to each patient. Informed consent has been provided by each patient. Since the patients were lay persons, with different levels of educational background, the questionnaire was distributed under the interview-format, by three, pre-trained interviewers. The interviewers read literally each item to the patient, as well as the answer possibilities, and registered the patients’ option.

Missing data

According to Slade [1], the questionnaire is discarded if more than nine responses are left blank or marked with the “don’t know” option. In our sample, no questionnaire was discarded based on these criteria. Missing answers (<1%) were imputed using item mean imputation.

Score computing

Scores were calculated by summing, for each patient, the impact responses in the score interval 0–4, within each subscale. An overall OHIP-49Ro score was also computed. Mean values for each subscale, within the study sample (N = 150) were obtained. Item weights were not generated, due to the fact that their application is questionable in the literature [17,18,21].

Data analysis

The data was screened for indices of non-normality. Univariate normality at item level was assessed through computing skewness and kurtosis for each item, while multivariate normality was assessed through Mardia multivariate kurtosis test. Normality indicators, univariate and bivariate descriptive statistics at item and scale level were computed using the IBM SPPS (version 22) software package.

We used the Confirmatory Factor Analysis (CFA), representing the sequential model testing approach recommended by Joreskog [22], to evaluate the factor structure and construct validity of the OHIP-49Ro. In the first step, separate single-factor models corresponding to individual subscales were specified in order to assess convergent validity. In the second step, each two-factor model obtained by pairing all the subscales was tested in order to assess discriminant validity of the factors. Finally, all factors were included in the full seven-factor model. Factor correlations for the final model were set free, excluding the possibility of an orthogonal factor structure. For all measurement models, factors metric was set by the unit loading identification (ULI) method, constraining item loading of one item on its latent construct to unity for each scale.

In order to test the specified measurement model, the following goodness of fit statistics was used: the Satorra–Bentler scaled chi-square (S–Bχ2) test (with robust standard errors adjustment for non-normality), the comparative fit index (CFI) [23], the Tucker-Lewis fit index (TLI) [24], and the RMSEA (root mean square error of approximation) [25]. CFI and TLI values close to 0.95, along with RMSEA values close to .06, were accepted as indicators of a good model fit [26]. The robust maximum likelihood estimation method was used to estimate the parameters values of the measurement model. The data was analyzed using the MPLUS program (version 7.2) [27]. The coefficient alpha reliabilities of the factors in the final model was also assessed, values above 0.70 indicating acceptable reliability.

Results

Item level univariate descriptive statistics and normality analysis

The items had skewness values ranging from 0.13 to 2.74 and kurtosis ranging from 1.39 to 7.26, values, that are above the established cutoffs of |2| and |7| [28], indicating that item scores are not normally distributed. Multivariate normality was assessed using Mardia’s normalized (standardized) coefficient of multivariate kurtosis. That statistic was 367.49, for the 49 items of the OHIP-49Ro, a value that is well over the recommended cutoff of |3.0| [29] and the items were determined to be multivariate non-normal. Mean score of OHIP-49Ro items ranged from 0.27 to 2.12 and standard deviation from 0.61 to 1.42.

Subscale level univariate descriptive statistics, inter-item and subscale correlations and internal consistency

Subscale mean ranged from 2.66 (Social Disability) to 12.52 (Physical Pain), the standard deviation being the highest for Physical Disability (SD=7.89) and lowest for Social Disability (SD=3.27) (Table I).

Table I

Univariate and bivariate descriptive statistics for OHIP-49Ro total and subscale scores. (N=150).

	Scale (items)	Mean	Std. Deviation	1	2	3	4	5	6	7	8
1	FL (9)	11.90	7.19	.798
2	PhyP (9)	12.52	7.53	.723**	.840
3	PsyD (5)	6.38	5.58	.537**	.580**	.899
4	PhyD (9)	8.26	7.89	.670**	.787**	.578**	.883
5	PsyDis (6)	6.25	5.53	.543**	.638**	.691**	.686**	.869
6	Sdis (5)	2.66	3.27	.286**	.429**	.510**	.423**	.656**	.746
7	H (6)	2.70	3.59	.289**	.380**	.469**	.337**	.486**	.693**	.751
8	OHIP-49Ro	50.70	32.46	.798**	.877**	.787**	.870**	.844**	.643**	.578**	.954

- the correlation is significant at 0.01 level;

FL - Functional Limitation, PhyP- Physical Pain, PsyD- Psychological Discomfort, PhyD- Physical Disability, PsyDis- Psychological Disability, Sdis-Social Disability, H- Handicap, OHIP-49Ro - Total score

Inter-item correlations for the 9 items of the Functional Limitation subscale ranged from r=0.117 to r=0.555, six of the 36 item correlation being statistically non-significant (p>0.05). For the Physical Pain subscale none of the 36 item correlation was found to be non-significant, their values ranged from r=0.167 to r=0.526, all of them being positive. Correlation between Psychological Discomfort subscale items ranged from r=0.166 to r=0.924, all of them being found significant (p<0.01). Inter-item correlations for the 9 items of the Physical Disability scale ranged from r=0.117 to r=0.555, none of the 36 item correlation being statistically non-significant (p>0.05). Correlation between Social Disability subscale items ranged from r=0.308 to r=0.606, all of them being found significant (p<0.01). At last, inter-item correlations for the 6 items of the Handicap scale ranged from r=0.220 to r=0.592, none of the 15 item correlations being statistically non-significant (p>0.05).

The correlations between the OHIP-49Ro subscales were all positive, statistically significant (Table I), and consistent with those found in other validity studies [17,18,19]. Cronbach’s Alpha coefficients values are above 0.7 for all subscales, providing support for the internal consistency of OHIP-49Ro scale scores (see the main diagonal of Table I).

Confirmatory factor analysis

In the first step of the confirmatory factor analysis, all single-factor models (corresponding to individual subscales) were tested, in order to assess the convergent validity of each subscale. Authors considered overall fit indices and standardized factor loadings. Four of the seven chi-square statistics for model fits were significant, but given the relative large sample size, these indices have a reduced diagnostic value. RMSEA values ranged from 0.019 to 0.11, offering acceptable support to model fit. The great majority of CFI and TLI indices are all about the established cut-off (0.95) indicating an excellent model fit (Table II). Factor loadings were all significant and their standardized value ranged from 0.353–0.696 (for Functional Limitation), 0.362–0.804 (for Physical Pain), 0.629–0.904 (for Psychological Discomfort), 0.468–0.918 (for Physical Disability), 0.59–0.841 (for Psychological Disability), 0.522–0.680 (for Social Disability) and 0.291–0.86 (for Handicap).

Table II

Confirmatory factor analysis model fit indicators (N=150).

Model		S–Bχ2	df	p	χ 2/df	CFI	TLI	RMSEA	RMSEA 95%CI
One Factor*	FL	43	25	.014	1.72	.941	.914	.073	.03–.1
	PhyP	40.24	26	.037	1.54	.964	.95	.066	.01–.09
	PsyD	4.9	3	.179	1.635	.996	.986	.065	.001–.165
	PhyD	61.34	22	.001	2.789	.948	.914	.110	.078–.143
	PsyDis	16.74	7	.019	2.392	.976	.948	.097	.037–.157
	Sdis	3.16	3	.367	1.056	.999	.997	.019	.001–.141
	H	10.09	8	.258	1.262	.989	.98	.042	.001–.110
Two Factor	FL&PhyP	327.62	131	.001	2.501	.795	.760	.1	.087–.114
	FL&PsyD	105.19	72	.007	1.461	.959	.949	.056	.030–.078
	FL&PhyD	351.72	127	.001	2.769	.824	.788	.109	.096–.123
	FL&PsyDis	120.24	85	.007	1.415	.954	.944	.053	.028–.073
	FL&Sdis	94.97	72	.036	1.319	.953	.941	.046	.013–.07
	FL&H	139.27	86	.001	1.619	.901	.88	.064	.044–.084
	PhyP&PsyD	116.97	73	.001	1.602	.953	.941	.064	.041–.084
	PhyP&PhyD	349.77	128	.001	2.733	.838	.806	.108	.094–.121
	PhyP&PsyDis	150.02	86	.001	1.744	.930	.915	.071	.051–.089
	PhyP&Sdis	97.08	73	.031	1.33	.96	.951	.047	.015–.07
	PhyP&H	127.89	87	.003	1.470	.935	.922	.056	.034–.076
	PsyD&PhyD	58.07	40	.032	1.452	.974	.964	.055	.017–.084
	PsyD&PsyDis	85.11	39	.001	2.182	.954	.936	.089	.063–.115
	PsyD&Sdis	34.27	30	.27	1.142	.994	.990	.031	.001–.072
	PsyD&H	58.07	40	.032	1.452	.974	.964	.055	.017–.084
	PhyD&PsyDis	254.58	82	.001	3.105	.869	.832	.119	.102–.136
	PhyD&Sdis	159.52	69	.001	2.312	.908	.879	.094	.075–.113
	PhyD&H	169	83	.001	2.036	.914	.891	.083	.065–.101
	PsyDis&Sdis	73.3	39	.001	1.879	.950	.929	.077	.049–.104
	PsyDis&H	88.33	50	.001	1.767	.942	.923	.072	.046–.096
	Sdis&H	74.57	39	.001	1.912	.933	.906	.078	.051–.105
Seven Factor	All subscales	2193.74	1091	.001	2.011	.740	.72	.082	.077–.087

Due to wording similarities several error variance correlates were allowed for each factor;

FL - Functional Limitation, PhyP- Physical Pain, PsyD- Psychological Discomfort, PhyD- Physical Disability, PsyDis- Psychological Disability, Sdis-Social Disability, H- Handicap; CFI – Comparative Fit Index, TLI - Tucker-Lewis Fit Index, RMSEA - Root Mean Square Error of Approximation

The purpose of testing two-factor models, obtained by combining each pair of subscales, was to test discriminant validity of the factors and identify possible double loading items. In order to achieve this goal, correlation between factors and modification indices of possible cross-loaded items were analyzed.

The correlation between factors ranged from 0.32 (between Physical Disability and Handicap) to 0.984 (between Social Disability and Handicap). Besides the high correlation between Social Disability and Handicap there are other three correlations which are seriously questioning the discriminant validity of OHIP-49Ro scales, namely the correlation between Psychological Discomfort and Psychological Disability (r=0.821), Social Disability and Psychological Disability (0.830) and Functional Limitation and Physical Pain (r=0.856).

Regarding modification indices, several two-factor models showed an unacceptable low model fit, which could be increased by specifying cross-loaded items. Thus, the fit of Functional Limitation and Physical Disability model could be improved, by loading items 16 (“sore spots”) and 18 (“uncomfortable dentures”) on the Physical Disability factor too (reducing with 53.54, respectively with 25.87 the value of chi square). In order to increase the Functional Limitation and Physical Pain model fit, item 7 (“food catching in teeth/dentures”) should also be loaded on the Physical Pain factor (reducing the computed chi square with 71.62). The statistical software suggested three modifications in order to improve the fit of the Physical Pain and Physical Disability model: item 11 (“headaches”) and 19 (“worried about dental problems”) should also be loaded on the Physical Disability factor (reducing the chi square value with 21.66 and 19.07) and item 28 (“avoiding eating some foods”) should be loaded on the Physical Pain factor too (reducing the chi square value with 21.92).

The Physical Disability and Psychological Disability two factor model fit could be improved by loading item 32 (“interrupting meals because of problems with tooth/mouth/dentures”) on the Psychological Disability factor too (reducing the chi square value with 20.07), loading item 34 (“upset because of dental/oral problems”) and 35 (“difficulty in relaxing because of dental/oral problems”) on the Physical Disability factor too (reducing the chi square value with 17.98 and 17.97). Another specification to improve the Physical Disability and Social Disability model fit would be to load item 39 (avoiding going out) on the Physical Disability factor too (reducing the chi square value with 17.98 and 18.51). The last suggested specification change in order to increase the fit of Physical Disability and Handicap model with the data would be to load items 32 and 26 (less flavor in the food) on the Handicap factor too, in this way reducing the chi square value by 12.3 and 10.03.

The cumulative effect of these specification errors affects the model fit of the seven-factor model, all the model fit indicators being situated below the accepted threshold. At last, some items of the full seven-factor model have also low standardized factor loadings: item 7 (0.342), item 27 (0.314) and item 45 (0.292).

Discussions

The present study’s main focus was to investigate the factor structure and construct validity of the OHIP-49Ro, in order to proper assess the quality of its cross-cultural adaptation.

The evaluation of the factor structure was previously applied on the original OHIP-49 instrument, using a Confirmatory Factor Analysis, as well as an Exploratory Factor Analysis, representing application of the more complex Structural Equation Modeling [30] and providing information about whether items developed to measure a specific dimension manage to achieve that goal, or about the ability of the index to quantify seven distinct dimensions [30,31].

The results obtained in this study tend to justify only partially the OHIP-49Ro validity:

Firstly, the high level of correlation between several factors (the high correlation between Social Disability and Handicap, the correlation between Psychological Discomfort and Psychological Disability and the correlation between Social Disability and Psychological Disability, Functional Limitation and Physical Pain) could indicate that the conceptual delimitation between two (or more) factors is not clearly defined, or, that the constructs were correctly defined and operationalized in the original cultural context, but the same items do not load on the same way on the initially defined factors, given possible cultural differences. As a consequence, items, which are supposed to measure different dimensions, may actually measure the same factor. Also, the correlation between the sets of two factors, in the case of OHIP-Ro, can be caused by specific conceptual similarities between them, a situation noticed in other studies as well [30].

Locker [30] identifies correlations between sets of two factors, different from those obtained in the present study: Functional Limitation-Discomfort, Functional Limitation-Physical Disability; and similar to the present study: Discomfort-Psychological Disability. Locker proposes the merging of discomfort and psychological disability intro a single construct, obtaining thus a new model variant and also proposes a revised index model with a reduced number of items, due to the high standardized residual covariance values (>2.58) of the excluded items.

Secondly, another problematic result of our study is the absence of “simple structure” [32], namely there are many items which are loading on more than one factor (E.g. item 11 – “Have you had headaches because of problems with your teeth, mouth or dentures?” loads on both Physical Pain and Physical Disability factors). Such items, which can be attributed to more than one factor, during the development stage are usually merged with another item, or are removed out of the instrument’s structure.

Although the authors opted for the interview format, in order to overcome the variation in the educational profile of the patients (this applying approach being alongside with the methodology used in the validation of Hungarian [18], Chinese [19] or Brazilian [33] version), this educational state may also generate a reduced discriminating ability of the items’ meaning, resulting in items which simultaneously quantify two or more factors. The inclusion criteria did not follow a specific pathology of the patients, because the purpose of the study was to assess OHIP-49Ro’s psychometrical properties.

Further research is thus needed with the purpose of revising and perfecting the contextual item meaning, analysis of the items with multiple factorial load, reconstruction of such items and possible a clear conceptual identification of the seven factors.

OHIP-49Ro had presented positive and statistically significant correlations between the subscales, the Cronbach’s Alpha coefficient values indicating the OHIP-49Ro scale scores consistency. The findings were similar to other studies in the literature [17,18,19]. However, a test-retest reliability will be, needed in the future.

Conclusions

OHRQoL indices should represent standardized instruments, able to measure the impact of oro-dental pathology, in both physical and psycho-social dimensions. OHIP-49 meets such criteria, representing a general acknowledged complex instrument. The results of this study suggest a high internal consistency of the OHIP-49Ro instrument. Although the literature presents evidence of the different OHIP-49 validations’ concurrent and predictive validity, the OHIP-49Ro’s factorial structure remains unclear, thus highlighting the necessity of further studies on different samples and in different cultural and educational contexts, in order to obtain a final version, which can be applied on Romanian patient samples.