A Rasch Analysis between Schizophrenic Patients and the General Population.

BACKGROUND: The aim of this study was to test the General Oral Health Assessment Index Questionnaire (GOHAI) items for differential item functioning (DIF) according to demographic characteristics (gender, age) and mental health status (schizophrenic disorders versus general population) using Rasch analysis.
METHODS: This is a cross-sectional study using aggregated baseline data from a validation study of the GOHAI in the French general population (GP) and similar validation study in persons with schizophrenia (PWS). DIF was tested using the Partial Credit Model. DIF were estimated in different groups of patients.
RESULTS: The cohort comprised 363 persons: 65% were female, 83% were 25-45 years old and 30% were PWS.Five of the 12 items exhibited DIF. DIF effects were observed with schizophrenia for 3 items, with age for 3 items and with gender for 1 item. The variable "age" gave a significant explanation of the latent variable: the latent variable decreased with age (-0.40±0.08-p<0.001 for each increase of the age of 10 years). This decrease represented an effect size of 0.27 which can be qualified of a small to medium effect. The status of the individuals (GP versus PWS) and the gender did not significantly explain differences in the values of the latent variable.
CONCLUSION: The GOHAI scores may not be comparable across sub-groups defined by health status, age and gender without accounting for DIF. In the future, other studies should explore this way with other Oral Health related Quality of Life assessment tools and populations with mental illness.

Introduction

Measures of self-perceived oral health are useful tools for assessing oral health, identifying health disparities and evaluating the impact of health interventions at a population level [1, 2]. Indeed, Oral Health related Quality of Life (OHrQOl) measures, such as the General Oral Health Assessment Index Questionnaire (GOHAI) can provide valuable information about oral health and its components, such as pain and discomfort, dysfunctions or the psychosocial impacts of oral diseases [3].

The GOHAI has already been widely used in clinical or epidemiological studies worldwide as it is available in different languages. It includes three sets of response categories (three, five, and six categories) that generate a high GOHAI score for people with satisfactory oral health [3]. The questionnaire is adapted to study various types of adult populations. The French version of the GOHAI has been validated for use in a population aged 18-45 years old [4] while the Italian version has been adapted for an older population (mean age 75 years, range 59-95) [5]. A recent study showed that the French GOHAI had good psychometric characteristics for adults with schizophrenia and construct validity was supported by three factors [6]. Despite confirmation of three factors structure for PWS reporting of the total score remains a common practice which implicitly assumed a one-dimensional nature of the scale.

Schizophrenia is a severe, disabling psychiatric disorder with either episodic or continuous evolution that can result in physical, psychological and social problems related to both the disease and the potential side effects of its treatment [7, 8]. Persons with schizophrenia (PWS) have excess mortality (their life expectancy is reduced by 20%) and excess morbidity [9]. Among somatic comorbidities in PWS, poor oral health has been shown to contribute to the overall poor health of these patients [10]. Generally, schizophrenia leads to disturbances in the progression of thought, errors in contextual analysis and errors of logic. Often PWS do not recognise their health needs and delay seeking advice or treatment [11]. In recent decades, a number of studies have reported poor oral health in PWS [12, 13], but few studies have explored self-perceived oral health in PWS [14]. Oral health is a complex concept differently defined by each person according to the understanding of what a healthy mouth is, the type of symptoms already experienced, cultural values, past experience with the health care system, general health, psychosocial wellbeing, the impact of severe mental illness and age or gender [15].

Searching for differential item functioning (DIF) in an OHrQOL questionnaire is a way to explore the equivalence of items across population sub-groups [16]. DIF occurs when individuals with the same underlying (i.e., latent) level of health do not interpret a measure’s items in the same way. Awareness of this bias is of particular importance in clinical research where scale scores are used to investigate gender, ages or the mental status differences and ensure that derived scores are comparable across groups. A lack of measurement equivalence at the item level, may lead to spurious mean differences in the observed scores between this parameters, because one cannot be certain there is a meaningful difference, thereby making mean score differences un-interpretable [17]. It is thus fundamental to check whether items function similarly across different population groups in order to understand more clearly the psychological characteristics of OHrQOL. Consequently, in the presence of DIF, a latent variable model is well suited for data analysis because it can estimate the different values of the parameters related to each item for the different groups of individuals. Among the latent variable models, the Rasch analysis is frequently used because of its simplicity: only one parameter for each answer category is required. Moreover, with the Rasch model, it is possible to estimate parameters without bias even in the presence of missing data [18].

In OHRQOL, one study showed that the impact of DIF across gender on the overall score was minimal for the Children Perception Questionnaire (CPQ11-14) [19]. However, DIF were found in the CPQ11-14 for cultural factors and ethnicity [19, 20, 21]. For the Pediatric Quality of Life Inventory (PedsQL) the parent-proxy report was found to be inferior to the student self-report [22]. As far as we know, DIF has never been investigated for the GOHAI questionnaire. Furthermore, the impact of mental illness on DIF for OHrQOL questionnaires has never been explored.

The aim of this study was to test the GOHAI items for DIF according to demographic characteristics (gender, age) and mental health status (schizophrenic disorders versus general population) using Rasch analysis.

Methods

Data were extracted from two previous crosssectional studies.

The first one was the French validation study of the GOHAI conducted in 2000-2001 among a sample of 255 adults in the “Puy de Dome” department (France) [4]. Disadvantaged adults were over-represented in this sample and their age varied from 18 to 45 years. The sample included low-income earners benefiting from extended public health coverage.

The second study was the validation study of the GOHAI conducted in 2015 in a PWS sample [6]. Data were collected within a multicentre cross-sectional descriptive study (BUCCODOR). A cluster sampling method was used to recruit 108 PWS aged 21-75 years in the “Côte d’Or” department (France). Schizophrenia was defined according to the standard classification of mental disorders (DSM-V).

For both studies, data were collected using personal interviews and dental examinations. Details of the methodology used for sampling and data collection have been described in previous publications [4, 23].

This study has been approved by the ethics Committee for the Protection of Persons (CPP) number I of Eastern France (registration number: 2014-A00358-39) and the Comité National Informatique et liberté in Paris which gave permission and set conditions for the collection of personal information for the study.

The GOHAI is an OHrQOL questionnaire, which was initially validated for use in an elderly population in North America [3]. The questionnaire is composed of 12 questions with nine negative questions and three positive ones in order to discourage respondent acquiescence. The 12 questions assess physical functions (eating, talking and swallowing) for items 1, 2, 3 and 4 and psychosocial impacts (self-esteem, social withdrawal and worries about oral health) for items 6, 7, 9, 10 and 11. Items 5, 8 and 12 explore pain and symptoms (use of drugs to relieve pain, discomfort) related to the presence of oral diseases. There are five response categories with an associated score (l=always, 2=often, 3=sometimes, 4=seldom, and 5=never). Scores from the positively worded questions are reversed to calculate the global score so that the direction of all responses is the same. The GOHAI score is computed by adding up the scores of the 12 responses so that the highest score (60) indicates excellent oral health.

The sample size for Rasch analysis should be at least equal to the number of questions multiplied by the number of answer categories [24]. Given that a 5-point Likert scale was used, the minimal number of participants needed was 60. Nevertheless, a sample of at least 200 individuals is generally used [25]. The sample size (n = 363) obtained with the data from the two studies was thus considered sufficient.

For the Rasch analysis, item scores were ordered so that low values represented the worst level of OHrQOl and that high values represented satisfactory OHrQOl. For example, the answers to question 3 (“How often were you able to swallow comfortably?”) was coded from 0 (never) to 4 (always), while those to question 4 (« How often have your teeth or dentures prevented you from speaking the way you wanted?”) were coded from 0 (always) to 4 (never).

The Partial Credit Model (PCM) was applied to examine the GOHAI [24]. Ad-hoc tests of fit of the PCM were realized using RUMM 2030 software. We used the chi squared statistics of RUMM to test the fit of the PCM to the dataset.

The criteria of infit and outfit mean-square (MnSq) were set at 0.6 to 1.4. The fit of items outside this range was considered poor [26]. Items were characterized by using difficulty parameters δ with the unit of logit (a standardized score with mean as 0 and SD as 1 for unit of the latent variable) and were calculated for each item [27].

In case of dysfunctioning items (items presenting difficulty parameters in an unexpected order compared to the codes used for each answer category) they were recorded by collapsing adjacent answer categories, until each recoded item became a functioning item and supported the unidimensionality of the model [27].

In order to investigate the DIF, these parameters were estimated in different groups of patients: general population (GP) sample versus PWS, males versus females, younger (18-24) versus medium (25-45) or older subjects (45-74). Then the estimates of these parameters among groups were compared and only significant differences at 5% were retained.

Then, the latent variables were explained by the same variables as those used for the DIF analysis: gender, age, and health status of the individuals (GP or PWS). Interactions with age, gender and health status of the individuals were tested. Only significant variables at 5% were retained.

Results

The overall sample included 369 subjects; 65% were female, 30% were PWS and 75% were aged 25-45 years. The gender and age distributions of the population per group are presented in Table 1 and Fig. 1. The mean GOHAI scores for PWS and GP were 45.5 (SD=8.41) and 46.43 (SD=9.46), respectively.

Table 1

Gender and age distributions of subjects by schizophrenia or not

	person without schizophrenia	person with schizophrenia
	n(%)	n(%)
Gender
Male	117 (45.9)	68 (63)
Female	138 (54.1)	40 (37)
Age (years)
18-24	39 (10.7)	-
25-45	216 (59.5)	55 (15.1)
46-75	-	53 (14.6)

Values are represented as n (%)

Fig. 1

Age distribution between persons with and without schizophrenia

The fit of the PCM was significant (p<0.001), even if we adjust the statistics to a lesser number of individuals in order to evaluate the impact of the overpower of this test of fit due to the relatively large sample size (p=0.0293 with n=200). This could be explained by presence of DIF and of dysfunctioning items. In particular, items 3, 4, 5, 8, 10 and 11 presented a poor fit of the PCM to the data (Table 2 without DIF).

Table 2

Differential item functioning of the GOHAI items

Without DIF				With DIF
Items	Difficulty Parameters	Fit test (p-values)	DIF Group	DIF Sex	DIF Age	Difficulty Parameters	Fit test (p-values)
Item 1	-4.7±0.4	0.18				-4.6±0.5	0.27
	-2.6±0.3					-2.5±0.3
Item 2	-3.63±0.4	0.10				-3.5±0.4	0.71
	-2.04±0.3					-1.9±0.3
Item 3	-2.96±0.4	0.0008	GP		18-24	NA	0.60
	-3.47±0.3					-3.8±0.7
			GP		25-45	-2.4±0.5	0.0474
						-3.8±0.4
			PWS		25-45	-3.9±0.8	0.86
						-2.6±0.5
			PWS		46-75	-3.1±0.7	0.80
						-2.8±0.5
Item 4	-3.99±0.4	0.0227	GP			NA	0.61
	-3.40±0.3					-5.2±1.1
			PWS		25-45	-4.0±0.7	0.91
						-3.4±0.3
			PWS		45-75	-3.6±0.7	0.75
						-2.3±0.5
Item 5	-3.2±0.3	0.0167	PWS	M		-3.6±0.6	0.0269
	-1.8±0.3					-1.2±0.4
			GP	M		-2.3±0.4	0.0490
						-2.0±0.4
			PWS	F		-2.9±0.6	0.46
						-1.7±0.5
			GP	F		-3.7±0.5	0.32
						-1.8±0.4
Item 6	-4.6±0.5	0.55				-4.6±0.5	0.66
	-3.5±0.3					-3.4±0.4
Item 7	-3.0±0.3	0.26	PWS			-3.4±0.5	0.36
	-1.6±0.3					-1.5±0.4
			GP			-2.7±0.4	0.36
						-1.5±0.3
Item 8	-5.1±0.4	0.0331				-5.0±0.4	0.82
	-1.6±0.3					-1.5±0.3
Item 9	-3.8±0.3	0.12				-3.7±0.4	0.21
	-1.1±0.3					-1.0±0.3
Item 10	-4.1±0.4	0.0014			18-24	-4.2±1.2	0.63
	-1.8±0.3					-2.8±0.6
					25-45	-3.8±0.4	0.0348
						-1.6±0.3
					46-75	-4.6±0.7	0.0472
						-1.4±0.5
Item 11	-4.0±0.4	0.0095				-3.9±0.6	0.0842
	-3.1±0.3					-3.0±0.3
Item 12	-3.9±0.3	0.69				-3.8±0.4	0.18
	-0.4±0.3					-0.3±0.3
Global Fit		<0.001					<0.001
Global Fit (n=200)		0.0293					0.0697

GP: general population; PWS: persons with schizophrenia; M: male; F: female; NA: not applicable

In Fig. 2, the curves show the probabilities of response for item 1 as a function of the latent trait. The curves for all the items presented the same global appearance with a translation on the right or on the left on the curves following the items. We can see from these curve that modality 3 is rarely used.

Fig. 2

These curves show the measure relative to item difficulty and the probabilities of response as a function of the latent trait. The y-axis represents the probability (0–1) of responding to one of the rating categories, and the x-axis represents the different performance values in logits. This is question 1 of the questionnaire; the curves for all the items presented the same global appearance with a translation on the right or on the left on the curves following the items.

All items were considered dysfunctioning items, and answer categories coded 1 and 3 were collapsed with an adjacent answer category. Consequently, the answer categories “Never” and “Seldom” on the one hand and the answer categories “Sometimes” and “Often” on the other hand were collapsed. Note that for the three inversed items, the collapsed answer categories were “Always” and “Often” on the one hand, and “Sometimes” and “Seldom” on the other hand (Fig. 3). There are thus two difficulty parameters δ per item instead of four.

Fig. 3

For items 3, 5 and 7 by collapsing the terms sometimes and often, the adequacy of the Rasch model was better because there were two difficulty parameters δ per item instead of four.

After collapsing the answer categories, no item was dysfunctioning and the fit to the model was significantly improved even though it was still unsatisfactory (the p-value was still below 0.001).

DIF was searched for by exploring gender, age and health categories. Table 3 presents the items affected by DIF.

Table 3

Items significantly affected by DIF

Items	Health status	GP between 18 and 24 years to	Gender
	PWS vs GP	PWS between 46 and 75 years	F vs M
Item 3: How often were you able to swallow comfortably?	p<0.001	p=0.034
Item 4: How often have your teeth or dentures prevented you from speaking the way you wanted?		p=0.002 only for PWS
Item 5: How often were you able to eat anything without feeling discomfort?	p=0.003		p=0.033
Item 7: How often were you pleased or happy with the appearance of your teeth, gums or dentures?	p<0.001
Item 10: How often did you feel nervous or self-conscious because of problems with your teeth, gums or dentures?		p=0.011

GP: general population; PWS: persons with schizophrenia; M: male; F: female;

Among the 6 items (3, 4, 5, 8, 10 and 11) with an initial poor fit of the PCM, 4 presented DIF (3, 4, 5 and 10), while only one item (7) also presented DIF.

The handling of the DIF resulted in a significant improvement of the log-likelihood of the model (p=0.002), even if the fit of the PCM always is significant (p<0.001). By adjusting the size of the sample size to evaluate the impact of the overpower of the test of fit due to the relatively large sample size, we obtained an unsignificant fit (p=0.0697 with n=200), (Table 2 with DIF).The covariates (health status, age and gender) were introduced into the model in order to explain the values of the latent variable. The variable “age” gave a significant explanation of the latent variable: the latent variable decreased with age (-0.40±0.08-p<0.001 for each increase of the age of 10 years). This decrease represented an effect size of 0.27 which can be qualified of a small to medium effect. Nevertheless, this effect explained only 2% of the variance of the latent trait, due to the heterogeneity of the latent variable between all of the individuals of the sample. The health status of individuals (GP versus PWS) and gender did not significantly explain differences in the values of the latent variable (respectively p=0.41 and p=0.49).

Discussion

The aim of the study was to explore DIF of the GOHAI items according to demographic characteristics (gender, age) and mental health status (schizophrenic disorders versus general population) This research have a potential theoretical importance in increasing researchers’ understanding of the interplay between this parameters.

First we focused on one important prerequisite for such comparisons, measurement invariance.

The GOHAI showed initial problems with the basic requirements of the PCM. When the collapsing procedure was applied (producing a new three-level rating scale: 0 = never; 1 = sometimes; 2 = often/always) problems were solved and the measurement quality improved. After this adjustment, the unidimensionality was supported and the difficulty setting decreased to two per item instead of four.

In accordance with our results, a previous diagnosis of the GOHAI rating scale made by Franchignoni et al. showed that two (1 = seldom; 3 = often) of the five rating categories did not comply with the criteria for category functioning.5 When they collapsed them, using the same procedure as we did, the probability of selecting one of the three revised rating categories became a clear function of the level of ability shown by the subject. The Rasch analysis demonstrated the substantial unidimensionality of the GOHAI and pointed out the opportunity to decrease the number of response categories to three rating levels instead of the original five.

Participants were possibly unable to clearly distinguish between the categories ‘Always’ and ‘Very often’ and the categories ‘Seldom’ and ‘Never’. Another explanation could be that the evaluated symptom/dysfunction was always present or did not exist (never). The response categories “very often” and “rarely” would thus not be relevant if the symptom were constantly present or absent. For the non-constant situations, the central modality could thus be sufficient to describe the frequency of the event.

Some of the GOHAI items performed differently depending on the subgroup. Poor oral health may have a serious impact on quality of life, on everyday functioning, self-esteem and social inclusion but the perception of OHrQOL between PWS and GP seems to be different. OHRQoL is a psychological concept whereas symptoms are objective physical aspects. It is the impact of oral symptoms, rather than symptom itself, that is important.

The first explanation could be a misunderstanding of the positively worded questions of the GOHAI (n°3,5) and a particular difficulty for people to grasp the underlying concept of question 7 [6]. This explanation is in accordance with the results of previous studies on the cultural adaption of the English GOHAI. In the Chinese version of the GOHAI, question 3 was reworded negatively to improve understanding [28]. For the French validation of the GOHAI with PWS the lowest item-scale correlation coefficients were obtained for items 3, 5 and 7 [6].

The second hypothesis is that participants and particularly PWS may have found it difficult to perceive the concept of OHrQOL particularly for those questions. There are psychological and cultural aspects in the evaluation of OHrQOL that determine the way objective symptoms are interpreted as giving rise to impacts on quality of life [15]. For example, in PWS, the authors observed some paradoxical situations where patients with severe dental diseases reported good oral health [29]. In PWS, the perception of oral health strongly depends on the extent of the dental disease [13, 14] and on the intensity of the symptoms [10]. The extent of the dental disease among inpatients is also directly related to the intensity of the schizophrenia, the magnitude of negative symptoms associated with schizophrenia, and to the length of hospitalization [30, 31]. Thus, PWS frequently do not identify or express their health needs, the side effects of treatments or their pain [10, 11]. In persons with disabilities, it has also been shown that differences in HRQOL not only reflect health status differences but also functional differences related to disability [29]. Furthermore, the influence of functional differences on HRQOL scores may vary across different disability subgroups [30, 32]. Thus, the evaluation of OHrQOL may be potentially complex in people with disability or in PWS. This is emphasized for questions relating to satisfaction with appearance because psychological status determines vulnerability to low self-esteem and thus self-evaluation of oral health [33].

Although items with differential item function across mental status were identified, its impact on the overall score can’t be estimated. Further investigation should be conducted for a better understanding of the oral problems linked with the illness.

Concerning items 3,4 (ability to swallow and speak) and 10 (feel nervous or self-conscious) are less with older, this is accordance with the fact that the latent variable significantly decreased with age (-0.40±0.08-p<0.001) between the youngest individuals (18-25 years) and the oldest individuals. As in the study of Franchignoni et al, items evaluating important oral functions were items showing high difficulties [5]. In fact, the elderly more commonly experience difficulty in swallowing or speaking due to oral impairment. Communication difficulties strongly affect general health, social life and wellbeing and swallowing difficulties may threaten life of the most vulnerable ones [1, 14]. The sequelae of oral diseases, such as tooth loss, are not reversible and thus accumulate over time even if prosthetic treatments are provided. The impact of oral disorders on quality of life thus increases with age for everyone, including institutionalized psychiatric patients, who are generally older patients [1, 30, 31].

In a study conducted in older adults, high GOHAI scores were found to show no significant influence of age on the frequency and severity of the impact of oral diseases [32]. An explanation is that older people adapt to poor health and consider their impaired dental status as normal. In this situation, the objective dental status is disconnected from the subjective perception of oral health and quality of life.

The gender-related DIF identified for item 5 showed that women were more likely than men to report feeling discomfort for eating. Yau et al. showed a minimal impact of DIF across gender in Children with the CPQ 11-14 questionnaire [19]. Lin et al., in the evaluation of the Persian Pediatric Quality of Life Inventory Oral Health Scale no DIF items were found across gender [34]. Although one item across gender were detected, they were not possible to precise the impact of DIF in practice. Our results confirm the previous studies and the low impact of gender in DIFF in HRQOL scale.

Limitations

Our study was limited to a sample of French participants. To extend the generalizability of our results, we encourage scholars in this area to examine our proposed model with different samples across different countries.

Second, our samples are heteregenous. GP sample is more younger than PWS sample. These differences may explain why men GOHAI scores were higer in PWS population. It is also possible that some of the differnces were due to DIF related to socioeconomic status.

Conclusions

This study revealed the presence of DIF in five of the 12 GOHAI items based on the analysis of data from two French populations; PWS and a general adult population. We showed the GOHAI scores were not be comparable across sub-groups defined by health status, age and gender without accounting for DIF. In our study, the effect of DIF “age” explained only 2% of the variance of the latent trait, but the decreased with age of latent variable represented an effect size not negligible and can influence mean GOHAI score comparisons. DIF analysis with gender and health status indicated that differences at the item and scale level exist but we can’t shows differences in the values of the latent variable. In the future, other studies should explore this way with other OHRQOL assessment tools and populations with mental illness.