Validation of the Mental Health Continuum-Short Form and the dual continua model of well-being and psychopathology in an adult mental health setting.

OBJECTIVE: The growing evidence for the dual continua model of psychopathology and well-being has important implications for measuring outcomes in mental health care. The aim of the current study is to validate a measure of well-being as well as the dual continua model in adults with mood, anxiety, personality, and developmental disorders.
METHODS: 472 adult psychiatric outpatients filled out the Mental Health Continuum-Short Form (MHC-SF) and the Outcome Questionnaire before start of treatment.
RESULTS: Confirmatory factor analyses (CFA) confirmed the three-factor structure of emotional, psychological, and social well-being of the MHC-SF. The dual continua model had the best fit in the complete sample and the different diagnostic groups.
CONCLUSION: The MHC-SF is a reliable and valid instrument to measure well-being in the psychiatric population. Although relatively high correlations between psychopathology and well-being exist, the results underline the importance to measure well-being in addition to psychopathology in mental health care.

INTRODUCTION

In recent years, there is a growing interest in well‐being, both in society in general as well as in mental health care. This is also reflected in the World Health Organization (WHO) definition of mental health: “mental health is a state of well‐being in which the individual realizes his or her abilities, can cope with the normal stresses of life, can work productively and fruitfully, and is able to make a contribution to his or her community” (World Health Organization, 2005). There is a growing number of studies that demonstrate that patients appreciate well‐being as an important outcome of treatment (e.g., De Vos et al., 2017; Macaskill, 2012; Zimmerman et al., 2006) and several meta‐analyses have shown that it is possible to promote well‐being in clinical populations (Bolier et al., 2013; Sin & Lyubomirsky, 2009; Weiss, Westerhof, & Bohlmeijer, 2016). The current study is one of the first to address a comprehensive model of well‐being and its relation to psychopathology in a mental health setting.

In line with the WHO definition, Keyes (2005, 2007) defined well‐being as an important dimension of mental health. He distinguished between emotional, psychological and social well‐being. Emotional well‐being fits with a hedonic perspective on happiness. The main goal of hedonism is to maximize pleasure and minimize pain in life (Gallagher, Lopez, & Preacher, 2009; Kahneman, Diener, & Schwarz, 1999; Waterman, 1993). Emotional well‐being comprises the presence of positive affect and life‐satisfaction and the absence of negative affect (Diener, 1984; Diener, Suh, Lucas, & Smith, 1999). Psychological and social well‐being fit with the eudaimonic tradition, where the focus is on positive functioning. Psychological well‐being comprises six facets of optimal individual functioning and self‐realization: self‐acceptance, positive relations with others, autonomy, environmental mastery, personal growth, and purpose in life (Ryff, 1989; Ryff & Keyes, 1995; Ryff & Singer, 2008). Social well‐being reflects optimal functioning in social groups and society (Huppert & Whittington, 2003; Ryff, 1989). Keyes (1998) distinguished five facets of social well‐being: social acceptance, social integration, social contribution, social coherence, and social actualization.

As there were no other instruments measuring all three dimensions of well‐being, Keyes (2002) developed the Mental Health Continuum‐Short Form (MHC‐SF) as a short instrument to measure the three facets of emotional well‐being and the 11 facets of psychological and social well‐being. Its three‐factor structure, reliability, and construct validity have been established in nonclinical samples across various countries and cultures, including the United States (Robitscheck & Keyes, 2009), South Africa (Keyes et al., 2008), the Netherlands (Lamers, Westerhof, Bohlmeijer, Ten Klooster, & Keyes, 2011), France (Salama‐Younes & Ismaïl, 2011), Korea (Lim, 2014), Poland (Karaś, Cieciuch, & Keyes, 2014), Italy (Petrillo, Capone, Caso, & Keyes, 2015), Portugal (De Carvalho, Pereira, & Pinto, 2016), Iran (Joshanloo, 2016), and Argentina (Perugini, De la Iglesia, Solano, & Keyes, 2017). Recently, in Australian (Hides et al., 2016) and South African samples (de Bruin & Du Plessis, 2015), the presence of a general factor of well‐being was suggested by mentioning evidence for a bifactor model.

An important question is how well‐being is related to psychopathology. Tudor (1996) introduced the dual continua model of well‐being and psychopathology which was further developed by Keyes (2002; 2005). It states that well‐being and psychopathology are two related, but distinct dimensions of mental health. One way to assess the dual continua model is by using confirmatory factor analysis (CFA) to examine the distinctiveness of well‐being and psychopathology. A model with two related factors was found to describe the data best in different population studies, such as American adolescents between 12 and 18 year of age (Keyes, 2006), American university students (Keyes, Eisenberg, Dhingra, Perry, & Dube, 2012; Renshaw & Cohen, 2014), American adults (Keyes, 2005; 2007), South‐African adults (Keyes et al., 2008), Dutch adults (Lamers et al., 2011), Canadian adults (Gilmour, 2014), Italian adults (Petrillo et al., 2015), and Argentinian adults (Perugini et al., 2017).

Another way of assessing the dual continua model is by studying the relations between well‐being and psychopathology over time. A growing number of studies provide evidence for the predictive value of well‐being on psychopathology, even when controlling for psychopathology earlier in time. Well‐being reduces the risk for future incidence of psychopathology (Keyes, Dhingra, & Simoes, 2010; Lamers, Westerhof, Glas, & Bohlmeijer, 2015; Layous, Chancellor, & Lyubomirsky, 2014; Schotanus‐Dijkstra, Ten Have, Lamers, de Graaf, & Bohlmeijer, 2016; Trompetter, de Kleine, & Bohlmeijer, 2016; Trompetter, Lamers, Westerhof, Fledderus, & Bohlmeijer, 2017). People with low psychological well‐being are seven times more likely to be depressed ten years later (Mood & Joseph, 2010).

A number of studies have addressed several aspects of well‐being in people with mental health problems. These studies show that there is a relation between psychopathology and well‐being in the sense that people with mental health problems experience lower levels of well‐being than people without complaints. This has been found in adult patients with mood problems (Beekman, Deeg, Braam, Smit, & Van Tilborg, 1997; Keyes, 2005; Nierenberg et al., 2010; Seow et al., 2016; Wells & Sherbourne, 1999; Westerhof & Keyes, 2010), with anxiety complaints (Candilis et al., 1999; Mendlowicz & Stein, 2000; Stein & Kean, 2000; Stein & Heimberg, 2004; Keyes, 2005; Westerhof & Keyes, 2010), with a borderline personality disorder (e.g., Conklin, Bradley, & Westen, 2006; IsHak et al., 2013) and various other personality disorders (e.g., Cramer, Torgersen, & Kringlen, 2006; Eisen et al., 2006) as well as with developmental disorders, both for persons with autism spectrum disorders (Mazurek, 2014) and for individuals with attention‐deficit/hyperactive disorders (Adler et al., 2006). Despite these lower levels of well‐being, these studies also show that well‐being and psychopathology are relatively independent as there is a large variation in well‐being among people with psychopathological impairments. These studies suggest there are indications for the validity of the dual continua model in clinical populations. However, a more formal validation of the dual continua model, using the comprehensive model of well‐being, is still lacking in the clinical population.

Well‐being seems to be an important dimension of mental health and recovery, and the dual continua model implies that the absence of psychopathology does not guarantee the presence of well‐being (Bolier et al., 2013; Slade, 2010; Trompetter et al., 2016; Trompetter et al., 2017; Trompetter, Bohlmeijer, Veehof, & Schreurs, 2015). With this in mind, the aim of the current study is twofold.

The first objective is to validate a comprehensive instrument for measuring well‐being, the MHC‐SF in people with mood, anxiety, personality, and developmental disorders. We hypothesize that the three‐factor structure of emotional, psychological, and social well‐being in the MHC‐SF will be confirmed for the Dutch clinical population. After all, when the three‐factor structure of the MHC‐SF can be confirmed in clinical groups, not only well‐being in general but especially the three distinct aspects of emotional, psychological, and social well‐being may be useful aspects to measure to predict outcomes of treatment in the mental health care.

Because a study in the general population showed in an item response analysis that items of the MHC‐SF functioned similarly in people with and without psychopathological symptoms (Lamers, Glas, Westerhof, & Bohlmeijer, 2012), we hypothesize that the MHC‐SF will also be a reliable instrument in the clinical population.

The second objective is to study to what extent the dual continua model is applicable to people with psychopathological symptoms. We hypothesize that the dual continua model of well‐being and psychopathology is also a valid model in the clinical population. To reinforce this hypothesis, we hypothesize that the dual continua model will be confirmed in each of the four common psychopathological subgroups: mood disorder, anxiety disorder, personality disorder, and developmental disorder. After all, there is a wide variety of diagnostic patient groups in mental health care, and they may differ in mean levels and variances of well‐being (Cramer et al., 2006; Jones, Yates, Williams, Zhou, & Hardman, 1999; Rapaport, Clary, Fayyad, & Endicott, 2005; Soeteman, Verheul, & Busschbach, 2008; Wells & Sherbourne, 1999). This could mean that the dual continua model is not equally pronounced for all patient groups. Although we expect the dual continua model to be found in the clinical population, we therefore also investigated the four main patient groups to clarify. To know that we can apply the dual continua model in all four groups, the application of the model in the mental health care is even more important.

METHODS

Participants

In total, 472 respondents of an outpatient population were included in the present study. Most respondents were female, moderately educated, and lived with a partner and children or were single (see Table 1). One third had meaningful daily activities (school or (voluntary) work), a third was at home because of illness and a third was unemployed for other reasons. The respondents were classified into four common psychopathological groups on the base of their primary diagnoses: mood disorder (n = 168; 35.6%), anxiety disorder (n = 101; 21.4%), personality disorder (n = 122; 25.8%), and developmental disorder (n = 81; 17.2%).

Table 1

Major characteristics of respondents (N = 472)

	Mood disorder (n = 168) (35.6%)		Anxiety disorder (n = 101) (21.4%)		Personality disorder (n = 122) (25.8%)		Developmental disorder (n = 81) (17.2%)		Total (N = 472)
Gender n (%)
Male	72	(42.9)	34	(33.7)	31	(25.4)	54	(66.7)	191	(40.5)
Female	96	(57.1)	67	(66.3)	91	(74.6)	27	(33.3)	281	(59.5)
Age
Mean	44.4		39.3		36.8		36.3		40.0
Range	18–64		18‐63		19‐61		18‐61		18‐64
SD	11.2		11.2		10.6		11.6		11.6
Level of education n (%) a
Low	41	(24.4)	28	(27.7)	24	(19.7)	13	(16.0)	106	(22.5)
Moderate	59	(35.1)	25	(24.8)	45	(36.9)	44	(54.3)	173	(36.7)
High	12	(7.1)	15	(14.9)	20	(16.4)	9	(11.1)	56	(11.9)
Marital status n (%)
Single without children	42	(25)	27	(26.7)	47	(38.5)	25	(30.9)	141	(29.9)
Single with children	7	(4.2)	11	(10.9)	12	(9.8)	9	(11.1)	39	(8.3)
Married without children	39	(23.2)	14	(13.9)	28	(23.0)	10	(12.3)	91	(19.3)
Married with children	58	(34.5)	34	(33.7)	18	(14.8)	24	(29.6)	134	(28.4)
Daily activities n (%)
School	2	(1.2)	2	(2.0)	2	(1.6)	9	(11.1)	15	(3.1)
Volunteer work	5	(3.0)	3	(3.0)	5	(4.1)	3	(3.7)	16	(3.4)
Work	36	(21.4)	22	(21.8)	31	(25.4)	30	(37.0)	119	(25.2)
Sick at home	65	(38.7)	37	(36.6)	41	(33.6)	12	(14.8)	155	(32.8)
Unemployed	32	(19.0)	23	(22.8)	29	(23.8)	18	(22.2)	102	(21.6)

Low = primary school, lower vocational education; moderate = secondary school, intermediate vocational education; high = higher vocational education, university.

From the patients with the primary diagnosis in the category Personality Disorder, 61% had a secondary diagnosis on Axis I (39% mood disorder, 18% anxiety disorder, 4% developmental disorder). In patients with a primary diagnosis on Axis I, comorbidity with a personality disorder, labeled on Axis II, was as follows: in patients with a mood disorder 30%, with an anxiety disorder 29%, with a developmental disorder 5%.

Procedure

The present study used data collected in the context of Routine Outcome Monitoring (ROM), a standardized service to measure treatment effects. Patients in a mental health organization in the Netherlands completed questionnaires every 3 months from the initial interview to end of treatment. In the present study, only data completed before start of treatment were included. These data were gathered between March and August 2015. Patients agreed that their anonymized data may be used for research by passive consent. Inclusion criteria were an age between 18 and 65 years of age; full completion of the questionnaires on the same day; and a primary psychopathological classification in one of the four common psychopathological groups (mood disorder, anxiety disorder, personality disorder, and developmental disorder). A licensed psychiatrist or clinical psychologist based DSM‐IV classifications on an extensive interview, added this information to the electronic patient file, and suggested a primary diagnosis and corresponding specific care program. The suggested primary diagnosis and related treatment option were discussed and confirmed in multidisciplinary teams. Based on the primary diagnosis and therewith care allocations, the patients in our study were classified into the four subgroups. Initially, 654 respondents completely filled out the questionnaires. After applying the inclusion criteria, 472 respondents were included (72%). 116 respondents were excluded because they did not belong to one of the four psychopathological subgroups, 54 because they did not fill out the questionnaires on the same day, seven did not give permission to use their data, three did not fill in the questionnaires completely and two fell outside the age category of 18–65 years. Scores on the measures of psychopathology and well‐being did not significantly differ between included and excluded participants (t‐values with p > 0.30). More male participants were in the excluded (53.5%) versus the included group (40.5%; Pearson Chi‐square = 8.68, p = 0.003). In the included group men did not differ from women in levels of well‐being (t‐values with p > 0.30), but had lower levels of symptomatic distress (SD) (50.14 ± 17.02 vs. 54.79 ± 16.46; p = 0.00) and psychopathology (81.59 ± 26.39 vs. 86.60 ± 26.04; p = 0.04).

Measurements

Well‐being

Well‐being was measured using the Dutch version of the self‐reported MHC‐SF (Keyes et al., 2008; Lamers et al., 2011). The MHC‐SF comprises 14 items, which represent emotional well‐being (three items), psychological well‐being (six items), and social well‐being (five items). Respondents indicate the frequency of experienced feelings in the last month on a 6‐point scale ranging from 0 (never) to 5 (every day). They answered questions like “During the past month, how often did you feel: happy (emotional well‐being); confident to think or express your own ideas and opinions (psychological well‐being); that you had something important to contribute to society (social well‐being). The MHC‐SF has good psychometric qualities in the general population (De Carvalho et al., 2016; Hides et al., 2016; Joshanloo, 2016; Joshanloo, Wissing, Khumalo, & Lamers, 2013; Karaś et al., 2014; Keyes et al., 2008; Lamers et al., 2011; Lim, 2014; Perugini et al., 2017; Petrillo et al., 2015; Salama‐Younes & Ismaïl, 2011). The psychometric properties of the MHC‐SF in the psychiatric population will be examined in the present study.

Psychopathology

The Outcome Questionnaire (OQ‐45; De Jong et al., 2007) is a commonly used adult self‐report measure of psychopathology. The OQ‐45 is designed for repeated measurement of patient progress during and after therapy. It comprises 45 items which are answered on a five‐point‐rating scale from 0 (never) to 4 (almost always). It is composed of three subscales: SD, interpersonal relations (IR), and social roles (SR) performance. The SD scale (25 items, e.g., “I'm anxious”) assesses subjective complaints or intrapsychic functioning. The IR scale (11 items, e.g., “Often I have fights”) assesses the patients functioning in relation to partner, family and friends. The SR scale (nine items, e.g., “I feel like I'm not doing well with my work/schoolwork”) assesses whether one is able to fulfill societal obligations like functioning in school, work, and leisure. The instrument has demonstrated good validity and reliability across different countries (De Jong & Nugter, 2004; De Jong et al., 2007; De Jong et al., 2008). In the present study, the internal consistency for the total scale of the OQ‐45 was high (α = 0.94), and for the subscales high to marginal: SD α = 0.93; IR: α = 0.81; SR: α = 0.67.

Statistical analysis

For the statistical analyses of the data, the programs IBM Statistical Packages for the Social Sciences (SPSS) version 23.0 and LISREL 8.80 were used. To evaluate the psychometric properties of the MHC‐SF, the factor structure and internal consistency were analyzed. CFAs using robust maximum likelihood estimation were applied to evaluate whether the three‐factor structure of the MHC‐SF items could be confirmed in a clinical population. We tested and compared the fit of three possible models of the structure of the MHC‐SF: (1) a model with one factor, which represents well‐being, (2) a model with two related factors, representing the hedonic and the eudaimonic well‐being, and (3) the hypothesized model with three related factors of emotional, psychological, and social well‐being.

After testing the one factor model of well‐being, the two factor model is evaluated. The rationale for fitting this model was found in research exploring the extent to which eudaimonia and hedonia differ (Baumeister, Vohs, Aaker, & Garbinsky, 2013; Disabato, Goodman, Kashdan, Short, & Jarden, 2016; Huta & Ryan, 2010; Keyes, Shmotkin, & Ryff, 2002; Waterman, 1993) and research targeting eudaimonic treatment outcomes (Brandel, Vescovelli, & Ruini, 2017; Weiss et al., 2016).

Cronbach's alpha was computed as an indicator of internal consistency. Guidelines by Kline (2000) were applied: values below 0.70 were interpreted as low and unacceptable, above 0.70 as acceptable, above 0.80 as high and above 0.90 as very high.

To test if the dual continua model is applicable in the clinical population, we conducted the following analyses in both the total clinical population and separately in the four included psychopathological groups (mood disorder, anxiety disorder, personality disorder, and developmental disorder). First, Pearson correlations were computed between the total and subscale scores of the MHC‐SF and OQ‐45. Based on Hinkle, Wiersma, and Jurs (2003) correlations between 0.30 and 0.50 were considered low, between 0.50 and 0.70 moderate, between 0.70 and 0.90 high and above 0.90 very high. Differences in correlations between psychopathological groups were tested using Fischer's Z‐tests for independent samples. Second, CFAs using maximum likelihood estimation were applied to test and compare two models: (1) a model with one factor, where the subscale scores of psychopathology and well‐being are seen as two extremes on a continuum, (2) the dual continua model where psychopathology subscales and well‐being subscales reflect two related factors. To test the validation of the dual continua model, the distinction between overall psychopathology and overall well‐being was expressed.

In all CFAs, we used several fit indices to assess the fit of the models: Satorra‐Bentler scaled chi‐square (SBχ 2; Satorra & Bentler, 2001) to evaluate the three‐factor structure of the MHC‐SF items and the normal‐theory chi‐square to evaluate the dual continua model; and further: noncentrality parameter (NCP), Akaike's information criterion (AIC), root mean square error of approximation (RMSEA), comparative fit index (CFI), standardized root mean square residual (SRMR), goodness of fit index (GFI), and adjusted goodness of fit index (AGFI). The following values are considered indicative of good or, as noticed in parentheses, acceptable model fit: RMSEA ≤ 0.05 (≤0.08); SRMR ≤ 0.05 (≤0.10); CFI ≥ 0.97 (≥0.95); GFI ≥ 0.95 (≥0.90); AGFI ≥ 0.90 (≥0.85) (Schermelleh‐Engel, Moosbrugger, & Müller, 2003). When the χ 2, the NCP and the AIC are smaller than its equivalent parameter in the comparison model, it stands for a better fit (Schermelleh‐Engel et al., 2003). The Satorra‐Bentler scaled difference chi‐square test (ΔSBχ2) or normal chi‐square difference test was used to statistically compare the relative fit of nested models.

In the analyses, we applied p values of <0.05 as the minimum level of statistical significance.

RESULTS

Psychometric properties of the MHC‐SF

CFA was used to test the latent structure of the MHC‐SF items in the total clinical population. Given the ordinal nature of the items and the observed non‐normality of several item distributions (Skewness range −0.08 to 1.55; Kurtosis range −1.42 to 1.76), robust maximum likelihood estimation was used with standard errors and a chi‐square test statistic that are robust to non‐normality. In the CFAs, all three models showed an acceptable to good fit in the total sample, but the three‐factor model had the best fit to the data (see Table 2). When comparing the three models to each other, the SBχ 2, NCP and AIC showed the relative best fit for the three‐factor model. The CFI, SRMR, GFI, and AGFI all showed a good fit for the three‐factor model. Only the RMSEA indicated an acceptable fit. The AIC and NCP were smaller for the three‐factor model and the ΔSBχ 2 indicated a significantly better fit for the three‐factor model over the two‐factor model.

Table 2

Robust maximum likelihood estimation of CFA models of the latent structure of the MHC‐SF items

	Model 1	Model 2	Model 3
	Single factor	Two factors	Three factors
Fit indices	Well‐being	Hedonic and eudaimonic well‐being	Emotional, psychological and social well‐being
SB χ 2	281.2	207.7	176.6
df	77	76	74
NCP	204.2	131.7	102.6
AIC	337.2	265.7	238.6
RMSEA	0.075	0.061	0.054
CFI	0.978	0.986	0.989
SRMR	0.056	0.051	0.046
GFI/AGFI	0.86/0.81	0.89/0.85	0.91/0.87
	Model 1 vs. 2	Model 2 vs. 3
Scaled Δχ 2	66.62***	35.31***

Note. CFA, confirmatory factor analysis; MHC‐SF, Mental Health Continuum‐Short Form; SB χ 2, Satorra‐Bentler scaled chi‐square (smaller values indicate better fit); df, degrees of freedom; NCP, estimated noncentrality parameter (smaller values indicate better fit); AIC, Akaike's information criterion (smaller values indicate better fit); RMSEA, root mean square error of approximation (good fit ≤ 0.05; acceptable fit ≤ 0.08); CFI, comparative fit index (good fit ≥ 0.97; acceptable fit ≥ 0.95); SRMR, standardized root mean square residual (good fit ≤ 0.05; acceptable fit ≤ 0.10); GFI, goodness of fit index (good fit ≥ 0.95; acceptable fit ≥ 0.90); AGFI, adjusted goodness of fit index (good fit ≥ 0.90; acceptable fit ≥ 0.85).

***p < 0.001, two‐tailed.

The standardized solution of the three‐factor model showed that all MHC‐SF items had a high loading on their intended factor of emotional, psychological, or social well‐being with a minimum of 0.52 (item 8) and a maximum of 0.87 (item 3). There was a high correlation between the three latent variables, in particular between emotional and psychological well‐being (r = 0.91), followed by social and psychological well‐being (r = 0.87), and emotional and social well‐being (r = 0.76). Nevertheless, the three‐factor model still had the best fit to the data. The internal consistency for the total scale of the MHC‐SF was high (α = 0.92), as well as for the subscales emotional well‐being (α = 0.88) and psychological well‐being (α = 0.85). For social well‐being, the internal consistency was acceptable (α = 0.77). Hence, the hypotheses on the factor structure and reliability of the MHC‐SF are confirmed.

Dual continua model in the clinical population

The correlations as well as the mean levels and standard deviations of the MHC‐SF and the OQ‐45 and their subscales are presented in Table 3. In the complete sample, there was a high negative correlation between overall well‐being and psychopathology. Moderate negative correlation coefficients were found for the subscales of the MHC‐SF and OQ‐45 (r = −0.53 to −0.69). Well‐being only had a relatively low negative correlation with social roles dysfunction (r = −0.34 to −0.44).

Table 3

Comparison of standard scores and Pearson correlations between subscales and total scores on the MHC‐SF and the OQ‐45 for the complete sample and subgroups of psychopathology

	Well‐being
Psychopathology	Emotional well‐being	Psychological well‐being	Social well‐being	Total well‐being	M OQ‐45	SD OQ‐45
Complete sample (N = 472)
M MHC‐SF	2.1	2.0	1.6	1.9
SD MHC‐SF	1.3	1.2	1.1	1.0
Symptomatic distress	−0.66**	−0.65**	−0.56**	−0.69**	52.9	16.8
Interpersonal relations	−0.61**	−0.63**	−0.53**	−0.67**	17.8	7.3
Social roles	−0.42**	−0.43**	−0.34**	−0.44**	13.8	5.6
Total psychopathology	−0.68**	−0.69**	−0.58**	−0.72**	84.6	26.3
Mood disorders (n = 168)
M MHC‐SF	1.9a	1.9	1.5	1.7
SD MHC‐SF	1.4	1.2	1.1	1.1
Symptomatic distress	−0.72**	−0.73**	−0.67**	−0.77**	55.2	17.9
Interpersonal relations	−0.64**	−0.68**	−0.62**	−0.70**	19.0	7.2
Social roles	−0.44**	−0.52**	−0.47**	−0.52**	13.9	5.9
Total psychopathology	−0.73**	−0.76**	−0.69**	−0.79**	88.1	27.7
Anxiety disorders (n = 101)
M MHC‐SF	2.2	2.1	1.6	1.9
SD MHC‐SF	1.4	1.2	1.1	1.1
Symptomatic distress	−0.67**	−0.67**	−0.59**	−0.73**	55.3	16.7
Interpersonal relations	−0.67*	−0.68**	−0.53**	−0.71**	16.3	7.8
Social roles	−0.41**	−0.41**	−0.32**	−0.43**	13.4	6.0
Total psychopathology	−0.69**	−0.70**	−0.59**	−0.74**	85.0	27.1
Personality disorders (n = 122)
M MHC‐SF	2.2	2.1	1.6	1.9
SD MHC‐SF	1.2	1.1	1.0	1.0
Symptomatic distress	−0.56**	−0.52**	−0.47**	−0.57**	52.2	15.8
Interpersonal relations	−0.55**	−0.55**	−0.49**	−0.58**	18.3	7.1
Social roles	−0.36**	−0.31**	−0.27**	−0.34**	14.0	5.1
Total psychopathology	−0.60**	−0.56**	−0.51**	−0.61**	84.4	24.2
Developmental disorders (n = 81)
M MHC‐SF	2.5a	2.2	1.6	2.1
SD MHC‐SF	1.0	1.0	1.0	0.9
Symptomatic distress	−0.60**	−0.60**	−0.37**	−0.60**	46.3	14.6
Interpersonal relations	−0.53**	−0.56**	−0.41**	−0.59**	16.7	6.8
Social roles	−0.51**	−0.43**	−0.20	−0.43**	14.0	5.7
Total psychopathology	−0.64**	−0.63**	−0.39**	−0.64**	77.0	23.7

Note. SD, symptom distress; IR, interpersonal relations; SR, social role acceptance.

M, mean level of well‐being or psychopathology; SD, standard deviation.

**p < 0.01, two‐tailed

With regard to the four common psychopathological groups, high correlations were found between overall well‐being and psychopathology in patients with mood disorders (r = −0.79) and anxiety disorders (r = −0.74), whereas the correlations were moderate in patients with personality disorders (r = −0.61) and developmental disorders (r = −0.64). Only the correlation for mood disorders was significantly stronger than that of personality disorders (Z = −3.01; p = 0.0026) and developmental disorders (Z = −2.28; p = 0.0226). Relations between the subscales also tended to be highest for mood disorders and lowest for personality and developmental disorders. In the latter two groups, especially the correlation between social well‐being and social role was very low.

CFAs were applied to test the hypothesis of well‐being and psychopathology as two related but distinctive latent factors. Because of the continuous nature of the subscales scores of the MHC‐SF and the OQ‐45 and the finding that they were essentially normally distributed (Skewness range −0.20 to 0.86; Kurtosis range −1.16 to 1.16), “normal” maximum likelihood estimation was used.

The descriptive indices of the model with a single factor of well‐being and psychopathology revealed a poor fit to the data (see Table 4). There was a stronger fit for the dual continua model, which confirms our hypothesis that well‐being and psychopathology are distinct dimensions instead of the ends of one bipolar dimension. The normal theory weighted least squares χ 2, the NCP and AIC were much lower in the dual continua model in comparison with the single factor model, thus also favoring the dual continua model. The RMSEA showed an acceptable fit. The CFI, SRMR, GFI, and the AGFI all showed a good fit for the dual continua model to the data. Furthermore, chi‐square difference test statistic improved significantly for the model with two correlated factors over the model with one factor.

Table 4

Maximum likelihood estimation of CFA models of the latent structure of well‐being (MHC‐SF subscales) and psychopathology (OQ‐45 subscales)

	Model 1	Model 2
Fit indices	Single factor	Two related factors
χ 2	152.9	24.4
df	9	8
NCP	143.9	16.4
AIC	176.9	50.4
RMSEA	0.184	0.066
CFI	0.952	0.993
SRMR	0.054	0.025
GFI/AGFI	0.90/0.77	0.98/0.96
	Model 1 vs. 2
Δχ 2	128.51***

Note. CFA, confirmatory factor analysis; MHC‐SF, Mental Health Continuum‐Short Form; OQ‐45, Outcome Questionnaire 45; χ 2, normal theory weighted least squares Chi‐square (smaller values indicate better fit); df, degree of freedom; NCP, estimated noncentrality parameter (smaller values indicate better fit); AIC, Akaike's information criterion (smaller values indicate better fit); RMSEA, root mean square error of approximation (good fit ≤ 0.05; acceptable fit ≤ 0.08); CFI, comparative fit index; SRMR, standardized root mean square residual (good fit ≤ 0.05; acceptable fit ≤ 0.10); GFI, goodness of fit index (good fit ≥ 0.95; acceptable fit ≥ 0.90); AGFI, adjusted goodness of fit index (good fit ≥ 0.90; acceptable fit ≥ 0.85).

***p < 0.001, two‐tailed.

The standardized solution of the dual continua model showed that the three subscales of the MHC‐SF had a high loading on their intended factor of well‐being ranging from 0.78 to 0.92. The three subscales of the OQ‐45 had a high loading on the factor of psychopathology, with a range between 0.63 and 0.87. The latent correlation between well‐being and psychopathology was −0.84. Despite the high correlation, CFAs confirmed the hypothesis that well‐being and psychopathology are two related dimensions.

Next, we applied identical CFAs to examine the dual continua model in the four common psychopathological subgroups separately. Table 5 shows that in all groups the two related factors model gave a better fit for the data than a single factor model. These results were most prominent in the psychopathological groups of personality disorders and mood disorders. Chi‐square difference test statistics improved significantly for the model with two related factors in all subgroups as compared with the single‐factor model. In all subgroups, there was a strong and negative correlation between the latent factors of mental health and psychopathology, ranging from −0.72 (personality disorder), −0.81 (developmental disorder), −0.85 (anxiety disorder) to −0.88 (mood disorder).

Table 5

Maximum likelihood estimation of CFA models of the latent structure of the well‐being (MHC‐SF subscales) and psychopathology (OQ‐45 subscales) in psychopathological subgroups

	Mood disorder		Anxiety disorder		Personality disorder		Developmental disorder
	(n = 168)		(n = 101)		(n = 122)		(n = 81)
	Model 1	Model 2	Model 1	Model 2	Model 1	Model 2	Model 1	Model 2
Fit indices	Single factor	Two related factors	Single factor	Two related factors	Single factor	Two related factors	Single factor	Two related factors
χ 2	52.8	7.6	39.1	16.1	57.7	10.5	29.1	13.4
df	9	8	9	8	9	8	9	8
NCP	43.8	0.0	30.1	0.3	48.7	2.5	20.1	5.4
AIC	76.8	33.6	63.1	42.1	81.7	36.5	53.1	39.4
RMSEA	0.171	0.000	0.183	0.101	0.211	0.051	0.167	0.092
CFI	0.966	1.000	0.945	0.983	0.919	0.996	0.948	0.986
Standardized RMR	0.047	0.018	0.053	0.039	0.083	0.036	0.067	0.053
GFI/AGFI	0.91/0.78	0.99/0.96	0.89/0.73	0.95/0.87	0.86/0.68	0.97/0.93	0.89/0.75	0.95/0.86
	Model 1 vs. 2		Model 1 vs. 2		Model 1 vs. 2		Model 1 vs. 2
Δχ 2	45.28***		23.03***		46.97***		15.64***

Note. CFA, confirmatory factor analysis; MHC‐SF, Mental Health Continuum‐Short Form; OQ‐45, Outcome Questionnaire 45; χ 2, normal theory weighted least squares Chi‐square (smaller values indicate better fit); df, degree of freedom; NCP, estimated noncentrality parameter (smaller values indicate better fit); AIC, Akaike's information criterion (smaller values indicate better fit); RMSEA, root mean square error of approximation (good fit ≤ 0.05; acceptable fit ≤ 0.08); CFI, comparative fit index (good fit ≥ 0.97; acceptable fit ≥ 0.95); standardized RMR, standardized root mean square residual (good fit ≤ 0.05; acceptable fit ≤ 0.10); GFI, goodness of fit index (good fit ≥ 0.95; acceptable fit ≥ 0.90); AGFI, adjusted goodness of fit index (good fit ≥ 0.90; acceptable fit ≥ 0.85).

***p < 0.001, two‐tailed.

DISCUSSION

Studies in the general population have shown that the MHC‐SF is a reliable and valid instrument to comprehensively measure emotional, psychological, and social well‐being (De Carvalho et al., 2016; Hides et al., 2016; Joshanloo, 2016; Joshanloo et al., 2013; Karaś et al., 2014; Keyes et al., 2008; Lamers et al., 2011; Lim, 2014; Perugini et al., 2017; Petrillo et al., 2015; Robitscheck & Keyes, 2009; Salama‐Younes, & Ismaïl, 2011). Furthermore, studies in the general population have provided evidence for the dual continua model of well‐being and psychopathology (Gilmour, 2014; Keyes, 2005; 2006; 2007; Keyes et al., 2008; Keyes et al., 2012; Lamers et al., 2011; Lamers, Westerhof, Bohlmeijer, & Keyes, 2013; Perugini et al., 2017; Petrillo et al., 2015; Renshaw & Cohen, 2014; Westerhof & Keyes, 2008, 2010). In accordance with findings in the general population, the present study confirmed the three‐dimensional structure and the reliability of the MHC‐SF in the clinical population.

Furthermore it provided first evidence for the dual continua model in the clinical population and in the four most common psychopathological groups of mood disorder, anxiety disorder, personality disorder, and developmental disorder. These findings mean that well‐being explicitly should be included in mental health care.

Although the three‐dimensional model of well‐being and the dual continua model of well‐being and psychopathology provided the best fit to the data, the correlations between the well‐being dimensions and the correlations between well‐being and psychopathology were generally high. The latter correlations were highest for patients with mood disorders, followed by anxiety disorders, personality disorders, and developmental disorders. All correlations were also higher in comparison with those found in a study of the Dutch relative healthy population (Lamers et al., 2011). Possible explanations for the stronger relations found in the current study can be found in both clinical reasoning and theoretical considerations.

From a clinical perspective, the high correlation might be related to the high levels of burden and distress that people who seek initial treatment for their mental health problems experience. As a result, when filling out self‐report questionnaires they may differentiated less between aspects of well‐being and between well‐being and psychopathology than people in the general population. For example, Zautra, Berkhof, & Nicolson (2010) found that the correlation between positive and negative affect is higher when people experience higher levels of stress. The fact that the correlation was strongest in patients with a mood disorder supports this thought.

From a theoretical perspective, the high correlation between well‐being and psychopathology can be the result of the way psychiatric disorders are defined. The DSM‐IV diagnosis not only includes symptoms of psychopathology, but also the amount of suffering and dysfunction in daily and social activities (American Psychiatric Association, 2000). This could have strengthened the relationship between psychopathology and well‐being in the clinical population. In future research, it would be helpful to better distinguish symptoms and functioning when studying the relationship between psychopathology and well‐being.

The relation between well‐being and psychopathology also might be stronger in this study because the OQ‐45 was used for the clinical population while the Brief Symptom Inventory (BSI) was used in the general population study of Lamers and colleagues (2011). While the OQ‐45 assesses SD as well as IR and SR, the BSI only assesses symptoms. However, the correlations between well‐being and SD were as high as or even higher than those with IR and SR, respectively.

Furthermore, the BSI only has negatively formulated items, while the OQ‐45 also has positively formulated items in each of the subscales. Hence, there might be less overlap between the MHC‐SF and the BSI compared with the OQ‐45, which can result in an artificially deflated correlation in the earlier studies. In further research, it might be interesting to repeat the current research with another instrument for psychopathology like the BSI to assess whether the stronger relation is a result of the specific research instrument used or if it has a more substantive reason.

There are some further limitations to the current study that need to be considered. The four psychopathological groups are not homogeneous, since different disorders are represented in each group, for example patients with a unipolar and bipolar depression within mood disorders, or autism an attention‐deficit/hyperactivity disorder within developmental disorders, and several patients had a comorbid diagnosis. Cramer and colleagues (2006) showed that comorbidity of psychiatric disorders and personality disorders is related to stronger impairments of well‐being. In further research it will be interesting to investigate the influence of comorbidity.

Patients may also vary because some have first‐time complaints and others more chronic psychopathological impairments. It is therefore recommended for future research to further differentiate and include more specific diagnoses, comorbidity, and illness history. It might be recommended also to assess the dual continua model with disorder specific instruments rather than with a general questionnaire of psychopathology. Last, the study had a cross‐sectional design, measuring well‐being and psychopathology and their intercorrelation only before the treatment was started. It remains to be explored whether various treatments affect well‐being and psychopathology in a similar way in all patients. A previous study indicated that whereas both well‐being and psychopathology improved during psychotherapeutic treatment, only a small proportion of patients improved on both well‐being and psychopathology (Trompetter et al., 2017). Therapies that focus on the promotion of well‐being might be especially relevant for patients who recovered in terms of psychopathology, but still have low levels of well‐being (Fava & Ruini, 2002; Rafanelli et al., 2000, 2002; Steger, Kahdan & Oishi, 2008; Tugade & Frederickson, 2004).

Finally, although in the general population the items of the MHC‐SF appeared to function equally in different demographic groups (Lamers et al., 2012), it is important to take the demographic and clinical characteristics of the psychopathology groups into account and determine if any of these characteristics impact fit. For example, some of the groups appear to have more male participants than other groups or more unemployed participants than other groups. Similarly, some of the groups have poorer numerical values for model fit than other groups. It is possible that some of the demographic and clinical characteristics are associated with quality of fit. Future studies using larger samples could further examine this issue by testing the measurement invariance of the MHC‐SF between relevant patient characteristics such as age, gender, and education.

Despite mentioned limitations, the three‐factor model of well‐being and the dual continua model of well‐being and psychopathology provided the best fit to data of patients with different psychopathological disorders, suggesting that the dual continua model is widely applicable in mental health care. However, the high correlation between emotional and psychological well‐being gives thought of the extent to which it makes practical sense to differentiate between these aspects of well‐being in the clinical population. The data of the current study are drawn from a larger study examining treatment for psychological disorders. Follow‐up studies could demonstrate the incremental validity of the separate constructs of emotional, psychological, and social well‐being in predicting treatment outcomes.