The wide-ranging life outcome correlates of a general psychopathology factor in adolescent psychopathology.

BACKGROUND: The structure of psychopathology has been much debated within the research literature. This study extends previous work by providing comparisons of the links between psychopathology and several life outcomes (temperamental, economic, social, psychological and health) using a three-correlated-factors model, a bifactor model, a revised-bifactor model and a higher-order model.
METHODS: Data from a sample of Dutch adolescents were used (n = 2 230), and psychopathology factors were modelled using self-reported and parent-reported longitudinal data from youth across four assessments during adolescence, from ages 11 to 19. Outcome variables were assessed at age 22 using adolescent-reports and parent-reports and more objective measures (e.g. body mass index).
RESULTS: While no measurement model was clearly superior, we found modest associations between the psychopathology factors and life outcomes. Importantly, after taking into account a general factor, the associations with life outcomes decreased for the residual parts of thought problems (across all domains) and internalizing problems (for temperamental and psychological outcomes), but not for externalizing problems, compared with the traditional three-correlated-factors model. Patterns were similar for adolescent-reported and parent-reported data.
CONCLUSIONS: Findings suggest that a general factor is related to psychopathology and life outcomes in a meaningful way. Results are discussed in terms of individual differences in propensity to psychopathology and more broadly in light of recent developments concerning the structure of psychopathology.

Introduction

The structure of psychopathology has been much debated in the research literature. Traditionally, researchers have posited that the structure of psychopathology is characterized by the dimensions of internalizing (INT) and externalizing (EXT) psychopathology (and sometimes a thought problem dimension (THO)1, 2; for the INT/EXT/THO approach, see3, 4). More recently, it has been suggested that psychopathology is better defined by either a bifactor model or a higher‐order model reflecting one or more higher‐order dimensions in addition to INT, EXT and THO and subsequent lower‐order symptoms.5, 6, 7, 8, 9, 10, 11 Over the last years, the bifactor model proposing a general psychopathology factor (GEN), in addition to INT, EXT and THO, has been robustly replicated. However, interpretation and use of the GEN factor models have been shown to be complicated.12, 13 The current study aims at contributing to unveiling the construct validity of the GEN factor providing a direct comparison of the links between psychopathology and life outcomes using a three‐correlated‐factors model (model A; INT/EXT/THO), a bifactor model (model B; INT/EXT/THO/GEN), a revised‐bifactor model (model ′B; INT/EXT/GEN; excluding the THO factor after the THO variance was found to be almost completely subsumed by GEN3, 4;) and a higher‐order model (model C).

Across the literature, definitions of the GEN factor vary. Often, the GEN factor—and higher‐order dimensions more broadly—is assumed to reflect some kind of vulnerability.5, 7, 14 Additionally, the GEN factor may reflect a proneness towards distress that is inherent across diagnostic categories, and as such, people who are high in this proneness may be more likely to meet criteria for any given disorder.15 Further, the GEN factor has been suggested to parallel compromised cognitive or impulse control,16, 17, 18 poor intellectual function,5, 19 disordered form and content of thoughts5 or cybernetic dysfunction, which is failure to make progress towards important goals because of failure of characteristic adaptations.20 These approaches posit that the GEN factor reflects the propensity for adverse (mental health) outcomes.

Risks of general factor modelling have been described repeatedly.21, 22 Specifically, the sometimes‐suggested better fit of a bifactor approach compared with traditional approaches may often be due to its greater complexity, rather than a better reflection of the true structure of psychopathology.23, 24, 25, 26 As a result, superior model fit statistics (i.e. Bayes information criterion and Akaike information criterion) may not provide robust evidence that the GEN factor model better reflects the true structure of psychopathology than do traditional approaches.

In sum, whereas the replicability of the GEN factor has now been well established, its conceptual meaning is debated and it is unclear what the substantive meaning is of the residual parts of INT and EXT after taking into account the GEN factor.27 One way of unveiling the usefulness and meaning of the GEN factor is to investigate the link between the GEN factor and stable individual differences.17, 27 Some studies indeed have found support for the GEN factor being related to personality and temperament traits.5, 9, 17, 19 For example, Tackett and colleagues suggested that the GEN factor is largely accounted for by neuroticism.9 In addition, the construct validity of the GEN factor has been studied by examining associations with school functioning and academic outcomes5, 19, 28 and brain functioning.5, 15 For example, Lahey and colleagues provided evidence of robust and independent associations with a range of teacher‐reported school functioning measures from childhood to adolescence.29 Finally, studies have suggested that the GEN factor is related to more adverse family and developmental history, compared with INT and EXT.5, 15, 29

Taken together, in addition to the robust support that the structure of psychopathology is well reflected by a model including a GEN factor, research has increasingly assessed the GEN factor in terms of criterion validity to examine its scientific and clinical utility. Findings can provide insight for the GEN factor being reflective of a propensity for all types of psychopathology but also being important for a variety of life domains and partially accounting for the traditional link between INT/EXT and life outcomes. Moreover, findings seem to suggest that the GEN factor subsumes most of the ‘severe pathology' of the traditional INT, EXT and THO factors. As such, when the GEN factor is added to the model, the remaining traditional factors may reflect some modest, non‐pathological tendencies for problem behaviours, which may be less important for predicting long‐term life outcomes. Moreover, in our previous work14 and in the study by Caspi and colleagues,5 the THO factor (often including severe pathological symptoms such as psychotic experiences) was completely subsumed by the GEN factor.

Whereas the studies mentioned previously provide support for links between the GEN factor and a range of psychological constructs, they generally do not contrast the three‐correlated‐factors model to GEN factor models in terms of their relations with life outcomes. Moreover, to illuminate the extent that (a) the GEN factor accounts for the links between INT/EXT/THO and life outcomes and (b) the remaining INT/EXT/THO factors augment the GEN factor in explaining individual differences in life outcomes, it is crucial to compare the links between the various factors and life outcomes across models. In the current study, we compare the various psychopathology factors and their respective links with a range of major life outcomes. Specifically, with regard to the measurement models, the current study builds on our previous work replicating the approach of Caspi and colleagues.5, 14 That is, we compared the traditional correlated‐three‐factors model (INT/EXT/THO; model A) with the following three alternative models: a bifactor model (INT/EXT/THO and GEN; model B), a revised‐bifactor model (INT/EXT and GEN; model ′B) and a higher‐order model (model C). In the subsequent structural models, a range of life outcomes were added to the models. Life outcomes were assessed in line with previous work using the same data30 in five major life domains: temperament, economic outcomes, social outcomes, psychological outcomes and health behaviours. Based on previous correlational evidence,5 we hypothesized that the GEN factor is related to more adverse outcomes in all domains, whereas INT, EXT and THO are expected to be related to only some of the outcome variables after the GEN factor is taken into account. Specifically, in the three‐correlated‐factors model, EXT was expected to show the strongest links with social outcomes, whereas INT is expected to show the strongest associations with psychological outcomes. Additionally, we hypothesized that the GEN factor accounts for a substantial part of the links among INT, EXT, THO and life outcomes, as revealed in the literature in which factor scores were based on the traditional three‐correlated‐factors models. As such, we expected that most of the links between respectively INT, EXT, THO and life outcomes would decrease or would not remain significant in a model including a GEN factor. Finally, to test the robustness of the findings and possible effects of common method variance, relations among parent‐reported psychopathology and (primarily adolescent‐reported) life outcomes were also analysed.

Methods

Sample

The TRacking Adolescents' Individual Lives Survey (TRAILS) is a large prospective cohort study of Dutch adolescents who were followed biennially or triennially from 11 to at least 25 years of age. The current research uses data from all five assessment waves (T1 to T5). Children born between 1 October 1989 and 30 September 1991 were eligible to participate, providing they met the inclusion criteria and their schools were willing to participate.31 Over 90% of the schools enrolling a total of 2 935 eligible children agreed to participate in the study. Seventy‐six percent of these children and their parents consented to participate (T1, n = 2 230, mean age = 11.1 years, standard deviation = 0.6, 50.8% girls). Subsequent data collection waves had good retention rates (T2 mean age 13.6, 96%; T3 mean age 16.3, 81%; T4 mean age 19.1, 84%; and T5 mean age 22.3, 80%). Non‐response and attrition during follow‐ups was somewhat higher in males and in adolescents of non‐Western ethnicity, with divorced parents, low socio‐economic status, low intelligence quotient and academic achievement and poor physical health and with behaviour and substance use problems.32 Non‐response showed little to no association with urbanization, parental religiousness, being an only child or recent self‐reports of anxiety and mood problems.33 Each assessment wave was approved by the national ethical committee (CCMO, http://www.ccmo.nl).

TRAILS data are not open source but accessible for researchers outside the TRAILS consortium by submitting a publication proposal. For information on access to the data, see https://www.trails.nl/en/hoofdmenu/data/data-use. All codebooks are available from https://easy.dans.knaw.nl/ui/home. To enable the reproducibility of our analyses, all descriptions and correlations for all study variables are reported in Table S2. All Mplus output files are available at https://osf.io/8275s/.

Measures

A list of variable names and response categories are presented in the supporting information.

Psychopathology. Psychopathology was assessed using a range of adolescent‐report and parent‐report measures. Adolescent‐report measures included the Youth Self Report (T1, T2 and T3), the Adult Self Report (ASR; T4), the Revised Child Anxiety and Depression Scale (T1, T2 and T3) and the Community Assessment of Psychic Experiences (T3). The Youth Self Report and ASR were used to assess anxious‐depressed, withdrawn‐depressed, aggressive behaviour, delinquent behaviour, attention‐hyperactivity problems and thought problems.34, 35 The Revised Child Anxiety and Depression Scale was used to asses generalized anxiety disorder, social anxiety, separation anxiety, panic disorder and obsessive‐compulsive disorder.36, 37 The Community Assessment of Psychic Experiences was used to assess psychotic experiences (both frequency and distress38;).

Parent‐reported psychopathology was also used in the current study. Parent‐reported psychopathology of the adolescent was assessed using the Child Behavior Checklist (T1, T2 and T3), one of the most commonly used parent‐report questionnaires in child and adolescent psychiatric research.39 The symptom dimensions covered by the Child Behavior Checklist are anxious‐depressed, withdrawn‐depressed, aggressive behaviour, delinquent behaviour, attention‐hyperactivity problems and thought problems.

Outcomes. Outcomes covering five life domains were included in the analyses: temperament, economic outcomes, social outcomes, psychosocial outcomes and health behaviours. Outcome variables were selected to correspond with previous work on the TRAILS data.30 Continuous outcome measures were utilized as much as possible. For the few outcome measures for which this was not possible (e.g. being pregnant as a teenager vs. not being pregnant as a teenager), we used binary variables. All outcomes were assessed at T5 when participants were 22 years old, with the exception of suicidal ideation, which was assessed at T4 (and not at T5).

Five temperament traits were included: effortful control, frustration, fearfulness, affiliation and shyness. Temperament was assessed using the Early Adolescent Temperament Questionnaire (parent‐reported40;).

Four economic outcomes were included: (1) attained educational level or current level if still studying; (2) receiving social security benefits due to unemployment or long‐term illness; (3) absenteeism from work; and (4) serious financial difficulties in the past 2 years (adolescent‐reported).

Five social outcomes were included: (1) antisocial behaviour (assessed with the Anti‐social Behavior Questionnaire; adolescent‐reported41); (2) teenage pregnancy (adolescent‐reported); and (3) being let down by a friend or relative, (4) having a serious conflict with somebody at least twice and (5) physical assault (including rape). Teenage pregnancy, being let down, serious conflict and physical assault were all assessed with the Life Event Checklist, asking the adolescents for events that occurred in the last 2 years (adolescent‐reported42).

Six psychological outcomes were included: (1) lifetime serious suicidal ideation (measured at T4; adolescent‐reported43;); (2) use of specialty mental health services as registered in the Psychiatric Case Register North Netherlands: lifetime day treatment or inpatient care; (3) use of specialty mental healthcare in the past 2 years; (4) low levels of happiness and/or life satisfaction (adolescent‐reported; TRAILS questionnaire); (5) poor sleep quality as indexed by the Nottingham Health ;); and (6) feeling lonely in the past 6 months (adolescent‐reported; T5 ASR35;).

Five health behaviours were included: (1) daily smoking (10+ cigarettes per day); (2) alcohol use; (3) cannabis use; (4) body mass index (BMI; as measured during physical examinations or, if unavailable, by adolescent report); and (5) subjective physical health. Health behaviours were assessed with five questions (adolescent‐reported except for BMI; TRAILS questionnaire).

Statistical analyses

All analyses were performed within a structural equation modelling framework in Mplus 8 (45). In the current study, we examined how a three‐correlated‐factors model, a bifactor model, a revised‐bifactor model and a higher‐order model of psychopathology compared with each other when examining their associations with a range of life outcomes. To do so, we specified two measurement models presented in the work of Laceulle and colleagues (14): model A (including INT, EXT and THO) and model ′B (including the GEN factor, and INT and EXT, with INT and EXT being allowed to correlate, and the THO factor being omitted from the model). We extended the previous work by also specifying model B (included the THO factor) and model C (including a GEN factor that was indicated by INT, EXT and THO). We began our model specification by specifying measurement models such that scale scores for the various psychopathology measures were used as manifest indicators of latent symptom variables, and then, depending on the model, latent symptom variables were specified as indicators of the INT, EXT, THO factors and/or a GEN factor. We then specified structural models in which we entered scale scores for the outcome variables as manifest variables, allowed the outcome variables to covary and then regressed them onto the INT, EXT, THO factors and/or GEN factor. We chose a Bayesian approach for estimating our models. This approach differs from the frequentist approach (e.g. maximum likelihood) in the conceptualization and computation of estimates, such that it uses both prior parameter distributions and the likelihood to create posterior distributions from which estimates are drawn. For the applied researcher, this approach is recommended for models that have shown convergence problems, including negative residual variances,46 and for models that are structurally similar to bifactor models, such as multitrait multimethod models.47, 48 Each model was run at least twice, such that proportional scale reduction factor was examined after the first run, and iterations were increased by at least a factor of 2 using the FBITER command in Mplus. Model convergence was determined by obtaining a proportional scale reduction value of <1.1.

Results from the models using Maximum Likelihood Robust estimation are available here: https://osf.io/8275s/.

The Bayesian estimation approach also provides statistics that differ from traditional analytic techniques. Specifically, this approach does not yield model fit indices that can be interpreted in the same way as a reader might interpret values for popular incremental fit indices such as the Tucker–Lewis index or the comparative fit index and absolute measures of fit such as the root mean square of approximation. Instead, the Bayesian estimation approach provides the comparative measures of the Bayesian information criteria (BIC) and deviance information criteria (DIC49, 50;). Values of the BIC and DIC indicate how well a model might fit the data when compared with an alternative model, with lower values indicating better model fit. Model fit indices are shown in Table 1.

Table 1

Model fit statistics for measurement and structural models using adolescent reports and parent reports

			Model
			Model A (correlated‐three‐factors)	Model B (bifactor)*	Model ′B (revised‐bifactor)	Model C (higher‐order)
Adolescent report
Measurement		DIC	180,215.27	179,306.82	179,624.57	180,246.32
		pD	179.11	−6.82	184.84	176.24
		BIC	181,235.80	180,667.64	180,688.32	181,272.20
Structural		DIC	256,566.04	254,944.08	255,309.84	256,425.34
		pD	877.66	680.91	637.48	565.15
		BIC	259 638.35	258,713.64	258,934.59	259,756.40
Parent report
Measurement		DIC	72,396.18	72,248.29	72,087.93	72,180.93
		pD	80.44	−177.16	83.47	79.53
		BIC	72,841.90	73,300.64	72,565.44	72,636.04
Structural		DIC	147,979.35	147,644.26	147,646.69	148,171.16
		pD	518.85	544.92	525.39	466.83
		BIC	151,012.42	150,859.59	150,707.39	150,931.14

Fit statistics should be interpreted with caution as the measurement model B did not converge adequately for both adolescent report and parent report. DIC, deviance information criterion; pD, posterior mean of the deviance minus the deviance of the posterior mean; BIC, Bayesian information criterion.

Additionally, the Bayesian estimation approach provides posterior standard deviations—the variance of the posterior distribution for the parameter—and can be seen as a Bayesian analogue to standard errors. Furthermore, in general, given the large sample size and number of estimated associations caution is needed with interpreting the p‐values. Instead, interpretation of the effects is based on standardized effects (βs) and explained variance (R 2). To test the robustness of the findings and effects of common method variance, parent‐reported psychopathology was also analysed. Descriptive statistics and correlations among all study variables are shown in the supporting information.

Results

Descriptive statistics

Descriptive statistics and correlations among all study variables are reported in the supporting information. Plots showing the associations between psychopathology and the various outcomes are also shown in the supporting information. Plots were created in R51 using the ggplot2 package.52

Associations between psychopathology and the various outcomes

Adolescent‐reported psychopathology and life outcomes. Model fit statistics for both the structural and measurement models are shown in Table 1. Path diagrams, factor loadings and ancillary statistics for each of the measurement models can be found in the supporting information. Measurement model B, the bifactor model, did not converge according to the criteria described previously, and as such, fit statistics should be interpreted with caution.5, 14 All other models, including the structural model B, converged according to the criteria described previously. When examining model fit indices such as DIC and BIC, differences between models were small, indicating that there was no clear superior model but rather that the relative quality of the models when compared with another was similar.

The structural models showed a range of associations between the psychopathology factors and outcomes in the various models (Table 2). First, the THO factor showed modest to moderate links with several outcomes in model A (the three‐correlated‐factors model). While more thought problems were generally related to less adaptive outcomes, there were a few exceptions where more thought problems were related to more adaptive outcomes (e.g. shyness: β = −0.25, alcohol use: β = −0.08). When adding GEN to the model (model B), these associations with maladaptive outcomes decreased in effect size as a substantial amount of variance shifted from the THO factor to the GEN factor. Additionally, substantial decreases were, for example, found for fear (β = 0.13 to 0.04), social security benefits (β = 0.20 to 0.10), interpersonal conflicts (β = 0.25 to 0.15) and cannabis use (β = 0.16 to 0.08). In model ′B, the separate THO factor was not specified, forcing all variance into the GEN factor. Second, the INT factor showed moderate to strong links with most temperamental and psychological outcomes in model A (the three‐correlated‐factors model). When adding a GEN factor to the model (model B) associations decreased in effect size as a substantial amount of variance shifted from the INT factor to the GEN factor. For example, the link with shyness decreased from β = 0.43 in the three‐correlated‐factors model to β = 0.08 in the revised‐bifactor model. Also, several links were revealed with the health, social and economic outcomes, indicating that adolescents reporting more INT were lower on, for example, smoking, cannabis and alcohol use, substantial antisocial behaviour and interpersonal conflict and higher on attained educational level. These patterns with INT being related to adaptive outcomes became sometimes even more prominent after adding the GEN factor. Third, the EXT factor showed mainly associations with health outcomes and with a few temperamental and social outcomes. The associations that were found in model A (e.g. less adaptive health outcomes, lower effortful control and more substantial antisocial behaviour) remained in models B and ′B when adding the GEN factor to the model, indicating that little variance shifted to the GEN factor. Fourth, the GEN factor showed substantial links with almost all outcomes in models B, ′B and the higher‐order model (model C). This was also the case for more severe outcomes such as suicidal ideation and recent psychiatric hospitalization, outcomes that were not related to INT and EXT in any of the models. In model ′B (after forcing all THO variance to the GEN factor), however, the strength of the associations was larger than in model B. Strongest associations were found for temperamental outcomes (e.g. effortful control (β = −0.23), frustration (β = 0.29) and fear (β = 0.34)) and psychological outcomes (e.g. suicidal ideation (β = 0.30), specialist mental healthcare (β = 0.30), being unhappy (β = 0.34) and loneliness (β = 0.36)). Models ′B and C showed highly similar associations between the GEN factor and the various outcomes. Finally, variance explained by all psychopathology factors together was calculated for each of the outcome variables. In general, the amount of explained variance was rather modest, with the largest amounts being found for the temperamental outcomes (8–16%). Plots showing the standardized regression coefficients for each factor model and for each outcome domain can be found in the supporting information.

Table 2

Associations between adolescent‐reported problems and various life outcome variables

		Model A (three‐correlated‐factors)							Model B (bifactor)
						95% CI							95% CI
		B	Posterior SD	β	p	Lower 2.5%	Upper 2.5%	R 2	B	Posterior SD	β	p	Lower 2.5%	Upper 2.5%	R 2
Temperament
Effortful control	GEN							0.161	−0.076	0.021	−0.104	0.000	−0.117	−0.035	0.159
	EXT	−0.263	0.039	−0.360	0.000	−0.344	−0.190		−0.233	0.032	−0.319	0.000	−0.296	−0.171
	INT	0.042	0.051	0.058	0.196	−0.070	0.138		0.013	0.026	0.018	0.315	−0.038	0.065
	THO	−0.001	0.070	−0.002	0.493	−0.132	0.145		0.007	0.034	0.009	0.421	−0.059	0.075
Frustration	GEN							0.141	0.139	0.020	0.198	0.000	0.099	0.179	0.131
	EXT	0.210	0.038	0.300	0.000	0.138	0.290		0.188	0.032	0.269	0.000	0.127	0.251
	INT	0.098	0.057	0.139	0.040	−0.033	0.204		0.024	0.026	0.035	0.174	−0.027	0.075
	THO	−0.041	0.072	−0.058	0.265	−0.196	0.088		−0.038	0.035	−0.054	0.137	−0.106	0.031
Fear	GEN							0.119	0.219	0.019	0.321	0.000	0.181	0.257	0.110
	EXT	−0.020	0.038	−0.029	0.296	−0.094	0.053		0.010	0.030	0.015	0.365	−0.050	0.068
	INT	0.180	0.072	0.264	0.027	−0.060	0.284		0.043	0.026	0.063	0.052	−0.009	0.094
	THO	0.089	0.069	0.130	0.090	−0.045	0.231		0.026	0.033	0.039	0.208	−0.037	0.093
Affiliation	GEN							0.072	0.025	0.017	0.044	0.071	−0.008	0.060	0.115
	EXT	−0.129	0.033	−0.221	0.000	−0.201	−0.068		−0.061	0.027	−0.105	0.011	−0.116	−0.009
	INT	−0.057	0.052	−0.097	0.093	−0.163	0.041		−0.149	0.034	−0.257	0.008	−0.194	−0.104
	THO	0.140	0.066	0.240	0.007	0.032	0.286		0.076	0.032	0.130	0.010	0.016	0.137
Shyness	GEN							0.079	0.116	0.024	0.145	0.000	0.068	0.163	0.160
	EXT	−0.029	0.047	−0.035	0.274	−0.115	0.071		−0.083	0.038	−0.104	0.016	−0.157	−0.008
	INT	0.343	0.134	0.426	0.026	−0.242	0.506		0.298	0.061	0.373	0.008	0.231	0.365
	THO	−0.201	0.103	−0.249	0.009	−0.426	−0.039		−0.126	0.047	−0.158	0.007	−0.209	−0.042
Economic outcomes
Attained educational level (low)	GEN							0.071	0.005	0.020	0.007	0.407	−0.033	0.043	0.083
	EXT	0.088	0.037	0.136	0.013	0.012	0.158		0.093	0.031	0.145	0.002	0.031	0.154
	INT	−0.141	0.076	−0.218	0.028	−0.264	0.093		0.053	0.027	0.083	0.025	0.000	0.103
	THO	0.173	0.074	0.267	0.006	0.051	0.335		0.076	0.035	0.117	0.015	0.009	0.144
Frequent absenteeism work/study	GEN							0.024	0.500	0.178	0.085	0.002	0.152	0.849	0.024
	EXT	−0.014	0.336	−0.002	0.483	−0.688	0.634		−0.044	0.284	−0.007	0.436	−0.608	0.501
	INT	0.159	0.418	0.027	0.345	−0.699	0.968		0.476	0.236	0.081	0.024	0.007	0.934
	THO	0.744	0.595	0.127	0.093	−0.394	1.985		0.316	0.303	0.054	0.144	−0.262	0.941
Social security benefits	GEN							0.047	0.034	0.008	0.121	0.000	0.017	0.051	0.043
	EXT	0.005	0.016	0.017	0.386	−0.028	0.036		0.008	0.014	0.028	0.287	−0.020	0.034
	INT	−0.001	0.022	−0.004	0.477	−0.048	0.040		0.017	0.011	0.060	0.068	−0.006	0.039
	THO	0.058	0.030	0.204	0.021	0.003	0.121		0.029	0.016	0.104	0.026	0.000	0.060
Serious financial difficulties	GEN							0.093	0.129	0.026	0.149	0.000	0.078	0.180	0.091
	EXT	0.113	0.049	0.130	0.015	0.013	0.208		0.122	0.041	0.141	0.003	0.039	0.201
	INT	−0.009	0.068	−0.011	0.441	−0.148	0.121		0.018	0.032	0.021	0.287	−0.045	0.082
	THO	0.172	0.089	0.199	0.023	0.003	0.360		0.098	0.046	0.114	0.016	0.011	0.189
Social outcomes
Substantial antisocial behaviour	GEN							0.129	0.003	0.004	0.024	0.218	−0.004	0.010	0.158
	EXT	0.045	0.007	0.354	0.000	0.031	0.059		0.051	0.006	0.403	0.000	0.040	0.063
	INT	−0.018	0.011	−0.142	0.046	−0.036	0.011		−0.014	0.006	−0.112	0.009	−0.024	−0.004
	THO	0.001	0.013	0.010	0.457	−0.024	0.027		−0.006	0.006	−0.044	0.188	−0.018	0.007
Teenage pregnancy	GEN							0.053	−0.005	0.007	−0.019	0.253	−0.019	0.009	0.069
	EXT	0.031	0.014	0.118	0.014	0.003	0.057		0.027	0.012	0.102	0.013	0.003	0.049
	INT	−0.026	0.020	−0.101	0.075	−0.064	0.019		0.012	0.010	0.046	0.102	−0.007	0.031
	THO	0.028	0.025	0.106	0.117	−0.019	0.079		0.021	0.013	0.079	0.051	−0.005	0.046
Recent let down experience	GEN							0.078	0.058	0.014	0.124	0.000	0.030	0.085	0.074
	EXT	0.048	0.027	0.102	0.040	−0.006	0.101		0.061	0.022	0.130	0.004	0.017	0.105
	INT	−0.005	0.036	−0.011	0.439	−0.081	0.064		−0.004	0.018	−0.009	0.403	−0.039	0.031
	THO	0.077	0.049	0.165	0.046	−0.016	0.179		0.036	0.025	0.076	0.070	−0.014	0.083
Recent interpersonal conflicts	GEN							0.084	0.048	0.013	0.113	0.000	0.023	0.072	0.082
	EXT	0.043	0.025	0.101	0.048	−0.009	0.091		0.045	0.021	0.107	0.019	0.003	0.085
	INT	−0.035	0.038	−0.081	0.150	−0.111	0.044		−0.006	0.016	−0.014	0.355	−0.038	0.026
	THO	0.105	0.047	0.246	0.009	0.023	0.206		0.066	0.024	0.154	0.008	0.020	0.112
Physical assault, including rape	GEN							0.025	0.006	0.006	0.033	0.135	−0.005	0.018	0.029
	EXT	−0.004	0.011	−0.022	0.353	−0.028	0.017		−0.004	0.009	−0.019	0.347	−0.023	0.014
	INT	−0.019	0.017	−0.101	0.093	−0.053	0.017		0.007	0.008	0.038	0.177	−0.008	0.022
	THO	0.041	0.022	0.216	0.019	0.003	0.088		0.025	0.011	0.132	0.013	0.004	0.046
Psychological outcomes
Suicidal ideation	GEN							0.124	0.063	0.009	0.224	0.000	0.046	0.080	0.124
	EXT	−0.003	0.016	−0.012	0.416	−0.035	0.028		−0.008	0.014	−0.030	0.262	−0.036	0.018
	INT	0.071	0.031	0.252	0.027	−0.027	0.117		0.064	0.016	0.228	0.008	0.042	0.088
	THO	0.036	0.029	0.128	0.105	−0.023	0.093		0.029	0.015	0.103	0.025	0.000	0.057
Psychiatric hospitalization	GEN							0.048	0.024	0.006	0.124	0.000	0.013	0.035	0.047
	EXT	0.016	0.010	0.084	0.053	−0.004	0.037		0.009	0.009	0.047	0.148	−0.008	0.026
	INT	0.037	0.018	0.195	0.028	−0.010	0.067		0.027	0.009	0.141	0.008	0.012	0.042
	THO	−0.007	0.020	−0.037	0.347	−0.050	0.029		0.001	0.009	0.004	0.468	−0.018	0.019
Specialist mental healthcare	GEN							0.113	0.148	0.022	0.196	0.000	0.104	0.191	0.115
	EXT	0.119	0.040	0.157	0.002	0.042	0.198		0.078	0.033	0.103	0.010	0.012	0.142
	INT	0.142	0.069	0.189	0.030	−0.042	0.257		0.139	0.037	0.184	0.008	0.081	0.196
	THO	0.035	0.074	0.046	0.314	−0.119	0.175		0.037	0.036	0.049	0.148	−0.034	0.107
Unhappy and/or dissatisfied	GEN							0.153	0.441	0.045	0.299	0.000	0.353	0.528	0.166
	EXT	0.054	0.085	0.036	0.255	−0.107	0.229		0.011	0.068	0.007	0.435	−0.124	0.143
	INT	0.523	0.195	0.352	0.026	−0.295	0.775		0.378	0.085	0.256	0.008	0.260	0.497
	THO	0.029	0.161	0.019	0.425	−0.315	0.327		0.013	0.074	0.009	0.421	−0.128	0.161
Sleep problems	GEN							0.073	0.187	0.033	0.173	0.000	0.122	0.253	0.086
	EXT	−0.031	0.063	−0.029	0.307	−0.155	0.091		−0.054	0.055	−0.050	0.159	−0.165	0.050
	INT	0.068	0.087	0.063	0.212	−0.116	0.235		0.150	0.048	0.139	0.008	0.064	0.233
	THO	0.255	0.116	0.236	0.013	0.041	0.495		0.180	0.062	0.166	0.007	0.067	0.298
Loneliness	GEN							0.211	0.234	0.018	0.385	0.000	0.199	0.269	0.208
	EXT	−0.005	0.036	−0.009	0.440	−0.072	0.069		−0.018	0.028	−0.029	0.262	−0.071	0.037
	INT	0.344	0.122	0.560	0.026	−0.260	0.461		0.152	0.036	0.250	0.008	0.099	0.203
	THO	−0.078	0.076	−0.128	0.114	−0.243	0.048		−0.045	0.031	−0.074	0.068	−0.106	0.016
Health behaviours
Cigarette smoking (daily)	GEN							0.075	0.009	0.011	0.025	0.209	−0.013	0.030	0.096
	EXT	0.080	0.021	0.220	0.000	0.037	0.120		0.107	0.017	0.294	0.000	0.072	0.140
	INT	−0.065	0.037	−0.180	0.030	−0.125	0.041		−0.038	0.016	−0.105	0.011	−0.066	−0.009
	THO	0.056	0.038	0.153	0.055	−0.013	0.138		0.014	0.019	0.040	0.220	−0.022	0.052
Alcohol use	GEN							0.130	−0.055	0.155	−0.010	0.363	−0.361	0.246	0.163
	EXT	1.348	0.297	0.251	0.000	0.791	1.956		1.597	0.253	0.297	0.000	1.118	2.116
	INT	−0.419	0.406	−0.078	0.144	−1.092	0.567		−0.841	0.267	−0.156	0.008	−1.259	−0.413
	THO	−0.423	0.528	−0.079	0.185	−1.546	0.552		−0.356	0.280	−0.066	0.091	−0.900	0.179
Cannabis use	GEN							0.116	−0.118	0.287	−0.012	0.341	−0.678	0.450	0.138
	EXT	2.380	0.527	0.248	0.000	1.357	3.435		2.734	0.459	0.286	0.000	1.807	3.625
	INT	−1.946	1.040	−0.203	0.027	−3.259	2.266		−0.212	0.366	−0.022	0.278	−0.897	0.530
	THO	1.549	0.917	0.162	0.043	−0.225	3.400		0.726	0.478	0.076	0.057	−0.165	1.714
High body mass index	GEN							0.019	0.095	0.131	0.023	0.234	−0.163	0.349	0.029
	EXT	0.566	0.240	0.135	0.009	0.101	1.053		0.423	0.206	0.101	0.020	0.020	0.828
	INT	0.139	0.308	0.033	0.311	−0.444	0.788		0.428	0.181	0.102	0.012	0.085	0.757
	THO	−0.190	0.431	−0.045	0.313	−1.113	0.607		−0.147	0.222	−0.035	0.253	−0.580	0.298
(Low) subjective health	GEN							0.121	0.223	0.025	0.269	0.000	0.175	0.272	0.113
	EXT	0.059	0.047	0.072	0.099	−0.031	0.154		0.050	0.039	0.060	0.102	−0.027	0.124
	INT	0.182	0.080	0.220	0.027	−0.043	0.313		0.088	0.034	0.106	0.010	0.023	0.151
	THO	0.079	0.084	0.096	0.165	−0.096	0.239		0.055	0.042	0.066	0.095	−0.027	0.137

GEN, general psychopathology; EXT, externalizing problems; INT, internalizing problems; THO, thought problems.

Sex of adolescent (1 = boys, 0 = girls) was included as a covariate in all analyses. All p‐values are one‐tailed.

Parent‐reported psychopathology and life outcomes. Model fit statistics for both the structural and measurement models are shown in Table 1. Path diagrams, factor loadings and ancillary statistics for the measurement models, as well as the associations with outcomes and plots can be found in the supporting information. All models except the measurement model for model B converged adequately with no clear superior model. Most of the patterns found were similar to those indicated for adolescent‐reported psychopathology. However, associations between the parent‐reported psychopathology factors and the (also parent‐reported) temperamental outcomes were stronger in this batch of models compared with the models with adolescent‐reported psychopathology, suggesting common method variance effects.

Discussion

In the current study, we started with multiple measurement models (i.e. the traditional three‐correlated‐factors model (model A), the bifactor model (model B), the revised‐bifactor model (model ′B, as proposed by Caspi et al., 2013) and a higher‐order model (model C)) and then extended these models with the various life outcomes (i.e. structural models A, B, ′B and C). Bayesian fit indices (BIC and DIC) were slightly lower for the three‐correlated‐factors model, but differences were small and do not allow strong conclusions regarding the dominance of any of the models. To probe the usefulness of the different models and the GEN factor in particular, we tested the links between psychopathology and life outcomes across the various models.

First, the THO factor was related to many outcomes in model A. However, the correlations decreased in magnitude when a GEN factor was added to the model, suggesting that a substantial amount of variance (shared between the THO factor and the outcomes) shifted from the THO factor to the GEN factor. This seems to support our earlier findings14 when we found that a bifactor model in which a separate THO factor was not specified (thus forcing all variance into the GEN factor; model ′B) converged better than the bifactor model with a separate THO factor specified (model B). Moreover, this finding might bolster the idea that the GEN factor may primarily reflect disordered form and content of thoughts, which—as suggested by Caspi and colleagues (2013)—may capture the extreme of practically every disorder.5 Second, for INT, somewhat similar associations were found. Specifically, the INT factor showed substantial links with young adult fear, shyness, feelings of unhappiness and loneliness in the three‐correlated‐factors model (model A) that could partially be explained by the GEN factor. Notably, INT was related to a higher attained educational level both in model A and in the bifactor models (models B and ′B), suggesting that INT may have some advantage, which becomes most obvious when controlling for the underlying psychopathology. This findings is in line with recent findings demonstrating a positive link between internalizing problems and academic achievement28 and raises the possibility that individuals with internalizing tendencies but no severe psychopathology have more attentional control,53, 54, 55 which make them do better at school and work.56 Third, with regard to EXT, most of the associations revealed in model A remained in the bifactor models (models B and ′B) (e.g. low effortful control, frustration, antisocial behaviour, smoking, alcohol use, cannabis use and high BMI). Earlier research has found little evidence for an association between EXT and outcomes during adolescence,28 which may be explained by the limited selection of outcomes assessed (i.e. criminal, academic and affective outcomes). However, results from both studies seem to suggest that EXT, in addition to the link between EXT and outcomes, is rather independent of the GEN factor. Moreover, our findings suggest that we included some adolescent‐specific symptoms (e.g. delinquency) that may not be very indicative of underlying psychopathology and health outcomes that are not strongly related to general or severe psychopathology.57 Assessing more chronic or life‐course‐persistent externalizing problems (e.g. antisocial personality disorder) and measuring the health outcomes later in adulthood may shed light on this issue. Finally, with regard to the GEN factor, higher scores on the GEN factor were related to most adverse outcomes across life domains. This supports previous findings providing support for correlates of the GEN factor across different domains of functioning, including personality,5, 9, 17, 19 academic functioning5, 19, 28 and well‐being.28 Moreover, the GEN factor was related to some real‐life and severe adverse outcomes (i.e. BMI and psychiatric hospitalization) that only showed very small links with the psychopathology factors in model A. This is an important finding as it suggests that the GEN factor may not be entirely reflective of individual differences in response styles.5, 29, 58

To investigate the robustness of the associations between psychopathology and the life outcomes, and criterion validity of the GEN factor more specifically, the same links were examined using parent‐reported psychopathology. Patterns found were generally similar to those found for adolescent‐reported psychopathology. However, many of the associations with temperament were smaller in size for the adolescent‐reported psychopathology models, likely reflecting effects of common method variance. Relatedly, for more objective life outcomes such as psychiatric hospitalization and BMI, the strength of the effects was similar for adolescent‐reported and parent‐reported psychopathology. Some exceptions were found with regard to social outcomes (i.e. feeling let down by a close other and experiencing a recent interpersonal conflict), where links were weaker when externalizing problems were reported by the adolescent. It could be that parents are more perceptive to certain externalizing social problems of their child than adolescents are.

Taken together, our findings contribute to the understanding of the construct validity of a GEN factor as well as the meaning of the residual parts of INT and EXT that do not overlap with the GEN factor. Specifically, comparison of the associations with outcomes across the various models demonstrates that each higher‐order model has its own implications for the (interpretation of) both the factors59 and the associations found (e.g.5, 60). While this may not show from the small differences in model fit statistics between the measurement models, the associations with outcomes vary dependent on whether and how the GEN factor is taken into account. For example, whereas an association of β = 0.264 is found between INT and the temperamental trait fear in model A, the association drops to β = 0.063 after taking into account GEN. This suggests that associations traditionally assumed to be domain specific may instead be reflective of an individual's general propensity to develop psychopathology.

The current study is not without limitation. Alternative measurement models have been proposed, some of which we took into account (a higher‐order model), and some of which we did not take into account (removing intercorrelations between factors, as this would hinder shared variance by any two latent factors not shared with the third ) or using different assignment of problem domains to the broader factors.61, 62, 63, 64 Notably, Kotov and colleagues introduced a Hierarchical Taxonomy of Psychopathology (HiTOP11;). In addition to proposing a higher‐order model, HiTOP provides a classification system explicitly stating causal influences on psychopathology dimensions and capturing the full range of psychiatric problems (including personality disorders and eating pathology). As such, HiTOP bridges dimensional models including bifactor approaches with the broad yet categorical approaches to psychopathology as described in the DSM‐5.

Results for models specifying a bifactor structure

Additionally, although all life outcomes were measured after the assessment of psychopathology, we could not control for all outcomes prior to the first measurement of psychopathology. As such, we cannot be conclusive regarding the direction of the effects, nor do we know whether the findings hold when individuals are older (which may point into the direction of a scarring effect30, 65, 66;). Relatedly, the longitudinal design was taken into account by modelling an assessment factor into each model to address age and wave‐related variance. Also, previous work in this field has rarely address the structure of psychopathology longitudinally.58, 67 Future research should take a more explicit developmental approach examining measurement invariance and age‐graded changes in the various models.

Furthermore, for most of the associations, both psychopathology and the outcomes were measured using reports from the same informant (i.e. the adolescent). As such, for these associations, links may partially be the result of correlated measurement error. However, all outcomes in the temperament domain were measured with parent report, and objective measures were used for psychiatric hospitalization (measured using the Psychiatric Case Register North Netherlands) in the psychological domain and BMI (measured during the physiological assessments) in the health domain. Also, outcome items were omitted that were very similarly worded or almost identical to the symptoms, and life events or behavioural outcomes were used whenever possible (i.e. limiting conceptual dependence). Nonetheless, more objective and real‐life outcomes would allow to provide a (more) robust test of the extent to which the GEN factor reflects individual differences in response style.5, 29, 58, 68 Relatedly, although a wide range of outcomes across several domains was taken into account, the current study was by no means exhaustive. For example, only five temperamental traits were taken into account including multifaceted constructs. For example, the negative associations between effortful control and psychopathology may be suppressed by some facets (i.e. activation control) that are positively associated with psychopathology.69 As such, an even broader range of outcomes and more specific individual characteristics might be important to consider.

Finally, it is important to keep in mind that virtually all associations represent weak to modest effects. Regarding the measurement models, differences found might mainly reflect the complexity of the models and are too small to draw strong conclusions in favour of a particular model. Also, the associations between psychopathology factors and outcomes were small in magnitude, as was the amount of variance in outcomes explained by the psychopathology factors. As such, the practical significance of our results is limited. Nonetheless, small effects may accumulate and as such should not be outright rejected,70 and several relative changes in associations across models might be interpreted as moderate (25–49%) to strong (>50%; based on standards used in intervention studies; e.g.71).

The current study sought to increase our understanding of the structure of psychopathology and the GEN factor in particular by providing relative comparisons between multiple measurement models and structural models examining the associations between psychopathology and life outcomes. While there was no clear superior model, modest associations between the psychopathology factors and life outcomes were found that suggest meaningful differences across the various models. Importantly, after taking into account the GEN factor, the associations with life outcomes decreased for the residual parts of THO (all domains) and INT (temperamental and psychological outcomes), but not for EXT, compared with the traditional three‐correlated‐factors model. These findings suggest that problems in the externalizing domain are not always pathological during adolescence and that the residual part of EXT adds explanatory power to the GEN factor in predicting life outcomes.

Table S1. List of study variable names and response categories

Table S2. Descriptive statistics for continuous life outcome variables

Table S3. Factor loadings and covariances for unconditional model A (three‐correlated‐factors) using adolescent report

Table S4. Factor loadings and covariances for unconditional model B (bifactor) using adolescent report

Table S5. Factor loadings and covariances for unconditional model ′B (revised‐bifactor) using adolescent report

Table S6. Factor loadings for unconditional model C (hierarchical) using adolescent report

Table S7. Factor loadings and covariances for unconditional model A (three‐correlated‐factors) using parent report

Table S8. Factor loadings and covariances for unconditional model B (bifactor) using parent report

Table S9. Factor loadings and covariances for unconditional model ′B (revised‐bifactor) using parent report

Table S10. Factor loadings for unconditional model C (hierarchical) using parent report

Table S10. Ancillary statistics for all unconditional models

Table S11. Correlations among mean plausible values for the general factor of psychopathology from unconditional models

Table S12. Associations between parent‐reported psychopathology and various life outcome variables in all four models

Click here for additional data file.

Figure 1a. Associations between self‐reported psychopathology and temperament variables

Figure 1b. Associations between self‐reported psychopathology and economic outcomes

Figure 1c. Associations between self‐reported psychopathology and social outcomes

Figure 1d. Associations between self‐reported psychopathology and psychological outcomes

Figure 1e. Associations between self‐reported psychopathology and health behaviors

Figure 1f. Associations between parent‐reported psychopathology and temperament variables

Figure 1g. Associations between parent‐reported psychopathology and economic outcomes

Figure 1h. Associations between parent‐reported psychopathology and social outcomes

Figure 1i. Associations between parent‐reported psychopathology and psychological outcomes

Figure 1j. Associations between parent‐reported psychopathology and health behaviors

Click here for additional data file.