Associations between critical consciousness and well-being in a national sample of college students during the COVID-19 pandemic.

Critical consciousness (CC) may promote well-being, particularly during the COVID-19 pandemic. In a national survey of 707 college students conducted in April 2020, we first validated the Short Critical Consciousness Scale (ShoCCS) among youth groups not often specifically examined in CC measurement (i.e., Asian, immigrant-origin, LGBQ+, and women youth). Next, we examined associations between ShoCCS subscales and validated measures of both anxiety (Generalized Anxiety Disorder-7) and hopefulness (The Individual-Differences Measure in Hopefulness). The ShoCCS achieved measurement invariance across racial/ethnic groups and immigrant-origin status, and partial invariance among LGBQ+ and women-identifying youth. We found critical reflection and action associated with anxiety for the full sample, but no evidence of moderation by sociodemographic factors. ShoCCS subscales were differentially associated with hopefulness for Asian youth and LGBQ+ youth. This study contributes to the evolution of CC measurement and extends the field by identifying well-being associations during the onset of the COVID-19 pandemic.

The global COVID‐19 pandemic amplified existing stress and anxiety disparities among young adult college students (Hoyt et al., 2021). As college youth experience the deleterious and unequal effects of the COVID‐19 pandemic on well‐being, protective factors must be identified, particularly for youth experiencing marginalization. One potential avenue to promote well‐being among youth experiencing marginalization is critical consciousness (CC) (Cadenas et al., 2021; Christens & Peterson, 2012; Godfrey et al., 2019). CC refers to the process through which individuals experiencing societal marginalization develop critical analysis of oppressive social conditions and enact individual or collective change to address these inequities (Diemer et al., 2017; Freire, 1993). As youth become aware of, and engage in, action to challenge oppressive systems, CC could have implications for well‐being, although the net effect (i.e., positive or negative) is uncertain.

Investigating the relation between CC and mental health is ever more important as the mounting economic strain, surging health inequities, and contentious sociopolitical climate in the United States may serve to catalyze college students' analysis of, and resistance to, the systems underlying these monumental societal challenges (e.g., analysis of systemic racism in the healthcare system) (Wilf et al., in prep). To explore associations between CC and well‐being among college youth during this unprecedented era, it is first critical to assess how CC measurement tools are responsive to the many marginalizing forces (e.g., escalations of White supremacy, xenophobia during the COVID‐19 pandemic that manifest in anti‐Asian racism) that influence college youth's development. We define youth experiencing marginalization as those who experience societal oppression based on race/ethnicity, gender, sexuality, immigrant‐origin status, socioeconomic position (SEP), and/or other social forces that constitute institutionalized, interpersonal, and/or internalized oppression (Crenshaw, 1993). In the current study, we seek to validate one of the newest CC measurement tools, the Short Critical Consciousness Scale (ShoCCS) (Diemer et al., 2020), across racial/ethnic, gender, sexuality, and immigrant‐origin status college youth groups. Then, upon validation, we assess associations between CC and well‐being (i.e., anxiety and hopefulness) in the early months of the COVID‐19 pandemic.

CRITICAL CONSCIOUSNESS

CC (Freire, 1993) is today conceptualized as three reciprocal subdomains (Diemer et al., 2020; Watts et al. 2003). Critical reflection is the analysis and rejection of oppressive societal inequities (e.g., analysis of historically rooted racism), and critical motivation represents one's perceived capacity and role in enacting change (e.g., sense of efficacy in addressing manifestations is racism) (Watts et al., 2011). Critical action is engagement in activities to challenge social inequities (e.g., participating in a protest; organizing community members) (Watts et al., 2011). The development of CC has been hailed as the “antidote to oppression” (Diemer & Rapa, 2016), and as “psychological armor” that provides an internal resource for resistance (Seider & Graves, 2020), based on evidence linking CC with relational, social‐emotional, educational, occupational, and community and civic outcomes (Heberle et al., 2020).

There is a range of newly developed CC measures (Diemer et al., 2020; McWhirter & McWhirter, 2016; Rapa et al., 2020; Shin et al., 2016, 2018). The ShoCCS is novel in its brevity and thus may present an opportunity for broad use. CC scales, including the ShoCCS, are often validated in samples of majority Black‐ and Latinx‐identifying youth, with Asian‐identifying youth often underrepresented (Anyiwo et al., 2020; e.g., Diemer et al., 2020; Rapa et al., 2020). CC scales are often not examined for invariance based on immigrant‐origin, LGBQ+ (lesbian, gay, bisexual, queer/questioning), and woman‐identifying youth (Bañales et al., 2020; Rapa et al., 2020). These groups are collectively rooted in a system of interlocking oppressions that perpetuate the White supremacy patriarchy (Hooks, 2000) operating to disadvantage populations in the United States that are not White heterosexual, cis‐gender men (Golash‐Boza et al., 2019).

Asian youth's distinct experiences navigating the “model minority” myth and/or coming from immigrant families who espouse optimism about the future could affect how they develop CC (Godfrey et al., 2019; Wray‐Lake et al., 2017). Indeed, immigrant‐origin youth's generational status (Wray‐Lake et al., 2015) or documentation status (Cadenas et al., 2021) could affect CC (e.g., critical motivation may help undocumented youth cope with discrimination). Education for CC can support the well‐being of LGBQ+ youth through empowerment against discrimination (Wagaman, 2016), and critical reflection in men and women may evolve uniquely based on intersections of racial and gender‐based discrimination that lead young women to be “doubly conscious” of inequities (Singh et al., 2020).

CC is also understudied in emerging adulthood (typically defined as ages 18–29; Arnett, 2000), an important developmental context for understanding the CC of youth experiencing marginalization (Bañales et al., 2020). Emerging adults are increasingly independent, exercising more agency over their lives, and constructing their trajectories into adulthood (Arnett, 2000). Notably, emerging adulthood theory has been critiqued for excluding youth whose life circumstances (e.g., social class) and/or cultural backgrounds do not support the delay of adulthood (Schoon & Lyons‐Amos, 2016). Accordingly, emerging adulthood theory may at times be incompatible with CC theory's focus on socially marginalized groups (including lower income youth). However, the youth in the current study align more with the central tenets of emerging adulthood theory as we examined CC development in full‐time college‐enrolled youth. College can be particularly formative to CC development for youth from marginalized groups (Cadenas et al., 2021; Hope & Spencer, 2017; Hope et al., 2018), and more work is needed particularly to understand how gender, sexual orientation, and immigrant‐origin identities further inform CC during this developmental period and specific context.

WELL‐BEING AND CC

The context of college can also put emerging adults, particularly those from marginalized groups, at unique risks for mental health challenges (Arnett, 2000; Hope et al., 2018). The relationship between CC and anxiety is likely particularly complex. Youth who possess a critical lens of inequities may actually be more likely to experience negative socioemotional health than their less critically reflective peers (Godfrey et al., 2019). Critical action may result in increased emotional stress (DeAngelo et al., 2016; Ortega‐Williams et al., 2020) and may cause fatigue and exposure to discrimination (Ballard & Ozer, 2016; Hope et al., 2018). Critical action is associated with higher stress and anxiety among Black college students (Hope et al., 2018), but critical action may also buffer against anxiety as youth channel energy into enacting collective critical action (Heberle et al., 2020), and it is associated with lower psychological distress among racially diverse LGBTQ youth (Fine et al., 2018) and Latinx college youth (Hope et al., 2018). Anxiety may deter the development of critical motivation; undocumented Latinx youth's critical motivation was not found directly linked to less anxiety (Cadenas et al., 2021). Alternatively, increased critical motivation was associated with less anxiety among students at majority Black and Latinx high schools (Christens & Peterson, 2012). Importantly, anxiety was on the rise in college populations prepandemic (Duffy et al., 2019), and has further increased during the COVID‐19 pandemic (Hoyt et al., 2021; Huckins et al., 2020), due to myriad factors, including fears of the virus and social distancing mandates (Son et al.,2020).

It is also important to consider CC's relationship to positive measures of well‐being (Christens et al., 2018; Johnson & Wood, 2017). Experiences with marginalization could impact hopefulness among college youth, but there is less research on this complex relationship (Christens et al., 2018). Youth who experience marginalization and have high levels of CC may be able to better protect their well‐being and foster hope (French et al., 2020). Critical action has been associated with better psychological well‐being among highly sociopolitically engaged youth in Hong Kong (Chan et al., 2020), while critical analysis was associated with high hopefulness for Latinx adolescents (Christens et al., 2018). The COVID‐19 pandemic may affect youth's hopefulness (Huckins et al., 2020), and the relationship between hopefulness and CC in particular. Youth may feel less hopeful and thus cultivate less critical motivation in the face of increasing inequities, higher uncertainty, worse economic prospects, and/or the loss of in‐person social networks that may foster or sustain hope. As we deepen our understanding of college students' functioning during this new COVID‐19 era, researchers must examine well‐being among diverse subgroups of college students to effectively design supports and intervene in response to students' different experiences with marginalizing forces (Holmes et al., 2020).

THE PRESENT STUDY

Our study aims to contribute to the evolving psychometric literature on CC by validating the ShoCCS (Diemer et al., 2020) with socially marginalized college‐age youth for the first time, and to begin extending the study of CC to well‐being outcomes during a time when the COVID‐19 pandemic makes well‐being support to marginalized youth groups paramount. Given the particular circumstances of the study (i.e., that we collected data during a pandemic and with a population largely understudied in the CC literature), we conceive the current study as exploratory and focus on the marginalized subgroups for whom we had statistical power to include within our analyses. The study has two primary aims: (1) examine the extent to which the ShoCCS scale, which was originally validated on younger, ethnically/racially marginalized adolescent youth, is maintained in its factor structure and subscale internal consistency for less commonly studied youth in CC research, including Asian, immigrant‐origin, LGBQ+, and women college emerging adult youth; and (2) investigate the relationships between the three ShoCCS subscales (critical reflection, critical motivation, and critical action) and well‐being (anxiety, hopefulness) during the COVID‐19 pandemic.

METHODS

Study sample

Participants were recruited via Instagram ads that targeted emerging adult college students. From 2887 clicks over a 5‐day period, 1331 unique individuals completed the screening questionnaire, and 1225 met eligibility criteria for the study (i.e., full‐time college students in the United States, ages 18–22, with a verified.edu email). Consented participants completed the survey via Qualtrics in late April 2020 until we reached budget capacity (n = 725). Our final analytic sample was reduced to 707 after validity checks. All study procedures were approved by Fordham University's Institutional Review Board. A detailed description of the social media recruitment methods and study design is described elsewhere (Cohen et al., 2020; Hoyt et al., 2021).

The full sample included 707 full‐time college students (61.0% women; 34.4% men; 4.7% transgender, nonbinary, genderqueer, or other) between the ages of 18–22 (m = 20.0, SD = 1.3) from across 49 U.S. states and Washington D.C., and attending 374 universities (Table 1). The sample was majority White (54.0%); 20.4% identified as Asian/Pacific Islander, 9.1% as Latinx, 5.2% as Black, 1.1% as Middle Eastern/North African, and 10.1% as mixed race/ethnicity. Over one‐third (38%) of participants were of immigrant‐origin, with more second‐generation (i.e., youth born in the United States who have one or both parents who were born outside of the country) (n = 219) than first‐generation (i.e., youth born outside of the United States) (n = 68). The sample's sexual orientation composition included 71.2% heterosexual‐identifying students, while 12.6% identified as bisexual, 6.8% as gay or lesbian, and the remaining 9.4% as another sexual orientation, including asexual, queer, pansexual, or questioning. Participants' SEP were economically diverse, with 33.8% of students from low‐income homes (i.e., household income was <$54,000 per year; 200% of the federal poverty line for a family of 4), 28.7% from middle‐income homes ($54,000–$99,999), and the remaining 37.5% from higher‐income homes (i.e., $100,000 or more). Our sample was reflective and, in some ways, more diverse than the national college undergraduate population (Table 1).

Table 1

Demographic profile of sample (N = 707)

	%	n	National %
Race/ethnicity1
White	54.3	384	52.9%2
Black/African American	5.2	37	15.1%2
Asian/Asian American or Pacific Islander	20.4	144	7.6%2
Middle Eastern/North African	1.1	8	(not available)
Hispanic/Latinx	8.9	63	20.9%2
Mixed race/ethnicity3	10.1	71	(not available)
Gender4
Women	61.0	431	57%5
Men	34.4	243	435
Transgender and gender diverse	4.6	33	–6
Sexual orientation7
Heterosexual or straight	71.2	503	90%6
LGBQ+	28.8	187	10%6
Family's typical annual household income (pre‐COVID‐19)8
Less than $54,000	33.8	279	20%9
$54,000–$99,999	28.2	199	34%
$100,000 and over	37.5	269	46%
Immigrant‐origin10
Immigrant‐origin	38	268	28%11
Nonimmigrant‐origin	62	439	72%11
Disability status
Reported a registered disability	8.9	63	19.4%11
Did not report a registered disability	91.1	664	80.6%11

Batalova and Feldblum (2020).

Race/ethnicity was measured by asking participants to check all the racial/ethnic groups with which they identified, which included a write‐in option; we created five mutually exclusive, comprehensive racial/ethnic groups accordingly (i.e., White, Black, Asian, Latinx, and multiracial). We dropped the small group (n = 8) of Middle Eastern/North African participants from analysis due to small sample size.

Racial/ethnic groups reported for largest ethnic/racial groups in the United States; percentages reflective of all undergraduate students (United States Census Bureau, 2018).

Students who reported two or more racial and/or ethnic groups (e.g., Black/African American and Hispanic/Latinx, Black/African American and White, Hispanic/Latinx and White) were coded as mixed race/ethnicity.

We collapsed gender responses into three categories: man, woman, and TGD. However, due to insufficient statistical power needed for CFA and MI analysis, we excluded TGD (n = 33) from all analyses.

The statistics on male and transgender and gender diverse students are not provided. The 57% female statistic includes all undergraduate students who identified as female. Transgender and gender diverse youth were not included (U.S. Department of Education, National Center for Education Statistics, 2019).

The 10% estimate includes both LGBQ+ and transgender/gender diverse youth (Postsecondary National Policy Institute, 2020).

Sexual orientation was operationalized as a binary variable: straight/heterosexual or LGBQ+.

SEP was measured by estimating the participant's family's typical annual household income pre‐COVID‐19 pandemic; with income groups reduced to three for analysis purposes: <$54,000, $54,000–$99,999 and >$100,000.

The national percentages for undergraduate income levels are based on the income to poverty ratio of dependent students' parents. We align the $54,000 group in our sample to those who are defined as “in poverty”, and the $54,000–$99,999 group to groups defined as “near poverty” and “lower‐middle income”. Students with family incomes above $100 are aligned with “middle income” and “higher income” groups (Pew Research Center, 2019).

Participants reported where they and each of their parents were born; participants were characterized as immigrant‐origin if they and/or at least one birth parent were born outside of the United States (Suárez‐Orozco et al., 2015).

U.S. Department of Education, National Center for Education Statistics, 2019).

Measures

CC: The ShoCCS

The ShoCCS was validated with primarily middle and high school Black and Latinx students who were likely to have experienced marginalization based on their racial/ethnic identity (Diemer et al., 2020). The measure includes 13 items representing three sub‐domains: critical reflection (4 items; e.g., “certain racial or ethnic groups have fewer chances to get good jobs”), critical motivation (4 items; e.g., “it is important to correct social and economic inequality”), and critical action (5 items; e.g., “participated in a civics rights group or organization”). Items use Likert scales ranging from 1 to 6 for critical reflection and motivation (1 = strongly disagree to 6 = strongly agree) and 1–5 for critical action (1 = never did this to 5 = at least once a week). Each subscale is to be independently summed and analyzed. The reliability estimates were all strong in our sample with Cronbach's alphas of reflection = 0.92, motivation = 0.77, and action = 0.84.

Anxiety: The Generalized Anxiety Disorder‐7 (GAD‐7)

The GAD‐7 (Spitzer et al., 2006) uses seven items to assess generalized anxiety disorder. Each item is rated on a 4‐point scale (0 = not at all to 3 = severely, it bothered me a lot). Sample items include “not being able to stop or control worrying”. The total score is generated by adding the results from each item, providing a 0–21 range, with cutoff points of 5, 10, and 15 for mild, moderate, and severe anxiety symptoms, respectively. The scale has been validated with college‐age populations (Duffy et al., 2019) and demonstrated strong internal consistency within a college sample (Cronbach's α = 0.90) (Byrd‐Bredbenner et al., 2020). The reliability estimate was strong in our sample (Cronbach's α = 0.922).

Hopefulness: individual‐differences measure of hope

The individual‐differences measure of hope (Snyder et al., 1991) includes two subscales (agency and pathways) that provide a cumulative hope score. We used eight items (split evenly between the two domains); items are scored on a Likert scale: 1 = definitely false to 8 = definitely true. Sample questions include “I energetically pursue my goals” (agency) and “Even when others get discouraged, I know I can find a way to solve the problem” (pathways). Studies with college students demonstrate acceptable internal consistency (Cronbach's α = 0.89) (Snyder et al., 1991). The reliability estimates were all strong in our sample with Cronbach's alphas of both subscales: agency (α = 0.824) and pathway (α = 0.840), as well as across the full scale (α = 0.891).

Sociodemographic variables

Sociodemographic variables were used to help determine potential differences by group. Gender was examined using a woman/man binary variable due to the limited sample of TGD youth that was insufficient for measurement invariance (MI) analyses. Sexual identity was examined using two groups: heterosexual/straight youth and youth who identify as LGBQ+ (e.g., gay, bisexual youth). Immigrant‐origin status was examined by combining youth who were either born outside of the country or who reported one or both parents born outside of the country (Suárez‐Orozco et al., 2015). We had five main categories of race/ethnicity (Latinx, White, Black, Asian/Asian American and Pacific Islander, and Multiracial and other), but for the purposes of increasing statistical power also did some analyses using just two groups: White participants and participants of color. Middle Eastern/North African participants were included in the multiracial and other group due to the small sample size (n = 8). SEP was measured in three groups: lower income, with an annual prepandemic household income of <$54,000; middle‐income, with an income ≥$54,000 and <$100,000, and higher income of ≥$100,000. Disability status was measured by self‐reported disability.

Analytic strategy

All analyses were conducted in R (RStudio Team, 2020). There were no missing data because the survey used a forced‐choice methodology. After performing descriptive analyses using the psych (Revelle, 2020) package, we conducted confirmatory factor analysis, MI, and latent mean comparisons using the lavaan (Rosseel, 2012) package to determine if the scales accurately captured CC, hopefulness, and anxiety across the sample. The hopefulness and anxiety measures are well‐established; we found that the CFA results for both scales were strong and did not proceed with further analysis of these scales.

Given the skewed and categorical nature of the CC data, and in line with recent CFA analysis of CC subscales (Rapa et al., 2020), we used robust maximum likelihood to perform confirmatory factor analysis and MI analysis (i.e., MLR) (Kline, 2011). The CFA restrained items from loading onto more than one factor to rigorously test underlying factor structure. The model fits were evaluated using the Comparative Fit Index (CFI) and Tucker‐Lewis Index (TLI) with values >0.95 indicating excellent fit and values >0.9 indicating acceptable fit, the root square error of approximation (RMSEA) with values <0.06 indicating excellent fit and values >0.10 suggesting poor fit, and standardized root mean square residual (SRMR) with values >0.8 acceptable (Hu & Bentler, 1999). We then used multiple group confirmatory factor analysis (MGCFA) to test for MI across groups using semTools (Jorgensen et al., 2020). Invariance was determined by a <0.01 change in CFI coupled with a >0.015 change in RMSEA between each step: configural, metric, and scalar (Chen, 2007). When changes in CFI were >0.01, modification indices were examined to identify parameters contributing to noninvariance, and to free these parameters until invariance is achieved (Putnick & Bornstein, 2016). We used CFI and RMSEA changes instead of χ 2 difference tests (Byrne et al., 1989) because χ 2 differences are sensitive to sample size (Oishi, 2007). To maximize statistical power, we first conducted MI between White participants and participants of color (n = 323); we treated MI across the five ethnic/racial groups as exploratory.

Upon confirming invariance, we conducted further exploratory analyses to compare latent mean differences of CC by race/ethnicity (White, Asian, Latinx, Black, and multiracial students), gender, sexual identity and immigrant origin status. Comparisons were made to maximize power by fixing the latent mean of the group with the largest sample size to zero and then freely estimating the latent means for the other group(s) and testing for significant differences. Last, to test for interactions, we changed to the R Stats (R Core Team, 2013) package, where we ran regressions to assess associations between each CC subscale with the outcomes of anxiety and hopefulness, and tested interactions between CC and different group identities (i.e., Asian, Latinx, immigrant‐origin, LGBQ+, women). The shift to manifest variables using regression analyses was reasoned to allow for increased interpretability of demographic moderators (Little et al., 2007) and to align with our exploratory approach wherein we did not approach the CC and well‐being relationship with a specific theory (conversely, the use of latent CC and well‐being variables via structural equation modeling would need to be predicated on clear theoretical relationships between variables, see Cohen et al., 1990). Moreover, we used anxiety and hopefulness measures routinely assessed as manifest variables, thus aligning with the broader study of these phenomena (e.g., anxiety: Ganson et al., 2021; Liu et al., 2021; hopefulness: Bryce et al., 2020; D'Amico Guthrie and Fruiht, 2020) Significant interactions were probed using simple slope analysis (Aiken et al., 1991) one standard deviation at, below and above the mean using emtrends (Lenth, 2018). All regressions controlled for disability status, as ableism acts as a force affecting youth (Fine, 2019), and household income, since lower SEP is associated with increased critical action (Roy et al., 2019). As a sensitivity test, we then reran analyses controlling for other CC subdomains (e.g., controlling for critical motivation and critical action when regressing critical reflection).

RESULTS

Whole sample confirmatory factor analysis

For the ShoCCS subscales, alphas for the full sample ranged from strong to acceptable. A baseline CFA with the full sample was conducted. The model was an adequate fit for the data (n = 707, χ 2[62] =331.388, p< 0 .001; CFI = 0.938, TLI = 0.922, RMSEA = 0.083 (confidence interval: 0.074, 0.092), SRMR = 0.066). The RMSEA and SRMR values met their suggested threshold for good fit; CFI and TLI were acceptable, though below their good fit thresholds of 0.95 (Hu & Bentler, 1999). All items loaded significantly on to their latent constructs (see Table 2). The model is depicted for the full sample in Figure 1.

Table 2

Descriptive statistics and measurement model: confirmatory factor loadings (N = 707)

Latent variable and indicators	Standardized estimate	SE	R 2
Factor 1: Critical analysis (α = 0.92; M = 4.54, SD = 1.22; skew = −0.86, kurtosis = 0.30)
Certain racial or ethnic groups have fewer chances to get good jobs	0.928	0.035	0.861
Certain racial or ethnic groups have fewer chances to get ahead	0.950	0.047	0.903
Women have fewer chances to get ahead	0.835	0.050	0.689
Poor people have fewer chances to get ahead	0.717	0.076	0.514
Factor 2: Critical motivation (α  = 0.77; M = 5.23, SD = 0.76; skew = −1.15, kurtosis = 1.66)
It is important for young people to know what is going on in the world	0.637	0.027	0.406
It is important to correct social and economic inequality	0.667	0.084	0.445
It is my responsibility to get involved and make things better for society	0.742	0.066	0.550
People like me should participate in the political activity and decision making of our country	0.697	0.068	0.486
Factor 3: Critical action (α  = 0.84; M = 1.77, SD = 0.86; skew = 1.24, kurtosis = 0.84)
Participated in a civil rights group or organization	0.795	0.055	0.633
Participated in a political party, club, or organization	0.702	0.080	0.493
Contacted a public official by phone, mail, or email to tell him/her how you felt about a particular social or political issue	0.601	0.050	0.361
Joined in a protest march, political demonstration, or political meeting	0.752	0.036	0.565
Participated in a human rights, gay rights, or women's rights organization or group	0.754	0.068	0.569

Figure 1

Standardized factor loadings among CC subscales and correlations between critical reflection (Rfl), critical motivation (Mtv) and critical action (Act) for total sample (N = 707). Critical reflection, critical motivation, and critical action are the latent domains of critical consciousness and arrows between the circles represent the correlations between latent constructs. The boxes represent the indicators within each domain of critical consciousness. The one way arrows between the latent constructs and indicators represent the factor loadings. CC, critical consciousness

MI by group

First, invariance between White participants and participants of color was strong. The five ethnic‐racial groups (White, Black, Asian, Hispanic/Latinx, multiracial) and immigrant‐origin youth (binary variable) also demonstrated strong MI. For all, the configural invariance was acceptable; metric and scalar invariance changes in CFI and RMSEA were well under their respective cutoffs respectively (Table 3). MI by sexual orientation (straight/heterosexual or LGBQ+) and gender (man or women) was partial. In both tests, the configural test was acceptable and metric invariance was confirmed with minimal CFI and RMSEA changes. However, scalar invariance for sexual orientation and gender were 0.015 and 0.019, respectively, above the 0.01 cutoff. Based on modification indices, one scale intercept (#13; “Participated in a human rights, gay rights, or women's rights organization or group”) was freed for both groups, resulting in partial scalar invariance by sexual orientation and gender (men and women) (Table 3).

Table 3

Measurement invariance

Subgroup	Model	χ2 (df)	CFI	RMSEA	Change in CFI	Change in RMSEA	Decision
Ethnic/racial groups (N of groups = 2)	Configural invariance	471.20 (124)	0.930	0.089			Accept
	Metric invariance	486.13 (134)	0.929	0.086	0.001	0.003	Accept
	Scalar invariance	507.28 (144)	0.925	0.084	0.002	0.002	Accept
Ethnic/racial groups (N of groups = 5)	Configural invariance	726.92 (310)	0.918	0.098			Accept
	Metric invariance	800.01 (350)	0.912	0.095	0.006	0.002	Accept
	Scalar invariance	860.55 (390)	0.908	0.092	0.004	0.003	Accept
Gender (N of groups = 2)	Configural invariance	431.39 (124)	0.932	0.086			Accept
	Metric invariance	459.94 (134)	0.928	0.085	0.004	0.001	Accept
	Scalar invariance	557.59 (144)	0.909	0.092	0.019	0.007	Reject
	Partial scalar invariance	509.64 (143)	0.919	0.087	0.009	0.002	Accept
Sexual identity (N of groups = 2)	Configural invariance	399.79 (124)	0.940	0.079			Accept
	Metric invariance	439.45 (134)	0.934	0.080	0.006	0.001	Accept
	Scalar invariance	518.69 (144)	0.919	0.086	0.015	0.005	Reject
	Partial scalar invariance		0.929	0.080	0.004	0.000	Accept
Immigrant‐origin status (N of groups = 2)	Configural invariance	405.52 (124)	0.941	0.081			Accept
	Metric invariance	421.37 (134)	0.940	0.079	0.001	0.002	Accept
	Scalar invariance	434.81 (144)	0.939	0.076	0.001	0.002	Accept

Abbreviation: RMSEA, root square error of approximation.

Group based differences

When comparing participants of color to White participants, participants of color demonstrated higher mean reflection (z = 2.600, p < 0.01). When we then compared each subgroup within the participants of color grouping, we found that Asian participants demonstrated higher mean reflection (z = 2.014, p < 0.05) and lower mean action (z = −2.520, p < 0.01) than their White counterparts, and Latinx participants demonstrated higher mean motivation (z = 2.03, p < 0.05). Black and multiracial participants demonstrated no significant mean differences from their White counterparts. We found no statistically significant differences between immigrant‐origin youth and their peers in any CC subdomain.

Upon freeing item 13, analyses of latent mean differences revealed significant mean differences for each CC subdomain by gender and sexual orientation. Straight/heterosexual youth had significantly lower mean reflection (z = −6.999, p < 0.001), motivation (z = −4.799, p < 0.001), and action (z = −6.724, p < 0.001) than their LGBQ+ counterparts. Men demonstrated significantly lower mean reflection (z=−5.763 p < 0.001), motivation (z = −4.134, p < 0.001), and action (z = −2.238, p < 0.05) than their women counterparts.

Regression analyses

Separate multiple regression analyses were run for the two well‐being outcomes—anxiety and hopefulness—including the full set of sociodemographic variables and covariates (i.e., family income, immigration status, race/ethnicity, sexual identity, gender and disability). We found no significant interactions between race/ethnicity, gender, sexual identity, immigration status and domains of CC on anxiety, but we did find significant main effects for each: critical reflection and anxiety (b = 1.02, SE = 0.28, p < 0.001), critical motivation and anxiety (b = 0.92, SE = 0.18, p < 0.001), and critical action and anxiety (b = 1.02, SE = 0.25, p < 0.001). Anxiety was positively associated with each CC subscale. As a sensitivity analysis, we also ran the regressions controlling for the other CC subdomains, and significant positive main effects were still identified for critical reflection and anxiety (b = 0.69, SE = 0.21, p < 0.001) and critical action and anxiety (b = 0.81, SE = 0.25, p < 0.01). The relationship between critical motivation and anxiety was no longer significant (b = 0.41, SE = 0.31, p = 0.19).

For hopefulness, when controlling from other CC subdomains, we found a significant interaction between sexual identity and critical reflection (b = 0.23, SE = 0.09, p < 0.05). Simple slope analysis revealed that at high levels of critical reflection, LGBQ+ youth had higher levels of hopefulness (b = 0.31, SE = 0.10, p < 0.01). The interaction was not significant at low or mean levels of critical reflection. There were also significant interactions by race/ethnicity for Asian youth and critical reflection (b = 0.28, SE = 0.09, p < 0.01) and critical action (b = −0.27, SE = 0.13, p < 0.05). Simple slope analysis showed that for Asian youth only, low (b = −0.64, SE = 0.18, p < 0.001) and mean (b = −0.36, SE = 0.14, p < 0.01) levels of critical reflection related to lower hopefulness. Also, for Asian youth only, mean (b = −0.36, SE = 0.14, p < .01) and high (b = −0.58, SE = 0.18, p < 0.001) levels of critical action related to lower hopefulness. Finally, there was a significant interaction for critical reflection by immigrant‐origin status (b = 0.15, SE = 0.07, p < 0.05); however, probing this interaction revealed that neither group's simple slope was statistically significant (see Appendix Tables 1–3 for full regression results).

DISCUSSION

As youth who are experiencing marginalization navigate the unequal mental health effects of the COVID‐19 pandemic (Hoyt et al., 2021), their levels of CC may play an integral role in fostering positive youth development that encompasses hopefulness and work toward social change. Our study sheds light on this relation by contributing five key findings to the fields of CC and well‐being, ultimately demonstrating that relationships between CC and anxiety, and CC and hopefulness, may be differentially associated across and among different marginalized groups. Overall, this study exposes important nuances that can inform intervention and prevention programs to support the well‐being and CC development of college youth from marginalized groups.

First, our findings evaluating the ShoCCS among emerging adults (aged 18–22 years) are the first to suggest that the ShoCCS fits as well with emerging adults as with adolescents, albeit with the same limitations due to non‐normality. The distributions of CC scores were reflective of the original findings from the adolescent sample assessed in the ShoCCS creation (Diemer et al., 2020). While the lower critical action mean in our sample was surprising given the increased opportunities for civic engagement and sociopolitical involvement during emerging adulthood (Flanagan & Levine, 2010), we must also factor in the limitations on in‐person action due to the COVID‐19 pandemic.

Second, in line with calls to perform measurement equivalence testing across subgroups (Godfrey & Burson, 2018; Innamorati et al., 2011; Rapa et al., 2020), we find relatively strong MI across racial/ethnic and immigrant generational groups. This may be because CC has mostly been studied in relation to race and ethnicity (Heberle et al., 2020), and thus extant measures of CC are better poised to capture experiences in this domain. Surprisingly, though prior work suggests that increased experiences with marginalization spark CC development (Mathews et al.,  2019), we did not find significant differences between mean levels of each CC subdomains for some racially marginalized groups (Black and multiracial youth) and immigrant‐origin youth.

We present three theories to explain this potential deviation from CC theory. First, we theorize that the college context may heighten CC development across ethnic/racial groups (Cadenas et al., 2018; Hope et al., 2016). Alternatively, the onset of the COVID‐19 pandemic may have sparked CC development across ethnic/racial groups as societal inequities were brought to the fore. Finally, we theorize that this equivalence may in fact be a result of the CC measure; while CC is celebrated as a civic engagement measure robust against White civic norms, the individual indicators may still be biased to reflect White youth's experiences. For example, critical motivation indicator #8 states “People like me should participate in the political activity and decision making of our country.” White youth may be more likely than their peers of color to affirm this indicator because White people continue to dominate America's political landscape (Delgado & Stefancic, 2017). Meanwhile, the critical reflection measures solely capture understandings of disadvantage. White youth may recognize that other social groups are disadvantaged, yet recognizing disadvantage does not equate to the recognition of privilege and the disavowal of White supremacy (Hagerman (2020), potentially biasing results toward a superficial critical reflection for White youth. We encourage further investigation into the scales to more strongly capture experiences with marginalizing systems.

Meanwhile, the differences we find in CC for Asian and Latinx compared to White youth support CC theory that experiences of racial/ethnic marginalization lead to CC development (Mathews et al., 2019). While CC measures have primarily been designed to address the experiences of racially/ethnically marginalized youth (Diemer et al., 2020; Seider & Graves, 2020), the differences observed across CC subdomains for women, and LGBQ+ students may reflect how experiences of marginalization within one system may inform more broad understandings of structural oppressions and encourage resistance (Burson & Godfrey, 2020).

Fourth, and in contrast, total MI was not achieved based on sexual orientation or gender identity, prompting important considerations for future research using the ShoCCS to compare across gender and sexual orientation. The partial invariance identified both by sexual orientation and gender are important findings in understanding how the ShoCCS critical action subscale may be “easier” to endorse for women and LGBQ+ groups. The prompt may disproportionately provoke affirmative responses among these groups, as it asks about participation in either a “human rights, gay rights, or women's rights organization”; thus, action taken on behalf of certain racial/ethnic groups or other issues (e.g., the environment, eliminating student debt) may not be considered “human rights.” In this case, men's score on item 13 (mean = 1.49) was statistically less at the p < 0.001 level than their women counterparts (mean = 1.97). Even more different (p < 0.001) were the mean score of LGBQ+ youth (mean = 2.63) than their straight counterparts (mean = 1.58). Our findings thus extend the limited field measuring CC by sexual orientation (Wagaman, 2016) and gender (Rapa et al., 2020; Singh et al., 2020). As of now, future researchers who seek to examine group differences by gender or sexual‐orientation using the ShoCCS should consider freeing item 13 before any subsequent analyses if they want to achieve accurate (i.e., equivalent) mean‐based differences (see Bañales et al., 2020 for a recent example of how a lack of MI in CC‐related measures prevented further comparative study among ethnically/racially marginalized emerging adults). Future research should discern how marginalization based on gender and sexuality may produce unique levels and forms of the construct (Singh et al., 2020) and consider ways to reconcile noninvariance.

Ultimately, we sought to extend the study of CC to associations with well‐being. After controlling for sociodemographic variables, each CC subscale was positively associated with anxiety for the sample. Our findings align with recent research that also identifies a positive association between anxiety and political behaviors among some college students (Ballard et al., 2020). However, none of the interactions between CC subscales and sociodemographic groups were statistically significant, limiting our additions to the existing mixed literature on the relationship between anxiety and CC by subgroup (Ballard & Ozer, 2016; Poteat, Godfrey, et al., 2020). Notably, we also found that the relationship between critical motivation and anxiety, and critical motivation and hopefulness, were no longer significant upon controlling for other subdomains, suggesting that the critical motivation scale may require further analysis to determine if it is fully capturing a unique construct. Given that we collected data during the early months of the COVID‐19 pandemic, these findings should be further investigated in other contexts. Meanwhile, there were significant interactions between CC subscales and two of the sociodemographic subgroups (race/ethnicity, sexuality) in models predicting hopefulness.

Critical reflection was associated with more hopefulness in LGBQ+ youth with high levels of critical reflection. For LGBQ+ youth, perceptions of inequality may coexist with feeling hopeful (Poteat, Godfrey, et al., 2020). Poteat, Godfrey, et al. (2020) suggest that hope may support LGBQ+ youth's agency; our findings add to the literature by suggesting that the process of critical reflection may aid in cultivating hopefulness. A potential phenomena we may be capturing is that of critical hopefulness, a construct theorized to capture the experience of youth who are at once critically aware of societal inequalities and hopeful in their ability to enact change (Christens et al., 2013). The findings furthermore build off prior research that demonstrates the ways in which scale development (in this case CC scale development) may be a tool in assessing youth's well‐being (Innamorati et al., 2011).

We recommend that CC continue to be explored as a public health intervention that can support youth experiencing marginalization (Hope & Spencer, 2017), albeit with precautions in mind given the fact that we do find CC associated with anxiety across the sample. Specifically, college service providers, including mental health therapists and campus‐based organizations, may consider embedding a CC lens into their work with marginalized youth. For example, to take the onus off of the individual, clinicians may consider highlighting the structural nature of the challenges youth face, rather than focusing on individual level factors. In line with CC theory, such work should also consider supporting youth in ways to take action against these forces and building their sense of efficacy to do so. Finally, it is of paramount importance that these efforts remain rooted in context and account for the specific strengths and needs of differentially marginalized youth. Indeed, more research is needed to understand the relationship between anxiety and CC across groups.

Meanwhile, that for Asian youth, lower and mean levels of critical reflection are associated with less hopefulness suggests that cultivating further critical reflection may encourage hope. Yet, the picture is complicated by the findings that mean and high levels of critical action are also associated with less hopefulness. These latter findings suggest that engagement in efforts to disrupt marginalizing systems may not foster hopefulness among Asian youth in our sample. Though limited research exists, our findings complicate extant literature that suggests that Chinese youth (in early adolescence) are significantly less likely to be “critical and discontented” and more likely to be “acritical and contented” (Godfrey et al., 2019). Our findings must again be contextualized within a time period where Asian youth were experiencing increases in anti‐Asian discrimination as they confronted COVID‐19 pandemic rhetoric such as “the Chinese virus” and alarming spikes in interpersonal violence (Tessler et al., 2020). Such conditions may reasonably hamper hopefulness given the timing of the survey in April, 2020. In sum, it is not yet clear why associations between CC subdomains and hopefulness are found uniquely among Asian youth in the sample. As Asian youth represent a vast array of distinct ethnic groups and heterogeneity in civic engagement experiences (Wray‐Lake et al., 2017), investigation into Asian youth subgroup differences could further advance our understanding.

Limitations and future directions

Our findings should be interpreted with certain limitations in mind. Our data are cross‐sectional in nature and so we could not assess temporality (i.e., if CC affects well‐being and/or vice versa). Our study sample was not nationally representative, limiting the generalizability of our study. We also had small samples of Black and Latinx students, limiting our understanding of MI across ethnic/racial groups. In comparison, we had enough Asian youth participants to detect statistically significant differences. Similarly, we were statistically powered to look at CC by only two genders: women and men; and thus, future work should purposefully sample for transgender and gender diverse youth. We studied only two well‐being outcomes during a time of heightened well‐being challenges; future research can extend well‐being associations to other indicators and to additional contexts to provide further insights into the psychological implications of CC.

Furthermore, performing intersectional research that examines relationships between well‐being and CC among multiply marginalized youth was beyond our study scope due to limited statistical power. We support recommendations to examine intersections of marginalizing systems (Godfrey & Burson, 2018), particularly as youth may be subject to multiple systems of oppression (or privilege) that shape their well‐being (Ching et al., 2018). For example, despite Asian youth's comparatively lower levels of anxiety, Asian students who identify as sexual minorities experience increased anxiety than their heterosexual peers (Ching et al., 2018). More broadly, we concur that further investigation of how White youth's unique racial privilege informs their CC is needed (Rapa et al., 2020). Moreover, there are additional influences on CC not studied here, like the potential relationship between well‐being and CC for disabled youth.

CONCLUSION

Through this study, we completed analyses that support and complicate the initial validation of a short measure of CC, and draw initial links between CC and well‐being during the onset of a pandemic. Interestingly, the links we identified to hopefulness exist in two understudied populations of youth (LGBQ+ and Asian youth). Our contribution to the study of well‐being has meaningful implications in both research and practice contexts. We encourage practitioners to develop targeted approaches to marginalized youth that help them to both critically reflect on societal inequities and develop efficacy to confront such inequities, while also remaining sensitive to well‐being ramifications. Such work is only increasing in importance as issues of well‐being continue to rise through the COVID‐19 pandemic.

PEER REVIEW

The peer review history for this article is available at https://publons.com/publon/10.1002/jcop.22678

CONFLICT OF INTERESTS

The authors declare that there are no conflict of interests.

Supplementary information.

Click here for additional data file.