Development and validation of the 4-Factor Critical Consciousness Scale.

Assessment of critical consciousness among individuals can provide a proxy measure of the readiness of communities, and individual decision-makers within, for social changes that address root causes of ill health. Critical consciousness, as conceived by Paolo Freire, emerges as a consequence of praxis. This iterative, recursive process of reflection and co-created knowledge enables community members to identify salient issues and the actions they want to take to address those issues. Public health and other social science researchers who engage in social- and population-level intervention work need a validated instrument that measures critical consciousness. Our purpose was to develop an instrument that can measure 4 key constructs of critical consciousness (passive adaptation, emotional engagement, cognitive awakening, and intentions to act) in an individual, relative to any salient community issue. We conducted two studies (Initial: June 2018; Retest: October 2019) to develop and validate this instrument. The same sampling strategy was used for both studies, but each study was conducted with a discrete cohort of participants. We used Amazon's Mechanical Turk to recruit and incentivize study participants. Data from the Initial study were used in an iterative process to evaluate construct validity and test our theoretical assumptions. Exploratory factor analyses were used to determine the best model fit that gave the greatest subscale reliability and validity. In the Retest study, confirmatory factor analysis was conducted and construct validity was verified. Our results indicated adequate construct validity as evidenced by good model fit. Additionally, the good fit of the data to the 4-factor structure confirmed our theoretical understanding of critical consciousness.

Introduction

Since the 1980s, public health has attempted to shift thinking about health from an over-emphasis on individual responsibility for health behaviors and outcomes to an appreciation of embedded social and environmental factors that influence health (McLeroy et al., 1988). Public health recognizes that structures which support or hinder human behaviors need to be addressed at a community level as necessary precursors to community change, and emphasizes the importance of mediating factors and social determinants that are fundamental to public and population health. According to Healthy People 2020 (2017), social determinants encompass five key areas: health and healthcare, education, economic stability, social and community context, and neighborhood and built environment. Community-led efforts targeting policies, systems, and social-ecological factors, as exemplified in community-based participatory research (CBPR), are more likely to be effective than those that target only individual biology, motivations, or attitudes (Glanz & Bishop, 2010). Consequently, public health and social science researchers who engage in social- and population-level work need a way to objectively measure the effectiveness and impact of their interventions at socio-ecological levels beyond the individual.

Background

A social change approach that emphasized examining and addressing the root causes of social problems was popularized by Paulo Freire, a Brazilian educator and philosopher who taught adult literacy using participatory methods (Freire, 2000, 2005). While addressing issues of community concern, he realized that our level of consciousness influences how we interpret reality and act. At the lowest level of consciousness, people passively accept the status quo and their helpless attitude perpetuates their oppression by those they perceive to be superior or more powerful. Freire's approach used a process of problem-posing “liberating dialogue” to help small groups of oppressed people engage in a thoughtful examination of their collective experience. From group discussions, awareness of common problems and shared concerns emerged, followed by group reflection on and identification of underlying causes and possible solutions. Collectively, they determined their best options, and took responsibility for actions that would change the status quo, thus engaging in “praxis: reflection and action upon the world in order to transform it” (Freire, 2000, p. 51). Through this organic process, individuals eventually shifted to the highest level—that of critical consciousness (CC; Freire, 2000; 2005). Similar to achieving critical mass in a chain reaction, achieving CC results in a self-sustaining process of social change that simultaneously emerges from and enables ongoing community transformation. This approach, routinely integrated into CBPR, has become an accepted method to produce social change to address root causes of ill health and health disparities (Minkler & Wallerstein, 2003).

Freire's brilliance was that he developed a repeatable process that moved people to a state of CC, first by listening to their stories, then by emotional engagement around the issues that were most important to them in their daily lives, and then by a process of co-created learning through ongoing discussion, reflection, and action (Freire, 2000, 2005). Participating in the decision-making process helps people realize that they have the power to bring about change (Minckas et al., 2020). This recursive process of discussion, reflection, and action has been successfully applied in social work education and practice, worker health and safety education, work with abused women, homelessness (Carroll & Minkler, 2000), health education, community development, youth programs, college courses, and literacy programs (Wallerstein & Sanchez-Merki, 1994). Whether an individual is a member of a privileged or oppressed group, development of CC can “help individuals to understand their role in a system of oppression” (Thomas et al., 2014, p. 487).

What is critical consciousness?

Carlson et al. (2006) stratified the evolution of CC into distinct “hierarchical levels of cognitive-emotional interpretations that moved participants from passive adaptation to … emotional engagement, cognitive awakening, and intentions to act” (p. 843). As Freirean concepts are applied, the process of praxis generates a community's ability to perceive the effects of social injustice and structural inequality, their increased agency for change, and their sense of empowerment to take action, which, taken together, are evidence that CC has developed within the members of that community. In other words, CC emerges in the individuals and key decision-makers of a community as a consequence of praxis. It is this iterative, recursive process of reflection and collective co-created knowledge that enables community members to identify issues that are important to them and to feel able and compelled to take action to address those issues. Critical consciousness is, therefore, neither reflection nor action, but the dynamic state of being that connects the two.

How is critical consciousness being measured?

Many researchers have used qualitative methods (Carlson et al., 2006; Minkler et al., 2006; Minkler & Wallerstein, 2003; Wallerstein & Sanchez-Merki, 1994; Wang & Burris, 1994), but fewer researchers have attempted to measure CC quantitatively, and researchers have not achieved consensus about the components of CC (Diemer et al., 2016; Jemal, 2017). There have been efforts to develop validated instruments specifically intended to measure CC based on oppression and liberation theory, issues of societal equity, and sociopolitical development (Thomas et al., 2014). Watts, Williams, and Jagers (2003) described the growth of CC as a process of “knowledge, analytical skills, emotional faculties, and capacity for action in political and social systems” (p. 185). Watts, Diemer, and Voight (2011) characterized CC as consisting of critical social analysis, political efficacy, and participation in political or civic action, whereas Diemer et al. (2015) and Seider et al. (2020) described CC as critical reflection, critical motivation, and critical action. Diemer et al. (2017) further characterized critical reflection as “a critical analysis of societal inequalities or endorsement of egalitarian ideologies” (p. 477).

These instruments are all based on a theoretical formulation of CC that is fundamentally different from Carlson et al.’s (2006) interpretation of CC which is used by many Freirean scholars in the public health field. Although they assess CC using one, two, or three subscales, a recent analysis noted that all existing instruments had failed to incorporate all dimensions or sub-components of CC posited by their developers (Jemal, 2017). More importantly, many measures of CC (Diemer et al., 2017, Thomas et al., 2014; McWhirter & McWhirter, 2016) were developed for specific adolescent or young adult populations with certain sociopolitical and demographic characteristics and therefore are not suitable for communities which include older adults or different population characteristics.

An additional instrument developed by Shin et al. (2016) is suitable for a broad age demographic, but like the other instruments, it was developed and validated with a focus on specific sociopolitical topics selected by the developers. None of these instruments measured the development of CC relative to actions within communities around issues that were self-identified by the community. Moreover, these instruments appear to be measuring the result of CC—the ability to perceive societal oppression and take action—rather than the process of CC development.

Critical consciousness and empowerment

Zimmerman and Rappaport (1988) reported a positive correlation between psychological empowerment and citizen participation. Watts, Diemer, and Voight (2011) related both CC and sociopolitical theory to psychological empowerment, describing CC as “having cognitive, emotional, and behavioral dimensions, which researchers use to assess an individual's capacity to engage in change activities and the extent to which they actually do so” (p. 51). Freire's (2000, 2005) problem-solving approach “has become almost synonymous with the philosophy of empowerment and participation in public health and community development” (Carlson et al., 2006, p. 838). Praxis is essential to a community's engagement throughout the process, from identifying common problems to recognizing their root causes and finding collective solutions. Individual and collective empowerment develop in parallel with the evolution of CC (Freire, 2000, 2005) and both are evidenced by effective social justice action and sustainable changes that improve our social environment, behavioral choices, and health outcomes.

Why develop another critical consciousness instrument?

Scholars and researchers have successfully used Freirean methodology in a variety of situations and settings: Community health concerns (Carlson et al., 2006), housing advocacy, inner-city single motherhood (Carroll & Minkler, 2000), community organizing (Minkler, Frantz, & Wechsler, 2006), youth alcohol and substance abuse prevention (Wallerstein & Sanchez-Merki, 1994), and women's reproductive health (Wang & Burris, 1994). However, without an appropriate validated instrument to measure CC as it relates to broader social issues and health concerns, it has been difficult for researchers who use Freirean methodology in CBPR to quantitatively measure the efficacy or impact of their interventions on changes in CC at the individual level or to quantify how much change in consciousness occurred as a result of a particular intervention.

The narrower and targeted lens of existing instruments does not meet the broader needs for an instrument that is versatile enough to measure the extent of individual CC for any intervention which involves community engagement in identifying salient issues, generating solutions, and advocating for change. The ideal instrument would also be able to indicate directionality and quantify how much change occurred for each component of CC as a result of a particular intervention. This article reports on the development of an instrument that measures the four constructs of cognitive-emotional interpretations that comprise CC. Findings reported in this paper were part of a larger evaluation project that assessed the impact of an intervention on the CC of participants.

Key constructs for measuring critical consciousness

Our purpose with the research reported here was to develop an instrument based on Carlson et al.’s (2006) four constructs of CC described below, that can be used by researchers with individuals and communities, relative to any community-identified issue:

Passive Adaptation. Awareness of an issue may or may not be present but cognitive attention to its salience for current life is too weak to motivate giving the issue attention over current life concerns. (Non-awareness certainly allows for passive adaptation, but awareness can also be present and still allow for passive adaptation.)

Emotional Engagement. A visceral and emotional reaction coming from new stimuli (typically experienced visually or verbally) that challenges previously static thinking.

Cognitive Awakening. A visceral emotional reaction gives way to questioning one's current understanding of the issues' salience, implications, and their root causes. Assessment of action and/or inaction is weighed with a cost/benefit calculus, increasing motivation to learn more about an issue.

Intentions to Act. A decision to act is made, based on awareness of the impact of an issue and dissatisfaction with the status quo. Targets for change (likely both accurate and inaccurate) are identified, and commitment to take action is greater than daily life concerns.

Materials and methods

Our Institutional Review Board exempted this study from full review because our purpose was to develop and validate an instrument. We conducted two studies: In the Initial study, a preliminary 37-item scale was developed and then reduced using analysis of subscale reliabilities and model fitting to a final 9-item scale, which was validated in the Retest study. The research design was cross-sectional and involved survey data collection, but without experimental manipulation or observation of participants. The same sampling strategy was used for both studies, but each study was conducted with a discrete cohort of participants.

Sampling strategy

We used Amazon's Mechanical Turk (“MTurk”) to recruit and incentivize study participants. MTurk is an online marketplace for work that requires human intelligence, such as psychological or social science research. Over the last decade, MTurk has become popular as a means of quickly recruiting participants for studies in the health, behavioral, and social sciences, and as a way to obtain high-quality data inexpensively (Buhrmester et al., 2011; Shank, 2016). Researchers can easily set qualification criteria, and rapidly complete surveys and other data collection with robust sample sizes.

The quality and reliability of MTurk results have been evaluated and compared to studies using traditional sampling methods; data obtained through MTurk samples are considered trustworthy (Buhrmester et al., 2011; Goodman et al., 2013; Lund et al., 2018). MTurk samples were also found to be more representative of the general population than traditional student samples (Buhrmester et al., 2011; Goodman et al., 2013) with “a relatively diverse group in terms of age, race, social class, sexual orientation, and political affiliation [and] a very similar composition to the general U.S. population in terms of gender, ethnicity, income, and education level” (Shin et al., 2016). The rural/urban geographic distribution and occupations of MTurk workers (“MTurkers”) are comparable to those obtained in professional polling samples, and characteristics of MTurkers are similar to those on other online survey platforms (Huff & Tingley, 2015).

Sampling method used in both studies

Inclusion and Exclusion of Participants. We only recruited participants on the MTurk platform, so anyone who was not an MTurker was excluded from participation by default. In addition to the requirement that study participants must be at least 18 years old, we also established four inclusion criteria for MTurkers. Items in our instrument assumed participants had some familiarity with health, behavioral, and social issues or concerns in the US. Therefore, we required participants to be located in the U.S. and be U.S. high school graduates. We also required MTurkers to have completed at least 500 MTurk assignments (“jobs”) with at least a 95% overall job approval rate to ensure that participants had previously demonstrated the ability to meet quality standards. We paid above-market rate and MTurkers received an additional bonus payment if they met our pre-established quality benchmarks.

Data Collection. We used Qualtrics' survey platform to create an electronic survey for each study, and MTurk's “Survey Link” template to make them available to MTurkers. For each item, we used a 5-point Likert-type scale ranging from 1 (strongly disagree) to 5 (strongly agree). Participants were asked to respond to each item but were not required to respond to every item. Our sampling strategy and incentives for survey completion resulted in no missing data.

Sampling Procedures. To compensate for our Eastern time zone location and to increase the probability of obtaining the most diverse sample possible from throughout the US, we published three batches of jobs in MTurk at various times over two days in June 2018 (Initial study) and July 2019 (Retest study). Workers self-selected into the Initial study by accepting our job from their task list. Because it is not known how many eligible MTurkers saw our job but did not accept it, we were unable to calculate a response rate for either study. The same qualification criteria were used for both studies; however, workers who had participated in the Initial study were not eligible to participate in the Retest. Retest benchmarks and quality controls were identical to those used in the Initial study. After each batch of surveys were submitted in Qualtrics, quality control items in the data were analyzed. Surveys were rejected if quality controls were not met.

Sample Size. There is no consensus about determining adequate sample size for factor analysis. Moreover, recommendations are often contradictory (MacCallum et al., 1999). A recent review concluded that 40%–60% of exploratory factor analyses (EFA) have 200 or fewer participants (McNeish, 2017). Traditional power analysis suggested a sample of size 200 or more was needed to detect an effect of size d = 0.20 with power 1 – β = 0.80 and α = 0.05. Hu and Bentler (1999) demonstrated that type I and type II errors were minimal when sample size was at least 250. Our intended final sample size was 250 or more for both studies; our achieved sample sizes were 249 (Initial) and 315 (Retest).

Initial study: model development, exploratory factor analysis, and model fitting

The objective of the Initial study was to develop an instrument, use it to collect a data sample, conduct EFA on the data, and investigate the best model fit and factor structure for our theoretical assumptions that gave the best subscale reliabilities.

Item Generation and Instrumentation. We operationalized key constructs of CC with statements that were intentionally worded using the generic phrase “problem/concern” so that the instrument could be used without modifying the wording to measure the CC of participants by directing them beforehand to focus on a particular issue. We drew from the foundational literature of Freire (2000, 2005) and the seminal works of Carlson et al. (2006), Minkler & Wallerstein (2003), and Wallerstein & Sanchez-Merki (1994), and from our own extensive experience to generate the initial items. We conducted a content analysis of Carlson et al. (2006) and pilot-tested items over a dozen years during research studies conducted by the authors and our colleagues related to various health issues in the US (Herget et al., 2015; Ewald et al., 2019), Ireland, and Kenya (Corneli et al., 2021). The draft version of the scale consisted of 39 newly written statements grouped in subscales for each of four constructs. In preparation for the Initial study, items were revised as a result of multiple rounds of expert review by public health and education researchers/practitioners and a psychometrician regarding content, clarity, and parsimony. Based on this feedback, we split one question into two, eliminated three questions, and made other minor edits to produce the preliminary scale, which consisted of 37 items, as shown in Appendix A.

Quality of Measurements. During data collection, we used several methods to enhance the quality of measurements. Interspersed among the 37 statements were two “attention check” items (i.e., “for this item, answer this way”). Demographic questions included participant age, gender, ethnic background, racial identity, and highest completed level of education. Additional quality controls included two qualification check items (a response option of “less than high school” for the prompt “What is your highest completed level of education?” and a continuous variable data entry field for the prompt “How old were you on your last birthday?”, minimum length of time spent on each page, and minimum total completion time for the entire survey. When MTurkers were paid for one job, they were coded as ineligible for all other jobs.

Participant Characteristics. Participants’ average age was 39.5 years (SD = 11.6 years), the sample was fairly evenly split between females (52.2%) and males (47.4%), and was predominantly White or Caucasian (75.0%). More than 90% of participants had attended college, with 41.0% indicating a 4-year degree was the highest level of education completed.

Data Diagnostics. Data were analyzed for outliers, but no extreme values were identified. Two survey items were reverse coded to align directionality of responses. Responses to survey items in each subscale were averaged to derive composite scores for each subscale. Data were not normally distributed, but skewness and kurtosis were not severe enough to warrant data transformation.

Psychometrics and Analytic Strategy. We used descriptive statistics to assess data suitability for factor analysis, and assessed internal consistency of subscales using reliability coefficient alpha; a value < 0.5 is unacceptable; 0.5 to < 0.6 is poor; 0.6 to < 0.7 is acceptable; 0.7 to < 0.9 is good, and ≥ 0.9 is considered excellent reliability (Streiner, 2003, p. 102). We conducted EFA using Maximum Likelihood (ML) extraction and varimax rotation (Costello & Osborne, 2005) and the most common indices for root-mean-square error of approximation (RMSEA), Tucker-Lewis index (TLI), comparative fit index (CFI), and standardized root-mean-square residual (SRMR). A good model fit would yield RMSEA ≤ 0.06, TLI ≥ 0.95, CFI ≥ 0.95, and SRMR ≤ 0.05 (Hu & Bentler, 1999). We used an iterative process to evaluate construct validity, determine which items gave the best reliability coefficient for each subcale, test our theoretical assumptions, and determine the best model fit that aligned with our theoretical model. Based on the results of multiple reliability and factor analyses, we reduced the preliminary 37-item scale to a 9-item final scale with four constructs. The resulting 4-Factor Critical Consciousness Scale (4-FCCS; Strack et al., 2018) is shown in Appendix B.

Retest study: validation, confirmatory factor analysis, and model fitting

The final scale received scores of 74.4 for Flesch Reading Ease and 6.4 for Flesch-Kincaid Grade Level, similar to the preliminay scale (which had scores of 67.0 and 7.1, respectively), ensuring that the instrument would be appropriate for use by both adolescent or adult participants.

Participant Characteristics. Participants’ average age was 38.2 years (SD = 11.3 years), the sample was fairly evenly split between females (54.3%) and males (45.1%), and was predominantly White or Caucasian (74.6%). Almost 90% of participants had attended college, with 41.6% indicating a 4-year degree was the highest level of education completed.

Psychometrics and Analytic Strategy. We assessed construct validity and internal consistency of subscales using reliability coefficient alpha with the same reliability indicators as previously described. We conducted confirmatory factor analysis (CFA) using ML extraction and varimax rotation (Costello & Osborne, 2005) to assess the theoretical model and demonstrate discriminant validity, and used the same model fit indices as previously described.

Results

For both studies, we used SPSS Version 26 for data transformation and analyses, and AMOS Version 26 for factor analyses.

Results from the initial study

Statistics and Data Analysis. There were no missing data. Examination of the interquartile range confirmed no extreme values that would be considered outliers; therefore, no participants were eliminated. Responses to subscale survey items were averaged to compute composite scores for each subscale. No data transformation was required. As shown in Table 1, responses were within approximately one standard deviation of subscale means; the largest standard deviation was for Emotional Engagement (1.07) and the smallest was for Cognitive Awakening (0.84). The subscale means ranged from 1.93 (somewhat disagree) for Passive Adaptation to 4.07 (somewhat agree) for Cognitive Awakening.

Table 1

Psychometric properties for subscales.

Subscale	M	SD	Range of Subscale Means
Passive Adaptation	1.93	.87	1.76, 2.65
Emotional Engagement	2.92	1.07	2.48, 4.04
Cognitive Awakening	4.07	.84	2.33, 4.39
Intentions to Act	3.79	1.03	2.90, 4.17

Note. N = 249. Minimum and maximum values for each item ranged from 1.00 to 5.00, respectively.

Reliability. We assessed internal consistency of the subscales using reliability coefficient alpha (Streiner, 2003). To achieve unidirectional values for each subscale, four items in the preliminary scale were reverse coded. Appendix A shows EFA factor loadings for all 37 items. We eliminated 11 items that loaded below 0.40 to all factors and an additional 12 that cross-loaded significantly to more than one factor and/or explained less than 50% of variance. The remaining 14 items were individually evaluated using a step-wise elimination/replacement approach, until each subscale achieved the highest reliability coefficients that explained the greatest amount of variance. As shown in Table 2, coefficient alpha values for the final subscales ranged from 0.65 to 0.84, which are considered acceptable to good. Our process produced evidence of face and content validity.

Table 2

Internal consistency of subscales using coefficient alpha.

Subscale	Preliminary			Final
	Reverse Coded Items a	Item Count b	Coefficient α	Item Count c	Coefficient α
Passive Adaptation	2	9	.785	2	.650
Emotional Engagement	0	8	.753	2	.681
Cognitive Awakening	2	12	.785	3	.747
Intentions to Act	0	8	.884	2	.844

Four items in the preliminary scale were reverse coded to achieve unidirectional values for each subscale.

The preliminary scale contained 37 items.

The final scale contained 9 items.

Model Fitting. The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.822, which is considered very good (Hair et al., 2006). Bartlett's test of sphericity was significant (χ2(36) = 818.731, p < .001), indicating the data were suitable for factor analysis. Following EFA evaluation of factor structure, which demonstrated discriminant validity, we assessed the theoretical model using the most common indices: RMSEA, TLI, CFI, and SRMR (Hu & Bentler, 1999). Our results indicated construct validity as evidenced by good model fit, with RMSEA < 0.001 (90% CI = [0.000, 0.039]), TLI = 1.01, CFI = 1.00, and SRMR = 0.021.

Results from the retest study

Reliability. As shown in Table 3, coefficient alpha ranged from 0.675 to 0.923, which is considered acceptable to excellent internal consistency. One item in the final scale required reverse coding to align directionality.

Table 3

Internal consistency and factor analysis for each of the four subscales.

Subscale	Reverse Coded Items a	Item Count	Coefficient α	Factor b
Passive Adaptation	1	2	.675	1 (negative)
Emotional Engagement	0	2	.817	3
Cognitive Awakening	0	3	.793	1 (positive)
Intentions to Act	0	2	.923	2

One item in the final scale was reverse coded to achieve unidirectional values for each subscale.

Items that assess Passive Adaptation and Cognitive Awakening loaded to one factor; factor loadings for Passive Adaptation were negative whereas those for Cognitive Awakening were positive. Because these are two distinct subscales, during EFA and CFA, these factor loadings were analyzed as if they were two factors.

Model Fitting. Following CFA, the model of best fit included only three factors, but items from two subscales loaded in opposite directions to one factor, which was consistent with our four-factor theoretical model of CC. Standard errors, regression weights, and correlations are shown in Fig. 1. Model fitting again indicated construct validity as evidenced by good model fit, with RMSEA = 0.057 (90% CI = [0.031, 0.082]), TLI = 0.973, CFI = 0.984, and SRMR = 0.030.

Fig. 1

The “best fit” factor structure with values from the retest study

Note. The “best fit” 9-item factor structure, with standard errors, regression weights, and correlations indicated good model fit and evidence of construct validity. Values shown are from Retest study data.

Comparison of 4-FCCS with other measures of critical consciousness

We used the same rigorous standards to assess model fit as most other published CC instruments. A model that explains a good amount of covariance and is parsimonious will have RMSEA < 0.06; one that explains a good amount of variance compared to a null model will have TLI and CFI > 0.95; and one that leaves little covariance unexplained will have SRMR < 0.08 (Hu & Bentler, 1999). As shown in Table 4, our results are better than all of these cutoffs, indicating a good model fit.

Table 4

Comparison of fit indices for initial models of critical consciousness.

Model	RMSEA		TLI(>.95)	CFI(>.95)	WRMR(>1.00)	SRMR(<.05)
	Value(<.06)	Confidence Interval
4-FCCS	.057	90% CI = [.031, .082]	.973	.984		.030
CCS a	.070	90% CI = [.06, .08]	.96	.96	1.04
CCCM b	.077			.972		.074
MACC c	.064			.962		.032

Note. RMSEA = root-mean-square error of approximation; CI = Confidence Interval; TLI = Tucker-Lewis index; CFI = comparative fit index; WRMR = weighted root-mean-square residual; SRMR = standardized root-mean-square residual.

Critical Consciousness Scale (Diemer et al., 2017).

Contemporary Critical Consciousness Measure (Shin et al., 2016).

Measure of Adolescent Critical Consciousness (McWhirter & McWhirter, 2016).

Except for the RMSEA confidence interval, our model fit statistics for the 4-FCCS exceeded statistics reported during initial validations of those instruments that used the same methods, as shown in Table 4. Diemer et al.’s (2017) narrower confidence interval reflects his much less diverse study population: Black, underprivileged teenagers between the ages of 13 and 19.

Discussion

The purpose of this study was to develop and validate an instrument that can be used by public health and social science researchers to assess key constructs of CC, from passive adaptation to intentions to act, relative to any salient community-identified issue. Our EFA did not identify four discrete factors as expected. Although items in the Emotional Engagement and Intentions to Act subscales loaded to separate factors, items in the Passive Adaptation and Cognitive Awakening subscales loaded in opposite directions to one factor. We concluded that these findings are entirely consistent with our theoretical model of CC: As Cognitive Awakening increases, we would expect a parallel decrease in Passive Adaptation. Therefore, we proceeded to assess model fit using a four-factor structure.

The findings of our EFA and CFA indicated that the 4-FCCS had acceptable to good reliability and excellent construct validity. Moreover, assessments of the internal consistency of the subscales were equivalent and model fit statistics exceeded those reported during initial validations of the CCS, CCCM, and MACC. The good fit of our data to the 4-factor structure confirms our theoretical understanding of CC proposed by Carlson and colleagues (2006). Additionally, the 4-FCCS reflects all components of our theoretical model, whereas other instruments have not, as discussed by Jemal (2017).

Four CC instruments were developed for and are thus limited to use with specific populations (Diemer et al., 2017; McWhirter & McWhirter, 2016; Shin et al., 2016; Thomas et al., 2014). Because these instruments all measure CC in terms of certain sociopolitical activities or awareness of specific types of societal oppression, they are not appropriate or useful for research about general health-related topics or social determinants affecting health. In contrast, the 4-FCCS was developed for general populations, validated using a diverse group of participants, and designed to assess elements of CC relative to any health condition, neighborhood issue, or societal problem. For these reasons, the 4-FCCS instrument can be used across larger populations and target a broader variety of issues.

Strengths and limitations

This is the first investigation of the 4-FCCS; therefore, the originality of this research is both a strength and a limitation. Although this study is innovative, findings cannot be supported by previous research. One of the strengths of the 4-FCCS is that it was intentionally designed to measure the constructs of CC itself relative to the mediating factors and social determinants that underlie disparities in individual and population helath, irrespective of an individual's marginalized or socioeconomic status (Diemer et al., 2016; McWhirter and McWhirter, 2016; Shin et al., 2016), sociopolitical development (Thomas et al., 2014; Watts, Williams, & Jagers, 2003), community organizing efforts, sociopolitical activism, or civic engagement (Diemer et al., 2017). Another strength is that development of the 4-FCCS was not limited to assessing CC in underprivileged or disadvantaged populations. It can therefore be used to measure CC in those who are marginalized, as well as in those with privilege “who have different levels of access to resources and power and can serve as allies for the oppressed” (Thomas et al., 2014, p. 488).

A limitation of these studies relates to the self-reported nature of the instruments, which potentially elicited socially desirable responses. We minimized the potential negative impact of this by informing participants beforehand that the purpose of the studies was instrument development and their responses would be confidential. Additionally, the studies were conducted using convenience rather than random sampling, and the data are cross-sectional, not longitudinal, so no causal inferences should be drawn. Additional research using more rigorous sampling strategies involving a variety of populations is necessary.

Implications for research

This instrument addresses the need for measuring the effectiveness of interventions, and provides public health and social science researchers with the ability to quantitatively measure CC relative to a wide variety of research topics. Researchers using Freirean methodology and praxis now have a validated instrument that can assess the extent of CC within a community by measuring CC of individual decision-makers and stakeholders in that community, yielding greater statistical power than traditional community-level measures to draw accurate conclusions about a population using sample data. The measurement of support for and intentions to act to address root causes of ill-health allows for a proxy measure of movement within a community prior to the adoption of a policy or systems change and is not limited by a delayed measure of changes within individuals of the community.

Additional research using the 4-FCCS could improve our understanding of CC as a phenomenon. In short, it is unknown if CC is problem/concern-specific or universal, a stable trait or temporary state. Future studies that use the 4-FCCS to assess changes of CC would provide evidence of its malleability. Ideally, such an instrument would not only measure the degree of CC but would also be able to indicate directionality and quantify how much change occurred as a result of a particular intervention. Such quantitative assessments could also provide researchers with empirical data to inform improvement of evidence-based practice.

Implications for practice

Like the CCCM, the 4-FCCS was developed and validated using participants recruited through MTurk. Findings of this research paper confirm the results of other studies that MTurk participants are at least as diverse as community and academic samples, and similar to the general U.S. population. Public health researchers and practitioners ought to consider this very convenient and low-cost sampling strategy for other research and evaluation projects.

Further research investigating the various components of CC within a community before implementing intervention activities may help researchers determine the most appropriate types of interventions for a given level of CC. Pre-intervention subscale values may imply targets for health promotion or intervention activities that are ideally suited for the specific community. Using the instrument as a pretest/posttest will also allow researchers to assess the effectiveness of an intervention. This is especially helpful in assessing efforts to shift community support for addressing root causes of social problems, as well as the degree to which support for taking action is present.

Most importantly, this instrument allows public health and social science researchers to investigate CC in individuals and communities. As Freire (2000) and Carlson et al. (2006) conceptualized it, CC has more to do with a person's internal state of being which mediates actions than their outer behavior. The 4-FCCS focus on CC as an internal state is unique and enables researchers to assess the readiness of individuals and communities for the process of social change.

Conclusion

Clinical research, although well-intentioned, has often failed to appreciate the profound influence that shifts in our social and ecological environments can have on our collective health. The ability of CBPR to motivate community actions leading toward healthier environments is largely dependent on leveraging its potential to shift participants and community members from a state of passive adaptation about a particular issue to a level of community awareness that leads to actions. The 4-FCCS instrument differs from individual-level behavior change evaluation measures (e.g., knowledge gains, behavioral shifts) by focusing on social-environmental influences surrounding individuals, such as stakeholder support to take action to address determinants of—and actual changes to—policies, systems and environments.

The more community members, stakeholders, and decision makers develop CC, the more likely they are to embrace and support social change efforts. Movement among many individuals' CC within a community can provide a proxy measure of the collective impact of an intervention or the community's movement toward change. Although individual-level impacts from public health interventions are considered longer-term outcomes, the 4-FCCS enables the measurement of short-term outcomes, such as shifts in increased awareness of and intentions to act on community-identified issues. In conclusion, the 4-FCCS can be used by public health and social science researchers with individuals and communities to investigate any salient community issue.

Author contributions

Muhsin Michael Orsini: Conceptualization, Methodology, Validation, Formal Analysis, Investigation, Data Curation, Writing – Original Draft, Writing – Review & Editing, Visualization, Project Administration.

D. Rose Ewald: Methodology, Validation, Formal Analysis, Investigation, Data Curation, Writing – Original Draft, Writing – Review & Editing, Visualization, Project Administration.

Robert W. Strack: Conceptualization, Validation, Writing – Review & Editing, Supervision, Project Administration, Funding acquisition.

Funding

This work was supported by the National Institutes of Health [grant number R41NR015736].

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Robert W. Strack has copyrighted the 4-Factor Critical Consciousness Scale to Robert Werner Strack, Muhsin Michael Orsini, D. Rose Ewald.

Preliminary scale showing EFA factor loadings for Eeach subscale.

Factor name and items	EFA factor loadings							CFA factor loadings
	1	2	3	4	5	6	7	1	2	3
Passive Adaptation
I have no idea why the problem/concern exists.	−0.534	−0.239	−0.002	−0.045	0.218	−0.024	0.079	−0.380	0.043	−0.246
I know about the problem/concern but I have too many other concerns in my life right now to worry about it.	−0.258	−0.255	−0.144	0.057	−0.075	0.354	−0.002	−0.296	−0.155	−0.263
There doesn't seem to be much we can do about the problem/concern.	−0.500	−0.431	0.019	−0.028	0.076	0.105	−0.114	−0.437	0.008	−0.471
The problem/concern has always been an issue in our community. (R)	0.000	−0.033	−0.156	−0.015	−0.109	−0.020	−0.140	−0.001	−0.194	−0.054
The problem/concern is an issue for certain people and it is their problem.	−0.453	−0.095	0.000	−0.315	0.035	0.428	−0.019	−0.667	0.011	−0.136
The problem/concern is an issue we should all care about. (R)	−0.588	−0.137	−0.241	−0.146	−0.105	0.227	−0.063	−0.611	−0.287	−0.176
Because the problem/concern impacts other people, it does not concern me.	−0.683	−0.168	−0.165	−0.181	−0.026	0.460	0.167	−0.735	−0.185	−0.192
I am not concerned about the problem/concern.	−0.640	−0.114	−0.230	−0.115	0.073	0.347	0.074	−0.668	−0.201	−0.136
I blame other people for the problem/concern.	−0.201	−0.110	0.009	−0.136	0.034	0.552	−0.181	−0.438	0.032	−0.170
Emotional Engagement
Learning about the problem/concern makes me emotional (e.g., sad, angry, helpless).	0.133	0.047	0.733	0.091	0.115	−0.056	−0.075	0.154	0.721	−0.015
I feel helpless about the problem/concern.	−0.001	−0.310	0.533	0.078	0.149	0.205	−0.123	−0.042	0.549	−0.367
I feel angry about the problem/concern.	0.123	0.136	0.746	0.053	0.099	−0.016	−0.012	0.116	0.722	0.085
I feel sad about the problem/concern.	0.277	0.112	0.661	0.028	0.032	−0.128	0.085	0.293	0.610	0.098
I feel hopeful that we can find answers to address the problem/concern.	0.238	0.264	−0.041	−0.012	0.054	−0.256	0.644	0.339	−0.006	0.386
I wish I could do something about the problem/concern, but I am not hopeful there is a solution.	−0.115	−0.276	0.326	0.067	0.047	0.252	−0.246	−0.176	0.325	−0.348
I don't understand how people can ignore the problem/concern.	0.372	0.308	0.337	−0.096	0.138	−0.090	0.196	0.290	0.391	0.326
The more I learn about the problem/concern, the more I get upset that it has not been addressed.	0.367	0.151	0.649	0.013	0.154	−0.073	0.167	0.337	0.662	0.153
Cognitive Awakening
I wonder who is responsible for the problem/concern.	−0.129	0.011	0.266	0.012	0.764	−0.050	0.112	−0.106	0.503	0.013
I would like to know who is responsible for the problem/concern in our community.	0.106	0.063	0.256	−0.027	0.920	0.048	−0.018	0.007	0.549	0.050
I believe that we share responsibility for the problem/concern.	0.401	0.093	0.099	0.406	0.166	−0.375	0.216	0.640	0.170	0.169
The more I know about the problem/concern, the more it makes me realize why it needs to be addressed.*	0.635	0.198	0.240	0.140	0.117	−0.178	0.124	0.638	0.300	0.237
Even though the problem/concern is complicated, it needs to be addressed.	0.691	0.083	0.156	0.223	0.008	−0.066	0.186	0.689	0.190	0.151
The problem/concern is primarily caused by the behavior of individuals. (R)	0.002	−0.038	−0.038	0.289	−0.077	−0.047	−0.102	0.133	−0.058	−0.044
The problem/concern is primarily influenced by the social conditions in our communities.	0.122	0.089	0.325	0.376	0.193	−0.108	−0.007	0.263	0.398	0.093
Communities should work together to address the problem/concern.	0.583	0.124	0.210	0.508	0.148	−0.028	0.226	0.687	0.305	0.206
Communities contribute to the size of the problem/concern.	0.185	0.178	0.276	0.591	0.146	−0.028	0.069	0.380	0.346	0.198
Communities are unfairly blamed for the size of the problem/concern. (R)	0.139	0.115	−0.017	0.417	−0.042	−0.005	0.043	0.286	0.000	0.134
I used to deny or avoid the problem/concern, but I don't feel comfortable doing that anymore.	0.049	0.035	0.302	0.038	0.216	−0.197	0.204	0.142	0.354	0.059
I believe that we can influence the problem/concern.	0.488	0.394	0.083	0.298	0.039	−0.230	0.254	0.596	0.139	0.475
Intentions to Act
I believe that we share responsibility for changing the problem/concern.	0.490	0.266	0.217	0.465	0.062	−0.342	0.157	0.709	0.258	0.324
I am able to imagine changing the problem/concern.	0.290	0.547	0.013	0.101	0.129	−0.192	0.243	0.325	0.097	0.603
I have some thoughts about how to address the problem/concern.	0.209	0.806	0.087	0.069	0.003	−0.030	−0.066	0.137	0.144	0.759
I would support community actions to address the problem/concern.	0.585	0.365	0.112	0.363	0.046	0.078	0.287	0.585	0.196	0.438
If someone asked me, I would do my part to help address the problem/concern.	0.526	0.375	0.198	0.339	0.032	−0.071	0.224	0.579	0.252	0.439
I plan to talk to someone to help address the problem/concern.	0.124	0.437	0.390	0.222	0.185	−0.085	0.089	0.213	0.467	0.439
I think we have the ability to make a difference in addressing the problem/concern.	0.318	0.577	0.081	0.252	0.053	−0.193	0.220	0.413	0.137	0.632
I know of some actions we could take to address the problem/concern.	0.156	0.830	0.086	0.137	−0.005	−0.056	0.086	0.134	0.144	0.835

Note. R = items were reverse coded to achieve unidirectional values for each subscale. * = item was slightly edited for consistency in the Retest study; in the Initial study, it was worded as “Knowing more about the problem/concern makes me realize why it needs to be addressed.” The nine bolded items have the best combination of subscale reliabilities and explanation of variance.

Final 4-Factor critical consciousness scale.

4-Factor Critical Consciousness Scale
Passive Adaptation
The problem/concern is an issue we should all care about. (R)
I am not concerned about the problem/concern.
Emotional Engagement
Learning about the problem/concern makes me emotional (e.g., sad, angry, helpless).
The more I learn about the problem/concern, the more I get upset that it has not been addressed.
Cognitive Awakening
I believe that we share responsibility for the problem/concern.
The more I know about the problem/concern, the more it makes me realize why it needs to be addressed.
Even though the problem/concern is complicated, it needs to be addressed.
Intentions to Act
I have some thoughts about how to address the problem/concern.
I know of some actions we could take to address the problem/concern.

Note. R = item was reverse coded to achieve unidirectional values for subscale. All items are scored using a 5-point Likert-type scale ranging from 1 (strongly disagree) to 5 (strongly agree). Copyright 2018 by Robert Werner Strack, Muhsin Michael Orsini, and D. Rose Ewald. All rights reserved. Used with permission.