Creation and validation of the barriers to alcohol reduction (BAR) scale using classical test theory and item response theory.

Those who binge drink are at increased risk for alcohol-related consequences when compared to non-binge drinkers. Research shows individuals may face barriers to reducing their drinking behavior, but few measures exist to assess these barriers. This study created and validated the Barriers to Alcohol Reduction (BAR) scale. Participants were college students (n = 230) who endorsed at least one instance of past-month binge drinking (4+ drinks for women or 5+ drinks for men). Using classical test theory, exploratory structural equation modeling found a two-factor structure of personal/psychosocial barriers and perceived program barriers. The sub-factors, and full scale had reasonable internal consistency (i.e., coefficient omega = 0.78 (personal/psychosocial), 0.82 (program barriers), and 0.83 (full measure)). The BAR also showed evidence for convergent validity with the Brief Young Adult Alcohol Consequences Questionnaire (r = 0.39, p < .001) and discriminant validity with Barriers to Physical Activity (r = -0.02, p = .81). Item Response Theory (IRT) analysis showed the two factors separately met the unidimensionality assumption, and provided further evidence for severity of the items on the two factors. Results suggest that the BAR measure appears reliable and valid for use in an undergraduate student population of binge drinkers. Future studies may want to re-examine this measure in a more diverse sample.

Alcohol use is excessively common on college campuses, with nearly 60% of students endorsing past-month alcohol use and two-thirds of those endorsing alcohol use reporting past-month binge drinking (SAMHSA, 2015). Binge drinking while in college has serious consequences for students ranging from a hangover or being late to class (Perkins, 2002) to blacking out (Merrill et al., 2016), sleep disorders (Miller, Janssen, & Jackson, 2016), increased risk of sexual assault (Abbey, 2002), and physical injury or death from motor vehicle accidents (Perkins, 2002). While college students do not necessarily view all consequences of drinking as negative (e.g. reduced stress from drinking was seen as a positive benefit), research suggests that the incidence of negative consequences doubles for each successive drink per day (incidence rate ratio = 2.34) (Barnett et al., 2014). Alcohol misuse (i.e. heavy drinking or binge drinking) is also associated with higher rates of health consequences, including certain types of cancer, cirrhosis of the liver, and cardiovascular disease (Rehm et al., 2009), and despite being a highly modifiable behavior, binge drinking contributes greatly to global mortality and morbidity (Control and Prevention, 2012; Mokdad, Marks, Stroup, & Gerberding, 2004). College students who binge drink have been shown to be more likely to experience at least one alcohol-related problem while in college compared to their non-binge drinking peers (Jennison, 2004). Thus, the college years are an important time to intervene to reduce the risk of these negative consequences.

However, there may be significant barriers that individuals face when attempting to reduce their alcohol consumption, especially in those that binge drink. The widespread nature and normative environment of alcohol use on college campuses may make students reluctant to seek treatment to help reduce their drinking (Baer, 2002). Peer influences may also contribute to difficulty in reducing alcohol use and maintaining that reduction. Among social networks where peers are not accepting of reduced drinking, even when individuals have been mandated to receive treatment for alcohol use, peak blood alcohol content sharply rose over time compared to social networks which did accept reduced drinking (Reid, Carey, Merrill, & Carey, 2015). Therefore, peers represent a major barrier to cutting down or stopping alcohol use.

However, few studies have focused specifically on peers or other barriers. Two major types of barriers to changing drinking behavior have been identified: client barriers and program barriers (Schober & Annis, 1996). Client barriers include psychosocial risk factors (e.g. situations leading to increased drinking), stigma associated with receiving treatment for alcohol misuse (e.g. fear that being seen in alcohol treatment programs may decrease one's social standing), and insufficient social or financial resources to seek help. The remaining were program barriers, including the type of treatment (multimodal or unimodal), the program's sensitivity to the needs of the individual, whether the client thought the treatment would help, lack of consistent therapists for clients to see, and requiring abstinence instead of controlled drinking (Schober & Annis, 1996). Some of these barriers identified by Schober and Annis are included in recent studies of alcohol use (Dawson, Goldstein, & Grant, 2007; Reid et al., 2015). However, no comprehensive measure of these barriers exists, limiting research in this area.

Given that there are significant health outcomes related to alcohol use and misuse on college campuses, and that college students who binge drink face barriers to reducing their alcohol consumption, establishing ways to measure these possible barriers merits further investigation. Moreover, as men are six times more likely to develop alcohol dependence if they continue heavy drinking patterns post-college, and women are twelve times more likely (Jennison, 2004), identifying barriers to reducing alcohol use during the college years could lead to interventions to reduce heavy consumption. It may well be that assessing these barriers could assist care-providers in working with college students to reduce unsafe drinking practices, similar to research of smoking cessation barriers predicting smoking abstinence (Martin, Cassidy, Murphy, & Rohsenow, 2016). However, it is currently difficult to assess barriers to seeking alcohol treatment among college students who rarely seek help (Caldeira et al., 2009) and likely have unique needs compared to other groups (e.g. combat veterans) (Santiago et al., 2010). Therefore, our study addresses this need in the literature for a measure that assesses barriers to alcohol reduction specifically for college students, using both classical test theory (CTT) and item-response theory (IRT). CTT provides information on reliability, validity, and dimensionality of a measure (Nunnally, 1978), whereas IRT provides information on the severity or difficulty of each item (Andrich, 1988; Rasch, 1960), such as with respect to reducing substance use (Kahler, Strong, & Read, 2005).

The purpose of the current study was to develop and validate a measure of barriers to alcohol use reduction, specifically in a sample of college student binge drinkers. Our hypotheses were that the final measure would achieve acceptable internal consistency levels (≥0.70: Nunnally, 1978) using the recommended coefficient omega (Dunn, Baguley, & Brunsden, 2014), convergent validity with the Brief Young Adult Alcohol Consequences Questionnaire (BYACQ: Kahler et al., 2005), and discriminant validity with Barriers to Physical Activity. Consistent with Schober and Annis (1996), we expected two facets would emerge within a single overall construct of barriers to alcohol reduction using both exploratory and confirmatory factor analysis (i.e., EFA & CFA). Finally, IRT analyses were expected to show that all items provided a cohesive list of item severity with higher values showing greater perceived barriers to alcohol reduction.

Methods

Participants

Participants for this study were drawn from a larger study on physical activity, adversity, and alcohol use (N = 720); 363 of the 720 participants completed the alcohol use portion of the study, with 230 reporting an incidence of binge drinking in the past month. These 230 participants constituted a relevant convenience sample of undergraduate students at a large, rural university in the northeastern United States who received one-point extra course-credit as compensation. This reduced sample was predominantly female (N = 172, 74.8%) and white (N = 179, 77.8%). All participants were asked to give their initial informed consent by reading the online consent document and checking that they read the document and were willing to participate. IRB approval was obtained prior to conducting this study.

Materials

Demographics

Participants completed a demographic questionnaire asking about gender identity, racial/ethnic identity, and age.

Alcohol use

Participants were asked about the frequency of their drinking behavior (past-week and past-month), and about their frequency of binge drinking.

Barriers of alcohol reduction

We developed the initial items for the Barriers of Alcohol Use Cessation (BAR) scale based on barriers identified by Schober and Annis (1996). In total, we identified 13 different barriers grouped into two categories of personal/psychosocial barriers and program barriers. The personal/psychosocial barriers included: stigma around reducing drinking, drinking in social situations, drinking alone, drinking while celebrating, drinking to cope with anxiety, drinking to cope with sadness, drinking to cope with life stressors, feeling like they could not drink less, and the program barriers included: difficulties affording treatment, low social support, treatment programs not meeting needs, treatment programs not working, and treatment programs not having a consistent therapist. These items were reworded for clarity, and placed on a 4-point rating scale (0 = False, 1 = True and Hardly at All Important, 2 = True and Moderately Important, 3 = True and Very Important) which has been used as response options in other measures of addictive behavior (Rohsenow et al., 2003). All 13 items were reviewed for content validity by the third author, who has a PhD in clinical psychology with advanced training in addiction and alcohol use.

Brief Young Adult Alcohol Consequences Questionnaire

The BYACQ has 25 items in which participants endorse whether or not they have experienced consequences of alcohol use rated on a “yes” or “no” scale (Kahler et al., 2005; Read, Kahler, Strong, & Colder, 2006). A sample item is “While drinking, I have said or done embarrassing things.” This scale has evidence for convergent validity with heavy drinking and frequency of drinking and good internal consistency levels (α = 0.83) (Kahler et al., 2005), and has been validated using IRT (Kahler et al., 2005). We used the BYACQ to assess convergent validity with our newly created barriers measure.

Barriers to physical activity (BPA)

Data for this study were gathered as part of a larger study on health behaviors. One of the scales measured was barriers to physical activity, which was a 13-item scale where 1 = Not a Barrier and 5 = Very Much a Barrier (Salmon, Owen, Crawford, Bauman, & Sallis, 2003). Salmon et al. found acceptable internal consistency among these items (Cronbach's alpha = 0.73). This measure was used to assess discriminant validity with the BAR.

Procedure

Participants for this study were recruited through several health-related undergraduate-level courses. If a student chose to participate, they were directed to an online survey link. After providing consent, participants completed several surveys including the BAR, BYACQ, BPA and demographic surveys. After completion of the study, participants were given one extra course-credit point for participating. If they were in multiple classes (i.e. in both a psychology and kinesiology course), they received course credit in each class.

Analysis plan

CTT analyses consistent with Nunnally (1978) and Fabrigar, Wegener, MacCallum, and Strahan (1999) were conducted, followed by IRT analysis based on guidelines from An and Yung (2014). In the CTT analyses, a significant positive correlation with the BYACQ would suggest evidence of convergent validity, while a non-significant correlation with the BPA measure would suggest evidence of discriminant validity. Reliability estimates above 0.70 would indicate good internal consistency (Cronbach, 1951; DeVellis, 2016; Dunn et al., 2014). Exploratory structural equation modeling (ESEM) provides measures of dimensionality by combining exploratory and confirmatory factor analyses. Loadings ≥ |0.30| were used to determine if an item loaded well on a construct (Fabrigar et al., 1999), and a standardized root mean square residual (SRMR) below 0.08 and a chi-square to degrees of freedom ratio below 5.0 indicate acceptable model fit (Hu & Bentler, 1999; Tabachnick & Fidell, 2013; Wheaton, Muthén, Alwin, & Summers, 1977). In the IRT analyses, eigenvalues, slopes, and difficulty parameters were interpreted to test for unidimensionality, determine how well each item loaded onto the construct, and the severity of each item (An & Yung, 2014; Kahler et al., 2005).

Results

Prior to the main analyses, descriptive statistics were analyzed to check assumptions. No issues of multicollinearity were found between items. However, the data were highly positively skewed, ranging from 1.51 to 4.25, and highly kurtotic, ranging from 2.66 to 17.17. Standard transformation techniques (e.g. adding 1 to remove all 0 s and performing a log10 transformation or square root transformation) did not remove the non-normality of these data. Thus, analyses were performed on the raw scores, as normality is not a true assumption of exploratory factor analysis (EFA) (Harlow, 2014). A small amount (0.05%) of the data were missing, which were imputed using maximum likelihood estimation with the expectation-maximization algorithm. After the data were imputed, any non-whole number results were rounded to the nearest discrete category. As the planned polytomous IRT analysis requires a set of discrete categories, rounding seemed to be the appropriate technique to use in this situation based on the suggestions of Graham (2009).

Because the sample size (n = 230) was relatively small, it was not feasible to use the standard approach of splitting the dataset and conducted the EFA on one portion and CFA on the remaining portion. However, the ESEM approach which allows researchers to conduct EFA within a CFA framework (Marsh, Morin, Parker, & Kaur, 2014). Essentially, ESEM conducts two exploratory factor analyses in two sets of variables, estimates the correlation between factors, and also estimates the correlations between the items on factor 1 with the second latent variable (and vice versa). This makes ESEM a flexible way to measure dimensionality, as the main difference between ESEM and CFA is CFA does not allow for the cross-item loading. Thus, due to the flexibility of ESEM and the limited amount of data, ESEM was used to test the dimensionality of the BAR.

All ESEM analyses were conducted using the R package psych version 1.7.5 (Reville, 2017) using maximum likelihood extraction and promax rotation as it was expected that factors would be correlated. ESEM analyses typically use an a priori factor structure based on the items written for the measure (Marsh et al., 2014). Using the theoretical guidance from the personal/psychosocial barriers and program barriers described by Schober and Annis (1996), and empirical guidance from conducting a MAP test which suggested extracting two factors, we identified a factor structure where items 1 through 8 (personal/psychosocial barriers) loaded on the first factor, and items 9 through 13 (program barriers) loaded on the second factor. In the initial model, item 1 (“I would feel stigmatized for needing help to drink less”) did not load above |0.29| and was subsequently dropped. The ESEM was then re-conducted and showed acceptable fit as the standardized root mean square residual (SRMR) was below 0.08, SRMR = 0.07 (Hu & Bentler, 1999). The chi-square test was significant, χ2 (43) = 159.35, p < .001, but chi-square tests tend to be sensitive and a significant result does not necessarily indicate poor fit (Harlow, 2014; McIntosh, 2007). Moreover, the chi-square to degrees of freedom ratio (159.35/43) was 3.71, which although above some restrictive guidelines of 2.0 it falls under permissive guidelines of 5.0, further indicating acceptable model fit (Tabachnick & Fidell, 2013; Wheaton et al., 1977). It should also be noted that the chi-square value is likely inflated due to the non-normality of the data, and a robust chi-square analysis could result in a lower chi-square to degrees of freedom ratio (Satorra & Bentler, 1994). There was a moderate positive correlation between the two factors (r = 0.47), and the loadings are shown below in Table 1. The slightly revised 12-item BAR is given in the Appendix A, dropping the original item 1 and renumbering.

Table 1

Exploratory structural equation modeling results.

Items (Note: Item 1 not listed as it was dropped due to low loadings.)	Loadings
	Factor 1	Factor 2
2. It would be difficult for me to drink less in social situations.	0.42	–
3. It would be difficult for me to drink less while alone.	0.40	–
4. It would be difficult for me to drink less while celebrating.	0.42	–
5. Drinking helps me cope with feelings of anxiety.	0.85	–
6. Drinking helps me cope with feelings of sadness.	0.87	–
7. Drinking helps me cope with life stressors (e.g. work, family, money, etc.).	0.76	–
8. I feel like I could not drink less even if I tried.	0.32	–
9. I feel like I cannot afford treatment to help me drink less.	–	0.53
10. I feel like my friends and family would not support me in getting treatment to help me drink less.	–	0.56
11. I feel like treatment programs to help me drink less would not cater to my needs.	–	0.76
12. I feel like a treatment program to help me drink less would not work for me.	–	0.77
13. I feel like a treatment program to help me drink less would not have a consistent therapist for me to see.	–	0.80

Reliability analyses showed the first factor had good internal consistency (coefficient omega = 0.78, 95% CI [0.70, 0.83]), as did the second factor (omega = 0.82, 95% CI [0.75, 0.87]), and the entire scale (omega = 0.83 95% CI [0.77, 0.87]). The correlation between the full BAR and the BYACQ provided convergent validity evidence (r = 0.39, p < .001). Discriminant validity support was found with a nonsignificant relationship between the BAR and the Barriers to Physical Activity variable, (r = −0.02, p = .81).

Based on the ESEM results, IRT analyses were conducted on the items comprising the two factors. All IRT analyses were conducted in SAS 9.4 using PROC IRT (SAS Institute, 2013). In accordance with An and Yung (2014), the first IRT step is to check for unidimensionality. Since the ESEM results suggested two factors, the IRT analyses were conducted on each factor separately to see if the assumption of unidimensionality was met within each factor. Starting with factor 1, there was evidence for unidimensionality as the first eigenvalue (4.02) explained 57.37% of the variance within the factor. Similar results were found for factor 2, where the first eigenvalue (3.91) explained 78.18% of the variance. Based on the results of the eigenvalues and the amount of variance explained, it was reasonable to conclude that the unidimensionality assumption was met for both factors.

The second step in the IRT analyses is to check if the items load well onto the latent construct. This was done by examining the slope parameters for each item, and any item with a slope below 0.5 is considered a poor indicator (An & Yung, 2014). All the items on both factors had a slope above 0.5, so all items were retained for future analyses. Slope estimates are shown in Table 2, Table 3 below.

Table 2

Item response theory results for the personal/psychosocial factor.

Item	Parameters
	Severity threshold 1	Severity threshold 2	Severity threshold 3	Slope
It would be difficult for me to drink less in social situations.	−0.02	1.17⁎⁎⁎	3.21⁎⁎⁎	.97⁎⁎⁎
It would be difficult for me to drink less while alone.	1.74⁎⁎⁎	2.20⁎⁎⁎	3.11⁎⁎⁎	1.24⁎⁎⁎
It would be difficult for me to drink less while celebrating.	−.69⁎⁎⁎	.67⁎⁎⁎	2.81⁎⁎⁎	.99⁎⁎⁎
Drinking helps me cope with feelings of anxiety.	.34⁎⁎⁎	1.05⁎⁎⁎	1.89⁎⁎⁎	3.97⁎⁎⁎
Drinking helps me cope with feelings of sadness.	.35⁎⁎⁎	.90⁎⁎⁎	1.91⁎⁎⁎	4.74⁎⁎⁎
Drinking helps me cope with life stressors (e.g. work, family, money, etc.).	−.06	.69⁎⁎⁎	1.88⁎⁎⁎	2.86⁎⁎⁎
I feel like I could not drink less even if I tried.	2.63⁎⁎⁎	3.21⁎⁎⁎	4.39⁎⁎⁎	.90⁎⁎⁎

Note: = n.s.

= p < .001.

Table 3

Item response theory results for the program factor.

Item	Parameters
	Severity threshold 1	Severity threshold 2	Severity threshold 3	Slope
I feel like I cannot afford treatment to help me drink less.	1.50⁎⁎⁎	1.81⁎⁎⁎	2.22⁎⁎⁎	2.44⁎⁎⁎
I feel like my friends and family would not support me in getting treatment to help me drink less.	1.67⁎⁎⁎	1.84⁎⁎⁎	2.29⁎⁎⁎	2.48⁎⁎⁎
I feel like treatment programs to help me drink less would not cater to my needs.	1.16⁎⁎⁎	1.52⁎⁎⁎	1.91⁎⁎⁎	3.84⁎⁎⁎
I feel like a treatment program to help me drink less would not work for me.	1.07⁎⁎⁎	1.49⁎⁎⁎	1.82⁎⁎⁎	3.77⁎⁎⁎
I feel like a treatment program to help me drink less would not have a consistent therapist for me to see.	1.26⁎⁎⁎	1.53⁎⁎⁎	1.98⁎⁎⁎	6.63⁎⁎⁎

Note: = n.s.

= p < .001.

The final step in analyses was to examine the difficulty parameter for each item (An & Yung, 2014). Kahler et al. (2005) suggests that in the case of substance use measures, difficulty parameters can be interpreted as a severity estimate. In this case, negative items are considered less severe, and positive items are considered more severe. The further away from 0 the difficult parameter is, the item is then considered more or less severe. In the case of polytomous IRT, severity parameters are interpreted within-item. For example, for item 2 (“It would be difficult for me to drink less in social situations”), the severity for responding with 1 (True and hardly at all important) was −0.02. However, the severity for responding with 3 (True and very important) was 3.22, suggesting that if social drinking was a very important barrier for the individual, it could be considered a very severe indicator. If social drinking was of low importance, then it could be considered much less severe. Severity and estimates for all items on both factors are shown in Table 2, Table 3 below.

Discussion

The purpose of this study was to create and validate a scale to measure barriers individuals may face when reducing their alcohol use. We created this scale using a sample of college student binge drinkers because they have been shown to be at risk for alcohol-related problems during college and alcohol misuse later in life (Abbey, 2002; O'Malley & Johnston, 2002; Perkins, 2002; Rehm et al., 2009; Tyler, Schmitz, Adams, & Simons, 2017; Wechsler, Lee, Kuo, & Lee, 2000). Additionally, college students may face great difficulty in reducing drinking due to the social influences of their peers (Reid et al., 2015). CTT results led to a 12-item scale with good reliability, convergent and discriminant validity and two sub-factors. IRT results revealed that the BAR items were good indicators of a single underlying construct within both factors, which met assumptions for the IRT analyses (An & Yung, 2014).

Consistent with the literature, our results from this study provided evidence for a general factor of barriers that could be delineated into two sub-factors (Schober & Annis, 1996). The first factor contained items describing personal and psychosocial barriers which were mainly reasons participants drank, whether for social reasons or for coping reasons. The second factor addressed perceives barriers against treatment programs, like an inability to afford treatment, lack of support from friends and family, feeling treatment programs would not cater to their individual needs, feeling like a treatment program would not work, and feeling like a treatment program would provide them with a consistent clinician.

Findings from the IRT analyses suggested that some of the BAR items were particularly severe. On the personal/psychosocial factor, a response of “True and very important” to item 8 (“I feel like I could not drink less even if I tried”) had a severity parameter of 4.39, which was the highest overall across both factors. The severity of this item makes theoretical sense, as feeling like one could not drink less even if he or she tried is a symptom of alcohol dependence (American Psychiatric Association, 2013). On the program barriers factor, a response of “True and very important” to item 10 (“I feel like my friends and family would not support me in getting treatment to help me drink less”) had a severity parameter of 2.29 which was the highest on the program barriers factor. A lack of social support for seeking help from a program may well be a significant barrier in the undergraduate student population. The lack of support could also be an indication of peer influence to drink alcohol, similar to the results of Reid et al. (2015).

Our results should be understood within the context of the limitations present in the study. First, the sample was predominantly white and female, and may not generalize to samples with different demographics. Considering that male college students tend to binge drink more often than female college students (White & Hingson, 2014), it may well be that the loadings and severity of items could be different in another male dominated sample. Second, the response options could be considered “double-barreled” as the questions ask both true and false as well as importance within the same response option. Future studies may want to re-validate the BAR using a different scale to address this concern. Third, it would be worthwhile to add several more items to assess the program barriers factor to equalize the number of items across subscales. Fourth, the use of ESEM was beneficial in this analysis due to the limited sample size as it allowed for both exploratory and confirmatory analyses in a single step. However, an inherent drawback is the lack of a second sample to validate these findings on a separate independent sample, and the restrictiveness of CFA may be desirable when validating new measures (Marsh et al., 2014). Fifth, the relatively few fit indices currently provided by the psych package do not allow for as robust of an interpretation of the ESEM results as might be desired. Thus, it would be preferable to have a larger and more diverse sample to more fully verify the nature of the BAR, possibly assessing invariance across different subsets of the data (e.g., by gender or ethnicity) using multiple sample CFA.

The results of this study provide evidence that the BAR appears reliable and valid for use in this binge-drinking college student sample. This may present promising implications for its usefulness in treatment planning for interventions targeting problematic drinking among college students. Because students who binge drink during college are at significantly increased risk for heavy drinking later in life (Jennison, 2004), this presents an important time to intervene to prevent future consequences. This measure could be used as a follow-up to brief screeners of alcohol use in college students who report an occasion of binge drinking by healthcare and counseling services on university campuses, or by providers who are seeing college students in their practice. It could also be particularly useful to utilize in intervention-planning for college students who are specifically seeking services to help them reduce their drinking or for those who have been mandated to seek treatment following an alcohol-related violation. The ability for clinicians to determine what barriers are most important to their clients by the rating scale in this measure could prove to have significant clinical utility, as it may enable the discerning of what areas the client feels should be addressed most urgently. This then provides for the opportunity to tailor interventions for alcohol reduction, which has been shown in extant literature to enhance the impact of interventions (Gagliardi, 2011; Ryan & Lauver, 2002). The clinician would be able to use this information to clarify misconceptions about certain barriers and provide strengths-based coping skills to deal with other barriers. The addition of this information may well increase the effectiveness of interventions.