Predictive utility of the brief Screener for Substance and Behavioral Addictions for identifying self-attributed problems.

BACKGROUND AND AIMS: The Brief Screener for Substance and Behavioral Addictions (SSBAs) was developed to assess a common addiction construct across four substances (alcohol, tobacco, cannabis, and cocaine), and six behaviors (gambling, shopping, videogaming, eating, sexual activity, and working) using a lay epidemiology perspective. This paper extends our previous work by examining the predictive utility of the SSBA to identify self-attributed addiction problems.
METHOD: Participants (N = 6,000) were recruited in Canada using quota sampling methods. Receiver Operating Characteristics (ROCs) analyses were conducted, and thresholds established for each target behavior's subscale to predict self-attributed problems with these substances and behaviors. For each substance and behavior, regression models compared overall classification accuracy and model fit when lay epidemiologic indicators assessed using the SSBA were compared with validated screening measures to predict selfattributed problems.
RESULTS: ROC analyses indicted moderate to high diagnostic accuracy (Area under the curves (AUCs) 0.73-0.94) across SSBA subscales. Thresholds for identifying self-attributed problems were 3 for six of the subscales (alcohol, tobacco, cannabis, cocaine, shopping, and gaming), and 2 for the remaining four behaviors (gambling, eating, sexual activity, and working). Compared to other instruments assessing addiction problems, models using the SSBA provided equivalent or better model fit, and overall had higher classification accuracy in the prediction of self-attributed problems. DISCUSSION AND
CONCLUSIONS: The SSBA is a viable screening tool for problematic engagement across ten potentially addictive behaviors. Where longer screening tools are not appropriate, the SSBA may be used to identify individuals who would benefit from further assessment.

The past decade has seen marked shifts in the conceptualization and diagnosis of addictive disorders. Gambling Disorder is the first nonsubstance-related addiction to be included in the Diagnostic and Statistical Manual of Mental Disorders – 5th Edition (DSM-5; American Psychiatric Association, 2013) under “Substance-Related and Addictive Disorders” and the recent revision of the International Classification of Diseases (ICD-11; World Health Organization, 2019) includes both gambling and gaming addiction. Compulsive sexual behavior is also included, though not currently as an addictive disorder. These shifts in addiction conceptualization raise fundamental classification and nosological issues, including how to operationally define “addiction,” what behaviors could be considered addictive, and how to accurately measure problematic engagement in such behaviors (Billieux, Schimmenti, Khazaal, Maurage, & Heeren, 2015; Grant & Cham berlain, 2016; Musetti et al., 2016).

Lay epidemiological approaches may contribute to resolving these issues, given their emphasis on the perspectives and experiences of the general public with potentially addictive behaviors. Lay epidemiology proposes that “fields of symptomology, nosology, etiology, and epidemiology have identifiable counterparts in the thoughts and activities of people outside the formal medical community” (Davison, Smith, & Frankel, 1991, p. 6). We previously used this approach to develop an instrument to assess self-attributed signs and symptoms of addiction problems across four substances and six behaviors in community surveys (Schluter, Hodgins, Wolfe, & Wild, 2018). This paper extends on our previous work by examining the utility of the brief Screener for Substance and Behavioral Addictions (SSBA; Schluter et al., 2018) to predict self-attributed problems with potentially addictive behaviors (herein referred to as self-attributed addiction problems), which may have utility in aligning treatment services with population need.

Two primary arguments supported the development of the SSBA. First, the addiction field is increasingly interested in operational definitions of “addiction” that are based on common etiologic mechanisms and features rather than specific substances or behaviors (Griffiths, 2017; Tunney & James, 2017). Several unified theories of addiction have been proposed, which define addiction based on core features common across substances and behaviors or underlying mechanisms (e.g., Shaffer et al., 2004). A consistent conceptualization of addiction also lends itself to the development of effective interventions that can be applied across substances and behaviors (Kim & Hodgins, 2018). However, these theories typically prioritize signs and symptoms of problematic involvement that were developed to define substance-specific disorders. When considering nonsubstance-related behaviors, this approach to theory and measurement development is problematic, as it can lead to diagnostic reification and discourage the development of a valid conceptualization (Hyman, 2010).

Alternatively, a phenomenological approach has been recommended, though it is underutilized in behavioral addictions research (Kardefelt-Winther et al., 2017). Consistent with this recommendation, the SSBA was developed using a lay epidemiology approach. In particular, common signs and symptoms of problematic involvement across substances and behaviors were derived phenomenologically from the experiences of individuals in the general population. Wild et al. (2015) provide a full description of the item development process). The target behaviors included in the SSBA share features that activate the reward-based learning system as a putative underlying mechanism, suggesting the potential for dysregulated associative learning and reward processing (Holden, 2001; Potenza, 2006, 2017), and have been identified as concerns for some individuals in the Canadian population (Konkolÿ Thege et al., 2015). We then identified a small, psychometrically optimal set of items to assess self-attributed signs and symptoms of addiction across the four substances (alcohol, tobacco, cannabis, and cocaine), and six behaviors (gambling, shopping, videogaming, eating, sexual activity, and working; Schluter et al., 2018). The four items per target behavior showed uniformly high loadings on a single factor, suggesting unidimensionality for each subscale.

A second argument supporting the development of the SSBA is that extant clinical screening instruments are insufficient or impractical for measuring problematic engagement in substances and behaviors in community epidemiology, i.e., when population-wide assessment is at issue. Measures exist for specific substances including alcohol (Bush, Kivlahan, McDonell, Fihn, & Bradley, 1998), tobacco (Etter, Le Houezec, & Perneger, 2003), cannabis (Adamson et al., 2010), and gambling (Ferris & Wynne, 2001). However, these scales have heterogeneous item formats and conceptual approaches, which makes comparison across behaviors difficult. Additionally, scales that do assess multiple substances and behaviors are either too long to be used for population-level screening, or do not have demonstrated reliability and validity for use in community samples (e.g., the Shorter PROMIS Questionnaire; Christo et al., 2003). The SSBA uses the same four items assessing signs and symptoms across all 10 target substances and behaviors, allowing for comparison across these behaviors and is brief enough to be used for population-level screening. As such, it may be a less costly method of screening for a broad spectrum of potentially addictive behaviors in the general population.

Extending our previous work, the purpose of this paper was to examine the utility of the SSBA as an instrument that could be used to identify self-attributed addiction problems from a lay epidemiologic perspective and to identify people who may benefit from subsequent clinical assessment. To this end, our specific aims were to (a) establish threshold scores on the SSBA for each target substance and behavior that could be used to identify potentially affected individuals, and (b) examine the utility of the SSBA subscales in predicting self-attributed addiction problems, compared to extant clinical screening tools.

Methods

Participants and procedure

In this study, we performed secondary analyses of data described in Schluter et al. (2018). Analyses were conducted using a large Canadian online panel sample (N = 6,000) from which the measure was developed. We contracted the Ipsos Reid Canadian Online panel to recruit participants and conduct online assessments in November and December of 2013. To maximize participation, email reminders were sent one and two weeks following the initial invitation and an incentive was provided. The data were obtained using quota sampling in order to ensure that the sample's regional and age/gender composition was representative of the English-speaking Canadian population aged 18 years or older, based on the 2011 Canadian Census data (Table 1). Unweighted data were used in the present set of analyses, as this is appropriate for receiver operating characteristic analyses (Keilwagen, Grosse, & Grau, 2014).

Table 1.

Demographic description of study sample (N = 6,000)

Characteristic	N	%
Gender
Male	2,770	46.2
Female	3,230	53.8
Age range
18–34	1,393	23.2
35–54	2,336	38.9
55+	2,271	37.9
Region
British Columbia	1,040	17.3
Alberta	727	12.1
Saskatchewan/Manitoba	492	8.2
Ontario	3,001	50.0
Quebec–English	190	3.2
Atlantic	550	9.2
Education
Less than high school	305	5.1
High school diploma	1,191	19.9
Some postsecondary	1,101	18.4
Postsecondary diploma	1,448	24.1
University undergraduate degree	1,123	18.7
University graduate or professional degree	816	13.6
Unknowna	16	0.3
Marital status
Married or common law	3,686	61.4
Separated or divorced	607	10.1
Widowed	240	4.0
Single	1,441	24.0
Unknowna	26	0.4
Employment
Parttime (<30 hours/week)	758	12.6
Fulltime (>30 hours/week)	2,750	45.8
Unemployed	347	5.8
Student	260	4.3
Retired	1,286	21.4
Disability	356	5.9
Other	186	3.1
Unknowna	57	1.0

Due to small cell sizes, ‘don't know’ and ‘prefer not to answer’ responses are grouped together.

Measures

Screener for Substance and Behavioral Addictions (Schluter et al., 2018)

The SSBA is a short screening instrument for measuring self-attributed addiction problems in the general population. The SSBA is comprised of four selfreport item stems, each reflecting a distinct sign or symptom of potentially problematic involvement (“I did it too much;” “Once I started, I couldn't stop;” “I felt I had to do it in order to function;” and “I continued to do it, even though it caused problems”) administered for each of four substances (alcohol, tobacco, cannabis, and cocaine), and six behaviors (gambling, shopping, videogaming, eating, sexual activity, and working). Each item is rated in terms of frequency in the previous 12 months on a 5-point Likert scale: 0 = None of the time, 1 = A little of the time, 2 = Some of the time, 3 = Most of the time, and 4 = All of the time. Two additional response options were available: “I didn't do this at all” and “Don't know/prefer not to say.” Participants endorsing either of these latter response options were excluded from the present analyses. Participants were also provided with brief definitions of each behavior. To reduce the risk that participants would misunderstand what types of problems the questions are meant to address, the descriptions for behaviors that are involved in order disorders (namely eating and sex) included excessive involvement (see Appendix A for the measure). Table 2 presents the score ranges, means, and medians for the summary scores of SSBA subscale scores for each target behavior for the overall sample, and only among individuals who endorsed using the target substance or engaging in the target behavior in the past 12 months (from Schluter et al., 2018). Coefficient alphas ranged from 0.87 to 0.95, indicating good internal reliability.

Table 2.

Descriptive statistics for the SSBA among the entire sample and among individuals who endorsed engagement in the target behavior in prior 12-months

Target Behavior	Overall Sample				Past Year Behavior Engagement
	N	Range	M(SD)	Median	N	Range	M(SD)	Median
Alcohol	3,924	0–16	0.9(3.3)	0	351	0–16	2.3(3.5)	0
Tobacco	3,006	0–16	3.5(5.0)	0	151	0–16	6.6(5.5)	6
Cannabis	2,517	0–16	1.3(3.2)	0	63	0–16	3.7(4.8)	1
Cocaine	2,216	0–16	0.8(3.0)	0	11	0–16	8.4(5.8)	8
Gambling	3,111	0–16	1.4(3.0)	0	209	0–16	1.6(3.1)	0
Shopping	4,679	0–16	1.7(2.9)	0	74	0–16	1.3(2.5)	0
Video gaming	2,349	0–16	1.9(3.3)	0	232	0–16	2.6(3.7)	1
Overeating	4,233	0–16	2.0(3.6)	2	285	0–16	4.1(3.8)	3
Sex	4,173	0–16	1.6(3.0)	0	328	0–16	1.6(3.1)	0
Overworking	3,972	0–16	2.6(3.6)	1	167	0–16	4.7(3.9)	4

The overall sample excludes participants with any responses coded as missing cases for analysis (“I didn't do this at all” or “I don't know/prefer not to say”). Past-year behavior engagement descriptive statistics refer to the individuals randomly assigned to complete an additional validation measure and who endorsed any (not necessarily problematic level of) past-year engagement in the given behavior. This table is adapted from Schluter et al. (2018).

Self-attributed problems, perceived need for behavior change, and help-seeking

To develop suitable threshold scores for predicting self-attributed problems, participants were randomly assigned to one of the 10 target substances and behaviors and answered three additional questions: (1) Thinking back over your life, have you ever personally had a problem with [target substance or behavior]? (2) Thinking back over your life, have you ever felt that you needed to reduce your [target substance or behavior]? and (3) Thinking back over your life, have you ever tried to get formal help to change your [target substance or behavior]? Each question had four response options: “No,” “Yes, in the past 12 months,” “Yes, but not in the past 12 months,” or “Don't know/prefer not to say.” Respondents endorsing the two latter options were excluded from analyses of that variable.

Validation screening instruments

Participants completed a validation instrument for the behavior to which they were randomly assigned. If participants indicated that they had not engaged in that behavior within the past 12 months, they did not complete the validation measure. Given that many behaviors lack a gold-standard for measurement, these instruments were selected from a systematic psychometric review of instruments because they demonstrated acceptable psychometric properties in epidemiological studies at the time of the literature review, were developed by expert clinical researchers using signs and symptoms that they regarded as important indicators of addiction problems, and were reasonably brief (Wild et al., 2010): the Alcohol Use Disorders Identification Test, Consumption subscale (AUDIT-C; Bush et al., 1998); the Cigarette Dependence Scale – 5 (CDS-5; Etter et al., 2003); the Cannabis Abuse Screening Test (CAST; Legleye, Karila, Beck, & Reynaud, 2009); the Obsessive-Compulsive Cocaine Use Scale (OCCUS; Hormes, Coffey, Drobes, & Saladin, 2012); the Problem Gambling Severity Index (PGSI; Ferris & Wynne, 2001); the Compulsive Buying Measurement Scale (CBMS; Valence, d'Astous, & Fortier, 1988); the Game Addiction Inventory for Adults (GAIA; Wong & Hodgins, 2013); the Yale Food Addiction Scale (YFAS; Gearhardt, Corbin, & Brownell, 2009); the Sexual Compulsivity Scale (SCS; Kalichman et al., 1994); and the Work Addiction Risk Test (WART; Flowers & Robinson, 2002). Internal consistency in the present study was good or excellent for most external measures (α = 0.80–0.98), apart from the CDS-5 (α = 0.75).

Baratt Impulsiveness Scale (BIS-11; Patton, Stanford, & Barratt, 1995)

The BIS-11 was included as a measure of impulsivity. The BIS-11 is composed of 30 items that are scores on a four-point likert scale. Total scores range from 30 to 120, with higher scores indicating greater impulsivity. Internal consistency in the present study was good (α = 0.85).

Demographics

The final set of questions assessed participants' demographic characteristics including sex, age, educational level, marital status, employment status, and household income (Table 1).

Statistical analyses

All analyses were conducted in R (R Core Team, 2013) using the following packages: generalhoslem (Jay, 2019), rcompanion (Mangiafico, 2019), nonnest2 (Merkle & You, 2018), and pROC (Robin et al., 2011). We first used receiver operating characteristics (ROCs) analyses to assess the performance of the SSBA subscales in predicting self-attributed past-year problems, need for behavior change, and help seeking, and to establish a threshold score on the SSBA to identify the need for more detailed assessment for each behavior; ROC analyses were conducted separately for each behavior. Area under the curve (AUC) values between 0.70 and 0.90 are considered moderate and values of 0.90 and greater are considered high (Streiner & Cairney, 2007).

In the absence of a clear gold standard against which to establish thresholds, individuals' self-attribution of a problem with the given behavior in the previous 12-months was used to indicate a positive case as this criterion aligns with the lay epidemiologic focus of the SSBA. Self-reported need for behavior change and history of treatment seeking represent more liberal and conservative boundaries on implications of experiencing signs and symptoms of potential addiction problems, respectively, and were thus also measured as comparisons.

The Youden Index was used to determine the recommended threshold score for SSBA scores for each target behavior. This index reflects the overall ability of a measure to detect positive and negative cases (Youden, 1950). The score that maximizes both sensitivity and specificity is determined to be “optimal.” In a few cases the recommended score was different from most other target behaviors. However, in all cases, the sensitivity and specificity of the recommended threshold were not significantly different from the score recommended for other behaviors. Therefore, we selected the latter score for greater consistency.

To further examine the SSBA's utility for predicting self-attributed problems, perceived need for behavior change, and past treatment seeking, the performance of each subscale score was compared to the performance of the relevant clinical screening instrument for that behavior. Two logistic regression analyses were run for each target behavior, with past-year self-attributed problem as the criterion variable. The first model for each target behavior included the SSBA subscale score, and the second model included the extant screening measure. Except for cocaine (due to low base rates of cocaine use) age, gender, employment, education, marital status, and impulsivity were entered into each model as covariates because these characteristics are associated with addiction problems (Becker, McClellan, & Reed, 2017; Swendsen et al., 2009).

Finally, a comparison of overall classification accuracy and model fit were conducted in order to further examine the predictive validity of the SSBA relative to commonly used screening instruments developed by clinical researchers. For each behavior, the two models were compared using Vuong's likelihood-ratio test (Vuong, 1989). Each pair of models were first examined for their indistinguishability (i.e., whether the models provide the same fit to the criterion variable of interest). In cases where the two models were distinguishable (P-value ≤0.05) the nonnested likelihood ratio test was used to compare model fit. In cases were the two models were indistinguishable (i.e., nested), the robust likelihood ratio test was used to compare model fit. Overall classification accuracies of the regression models were also calculated.

Ethics

The study procedures were carried out in accordance with the Research Ethics Board at The University of Alberta. The Research Ethics Board at the University of Alberta approved the study. All subjects were informed about the study and provided consent.

Results

Descriptive statistics

The sample consisted of 2,770 males (46.2%) and 3,230 females (53.8%) who ranged in age from 18 to 93 (M = 48.32, SD = 15.21). Most participants reported that they were married or common law (n = 3,686, 61.4%), and employed fulltime (n = 2,750, 45.8%) or retired (n = 1,286, 21.4%). Demographic characteristics of the sample are presented in Table 1. Table 3 presents Spearman correlations between summary scores for the SSBA and the other screening/validation measures, and the means and standard deviations of these other measures.

Table 3.

Spearman's Rho Correlation Coefficients between the SSBA and other screening/validation measures

	AUDIT-C	CDS-5	CAST	OCCUS	PGSI	CBMS	GAIA	Yale	SCS	WART	BIS-total
Alcohol	0.63	−0.13	0.26	0.58	0.43	0.32	0.23	0.19	0.24	0.12	0.31
Tobacco	0.26	0.48	0.38	0.30	0.30	0.20	0.22	0.26	0.23	0.16	0.28
Marijuana	0.23	0.00	0.74	0.77	0.42	0.27	0.36	0.29	0.40	0.24	0.34
Cocaine	0.30	−0.21	0.20	0.82	0.57	0.31	0.45	0.50	0.34	0.03	0.36
Gambling	0.26	−0.09	0.41	0.74	0.72	0.37	0.38	0.36	0.35	0.26	0.35
Shopping	0.07	0.00	0.11	0.73	0.38	0.35	0.45	0.40	0.22	0.20	0.35
Gaming	0.17	−0.08	0.30	0.19	0.34	0.35	0.71	0.47	0.40	0.24	0.36
Overeating	0.04	−0.05	0.26	0.57	0.27	0.26	0.41	0.70	0.28	0.27	0.32
Sexual activity	0.19	−0.05	0.40	0.91	0.37	0.29	0.31	0.22	0.50	0.21	0.31
Overworking	0.22	0.02	0.39	0.83	0.35	0.24	0.30	0.15	0.27	0.24	0.20
M	3.53	18.76	5.34	15.00	1.65	7.98	47.44	2.17	12.81	61.19	58.62
SD	2.59	3.66	5.02	15.82	3.75	3.31	23.19	1.73	5.11	13.14	11.02

AUDIT-C = Alcohol Use Disorders Identification Test (Consumption); CDS-5 = Cigarette Dependence Scale – 5; CAST = Cannabis Abuse Screening Test; OCCUS = Obsessive-Compulsive Cocaine Use Scale; PGSI = Problem Gambling Severity Index; CBMS = Compulsive Buying Scale; GAIAs = Game Addiction Inventory for Adults; Yale = Yale Food Addiction Scale; SCS = Sexual Compulsivity Scale; WART = Work Addiction Risk Test; BIS = Barratt Impulsivity Scale. This table is adapted from Schluter et al. (2018).

Receiver Operating Characteristics

Receiver Operating Characteristic (ROC) curves plot the probability of a truepositive test result (sensitivity) against the probability of a truenegative test result (specificity; Youngstrom, 2014). The AUC indexes the accuracy of the measure. Table 4 reports the AUCs in relation to self-attributed addiction problems, perceived need for behavior change, and past help-seeking, with 95% confidence intervals. AUC values were between 0.73 and 0.94, indicating moderate to high accuracy for these outcomes. Using self-attributed problems as the criterion, the determined threshold score was 3 for six of the target behaviors (alcohol, tobacco, cannabis, cocaine, shopping, and gaming), and 2 for the remaining four behaviors (gambling, eating, sexual activity, and working). Table 4 also reports these threshold scores, and their corresponding sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each target behavior and for each outcome variable.

Table 4.

Area under the curve (AUC), 95% confidence intervals for AUC, optimal threshold scores, and corresponding sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and true positives for each behavior broken down by predictor variable

SSBA subscale	Threshold	Problem attribution							Perceived need for behavior change							Help-seeking
		AUC	95% CI s	Sensitivity	Specificity	NPV	PPV	True positives	AUC	95% CI	Sensitivity	Specificity	NPV	PPV	True Positives	AUC	95% CI	Sensitivity	Specificity	NPV	PPV	True positives
Alcohol	3	0.82	0.76–0.88	0.70	0.82	0.93	0.45	0.12	0.89	0.85–0.93	0.79	0.86	0.94	0.61	0.16	0.94	0.91–0.96	1	0.76	1	0.17	0.05
Tobacco	3	0.86	0.81–0.91	0.85	0.75	0.89	0.68	0.32	0.86	0.81–0.91	0.81	0.80	0.83	0.78	0.38	0.77	0.90–0.85	0.85	0.60	0.95	0.31	0.15
Cannabis	3	0.85	0.74–0.95	0.73	0.90	0.95	0.53	0.10	0.85	0.73–0.96	0.75	0.87	0.97	0.4	0.07	0.76	0.52–0.99	0,67	0.83	0.98	0.13	0.02
Cocaine	3	0.91	0.84–0.99	1	0.75	1	0.25	0.17	0.88	0.80–0.97	1	0.82	1	0.5	0.15	0.90	0.82–0.99	1	0.75	1	0.25	0.08
Shopping	3	0.80	0.73–0.86	0.62	0.84	0.94	0.37	0.08	0.77	0.71–0.82	0.46	0.86	0.81	0.51	0.11	0.87	0.77–0.97	0.86	0.80	0.99	0.12	0.02
Gaming	3	0.73	0.65–0.80	0.57	0.81	0.86	0.48	0.13	0.86	0.80–0.91	0.75	0.83	0.94	0.51	0.14	0.81	0.68–0.94	0.85	0.75	0.99	0.14	0.03
Gambling	2	0.75	0.68–0.82	0.59	0.86	0.88	0.54	0.13	0.86	0.79–0.93	0.80	0.86	0.96	0.48	0.12	0.91	0.80–1.00	0.91	0.80	1	0.15	0.03
Overeating	2	0.83	0.79–0.87	0.81	0.75	0.83	0.73	0.37	0.83	0.79–0.87	0.77	0.80	0.75	0.81	0.63	0.76	0.69–0.83	0.84	0.54	0.96	0.22	0.18
Sexual Activity	2	0.77	0.68–0.87	0.58	0.78	0.94	0.25	0.06	0.74	0.65–0.82	0.69	0.78	0.97	0.22	0.05	0.81	0.68–0.95	0.75	0.76	0.99	0.09	0.02
Overworking	2	0.77	0.72–0.82	0.83	0.68	0.90	0.55	0.27	0.81	0.76–0.85	0.80	0.67	0.88	0.54	0.26	0.74	0.63–0.85	0.83	0.54	0.97	0.12	0.05

For each predictor variable, endorsement in the past 12-months was used to indicate a positive case. Individuals who had not engaged in the behavior in the prior 12 months were excluded from analysis. For a given behavior, true positives refer to the proportion of individuals who responded “yes” to self-attributed problem, need for behavior change, or help-seeking behavior and met the SSBA threshold score, out of all participants included in the ROC analyses for that behavior.

Logistic regression analyses and model fit

To further examine the predictive utility of the SSBA, the performance of each subscale score was compared to the performance of the relevant external validation instrument for that behavior using logistic regression analyses. Hosmer–Lemeshow tests were nonsignificant (ps 0.24–0.96), indicating that the hypothesis of good model fit for each model was tenable. After controlling for the included covariates, all SSBA subscale scores and external screener scores (apart from cocaine) accounted for significant variance in problem attribution in their respective models (χ2 = 8.37–38.75; ps < 0.001). The SSBA cocaine subscale did not significantly predict problem attribution (χ2 = 1.29, P = 0.26) while the external screener did so (χ2 = 4.96, P = 0.03). Vuong's test indicated that the first model (with SSBA screener) provided a better fit to the three outcomes of interest for alcohol, cocaine, gambling, work, sex, and tobacco (Table 5). For cannabis, eating, gaming, and shopping the fit for the two models were equivalent. In the latter target behaviors however, the overall accuracy of the model employing the SSBA was higher or equivalent to the accuracy of the model employing the external measure.

Table 5.

Logistic regression analyses predicting self-attributed problem for each behavior and comparison of model fits between the SSBA and external screening measure, among participants who completed both the SSBA and external screening measure

	n	Model	Pseudo R2 (Nagelkerke)	Accuracy	Vuong's test for model equivalence
					z	P
Alcohol	298	1	0.48	0.89	2.03	0.02
		2	0.40	0.88
Tobacco	84	1	0.57	0.83	17.67a	<0.001
		2	0.38	0.76
Cannabis	51	1	0.76	0.94	7.88a	0.11
		2	0.66	0.80
Cocaine	11	1	0.16	0.91	3.67a	0.002
		2	0.53	0.73
Shopping	369	1	0.31	0.87	0.73	0.23
		2	0.27	0.88
Gaming	185	1	0.32	0.80	−1.30	0.90
		2	0.37	0.78
Gambling	175	1	0.34	0.78	1.77	0.04
		2	0.25	0.75
Overeating	121	1	0.48	0.74	3.93a	0.16
		2	0.50	0.75
Sexual activity	268	1	0.42	0.91	14.21a	<0.001
		2	0.50	0.93
Overworking	139	1	0.40	0.81	4.80a	0.02
		2	0.36	0.82

Robust likelihood ratio test statistic calculated due to nested models. For each behavior model 1 included the SSBA subscale score, and model 2 included the extant screening measure. Except for cocaine (due to low rates of reported cocaine use) age, gender, employment, education, marital status and impulsivity were included in each model as covariates. Classification accuracy was calculated from the models which included the covariates.

Discussion

The aim of this report was to examine and report the accuracy and predictive utility of the SSBA as a screening measure for self-attributed problems across potentially addictive behaviors. AUC values were moderate-to-high across the target behaviors, demonstrating the overall ability of each subscale to discriminate between individuals who did and did not self-report problematic engagement in a target behavior. The determined threshold score was 3 for six of the target behaviors (alcohol, tobacco, cannabis, cocaine, shopping, and gaming), and 2 for the remaining four behaviors (gambling, eating, sexual activity, and working). Given the purpose of the measure is to identify people who would benefit from further assessment, the lower threshold scores limit the number of people incorrectly screened as negative. Further, optimal threshold scores demonstrated good sensitivity and specificity, indicating acceptable identification of people with and without self-attributed problematic engagement in potentially addictive behaviors.

The PPV indicates the percentage of individuals falling above the established threshold that do show problematic engagement. As the base rate occurrence of the behavior diminishes, accurate classification of the presence of the problematic behavior becomes more difficult. Prevalence estimates of addictive behaviors vary, depending on diagnostic criteria used. Gambling disorder is the most studied nonsubstance related addiction and prevalence estimates range from 1 to 2% of the general population based on formal diagnostic criteria (Welte, Barnes, Wieczorek, Tidwell, & Parker, 2001) to 3.4% of individuals self-reporting problematic addictive behavior (Konkolÿ Thege et al., 2015) and 3.8% of self-reported gambling problems (Afifi, Cox, Martens, Sareen, & Enns, 2010). Regardless, the recommended SSBA threshold improves upon the accuracy of this base rate, with a PPV of 54%; for an individual who screens positive on the SSBA, the probability that they self-report problematic gambling is 0.54, compared to the base rate probability of 0.038. In addition, the NPVs were high across all behaviors, indicating that these scores correctly identify most individuals who do not self-attribute problematic addictive behaviors.

For a given behavior, the proportion of true positives indicates the number of positive cases identified by a predictor variable (problem attribution, need to change, or help seeking) and who meet the SSBA subscale threshold for that behavior among individuals who had engaged in the behavior in the prior 12-months. For example, the proportion of true positives for a self-attributed problem with alcohol in the present study was 0.12 (Table 3), indicating a prevalence of 12% among current drinkers. In 2016, approximately 12.3% of the Canadian population age 15+ that reported drinking in the past year also experienced problematic alcohol use (calculated as the number of Canadians with problematic drinking divided by the number of Canadians who consumed alcoholic beverages in the preceding 12-months; World Health Organization, 2018). Similarly, the proportion of true positives for a self-attributed problem with cannabis use was 0.10, and research suggests that approximately 9% of individuals who use cannabis at least once will become dependent (Volkow, Baler, Compton, & Weiss, 2014). The similar prevalence rates of self-attributed problems in the present study lend further support to the validity of the recommended threshold scores.

Comparison of the SSBA to extant screeners also provided support for the utility of the SSBA. In six out of the 10 behaviors (alcohol, tobacco, cocaine, gambling, sex, and work) model one (which included the SSBA subscale score) provided a significantly better fit than model two (which included the external screening measure). In the other four cases, the models were equivalent, though the accuracy of model one tended to be slightly higher than that of the second model. Across all behaviors, logistic regressions indicated that SSBA scales were 4.2% more accurate than the clinical screening instruments. This is particularly advantageous for assessing these target behaviors when considering the length of the clinical validation tools (3–25 items, M = 14), compared to the use of only four items per behavior (40 total) on the SSBA. The use of four items per behavior is also advantageous over singleitem assessments of problem attribution. Multiitem screeners with multiple response options offer increased reliability and the ability to use continuous scores to reflect a range of problem severity.

Screening measures with heterogeneous items for different behaviors may offer an advantage over measures such as the SSBA because items can be tailored to better suit a single behavior. For example, the SSBA item “I did it too much” may have greater face validity when assessing gambling versus overeating. We previously determined that the four items included in the SSBA do tap into a single construct across the target behaviors (Schluter et al., 2018). Additionally, content validity was used as a criterion when developing the SSBA, which suggested that items from four major domains should be included.

Limitations

This study has several limitations. First, all target behaviors were measured with the same questions, resulting in some shared method variance across behavioral screens. However, this feature allows us to measure a single underlying addiction construct across all behaviors. Second, despite the large sample size, some target behaviors such as cocaine had very low rates of endorsement, resulting in small numbers for the ROC analyses. Third, we were limited in our selection of appropriate screening measures with which to compare the SSBA. The external measures needed to demonstrate acceptable psychometric properties in the literature and a recommended threshold score to support their use as a screening tool. Fourth, the current study involved the same sample from which the SSBA was initially developed. However, the logistic regressions used the total SSBA scores to predict problem attribution (versus threshold scores), which mitigates the risk that the SSBA's performance could be inflated when compared to the other screening instruments. Nevertheless, additional research should examine the SSBA's performance in additional samples. Finally, self-identified need for behavior change, problem attribution, and help seeking were used to indicate a positive case. Without corroborating information such as a clinical interview, it is difficult to confirm whether participants had, in fact, experienced a problem with an addictive behavior. As with all psychometric scales, further validation data may lead to greater confidence in the scale or to refinement. It will be important to cross validate these results in community samples, and calculation of threshold scores for clinical samples (with higher base rates) would also be a valuable future step. Similarly, inclusion of other problematic behaviors in the scale (e.g., exercise) would require validation.

Conclusion

In sum, the SSBA appears to be a viable screening measure of self-attributed problematic engagement across ten potentially addictive behaviors. Moreover, its accuracy and brevity may lend itself to more efficient screening in the general population than current measures and may be used to better align treatment services with population need. The measure demonstrated good accuracy at detecting self-attributed problematic engagement in potentially addictive behaviors, perceived need to change the behavior, and treatment-seeking in the prior 12 months. The recommended threshold scores showed good predictive ability (as indicated by the sensitivity, specificity, PPV, and NPV). When the SSBA's performance was compared to that of established clinical screeners it demonstrated roughly equivalent, if not slightly better accuracy. The use of four items per behavior also provides a range of scores that may prove useful as a measure of severity of problematic involvement, although future research will need to examine this possibility. The range of scores will also likely facilitate alternative thresholds for other populations, including clinical samples.

Funding sources

The research was funded by a grant from the Palix Foundation.

Author's contribution: MGS: conceptualization, methodology, formal analysis, writing; DCH: conceptualization, methodology, investigation, writing, supervision; BKT: conceptualization, formal analysis, writing; TCW: conceptualization, investigation, writing.

Conflict of interest: The authors declare no conflict of interest.