Predicting tobacco product initiation from intentions to use: Comparing the validity of item analysis methods.

Items measuring tobacco use intentions are used to predict future use. Researchers combine items using different methods; however, no research has compared these methods' predictive validity. Here, we compare how well six methods of analyzing four intention items predict initiation of cigarettes, e-cigarettes, snus pouches, and other smokeless tobacco one year later. We analyzed youth and young adult never users from the US Population Assessment of Tobacco and Health Study. We compared six methods of analyzing Wave 3 intention items in predicting Wave 4 use: susceptibility scoring (susceptible is not answering "definitely no" to all items); dichotomizing the four-item average using two cut-points on the 1-4 response scale; and dichotomizing one item (next year use intention) with three cut-points. Analyses (1) tested whether each single-item predicted initiation; and (2) compared each method's (a) true positive rate (rate of correctly identifying future initiators), (b) true negative rate (rate of correctly identifying future non-initiators), and (c) model fit. Results were similar across products and age groups. Averaging items best predicted initiation in regression. Susceptibility scoring had the highest true positive rate but lowest true negative rate. False positives (incorrectly identifying someone as a future initiator) were best minimized by averaging items with a cutoff of 3, or using the single item with a 3 or 4 cutoff. Findings suggest researchers predicting tobacco use initiation using regression should average the four items; and researchers seeking to identify likely initiators should use different analytic methods depending on if they seek to maximize true positives or minimize false positives.

Introduction

Tobacco researchers, advocacy groups, and regulators seek to predict whether people will initiate tobacco use. They seek to understand how product features, advertisements, tobacco prevention campaigns, and other factors affect people’s likelihood of using tobacco products for the first time (e.g., Altman et al., 1996, Bunnell et al., 2015, Evans et al., 1995, Feighery et al., 1998, Pierce et al., 2017, Unger et al., 1995, Weiss et al., 2006). For example, regulators seek to estimate the number of people who may start using a tobacco product newly introduced to the market (e.g., FDA, 2019), and public health educators seek to understand tobacco never-users’ likelihood of using tobacco to better target interventions (e.g., Huang et al., 2005, Lipkus et al., 2015).

One tool used to predict product initiation is asking study participants to report their intentions to purchase, try, or use a product. For example, participants may be asked, “Do you think you will smoke a cigarette in the next year?”, by providing a response on a scale from “Definitely not” to “Definitely” (e.g., Bunnell et al., 2015).

Although such items have been found to predict subsequent use of various tobacco product types (e.g., Orlan et al., 2019, Pierce et al., 1995, Seo et al., 2020;), there is no consensus on how to analyze intentions to best predict product initiation. In public health research, studies typically use susceptibility scoring to understand which people are likely to initiate product use. Susceptibility is defined as “the absence of a firm resolve” to remain a never user (Pierce et al., 1995, p. s47; Strong et al., 2015, p. 863). When people are asked multiple questions about their intentions to use (e.g., likelihood of trying the product if offered by a friend), people are classified as susceptible to using the product unless they respond “definitely not” on every question (e.g., Choi et al., 2001, Pierce et al., 1995, Pierce et al., 2005). Prior research has found that, depending on which items are included, this classification method identified roughly 40–80% of adolescent never-smokers who tried cigarettes in the next three to four years (Nodora et al., 2014, Pierce et al., 2005) and roughly 40–80% of adolescent never-smokers who became established cigarette smokers four to six years later (Nodora et al., 2014, Strong et al., 2015). Susceptibility also predicted adolescents’ and young adults’ initiation of cigars, e-cigarettes, hookah, and smokeless tobacco (Orlan et al., 2019, Pierce et al., 2018, Seo et al., 2020). For example, susceptibility to e-cigarette use predicted whether nicotine-naïve US adolescents tried e-cigarettes within the next year, with a sensitivity (true positive rate) of 57% and a specificity (true negative rate) of 73% (Seo et al., 2020). Susceptibility to use tobacco products has also been linked to perceived product risks (Gentzke et al., 2019), tobacco advertising receptivity (Evans et al., 1995, Feighery et al., 1998, Pierce et al., 2017, Unger et al., 1995), and exposure to pro- and anti-tobacco messages (Altman et al., 1996, Weiss et al., 2006).

In contrast, tobacco companies have often used different methods of analyzing use intentions. In some applications submitted to FDA to market products, companies have submitted research in which participants were considered likely to use the products if they selected the top two categories of a six-point response scale (e.g., “Very likely” or “Definitely”) on questions about their likelihood of trying the product, trying the product if one of their best friends offered it to them, and using the product regularly if they tried and liked it. The stated rationale for this method of analyzing the data is that it will identify people who have a relatively strong interest in the product while minimizing false positives. Companies have argued that the top-two box percentage is commonly used in marketing research and has been shown to predict likelihood of trial of other product types. In contrast, public comments from tobacco researchers have argued that the top-two box percentages will underestimate future product uptake because many people lack an intention to use the product but nonetheless would be susceptible to using it. The comments pointed out that many decisions to try products may be un-planned and thus may not be captured by top-two box percentages and argued that people should be considered at-risk of initiating use of products if they responded by saying anything other than “definitely not.”.

Other tobacco applications and research have analyzed intentions by calculating average responses or using single items without using any cut-points or dichotomization (e.g., Hines et al., 2000, Katzet al., 2018, Shadel et al., 2020, Stroup and Branstetter, 2018). The advantage of this approach is that it avoids dichotomizing items, which can reduce statistical power by eliminating meaningful variation in the data (Altman and Royston, 2006, MacCallum et al., 2002, Royston et al., 2006, Streiner, 2002). For example, using the top-two box percentage treats participants as equally likely to use the product if they respond at the bottom (e.g., “definitely not”) or near the bottom (e.g., “very unlikely”) of the scale, and equally likely to use the product if they respond at the top of the scale (e.g., “definitely”) or near the top of the scale (e.g., “very likely”). Analyses of mean scores will capture such differences, which is important if such differences reflect meaningful variation in people’s propensity to use products.

We compared the predictive validity of intentions when analyzed in several ways: a single-item measure using top one- or two-box scoring or susceptibility scoring; a four-item measure using various cut-offs or susceptibility scoring; and a four-item measure scored as the mean. We conducted analyses for four product types varying in use prevalence (cigarettes, e-cigarettes, snus, and other smokeless tobacco) and for youth and young adult never users of products. Analyses (1) tested whether each single-item was valid for predicting product initiation (monotonically associated with likelihood of future initiation); and (2) compared each method of scoring intentions in terms of its (a) true positive rate (i.e., rate of correctly identifying people who later initiate product use), (b) true negative rate (i.e., rate of correctly identifying people who do not later initiate), and (c) model fit when predicting future product initiation.

Method

Data source

The Population Assessment of Tobacco and Health (PATH) Study is an ongoing, nationally-representative, longitudinal cohort study of U.S. adults and youth. We analyzed youth and adult data from Waves 3 and 4, collected approximately one year apart from October 2015-October 2016 and December 2016-January 2018, respectively, and made available in the Public Use File (US DHHS, 2019; https://doi.org/10.3886/Series606). These were the most recent waves that were available at the time of analysis.

The PATH Study recruitment employed a stratified, address-based, area-probability sampling design that oversampled adult tobacco users, African Americans, and young adults (18–24 years). Audio computer-assisted self-interviews were conducted in-person with 28,148 adults and 11,814 youth at Wave 3, and 27,757 adults and 11,059 youth at Wave 4, with an overall weighted response rate of 73.5% (adults) and 79.5% (youth) at Wave 4. The PATH Study was conducted by Westat and approved by Westat’s institutional review board. Further details and study instruments are available elsewhere (Hyland et al., 2017; https://doi.org/10.3886/Series606).

The study’s longitudinal design allowed us to assess whether youth and young adults’ intentions to use tobacco products at Wave 3 predicted whether they used the products within the next year, by Wave 4.

Measures

Intentions to use

At Wave 3, participants reported their intentions to use cigarettes, e-cigarettes (“electronic nicotine products”), snus pouches, and other smokeless tobacco. For each product, the intention items were:

Do you think you will [smoke/ try/ use] [product type] in the next year?

Do you think that you will try [product type] soon?

If one of your best friends were to offer you [product type], would you [smoke/ try] it?

Have you ever been curious about [smoking/ using] [product type]?

Response options for the first three items were: 1 = definitely not, 2 = probably not, 3 = probably yes, and 4 = definitely yes, and for the fourth item were: 1 = not at all curious, 2 = a little curious, 3 = somewhat curious, and 4 = very curious. When the four items were averaged to create a single index for each product type (using the values [1-4] just noted), internal consistency reliability was high: unstandardized Cronbach alphas calculated using unweighted data in our analytic sample ranged from 0.78 to 0.86 across age groups and product types.

Product initiation

At Wave 4, for each product type, youth were asked: “In the past 12 months, have you [smoked/ used] a [cigarette/ electronic nicotine product/ snus/ smokeless tobacco] even one or two [puffs/ times]?” The words “smoked” and “puffs” were only used for cigarettes. Electronic nicotine products and smokeless tobacco were further described in the survey (US DHHS, 2019).

At Wave 4, young adults were first asked the same question, but about “past 30 days” instead of the “past 12 months.” Participants who did not respond “yes” were asked the question about the past 12 months.

For each product type, youth and young adults were classified as having used the product if they responded “yes” on the item about the past 12-months or 30-days (respectively).

Sample

The analytic sample includes youth (ages 12–17) and young adults (ages 18–24) who were classified as never using each product type at Wave 3, who reported their intentions to use the product on all four items at Wave 3, and who reported whether they used the product at Wave 4. Never use was defined independently for each product type (i.e., never users of each product type could have reported previously using one or more of the other product types). Sample sizes for youth never users were: nCIGARETTES = 8139; nE-CIGARETTES = 7298; nSNUS POUCHES = 7146; nOTHER SLT = 7777. Sample sizes for young adult never users were: nCIGARETTES = 3857; NE-CIGARETTES = 4054; nSNUS POUCHES = 6503; nOTHER SLT = 6261.

Analyses.

For each product, we analyzed Wave 3 intention scores in several ways to predict product initiation one year later:

single-item methods: analyzing the single-item (“Do you think you will [smoke/ try/ use] [product type] in the next year?”) using cutoffs of “probably not,” “probably yes,” and “definitely yes.”

four-item average using cutoffs of mean greater than 1 (i.e., susceptibility scoring), mean ≥ 2, mean ≥ 3, mean = 4, or without dichotomizing (i.e., continuous mean score).

Predictive validity analyses examined how well these intention scores predicted Wave 4 product use, using descriptive statistics and binary logistic regressions in SAS Version 9.4 and SUDAAN 11.0.3. Analyses were weighted to produce nationally representative estimates and to account for the PATH Study's sampling design, using the Wave 4 longitudinal full-sample weights for the Wave 1 cohort. The balanced repeated replication (BRR) method with Fay's adjustment (0.3) was used (US DHHS, 2019). Regressions did not adjust for demographics as we sought to capture the predictive utility of intentions without regard for whether the intentions caused or uniquely explained subsequent product initiation. We conducted all analyses separately for youth and young adults.

Results

All results are presented separately for each product type and each age group (youth and young adults).

Descriptive statistics

Table 1 shows the percentages of people in each response category on the single-item measure (intentions to use in the next year and the mean of the four-item measure. Intentions to use were lowest for snus and other SLT and highest for e-cigarettes.

Table 1

Weighted percentages (and 95% CIs) in each intention category at Wave 3 of the PATH Study.

Product Type	Youth
	Single Intention Item (use in next year)				Mean of 4 Item Intention Measure
	1	2	3	4	1.00(non-susceptible)	1.01–1.99	2.00–2.99	3.00–3.99	4.00
Cigarette	88.6(87.7–89.5)	10.6(9.9–11.4)	0.7(0.5–0.9)	0.1 †(0.0–0.2)	71.2(70.1–72.2)	22.8(21.9–23.8)	5.7(5.2–6.4)	0.3(0.2–0.4)	0.0 †(0.0–0.0)
E-Cigarette	86.0(85.1–87.0)	12.7(11.8–13.6)	1.2(0.9–1.5)	0.1 †(0.1–0.3)	72.3(71.1–73.5)	19.1(18.0–20.2)	8.0(7.4–8.7)	0.5(0.4–0.7)	0.1 †(0.0–0.3)
Snus	96.1(95.7–96.6)	3.7(3.2–4.1)	0.2(0.1–0.4)	0.0 †(0.0–0.1)	91.2(90.5–91.8)	7.2(6.7–7.8)	1.5(1.3–1.9)	0.1 †(0.0–0.2)	0.0 †(0.0–0.1)
Other SLT	95.2(94.6–95.6)	4.6(4.1–5.1)	0.2 †(0.1–0.4)	0.1 †(0.0–0.2)	89.0(88.2–89.6)	8.7(8.0–9.3)	2.3(2.0–2.7)	0.1 †(0.1–0.3)	–
Product Type	Young Adults
	Single Intention Item (use in next year)				Mean of 4 Item Intention Measure
	1	2	3	4	1.00(non-susceptible)	1.01–1.99	2.00–2.99	3.00–3.99	4.00
Cigarette	85.6(84.0–87.1)	13.8(12.4–15.4)	0.5(0.3–0.7)	0.1 †(0.0–0.4)	68.2(66.3–70.1)	25.3(23.7–26.8)	6.3(5.4–7.4)	0.1 †(0.1–0.3)	0.1 †(0.0–0.5)
E-Cigarette	78.9(77.1–80.6)	19.6(18.0–21.3)	1.2(0.9–1.6)	0.3 †(0.1–0.6)	62.6(60.5–64.7)	24.0(22.1–26.0)	12.6(11.3–14.1)	0.7(0.5–1.1)	0.1 †(0.0–0.4)
Snus	95.0(94.3–95.6)	4.8(4.2–5.4)	0.2(0.1–0.4)	0.0 †(0.0–0.1)	90.6(89.7–91.4)	7.5(6.8–8.3)	1.7(1.5–2.1)	0.2 †(0.1–0.3)	0.0 †(0.0–0.1)
Other SLT	95.3(94.6–95.8)	4.5(4.0–5.2)	0.2 †(0.1–0.4)	0.0 †(0.0–0.1)	90.6(89.6–91.5)	7.1(6.2–8.1)	2.3(1.9–2.7)	0.1 †(0.0–0.2)	–

† Estimate should be interpreted with caution because it has low statistical precision. It is based on a denominator sample size of less than 50, or the coefficient of variation of the estimate or its complement is larger than 30%.

Table 2 shows rates of product initiation (past year use) reported at Wave 4. As observed for intentions to use, rates of initiation were lowest for snus and other SLT and highest for e-cigarettes.

Table 2

Rates of product use initiation (past 12-month use) at Wave 4 among Wave 3 never users of products in the PATH Study (2015–2018).

Product Type	Wave 4 Product Initiation Rates: % (95% CI)
	Youth	Young Adults
Cigarettes	3.4 (3.0,3.9)	6.9 (6.0,7.9)
E-Cigarettes	6.0 (5.4,6.7)	7.4 (6.4,8.4)
Snus	0.6 (0.4,0.8)	1.5 (1.2,1.9)
Other SLT*	1.2 (1.0,1.5)	1.6 (1.3,2.1)

Note. Initiation was operationalized as moving from being a never user of that product type to being an ever user of that product type, ignoring other tobacco products.

*SLT stands for smokeless tobacco.

Monotonic associations

Fig. 1 shows initiation rates at Wave 4 among never users who selected each response option (1–4) on each of the four intention items at Wave 3. Results were similar for both age groups. For each item, rates of initiation generally appeared higher at each progressively higher level of intentions. That is, there appeared to be a positive monotonic association between intentions to use at Wave 3 and rates of initiation at Wave 4, with no clear thresholds or disjoints. However, for rarely used product types (snus and other SLT), intentions were very low, which limited the sample sizes of participants in the top-two box categories for each item and limited the statistical precision of the estimated initiation rates in those categories.

Fig. 1

Initiation rates (weighted percentages) for each product type at Wave 4 among never users of that product who selected each response option (1–4) on the four intention items at Wave 3 of the PATH Study. Note: For the first three intention items, 1 = “Definitely not”; 2 = “Probably not”; 3 = “Probably yes”; 4 = “Definitely yes.” For the “Curious about” item, 1 = “Not at all curious”; 2 = “A little curious”; 3 = “Somewhat curious”; 4 = “Very curious.” † Estimate should be interpreted with caution because it has low statistical precision. It is based on a denominator sample size of less than 50, or the coefficient of variation of the estimate or its complement is larger than 30%. ‡ Denominator sample size of less than 5.

True positives and negatives

Fig. 2, Fig. 3 show the predictive utility that would be achieved by using various thresholds for classifying people as likely to use products at Wave 4. For the mean of the four items, we examined three potential cutoff values: classifying people as likely to use a product type if their mean intention was (a) above 1 (susceptibility scoring), (b) greater than or equal to 2, and (c) greater than or equal to 3. For the single item, we also examined three potential cutoff values: classifying people as likely to use a product type if they responded (a) “Probably No” or greater, (b) “Probably Yes” or greater, and (c) “Definitely Yes.”.

Fig. 2

True positive rates (blue) and false negative rates (red): Among people who used the product type at Wave 4, how many would be correctly identified as future users (true positives) vs. incorrectly identified as future non-users (false negatives) by each classification measure. Note: When predicting product use with each of the six methods, we dichotomized at the level indicated. For example, the 4 Item Susceptibility method predicted that participants would use the product type if they fell into the susceptible category at Wave 3 (i.e., if they responded greater than 1 on any of the four intention items). Participants were classified as having used the product type at Wave 4 if they reported using the product type within the past 30 days or 12 months at Wave 4. † Estimate should be interpreted with caution because it has low statistical precision. It is based on a denominator sample size of less than 50, or the coefficient of variation of the estimate or its complement is larger than 30%. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 3

True negative rates (blue) and false positive rates (red): Among people who did not use the product type at Wave 4, the percent who would be correctly identified as future non-users (true negatives) vs. incorrectly identified as users (false positives) by each measure. Note: When predicting product use with each of the six methods, we dichotomized at the level indicated. For example, the 4 Item Susceptibility method predicted that participants would use the product type if they fell into the susceptible category at Wave 3 (i.e., if they responded greater than 1 on any of the four intention items). Participants were classified as not using the product type at Wave 4 if they reported not using the product type within the past 12 months at Wave 4. † Estimate should be interpreted with caution because it has low statistical precision. It is based on a denominator sample size of less than 50, or the coefficient of variation of the estimate or its complement is larger than 30%. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 2 shows the true positive and false negative rates for each potential cutoff value. Among people who initiated use of each product type at Wave 4, this figure indicates what percentage would have been correctly identified as future users (true positives) vs. incorrectly identified as future non-users (false negatives) by each analysis method. Susceptibility scoring showed the highest true positive rate, identifying most people who later initiated cigarettes and e-cigarettes (between 53.8 and 70.3% across age groups and product types) and a minority of people who later initiated snus and other SLT (between 19.1 and 42.7%). The single item measure failed to identify the majority of future product users even when using the lowest possible cutoff value: Across product types, 53.5–91.7% of youth and 55.6–80.6% of young adults who later used each product type at Wave 4 had, one year earlier, stated that they would “Definitely Not” use it in the next year. Using the four-item mean or single item with cutoff values of 2 or 3 had very low true positive rates, failing to correctly identify almost any future users of product types.

Fig. 3 shows the true negative and false positive rates for each potential cutoff value. This figure depicts, among people who did not initiate use of each product type at Wave 4, the percentage that would have been correctly identified as future non-users (true negatives) vs. incorrectly identified as users (false positives) by each analysis method. For cigarettes and e-cigarettes, susceptibility scoring correctly identified between two-thirds and three-quarters of people who did not initiate product use, and incorrectly identified about one-quarter to one-third of people as users (note that those people could still start using the products later). For snus and other SLT, susceptibility scoring correctly identified approximately 90% of non-users. Using higher cutoff values or using the single item measure with any cutoff had the highest true negative rates across products and age groups (Fig. 3).

Predictive validity

Table 3 shows the results of using Wave 3 intentions to predict Wave 4 product initiation in logistic regressions. We compared three methods of scoring intentions: (a) susceptibility scoring the four items, (b) taking the mean of the four items, and (c) using the most relevant single item (likelihood of use in the next year). All three methods predicted subsequent product initiation for all four product types and both age groups (youth and young adults). For youth for cigarettes and e-cigarettes, the mean of the four items appeared to be the best predictor, with slightly better model fit (based on Cox & Snell Pseudo-R) than susceptibility scoring and the single item. For young adults, for cigarettes, the four item mean and the single-item (use in next year) performed similarly well for cigarettes; for e-cigarettes, the four item mean and the susceptibility scoring method performed similarly well. For youth, all the methods were similarly poor predictors of snus use (Cox & Snell R <=0.001). For young adults, the four-item susceptibility measure accounted for slightly more approximate variance in snus initiation. All methods performed similarly for other SLT use in both age groups. Overall, the four-item mean method performed most consistently in terms of model fit in predicting initiation.

Table 3

Simple weighted logistic regressions of Wave 3 intentions predicting Wave 4 product initiation.

Product Type		Youth		Young Adults
	Measure	OR (95%CI)	Cox-SnellPseudo-R2 †	OR (95%CI)	Cox-SnellPseudo-R2 †
Cigarettes	4 Items Scored as Susceptibility	4.9 (3.7,6.3)	0.020	2.7 (2.1,3.5)	0.015
	4 Item Mean	6.4 (5.1,8.1)	0.029	4.7 (3.5,6.1)	0.029
	Single-Item (Use in Next Year)†	4.5 (3.5,5.8)	0.022	3.8 (2.9,4.9)	0.028
E-Cigarettes	4 Items Scored as Susceptibility	6.3 (4.9,7.9)	0.044	4.4 (3.3,6.0)	0.035
	4 Item Mean	5.7 (4.7,6.9)	0.052	3.5 (2.8,4.5)	0.034
	Single-Item (Use in Next Year)†	4.3 (3.6,5.2)	0.038	2.7 (2.1,3.5)	0.022
Snus	4 Items Scored as Susceptibility	2.5 (0.8,7.5)	0.001	6.0 (4.0,9.1)	0.009
	4 Item Mean	3.0 (0.9,9.8)	0.001	5.3 (3.5,7.8)	0.007
	Single-Item (Use in Next Year)†	1.9 (0.5,7.4)	< 0.001	3.9 (2.5,5.9)	0.004
Other SLT	4 Items Scored as Susceptibility	6.2 (4.1,9.5)	0.008	5.4 (3.2,9.3)	0.008
	4 Item Mean	6.9 (4.5,10.7)	0.009	6.8 (4.0,11.3)	0.009
	Single-Item (Use in Next Year)†	6.0 (3.5,10.3)	0.009	5.5 (3.3,9.2)	0.008

Note: Cox-Snell pseudo-R cannot be interpreted as variance explained. Initiation was operationalized as moving from being a never user of that product type to being an ever user of that product, ignoring other tobacco products.

† Retaining all four response options.

Discussion

Tobacco researchers, educators, and regulators assess people’s intentions in surveys to predict future tobacco use. This study is the first to compare analytic methods to help determine how to maximize the utility of four commonly used intentions items (e.g., Choi et al., 2001, Pierce et al., 1995, Pierce et al., 2005), using two populations vulnerable to tobacco use (youth and young adults) and examining use of four tobacco products (cigarettes, e-cigarettes, snus, and other SLT). These methods include using a single-item measure (intention to use in the next year) with top one- or two-box scoring or susceptibility scoring, a multi-item measure with various cut-offs or susceptibility scoring, and a multi-item measure scored as the mean with no cutoffs. While all methods demonstrated some predictive validity among youth and young adults—they all predicted tobacco initiation—the best method to use may depend on the goals of the research.

We examined several methods for predicting youth and young adults who are likely to use a tobacco product one year later. Researchers commonly use the susceptibility method of scoring (e.g., Choi et al., 2001, Nodora et al., 2014, Orlan et al., 2019, Pierce et al., 1995, Pierce et al., 2005, Pierce et al., 2018, Seo et al., 2020, Strong et al., 2015), and we found that this method identified the most future users (true positives). However, this method also had the highest rate of false positives (identified as future users but were not after one year; 25–35% for cigarettes and e-cigarettes). Given these findings, this method might be best for identifying target audiences for large-scale, lower cost interventions (e.g., public health campaigns), where there is little downside to being overly inclusive. However, community programs and public health practitioners seeking to identify people at the highest risk for using tobacco while minimizing false positives (e.g., for more expensive interventions aimed at a smaller group, such as mentoring) can use the method of using the single-item and a cutoff of “probably not,” which reduced the rate of false positives to under 20% for all products and age groups.

For researchers who only have space on a survey to include the single item to assess susceptibility, we found that “probably no” is the best cutoff. Using a “definitely yes” or “probably yes” cutoff identified fewer than 5% of future users. While these methods also had the lowest rate of false positives, this suggests analyses using these cutoffs may have little utility in future research seeking to identifying individuals likely to use tobacco.

A considerable shortcoming of all methods we compared is that large proportions of future users were misidentified at Wave 3 as future non-users. This was particularly pronounced for products with lower use prevalence—snus and other SLT—where even susceptibility scoring missed over half of future users. Many statistical estimates for these product types had low statistical precision because of the small number of participants who initiated use.

For research where the goal is to maximize the amount of variance predicted in a statistical model predicting future use, the method of averaging all four items performed well across product types. Therefore, for studies predicting future behavior, testing effects of an intervention, or comparing intentions between groups, averaging the four items appears most supported by study findings. However, for snus and other SLT, the susceptibility method and using a single item also performed equally well. Thus, it may be worthwhile for studies of these products to use the single-item measure to reduce respondent burden.

For research using intentions data to model population effects, we found there is predictive value in including each response option of the single item measure rather than only looking at top one or two categories, as each response option was monotonically associated with future tobacco use (i.e., each ascending response option was associated with a higher probability of use during the next year).

Our findings were similar between youth and young adults, even though these measures were first developed for youth (Pierce et al., 1995). This suggests that these items may be equally valid for use with young adults and provide some evidence that these measures could be used to compare intentions between these populations.

Strengths, Limitations, and Future Directions.

Strengths of this study include its generalizability: we used nationally-representative samples of both youth and young adults and compared analytic methods for four tobacco product types.

A key limitation is that we focused on predicting ever-use within one year. Other studies could examine predictions of behavior farther into the future or different behavioral outcomes. As we found that a large proportion of wave 4 tobacco users were misidentified by all methods at wave 3, future research could also seek to develop measures that more effectively predict future tobacco use. For example, a future analysis might evaluate the utility of intention items for one product type in predicting likelihood of using other product types, above and beyond the variance explained by intention items for those product types. Alternate items and approaches may be especially useful to investigate for snus and other SLT given their particularly high false negative rates.

Future research can also replicate our study over time and with other product types (e.g., cigars), as the predictive utility of these measures may vary along with changes in product use rates and intentions to use. Still, this study can provide a foundation for researchers to better understand and leverage the predictive utility of survey items measuring use intentions, across tobacco products and among youth and young adults.

Disclaimer

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Food and Drug Administration.

Funding statement

This manuscript is supported with Federal funds from the National Institute on Drug Abuse, National Institutes of Health, and the Center for Tobacco Products, Food and Drug Administration, Department of Health and Human Services, under contract to Westat (Contract No. HHSN271201100027C and HHSN271201600001C).

CRediT authorship contribution statement

Alexander Persoskie: Conceptualization, Methodology, Formal analysis, Writing – original draft, Writing – review & editing, Visualization. Erin Keely O'Brien: Conceptualization, Methodology, Writing – original draft, Writing – review & editing, Visualization.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.