Effectiveness of an optimized text message and Internet intervention for smoking cessation: A randomized controlled trial.

AIMS: To evaluate the effectiveness of a combined internet and text message intervention for smoking cessation compared with an internet intervention alone. The text message intervention was optimized for engagement in an earlier multiphase optimization (MOST) screening phase.
DESIGN: A parallel, two-group, individually randomized clinical trial (RCT) was conducted in a MOST confirming phase. Recruitment spanned December 2018 to March 2019. Follow-up was conducted at 3 and 9 months, beginning March 2019 and ending January 2020.
SETTING: United States: a digital study conducted among new registrants on a free tobacco cessation website. PARTICIPANTS: Eligible individuals were 618 adult current smokers in the United States, age 18 years or older who signed up for text messages during website registration (67.2% female, 70.4% white).
INTERVENTIONS: The treatment arm (WEB+TXT; n = 311) received access to the website and text messaging. The control arm (WEB; n = 307) received access to the website alone. MEASUREMENTS: The primary outcome was self-reported 30-day point prevalence abstinence (ppa) at 9 months post-randomization analyzed under intent to treat (ITT), counting non-responders as smoking. Secondary outcomes included 3-month measures of 30-day ppa, intervention engagement and intervention satisfaction.
FINDINGS: Abstinence rates at 9 months were 23.1% among WEB+TXT and 23.2% among WEB (OR = 1.00, 95% CI = 0.69-1.45; P = 0.99). WEB+TXT increased engagement with 5 of 6 interactive features (standardized mean difference (SMD) = 0.26-0.47, all P < 0.001) and repeat website visits (48.7% vs 38.9%, SMD = 0.14, P = 0.02). Satisfaction metrics favored WEB+TXT (satisfied: 96.3% vs 90.5%, SMD = 0.17, P = 0.008; recommend to friend: 95.9% vs 90.1%, SMD = 0.16, P = 0.028).
CONCLUSIONS: A randomized controlled trial found no evidence that a combined internet and text message intervention for smoking cessation compared with an internet intervention alone increased 9-month abstinence rates among adult current smokers in the United States, despite evidence of higher levels of intervention engagement and satisfaction at 3 months.

INTRODUCTION

Internet interventions are effective for smoking cessation [1, 2] and a recommended treatment approach [3]. Less studied are ways to further increase their effectiveness [2]. Given the well documented dose–response relationship between treatment intensity and outcomes in traditional cessation interventions [4], it is expected that increasing levels of treatment engagement with digital interventions may yield better outcomes. Optimizing engagement with internet behavior change interventions has been noted as a research priority [5, 6, 7, 8].

Text messaging is a promising channel for optimizing engagement. Nine of ten United States (US) adults own a mobile phone [9, 10], and text messaging is a preferred communication modality among mobile phone owners [11, 12]. It is a “push” technology that delivers intervention content to a user's phone without effort from the recipient. Text messages have a 98% open rate, typically within 3 minutes, and a response rate double that of phone, email, or social media [13]. Text messaging has been used successfully to deliver behavior change techniques (e.g. education, prompts and cues, goal setting, and support [14, 15, 16, 17, 18, 19]) and is a recommended modality for smoking cessation [3, 20, 21, 22]. It has been used to increase uptake of Quitline services [23], but has not been tested as a strategy to increase engagement with an internet cessation program.

In phase I of this research [24], we evaluated the impact of four text message design factors on adult smokers' engagement with an internet smoking cessation program. Guided by the multiphase optimization strategy (MOST) [25, 26], we conducted a factorial screening experiment to identify the optimal combination of factors that could maximize intervention engagement. The factors tested were: personalization (on/off), to enhance message relevance [15]; integration (on/off), to enable engagement via web and/or text message [27]; dynamic tailoring (on/off), to tailor messages based on program engagement over time [28]; and message intensity (tapered/abrupt drop‐off), to influence message salience based on delivery schedule [15]. Primary outcomes were 3‐month measures of engagement measured by automated tracking data. The greatest levels of engagement occurred with (i) message personalization on, (ii) integration on, (iii) dynamic tailoring on, and (iv) tapered message intensity [29].

This randomized controlled trial represents the second phase of this research, designed to evaluate the effectiveness of a combined internet and text message intervention compared to an internet intervention alone. We hypothesized that the combined intervention would yield higher quit rates than an internet intervention alone, and that this effect would be mediated by higher levels of engagement.

METHODS

Trial design

This study was a blinded, parallel, two‐group, individually randomized clinical trial that compared an internet and text message smoking cessation intervention (WEB+TXT) to an internet intervention alone (WEB) among adult smokers. The study was prespecified in the trial protocol [24] and registered on ClinicalTrials.gov on October 23, 2015. Drawing on our own research [30, 31, 32] and that of others [33, 34], we conservatively estimated that the 30‐day point prevalence abstinence (ppa) rate at 9 months for WEB would be 9% under intent to treat (ITT). For WEB+TXT, we conservatively estimated a quit rate of 16.5% under ITT based on several converging lines of evidence showing (i) the addition of text messaging in internet interventions had large effects on behavior change (d = 0.81, k = 4, 95% CI = 0.14, 1.49) [27], and (ii) roughly a doubling of abstinence among those with higher levels of treatment engagement in internet interventions [31, 35], which we expected the addition of text messaging would produce. These quit rates corresponded to an intervention odds ratio (OR) of 2.0, which can be detected with 80% at two‐sided α = 0.05 using n = 300 per study arm (n = 600 total). Results are reported according to Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online TeleHealth (CONSORT‐eHealth) [36] and Template for Intervention Description and Replication (TIDieR) [37] recommendations.

Participants

Participants were new registered users on a free internet smoking cessation program who signed up for text messages during enrollment. Eligibility criteria gathered during website enrollment determined who was invited to participate and included US residence (IP address), current smoking (every day/some days), and age (18 or older). The protocol [24] was approved by Advarra institutional review board (CR00086431).

Recruitment, enrollment, and randomization

Individuals were recruited following registration on BecomeAnEX.org (“EX”), an evidence‐based internet cessation program developed in 2008 by Truth Initiative in collaboration with Mayo Clinic. EX is promoted through paid search advertising and is highly ranked in organic search queries related to quitting smoking. There were no marketing strategies specific to the trial. Interested individuals completed online eligibility screening followed by informed consent. Acceptance of informed consent launched the baseline assessment. Those who completed the baseline and responded to the first system generated text message (confirming mobile number) were randomized in a 1:1 ratio using randomly permuted blocks of size 8 within predefined strata of sex and device type at registration (mobile/desktop). The random allocation sequence was generated by the statistician and automated by the web‐based clinical trials management system. Participants, investigators, and research personnel were masked to treatment allocation. Control participants may have suspected their allocation because they received no text messages supporting their efforts to quit.

Interventions

WEB‐based cessation program (WEB)

Accessible on any web‐enabled device, EX was designed around tobacco dependence treatment guidelines [38], Social Cognitive Theory [39], and the Mayo Clinic model for engaging tobacco users in cessation treatment [40]. At the time of this study, users could engage with six interactive features: (i) “Set Quit Date” assists users in selecting a quit date; (ii) “Track Smoking Triggers” allows users to track cigarettes and identify personal smoking triggers; (iii) “Beat Smoking Triggers” encourages identification of strategies to dissociate cigarettes from triggers; (iv) “Choose Quit Smoking Aid” educates users about medication and helps them create a medication plan; (v) “Build Support System” discusses the importance of social support and assists users in identifying supportive friends/family; and (vi) “EX Community” introduces users to a large online social network for smoking cessation in existence for over 12 years.

WEB + text messaging intervention (WEB+TXT)

Designed around tobacco cessation treatment guidelines [38] and Social Cognitive Theory [39] like the EX website, the text message program is designed to enhance motivation for quitting; educate users about addiction; encourage use of evidence‐based cessation methods (e.g. nicotine replacement therapy, peer support); build confidence and coping skills; praise success; reinforce benefits of quitting; and support users in maintaining abstinence. The 12‐week program is tailored to a participant's quit date, should they choose to set one. It is interactive, with bidirectional messages and keywords that allow users to request on‐demand support, and contains links to the EX website for additional information.

The optimized version of the text message program [29] was added to this standard program. Key features included: “personalization”, text messages personalized with EX username, chosen during registration; “dynamic tailoring”, messages were tailored on real‐time engagement with EX, reinforcing actions already taken or prompting exploration of features not yet used; “integration”, ability to engage with the six interactive features of EX via text message through special keywords and interactive messages; and “message intensity,” frequency of messages gradually tapered over 12 weeks [24]. Participants received a maximum of 69 engagement messages, delivered as follows: weeks 1–2, 14 messages/week; weeks 3–4, 7 messages/week; weeks 5–8, 4 messages/week; weeks 9–11, 3 messages/week; week 12, 2 messages [24]. For example, user texted MEDS to use the medication tool. System responded: “Which quitting medication are you planning to use? A = patch, B = gum, C = Chantix, D = Zyban, E = Meds aren't for me.” If the user said A (patch), the system responded, “Ok, we've got it saved in your EX Plan. Put the patch on when you wake up, and you're good to go. You can buy it without a prescription, but talk to your doctor about using. See your meds plan at LINK.” If the user said E (no meds), the system responded, “Ok, we've got it saved in your EX Plan. Medication can double your odds of success. Make sure you know the myths and misconceptions about the nicotine in medication: LINK ‐ If you change your mind, update your meds plan at LINK and if you don't, that's ok too!” Responses sent via text message were reflected on the website. For engagement with the EX community via text message, a special keyword (“TIPS”) delivered advice and guidance manually curated from actual community posts.

Measures

The baseline survey was conducted online. Follow‐up assessments at 3 and 9 months were conducted via mixed‐mode follow‐up. Online survey non‐responders were contacted by phone, text message, and mailed surveys. Mailed reminders about upcoming surveys and incentives for survey completion ($40/survey, $10 bonus payment for completion within 24 hours) were used to maximize follow‐up rates. The first reminder included $2 to encourage response by engaging the reciprocity norm [41].

At baseline, participants provided demographic information and frequency of internet use [42]. Smoking information included number of cigarettes per day, usual cigarette type, other tobacco product use, and time to first cigarette from the Fagerström Test for Nicotine Dependence. We assessed motivation to quit with the stages of change [43], Abstinence‐Related Motivational Engagement scale (ARME) [44], and a Likert item assessing desire to quit (1 = not at all, 5 = very much). Self‐efficacy was measured with the Smoking Situations Confidence Questionnaire (short form) [45] and a Likert item assessing confidence in quitting (1 = not at all, 5 = very much). Participants reported past year quit attempts, methods used to quit, and advice to quit from a healthcare provider. We also assessed problem drinking (Alcohol Use Disorders Identification Test [AUDIT‐C]) [46], depression (Patient Health Questionnaire‐2‐item [PHQ‐2]) [47], anxiety (Generalized Anxiety Disorder 2‐item [GAD‐2]) [48], smoking status of a spouse/partner, number of smokers in the household, and smoking status of their five closest friends.

The primary outcome was self‐reported 30‐day abstinence at 9 months. Secondary outcomes included 3‐month measures of 30‐day abstinence, intervention engagement (website visits, page views, time on site in minutes, use of six interactive features), and overall intervention satisfaction (1 = very dissatisfied, 2 = somewhat dissatisfied, 3 = somewhat satisfied, 4 = very satisfied) and whether they would recommend it to a friend. WEB+TXT participants rated their perception of the number of text messages received (too few, just right, too many) and of the messages themselves (e.g. “They helped me stay on track with quitting”; “They helped me build the confidence to quit”; 1 = Completely disagree, 2 = Disagree somewhat, 3 = Agree somewhat, 4 = Completely agree).

Analytic plan

In randomized trials, between‐group imbalances at baseline are the result of type I error and should not be evaluated for statistical significance. However, even such chance imbalances can bias treatment comparisons if characteristics that differ between study groups are prognostic of outcome and confound the treatment‐to‐outcome association. We used standardized mean differences (SMD) to gauge the magnitude of such between‐group imbalances [49]. For comparisons of binary and continuous variables, SMDs correspond to WEB+TXT versus WEB mean differences standardized to unit variance. Hence, Cohen's thresholds [50] for small/medium/large effects given by δ = 0.20, 0.50, and 0.80, respectively, can be used to assess their magnitude. For categorical variables with K >2 categories, SMDs are based on the square root of a Mahalanobis distance metric [51]. Descriptive statistics were also calculated for intervention satisfaction and intervention perception items.

We calculated abstinence rates under three missingness mechanisms [52]: (i) missing not‐at‐random (MNAR), in which subjects lost to follow‐up are treated as smokers, a convention commonly associated with ITT analyses; (ii) missing‐at‐random (MAR), in which missingness is modeled as a function of baseline covariates; and (iii) missing‐completely‐at‐random (MCAR), in which responders are regarded as representative of the full sample randomized at baseline. Although multiple imputation (MI) is a more popular approach to dealing with missing data under MAR, we chose to use Inverse probability of response weighting (IPRW), which rebalances the set of responders to be representative of the sample randomized at baseline. IPRW makes fewer assumptions than MI because it only requires correct modeling of the probability of non‐response, whereas MI requires correct modeling of the joint distribution of all missing data given the observed data [53]. We divided these propensity scores by the follow‐up rates in each arm and inverted the resulting ratios to create stabilized weights of unit mean under the assumption of no model misspecification. We assessed the success of the IPRW approach in reducing selection bias because of non‐response by comparing SMDs between respondents and non‐respondents before and after weighting. The stabilized weights were used to estimate logistic regression models for abstinence outcomes via the survey package in R [54].

Secondary analyses examined individual engagement metrics, including pageviews, time on site, return visits to the EX website, and use of the six key interactive features at the 3‐month follow‐up. In phase I of this research [24], outliers observed in pageviews, time on site, and website visits led us to dichotomize these variables at cut points that bifurcated the sample at a roughly 1:2 split. Cut points were ≥25 page views (32.2% of sample), ≥15 minutes of website use (29.8% of sample), and ≥1 website visit (38.5% of sample), respectively. We used the same cut‐offs in these analyses; sensitivity analyses confirmed robustness of findings to variation in these cut‐offs. Ever‐use of specific interactive features and website engagement metrics were analyzed via logistic regression. Overall use of the six interactive features (count variable, range 1–6) was analyzed via quasi‐Poisson regression with a scale parameter to account for under/overdispersion. Analyses were conducted via the glm function in R [54].

RESULTS

Between December 18, 2018 and March 11, 2019, 6241 new registered users on EX were screened and 618 were randomized. Follow‐up began March 18, 2019 and was completed on January 8, 2020. Follow‐up rates were 76.7% at 3 months (WEB: 79.8% vs WEB+TXT: 73.6%, P = 0.07) and 75.1% at 9 months (WEB: 77.2% vs WEB+TXT: 73.0%, P = 0.23) (Figure 1).

FIGURE 1

Study CONSORT diagram

Average age was 37.8 years (SD = 11.9). The sample was 67.2% female, 29.6% non‐White, 17.2% Hispanic, 42.4% had a high school degree or less, 63.4% reported household income of $35 000 or less. The majority (72.9%) smoked a pack or less per day and 97% smoked within 30 minutes of waking. As expected, variables of interest were balanced across arms at baseline (SMDs < 0.22), necessitating no adjustment for potential confounders. Imbalances in baseline characteristics of responders and non‐responders at each follow‐up were also assessed for all variables in Table 1, both before and after propensity weighting (full results not shown). Variables showing the largest imbalances between 3‐month responders and non‐responders after weighting were education (SMD = 0.27), previous cessation medication use (SMD = 0.25) and nicotine dependence (SMD = 0.22). Variables showing the largest imbalances between 9‐month responders and non‐responders after weighting were sex (SMD = 0.24), internet use (SMD = 0.25) and education (SMD = 0.21). Given that no SMD exceeded 0.30 at either follow‐up, we analyzed between‐group differences in smoking outcomes without further covariate adjustment for non‐response bias.

TABLE 1

Baseline characteristics of n = 618 randomized sample

	Full sample (n = 618)	WEB (n = 307)	WEB+TXT (n = 311)
Demographics characteristics
Age, mean (SD) y	37.80 (11.9)	37.41 (11.8)	38.11 (11.9)
Female, n (%)	415 (67.2)	205 (66.8)	210 (67.5)
Race, n (%)
White	435 (70.4)	208 (67.8)	227 (73.0)
African American	109 (17.6)	54 (17.6)	55 (17.7)
Other	51(5.3)	31 (10.1)	20 (6.4)
More than 1 race	23 (3.7)	14 (4.6)	9 (2.9)
Hispanic ethnicity, n (%)	106 (17.2)	57 (18.6)	49 (15.8)
Education, highest grade completed, n (%)
< High school	73 (11.8)	33 (10.7)	40 (12.9)
GED or HS graduate	189 (30.6)	94 (30.6)	95 (30.5)
Some college	230 (37.2)	122 (39.7)	108 (34.7)
College grad or graduate degree	126 (20.4)	58 (18.9)	68 (21.9)
Employment status, n (%)
Employed for wages	319 (51.6)	159 (51.8)	160 (51.4)
Self‐employed	47 (7.6)	24 (7.8)	23 (7.4)
Unemployed 1 year or longer	77 (12.5)	45 (14.7)	32 (10.3)
Homemaker	39 (6.3)	17 (5.5)	22 (7.1)
Student	27 (4.4)	15 (4.9)	12 (3.9)
Retired	21 (3.4)	8 (2.6)	13 (4.2)
Unable to work	88 (14.2)	39 (12.7)	49 (15.8)
Marital status, n (%)
Married	180 (29.1)	90 (29.3)	90 (28.9)
Divorced	115 (18.6)	56 (18.2)	59 (19.0)
Widowed	15 (2.4)	8 (2.6)	7 (2.3)
Separated	38 (6.1)	22 (7.2)	16 (5.1)
Never married	191 (30.9)	101 (32.9)	90 (28.9)
Member of unmarried couple	79 (12.8)	30 (9.8)	49 (15.8)
Annual income, n (%)
<25 K	280 (45.3)	147 (47.9)	133 (42.8)
25 K–35 K	112 (18.1)	55 (17.9)	57 (18.3)
35 K–50 K	98 (15.9)	40 (13.0)	58 (18.6)
50 K–75 K	71 (11.5)	35 (11.4)	36 (11.6)
75 K+	57 (9.2)	30 (9.8)	27 (8.7)
Insurance, n (%)
Uninsured	174 (28.2)	91 (29.6)	83 (26.7)
Insured	421 (68.1)	206 (67.1)	215 (69.1)
Unknown insurance status	23 (3.7)	10 (3.3)	13 (4.2)
Geography, n (%)
Rural	109 (17.9)	53 (17.7)	56 (18.1)
Suburban	233 (38.3)	120 (40.1)	113 (36.6)
Urban	266 (43.8)	126 (42.2)	140 (45.3)
Internet use (frequency), n (%)
Almost constantly	298 (48.2)	156 (50.8)	142 (45.7)
Several times a day	247 (40.0)	118 (38.4)	129 (41.5)
About once a day	43 (7.0)	15 (4.9)	28 (9.0)
Several times a week	18 (2.9)	10 (3.3)	8 (2.6)
Less often	12 (1.9)	8 (2.6)	4 (1.3)
Substance Use Characteristics
Cigarettes per day, n (%)
10 or less	135 (21.8)	65 (21.2)	70 (22.5)
11–20	316 (51.1)	159 (51.8)	157 (50.5)
21–30	128 (20.7)	64 (20.8)	64 (20.6)
31 or more	39 (6.3)	19 (6.2)	20 (6.4)
Usual cigarette type, n (%)
Menthol	277 (44.8)	140 (45.6)	137 (44.1)
Non‐menthol	316 (51.1)	152 (49.5)	164 (52.7)
No usual type	25 (4.0)	15 (4.9)	10 (3.2)
Past 30‐day other tobacco product use, n (%)
Cigars	17 (2.8)	7 (2.3)	10 (3.2)
Little cigars/cigarillos	108 (17.5)	46 (15.0)	62 (19.9)
E‐cigs	147 (23.8)	76 (24.8)	71 (22.8)
Hookah	21 (3.4)	7 (2.3)	14 (4.5)
Chew/snuff/snus	33 (5.3)	16 (5.2)	17 (5.5)
Past 30‐day marijuana use	160 (25.9)	79 (25.7)	81 (26. 0)
AUDIT‐C score 3 or greater, n (%)	199 (32.2)	104 (33.9)	95 (30.5)
Time to first cigarette, n (%)
<5 min	325 (52.6)	167 (54.4)	158 (50.8)
6–30 min	219 (35.4)	109 (35.5)	110 (35.4)
31–60 min	42 (6.8)	20 (6.5)	22 (7.1)
After 60 min	32 (5.2)	11 (3.6)	22 (7.1)
Motivation to quit, n (%)
Next 30 days	561 (90.8)	280 (91.2)	281 (90.4)
Next 6 months	55 (8.9)	26 (8.5)	29 (9.3)
Not thinking of quitting smoking	2 (0.3)	1 (0.3)	1 (0.3)
ARME short form score, mean (SD)	28.58 (6.4)	28.56 (6.2)	28.60 (6.6)
Desire to quit, mean (SD)	4.70 (0.53)	4.67 (0.56)	4.73 (0.49)
Confidence in quitting, mean (SD)	3.44 (1.04)	3.44 (1.03)	3.45 (1.05)
Smoking Situational Confidence Questionnaire score, mean (SD)	25.5 (8.13)	25.85 (8.16)	25.15 (8.11)
Quit attempt in past year, n (%)	344 (55.7)	162 (52.8)	182 (58.5)
Quit methods used in past year, n (%)
Cold turkey/unassisted/willpower only	285 (82.8)	135 (83.3)	150 (82.4)
Prayer	138 (40.5)	73 (45.3)	65 (36.1)
Pamphlet or book	17 (5.0)	7 (4.3)	10 (5.6)
Face‐to‐face counseling, 1:1 or in a group	14 (4.1)	3 (1.9)	11 (6.1)
Nicotine replacement medicines	126 (37.0)	60 (37.3)	66 (36.7)
Prescription medicines	38 (11.1)	20 (12.4)	18 (10.0)
Switching to e‐cigarettes	127 (37.2)	61 (37.9)	66 (36.7)
Switching brands or cutting back	144 (42.1)	70 (43.5)	74 (40.9)
Telephone counselling/Quitline	16 (4.7)	11 (6.8)	5 (2.8)
Internet quit smoking program other than EX	11 (3.2)	4 (2.5)	7 (3.9)
Alternative methods	15 (4.4)	10 (6.2)	5 (2.8)
Advised to quit past year, n (%)
No	167 (27.0)	81 (26.4)	86 (27.7)
Yes	379 (61.3)	188 (61.2)	191 (61.4)
Haven't seen a health professional in past yr	72 (11.7)	38 (12.4)	34 (10.9)
Psychosocial
PHQ‐2 score 3 or greater, n (%)	230 (37.2)	112 (36.5)	118 (37.9)
GAD‐2 score 3 or greater, n (%)	274 (44.3)	135 (44.0)	139 (44.7)
Partner smoking status, n (%)
Partner does not smoke	221 (35.8)	115 (37.5)	106 (34.1)
Partner smokes	216 (35.0)	97 (31.6)	119 (38.3)
No partner	181 (29.3)	95 (30.9)	86 (27.7)
No. of smokers in household, mean (SD)	0.54 (1.29)	0.60 (1.55)	0.49 (0.96)
No. of 5 closest friends who smoke, mean (SD)	2.29 (1.65)	2.36 (1.69)	2.22 (1.60)

AUDIT‐C = Alcohol Use Disorders Identification Test; PHQ‐2 = Patient Health Questionnaire‐2‐item; GAD‐2 = Generalized Anxiety Disorder 2‐item; GED = General Equivalency Diploma; HS = high school; ARME = Abstinence‐Related Motivational Engagement scale.

Table 2 presents abstinence outcomes at 3 and 9 months. No evidence of differences in 30‐day ppa rates under were observed under an MNAR missing = smoking assumption at the conventional two‐sided α = 0.05 level of significance at either 3 months or 9 months. Interestingly, the corresponding ITT abstinence rates rose across time in both arms, from 19.5% to 23.1% in WEB and from 18.6% to 23.2% in WEB+TXT. IPRW‐correction of responder‐only abstinence rates under a MAR mechanism showed intervention benefits at both 3 months and 9 months too weak to attain statistical significance. Again, IPRW‐corrected estimates of abstinence rates improved across follow‐ups, rising from 24.5% to 29.2% in WEB and from 24.9% to 30.4% in WEB+TXT. Finally, responder‐only analyses, unbiased under MCAR, resulted in slightly elevated abstinence rates, without materially affecting estimates of the intervention effect.

TABLE 2

30‐day point prevalence abstinence at 3‐ and 9‐months post‐randomization

	WEB	WEB+TXT	Rate difference	OR	P value
	n = 307	n = 311
3 month outcomes
No. responses	245	229
No. abstinent	60	58
Intent to treat, % (95% CI)	19.5 (15.5, 24.4)	18.6 (14.7, 23.4)	−0.9 (−7.1, 5.3)	0.94 (0.63, 1.41)	0.78
IPRW responder‐only, % (95% CI)	24.5 (19.1, 29.9)	24.9 (19.3, 30.6)	0.4 (−7.4, 8.3)	1.02 (0.67, 1.56)	0.92
Responder‐only, % (95% CI)	24.5 (19.5, 30.3)	25.3 (20.1, 31.4)	0.8 (−6.9, 8.6)	1.05 (0.69, 1.59)	0.83
9 month outcomes
No. responses	237	227
No. abstinent	71	72
Intent to treat, % (95% CI)	23.1 (18.7, 28.2)	23.2 (18.8, 28.2)	0.1 (−6.6, 6.7)	1.00 (0.69, 1.45)	0.99
IPRW responder‐only, % (95% CI)	29.2 (23.3, 35.0)	30.4 (24.4, 36.5)	1.3 (−7.1, 9.6)	1.06 (0.71, 1.59)	0.77
Responder‐only, % (95% CI)	30.0 (24.5, 36.1)	31.7 (26.0, 38.1)	1.8 (−6.6, 10.1)	1.09 (0.72, 1.64)	0.69

IPRW = inverse probability of response weighting.

Table 3 summarizes 3‐month engagement for all 6 interactive features. Moderate intervention effects in Cohen's nomenclature [50] were seen for Choose Quit Smoking Aid, Build Support System, Track Smoking Triggers, and Beat Smoking Triggers, whereas a small effect was observed for Visit Community. No differences were observed on Set Quit Date, possibly because of ceiling effects. Feature use was higher among WEB+TXT. No between‐arm differences were observed for pageviews or time on site. A small intervention effect was observed for return website visits.

TABLE 3

WEB vs WEB+TXT use differences at 3‐months post‐randomization

	WEB	WEB+TXT	Difference	SMD	P value
Used interactive features via either modality, % (95% CI)
Set quit date	84.7 (80.2, 88.3)	88.4 (84.4, 91.5)	3.7 (−1.7, 9.2)	0.077	0.175
Choose quit smoking aid	26.4 (21.8, 31.6)	49.2 (43.7, 54.7)	22.8 (15.2, 30.0)	0.342	<0.001
Build support system	5.9 (3.7, 9.1)	30.2 (25.4, 35.6)	24.4 (18.6, 30.1)	0.472	<0.001
Track smoking triggers	27.0 (22.4, 32.3)	55.3 (49.7, 60.7)	28.3 (20.6, 35.4)	0.424	<0.001
Beat smoking triggers	8.8 (6.1, 12.5)	29.9 (25.1, 35.2)	21.1 (15.0, 27.1)	0.392	<0.001
Visit community	18.3 (14.3, 23.1)	32.5 (27.4, 37.9)	14.2 (7.3, 20.9)	0.261	<0.001
No. features used, mean (SD)	1.71 (1.57, 1.86)	2.85 (2.66, 3.05)	1.14 (.90, 1.38)	0.523	<0.001
Website engagement, % (95% CI)
Pageviews ≥25	31.2 (26.3, 36.7)	34.2 (29.1, 39.7)	3.0 (−4.5, 10.4)	0.045	0.43
Time on site ≥15 min	43.2 (37.7, 48.9)	48.4 (42.8, 54.0)	5.2 (−2.8, 13.0)	0.074	0.20
Website visit >1	38.9 (33.5, 44.5)	48.7 (43.1, 54.3)	9.8 (−1.9, 17.6)	0.141	0.02

SMD = standardized mean difference.

Among 3‐month survey completers (n = 449), there were statistically significant differences in satisfaction measures favoring WEB+TXT (somewhat/very satisfied: 96.3% vs 90.5%, P = 0.008; would recommend EX: 95.9% vs 90.1%, P = 0.028). Among those randomized to WEB+TXT, the majority (70.7%) said the number of messages was “just right”; 23.6% said there were “too many,” and 5.8% said “too few.” Top scoring items on the 4‐point scale about text message satisfaction were: “They helped remind me to visit the EX website,” M = 3.4 (SD = 0.8); “I was able to access many program features by using text messages,” M = 3.4 (SD = 0.7); “They helped me stay on track with quitting,” M = 3.3 (SD = 0.8); and “They helped me know how to use quit smoking medication properly,” M = 3.3 (0.8). Lowest scoring items were: “I liked reading tips from other EX members,” M = 2.6 (SD = 1.0); and “The messages were written personally for me,” M = 2.7 (SD = 1.1).

DISCUSSION

This study compared the effectiveness of an integrated internet and text message smoking cessation program against an internet program alone in increasing abstinence among adult smokers. In the first phase of this research, the text message program had been optimized for treatment engagement, hypothesized to be a mediating pathway for treatment arm differences in abstinence based on prior research [30]. The combined internet and text message intervention improved engagement over the first 3 months of the study compared to the internet intervention alone. As hypothesized, engagement was found to be significantly higher in the intervention group for 5 of 6 interactive features, with the sixth likely subject to ceiling effects. Absolute quit rates under ITT analysis at 9 months were strong at roughly 30%. However, greater engagement at 3 months did not translate into higher abstinence rates at 9 months. Although others have also failed to find an association between abstinence and treatment engagement in digital interventions [55], our findings run counter to prior research [4]. Several possible explanations are worth considering.

First, our conceptualization of the types of engagement that drive abstinence may be flawed. It is possible that we “optimized” the text message intervention on dimensions not causally connected to abstinence (i.e. our “action theory” was supported, but our “conceptual theory” may have been inadequate) [56]. The utilization metrics examined are commonly used measures of engagement quantity in digital interventions, but do not capture all aspects of engagement [8]. Other aspects of engagement, such as one's subjective experience with an intervention [8], may be causally connected to abstinence. This was outside the scope of the current study, but should be considered in future studies. Alternatively, the hypothesis that “more is better” may also be flawed. Ongoing use of a tool to identify smoking triggers, for example, likely signals that a user continues to experience triggers and may still be smoking. Different types and intensity of engagement can have varied effects on outcomes [55, 57, 58]. The Goldilocks principle—“Not too much. Not too little. Just right.”—may be the most suitable in understanding digital engagement [59]. We may need to work backward to understand patterns of digital engagement among those who quit, accounting for nuanced motivations for engagement and the complex interplay of person factors and other drivers of self‐selected treatment engagement. Discussion of such analyses is beyond the scope of this manuscript, but is an important area for future research. Last, it may be that a greater magnitude of differences in engagement are needed to drive abstinence outcomes. Observed quit rates in both arms were substantially higher than projected in our original power analyses, potentially signaling the impact of ongoing platform optimizations or population‐based shifts in motivation and readiness to quit.

Other studies have also found that adding text messaging to another intervention did not increase abstinence. Whittaker et al. [20] pooled results from four studies that added text messaging to another smoking cessation intervention versus the other intervention alone. In three studies [60, 61, 62], no treatment arm differences were observed. In the fourth study [63], no treatment effect was observed at the primary endpoint, but group differences in a secondary outcome were included in the meta‐analysis. Taken together, these results and others [64, 65] suggest text messaging may not be potent enough to increase abstinence when added to intensive interventions.

Strengths of this study are that it involved a theory‐based and empirically tested text message intervention; complete data on an array of engagement metrics were available on all participants; and the intervention was tested in a diverse sample that reflected lower levels of income and education common among US smokers [10]. It also involved the population of interest, namely treatment‐seeking smokers who had demonstrated an interest in quitting and using a digital intervention to do so. To our knowledge, it is the first high quality study designed around MOST principles to examine multiple factors of text messaging. Despite null results, this work suggests several areas of research for further inquiry. Several limitations should be noted. We did not biochemically verify abstinence given demonstrated challenges in digital trials [66, 67], the lack of demand characteristics that would give rise to misreporting [68, 69], and our selection of 30‐day abstinence as a more rigorous primary endpoint. If misreporting is present, point estimates of abstinence in both arms may be overestimated. Given logistical considerations, our primary outcome in phase I of this research was focused on engagement whereas the primary outcome in phase II focused on abstinence.

In conclusion, this study found that despite producing higher levels of engagement and satisfaction, a combined internet and text message intervention yielded equivalent abstinence rates to an internet intervention alone. More nuanced analytic approaches may be needed to understand the relationship between digital engagement and cessation outcomes. Given that internet interventions are often complemented by text messages [70], there is much to learn about defining, measuring, and optimizing digital engagement.

CLINICAL TRIAL REGISTRATION

https://clinicaltrials.gov/ct2/show/NCT02585206

DECLARATION OF INTERESTS

A.L.G., S.C., M.S.A., and M.A.J. are employees of Truth Initiative, a non‐profit public health foundation that sells enterprise digital tobacco cessation programs to support its mission‐driven work. L.C.A. receives royalties for the sale of a smoking cessation text messaging program called Text2Quit and has stock in Welltock. All other authors declare no conflicts of interest.

AUTHOR CONTRIBUTIONS

Amanda Graham: Conceptualization; funding acquisition; methodology; supervision. George Papandonatos: Conceptualization; formal analysis; funding acquisition; methodology; software; validation. Sarah Cha: Conceptualization; data curation; funding acquisition; investigation; methodology; project administration; software; supervision; visualization. Michael Amato: Conceptualization; data curation; investigation; methodology; software. Megan Jacobs: Conceptualization; funding acquisition; investigation; methodology; software. Amy M. Cohn: Conceptualization; funding acquisition; investigation; methodology. Lorien Abroms: Conceptualization; funding acquisition; investigation; methodology. Robyn Whittaker: Conceptualization; funding acquisition; investigation; methodology.