Identifying Active Ingredients, Working Mechanisms, and Fidelity Characteristics Reported in Smoking Cessation Interventions in Dutch Primary Care: A Systematic Review.

BACKGROUND: Evidence-based smoking cessation interventions provided by healthcare professionals can be successful in helping citizens to quit smoking. Yet, evidence is needed about the active ingredients of these interventions, how these ingredients work and how they are implemented in practice. Such knowledge is required to effectively support healthcare professionals to optimally put evidence-based smoking cessation interventions to (inter)national practice.
OBJECTIVE: To identify active ingredients (including behavior change techniques), mechanisms of action and implementation fidelity reported in smoking cessation interventions in Dutch primary care settings and to relate these to intervention effectiveness.
METHODS: A systematic review was conducted by searching nine national intervention or funding databases, five international scientific databases and consulting 17 national smoking cessation experts. Out of 1066 identified manuscripts, 40 interventions were eligible for this review. Based on published protocols, information regarding behavior change techniques and mechanisms of action was systematically abstracted. Additionally, information regarding study characteristics and other active ingredients, effects on smoking behavior and implementation fidelity was abstracted. Comparative effectiveness concerning abstracted intervention characteristics was qualitatively explored.
RESULTS: Active ingredients, mechanisms of action and implementation fidelity were moderately to poorly reported. Interventions applying behavior change techniques and interventions with a single behavioral target (i.e. smoking-only versus multiple behaviors) seemed to provide stronger evidence for successfully changing smoking behavior.
CONCLUSION: Attention to and reporting on interventions' active ingredients (e.g. behavior change techniques), mechanisms of action and implementation fidelity are prerequisites for developing more effective evidence-based smoking cessation interventions to be successfully implemented in primary healthcare. IMPLICATIONS: This systematic review provides an overview of smoking cessation interventions in Dutch primary care settings, identified since the year 2000. Smoking cessation support is offered in various forms, but our qualitative findings show that interventions including more behavior change techniques and interventions targeting only smoking cessation (compared to multiple behaviors) might be more effective. Results also show that-based on available intervention reports-it is difficult to distinguish patterns of active ingredients (such as behavior change techniques), mechanisms of action and fidelity of implementation in relation to interventions' effectiveness. This means (quality of) reporting on these intervention characteristics should improve.

Introduction

Smoking is the most preventable cause of non-communicable diseases, such as cancer and cardiovascular disease, and premature death worldwide, yet more than 1 billion people are still smoking.[1] In the Netherlands, more than 20 000 Dutch adults die of smoking-related causes each year.[2] Smoking prevalence in the Netherlands is above the global average; 20% among adults.[3] To reduce the negative public health impact of smoking, it remains important to motivate current smokers to make an attempt to quit smoking. For instance, the implementation of national smoking bans and other public smoking restrictions in the Netherlands have contributed to the “de-normalization of smoking”.[4] Still, only 73% of Dutch smokers has ever tried to quit smoking, while 25% intends to quit smoking within six months.[5] Moreover, without any form of support only 5% of quitters actually manages to quit.[6] Additional quit smoking measures might therefore be warranted, especially when targeting the remaining current smokers who might be highly nicotine-dependent and/or not yet sufficiently motivated to quit.[7]

In line with WHO strategies and recommendations to control the tobacco epidemic,[1] the Dutch government is taking action to expand existing smoking cessation policies and to improve the availability of smoking cessation support to its smoking citizens. One of these actions, implemented in January 2020, is to exempt smoking cessation support (combining counseling with pharmacological cessation support) in Dutch primary care from an individual’s “own risk” excess for contracted healthcare.[2] This largely removes the financial barrier for individual smokers to seek cessation support. Given that around two-thirds of Dutch smokers annually visit a primary healthcare professional (i.e. general practitioner (GP), dentist or midwife[8]) this measure improves accessibility of smoking cessation support interventions.

Worldwide, different guidelines and protocols have been developed for healthcare professionals to support citizens to quit smoking. Although these are applied in different settings by different professionals, a meta-analysis (including 26 guidelines from 22 countries), indicated almost universal agreement regarding (1) the need to identify smokers, (2) to offer a quit advice, and (3) to offer behavioral and/or pharmacological support.[9] Professionals are generally advised to offer elaborate quit support, as evidence demonstrates that more extensive forms (e.g. weighing pros and cons of quitting, discussing cessation aids) of smoking cessation counseling are more effective than merely a quit advice.[10,11] However, currently there is no consensus on the operationalization of the optimal form for offering smoking cessation support, and on conditions for effectiveness. Yet, understanding why and how an intervention works is essential for developing effective smoking cessation interventions and for implementing them in an efficient way.[12,13]

Mapping active ingredients of a behavior change intervention could help our understanding of why and how an intervention works. Systematic methods like the Behavior Change Taxonomy,[14] been applied to various behavior change interventions,[15] including smoking cessation interventions,[12,13,16] though not yet specifically in a primary healthcare context. The active ingredients, or so-called behavior change techniques (BCTs), in this taxonomy are defined as an observable, reproducible and simplified component of an intervention designed to modify causal processes that regulate behavior.[14] The relationship between a BCT and behavior is assumed to be mediated by mechanisms of action (MoAs), which provide information on how a BCT is expected to impact behavior.[14,15] Hence, together BCTs and MoAs can offer a thorough understanding of what an intervention’s active ingredients are and how they are able to change smoking behavior.

Still, a better understanding of an intervention’s content in terms of BCTs and MoAs might not be sufficient in itself, as intervention impact is also determined by the degree of implementation of the intervention in practice.[17] An important part of intervention implementation is fidelity, which is operationalized through five distinct concepts: adherence, dose, quality of delivery, participant responsiveness, and program differentiation.[18] When applying these concepts to smoking cessation support in primary care, they reflect, for instance, to what extent a smoker receives all intervention components (adherence); how often a smoker is exposed to these components (dose); how well the intervention components are delivered by the intermediary, such as a healthcare professional (quality of delivery); to what extent smokers’ experiences and engagement are assessed (participant responsiveness); and how the intervention can be distinguished from other forms of smoking cessation support (program differentiation). Ultimately, providing insight in implementation fidelity of an intervention would complement a BCT- and MoA-based overview of how and why smoking cessation interventions work.

Existing evidence indicates that both mapping of BCTs and MoAs and reporting on fidelity is not routinely done as part of an evaluation study.[19,20] Therefore, the aim of the present study is to systematically review the active ingredients in existing smoking cessation support interventions embedded in Dutch primary care. Several specific questions are addressed: (1) Which active ingredients (i.e. BCTs and other intervention characteristics not included in the BCT taxonomy) are reported in Dutch smoking cessation interventions? (2) Are active ingredients related to intervention effectiveness and how (MoAs)? (3) To what extent is implementation fidelity of the intervention reported and is this related to intervention effectiveness? We aim to increase our understanding of why and how smoking cessation support interventions in Dutch primary care work. This will both yield valuable insights for research (i.e. an evidence-based framework that can be used in primary care settings elsewhere) and practice (i.e. optimally implementing existing interventions, or identifying relevant active ingredients that can be used to review new interventions).

Methods

Study Design

A systematic review was conducted to both quantitatively and qualitatively explore the use of various active ingredients in smoking cessation interventions in Dutch primary care. The present study was part of a larger project in which smoking cessation interventions for smoking citizens (present study) and interventions aimed at improving smoking cessation care by primary care professionals were investigated (submitted for publication elsewhere).

Search Strategies and Selection Criteria

To find relevant smoking cessation interventions in Dutch primary care, three sources (separately described below) were systematically searched to ensure a comprehensive and inclusive search strategy. All searches were conducted on August 2, 2019. After duplicate removal, identified interventions/studies were first screened on title/abstract and then on full-text following the inclusion and exclusion criteria (Table 1). Figure 1 illustrates the various reasons for exclusion of the interventions. In case eligibility could not be determined based on the available information, the intervention owner was contacted to request more information. The entire selection process was conducted by two researchers (D.d.R., E.M.), who verified each other’s work by checking whether articles were rightfully excluded. In case of discrepancies the researchers’ doubts were discussed with a third researcher (n = 7), after which agreement was reached in all cases.

Table 1.

Inclusion and Exclusion Criteria

	Inclusion	Exclusion
Intervention goal	Intervention targets smoking cessation or motivation to quit smoking, alone or in combination with other intervention foci	Study includes a measurement of smoking behavior, but smoking cessation is not targeted in the intervention. Intervention is targeted at smoking other drugs than tobacco (e.g. cannabis)
Role of healthcare professionals	At least one smoking cessation-targeted intervention component includes behavioral support (e.g. providing a quit advice or more extensive counseling) from a primary healthcare professional	Self-help interventions, interventions only including pharmacological support, or interventions offered by non-healthcare professionals (e.g. lifestyle coaches)
Setting	Data are available on effectiveness of the intervention in a Dutch primary care setting (i.e. first-contact care that does not require a medical referral, e.g. a GP, dentist or physiotherapist)	Data are only available on effectiveness of the intervention outside the Netherlands or from non-primary care settings
Data	The available data enable the comparison of the intervention to at least one other (intervention) group, or the comparison of pre- and post-intervention results	Only baseline data or only post-intervention data are available

Figure 1.

Flowchart study selection.

Online National Databases

Registered and available smoking cessation interventions in a primary care context were identified in Dutch intervention databases (i.e. the Centre for Healthy Living, the Trimbos Institute, the Dutch Youth Institute and the Stop Smoking Quality Register). Completed and ongoing studies about smoking cessation interventions in a primary care context were identified in Dutch funding databases (i.e. the Netherlands Organization for Health Research and Development, the Netherlands Organization for Scientific Research, the Dutch Cancer Society, the Lung Foundation Netherlands and the Dutch Heart Foundation). In all databases, the following search terms and their Dutch equivalents were entered: smoking, smoker, smoke, tobacco, addiction and cigarette.

Online International Scientific Databases

Five scientific databases were systematically searched: PubMed, Web of Science, Cochrane, Medline, and Eric. In each database a similar search string was used (see Appendix A), which was based on the inclusion and exclusion criteria (Table 1). Each search string combined keywords and relevant synonyms for concepts concerning smoking (e.g. smoke, tobacco), primary care (e.g. physician, dentist), study design (e.g. intervention, program), and geographical region (e.g. Dutch, Netherlands). Boolean operators were used to separate synonyms (OR) and to ensure relevant hits would include all relevant concepts (AND). Searches were limited on title and abstract and to studies published in 2000–2019. To provide a complete as possible overview for each intervention, articles covering potential supplemental material (e.g. study protocol, process evaluation, economic evaluation) were not excluded.

Dutch Smoking Cessation Experts

Seventeen Dutch smoking cessation experts (i.e. representatives of relevant public health organizations and research institutes) and their professional networks were consulted. Experts were contacted on August 14, 2019 and were given a response time frame of five weeks. Experts received an email, including a social media template for obtaining input from their online professional networks, requesting their input concerning Dutch smoking cessation interventions in primary care. Each expert received an overview of already identified interventions via databases described above.

Data Abstraction

After duplicate removal across data sources, data abstraction for each intervention included: study characteristics and effects on smoking behavior, active ingredients (and related MoAs) and characteristics of implementation fidelity (Table 2). The following types of active ingredients were abstracted: (1) general active ingredients, not related to behavior change or intervention content; (2) specific active ingredients, based on an evidence-based protocol to map health behavior change with the BCT taxonomy[14]; and (3) content-related active ingredients (other than BCTs), which address the intervention’s goal, target group or methodology.

Table 2.

Type of Data Abstracted for Each Intervention

Type of information	Abstracted data per intervention
Study characteristics	First author and year of publication
	Intervention format (e.g. online, face-to-face)
	Theoretical grounding of intervention content
	Description of target population Intervention target behavior(s)
	Smoking-related outcomes measured
	Effects per smoking-related outcome
Active ingredients: general	Use of a planning model for intervention development Incentives used for smoker and/or professional Co-creation with target population Compatibility with target population Training of intermediaries
Active ingredients: specific	BCTs: number, label, certainness, operationalization BCT smoking-related target behavior
Mechanism of action	MoA: number, label, operationalization
	MoA: single or multiple links with BCTs
	Explicitness of BCT-MoA link
	Empirical test of BCT-MoA link
Active ingredients: content-related	Ingredients concerning the target population (not BCTs)
	Ingredients concerning the intervention goal (not BCTs)
	Ingredients concerning the methods used (not BCTs)
Fidelity characteristics	Intervention name
	Intervention components
	Adherence*
	Dose*
	Quality of delivery*
	Participant responsiveness*

*Operationalization based on the work of Dusenbury et al.[18]

BCT, behavior change technique; MoA, mechanism of action.

Data abstraction was conducted by two researchers (D.d.R., E.M.). Based on previous work regarding BCT-MoA coding,[14,21] a set of coding rules (e.g. when to (not) code a specific BCT) was discussed and agreed upon by the researchers. To pre-test data abstraction, both researchers abstracted all available data from one intervention. Upon comparison of the abstracted data, agreement was high. Any inconsistencies were discussed and small changes were made to the coding rules, after which both researchers abstracted data from the remaining interventions. During abstraction, each researcher coded a different set of interventions and regular consensus meetings were held to ensure similar application of the coding rules. Upon completion of data abstraction, each researcher randomly checked 5% of the other researcher’s abstracted data, especially focusing on BCTs and MoAs, as coding these was deemed most sensitive for bias and inconsistencies. Again, agreement between researchers was high, as only one BCT was added for one intervention after cross-checking. In addition, as described in the BCT abstraction protocol,[14] the researchers kept track of their confidence level in assigning BCTs and MoAs (1 = highly confident, 2 = not sure). Researchers discussed all instances where they were not sure about the codes assigned (n = 37), resulting in 11 individual BCT codes being adjusted.

After completing data abstraction, the active ingredients of each intervention (Table 2) were summarized and reported back to the first author of the intervention’s effect paper. This enabled the authors to verify the coding work and provide feedback on any additional active ingredients that might have been missed. Resulting from this, two additional papers and seven responses with unpublished supplementary information (on eight interventions) were obtained from the authors, which subsequently also underwent data abstraction.

Data Preparation and Analysis

Data preparation consisted of three distinct steps. First, the intervention’s effect on smoking cessation was summarized based on the reported results. Given that most interventions did not report an effect size measure nor reported sufficient information to calculate an effect size measure, it was decided to create a qualitative interpretation of each intervention’s effects. This “qualitative effect size” ultimately consisted of three rudimental categories describing the extent to which the intervention managed to reduce smoking behavior: strong evidence (+), weak evidence (+/−) or no evidence (−). An intervention was categorized as providing “strong evidence” when post-intervention smoking behavior was statistically significantly improved compared to a control group or compared to baseline smoking behavior. Moreover, to receive the “strong evidence” category standardized smoking behavior had to be measured according to the Russel standard,[22,23] meaning that a 7-day point prevalence abstinence (7D-PPA) measure had to be used or a measure indicating longer abstinence than 7D-PPA (e.g. continued abstinence). The category “weak evidence” was used when less conservative measures were used (e.g. quit attempt), but the results showed a statistically significantly improvement compared to a control group or compared to baseline smoking behavior. Interventions were categorized as providing “no evidence” when smoking behavior was not statistically significantly reduced or when any measure of smoking behavior significantly increased compared to a control group or baseline measure.

Secondly, the active ingredients and working mechanisms identified in each intervention were mapped to systematically organize all data (Supplementary Tables 4 and 5). Descriptive analyses were conducted to explore any patterns concerning an intervention’s categorization as providing strong, weak or no evidence. Moreover, quality of implementation fidelity (Supplementary Table 6) was scored. For this purpose, a well-established operationalization of implementation fidelity was used[24] as this concept is known to be more often measured than many other implementation outcomes.[25] Fidelity consisted of four characteristics (i.e. adherence, dose, quality of delivery, participant responsiveness[18]), which were assessed for every intervention (score ranging from 0 to 3). After summing the four separate scores, this resulted in an overall score. This newly created score was labeled Quality Assessment of Implementation or QAI score, which could range from 0 to 12. Based on earlier work, specific assessment criteria were used to define cutoff scores of 50% and 75%,[26] informing the scoring categories (range 0–3). A score of zero was provided, whenever a characteristic was not reported in the intervention description. A “poor” score (1) was assigned when fidelity was below 50% (e.g. less than half of participants adhered to intervention components, received the appropriate dose or positively evaluated the intervention). An “acceptable” score (2) was assigned when fidelity was between 50% and 75% or when reported information generally described good fidelity but included too little data to assess a specific characteristic in detail. A “good” score (3) was assigned when fidelity was reported to be above 75%. To verify the QAI scoring, a second researcher (EM) also scored all fidelity descriptions. This resulted in a high inter-rater agreement (86/96 agreement; 90%); two scores were changed after discussing 10 individual discrepancies in the QAI scores.

Thirdly, abstracted data were systematically organized in tables to explore differences between each of the three evidence levels (+, +/−, and −) whenever possible. Qualitative descriptive analyses were used to answer the research questions.

Results

Intervention Effects and Study Characteristics

The selection process resulted in the inclusion of 40 unique smoking cessation interventions (Figure 1), which are categorized according to the level of evidence an intervention provides (Supplementary Table 3). Four interventions were categorized as providing strong evidence (#37-40; 10%), eight as providing weak evidence (#30–36; 20%) and 28 as providing no evidence to improve smoking behavior (#1–29; 70%). The four interventions providing strong evidence measured smoking behavior with 7D-PPA or continued abstinence and three of them used cotinine validation. Most interventions providing weak evidence showed significant improvements in smoking behavior measured as smoking status (y/n) or quit attempts undertaken. One intervention’s effect (#25) was evaluated at two different points in time, meaning that both effect evaluations are included in this review.[27,28] Yet, as the intervention was categorized to provide strong evidence only once, this intervention was categorized as providing weak evidence overall (Supplementary Table 3). Further, since in one specific paper[29] the authors reported on the effects of a large number of different interventions—and since the level of evidence was different for each intervention—these interventions are included separately.

Supplementary Table 3 provides an overview of the interventions’ study characteristics and Supplementary Table 6 describes the interventions’ content in more detail. Only one intervention (3%) used a group format, whereas all others targeted smokers individually. Moreover, most interventions used a face-to-face format (n = 31; 78%); the others used either an online (n = 2; 5%), telephone (n = 1; 3%) or blended format (i.e. face-to-face combined with online, n = 6; 18%). About two-thirds of the intervention descriptions (n = 28; 65%) mentioned the use of theory in developing or evaluating the intervention. Most interventions included populations either at-risk for or already having one or more chronic illnesses such as cardiovascular disease (n = 14; 30%) or COPD (n = 8; 20%), whereas a quarter included healthy populations (n = 10; 25%). Ten interventions (25%) were primarily aimed at changing smoking behavior, whereas the majority of interventions focused on multiple health behaviors, including smoking. A larger portion of the latter group of interventions was categorized as providing no evidence to improve smoking behavior (25/30; 83%), compared to the group of interventions primarily aimed at smoking (4/10; 40%).

Active Ingredients

The three types of identified active ingredients are summarized in Supplementary Table 4. First, specific active ingredients (i.e. BCTs) were identified, which are structured as BCT categories and individual BCTs in Supplementary Table 4. The BCT categories “social support” (n = 30; 75%), “shaping knowledge” (n = 26; 65%), “goals and planning” and “feedback and monitoring” (both n = 23; 58%) were most often reported. The individual BCTs “social support (unspecified)”, “information about antecedents” and “goal setting (behavior)” were most often identified, in respectively 25 (63%), 22 (55%), and 20 (50%) intervention descriptions. On average, about 10 individual BCTs were coded per intervention; more than 10 BCTs were coded in 17 intervention descriptions (43%), whereas in 14 descriptions three or less BCTs were coded (35%). The maximum number of BCTs coded in a single intervention description was 17 (intervention #34 & 38), whereas for four interventions (#11, 19, 24, and 33) only one BCT was described. On average, more BCTs were found for interventions providing strong evidence (16 BCTs), compared to weak (13 BCTs) and no evidence (9 BCTs).

Secondly, general active ingredients were identified. One intervention was developed with the use of a planning model (#29), incentives were used in five and three interventions for respectively smokers (#5, 7, 26, 30, and 39) and professionals (#5, 7, and 25). Additionally, for eight interventions (20%) co-creation was used, meaning the target population was involved in preparation and development. For intervention development almost a third (n = 12; 30%) took into account the intervention’s compatibility with the target population/setting. Most interventions (n = 31; 78%) used trained intermediaries to implement the intervention, including the four interventions providing strong evidence.

Thirdly, content-related active ingredients are summarized in Supplementary Table 4. In 80% of interventions (n = 31) active ingredients regarding the target population were found, e.g. describing tailoring of intervention content to individuals or a group of participants. Nine interventions (23%) included active ingredients regarding the intervention goal, such as giving participants autonomy to choose a behavioral goal. Finally, eleven interventions (28%) addressed active ingredients regarding the methodology, e.g. operationalized as implementing the intervention following a gradual (stage-based) approach. Overall, no apparent differences in content-related active ingredients were identified when comparing interventions on the three evidence levels.

Mechanisms of Action

Supplementary Table 5 provides an overview of the reported working mechanisms (i.e. MoAs) and BCT-MoA links that were hypothesized and tested. For nearly half of the interventions (n = 18; 45%) at least one MoA was described. MoAs “motivation” and “knowledge” were most frequently described in the studies included in the present review. As these MoAs were described ten times across nine and seven interventions respectively, this means that these MoAs were linked to more than one BCT in a single intervention. MoA “beliefs about capabilities” was also described frequently (i.e. nine times). In 17/49 cases, a single BCT was linked to a single MoA, e.g. hypothesizing action planning (BCT) would change behavior through reinforcement (MoA). In most cases a link between multiple BCTs and/or MoAs was described, e.g. hypothesizing that social support (BCT) would change behavior via knowledge and motivation (MoAs). Moreover, over 90% of hypothesized BCT-MoA links (n = 45) were not empirically tested. Three out of five tested links demonstrated a significant impact on smoking behavior: attitude was associated with quit smoking intention (intervention #4); and perceived susceptibility/vulnerability and general attitude/beliefs were associated with abstinence (intervention #38).

Fidelity of Implementation

Supplementary Table 6 outlines an intervention’s fidelity characteristics[18] and illustrates the assigned QAI scores. The total QAI scores averaged at 2.0 (4.5 for interventions providing strong evidence, 2.1 for weak evidence and 1.5 for no evidence) and ranged from 0 to 12. For most interventions, few fidelity characteristics were described. Overall, adherence was reported for 13 interventions (33%), dose ten times (25%), quality of delivery two times (5%), and participant responsiveness nine times (23%). Nearly half of these interventions received a good score for adherence (6/13), 40% of interventions a good score for dose (4/10), 100% for quality of delivery (2/2), and 11% of intervention received a good score for participant responsiveness (1/9).

Discussion

The objective of the present review was to describe the active ingredients (including BCTs), MoAs and fidelity characteristics of smoking cessation interventions for citizens in Dutch primary care settings. Ultimately, a systematic search and selection process resulted in inclusion of scientific papers describing 40 different interventions. Overall, active ingredients, MoAs and fidelity characteristics were only sparsely reported in these papers, limiting the identification of a clear association of these characteristics with intervention effectiveness. The majority of the 40 interventions found no clear evidence for successfully changing smoking behavior, as only four studies were classified as providing strong evidence of behavior change based on standardized outcomes measures. These findings illustrate the persisting need to develop evidence-based behavior change interventions that can effectively change citizens’ smoking behavior (assessed with standardized outcome measures) and the importance of systematically reporting on their active ingredients, MoAs and implementation fidelity.

Active Ingredients and Working Mechanisms

About ten BCTs per intervention were identified through using an evidence-based BCT abstraction protocol,[14] mostly belonging to BCT categories “social support”, “shaping knowledge”, “goals and planning” and “feedback and monitoring”. We observed that more BCTs were reported in interventions providing stronger evidence (i.e. strong > weak > no evidence). Otherwise, no further patterns were found in the present review regarding the type of BCTs used in effective interventions. This contrasts findings of Black and colleagues[16] who recently showed that several specific BCTs (e.g. social reward) and BCT categories (e.g. goals and planning) could be associated with a higher intervention success rate in their much larger review on smoking cessation RCTs across different contexts and countries. Several of these BCT categories were also frequently reported in our country-specific review, but no association with the level of evidence of an intervention was detected. These contrasting findings may show the importance of taking into account contextual differences (e.g. different healthcare system, different culture) of intervention implementations.[30] Future research should demonstrate if and how similar BCTs may impact the effectiveness of smoking cessation interventions in country-specific or healthcare-specific contexts, and comparing results across contexts, before advocating their use by intervention developers. The present review provides preliminary evidence on the importance of such contextual differences, by describing novel insights concerning the specific context of the Dutch primary healthcare setting.

General active ingredients—except training intermediaries—were not frequently reported in intervention descriptions. Despite this apparent underreporting, it is promising to see that many intervention developers use trained intermediaries for intervention implementation, as this is known to increase implementation quality[18] and subsequently intervention effectiveness.[17] Our results seem to support this, as all four interventions providing strong evidence reported using trained intermediaries. Furthermore, content-related active ingredients were not frequently reported either, except those addressing the target population (e.g. tailoring). Yet, no patterns were found in the reported content-related active ingredients when distinguishing interventions based on their level of evidence. Generally speaking, the present review has indicated that most active ingredients are not frequently reported, potentially indicating that they are also not frequently used. This exemplifies a need for Dutch intervention developers and researchers to better report on the use of active ingredients in their smoking cessation interventions in primary care.

Unfortunately, after systematically structuring reported MoAs in relation to identified BCTs, similar observations were made in terms of an apparent underreporting and a lack of patterns. Only 49 BCT-MoA links were found in the descriptions of the 40 included interventions and more than 90% of these links were not empirically tested. In comparison, the average number of identified MoAs was nearly four times higher in a recent review on behavior change interventions in general, but still the authors emphasized that BCT-MoA links are hardly tested.[15] This clearly indicates a research agenda to first of all report on the hypothesized MoAs and BCT-MoA links when reporting on intervention effectiveness, and second, to incorporate systematic testing of said MoAs and BCT-MoA links. Moreover, as our results showed that two-thirds of the interventions were theory-based, it seems to be a lack of explicit reporting of how theory was used that explains the underreporting of MoAs. This coincides with recent recommendations for researchers to engage in systematically describing their hypotheses about the theory-based mechanisms via which they expect their interventions to operate.[19] Researchers have previously acknowledged that interpreting and applying behavioral or psychological theory can be challenging, hence various initiatives have been launched to support the development of evidence-informed and theory-based behavior change interventions.[31,32] Supported by the findings of the present review, we would advocate systematic dissemination and use of such initiatives among and by behavior change researchers.

Three conclusions were drawn regarding the reported fidelity characteristics: first, there was considerable variation across studies regarding the degree of detail in which implementation fidelity was described; second, few authors adequately reported on implementation fidelity; and third, relatively low QAI scores were assigned based on the available information. Although other studies have found similar results,[20] this observation is worrisome as it has repeatedly been established that high-quality implementation fidelity can positively impact intervention effectiveness.[33] By assigning QAI scores we attempted to offer a concise overview of the quality of implementation fidelity, but this proved to be especially challenging when only limited information was available. Moreover, it was striking that the quality of delivery of an intervention was described in only two studies. Interestingly, both of these reports of quality of delivery were given a “good” score and overall, the two intervention descriptions represented the two highest QAI scores in our analysis (i.e. 9 and 12 points). Unfortunately, both interventions were not effective in changing smoking behavior. It is likely that other factors explain the lack of intervention effectiveness, for instance contextual factors of a specific healthcare setting,[30] interactions between healthcare professionals and the environment they are working in[34,35] or national policy on reimbursement of smoking cessation support.[2] Even though the findings of the present review offer limited insights in factors potentially associated with intervention implementation, more systematic and comprehensive evaluations of a wider range of implementation outcomes are needed to advance implementation science.[25] In conclusion, researchers, intervention developers and the scientific community would benefit from investing more resources in systematically conducting and reporting on evaluations of implementation (e.g. in separate publications or as an appendix of effect evaluations).

Intervention Characteristics

The 40 included interventions represented a wide array of formats for smoking cessation support with varying degrees of intensity, targeting various types of citizens. Because of this high degree of variation, it was difficult to unravel patterns of intervention characteristics potentially being associated with intervention effectiveness. Nevertheless, one potential trend worth mentioning is that interventions primarily targeting smoking cessation were less often labeled as providing no evidence (40%) than interventions primarily targeting other behaviors (83%). This effect may be confounded by the frequent use of non-standardized outcomes measures to assess smoking abstinence in studies not primarily targeting smoking. Alternatively, this may suggest that successful behavior change is more likely when intervention developers focus on developing interventions primarily targeting smoking behavior. However, it is noteworthy that fewer interventions primarily targeting smoking behavior than interventions targeting multiple behaviors were included. Although our review is in line with other studies suggesting that single-behavior change interventions are more successful than interventions targeting multiple behaviors, it is still debatable whether there is sufficient evidence to advocate for either single or multiple-behavior change interventions.[36] Arguably, targeting multiple behaviors would especially be beneficial when separate behaviors naturally co-occur and ultimately strive to achieve the same goal, such as targeting both physical activity and dietary intake for achieving weight loss. Hence, when targeting smoking cessation, it could be useful to target an often co-occurring behavior such as excessive alcohol consumption.[37] Because of the addictive nature of both behaviors, it may be useful to address them simultaneously in an integrated behavior change intervention.[38] Moreover, to increase successfulness of such a combined intervention, researchers could consider to still give the individual some choice in prioritizing a specific behavior to change first,[39] as support for autonomy has previously been demonstrated to be effective in the context of tailored smoking-related interventions.[40,41] Such an autonomy-support strategy may be especially applicable for interventions in a primary care setting, compared to patients dealing with substance use disorders in outpatient or inpatient care settings as behavior change may be less of a priority in these care settings.

Strengths and Limitations

A strength of the present review was the systematic, evidence-based methodology that was used to conduct the BCT-MoA coding and analysis, as this was based on earlier work.[14,15,21] In line with these protocols and recommendations, all work was conducted by two coders checking each other’s work and discussing any discrepancies in all coding phases. Moreover, all authors of intervention papers were contacted for verification of the identified active ingredients, further increasing the reliability of our analyses.

A limitation of the present review could be the rather crude classification of studies based on the level of evidence provided and the newly created QAI score to evaluate implementation fidelity. Still, both measures were based on existing evidence: standardized outcomes recommended in the research field of smoking cessation[22,23] were used to determine the level of evidence of interventions and a well-established operationalization of fidelity was used to assign the QAI scores.[18] The use of the QAI scoring system proved to be rather consistent in the present study, based on the high agreement between the first and second coder. Yet, additional validation procedures (e.g. comparing the QAI scoring system to a different fidelity assessment[26]) are required to substantiate reliability of this new scoring system, before the QAI score may be applied to systematically assess implementation fidelity in future studies.

Conclusion

This systematic review explored active ingredients, working mechanisms, and fidelity characteristics of smoking cessation interventions in Dutch primary care. We found that interventions that provide strong evidence (based on standardized smoking cessation outcome measures) include a higher number of specific active ingredients (i.e. BCTs) than interventions providing weak or no evidence, suggesting it could be worthwhile to systematically incorporate BCTs during intervention development. Further, the findings provide novel insights concerning the role of the country-specific (e.g. reimbursement of smoking cessation support) and healthcare-specific (e.g. trained primary care professionals specialized in smoking cessation) context in which smoking cessation interventions are offered. The impact of such contextual differences should be further understood (e.g. by conducting detailed process evaluations), before the use of specific active ingredients and working mechanisms can be advocated when developing and implementing interventions. Also, primary care interventions targeting multiple behavioral outcomes may be less effective in achieving smoking cessation, but specific behaviors (e.g. those that naturally co-occur with smoking) may still be combined under specific conditions (e.g. providing smokers with choice/autonomy). Future studies are encouraged to use standardized outcomes assessments.

The findings of the present review should be interpreted in light of substantial underreporting of active ingredients, working mechanisms and fidelity characteristics, illustrating a clear research agenda for intervention developers. Not before active ingredients, working mechanisms and implementation fidelity are all systematically reported by researchers and intervention developers, can reliable analyses be conducted and firm conclusions be drawn on the relationship of these aspects with intervention effectiveness.

Supplementary Material

A Contributorship Form detailing each author’s specific involvement with this content, as well as any supplementary data, are available online at https://academic.oup.com/ntr.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.