Content Validity of a Willingness to Quit Tool for Use with Current Smokers in Clinical Practice.

INTRODUCTION: Despite reductions in rates of smoking in the past decade, smoking remains one of the most significant public health concerns. Quitting smoking can result in reductions in a number of serious health conditions. The brief Willingness to Quit (WTQ) tool can be used in routine clinical practice to assess current willingness to quit and engage a patient-physician dialogue regarding smoking cessation. The overall aim of this study was to validate the content of a WTQ tool for use with current smokers in clinical practice.
METHODS: In-depth, qualitative interviews were conducted with 12 current smokers and five physicians. The interview was divided into two sections: concept elicitation (CE) followed by cognitive debriefing (CD). During CE, participants were asked questions exploring the different factors that can impact an individual's willingness to quit smoking. During CD, participants were given a copy of the WTQ tool and asked to comment on their level of understanding and interpretability of the items and the feasibility of completing the tool in clinical practice.
RESULTS: All of the current smokers (n = 12) and physicians (n = 5) interviewed indicated that the items were understandable and relevant to assess willingness to quit. The tool was considered simple and suitable for use in clinical practice.
CONCLUSION: The WTQ tool is a brief tool to assess willingness to quit and to engage communication between patients and physicians. All smokers should be offered smoking cessation support and facilitating a discussion on willingness to quit further supports a personalized quit plan. FUNDING: Pfizer Inc.

Introduction

Despite reductions in rates of smoking in the past decade, smoking remains a significant public health concern [1, 2]. Recent data from the Centers for Disease Control and Prevention (CDC) and national cohort studies indicate that smoking is causally, or suspected causally, related to approximately 500,000 deaths each year [3-5]. Quitting results in reductions in cardiovascular diseases, respiratory diseases, tobacco-related cancers, and all-cause mortality [6]. Research has shown there are benefits to quitting smoking at various ages [3], and even those who quit in middle age avoid most of their subsequent risk of lung cancer [7, 8].

However, the behavioral and addictive nature of smoking makes cessation difficult [9]. Counseling and medication can be effective in increasing cessation rates when targeted towards smokers who are willing to make a quit attempt [10]. Healthcare provider support is another important factor in successful smoking cessation [11]. Indeed, physician office visits can be seen as an opportunity to engage individuals to spontaneously adopt risk-reducing health behaviors such as smoking cessation [12, 13]. Patients are more likely to successfully quit smoking and remain a non-smoker 12 months later if they have received advice from their physician, however, brief and simplistic [14].

In support of this, the US Department of Health and Human Services guidelines on the treatment of tobacco use and dependence propose that smokers should be given a brief intervention to address their smoking at every physician consultation [15, 16]. Specifically, these guidelines recommend physicians follow the “five A’s” of treating tobacco dependence (Ask, Advise, Assess, Assist, and Arrange follow-ups) with every patient they see who smokes [15]. However, research suggests that the first two strategies, i.e., “Ask” and “Advise”, are more frequently used than the subsequent strategies [17, 18]. This indicates that the final three steps, i.e., “Assess”, “Assist”, and “Arrange”, are currently not being optimized in clinical practice, which will have implications for smoking cessation success. Patients are more likely to undertake physician cessation counseling when provided with the “five A’s” during a physician appointment [19]. Facilitating implementation of the final three strategies, by assessing a patient’s willingness to quit smoking, providing physician assistance in quitting, and arranging a follow-up appointment, may help to improve smoking cessation outcomes. In order to support physicians in smoking cessation efforts with patients, a brief willingness to quit (WTQ) tool for use in clinical practice has been developed. The WTQ tool assesses patients’ current willingness to quit smoking and facilitates open communication between patients and their physician where different smoking cessation methods can be discussed.

The original WTQ tool has been piloted in routine clinical practice as a tool to support dialogue with patients and their physicians on their willingness to receive assistance for quitting. The original tool comprised the first three questions and was developed in line with current public health service guidelines, but without any input from current smokers. The WTQ tool was developed on the basis of a literature evaluation of patient-reported information to engage smokers in a smoking cessation conversation. The tool had been used in clinical practice to support smoking cessation conversations between the patient and physician. Following the literature review and at the recommendation of physicians who use the tool in clinical practice, a fourth question assessing “willingness to reduce” was added. It was important to assess and confirm the content validity of the four-item tool in this target population. The US Food and Drug Administration (FDA) define content validity as the extent to which an instrument or tool measures the concept of interest [20]. Assessing content validity ensures that the items in the instrument or tool are appropriate for the intended use and population [20]. The industry recommended approach for assessing the content validity of clinical outcome assessments (COAs), such as patient-reported outcomes (PROs), clinician-reported outcomes (ClinROs), and patient-completed tools for use in clinical trials and clinical practice, is through qualitative concept elicitation and cognitive debriefing interviews [21]. Although the FDA provides regulatory guidance on the development of instruments for use in clinical trials, the WTQ tool is an instrument that will be used in clinical practice to aid communication between a patient and their healthcare provider on willingness to stop smoking and not as an endpoint in clinical trials.

The WTQ tool is a brief tool containing four items: “If I could quit smoking I would”, “I want to quit smoking because I worry about how smoking affects my health”, “I would be willing to make a plan to quit smoking”, and “I would be willing to cut down my number of cigarettes before quitting”. The tool takes less than 5 min to complete. The tool aims to understand a patient’s current level of willingness to quit smoking and enable further discussion between the patient and his/her physician or healthcare provider on how and when to initiate smoking cessation. Upon completion of the tool, physicians are encouraged to have an open dialogue and establish a personalized quit plan with smokers.

Objectives

The overall aim of this work was to assess the content validity of the WTQ tool and to ensure that it is suitable for use in clinical practice as a resource for physicians. The content validity was assessed using qualitative interviews with both current smokers and physicians.

Methods

A targeted peer-reviewed literature search was conducted in PubMed, PsycINFO, and EMBASE databases to identify key publications that included an assessment of willingness to quit in relation to quitting smoking. Searches were conducted using specific search terms yielding 555 abstracts, 16 of which met the inclusion criteria and were reviewed in full (n = 12 full-text articles and n = 4 abstracts).

Following the literature review, semi-structured, 1-h qualitative interviews were conducted with current smokers to explore their experience, willingness to quit smoking with a physician’s support, and explore their experience(s) of smoking cessation methods. The WTQ tool is intended to be interpreted by physicians in clinical practice. Thus, an additional five qualitative interviews were conducted with physicians to obtain their feedback about using the tool. All interviews were conducted remotely via telephone to allow for geographical diversity.

A total of 12 American-English-speaking adults, who were current smokers, participated in the study. A sample size of 12 was calculated, based on a sample sufficient to achieve conceptual saturation [22]. The interviews were conducted in two separate rounds of six participants per round to allow for interim analysis and to provide opportunity for adaptations to the interview guide if necessary. To meet the eligibility criteria, all participants were regular smokers of conventional, store-bought cigarettes (at least 10 per day) who had been smoking for at least 3 years. All participants must have had at least one quit attempt more than 3 months ago and must not have been taking any medications to quit smoking at the time of the interview. Participants were recruited via a third-party recruitment agency using a convenience sampling methodology. The third-party recruitment agency was responsible for identifying recruiting clinicians, who would discuss the study with potential participants on their behalf. Participants were invited to take part during a routine physician appointment and all participants provided written informed consent. Prior to the conduct of each interview, participants were informed of the nature of the research and were given the opportunity to ask any questions or withdraw from the study if necessary. Notably, no participants withdrew from this study after initially consenting to participate. All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1964, as revised in 2013.

Informed consent was obtained from all patients for being included in the study.

Physicians were also recruited via a third-party recruitment agency. To meet the inclusion criteria, all physicians had been practicing for at least 5 years and had experience of helping patients to quit smoking. Each physician used the WTQ tool with five patients in clinical practice and participated in a 1-h interview to provide feedback on concepts associated with willingness to quit smoking and their experience of using the tool with patients.

Semi-structured interview guides were developed for both the current smoker and physician interviews to ensure all concepts of interest were discussed. Interviews were conducted by experienced qualitative interviewers and were audio-recorded and transcribed verbatim. Ethical approval was sought from Copernicus Group Independent Review Board (CGIRB; ADE1-16-645).

Interview with Current Smokers

For the interviews with current smokers, the first half of the interview (30 min) was dedicated to concept elicitation, where participants were asked a series of open-ended questions to explore their smoking history and the different factors that can impact on willingness to quit smoking. Initial questions were designed to avoid leading participants in their responses, and provide participants with an opportunity to report concepts “spontaneously” (e.g., “Tell me about your smoking history”) followed by more focused questions designed to probe on issues that they may not have been mentioned during the course of the interview. Following the open-ended discussions, participants were given a copy of the WTQ tool, which includes four items assessing willingness to quit smoking. Participants were asked to complete the tool using a “think aloud” exercise [23] which involved the participant completing the questions speaking aloud their thoughts as they read and selected a response for each question. Participants were asked detailed questions regarding the meaning, understanding, and relevance of each question and instruction. Participants were also asked to comment on the wording used in the instructions and each item in the tool, and encouraged to suggest alternative wording if necessary.

Interview with Physicians

Prior to participating in the physician interview, each physician was required to use the WTQ tool in clinical practice with a minimum of five current smokers. Physicians were asked to record their observations on how well patients understood the tool, whether the tool was relevant, and any suggested changes and to share this feedback in a 1-h telephone interview. Similar to the interviews with current smokers, each interview was divided into a concept elicitation and cognitive debriefing section. During concept elicitation, physicians were asked a series of broad, open-ended questions designed to elicit information regarding their patients’ smoking history and willingness for assistance in relation to quitting smoking. Following this, physicians were asked to complete the tool using a “think aloud” exercise and to reflect on using the tool with their patients. Physicians were asked to assess the adequacy of conceptual coverage and to comment on whether anything was missing from the tool and identify any items that should be considered for removal from the tool.

Interview Analysis

A software package (ATLAS.ti) [24] was used to facilitate the storage, coding, and qualitative analysis of interview transcripts. Interviews with current smokers and physicians were analyzed in two separate ATLAS.ti databases. Quotes were grouped into concepts using thematic analysis methods and a coding tree was developed prior to analysis to establish how concepts and sub-concepts would be coded [24]. Concepts were considered spontaneously elicited if reported by the participant during the open-ended questioning. Following the initial open discussion, participants were asked about specific concepts relating to smoking. If a participant then reported a concept to be relevant, which had not already been mentioned in the open-ended discussion section, this concept was coded as probed. When a participant did not mention a concept spontaneously and did not report a concept to be relevant when probed, this was coded as “no impact”. Sub-concepts were not assigned spontaneous or probed codes. Conceptual saturation was assessed to ensure that the concepts elicited from participants had been fully explored during the interviews and provide evidence of an adequate sample size. To assess conceptual saturation, participants were divided into three equal groups based on the chronological order in which interviews were conducted to allow the concepts elicited spontaneously to be compared using a stepwise approach. Saturation was considered to be achieved if no new concepts emerged in the final group of interviews.

Results

Literature Review

Of the 16 articles/abstracts reviewed, six used a tool or multiple items to assess willingness to quit or reduce smoking, another six used a single question to assess willingness to quit or reduce smoking, and the remaining four mentioned willingness to quit or reduce smoking but did not specify how it had been assessed. Measures of willingness to quit identified from the literature review consisted of either a single item or up to three items, indicating that the WTQ tool is an appropriate length when compared to similar measures. The term “willingness to quit” was the most frequently used term throughout the measures identified in the literature review (9/16 articles provided specific information about items/measures of willingness to quit) and is therefore considered appropriate for use in the current tool.

In relation to the concepts covered in the WTQ tool, all 16 articles reported on willingness to quit smoking (“If I could quit smoking, I would”). Five articles specifically reported on willingness to quit in relation to health (“I want to quit smoking because I worry about how smoking affects my health”) and ten articles reported on the importance of an intervention to help individuals quit smoking (“I would be willing to make a plan to quit smoking”). Therefore, the term “willingness to quit” was used throughout the measure and is considered appropriate for use in the current tool because of the supportive literature on this topic.

Demographics and Clinical Characteristics of Current Smokers

The sample consisted of slightly more female participants (n = 7, 58.3%) than males (n = 5, 41.7%) and participants ages ranged from 25 to 68 years old. As per predefined sampling quotas, a range of education levels was achieved, with three participants reporting their highest level of education to be a high school diploma.

Demographic Characteristics of the Physicians

The five physicians had been in their role for a mean of 25.6 years (range 23–30 years), all worked in private practice settings, and reported that they discuss quitting smoking in every consultation with their patients who currently smoke. Physicians were recruited from four different states in the USA and reported that 11–40% of their patients were current smokers.

Concept Elicitation: Insights from Current Smokers

A total of seven distinct broad concepts associated with the impact of smoking were elicited from participants (see Fig. 1) and within each broad concept were more specific concepts. For example, the broad concept of “impact on the body” included the more specific concepts of smell of cigarettes (n = 11), reduced appetite and/or weight loss (n = 6), stained teeth and/or bad breath (n = 5), impact on taste (n = 3), and poor skin (n = 2). In addition, the broad concept of “impact on well-being” included the more specific concepts of reduces stress (n = 9), relaxing (n = 8), experiences of urges to smoke (n = 5), improves mental state (n = 4) and prevents boredom (n = 3) (Tables 1, 2).

Fig. 1

Overview of concepts discussed during qualitative interview

Table 1

Participant demographic characteristics

Demographic characteristics	Total (n = 12)
Age
Mean (range)	42.8 (25–68)
Median	38.5
Gender, n (%)
Female	7 (58.3)
Male	5 (41.7)
Education, n (%)
High school diploma	3 (25)
Undergraduate or bachelor’s degree	2 (16.7)
Graduate degree	3 (25)
Other	4 (33.3)
Work status, n (%)
Working full time or part time	10 (83.3)
Retired	1 (8.3)
Unemployed	1 (8.3)
Ethnicity, n (%)
Hispanic or Latino	3 (25)
Non-Hispanic or Latino	6 (50)
Missing data	3 (25)
Race, n (%)
White	9 (75)
Black/African American	0 (0)
Multiracial	0 (0)
Native American or Alaskan	0 (0)
North African or Middle Eastern	0 (0)
Other	3 (25)
Hispanic	3 (100)

There was considerable variety in the smoking history and previous quit attempts of the sample (as detailed in Table 2). Pre-recruitment quotas were met as reflected in the diverse sociodemographic and clinical characteristics of the sample

Table 2

Participant smoking-related characteristics

Smoking-related characteristics	Total (n = 12)
Age of first cigarette
Mean (range)	23.4 (11–50)
Median	19
Years full-time smoker
Mean (range)	13.75 (3–50)
Median	6.5
Average number smoked daily, n (%)
11–20	10 (83.3)
21–30	2 (16.7)
Nicotine dependence score (based on Fagerström) [25]
Low dependence	0 (0)
Low to moderate dependence	4 (33.3%)
Moderate dependence	8 (66.6%)
High dependence	0 (0)
Have you thought about quitting smoking? n (%)
I am thinking about quitting now (in the next month)	4 (33.3)
I am thinking about quitting in the next 3 months	5 (41.7)
I am thinking about quitting at some point in the future	3 (25)
I am not thinking about quitting	0 (0)
Number of previous quit attempts n (%)
1–2	5 (41.7)
3–4	3 (25)
5–6	2 (16.7)
11 or more	2 (16.7)
Methods used to help quit smoking? n (%)a
Prescription methods	2 (16.7)
Over-the-counter nicotine replacement therapy	7 (35)
Face-to-face individual counseling	0 (0)
Face-to-face group counseling	0 (0)
Telephone counseling	0 (0)
Hypnotherapy	0 (0)
E-cigarettes	1 (8.3)
Other	2 (16.7)
How difficult/easy is it to quit smoking? n (%)
Very easy	0 (0)
Somewhat easy	0 (0)
Neither difficult nor easy	0 (0)
Somewhat difficult	5 (41.7)
Very difficult	7 (58.3)
Best person suited to assist in attempt to quit smoking? n (%)
Doctor	6 (50)
Family member(s)	4 (33.3)
Dentist	1 (8.3)
Friend	1 (8.3)
Nurse	0 (0)
Pharmacist	0 (0)
Smoking cessation nurse	0 (0)
Ex-smokers	0 (0)

aParticipants could select more than one option

Notably, all participants (n = 12) spontaneously reported an impact associated with their body. Additionally, when probed, all 12 participants discussed the financial impact of smoking, but no participants mentioned the financial impact spontaneously. The least discussed impact was the emotional impact associated with smoking, with six participants (n = 6/12) reporting that they had not experienced any emotional impacts as a result of smoking. Table 3 provides the analyses of conceptual saturation, showing that conceptual saturation was achieved for all six impact concepts, as no new concepts emerged in the final set of interviews (“Methods”). Of note, financial impact was not included in the saturation analysis and it was not mentioned spontaneously. In addition, given that no new concepts emerged following the second set of participant interviews, we are confident that conceptual saturation was achieved in our sample.

Table 3

Conceptual saturation analysis: concepts reported spontaneously (n = 12)

A checkmark indicates that the concept was discussed spontaneously, while a cross indicates that the participant was probed or did not experience a given concept. Gray boxes show the first instance where a concept was discussed spontaneously

Concept Elicitation: Insights from Physicians

All five physicians spontaneously discussed the impact of smoking on health and provided further detail of the associated health conditions including chronic obstructive pulmonary disease (COPD), bronchitis, chronic sinus problems, and pneumonia. Physicians also briefly discussed each of the other broad impact categories (i.e., body, financial, well-being, family, social life, and emotional). A number of concepts were discussed by physicians which had not previously emerged from the qualitative interviews with smokers but which nonetheless were included in the broad impact categories, including smokers appearing cyanotic, social withdrawal during quit attempts, patients reporting an emotional connection to smoking, and a financial impact on other family members.

Cognitive Debriefing Findings

All participants (12/12, 100%) demonstrated good understanding of the instructions and the four items in the WTQ tool, and all items were reported to be relevant to participants. Participants were able to select a relevant response for each item and demonstrated good understanding of the range of response options. Minimal wording changes were suggested by participants throughout the cognitive debriefing interviews (Fig. 2). Supporting quotes from a number of participants are presented in Table 4.

Fig. 2

Overview of feedback for the WTQ tool from current smokers (n = 12)

Table 4

Quotes to support understanding of the WTQ tool

Items	Supporting quote
Instruction 1:  If you are a current smoker, please answer these simple questions to help understand your willingness to quit smoking	“They’re just trying to see what would be the easiest way for—to help us quit. And how willing we are” (female, age 38) “I think, you know, I think it’s pretty clear for me” (male, age 36)
Item 1:  If I could quit smoking I would	“Basically if it was easy to quit would you quit” (female, age 38) “I just want to be there for my kids as long as possible, so if I can quit smoking I would. I definitely would” (male, age 39)
Item 2:  I want to quit smoking because I worry about how smoking affects my health	“It’s very relevant because that’s, that’s the reason why I would like to quit, because of my health” (female, age 31) “I would like to stop smoking so that my health can no longer be affected by smoking” (female, age 47)
Item 3:  I would be willing to make a plan to quit smoking	“Like to set something, instill a plan. You know, how I’m going to go forward in actually quitting to smoke” (female, age 31)
Item 4:  I would be willing to cut down my number of cigarettes before quitting	“Like a step-down method, something where it’s not just quitting cold turkey, completely taking away all cigarettes at one time” (female, age 47)

The results of the cognitive debriefing interviews with physicians were similar to the current smoker interviews. All five physicians reported that their patients understood each item and that each item was relevant. Physicians reported that their patients found the tool easy to complete and were able to select a relevant response for each item.

General Feedback on WTQ Tool and its Use in Clinical Practice

All participants provided positive feedback about the WTQ tool and described it as “straightforward” (male, age 54), “questions are relatable and make you think” (female, age 25), “concise and to the point” (male, age 68), and helps to “open up my mind to quitting smoking” (male, age 36). Similarly, three of the five physicians (n = 3/5, 60%) described the positive aspects of the WTQ tool, explaining that it is beneficial to separate patients who are willing to quit. It was further described as “easy to use, well-designed” and “It’s simple. It’s basic. It’s very straightforward”.

All participants (12/12, 100%) reported that it would be appropriate for physicians to introduce the WTQ tool during a routine appointment. This was confirmed during the physician interviews as no physicians reported a burden associated with completing the WTQ tool in clinical practice. Four of the five physicians (n = 4/5, 80%) explained that the tool improved communication with their patients. The remaining physician indicated that she did not notice any change in communication, and notably indicated that she would be unable to develop a personalized plan during an appointment because of time constraints.

Changes Made to the Tool

Despite the majority of current smokers and physicians providing positive feedback, some minor changes were made to the WTQ tool following review of the results. At the top of the tool, the participant was originally asked to complete the month and year and this was changed to “Today’s date” for clarity. At the end of the tool the term “healthcare provider” was amended to “doctor or healthcare provider” following participant feedback.

Discussion

As smoking remains a significant public health concern, tools designed to facilitate communication between patient and physician regarding quitting are encouraged. The WTQ tool was developed to facilitate the discussion between patient and physician. The results of both the literature and qualitative interviews with current smokers and physicians confirm that the tool is content valid and helpful when used in clinical practice. All 12 current smokers and physicians and their patients understood and interpreted the items as intended and considered the items relevant to supporting smoking cessation efforts. All participants would be open to completing the tool during a routine clinical appointment and all but one physician reported that the tool could be easily administered and acted upon during clinical practice.

The literature review confirmed that the length and content of the WTQ tool was aligned with other available measures designed to support smoking cessation engagement with smokers. The concept elicitation part of the interview confirmed the vast array of different concepts associated with smoking and highlighted the difficulties facing participants when trying to quit smoking. Our data suggest that each smoker may be motivated to quit by different reasons and physicians need to discuss the reasons most important to smokers to increase the chances of quitting. Although the WTQ tool is not designed to assess motivation to quit, the tool will facilitate discussions regarding willingness to quit between smokers and their physicians and provide the stepping stone to developing a tailored quit smoking plan. Physician office visits can be seen as an opportunity to engage individuals to spontaneously adopt risk-reducing health behaviors, e.g., smoking cessation [12, 13]. The WTQ tool can help physicians further apply the “five A’s” strategy, proposed by the US Department of Health and Human Services, as a tool to assess, assist, and arrange smoking cessation support. It is expected that the proposed amendments to the WTQ tool, including changes to the date format and some minor wording changes, will improve the usability of the tool for both patients and physicians.

Study Limitations

Interviews were only conducted in the US; further research in additional countries would provide further confidence that the WTQ tool has cross-cultural validity and is appropriate for use outside of the USA. It is also important to note that the current study has not explored the predictive validity of the WTQ tool. Therefore, it is difficult to establish whether patients who have indicated that they are willing to quit on the WTQ tool actually succeed at quitting smoking after following their personalized quit plan.

Conclusions

The WTQ tool was developed to facilitate discussion between smokers and physicians regarding willingness to quit smoking. This study indicates that the WTQ tool is content valid in current smokers. The WTQ tool is a brief assessment, which can be used in routine clinical practice, to assess willingness to quit smoking and encourage patient–physician dialogue about smoking cessation methods and personalized quit smoking plans.