A vaping risks education program for school students: Evaluation of the solve mystery toolkit.

One in three grade 7 to 12 students in Canada report trying vaping or e-cigarettes. Despite consequences like nicotine addiction, impaired brain development, increased respiratory symptoms, and association with an increased risk of COVID-19 diagnosis, 48% of youth believe occasional vaping has little to no risk. There is a clear need for youth to learn about vaping consequences. We developed and piloted a novel free interactive educational program on vaping risks which has been used by over 800 grade 7 to 9 students. In post-program surveys, students reported a subjective increase in knowledge about the health consequences of vaping.

Introduction

E-cigarette use, or “vaping”, among children and youth in Canada is increasingly common. Thirty-four percent of youth in grades 7 to 12 report having tried e-cigarettes (Government of Canada. Summary of results for the Canadian Student Tobacco, Alcohol and Drugs Survey, 2019). Potential serious adverse consequences of vaping for youth including nicotine addiction (Helen and Eaton, 2018), impaired brain development (Yuan et al., 2015, England et al., 2015), and increased respiratory symptoms (Stanbrook, 2019, King et al., 2020, Gotts et al., 2019 Sep). Many vaping products also contain chemicals other than nicotine, which may be harmful (Stanbrook, 2019, Layden et al., 2020). E-cigarette use has also recently been associated with an increased risk of a diagnosis of COVID-19 (Gaiha et al., 2020, The Lancet Respiratory Medicine, 2020). Once youth start using vaping products, it is difficult for them to stop. Among young people aged 16 to 24 who started vaping, 52.2% tried unsuccessfully to quit, an average of 4.8 times (Al-Hamdani et al., 2020). Additionally, youth who vape are five times more likely to smoke combustible cigarettes (Osibogun et al., 2020 May). Despite this evidence, 48% of youth believe occasional vaping with nicotine has little to no risk (Government of Canada. Summary of results for the Canadian Student Tobacco, Alcohol and Drugs Survey, 2019).

Given common misconceptions around vaping risks, the often early initiation of this addictive practice, and well-established evidence of potentially severe health effects, there is a clear need for Canadian youth to learn about the consequences of vaping. To the best of our knowledge, health curricula on vaping are lacking or inaccessible for youth in Canada. Studies of rising rates of vaping in North America have concluded that there is a need for effective intervention strategies (Ingels et al., 2020 Sep, Harrell et al., 2017 Mar). It has been shown that youth state they would not vape if they learned that it was harmful to their health (Alexander et al., 2019 Mar). In other contexts related to both health and substance use, educational interventions have been successful in giving youth improved self-management (Bruzzese et al., 2011), greater knowledge, and increased likelihood of making healthier (non-drug-use) choices (Midford et al., 2014, McBride et al., 2004). An educational intervention on vaping could play an important role in reducing vaping initiation and promoting vaping cessation.

Over the past 18 months, our team developed and piloted a novel, free, interactive, multimedia educational program named the SOLVE (Short On or Off-Line Vaping Risks Education) Mystery Toolkit. Funding was provided through a Health Canada Substance Use and Addictions Program microgrant. The toolkit offers an innovative design to engage students as active participants. Toolkit content was reviewed by a team of teachers, physicians, students and public health advocates to ensure effective, current and high-quality material. The target demographic is grade 7, 8 and 9 students. The toolkit is available for use in the classroom or online to facilitate learning during the COVID-19 pandemic. It is designed for use by teachers and does not require any external experts or performers. Teachers are provided a user-friendly manual and access to an online portal to view their students’ answers should they use the online version.

We sought to assess increase in knowledge and ease of implementation of this novel vaping education program for middle school students by piloting it in multiple schools. From a policy perspective, this analysis will assist with optimizing implementation and take-up of this short educational program, as well as offer ideas on what educational interventions can be applied in other areas of health.

Methods

The toolkit was promoted through word-of-mouth, teacher conferences, social media and a website (), and used by several schools in Calgary, Alberta since January 2020. Students were introduced to the toolkit by teachers in a classroom or online setting. The program contains 3 elements: a mystery introduction, knowledge development and critical reflection and application. Initially, students are not aware of the subject of the activity. They are presented with a mystery scenario through a short video and various evidence files, and are required to use these resources to discover the role of vaping in the mystery. Next, students’ understanding of the health consequences of vaping is deepened through fill-in-the-blank worksheets and videos featuring health professionals. Finally, students are asked to apply this knowledge to real-life scenarios, such as speaking to a friend about vaping (Supplementary Figure A).

For quality improvement (QI) purposes, teachers were given the option of inviting their students to complete an anonymous survey, in which no identifiable information was collected. Survey participation did not affect access to the toolkit. The survey included two questions that elicited self-assessments of learning: students were asked to rate their increase in knowledge about vaping, and their current knowledge on vaping after completing the toolkit. No specific questions testing knowledge on vaping risks were included. To minimize bias concerning self-referential statements, students were also asked to rate how useful they felt the toolkit would be for other students in their grade.

Since uptake of educational programs is more likely to be successful if students experience intrinsic motivation, such as enjoyment (Ryan and Deci, 2000), students were asked to rate overall enjoyment, and to comment on what they liked and disliked. An optional teacher survey was recently implemented, asking teachers whether they would recommend the toolkit to colleagues, to comment on ease of use, and how it compares to other educational resources. Teachers were also given the option to provide other comments. The project did not require institutional review board approval as it fell under the umbrella of quality assurance/improvement according to the Tri-council Policy Statement Article 2.5.

We summarized the survey data, compared proportion of responses across grades using Fisher’s Exact (Stata, version 14) with a significance of p < 0.05, and conducted a brief thematic analysis of the descriptive comments (StataCorp. Stata Statistical Software: Release 14, 2015).

Results

Eight hundred twenty-eight grade 7, 8 and 9 students in Calgary, Alberta have used the toolkit. Feedback forms were optional; 58% of students (484/828) completed them with 87% (95% CI 84 – 90%) reporting an “excellent”, “good” or “satisfactory” increase in knowledge about health consequences of vaping following toolkit completion. Ninety-seven percent (95% CI 95 – 98%) described their knowledge on vaping risks after completing the toolkit as “excellent”, “good” or “satisfactory”. Ninety-percent (95% CI 86 – 92 %) felt the activities would be “excellent”, “good” or “satisfactory” for other middle school students (Fig. 1). These results were similar across grades with no statistically significant difference (p > 0.05) in the proportion of students in each grade reporting “excellent”, “good” or “satisfactory” compared to “poor” or “very poor” with respect to increase in knowledge, toolkit usefulness and knowledge after completion (Table 1).

Fig. 1

Percentage of student responses in assessing the the SOLVE Mystery Toolkit learning measures. Assessments were defined as “very poor,” “poor,” satisfactory,” “good,” and “excellent.” Learning measures include i) increase in vaping knowledge (n = 484), ii) usefulness for other students (n = 484), and iii) knowledge about vaping risks after toolkit completion (n = 385). Responses were averaged from all responding students in grades 7–9 in Calgary, AB.

Table 1

Number and percentage of student responses by grade and assessment for SOLVE Mystery Toolkit learning measures. Assessments were defined as “very poor,” “poor,” satisfactory,” “good,” and “excellent.” Learning measures include i) increase in vaping knowledge (n = 484), ii) toolkit’s usefulness for other students (n = 484), and iii) knowledge about vaping risks after toolkit completion (n = 385). Responses were averaged from all responding students in grade 7, grade 8, or grade 9, respectively.

		Very Poor	Poor	Satisfactory	Good	Excellent
Increase in Vaping Knowledge(n (%))	Grade 7(n = 104)	4 (4%)	3 (3%)	22 (21%)	24 (23%)	51 (49%)
	Grade 8(n = 276)	15 (5%)	23 (8%)	53 (19%)	118 (43%)	67(24%)
	Grade 9(n = 104)	7 (7%)	9 (9%)	13 (13%)	39 (38%)	36 (35%)
Usefulness for Other Students(n (%))	Grade 7(n = 104)	6 (6%)	7 (7%)	20 (19%)	35 (34%)	36 (35%)
	Grade 8(n = 276)	13 (5%)	18 (7%)	51 (18%)	104 (38%)	90 (33%)
	Grade 9(n = 104)	0 (0%)	8 (8%)	14 (13%)	43 (41%)	39 (38%)
Knowledge after Toolkit Completion(n (%))	Grade 7(n = 104)	3 (3%)	2 (2%)	16 (15%)	52 (50%)	31 (30%)
	Grade 8(n = 177)	4 (2%)	1 (1%)	27 (15%)	73 (41%)	72 (41%)
	Grade 9(n = 104)	0 (0%)	1 (1%)	7 (7%)	56 (54%)	40 (38%)

When asked about their overall enjoyment of the toolkit as a school assignment, 77% (95% CI 73 – 80%) of students felt their enjoyment was “excellent”, “good” or “satisfactory”. Grade 9 students reported the highest enjoyment (82% said “excellent”, “good” or “satisfactory”). When describing their favourite part of the activities, 3 themes emerged: 33% mentioned the detective/mystery aspect of the toolkit, 32% mentioned hearing from real-life peers and health-professionals in the videos, and 17% mentioned learning about vaping risks. Although not included as a question in the surveys, 4% of students commented on potential behavioural changes in the future. This included statements of quitting vaping and/or planning to never vape given new knowledge, such as “[the activities] gave me an opportunity to learn more about vaping so I know not to vape myself” and “now I know not to [vape] and will not [vape]”.

Among teachers who completed the survey (n = 8 due to recent implementation), 6/8 would “definitely” recommend the toolkit to other teachers, and 2/8 would “probably” recommend it. When ranking the toolkit compared to other educational resources, 1/8 felt it was “much better”, 6/8 felt it was “better” and 1/8 felt it was “similar”. Seven out of 8 teachers felt it was easy to use. Teachers commented that “students found it interactive and fun”, “[the toolkit] was smooth and easy-to-use”, “my students really enjoyed this”, and “I would definitely use this program again”.

Discussion

The SOLVE Mystery Toolkit has the potential to address an important knowledge gap (Government of Canada. Summary of results for the Canadian Student Tobacco, Alcohol and Drugs Survey, 2019), and help students understand the risks of vaping. First, the survey responses indicate that a vaping educational intervention in school as a method for teaching students about the risks of vaping deserves further study: the majority of students indicated substantial gains in their knowledge about the risks of vaping and felt confident in their knowledge of vaping following completion of the toolkit. Second, the survey data indicate that this content is highly relevant for students: most students in all three grades felt that completing these activities would be useful for their peers.

Third, a large majority of students reported at least a satisfactory level of enjoyment from this short educational program. Many students enjoyed the interactive mystery aspects of the toolkit, which engaged them in thinking about the possible effects of vaping.

Fourth, the feedback suggested that delivery of a vaping educational intervention in a “ready-to-use” toolkit format can be easily delivered in a classroom setting. Teachers do not need specialized training to employ the toolkit, no external guests are needed, and there is no cost to the school (except printing worksheets if used in class). This enables immediate scale-up. Finally, although not formally assessed, a small proportion of students provided unsolicited comments on changes in intention to vape in the future, suggesting the potential for behavioural impact.

This pilot analysis offers a preliminary assessment of this novel program. Limitations include lack of a validated measure of student learning and of student knowledge prior to program use, and optional survey completion which could lead to self-selection bias by survey completers.

Conclusions

Rapid uptake of the toolkit within the year following launch as well as positive feedback from teachers indicates there is a need for vaping educational interventions in schools across Canada. In feedback, students in all grades subjectively reported learning new information about the risks of vaping. Initial use of this toolkit and success among students and teachers suggest that similar approaches utilizing interactive, ready-to-use, teacher-delivered educational interventions may be of benefit in other health-education topics (Soole et al., 2008). Future directions in research include a more robust study of the toolkit, including pre- and post-toolkit assessments, as well as longer-term follow up to assess impact on future vaping behaviour.

With respect to next steps in education, the toolkit will be adapted for national use by the Heart and Stroke Foundation, and they will further refine it and enhance availability. Furthermore, the toolkit will be translated into French to enable national use in both official languages.

Overall, the feedback obtained for QI purposes suggests the value of this novel educational intervention and indicates promise for application to middle school education.

CRediT authorship contribution statement

Asha Hollis: Conceptualization, Methodology, Formal analysis, Investigation, Writing – original draft, Writing – review & editing, Project administration. Emily Downey: Conceptualization, Investigation, Writing – review & editing. Shaelene Standing: Investigation, Data curation, Writing – review & editing. Janet Leahy: Investigation, Data curation, Writing – review & editing. Kirsten Ebbert: Conceptualization, Writing – review & editing, Supervision. Aravind Ganesh: Conceptualization, Writing – review & editing, Supervision.

Declaration of Competing Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: AH discloses a microgrant received from Health Canada’s Substance Use and Addictions Program used to fund development of the SOLVE Mystery Toolkit. ED discloses membership in the advocacy group Stop Addicting Adolescents to Vaping and E-Cigarettes, and a monetary compensation for presenting the SOLVE Mystery Toolkit at a teachers conference with all compensation directly used to fund toolkit expenses. KE discloses Cystic Fibrosis Canada Clinical Fellowship Funding for 2020. AG declares grants (Canadian Institutes of Health Research, Canadian Cardiovascular Society, Alberta Innovates, Campus Alberta Neurosciences, Sunnybrook Research Institute INOVAIT Program), consulting fees (MD Analytics, CTC Communications Corp, MyMedicalPanel, Atheneum, Deep Bench), honoraria (Fig. 1, Canadian Association of Neuroscience Nurses), meeting travel support (American Academy of Neurology, Association of Indian Neurologists in America, American Heart Association, University of Calgary), a patent (provisional US 63/024,239), editorial board memberships (Neurology: Clinical Practice, Neurology, Stroke) and ownership of stocks (SnapDx, TheRounds.com). JL and SS have no conflicts of interest to disclose.