Wellness and resilience for college and beyond: protocol for a quasi-experimental pilot study investigating a dialectical behaviour therapy skill-infused college course.

INTRODUCTION: College students' mental health problems and suicidal behaviour are serious, persistent and prevalent public health issues. With the need for mental health support greatly exceeding the availability of on-campus treatment, a recent trend on college campuses is to offer courses designed to teach students strategies for developing mental health or resilience. While these courses are exceptionally popular among students, a paucity of research investigates the health outcomes associated with participation. The purpose of this study is to investigate the acceptability, appropriateness, feasibility and preliminary effectiveness of a college course grounded in skills from dialectical behaviour therapy (DBT) titled, 'Wellness and Resilience for College and Beyond'. METHODS AND ANALYSIS: During the spring and fall 2020 semesters, the course will be offered on five campuses in Southwestern Pennsylvania and West Virginia. The course consists of 15 weekly 2.5-hour lessons, weekly homework assignments and a final examination with content drawn from DBT, acceptance and commitment therapy and positive psychology. Undergraduate students aged 18-24 will self-select into the course and control subjects receiving 'university as usual' will be recruited to serve as a comparison group. Students who receive the course will complete measures of course acceptability, appropriateness and feasibility. All study participants will complete measures of adaptive coping skills use, emotion dysregulation and suicidality. ETHICS AND DISSEMINATION: All of the study procedures were approved as an exempt protocol for evaluation of educational curricula by the University of Pittsburgh Human Research Protections Office (HRPO); the study was approved as a research study by the institutional review board (IRB) of the fifth study site. The University of Pittsburgh HRPO served as the IRB of record for all except one study site, which required standard IRB review. Data from this study will be disseminated via conference presentations, peer-reviewed publications and via our online stakeholder learning collaborative. TRIAL REGISTRATION NUMBER: NCT04338256. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This study is among the first to investigate health outcomes associated with a college course.

This study includes five diverse campus sites ranging from small, private institutions to large public institutions.

Study limitations include a quasi-experimental design with participants self-selecting into the intervention condition.

Introduction

College is a time of excitement and transition for many young people. While some students thrive during this developmental period of uncertainty and growth, many others struggle. Suicide is currently the second leading cause of death for college students,1 and exceptionally high numbers of college students struggle with mental health problems.2–4 An epidemiological study of collegiate mental health found that globally, one-third of first-year college students screen positive for at least one major DSM (diagnostic and statistical manual of mental disorders) disorder.3 Given that the majority of mental health disorders has an age of onset between the teen years and the early 20’s, college can be a particularly vulnerable time for some students.5 American College Health Association data indicate that during the preceding 12-month period, 85% of students surveyed felt overwhelmed by their responsibilities, 58% felt overwhelming anxiety, 48% felt that things were hopeless, 35% felt so depressed that it was difficult to function and 10% seriously considered suicide.2 The 2017 survey of university and college counselling centre directors found that students with suicidal thoughts and behaviours now represent 25% of students seeking care in these centres, a 5% increase from the 2015 survey.4 A recent report from the Centers for Disease Control and Prevention also showed that while deaths by suicide in youth aged 10–24 years remained relatively stable between 2000 and 2007, there was a 56% increase between 2007 and 2017.6 These data indicate a pressing need for innovative, scalable and cross-cutting solutions to support adolescent and young adult mental health and wellness.

Given the prevalence of mental health symptoms and disorders on college campuses, a popular trend has been the establishment of ‘happiness classes’ such as Yale University’s ‘The Science of Wellbeing’ which aims to teach students evidence-based approaches for developing a happier and more satisfying life. The science of well-being includes lessons on misconceptions about happiness, overcoming biases, improving one’s happiness, implementing strategies to increase happiness and challenges that help learners work towards changing behaviours by changing their environment or with support from others.7 While such courses are proliferating across the country, a limitation of this work is the dearth of research demonstrating what health outcomes students gain and maintain from their participation. However, some literature indicate that such approaches could be effective. For example, a pilot study with 12 students who self-selected into a stress-reduction course using a cognitive behaviourally oriented approach found that students had significant reductions in anxiety and depression, and significant improvement in self-esteem from pretest to post-test, with all results maintained at 1-month follow-up.8 A pragmatic clinical trial of a four-session resilience programme integrated within university-orientation courses also found that students who received the programme had significant improvements in stress postintervention as well as depression at the end of the semester.9

The research on prevention efforts on college campuses to support mental health (including evaluation of college courses focused on well-being) has not kept pace with their rapid development and wide-scale implementation. Though systematic research has not yet established the prevalence of well-being courses on college campuses, the popularity of such courses (eg, Yale’s well-being course and Stanford University’s ‘Designing your Life’ course, which has been implemented in a variety of iterations on 15 campuses globally10) indicates that education as prevention is a popular and growing trend. Our team seeks to establish the health outcomes associated with prevention coursework using a course originally developed at the University of Washington (UW) by Dr James Mazza called ‘Wellness and Resilience for College and Beyond’. As with other college campuses with similar courses, it has become one of the most popular courses on campus, with nearly 1000 UW students enrolling per year. The course has become so popular at UW that Mazza’s team developed a follow-up course titled, ‘Thriving on the Path to Happiness’ which was fully enrolled at a section of 110 students the first quarter it was offered. The Wellness and Resilience Course (WRC) teaches students techniques to improve mental health from evidence-based psychotherapeutic interventions.

The WRC is primarily grounded in skills from dialectical behaviour therapy (DBT), an evidence-based treatment for chronically suicidal and self-injuring behaviour.11 To the best of our knowledge, there is no other college course which primarily teaches students DBT skills with the goal of improving students’ mental health. We focus most heavily on DBT skills because the model has repeatedly been shown to be effective for improving mental health outcomes in adolescents and adults.12–23 An advantage of this treatment is that, unlike therapies that target a specific symptom (eg, depression), DBT has demonstrated effectiveness24 25 for a wide range of mental health problems (eg, depression and anxiety,21 26 suicidality,15 16 addiction,14 27 28 eating disorders17 and increasing coping skill use),18 19 21 and thus, it is considered to be a cross-cutting (ie, transdiagnostic) treatment modality. Given the rates of suicidality in college populations, the WRC was developed to be potent enough to serve students already living with mental health disorders, but general enough to serve as prevention for those without such symptoms/disorders. DBT skills target four core skill domains: mindfulness, emotion regulation, distress tolerance and interpersonal effectiveness.29 The WRC course also includes concepts from the fields of positive psychology30 and acceptance and commitment therapy.31

The primary aim of this pilot study is to assess WRC’s acceptability, appropriateness and feasibility based on student reports. The secondary aim of this study is to compare changes in student’s self-reported use of adaptive coping skills over time for students in the WRC group versus a comparison group of students receiving university as usual (ie, not enrolled in the WRC). We hypothesise that students in the WRC will report greater improvements in use of adaptive coping skills when compared with controls. The tertiary study aim is to assess differences in changes of emotion dysregulation and suicide ideation over time between WRC and control groups.

Methods

Study design

This pilot study uses a quasi-experimental design to examine WRC’s acceptability, appropriateness and feasibility among students attending higher education institutions. This is a quasi-experimental study because we are not randomising participants (as this is beyond the scope of this feasibility study) and are instead allowing students to voluntarily enrol in the WRC (ie, the intervention group). In particular, we found individual-level randomisation to be extremely challenging given that students pay tuition for the WRC and thus are entitled to enrol in the course during the semester of their choosing as their schedules change from semester to semester. While class-level randomisation would be more feasible, implementing a comparison condition (ie, another type of course) is a large undertaking as it requires curriculum approval at sites, training of instructors and sufficient resources to deliver. Such an undertaking was well beyond the resources available for the current pilot, as would a cluster-randomised trial in which campuses were randomised to receive the WRC or ‘university as usual’. Instead, we will recruit a second group of students who are not enrolled in WRC but instead participate in ‘university as usual’ (ie, the comparison group). Participants in both groups will complete self-reported surveys at baseline (beginning of the semester and prior to WRC implementation for the intervention group), approximately 16 weeks after baseline (at the end of the semester, which is immediately after intervention implementation) and approximately 29 weeks after baseline (or 3 months after completion of the intervention). The University of Pittsburgh Human Research Protections Office (HRPO) approved all study procedures, as did site IRBs as required.

Patient and public involvement

The WRC was developed and refined over time based on student experiences and feedback about the course by the original course developers at the UW. Though students are the primary users of the WRC course, college campus staff, faculty and leaders are also a key stakeholder group. Throughout the study period, we will run an online stakeholder learning collaborative. The learning collaborative is composed of campus staff, faculty and leaders who will respond to questions about course content, implementation or other relevant topics via an email listserv on a monthly basis.

Setting and interventions

Setting

The pilot phase of this study was conducted on three campuses in Southwestern Pennsylvania including one large public university and two small private universities; results of this uncontrolled trial are forthcoming.32 Following the pilot phase, two additional campuses joined for this quasi-experimental phase. These campuses are located either in Southwestern Pennsylvania or West Virginia and include one large public university and a satellite campus of a large public university.

Training and implementation

Prior to the pilot study, interested faculty, staff and administrators from campuses throughout our region were invited to participate in a free 3-day training event designed to help participants learn to deliver the WRC. The training event was developed by the original course developer and outlined a variety of relevant topics including the rationale for the course, overview of course content with structured practice activities, assignment types and tips for grading and marketing the course to students. Course instructors involved with this study also have access to as needed consultation via email from the lead investigator and course developer.

Wellness and resilience course

As the WRC was originally implemented at the UW (which uses the quarter system), the course content was modified for delivery within a semester system. The course developer provided a full package of materials to deliver the course, which included PowerPoint slide decks for each lecture, homework assignments, a final examination and instruction sheets for grading and student feedback. The lectures were originally designed to be taught using teaching notes from the book, DBT skills training for emotional problem solving for adolescents (DBT STEPS-A), as this curriculum was designed to enable general education teachers to teach DBT skills to adolescents.33 All training attendees were given access to the course materials package and a copy of the DBT STEPS-A manual. During the pilot phase, lessons for each class meeting were further manualised by the first author, a DBT expert, who developed detailed teaching notes, instructions for activity and discussion facilitation and references to sections of the DBT STEPS-A manual should further information be required, to accompany each PowerPoint presentation; these materials are available to all course instructors. The final version of the WRC used in this study includes 14 weekly class sessions, reflective homework posts, skills practice tracking via ‘diary cards’ and a cumulative final examination, which are delivered over a standard 16-week semester. Each class session meets for 2.5 hours with approximately half of class time devoted to teaching new content (lecture) and the other half of class time devoted to practice and discussion in small groups. Please see the online supplementary material for a more detailed overview of the WRC lessons and assignments.

Participants and recruitment

Study participants will include students and course instructors at the five participating campuses (spring and fall 2020 semesters). Course instructors are clinicians from campus counselling centres (social work, counselling and psychology professionals or faculty) or faculty members from the fields of education or public health. All course instructors completed the training event, though one study site instructor changed jobs after the training occurred. A new instructor for this site was identified and supported by the trained instructor for that site, the study PI and a teaching assistant. WRC was approved by curriculum committees at each campus and given an appropriate number of course credits and a course number within the department offering the course. Individual sites are largely responsible for advertising the availability of the WRC as they see fit, though the study team did provide editable course marketing flyers and purchased social media ads to notify undergraduate students at each campus that a new course was available on their campus. On enrolment of the course, students are invited to participate in a research study investigating the effects of the course on student mental health and wellness. They are provided with a link to an online screener for eligibility via Qualtrics. Eligible participants are undergraduate students attending one of the five study sites, aged 18–24 years. If found to be eligible, students are directed to provide their contact information to allow enrolment by the study team. All surveys are delivered via RedCap electronic surveys; all participant identifying information is stored in a separate, secure database housed on secure university servers.

Though random assignment is beyond the scope of the present study, we are recruiting students from each participating campus who were not enrolled in the course to serve as controls receiving ‘university as usual’. Students are recruited through paid advertisements on social media and flyers distributed on campus or hung in campus buildings which allow them to access the same electronic eligibility screener used for students enrolled in the WRC. We anticipate enrolling 150 students in the intervention condition (based on the course section sizes as each site), thus we will aim to enrol a similar number into the comparison group. While testing intervention effects is not a primary outcome of this study, we are interested in exploring the appropriateness of students who receive ‘university as usual’ as a control group. Course instructors from each of the five campuses will be invited to complete brief, electronic measures of programme acceptability, appropriateness and feasibility at the end of each semester. Course instructors will be invited to complete these measures via email with a link provided via email.

Measures

WRC acceptability, appropriateness and feasibility are assessed by students who participate in the course and their instructors at the end-of-semester timepoint. Students and instructors will provide ratings on three brief, validated measures: the acceptability of intervention measure (AIM), the intervention appropriateness measure (IAM) and the feasibility of intervention measure (FIM).34 Each measure has four items, which are rated on a 5-point Likert scale ranging from completely disagree to completely agree. The AIM, IAM and FIM have demonstrated good structural validity, test–retest reliability and excellent internal consistency (α=0.85–0.91).

Use of adaptive coping skills will be measured at baseline, end-of-semester and at 3-month follow-up, using the validated DBT Ways of Coping Checklist.35 Student responses are scored for three validated subscales, the first for use of adaptive coping skills and two subscales for dysfunctional coping skills use, including general dysfunctional coping and blaming others. These scores have shown good internal consistency (α=0.84–0.96), good test–retest reliability, good criterion validity and sensitivity to change in both individuals with borderline personality disorder and diagnostically diverse psychiatric patient populations.35 36

Emotion dysregulation will be measured using the 18-item Difficulties in Emotion Regulation Scale—Short Form (DERS-SF).37 Similar to the original (long form) DERS,38 six subscales can be calculated indicating: (a) limited access to strategies for regulation, (b) non-acceptance of emotional responses, (c) difficulties with impulse control when distressed, (d) difficulties engaging in goal-directed behaviour when distressed, (e) lack of awareness of emotions and (f) lack of emotional clarity. The DERS-SF has been found to have sound psychometric properties, including good internal consistency (α=0.78–0.91). There is strong correspondence between the DERS-SF and long-form DERS with correlations ranging from 0.90 to 0.97 and 81%–94% shared variance.

Suicidal ideation, behaviours and risk will be assessed using the Suicide Behaviors Questionnaire—Revised Version (SBQ-R).39 At baseline, participants complete the SBQ-R focusing on lifetime and past 30-day suicide ideation, attempts, frequency of suicidal ideation over the previous 12 months, suicidal threat and self-reported likelihood of future suicide. Two validated cutoffs are typically used. A single-item measure to detect suicidal ideation and attempts has been shown to have sensitivity, specificity and positive predictive values all equal to 1.0 in a non-clinical sample of undergraduate students. An overall SBQ-R score represents the sum of four items and scores ≥7 are coded as being at risk of suicide with sensitivity, specificity and positive predictive values of 0.93, 0.95 and 0.70 in a non-clinical sample of undergraduate students.

Demographic survey items allow students to report potential confounders including age, gender, sexual orientation, socioeconomic status and past/current receipt of psychotherapy. Items related to the COVID-19 pandemic were also added to all surveys administered in April 2020 or later to allow us to adjust for these potential confounders. COVID-19 items include (but are not limited to) whether participants have tested positive or had family, friends or other loved ones who tested positive for COVID-19, how concerned participants were about access to basic needs (eg, housing, food), whether participants were responsible for the care of any children, elderly people or individuals with an illness, and if participants have a health condition that would heighten their risk for serious illness if they contracted COVID-19.

Analyses

For our primary outcomes, we will conduct descriptive statistics (ie, means and 95% CI) of WRC’s acceptability, appropriateness and feasibility among the intervention group only. Our benchmark for success on all three primary outcomes is a mean ≥4. Analyses will adjust for the cluster sampling (ie, students being nested within schools) and will be conducted in SAS V.9.4. We will not adjust for multiple tests due to the pilot nature of the study.40–46

For our exploratory aims, we will examine changes in adaptive coping, emotion dysregulation and suicidality over time between our intervention and comparison groups. First, we will assess demographic differences at baseline between participants in the intervention and comparison groups. We will compare the participant characteristics by study arm (eg, age, gender, sexual orientation, race/ethnicity, socioeconomic status, previous exposure to psychotherapy, diagnosed mental health conditions and baseline symptom scales for depression and other conditions). Variables significantly related to study arm will be tested for potential interactions and, in the absence of interactions, will be added as covariates in the analyses. All hypotheses will be two-sided tests with a significance level of 5% that adjust for school clustering.

We will then conduct multilevel multivariable linear or logistic regression models, depending on the distribution of the outcomes. We will conduct separate models for each follow-up period. Each model will include variables for time, study arm, their interaction and random effects for observations-within-student and students-within-school. These models will allow us to assess whether the interaction between time and study arm is significant and, if applicable, is in the hypothesised direction (ie, positive for adaptive coping). We will include as covariates any participant-level baseline characteristics associated with study arm. As a secondary analytic approach, we may estimate the effects of WRC on each outcome using inverse propensity score-based weighting.47 First, a logistic model will be fitted using the full dataset describing multivariable associations of potential confounders with intervention group. Individual weights will be calculated as the inverse of each participant’s probability of participation in the WRC. Balance of the conditioned groups will be assessed by comparing weighted distributions of key variables, and alternative methods of analysis including matching and stratification will be considered, if necessary, to attain a well-balanced control group. Finally, the effect sizes will be estimated based on linear or logistic regression models in the conditioned sample.

Power analyses

Based on the best practices for pilot studies,40–46 given our sample size for the intervention group (n=150) and 5% type I error rate, we have the ability to estimate 95% CI margin-of-errors of ≤0.24 for mean-based hypotheses (which we derived from the largest upper 95% CI limit of the SD from FIM, AIM and IAM in prior studies.34 48

Current trial status

Study subjects began enrolling at the end of December 2019 and data collection will continue through May 2021 when the final follow-up surveys are collected.

Ethics and dissemination

Ethics approval

All of the study procedures were approved as an exempt protocol for evaluation of educational curricula by the University of Pittsburgh HRPO; the study was approved as a research study by the institutional review board (IRB) of the fifth study site. The University of Pittsburgh HRPO served as the IRB of record for all except one study site (Carnegie Mellon University IRB), which required standard IRB review. No informed consent is required for students attending four of the five sites as the study protocol was exempt. Students attending the fifth study site provide their electronic informed consent in REDCap prior to beginning the first survey; all surveys are administered using REDCap. However, it is important to note that exempt study protocols are reviewed in detail by the HPRO prior to granting an exemption and investigators are still responsible for reporting all study modifications and completing continuing reviews per the standard human subjects’ monitoring processes. Thus, while the study is approved as an ‘exempt’ protocol at four of the five sites, the research team remains accountable to the oversight of the HPRO in ensuring the safety and welfare of all participants.

Safety considerations

Several precautions are taken within the WRC and the study to ensure participant safety. Within the WRC, students are informed that the course is not therapy nor should it be considered a part of any treatment plan. Course instructors were provided with instructions to follow their university policies and procedures if a student disclosed concerning information of any kind that would require an immediate response (eg, walking the student to the counselling centre for an evaluation). Within the study, all participants are treated as if they are at risk given the prevalence of mental health disorders in college students.3 At the beginning of each survey, participants are reminded that the research team does not review the data in real time and as such, if students are in need of assistance, they should use the resource sheets provided. Downloadable resource sheets tailored to the specific campus and local area of study sites is provided within all surveys. Directly following questions about suicidality, students receive direct instructions on what steps to take should they be at risk of harming themselves (eg, calling the national suicide hotline).

The study site requiring standard IRB review for this study required additional safety precautions for students participating from that campus. Specifically, branching logic was programmed into the survey such that students who respond to SBQ-R questions in a way that could indicate possible suicidality are presented with the option to have the director of health services of their campus reach out to them to provide direct assistance and referrals. Students are informed that while we would like to protect and maintain the confidentiality of their data, we also want to offer them the opportunity to get connected to resources given their responses to the questions. Students who indicate that they would like to be contacted from this campus provide a phone number and are informed that they will be contacted within 2 business days. A research assistant reviews all surveys from this study site as they are completed and makes reports to the health services director as needed.

Dissemination

Data from this study will be disseminated via conference presentations and peer-reviewed publications. Preliminary data will also be disseminated via our online stakeholder learning collaborative. This activity will allow us to receive rapid campus stakeholder feedback related to preliminary data, challenges in course implementation (eg, marketing techniques) or challenges with research procedures (eg, recruitment of control subjects).

Public health significance

Mental health disorders and suicidality are serious, persistent and prevalent public health concerns on college campuses.1–3 College students are presenting with mental health concerns in numbers too great to be addressed by individual-level services on campus, such as those offered by college/university counselling centres. Given this, innovative, scalable solutions for equipping young people with evidence-based skills to improve and maintain mental health are sorely needed. College courses designed to teach students such skills and facilitate the practice and generalisation of new, healthier behaviours are a practical solution because they are universal, scalable and financially sustainable for institutions of higher education (ie, students pay tuition to receive the course). While ‘happiness’ courses have become popular across the country, a paucity of research investigates the health outcomes associated with these courses. The WRC is significant and unique from other happiness or well-being courses in that it primarily teaches students skills from DBT, an evidence-based treatment originally developed for chronically suicidal behaviour.11 If the WRC can increase students’ use of adaptive coping skills and reduce emotion dysregulation and suicide ideation, colleges and universities can offer evidence-based prevention courses to their students on a large scale, thereby improving student mental health and making a substantial public health impact.

Limitations

This study protocol includes several limitations. First, the quasi-experimental study design makes the data more vulnerable to potential confounders. Individual-level randomisation is challenging when investigating the effects of college coursework for which students pay tuition. Though class-level or campus-level randomisation is more feasible, neither are within the resources of the present study. The primary purpose of the present study is to establish that the WRC is acceptable, appropriate and feasible prior to a large-scale trial. Should our preliminary findings be promising, we will pursue a fully powered evaluation of the WRC via a cluster-randomised controlled trial. Second, students self-select into the WRC. Third, control subjects do not receive a dose-matched intervention. Fourth, this study has limited generalisability due to the geographic region in which the research is being conducted. Fifth, as this is a pilot study and suicidal behaviour is a low-base rate behaviour, this study lacks power to adequately investigate suicidal behaviours. Finally, the COVID-19 pandemic has disrupted research internationally, including this project. We recognise that this may affect participants’ perceptions of the intervention (our primary outcomes) or our study’s exploratory outcomes. As such, we will examine if our outcomes were different in semesters before, during and following the pandemic. We also added questions specific to COVID-19 and will adjust for these as potential confounders if necessary.