An open trial of internet-based cognitive behavioral therapy for first year medical students.

OBJECTIVE: Medical students experience high rates of depression, and often face barriers to receiving traditional mental health services. Internet-based cognitive behavioral therapy (iCBT) programs offer a more accessible method of receiving care. Here, we conducted an open trial of an iCBT program for medical students and characterize program usage, program users, and self-reported psychosocial symptoms and coping skills.
METHODS: All incoming first year medical students at a large state-run university were invited to use an iCBT program which focused on mood management and mood symptom prevention. Participants received access to the 16-week program and completed measures of perceived stress, quality of life, and the development of cognitive and behavioral coping skills at baseline and end of program.
RESULTS: Of the 194 students in the class, 53 (27.32%) signed up to use the program. While the program attracted a representative portion of underrepresented minority students, program engagement among males was particularly low. Repeated use of the program was low. Self-reported symptoms of depression and anxiety were low at baseline, and continued to be low at end of program. Slight increases were observed from baseline to end of program in the self-reported use of cognitive coping skills.
CONCLUSIONS: Digital mental health tools appear to be of interest to first year medical students, but need to be better designed to support continued program use and to attract specific subgroups of students who may face additional barriers to seeking mental health services.

Introduction

Medical school presents a period of high psychological distress for many students. The overall prevalence of depression and depressive symptoms among medical students is 27.2%, which is higher than in the general population (Rotenstein et al., 2016). It is clear that many medical students would benefit from mental health services, but research consistently demonstrates that they are unlikely to access these services. Medical students are less likely than the general population to received appropriate treatment (Rosenthal and Okie, 2005; Tjia et al., 2005; Chew-Graham et al., 2003), and report high levels of barriers to mental health treatment. While some of the barriers are those commonly observed in general populations (e.g., cost, time), others are unique to their position in the healthcare system, such as privacy concerns that their fellow students or attending physicians will learn about their seeking treatment (Schwenk et al., 2010; Estabrook and Christianson, 2013; Hankir et al., 2014).

Medical schools across the United States are increasingly acknowledging the challenges of providing students with accessible mental health care, as well as the importance of its provision. The United States Liason Committee for Medical Education (LCME) standards for accreditation of medical education programs specifically state that medical schools must provide personal counseling services, healthcare services and access to care by professionals who have no involvement in their academic assessment. Further, the Action Collaborative on Clinician Well-Being and Resilience has formed within the National Academy of Medicine to identify strategies to prevent and treat emotional distress and promote well-being, starting with medical trainees.

Mental health programs delivered online may offer a solution. Internet-delivered programs are seen to circumvent many of the barriers to traditionally-delivered services (Renton et al., 2014). Given that the majority of medical students today are digital natives, and often use the internet to learn about and manage their own health (Kontos et al., 2014), this makes them a prime audience to use and benefit from internet-delivered mental health programs. A variety of internet-delivered mental health programs have been developed to both prevent and treat mental health problems, including, but not limited to, those based on evidence-based treatments such as cognitive-behavioral therapy and those based on the provision of social support (Clarke et al., 2015; Ali et al., 2015). Over the years, internet-based cognitive-behavioral therapy (iCBT) has repeatedly demonstrated efficacy for treating a wide range of mental health concerns (Andrews et al., 2010; Karyotaki et al., 2017). Results from a small pilot study of our iCBT program for medical students indicated the program was perceived as useful, and there was interest from students in making the program accessible to all students (Lattie et al., 2017).

Here, we present on an open trial of our self-guided iCBT program for first year medical students. The primary aims of this study were to examine program usage and to examine the demographics of students who used the program. Secondary aims of this study were to examine the impact of the program on perceived stress, quality of life, and the development of cognitive and behavioral coping skills.

Methods

Recruitment

Participants were recruited from the University of Illinois at Chicago first year medical student (M1) class. All incoming M1 s were eligible to participate in the intervention. Recruitment was conducted during orientation week. A clinical psychologist and the Senior Associate Dean of Students (authors JD and KK, respectively) made a brief 10-min presentation during orientation about mental health services available to students, including traditional face-to-face services not discussed in this paper as well as the web intervention. General information about the web intervention was presented along with screenshots. Students received an email shortly after the presentation inviting them to enroll in the intervention. Interested individuals could click on a link in the email to complete an online consent and baseline measures prior to receiving a login to the web intervention.

Procedure

Participants completed online questionnaires at baseline and end of treatment. Follow-up assessments were collected following the completion of fall semester finals. Study data were collected using Research Electronic Data Capture (REDCap), a secure web-based survey tool hosted at the University of Illinois at Chicago (Harris, 2009).

The ThinkFeelDo Medical Student Intervention

The ThinkFeelDo Medical Student Intervention (TFD-MSI) was a mobile accessible web application that consisted of 16 lessons and five tools. The original ThinkFeelDo basic program was developed at Northwestern University's Center for Behavioral Intervention Technology (CBITs) and was designed to treat major depressive disorder. TFD-MSI is a modified version of the original site that was re-written to focus on well-being, stress management, coping skills, and the management of depressive and anxious symptoms. The program was designed to be a resource for a broad array of medical students who may or may not be distressed at the time of program initiation and use. Medical student specific examples and content were also created, based on feedback from an initial pilot with students (Lattie et al., 2017).

Lessons included text and video and required approximately 10 min to complete. There were “core” lessons that covered basic topics of cognitive behavioral therapy (e.g. behavioral activation, cognitive restructuring) as well as medical student specific topics (managing test anxiety, preventing burnout, etc.). The site included five interactive tools designed to support implementation of cognitive behavioral skills. Tools included activity scheduling and monitoring (activity tracker), cognitive restructuring (thought tool), goal setting, relaxation, and mood tracking. The included tools are those known to be effective components of cognitive behavioral therapy-based interventions targeting both prevention and treatment (Beck, 2011).

Measures

Site use variables included frequency and duration of overall site use as well the frequency of use of site components, including the lessons and tools. Student demographics were reported on the baseline questionnaire, and class-wide demographics were pulled from student registration documents.

Psychological measures were all self-report and were included in the study to measure the array of psychological health factors targeted by the TFD-MSI program. The Medical Student Well-Being Index (MSWBI) was used to measure levels of psychological distress in the medical students (Dyrbye et al., 2010). The Perceived Stress Scale (PSS) was used to measure subjective experiences of stress (Cohen et al., 1983). Cognitive and Behavioral Response to Stress Scale (CB-RSS) was used to track the frequency and perceived usefulness of cognitive and behavioral coping skills utilized by the students (Miner et al., 2015). The Patient Health Questionnaire-8 (PHQ-8) was used to measure depressive symptoms (Kroenke et al., 2009). Anxiety was measured with the GAD-7 (Spitzer et al., 2006).

Statistical analyses

Descriptive statistics were performed on website use data to examine program usage and on psychosocial measures to examine student mental health. McNemar's tests were run to examine changes in the proportion of participants responding “yes” and “no” to items on the MSWBI (Dyrbye et al., 2010). Chi-square tests were run to examine the demographics of participants who started the program, compared to those who never started and compared to the entire M1 class. t-Tests were used to examine changes in psychosocial measures from baseline to follow-up (Table 1).

Table 1

Demographics of M1 class.

	Total M1 classn = 194
Gender
Male	92 (47.4%)
Female	102 (52.6%)
Ethnicity/race
Hispanic or Latino	43 (22.2%)
Asian, non-Hispanic/Latino	63 (32.5%)
Black or African American, non-Hispanic/Latino	15 (7.7%)
White, non-Hispanic/Latino	64 (32.9%)
American Indian or Alaska Native	0 (0%)
More than one race	6 (3.1%)
Unknown/not reported	3 (1.5%)

Results

Study sample

Out of the incoming medical student class of 194 students, a total of 53 students enrolled in the study, completed a baseline assessment, and were provided with a login to the TFD-MSI program. With an initial response rate of 27.32%, this indicates that many incoming medical students are interested in digital resources aimed at promoting mental health.

Characterization of study participants

Of the 194 students in the class, 53 (27.32%) signed up to use the TFD-MSI program. A significantly larger portion of female students (34.31% of females) signed up to use the program compared to male students (19.57% of males), χ2(1, N = 194) = 5.29, p = .02. The study sample had a similar proportion of students identifying as Hispanic or Latino (n = 10, 18.87%) as were in the total class (n = 43, 22.16%), and a larger proportion of students identifying as Black or African American (n = 6, 11.32%) than were in the total class (n = 15, 6.62%).

General site use

Of the 53 students who received logins to the TFD-MSI program, 36 students (approximately 68%) logged into the site at least once and are referred to as “users”. Table 2 compares the demographic and symptom profiles of the full sample of participants, the sample of participants who never logged into the program (the “never users”), and the sample of participants who logged in at least one time (the “users”). As seen in Table 2, male participants were disproportionately represented in the “never users” category, indicating that they were less likely to log in and check out the program website, χ2(1, N = 53) = 10.55, p = .001. A trend appeared in the students who identified as Hispanic or Latino appeared to be less likely to check out the program website, but this was not statistically significant, χ2(1, N = 53) = 1.82, p = .18.

Table 2

Demographics and baseline psychosocial measures for study participants.

	Total samplen = 53	Never usersn = 17	Program usersn = 36
Age M (SD)	23.76 (2.33)	23.18 (1.02)	24.00 (2.74)
Gender
Male	18 (34.0%)	11 (64.7%)	7 (19.4%)
Female	35 (66.0%)	6 (35.3%)	29 (80.6%)
Ethnicity
Hispanic or Latino	10 (18.9%)	5 (29.4%)	5 (13.9%)
Race
Asian	9 (17.0%)	3 (17.6%)	6 (16.7%)
Black or African American	6 (11.3%)	1 (5.9%)	5 (13.9%)
White	29 (54.7%)	11 (64.7%)	18 (50.0%)
American Indian or Alaska Native	0	0	0
More than one race	6 (11.3%)	2 (11.8%)	4 (11.1%)
Unknown/not reported	3 (5.7%)	0 (0%)	3 (8.3)
Psychosocial Measures M(SD)
PHQ-8	1.09 (1.85)	0.60 (0.97)	1.26 (2.04)
GAD-7	1.43 (3.22)	0.20 (0.42)	1.26 (2.04)
PSS	15.95 (6.54)	13.90 (5.70)	16.21 (6.52)
CB-RSS Cognitive Frequency	14.40 (5.48)	14.30 (6.75)	14.21 (5.09)
CB-RSS Cognitive Usefulness	18.51 (6.56)	19.60 (8.11)	18.12 (6.23)
CB-RSS Behavioral Frequency	21.64 (4.08)	20.5 (3.81)	21.82 (4.12)
CB-RSS Behavioral Usefulness	27.98 (4.63)	28.10 (6.29)	28.06 (4.15)

Of the “users”, the mean number of logins was 4.3 and the median number of logins was 2.5. The degree to which these students engaged with the lessons was variable (ranging from 0 to 16 lessons read), and the average student read 3–4 lessons (M = 3.61, SD = 3.30). Use of the thought tool was low with a mean use of 1.02 (SD = 2.79), as was use of the mood rating tool (M = 0.83, SD = 1.25) and the relaxation tool (M = 0.19, SD = 1.17). Engagement with the activity tracker tool was also variable (ranging from 0 to 95 activities recorded; M = 20.14, SD = 27.15).

Psychosocial outcomes

Of the 53 students who completed the baseline questionnaire, 21 returned for follow-up assessment and 18 of the 21 were students who had used TFD-MSI at least once. Students reported very low levels of depressive and anxious symptoms at baseline (GAD-7 M = 1.4, PHQ-8 M = 1.1), but several reported elevated levels of stress. A cutoff of 20 is sometimes used on the PSS, and ~30% of students reported “high stress” per this cutoff at baseline.

As seen in Table 3, there were not significant changes in PHQ-8, GAD-7, or PSS baseline to follow-up among those participants who returned for follow-up assessment. As reported on the CB-RSS, there was a slight increase in the frequency of cognitive coping skill use from baseline to follow-up.

Table 3

Psychosocial measures at baseline and follow-up.

	Baseline	Follow-up	Test of difference
Psychosocial measures M(SD)
PHQ-8	1.84 (2.41)	1.95 (2.56)	t(18) = −0.58, p = .567
GAD-7	2.50 (4.03)	1.65 (3.44)	t(16) = 1.65, p = .119
PSS	17.89 (6.82)	17.24 (7.93)	t(17) = 0.54, p = .595
CB-RSS Cognitive Frequency	14.50 (5.55)	16.83 (4.11)	t(17) = −3.19, p = .005
CB-RSS Cognitive Usefulness	17.95 (5.93)	19.21 (4.72)	t(18) = −0.91, p = .377
CB-RSS Behavioral Frequency	20.68 (4.35)	21.00 (4.57)	t(18) = −0.35, p = .728
CB-RSS Behavioral Usefulness	27.50 (5.34)	28.22 (4.11)	t(17) = −0.69, p = .502

While self-reported depressive and anxious symptoms were low at baseline, 16% of participants in the full sample reported that they'd been bothered by feeling down, depressed or hopeless in the last month, 20.4% reported they felt all the things they had to do were piling up so high they couldn't overcome them in the last month, and 24% reported that they'd been bothered by emotional problems in the last month.

In the sample of participants who returned for follow-up assessment, 19% reported that they'd been bothered by feeling down, depressed or hopeless in the last month, 23.8% reported they felt all the things they had to do were piling up so high they couldn't overcome them in the last month and 61.9% reported that they'd been bothered by emotional problems in the last month. This demonstrates that students who returned for follow-up assessment were more likely to have perceived themselves as having been bothered by emotional problems when first joining the study.

Among this group of participants at follow-up, 42.9% reported that they'd been bothered by feeling down, depressed or hopeless in the last month, 52.4% reported they felt all the things they had to do were piling up so high they couldn't overcome them in the last month, and 61.9% reported that they'd been bothered by emotional problems in the last month. While this increase seen in students reporting that they have felt depressed or stressed was not reflected in the more standard measures of mental health symptoms (i.e. the PHQ-8 and PSS), it represents a self-perception of poorer mental health at the end of the first semester of medical school.

Discussion

Interest in this online mood management program was high, with more than a quarter of students signing up to use it. While past research on online mental health programs for medical students has been limited, this level of interest is nearly equivalent to what was seen in a similar program for second and third year medical students at a New Zealand university (Moir et al., 2015) and slightly lower than a similar program for medical and health science graduate students at an American university (Howell et al., 2018), indicating that the desirability of such programs is likely widespread.

While interest was high, actual program usage was low. Nearly a third of students who signed up for the program never logged in to access it, and the average student who did log in read just 3–4 of the 16 available lessons. These low rates of continued use are common in iCBT programs that are universally available in nature (i.e., in which all members of a population are eligible to participate, rather than only people who are deemed at-risk for, or are diagnosed with a condition (Muñoz et al., 2016; Christensen et al., 2006).

While the relationship between usage and outcome is unlikely to be linear (Donkin et al., 2013), these programs strive to provide users with at least an introduction to psychotherapeutic concepts and tools. Thus, while common, these usage rates are not ideal and hint that there were problems with program design. Here, we define program design as not just the design of the user interface and functioning, but rather the design of all components that impact the user experience of the program (including, but not limited to program content).

This program was designed and tested as a fully self-guided, primarily text-based iCBT program for incoming first year medical students. Without a doubt, a self-guided program design has certain strengths. Muñoz (2010) referred to these types of programs as “non-consumable,” noting that there are not consumable constraints (such as clinician time that is used and unable to be reused). This means that self-guided programs can be delivered to large populations without the need for clinician time, and thus can be more scalable and delivered at a lower initial cost. However, there is a body of literature on coach-guided versus self-guided iCBT programs which generally finds that the provision of human support improves engagement and outcomes (Andrews et al., 2018; Richards et al., 2015), and a recent review on the cost-effectivness of online mental health programs suggested that guided programs may be more cost-effective than unguided programs, despite the higher initial cost per participant (Donker et al., 2015). By designing this program to be self-guided, we may have lost contact with students who would have been able to meaningfully engage with the program given additional support.

However, at this stage, it would be unwise to conclude that human support is a necessary ingredient. Support aimed at engaging users can come in forms other than human support, including but not limited to automated messaging (via email or SMS), virtual rewards, tailored success stories and other interactive materials. In a recent study that tested multiple engagement strategies using a factorial design, human support improved outcomes during the intervention period, but those who received automated messaging for support experienced more change during the post-intervention period (Kelders et al., 2015). To determine if medical students are more likely to engage in coach-guided iCBT programs than they are to engage in self-guided programs with other motivational supports included in the design, additional research needs to be conducted.

In addition to a more engaging program design, the low rates of continued program use observed in this study suggest a need for a comprehensive implementation plan. The field of implementation science, which studies the methods used to promote the systematic uptake of evidence-based practices into routine practice, has grown tremendously in the last two decades (; Brown et al., 2017). Lessons from the field of implementation science encourage use of diverse strategies to facilitate the uptake and adoption of evidence-based programs (Powell et al., 2015). While digital mental health program offer the promise of greatly expanding the accessibility of evidence-based care, this vision has largely been unrealized. There are few examples of successful implementations of such programs in routine care settings (Fleming et al., 2018; Rogers et al., 2017), and many researchers are beginning to point to the need to incorporate plans for implementation early in the design of digital mental health programs (Mohr et al., 2017). In this study, the program's availability was advertised during orientation (albeit from presumably trusted sources), and there was a lack of ongoing support for embedding the program into students' lives. While this approach allowed a sizeable portion of the student body to initiate the program, future research should focus on testing known implementation strategies such as using media to advertise program availability, identifying and preparing champions on campus (that is, individuals who dedicate themselves to supporting, marketing, and driving through the implementation effort), and using systematic reminders (such as, students be reminded each year of the program as those who expressed interest in the first year may more likely need it the second year with the increased stress of the USMLE Step 1 licensure exam and the start of clinical core rotations).

While digital mental health researchers have long been focused on closing the treatment disparity gap, it remains common for digital mental health interventions to be predominantly used by non-Hispanic White females (Titov et al., 2018; Margarita Alegría et al., 2016). Males were less likely than females to sign up for the program, and were less likely to sign into and use the program after having signed up for it. While women in the general population report higher rates of common mental health problems than do men (Schwenk et al., 2010; Whiteford et al., 2013), research on gender differences in medical student mental health has been mixed with some studies finding no differences in rates of depression by gender (Dyrbye et al., 2006). The rates of depression, anxiety, and suicidality among male medical students are far from negligible, and point to a need for effective ways to engage male in mental health promotion programs.

While a representative portion of Hispanic students signed up for the program, and a larger than representative portion of Black/African American students signed up for the program, there appeared to be room for growth in attracting and retaining ethnic and racial minority students for this mood management program. Past research has demonstrated that ethnic and racial minority individuals often face additional barriers to mental health services (McGuire and Miranda, 2008), and while research on comparative rates of mental health problems among minority medical students is mixed (Dyrbye et al., 2006), this continues to be an important and sensitive treatment gap. Future design work should focus on increasing the perceived usefulness, accessibility, desirability and credibility of this program for male students and ethnic and racial minority students to help close these treatment gaps.

This study is not without limitations. To promote access for all interested students, the TFD-MSI program was tested as an open trial, and did not include a control group. The program was introduced at a single school to a single class of incoming students, and thus may not generalize to other medical school environments. There is evidence that different aspects of the medical school environment (e.g. teaching methods used (Camp et al., 1994), existing stress management resources (Drolet and Rodgers, 2010)) has an impact on student mental health. Thus, the TFD-MSI program may be perceived and used differently by students in other medical school settings. Because this study took place in large metropolitan area, in which students may be able to more readily access mental health services, it is possible that students could be less interested in iCBT programs compared to students in more rural environments in which mental health services are more limited. While the decision to target incoming first year medical students with this intervention was made based on feedback gathered during our pilot study (Lattie et al., 2017), the fact that students were introduced to it during orientation week may explain why the sample endorsed much lower mood symptom reports than is typically observed in populations of medical students who are further into their training (Dyrbye et al., 2006).

There were very low reports of symptoms of depression and anxiety at baseline, and no changes observed in symptoms of depression, anxiety or stress from baseline to end of program. Prior to this study, we did not know the level of symptoms that would be documented at baseline in this medical student sample, but given the low levels of symptoms, there was a low likelihood that significant changes would be observed in the included follow-up time. Past iCBT interventions that focused on prevention and that enrolled participants without elevated symptoms at baseline have typically had longer follow-up periods (Deady et al., 2017). Any statistically significant decreases in depression and anxiety that could have been observed would be unlikely to be clinically meaningful due to the low symptom reports at baseline (Jacobson and Truax, 1991). However, consistent with our pilot study, there were changes observed in the frequency with which participants reported used cognitive coping skills, such as trying to notice what they were thinking when they got upset, or taking a moment to question their interpretation of what was happening when they got upset. Thus, there is indication that the program prompted participants to utilize healthy coping skills, which could have downstream beneficial effects not captured in our limited follow-up period. Thus, the limited follow-up period is a notable limitation.

As medical schools across the U.S. and around the world increasingly recognize the value of supporting student mental health and wellness throughout training, online mood management programs are likely to join the array of common mental health resources offered to students. This study demonstrates that students are interested in using such programs, and offers insights into how these programs can be designed and tested to reach their full potential.

Funding

ThinkFeelDo was developed with support from research grant from the United States National Institute of Mental Health (R01 MH095753) to David C. Mohr, Ph.D. Dr. Emily Lattie is supported by a research grant from the National Institute of Mental Health (K08 MH112878).

Declaration of competing interest

None.