Evaluating the Effectiveness of Online Educational Modules and Interactive Workshops in Alleviating Symptoms of Mild to Moderate Depression: A Pilot Trial.

INTRODUCTION: Depression is a common health concern in primary care with barriers to treatment well documented in the literature. Innovative online psychoeducational approaches to address barriers to care have been well received and can be cost effective. This pilot trial evaluated the effectiveness of an online psychoeducation curriculum intended to alleviate symptoms of depression while utilizing minimal staff resources.
METHODS: A small (n = 29) randomized control pilot study was conducted. Online psychoeducational content was delivered in 5 to 10-minute videos over 8 weeks. Participants engaged in moderated discussions on workshop topics. The Patient Health Care Questionnaire (PHQ-9) was used to measure pre/post scores. Two Likert scale questions were used to determine subjective changes in understanding of depression and coping skills.
RESULTS: Paired T-test analysis showed an average PHQ-9 improvement of 4.37 (P = .01) in the intervention arm and 1.81 (P = .172) in the control group. No significant difference in delta PHQ-9 score was found between groups via difference in difference analysis (P = .185). Effect size was 0.59. No improvement in Likert scores for question 1 or 2 were detected by paired T test in either group.
CONCLUSION: This pilot trial of interactive online psychoeducational content shows initial promise as there was a significant improvement in PHQ-9 scores within the intervention arm. The comparison of delta scores between intervention and control arms was not statistically significant although this is likely due to the underpowered nature of the pilot trial. This data trend justifies the need for a larger validation trial of this intervention.

RCT Entities:   Population Interventions Outcomes

Introduction

Depression is one of the most common mental health concerns addressed in primary care in the United States (US).[1] The estimated lifetime prevalence of depression in US adults is about 16%[2] and it was the second leading cause of disability in 2010.[3] Regular follow up with providers is key to managing depression and associated chronic medical co-morbidities.[4]

Several factors make access to adequate treatment challenging. In 2017, the psychiatric workforce was 6.4% below demand and this shortage is projected to nearly double by 2025.[5] Thus, some estimates report 43% to 60% of patients with mental illness are seen only in primary care settings.[6] Psychotherapy is one mainstay of treatment but Primary Care Physicians (PCP) rarely have time to address psychoeducation during an average primary care visit.[7]

A diverse range of online depression resources have emerged to fill this gap in care. In their 1-arm trial, Mohr et al found that utilizing mindfulness applications over 8 weeks led to a significant decrease in Patient Health Questionnaire-9 (PHQ-9) scores after the intervention.[8] Several 2-arm randomized control trials (RCT) have also shown positive effects of mindfulness-based smartphone apps on various aspects of wellbeing, compared with control groups.[9-11]

Beyond independently used mindfulness apps, studies have also evaluated the effectiveness of cognitive behavioral therapy (CBT) delivered via online platforms. A randomized clinical trial by Segal et al found significant improvements in depression symptomatology after participation in online structured CBT-based material.[12] Similar results are supported by 2 large meta-analyses.[13,14] Rollman et al included moderator-facilitated discussion groups with eight 50-minute computer-based CBT sessions.[15] Although it required a robust team of care managers, they found significant improvement in depression and anxiety symptoms, in the CBT group. Of note, the moderated support group plus CBT did not have benefit over CBT alone.[15]

Informed by successful elements and duration of previous models,[12,15,16] our 8-week intervention is designed to provide detailed yet easily accessible psychoeducational material. Furthermore, our intervention was designed to operate with minimal support staff to ensure sustainability in resource-limited outpatient offices. The aim of this study is to evaluate the effectiveness of our online depression education series and accompanying interactive discussion board on the symptoms of mild to moderate depression.

Methods

This study was designed as a randomized control pilot study. The trial consisted of an intervention and control group run in parallel with a 1:1 allocation ratio. Both arms were selected from a family medicine office in Reading, Pennsylvania. The inclusion criteria were participants 18 years and older, a PHQ-9 score between 5-19, and a diagnosis of depression. The exclusion criteria were concurrent diagnosis of bipolar, schizophrenia, personality disorder, intellectual disability, lifetime history of psychiatric admission or suicidal ideation within the past year. Participants meeting this criterion were felt to be high risk and thus not suitable for mental health services in an online environment. Participants were also excluded if they were enrolled in psychotherapy or if they did not have access to the internet. There were no upper age limit exclusions for the study.

All qualified study participants were contacted via telephone to complete an interview and PHQ-9. Participants were also asked to respond to 2 questions using a 5-point Likert scale ranging from strongly agree to strongly disagree. A value of 3 represented the comment “I don’t know.” The questions were: Q1: “I feel like I have a good understanding of depression,” and Q2: “I feel like I have good coping skills with my depression.”

A total of 243 individuals were eligible for the study. During the screening process, 163 were able to be contacted via phone; 53 met exclusion criteria and 81 declined to participate. A total of 29 were consented (Figure 1) via telephone. Verbal consents were recorded and stored on a password-encrypted digital device. Sample size was determined by the number of participants who met inclusion criteria.

Figure 1.

CONSORT flow diagram for enrollment and data analysis of patient sample in intervention and control arms.

Once all consents were complete, participants were divided into 3 groups: PHQ-9 5-10, 11-15, 16-19. The groups contained 12, 9, and 8 individuals, respectively. All participants were assigned a numeric value by a random number generator. Within each PHQ-9 severity group, participants were randomized in a 1:1 fashion to either the control or intervention arm. Neither the primary investigator for the study nor the participants were blinded to group assignment.

The duration of the trial was 8 weeks (December 2019 to February 2020). Individuals assigned to the control arm were notified of the allotment and received routine care at the discretion of their PCP (ie, office follow-up with or without pharmacotherapy). At the 4- and 8-week marks, the control group received gift cards (total 10 US Dollars). After the final week, a repeat PHQ-9 and Likert questionnaire was administered over the phone.

The intervention group underwent an 8-week online psychoeducation curriculum. Each week, they were asked to watch a 2- to 10-minute video that covered various psychoeducation topics. Information used in the videos was written at a grade school reading level. After each video, participants were expected to respond to a prompt in the discussion group. Pseudonyms were assigned to protect anonymity. A moderator reviewed the board daily to facilitate discussion. Video viewership was monitored. Participants in the intervention group received a gift card at 4 and 8-week intervals (total 10 US Dollars). A PHQ-9 and Likert survey were administered after the final week. At the conclusion of the study, each patient’s PCP received their final PHQ-9 score via a secure message in the electronic medical record so that appropriate management could be determined.

The videos were presented in an informal prerecorded psychoeducational format. Videos began with a discussion of how to get and give support. Later content covered common CBT tactics such as talking back to negative thoughts, identifying triggers for depression, and proactive ways to minimize depressive episodes. The final video focused on preparing long-term support systems.

The primary outcome of this study was change in PHQ-9 score after 8 weeks. The secondary outcome was a change in response to the 2 Likert scale questions. For the primary outcome, delta in scores within each arm was analyzed via a paired t-test utilizing SPSS version 25.0. Given the small sample size, a sensitivity analysis was run with a Wilcoxon non-parametric test to confirm the findings of the T-test. A comparison in the delta of PHQ-9 scores between the control and intervention groups was performed via a difference in difference analysis. Effect size was calculated utilizing Cohen’s d formula.

Participants who dropped out before the final PHQ-9 administration were excluded from analysis as a delta score could not be determined. Participants assigned to the intervention arm who did not start nor complete the video series were analyzed via intention to treat. No sub-group analysis was performed.

The nominal variables of the Likert questionnaire were replaced with numeric values 1 to 5. These results were analyzed via parametric means with t-testing. Intergroup deltas for each question was compared via difference in difference analysis.

Results

A total of 29 participants were enrolled in the study. There were 22 females and 7 males. The median age was 40. The self-reported race/ethnicity of participants were 4 Black, 12 Hispanic/Latino, and 13 White/Caucasian. Depression severity of participants was 12 mild (PHQ-9 5-9), 9 moderate (10-14), and 8 moderate-severe (15-19). At the start of the study, 21/28 participants were taking antidepressants. Demographic data is depicted in Table 1.

Table 1.

Demographic Data of Study Population.

Study Arm	Variable	Category	Count
Control	Age	<40	7
		≥40	7
	Gender	Male	4
		Female	10
	Race/Ethnicity	Black	0
		Hispanic	9
		White/Caucasian	5
	Depression severity	Mild	5
		Moderate	5
		Moderate-severe	4
	Depression medications	Yes	9
		No	5
Intervention	Age	<40	7
		≥40	8
	Gender	Male	3
		Female	12
	Race/Ethnicity	Black	4
		Hispanic	3
		White/Caucasian	8
	Depression severity	Mild	7
		Moderate	4
		Moderate-severe	4
	Depression medications	Yes	12
		No	3

Of the 29 participants, 14 were randomized to control and 15 to intervention groups. Within the intervention arm, 2 were lost to follow-up and 2 declined the final interview; of these 4 individuals, the average age and PHQ-9 score was 48 and 9.5, respectively. In the control group, 2 participants were lost to follow-up with 1 declining the final PHQ-9. The mean age and PHQ-9 of these participants was 28 and 7.6, respectively. No analysis was performed on dropouts as no final PHQ-9 score was obtained. Twenty-two participants were included in the final analysis (11 in both the intervention and control group).

In the control arm, the difference in mean PHQ-9 score from pre (12.36, (SD 4.65)) to post (10.55, (SD 5.07)) was not significant by paired t-test (P = .172). In the intervention arm, a significant reduction in scores from pre (10.55, (SD 3.48)) to post (6.18, (SD 3.46)) was found (P = .01).

In the control arm, the mean difference in Q1 score from pre (3.55 (SD 1.21)) to post (3.91 (SD 1.30)) trial was 0.36 (P = .476) and the difference in Q2 from pre (3.18 (SD 1.33)) to post (3.36 (SD 1.21)) was 0.18 (P = .506). In the intervention arm, the mean difference in Q1 score from pre (3.73 (SD 0.64)) to post (4.09 (SD 0.54)) was 0.36 (P = .167) and the difference in Q2 from pre (2.73 (SD 1.10)) to post (3.64 (SD 0.81)) was 0.91 (P = .074). These findings are depicted in Table 2.

Table 2.

Pre and Post PHQ-9 Scores and Question Responses for Intervention and Control Arms.

Study arm	Variable	Pre-result (n = 11)		Post result (n = 11)		P value
		Mean	SD	Mean	SD
Control	PHQ-9	12.36	4.65	10.55	5.07	.172
	Q1	3.55	1.21	3.91	1.30	.476
	Q2	3.18	1.33	3.36	1.21	.506
Intervention	PHQ-9	10.55	3.48	6.18	3.46	.010
	Q1	3.73	0.64	4.09	0.54	.167
	Q2	2.73	1.10	3.64	0.81	.074

Due to the small sample size, a sensitivity analysis of the T-test was run utilizing a Wilcoxon test. This analysis confirmed the results of the parametric test with the exception that intervention Q2 responses also showed significant change (P = .016). A difference in difference analysis (Table 3) was run to compare the mean PHQ-9 and Likert score differences between the intervention and control group. For PHQ-9 scores, the P value was 0.185 (d = 0.59). The results for comparison of Q1 and Q2 were P = 1.00 (d = 0) and 0.183 (d = 0.61), respectively.

Table 3.

Difference in Difference Analysis for PHQ-9 Scores and Questionnaire for Intervention and Control Arms.

Variable	Control (n = 11)		Treatment (n = 11)		P value	Effect Size
	Delta	SD	Delta	SD
PHQ-9	−1.81	4.09	−4.37	4.59	.185	0.59
Q1	0.36	1.63	0.36	0.81	1.00	0
Q2	0.18	0.87	0.91	1.51	.183	0.61

Of participants in the intervention arm, 72.7% completed the first video and 36.4% completed all 8 videos. Eighty-one-point 8% of participants posted at least 1 comment to the discussion board. The average number of posts to the discussion group was 12 (median 10, range 4-31).

Discussion

This randomized control pilot trial tested the effectiveness of online psychoeducational modules and discussion groups in treating mild to moderately severe depression. The study was designed to operate with minimal support staff relative to other online interventions.[15] If comparable results are seen, this intervention could provide a sustainable model for implementation into primary care offices.

The initial findings from the pilot study are promising although limited by a small sample size. Participants in the intervention group had a significant 4.37 improvement in their PHQ-9 scores compared to no significant improvement in the control arm. A mean improvement of >4 is clinically relevant as this can indicate a change in depression severity on the PHQ-9 scale. The symptom improvement in our study is comparable to other online modalities with average PHQ-9 improvements ranging from 2.70 to 3.60.[12,16]

While the data from pre and post PHQ-9 show trends of improvement within the intervention arm, a significant between-group difference was not found using the difference in difference analysis. This finding may be influenced by the small number of participants. With an effect size of 0.59, it is reasonable to suspect that a larger N would detect a significant difference between intervention and control groups.

Upon completion of the study, participants were asked questions about their understanding of depression and coping skill inventory (Q1 and Q2). While there was no intra or intergroup difference of response to Q1 nor Q2, the response change in Q2 for the intervention arm (“I feel like I have good coping skills with my depression.”), demonstrated a trend towards significance. This result is a promising reflection of the intervention as the 8-week curriculum focused on exploring coping skills.

The discussion group portion of the intervention arm was intended to reinforce the material presented in the educational modules. Participants voiced appreciation for the respect for privacy in the discussion forum, and the sense of shared hopefulness. As the weeks went on, there was a growing cohesion within the discussion group.

Regarding intervention participation, 81.8% of participants posted at least 1 comment in the discussion board. This is greater than a similar study where only 61.8% of participants posted ≥1 comment.[15] Furthermore, 72.7% of participants in this study watched at least 1 video (vs 83.6%)[15] and 36.4% completed all 8 videos (vs 36.7%).[15] This data is promising because despite less staff support for the intervention, there were similarities in the participation metrics when compared to the larger Rollman et al trial.[15]

An important purpose of the pilot study was to ensure that a less resource-intensive program could be conducted with adequate safety: specifically, regarding suicidality. The primary contact for participants was the website moderator who reviewed comments daily. The moderator underwent training to identify suicidal individuals and could consult a physician to facilitate emergency services. During the entire 8-weeks, there were no actively suicidal participants. Passively suicidal individuals were identified and managed via timely follow-up with their PCP’s office.

A limitation for this trial was the small sample size which restricts the ability to validate positive trends seen in the data. Another consequence of the small N was a greater proportion of women to men in the sample and unequal distribution of race/ethnicities within study arms due to random chance. A larger multi-site trial should be sufficiently powered to analyze current trends, ensure more equal gender representation and be better apt to equally distribute race/ethnicity through randomization. With a larger N, it will also be beneficial to perform sub analysis to determine if response to intervention varies by gender or race/ethnicity. The use of antidepressants among participants posed another potential limitation. Although we included antidepressant use in Table 1, we did not evaluate duration of antidepressant use and thus this represents a potential confounding factor.

In a future trial, we plan to follow up with patients at 6 months post-study to investigate lasting effects of the intervention. We also plan to divide intervention arms into online modules plus discussion group versus online modules alone to investigate whether discussion groups contribute additional value to improving PHQ-9 scores. Finally, qualitative measures may also be used to gain a deeper appreciation of which patients respond best to the online modules.

Conclusion

This pilot randomized control trial evaluated the effectiveness of psychoeducation modules and discussion forums in reducing PHQ-9 depression scores. The intervention was designed to be less resource intensive than prior models so it can be sustainably implemented in a variety of primary care settings. There were no safety concerns identified in this 8-week trial period. Results demonstrated trends in PHQ-9 improvement within the intervention arm. There was no statistical difference between control and intervention group. There was no significant improvement in either arm regarding participant response to questions of whether they felt improvement in their understanding of depression nor development of coping skills. This study is limited by its small sample, but results do show positive trends that validate the need for a larger multi-clinic trial to further test the effectiveness of the intervention.