Feasibility and clinical utility of a transdiagnostic Internet-delivered rational emotive and behavioral intervention for adolescents with anxiety and depressive disorders.

BACKGROUND: Transdiagnostic interventions delivered for children and adolescents show promising results. In order to increase access to treatment, recent developments in technology have led to alternatives to face-to-face interventions. Transdiagnostic Internet-delivered interventions can be effective for adult anxiety and depressive disorders, but research is more limited regarding the efficacy of such treatments for young populations diagnosed with anxiety and or depressive disorders.
OBJECTIVE: This study aims to investigate the feasibility and clinical utility of a transdiagnostic Internet-delivered intervention based on Rational emotive behavior therapy for adolescents diagnosed with anxiety and depressive disorders.
METHODS: We tested the preliminary efficacy of a six-week intervention in an open trial, pre-post design.
RESULTS: Our results suggest that the intervention is feasible and can be included as a treatment for adolescents with anxiety and depressive disorders, being associated with moderate to large pre-post effect sizes on self-reported anxiety and depressive symptoms, as well as on negative patterns of thinking and knowledge acquisition. Satisfaction with the intervention was high.
CONCLUSIONS: Transdiagnostic Internet-delivered interventions for adolescents with anxiety and depressive symptoms are feasible. Future testing of the efficacy of such interventions in randomized controlled trials should be conducted.

Introduction

Anxiety and depressive disorders, commonly characterized as internalizing disorders, are prevalent in adolescence (Polanczyk et al., 2015), with a significant impact for the affected youth, family and society (Pape et al., 2012). The lifetime prevalence of anxiety disorders is almost 30%, while the prevalence of depression in adolescents is estimated to be around 7.7%–15.9% (Merikangas et al., 2010). Anxiety and depressive disorders in adolescence persist into adulthood (Ranøyen et al., 2018). Furthermore, the presence of depression in adolescence increases the risk of both depression as well as anxiety disorders later in life (Johnson et al., 2018), and it is associated with poor adult functioning (e.g., physical health, risky behavior, financial/educational and social functioning) (Copeland et al., 2020). There is a high comorbidity between anxiety and depressive disorders in children and adolescents (Melton et al., 2016), and this results in an increase in severity of symptoms, suicide attempts, impaired global functioning, and a poorer response to treatment (Garber and Weersing, 2010).

Despite the fact that evidence-based treatments for adolescent anxiety and depressive disorders exist, disorder-specific approaches do not take into consideration comorbidities between disorders (García-Escalera et al., 2016) which are rather the rule and not the exception. Up to 82% of children and adolescents with depressive disorders present a comorbid anxiety disorder (Ghandour et al., 2019). A meta-analysis investigating temporal changes in treatment for child and adolescent mental health reported that the effect sizes for depressive symptoms decreased significantly over time (Weisz et al., 2019). Moreover, psychotherapy for youth depression has only a modest effect size according to another meta-analysis, in which the effect sizes were in the small to medium range at post-treatment (g = 0.36) or follow-up assessments (g = 0.21) (Eckshtain et al., 2020). Therefore, given these results, there is a need for novel transdiagnostic treatment approaches for child and adolescents anxiety and depression.

Although there is no uniform definition on transdiagnostic interventions, as transdiagnostic approaches are heterogeneous (Fusar-Poli et al., 2019), Schaeuffele et al. (2021) propose a broad definition of them. Namely, transdiagnostic treatments represent those interventions that target several comorbidities at the same time either by targeting common mechanisms across disorders or by tailoring interventions for each patient's unique clinical presentation of comorbidities. Transdiagnostic cognitive-behavioral interventions, in which both anxiety and depressive disorders are targeted within the same protocol, have proven to be effective both in a traditional face-to-face format, both for adults and with preliminary results for children and adolescents (García-Escalera et al., 2016), and online (Păsărelu et al., 2017). However, little research has been conducted to determine the efficacy of transdiagnostic Internet-delivered interventions developed for children and adolescents (Wahlund et al., 2020). Transdiagnostic Internet-delivered interventions are effective for adults with anxiety and depressive disorders, associated with medium to large effect sizes for anxiety and depressive outcomes, and moderate effect sizes for quality of life (Păsărelu et al., 2017). Research on the efficacy of transdiagnostic Internet-delivered interventions for young populations is scarce. In fact, the single transdiagnostic Internet-delivered intervention developed for adolescents with anxiety and depressive symptoms is the one developed by Sandín et al., 2019, Sandín et al., 2020. The preliminary results indicated that a transdiagnostic Internet intervention based on the Unified Protocol for Transdiagnostic Treatment of Emotional Disorders in Adolescents (UP-A; Ehrenreich-May et al., 2017, Ehrenreich-May et al., 2018) was feasible and significantly reduced anxiety and depressive symptoms in a sample of Spanish adolescents (N = 12) (Sandín et al., 2020).

Transdiagnostic approaches have been used both in the treatment and prevention of internalizing problems in adolescents. Namely, the UP-A adapted as a universal prevention program delivered in a school context (N = 151) reduced anxiety and depressive levels as compared to a waitlist control condition for those students with a higher baseline level of emotional problems (García-Escalera et al., 2020). Also, research into the mechanisms of change indicated that changes in two constructs, namely experiential avoidance and distress tolerance, were associated with changes in symptoms after treatment (Sherman and Ehrenreich-May, 2020). Preliminary data on transdiagnostic treatments are also available for children with bullying-related anxiety and depression (Chu et al., 2015), adolescents at risk of developing serious mental illness (Weintraub et al., 2020), or transdiagnostic intervention delivered via telehealth for children with emotional disorders (Kennedy et al., 2020).

Rational emotive and behavioral therapy (REBT; Ellis, 1994) is particularly suitable to be implemented as a transdiagnostic intervention given the fact that rational beliefs play a central role in changing dysfunctional emotions into functional emotions and maladaptive behaviors into adaptive behaviors. REBT is considered the first approach in the cognitive behavioral therapy (CBT) tradition (Dryden, 2019, Dryden, 2021) and the emphasis on evaluative beliefs distinguishes REBT from other forms of CBT. In the ABC framework of psychological disturbance, A stands for antecedents, B for beliefs and C for consequences. Perhaps the most fundamental premise of REBT is that inflexible (rigid) beliefs (B) are at the root of psychological distress. Therefore, irrational beliefs (e.g., demandingness, awfulizing, catastrophic thinking and global evaluation) can be considered transdiagnostic factors that exist across psychiatric disorders. Through REBT, patients are educated to identify and change irrational beliefs that are associated with dysfunctional emotions (e.g., anger, anxiety) with rational beliefs (e.g., flexible beliefs based on preferences, non-awfulizing, discomfort tolerance, unconditional acceptance beliefs) that are associated with functional emotions (e.g., frustration, fear).

Evidence to support irrational beliefs as transdiagnostic factors includes that a) there is a moderate positive association between irrational beliefs and distress (e.g., general distress, anxiety, depression, anger, guilt) according to the results of a meta-analysis (Vîslă et al., 2016), b) irrational beliefs are mechanisms of change in interventions that aim to reduce emotional distress (Smith, 1983; Szentagotai et al., 2008), c) irrational beliefs appear across multiple mental health problems, such as: anxiety disorders, depressive disorders, post-traumatic stress disorder, psychosis, addictions (David et al., 2018; Rezaeisharif et al., 2021; Vîslă et al., 2016) d) are investigated as predictors or maintenance factors of psychological distress, namely for PTSD (Hyland et al., 2015), depression and anxiety in patients with diabetes (Vassou et al., 2021), paranoia (Soflau and David, 2019). Moreover, there is extensive evidence that catastrophic thinking is a transdiagnostic process across mental disorders, such as anxiety disorders (panic, phobia), obsessive-compulsive disorder, posttraumatic stress disorder, health anxiety, pain, psychosis, irritable bowel syndrome, and traumatic brain injury (Gellatly and Beck, 2016). Furthermore, REBT is one of the first interventions that focused on transdiagnostic outcomes in effectiveness studies (David et al., 2018).

Previously presented evidence regarding the role of irrational beliefs across disorders is in line with Mansell et al. (2009) criteria to define transdiagnostic approaches. Transdiagnostic processes, namely in this case irrational beliefs, must be examined in both clinical and nonclinical samples and be present in minimum four disorders to meet their criteria. Irrational beliefs in REBT are according to Mansell and collaborators criteria (2009) "mechanistically transdiagnostic constructs" as they are involved in the development and maintenance of psychopathology. We believe that REBT is appropriate for implementation as a transdiagnostic intervention as it is consistent with recent definitions of transdiagnostic therapies (Schaeuffele et al., 2021), namely, it is a treatment that simultaneously addresses several comorbidities by providing a protocol that targets shared mechanisms (e.g., irrational beliefs) between disorders.

Meta-analyses indicate that REBT is an effective therapy both for the adult population and adolescents with various mental health conditions (David et al., 2018). Given these results, so far REBT interventions have been tested in randomized controlled trials with Romanian children and adolescents either as universal prevention programs (Păsărelu and Dobrean, 2018), interventions for adolescents diagnosed with major depressive disorder (Iftene et al., 2015) or as part of multimodal treatments for children with attention-deficit/hyperactivity disorder (David et al., 2021). However, so far, no Internet-delivered intervention based on REBT theory has been developed.

The aims of the present study were to investigate the acceptability, usability, and clinical utility of a REBT Internet-delivered intervention for Romanian adolescents diagnosed with a primary anxiety or depressive disorder. Second, we investigated whether changes in proposed mechanisms, namely negative thinking (irrational beliefs) and declarative knowledge about anxiety, depression, and treatment, are associated with treatment outcome.

Method

Participants

Participants were recruited between April 2020 and September 2020 through online advertisements. Inclusion criteria for the present study were: (1) age between 12 and 17 years, (2) ability to read and write in Romanian, (3) current primary diagnosis of anxiety and/or depressive disorder based on DSM-5 criteria (Social Anxiety Disorder, Generalized Anxiety Disorder, Panic Disorder, Separation Anxiety, Unspecified Anxiety Disorder, Persistent Depressive Disorder, or Major Depressive Disorder), (4) access to Internet.

Exclusion criteria: primary diagnosis of autism spectrum disorder/psychosis/bipolar disorder, severe depressive symptoms, risk of suicide, currently following another treatment (psychotherapy sessions, pharmacological treatment).

Sixty-eight participants were interested in participating in the study. The total number of participants eligible was 15 children and adolescents (see Fig. 1), aged between 12 and 16 years old (Mage = 13.53, SD = 1.45, 66.7% girls). See Table 1 for demographic information of the sample.

Fig. 1

CONSORT diagram of feasibility study design.

Table 1

Characteristics of the sample.

Characteristics
Mean age, SD	13.53	1.45
Adolescent gender
Male	5	33.3%
Female	10	66.6%
Primary diagnosis
Social Anxiety	2	13.3%
Generalized anxiety disorder	6	40%
Unspecified anxiety disorder	3	20%
Major depressive disorder	4	26.7%
Internalizing problems YSR baseline, Mean, SD	30.80	9.49

Note. SD = standard deviation, YSR = Youth Self-Report.

Procedure

The study was approved by the Ethics Committee of Babeș-Bolyai University, Romania. Eligible participants were recruited online. The ads were posted on social media, disseminated through online invitations sent to public schools to reach the parent groups. The ads were directed at parents and contained information regarding the Internet-delivered intervention and the link to consent and screening forms.

After parents signed the informed consent and adolescents signed the informed assent, adolescents were invited to complete a screening assessment. Participants having scores above one standard deviation above the mean at Internalizing problems based on national norms on the Youth Self Report (YSR; Achenbach and Rescorla, 2001), were invited to a clinical interview. The interviews were conducted via telephone by a child psychiatrist based on a clinical interview, namely Kid-SCID (Hien et al., 1994). Eligible participants completed the baseline measures and were directed to the Internet intervention platform. They interacted with the platform for six weeks. At post-intervention assessments, adolescents completed a set of online scales measuring primary anxiety and depressive symptoms, negative patterns of thinking, satisfaction with the intervention, and rated program usability.

Measures

Demographic information

Basic information regarding participants' age and gender was collected using a demographic questionnaire.

Screening for adolescents in the clinical range for anxiety/depressive problems

The Youth Self-Report (YSR; Achenbach and Rescorla, 2001) consists of 112 items that assess behavioral and emotional problems for children and adolescents aged 11–18 in the previous six months. Answers are rated on a three-point Likert scale, (0—Not true, 1—Somewhat or sometimes true, 2—Very true or often true). Scores can be computed in two broad-band syndrome scales (Externalizing and Internalizing Problems), eight syndrome scales (Withdrawn, Somatic Complaints, Anxious/Depressed, Social Problems, Thought Problems, Attention Problems, Rule-breaking Behavior and Aggressive Behavior) and six DSM - Oriented scales (Affective problems, Anxiety problems, Somatic problems, ADHD Problems, Oppositional defiant problems, Conduct problems). We computed scores for the Internalizing scale. Alpha Cronbach for this scale was 0.92.

Clinical diagnosis of primary anxiety/and or depressive disorder

Two child psychiatrists administered by telephone a clinical interview developed for the present study to evaluate the presence of the criteria for anxiety and/or depressive disorders based on the Diagnostic and Statistical Manual of Mental Disorders—5th edition (DSM-5; American Psychiatric Association [APA], 2013). The interview was based on the Romanian version of the Structured Clinical Interview for DSM-IV Childhood Diagnoses (KID-SCID; Hien et al., 1994) with updated criteria for anxiety and depressive disorders based on DSM-5.

Anxiety symptoms

The Beck Anxiety Inventory (BAI-Y) from the Beck Youth Inventories second edition (BYI-II; Beck, 2001) was used to measure anxiety. This scale consists of 20 items rated on a 4-point Likert scale rated from 0 (Never) to 3 (Always). Alpha Cronbach for this scale was 0.74 at pre-intervention and 0.82 at post-intervention.

Depressive symptoms

The Beck Depression Inventory (BDI-Y) from BYI-II (Beck, 2001) was used to measure depression. This scale consists of 20 items rated on a 4-point Likert scale rated from 0 (Never) to 3 (Always). Alpha Cronbach for this scale was 0.93 at pre-intervention and 0.93 at post-intervention.

Irrational beliefs

The Child and Adolescent Scale of Irrationality (CASI; Bernard and Cronan, 1999) is a scale that measures irrational beliefs in children and adolescents aged between 10 and 17 years. The scale contains 28 items, rated on a 5-point Likert scale (1 - Strongly disagree, 5 - Strongly agree). The score can be computed on the total scale (Total Irrationality), as well as for subscales: Self-downing, Intolerance of frustrating rules, Intolerance of work frustration and Demands for Fairness. Alpha Cronbach for this scale was 0.88 at pre-intervention and 0.92 at post-intervention.

Negative automatic thoughts

The Automatic Thoughts Questionnaire (Hollon and Kendall, 1980) is a 15 items instrument, rated on a 5-point Likert scale (1 – Never; 5 – Almost always) was used to measure negative automatic thoughts. Alpha Cronbach for this scale was 0.94 at pre-intervention and 0.89 at post-intervention.

Program usability

In order to measure usability of the Internet-delivered intervention we adapted the System Usability Scale (SUS; Brooke, 1996). The scale contains 10 items, rated from 1 (Strongly disagree) to 5 (Strongly Agree). Scores range between 0 and 100, with higher scores indicating a higher usability of the platform. Alpha Cronbach for this scale in the present study was acceptable, namely 0.77.

User-friendliness

We asked adolescents “How friendly did you find the platform?” in order to capture their perception on the friendliness of the Internet-delivered intervention. Answers were rated from 1 (Very unfriendly) to 10 (Extremely friendly).

Satisfaction with the program

Similar to other studies (Watts et al., 2013), we measured satisfaction with the program using several questions. First, we asked participants “How satisfied are you with this program?”. Responses are rated on a nine-point Likert scale, ranging from 1 (Not at all satisfied) to 9 (Very satisfied). The second question was “Would you recommend this program to a friend?” with answers rated on a nine-point Likert scale ranging from 1 (Definitely not) to 9 (Certainly yes). Third, we asked “Did you like this program?” and fourth “Did this program helped you to cope with your problems?” with answers rated on a nine-point Likert scale ranging from 1 (Not at all) to 9 (Very much). Alpha Cronbach for this scale in the present study was excellent, namely 0.91.

Knowledge test

We developed a knowledge test based on previous instruments developed to measure knowledge acquisition in Internet-delivered interventions (Andersson et al., 2012; Berg et al., 2020). The test contains 18 items, all formulated as statements with a True/False response choice. In addition, certainty ratings were added for each item, namely, participants were asked about their level of certainty with each response (“Guessing,” “Pretty certain,” or “Totally certain”). This scale was constructed to measure knowledge regarding functional and dysfunctional emotions, rational and irrational beliefs, adaptive vs maladaptive behaviors in accordance with the REBT theory. Scores can be calculated in two ways: first, correct answers are summed; second, scores are weighted in line with the degree of certainty of the respondent (Andersson et al., 2012). Alpha Cronbach for this scale was 0.82 at pre-intervention and 0.89 at post-intervention.

Qualitative feedback

We added two open ended questions: “What did you most like about this Internet intervention?” and “What did you not like about this Internet intervention?” for obtaining the qualitative feedback from adolescents.

Intervention

The transdiagnostic Internet-delivered intervention consisted of nine modules delivered over six weeks. The content of the nine modules was developed based on the Rational emotive behavior therapy (REBT; Ellis, 1994), and previous research conducted with children and adolescents (Păsărelu and Dobrean, 2018). REBT focuses on a specific type of cognition, namely rational/ irrational beliefs, which are proximal for functional/ dysfunctional emotions and adaptive/ maladaptive behaviors. The REBT framework is particularly suitable for transdiagnostic approaches in the treatment of mental health problems, further evidence supports the idea that catastrophic thinking is a transdiagnostic process across psychiatric disorders (Gellatly and Beck, 2016). Empirical data summarized in a meta-analysis indicated that irrational beliefs are significantly and positively related to general psychological distress (Vîslă et al., 2016). Our Internet-delivered REBT program incorporates four core principles including distinguishing between functional and dysfunctional emotions, identifying irrational beliefs (absolutistic demands, awfulizing, low frustration tolerance, global evaluations of self/others/life) that lead to dysfunctional emotions, replacing irrational beliefs with rational beliefs (e.g., flexible attitudes, discomfort tolerance, non-awfulizing, unconditional acceptance of self/others/life) and reducing maladaptive behaviors. Evidence-based techniques used in this program include psychoeducation, cognitive restructuring, relaxation, behavioral activation, relapse prevention, all strategies that cut across disorder-specific protocols for adolescents with anxiety and depressive disorders, similar to other transdiagnostic interventions (Ehrenreich-May et al., 2018; Sandín et al., 2020). The distinctive feature of our REBT program is related to the type of beliefs disputed, namely irrational beliefs, which are rigid and extreme beliefs, and represent evaluative beliefs not inferential/descriptive beliefs as in Cognitive Therapy (David et al., 2010). According to the ABC framework of distress, the core principle of REBT is that unhealthy emotions are the results of irrational thinking about an event and not the results of the event itself. Therefore, if the person holds irrational/rigid beliefs towards the activating event, this leads to unhealthy emotions (called dysfunctional emotions), whereas the person holds rational beliefs towards the same activating event, this leads to healthy (functional) emotions. Therefore, through our program, adolescents learn to identify irrational beliefs (beliefs that do not have a logical, pragmatic and/or empirical support), dispute them, and practice new, rational beliefs. Of particular importance, adolescents learn to identify the four types of irrational beliefs: demandingness (“I must be good in everything I do”), low frustration tolerance (“I can't stand doing this!”), catastrophic thinking (“It's the worst thing that happened to me!”) and global evaluation (“I am bad”/“Other adolescents are bad”/“Life is bad”). Modules were delivered in the following sequence: first three modules in the first week of treatment, Module 4 in the second week, Module 5 in the third week, Module 6 in the fourth week, Module 7 in the fifth week, and finally, Modules 8 and 9 in the sixth week. See Fig. 2 for a screenshot of the modules from the Internet-delivered intervention and Table 2 for the content of each module.

Fig. 2

Screenshot of the structure of the nine modules from the transdiagnostic Internet-delivered intervention.

Table 2

Session contents.

Module name	Child content	Key focus
Module 1. Introduction	Psychoeducation regarding the program	Introduction to programReview information regarding the program, therapeutic orientationIdentify program goals
Module 2. Emotions	Difference between functional and dysfunctional emotions	Discuss the difference between different types of emotions
Module 3. Anxiety and depression	Psychoeducation regarding anxiety and depressive disorders	Discuss the difference between fear and anxiety, how anxiety and depression can be recognized from physical sensations, behaviors, thinking patternsIntroduce mood record
Module 4. Relaxation	Respiration and relaxation exercises	Learn breathing and relaxation exercisesCreate a relaxation plan
Module 5. Thinking patterns	Identifying, disputing and changing irrational beliefs	Introduce the concept of irrational thinkingDiscuss modalities to dispute irrational thinking and develop rational thinking
Module 6. Learn problem solving	Learn to solve practical problems	Discuss the distinction between emotional and practical problemsLearn a problem solving approach
Module 7. Gain control over your behavior	Exposure and behavioral activation	Discuss implementation of home exposure and behavioral activation
Module 8. Positive emotions	Practice positive emotions	Discuss importance of positive emotionsLearn strategies to increase positive emotions (gratitude exercises)
Module 9. Keep your emotions under control	Gaining maintanance	Review and discuss program contents and skills achieved through this programCreate a first aid mental health kitPlan for future stressful situationsCertificate of attendance

Each of the modules consisted of a structured format, with the next components of each session: module objectives, main content of this module, summarizing what was learned in that module, homework, and reward (a downloadable poster with contents achieved in each module). Each module has both written and audio or video contents.

In the first module (Introduction), adolescents learn about the program (number of sessions, therapeutic orientation). Confidentiality is discussed and expectations regarding adolescent's active involvement in the program are set. The rationale for the homework is presented.

In the second module (Emotions) the active ingredient is psychoeducation regarding emotions. Adolescents learn to distinguish between various emotions, the main components of an emotion (subjective experiences, physiological responses, cognitive processes, behavioral responses), and the distiction between functional (i.e., emotions such as: concern, sadness, annoyed, that help you achieve your goals and are associated with rational beliefs) and dysfunctional emotions (i.e., emotions such as anxiety, depression, anger, guilt, that obstruct one's progress in achieving goals and are associated with irrational beliefs).

In the third module (Anxiety and depression) the active ingredient is psychoeducation regarding anxiety and depression. Namely, adolescents learn the distinguish between fear and anxiety, between sadness and depression. They identify body signs, thinking patterns and behavioral responses specific to anxiety and depression.

In the fourth module (Relaxation) adolescents learn the relationship between physiological activation and anxiety. They learn breathing exercises and progressive muscle relaxation and are encouraged to create a relaxation plan.

In the fifth module (Thinking patterns) the active ingredient is cognitive restructuring. The transdiagnostic process in this module is represented by irrational beliefs, which according to REBT theory are evaluative beliefs that are associated with dysfunctional emotions. Namely, adolescents learn to identify irrational beliefs, dispute them and change them with rational beliefs. The ABC framework is presented with several examples, then for each irrational belief they practice alternative, rational beliefs as follows:

Instead of rigid thinking/absolutistic demands (“I must succeed in everything”, “Everyone must love me”) adolescents learn flexible, preferential thinking (“I would prefer to succeed in achieving my goals, however, I can accept that sometimes this does not happen”, “I would prefer that people love me but I can accept that some colleagues just don't like me”).

Instead of catastrophic/awfulizing beliefs (“It's awful to make mistakes during a presentation”) adolescents learn non-catastrophic thinking (“I don't like to make mistakes during an oral presentation, nevertheless, more terrible things might happen”)

Instead of low frustration tolerance (“I can't stand to receive a small grade at school”) they learn to increase frustration tolerance (“I don't like to receive small grades, but I can stand this”)

Instead of global evaluation of self (“I am stupid because I failed a test”), others (“My colleagues are bad because they laugh in classroom”) or life (“Life is bad”) adolescents learn unconditional acceptance of self/others/life (“I am a valuable human being who can have non-desired behaviours”).

In the sixth module (Learn problem solving) the active ingredient is problem solving skills. A distinction is made between emotional problems and practical problems based on the ABC framework. At this point, adolescents learn problem solving in order to change the Activating event (the A component of the ABC model) and, consequently, prevent the problem from arising in the first place. Common mistakes in problem solving are presented. Adolescents learn a 10-step protocol to solve practical problems.

In the seventh module (Gain control over your behavior) the active ingredients are behavioral exposure and behavioral activation. Namely, after presenting the rationale for these two techniques, adolescents are encouraged to create an exposure plan and a daily activities plan.

In the eight module (Practice positive emotions) the active ingredient is positive emotions. Adolescents learn in this module that reducing negative emotions does not entail increasing positive emotions. Therefore, active efforts need to be made to increase positive emotions. Several strategies and exercises from positive psychology (reward plan, three good things) are used in this module.

In the ninth module (Keep your emotions under control) the active component is relapse prevention. A summary of the program and its main content in each module is presented. Adolescents are rewarded for their gains, receive a diploma for program completion, and are supported to identify future situations where they could feel anxiety or depression. Furthermore, for identified situations, adolescents examine the strategies learned that can be used to manage their dysfunctional emotions that could appear.

Exercises for adolescents are available in the content of the module, as well as for homework assignments (see Fig. 3).

Fig. 3

Screenshot of an exercise from Module 4 from the transdiagnostic Internet-delivered intervention.

The intervention was guided by two psychotherapists, licensed practitioners under supervision training in Cognitive Behavioral Therapy, trained by the first author on the treatment protocol. They provided feedback on the worksheets from modules and homework assignments as well as support and answered adolescents' questions. Therefore, the level of involvement was minimal and they acted as coaches providing feedback on in-module activities and homework that served as a motivation boost for the activity of adolescents on the platform. Each adolescent was supported by a single therapist, all conversations between adolescent and therapist were asynchronous through the secured platform. There were no phone contacts were between therapists and the adolescents. The criteria to deliver a new module was temporal, each week participants received access to new modules. The intervention was only for adolescents, and parents were not contacted regarding treatment contents.

Adolescents were encouraged to work through modules at their own pace, having seven days before new content is available. Therefore, on each treatment week adolescents could log-in either one time when new modules were sent, or on multiple times during that week.

Statistical analysis

Data were analyzed using SPSS version 26 (IBM Corp). First, the normality of the distribution was checked with the Shapiro-Wilk test. The results indicated that the data was normally distributed for all the variables of interest with adequate levels of skewness and kurtosis.

Paired sample t-tests were conducted to investigate changes between pre-intervention and post-intervention assessments. Change scores were computed for primary and secondary outcomes. Pearson correlation coefficients were computed for the association between proposed mechanisms of action (changes in irrational beliefs, knowledge about treatment) and changes in primary outcomes (anxiety and depressive symptoms scores) (Table 3).

Table 3

Clinical outcomes.

Outcome	Baseline M (SD)	Post-intervention M (SD)	Cohen's d effect size with 95% CI
Anxiety	28.66 (7.61)	15.21 (7.79)	1.85 [0.96, 2.72]
Depression	25.06 (12.58)	9.78 (8.75)	1.31 [0.57, 2.02]
Irrational beliefs	85.80 (18.42)	67.50 (21.76)	0.69 [0.09, 1.26]
Negative automatic thoughts	42.06 (15.41)	23.92 (9.32)	1.44 [0.67, 2.19]
Knowledge raw	12.20 (1.37)	13.64 (1.44)	−1.80 [−1.39, −0.18]
Knowledge weighted	5.50 (3.05)	9.21 (2.86)	−0.97 [−1.63, −0.29]

Note. M = Mean score; SD = Standard deviation; CI = Confidence interval.

Results

Usability and acceptability

Participants completed a mean number of 6.26 (SD 2.52) modules of the nine modules available. The number of modules read by participants ranged between 3 and 9, with 42.9% children completing all nine modules, and 21.4% completing five modules. Participants' log-ins also ranged between 2 and 39 log-ins, with a mean of 10.66 (SD 9.52).

On the Usability scale, the mean was 82.50 (SD = 11.13).

Participants answered the question “How friendly did you find the platform?” and responses were between 8 and 10. Participants perceived the transdiagnostic Internet-delivered intervention user-friendly, M = 9.21, SD = 0.80.

For the question “How satisfied are you with this program?”, the mean scores were high, namely M = 7.92 = 3, SD = 1.07. For the second question, “Would you recommend this program to a friend?” participants mean scores were also high, M = 7.93, SD = 1.20. When asked “Did you like this program?” participants' mean scores were high, M = 8.14, SD = 0.94. For the question “Did this program helped you to cope with your problems?”, participants perceived the program helpful, M = 7.71, SD = 0.99.

Clinical utility

Primary anxiety/depressive symptoms

Our results indicated significant changes from pre-intervention to post-intervention for our primary outcomes, namely anxiety symptoms t(14) = 6.95, p < .001 and depressive symptoms t(14) = 4.92, p = .002.

Secondary outcomes

Negative patterns of thinking

Irrational beliefs decreased from pre-intervention to post-intervention t(14) = 2.59, p = .022. Negative automatic thoughts also decreased significantly from pre-intervention to postintervention t(14) = 5.41, p < .001.

Knowledge test

On the knowledge test, there was a significant improvement in scores from pre-intervention to post-intervention both for raw scores, t(14) = −2.99, p = .010, as well as for weighted scores, t(14) = −3.52, p = .004.

Change in irrational beliefs and anxious/depressive symptoms

Our results indicated a non-significant correlation between changes in total irrational beliefs between pre-intervention to post-intervention and change in anxiety (r = 0.02, p = .943) or depressive symptoms (r = 0.16, p = .584).

Change in knowledge and anxious/depressive symptoms

The results didn't indicate a significant correlation between changes in knowledge between pre-intervention to post-intervention and change in anxiety (r = 0.14, p = .645) or depressive symptoms (r = 0.35, p = .232).

Discussion

Anxiety and depressive disorders have a major impact on the lives of children and adolescents. Despite the fact that evidence-based treatment has been developed, numerous barriers in terms of access to such treatment exist, such as limited knowledge about treatment, stigma, financial costs, logistical barriers (Radez et al., 2020). In order to increase dissemination of evidence-based interventions, there has been a rapid proliferation of new modalities to deliver such interventions (e.g., online, mobile, web-based, mobile apps; Păsărelu et al., 2020). Therefore, interventions delivered over the Internet have the potential to overcome such barriers.

To our knowledge, this is the first Internet-delivered intervention developed based on the REBT approach in a clinical sample of children and adolescents diagnosed with anxiety and/or depressive disorders. Our results indicated that the intervention had a significant effect on reducing anxiety and depression in adolescents, and it was presented in a friendly manner. Satisfaction with the intervention was also high and the program was rated as user-friendly and helpful. The usability score for the intervention was in the good to excellent range according to existing guidelines to interpret usability scores (Brooke, 1996). Our primary outcomes of anxiety and depressive symptoms were significantly reduced, which is consistent with the literature on the efficacy of Internet transdiagnostic interventions for adults (Păsărelu et al., 2017). Within groups effect sizes were in the moderate to large range for our primary outcomes, namely anxiety and depressive symptoms. Negative patterns of thinking (negative automatic thoughts and irrational beliefs) also decreased significantly from pre-treatment to post-treatment. Also, moderate to large effect sizes were found for negative automatic thoughts, while for irrational beliefs and treatment knowledge within-group effect sizes were in the small to moderate range.

We did not find significant associations between change scores in proposed mechanisms, namely change scores in negative patterns of thinking (irrational beliefs), knowledge acquisition, and change scores in primary outcomes (anxiety and depressive symptoms). This could be explained by the fact that our study was underpowered given the small sample size included for this feasibility trial. However, findings are in line with previous findings by Berg et al. (2020) where knowledge changes have not been related to changes in depressive symptoms in a sample of adolescents suffering from depression.

The major limitations of the present study are the lack of a comparison group, as well as the small sample size. However, as the main objective of the present study was to determine the feasibility of a new developed transdiagnostic Internet-delivered intervention for children and adolescents with a primary diagnosis of anxiety or depressive disorder, we obtained preliminary results that will guide us when conducting a larger clinical trial using this intervention. Future research should involve larger samples of adolescents, compare with a control group, and investigate both short- and long-term efficacy, as well as mechanisms of change. Another limitation of the present study was the lack of blind assessments conducted at post-intervention. Clinician ratings of symptom improvement and functioning would be of particular importance. Another important limitation regarding assessments used in this study is regarding the measurement of anxiety symptoms with Beck Anxiety Inventory from Beck Youth Inventories (Beck, 2001), which better captures physiological symptoms of anxiety and panic symptoms. The use of a different assessment tool that can identify changes in social anxiety, generalized anxiety or other types of anxiety symptoms would be particularly useful, as well as the use of online transdiagnostic assessment instruments that capture both the global severity and impairment associated with anxiety symptoms (González-Robles et al., 2018). Also, future research regarding irrational beliefs assessment need to be conducted in order to establish instruments sensitive to treatment changes (David et al., 2019).

The development of an Internet-delivered intervention is an innovative approach to deliver evidence-based treatments for adolescent populations presenting multiple comorbidities. Through this study, our aim was to overcome existing problems in the implementation of adult interventions for adolescents, with the aim of developing psychological content that is age appropriate for adolescents. Future research should involve larger samples of participants, randomly assigned to different experimental conditions (e.g., with or without parental involvement; Păsărelu and Dobrean, 2016), and include measures of clinical improvement collected at both short- and long-term follow-up.

In conclusion, in this study we tested the feasibility and clinical utility of a novel REBT Internet-delivered intervention developed for adolescents with a primary diagnosis of anxiety or depressive disorder. Significant improvements from pre-intervention to post-intervention were found for anxiety and depressive symptoms, as well as negative patterns of thinking. The development of a new transdiagnostic Internet intervention requires iterative testing initiatives and adaptation in order lead to a final evidence-based intervention with an intuitive design. Users' feedback and results from this open trial study will be incorporated and the intervention will be tested in a randomized clinical trial.

Funding

Part of this work was supported through the project: Entrepreneurship for innovation through doctoral and postdoctoral research, POCU/380/6/13/123886 co-financed by the European Social Fund, through the Operational Program for Human Capital 2014- 2020 awarded to Dr. Costina-Ruxandra Păsărelu and a grant awarded to Dr. Anca Dobrean from the Romanian Ministry of Education and Research, CNCS – UEFISCDI [grant number PN-III-P4-ID-PCE-2016-0861].

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.