Assessing the efficacy and acceptability of an internet-delivered intervention for resilience among college students: A pilot randomised control trial protocol.

BACKGROUND: Exposure to new stressors places college students at increased risk for developing mental health problems. Preventive interventions aimed at enhancing resilience have the potential to improve mental health and well-being in college students and internet-delivery may improve access to these interventions. However, few studies have evaluated the efficacy of online interventions for resilience in college students. The present study seeks to assess the feasibility [initial efficacy and acceptability] of a newly developed internet-delivered intervention for resilience provided with human or automated support, in a sample of college students.
METHOD: A pilot randomised controlled trial including three groups: 1) an intervention group with human support; 2) an intervention group with automated support; and 3) a waiting list control group. The intervention, Space for Resilience, is based on positive psychology and consists of seven modules, delivered over a period of eight weeks. Primary outcomes measures will include the Connor-Davidson Resilience Scale (CD-RISC) and the Pemberton Happiness Index (PHI). Secondary outcomes measures will include the Brief Resilience Scale (BRS), the Patient Health Questionnaire - 4 items (PHQ-4), the Rosenberg Self-Esteem Scale (RSES), and the Perceived Stress Scale - 4 items (PSS-4). Acceptability will be examined using the Satisfaction with Treatment (SAT) questionnaire. Analysis will be conducted on an intention-to-treat basis. DISCUSSION: The study seeks to establish the initial efficacy and acceptability of an internet-delivered intervention for resilience with human support and automated support. Apart from determining the impact of the intervention on acceptability and effectiveness, this study will be a first to explore more clearly the relative benefits of different support modes.

Background

During the transition to adulthood, college students experience numerous psychosocial changes that play an important role in determining future developmental outcomes (Bayram and Bilgel, 2008). They face specific social and academic pressures which place them at increased risk for developing mental health problems (Bayram and Bilgel, 2008). Mental health problems are therefore particularly pervasive in this cohort with a 12-month prevalence rate of approximately 20% for mental disorders including post-traumatic stress disorder (PTSD) and anxiety and depressive disorders (Auerbach et al., 2016). These problems have long-term negative effects at an individual and societal level such as poor academic achievement, college attrition and future functional impairment (Harrer et al., 2018).

Despite the existence of adequate treatment, the most recent World Mental Health survey reported that only 6.7–23.1% of college students received treatment for their mental health disorder (Auerbach et al., 2016). One of the reasons for low treatment rates is related to low levels of help-seeking behaviours among college students, who report beliefs that stress is normal in university, not seeing their needs as serious and not having time for treatment as barriers to seeking treatment (Downs and Eisenberg, 2012; Regehr et al., 2013). Given the prevalence of mental disorders in college students and poor help-seeking behaviours, it is important to develop effective strategies that prevent against the development of these disorders and related patterns of poor help-seeking once they have occurred. A preventive approach that focuses on promoting well-being, rather than a problem-focused approach, may be particularly attractive to college students given perceptions that mental health problems are normal or not serious (Bolier and Abello, 2014). In this vein, the European Commission health policy of the World Health Organisation (WHO, 2013) also proposed the adoption of prevention-oriented care in order to promote mental well-being and resilience in individuals.

Interventions that aim to promote resilience and well-being may therefore be especially relevant for college students. Such interventions include any forward-looking programme that seeks to enhance individual, group or population resilience in an attempt to prepare users for future adversity (Leppin et al., 2014). Intervention approaches may include positive psychology, cognitive behavioural therapy, acceptance and commitment therapy and mindfulness (Macedo et al., 2014). Regardless of the modality, theoretical basis or methods used however, resilience interventions often seek to enhance protective factors (e.g., Burton et al., 2009; Waite and Richardson, 2004; Steinhardt and Dolbier, 2008). Research has generally demonstrated modest improvements in resilience following training (Joyce et al., 2018; Leppin et al., 2014), where even small preventive effects can produce substantial benefits at a community or population level (Sorensen et al., 1998; as cited in Vanhove et al., 2016).

Despite the growing popularity of resilience training, the majority of these programmes are delivered face-to-face, raising significant issues surrounding accessibility and engagement (Joyce et al., 2018). These include cost, availability of services, waiting lists, transportation and stigma (Herrero et al., 2018). Online or internet-delivered interventions represent a cost-effective tool for combating barriers to treatment while facilitating a person-centred environment where individuals can actively contribute to their own well-being (Richards et al., 2016). However, notwithstanding potential benefits, a recent systematic review noted a lack of studies exploring the effects of online resilience training (Joyce et al., 2018). Of the studies that have been conducted however, findings provide initial support for the efficacy of internet-delivered resilience interventions (e.g., Abbott et al., 2009; Masselink, 2013; Rose et al., 2013). This finding poses the need for conducting pilot and feasibility trials that inform the design of further confirmatory studies (Moore et al., 2011).

Meta-analytic evidence has shown that supported internet-delivered interventions do better, potentially reducing attrition rates by 30–40% (Richards and Richardson, 2012) and is associated with larger effect sizes than unsupported interventions (Wright et al., 2019). However, the cost associated with the provision of human support reduces both the scalability and viability of online interventions (Schueller et al., 2016). Moreover, research investigating different types of human supporters demonstrates no difference in overall effects (e.g., Titov et al., 2010). Rather, it is the availability of some degree of user support that seems to be important. In this sense, the use of automated support (i.e., automated reminders or emails) may represent a more cost- and time-efficient alternative to human support and different studies have shown positive results (Baumeister et al., 2014; Campos et al., 2019; Mira et al., 2017; Titov et al., 2013), but more research is needed to support these findings.

Therefore, the main goal of this study is to explore the feasibility [initial efficacy and acceptability] of an internet-delivered intervention for promoting resilience and well-being in college studies, compared to a waiting list control group. A secondary objective of the present study includes exploring the differential effects between human versus automated types of support.

Method

Study design

The present study is a three-arm, parallel group, exploratory pilot randomised controlled trial with an allocation ratio of 1:1:1. Participants will be randomised to one of three groups: 1) intervention with human support; 2) intervention with automated support; and 3) waiting list control group. Participants in the active intervention groups (intervention with human or automated support) will complete the intervention over a period of eight weeks and participants in the waiting list group will have access to the intervention once the eight-week waiting period is over. Fig. 1 depicts the CONSORT flow diagram for the trial.

Fig. 1

CONSORT flow diagram.

Sample size

Given the lack of research in this area, previous data on effect sizes for resilience interventions in college samples are not available (Herrero et al., 2018). However, taking a conservative approach, a small effect size for well-being outcomes (d = 0.2) is expected based on a meta-analysis of RCTs on positive psychology interventions by Bolier et al. (2013). Taking an equally conservative approach, at the minimum, a similar effect size for resilience outcomes is anticipated (Herrero et al., 2018; Joyce et al., 2018). Therefore, given a small expected effect size of 0.2 for resilience and well-being outcomes and recent guidelines for estimating sample size for pilot RCTs designed with 90% power and two-sided 5% significance, based on a non-central t-distribution approach, a sample size of 75 was determined (25 per arm; Whitehead et al., 2016).

Eligibility criteria

Eligibility criteria are outlined in Table 1.

Table 1

Eligibility criteria.

Inclusion criteria	Exclusion criteria
Above 18 years of age	Individuals with psychotic or bipolar disorder
Registered student at the university	Individuals at risk of suicide
	Individuals currently in psychological treatment

Recruitment, randomisation and blinding

An email advertising the study will be sent to all registered undergraduate and postgraduate students at Trinity College Dublin (TCD), Ireland, by the university's student counselling service. The study will also be advertised through posters placed around the university campus and by the university's undergraduate and graduate students' union in their weekly email newsletter and on their social media (Facebook page). Students interested in taking part in the study will be able to visit an online platform via a web address where they will create an account using their full name, e-mail address and mobile phone number. Participants will receive information about the study and consent will be obtained on the platform. Eligible participants will be randomly assigned to one of three groups: human support intervention group, automated support intervention group, and wait-list control group. The randomised allocation schedule will be generated through computer algorithms administered by an independent researcher who will be unaware of the characteristics of the study. The randomisation process will be performed in blocks of 12 with 3 groups using the Random Allocation Software. Participants will have agreed to participate before the randomised allocation without knowing which group they will be assigned to. However, for practical reasons, participants and researchers will not be blind to the allocation.

Intervention

The intervention programme is called Space for Resilience and has been developed by SilverCloud Health, a provider of online healthcare programmes. It is a seven-module online intervention that aims to promote well-being and resilience delivered on a Web 2.0 platform using media-rich interactive content. The intervention is based primarily on positive psychology (Seligman, 2004) and comprises cognitive components previously incorporated in other resilience interventions (Chmitorz et al., 2018), including cognitive flexibility, optimism, challenging negative self-talk (Lomas and Ivtzan, 2016), behavioural activation (Ekers et al., 2014) and active coping (Lee et al., 2013), as well as information on social support (Prati and Pietrantoni, 2009), lifestyle factors (Steptoe et al., 2015) and values (Ho et al., 2010).

Each module follows a structured format that incorporates videos, informational content, interactive activities, mindfulness meditations, homework suggestions and summaries. In addition, personal stories and accounts from other users are incorporated into the presentation of the material. The content of each module is described briefly in Table 2.

Table 2

Space for Resilience: Description of module content.

Module name	Brief description
Building resilience	This opening module introduces the concept of resilience (Helmreich et al., 2017) and allows the user to analyse their current levels of resilience and set goals for the programme. The practice of mindfulness (Davis and Hayes, 2011) and its relevance for resilience is also introduced.
Purpose	This module focuses on purpose, meaning and values (Ho et al., 2010; Seligman, 2004). The user is encouraged to identify their values, what matters most to them in life and their passions, and find ways to incorporate these into the key life roles they undertake.
Self	This module focuses on self-esteem and self-compassion. The user is encouraged to identify their strengths and align them to their values and passions (Seligman, 2004). Users are also invited to challenge their negative self-talk and replace it with more compassionate statements (Zessin et al., 2015).
Connections	This module supports users in reflecting on their social networks and improving their relationships and communities. Information about communication styles is provided and the user is given tips for improving their communication skills (Afifi et al., 2016; Prati and Pietrantoni, 2009).
Body	This module focuses on creating a healthy lifestyle (Steptoe et al., 2015) by developing positive habits for sleep, diet and exercise. Behavioural activation techniques (Ekers et al., 2014) are provided and the user can track their daily lifestyle choices and observe how they impact on their mood.
Mind	This module focuses on thoughts and offers balanced optimism (Ho et al., 2010; Prati and Pietrantoni, 2009) and gratitude (Wood et al., 2010) as alternatives to negative or distorted thinking (Lomas and Ivtzan, 2016).
Moving forward	This final module looks at active coping methods (Lee et al., 2013) for dealing with problems, and prepares the user for coming to the end of the programme. Users have the opportunity to review their progress since starting the programme and set goals for the future.

Support

Human

Supporters assigned to participants in the intervention with human support group will be counsellors or trainee counselling staff working at the university's student counselling service who already widely use the SilverCloud depression and anxiety interventions. They will be able to access information on participant levels of engagement with the programme through the online platform. Supporters will use this information to provide individualised reviews to participants. A review is when the supporter has seen the work and progress of the participant as they use the programme and thereafter formulates a reply that acknowledges and affirms the progress that has been made, encourages the user to continue using the programme and at times may actively direct the participant to content in any of the modules available in the programme. A review can also provide answers to any questions that the user may have and help resolve any difficulties experienced with the platform. Throughout, the supporter demonstrates empathy and care for the user's journey. Reviews are designed to support participant progress with the programme and will be sent via messages on the online platform. Participants in the intervention with human support group will receive four reviews that will be offered fortnightly during the eight-week intervention period. The decision to have four reviews is based on the delivery of a preventative intervention and considering that participants have sufficient self-efficacy to self-administer the programme with minimal support provided.

Automated

Support for participants in the intervention with automated support group will consist of generic, precast reviews. Reviews are also designed to support participant progress with the programme and will be sent via messages on the online platform. However, reviews will not be individualised based on each participant's level of engagement with the programme. Messages were pre-written by clinicians with many years of clinical experience and knowledge of delivering online support. All the messages follow a similar structure, where participants are prompted about their level of engagement, they are encouraged to continue using the intervention and specific bits of content are recommended. Participants in the intervention with automated support group will receive four reviews during the eight-week intervention period. Reviews are scheduled to be sent at the moment of sign-up, week 2, week 4 and week 7.

Measures

All data will be collected online for screening purposes and also online at follow-up. If deemed necessary, data collection via the online platform will be complemented with phone call and email reminders for participants to increase data retention.

Screening measure

The Sociodemographic Information and Clinical History Questionnaire (based on a previous version by Richards et al., 2013): This measure collects sociodemographic and clinical information about participants including experience of counselling or psychotherapy, drug and alcohol use, previous diagnosis of an organic or serious mental health disorder including schizophrenia, psychosis or bipolar disorder, and risk of suicide. This measure also includes a question on the participant's preference for which intervention group they would be allocated to (intervention with human or automated support). However, participant responses will not influence allocation. The Sociodemographic Information and Clinical History Questionnaire will be administered at baseline only.

Primary outcome measures

The Connor-Davidson Resilience Scale (CD-RISC; Connor and Davidson, 2003): The CD-RISC is a 25-item self-report questionnaire that assesses the ability to cope with stress. Respondents are asked to indicate their response using a 5-point Likert scale from 0 to 4 (0 = not true at all, 4 = true nearly all of the time). Scores range from 0 to 100, with higher scores reflecting greater resilience. Previous studies have shown that the CD-RISC has convincing evidence for its concurrent validity and good internal consistency (Cronbach's alpha above 0.70; Yu and Zhang, 2007; Singh and Yu, 2010). The CD-RISC will be administered at baseline and post-intervention.

The Pemberton Happiness Index (PHI; Hervás and Vázquez, 2013): The PHI is a 21-item integrative, self-report measure of well-being. The scale includes 10 items related to different domains of remembered well-being (i.e., general, hedonic, eudaimonic and social well-being) and 11 items related to experienced well-being (i.e., positive and negative emotional events that possibly happened the day before). For remembered well-being, participant agreement with statements is rated using an 11-point Likert scale (0 = fully disagree, 10 = fully agree). For experienced well-being, participants indicate whether or not they experienced specific events the day before with a dichotomous response (yes/no). Hervás and Vázquez (2013) found that the PHI had good internal consistency (Cronbach's alpha above 0.82). The PHI will be administered at baseline and post-intervention.

Secondary outcome measures

The Brief Resilience Scale (BRS; Smith et al., 2008): The BRS is a 6-item self-report measure assessing the ability to bounce back or recover from stress. The scales consist of six items. Participant agreement with statements is rated using a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). The BRS has shown strong convergent validity and good internal consistency with college students (Cronbach's alpha above 0.80; Smith et al., 2008). The BRS will be administered at baseline and post-intervention.

The Patient Health Questionnaire – 4 items (PHQ-4; Kroenke et al., 2009). The PHQ-4 is a brief 4-item self-report inventory of depression and anxiety. Participant responses are rated on a 4-point Likert scale (0 = not at all, 3 = everyday). This measure combines the items from the Patient Health Questionnaire – 2 items (PHQ-2; Kroenke et al., 2003) and the Generalised Anxiety Disorder – 2 item scale (GAD-2; Kroenke et al., 2007). Internal reliability is good for all scales (Cronbach's alpha above 0.80). A principal components analysis supports the factorial validity of the PHQ-4 with the 2 items related to depression in the PHQ-2 and the 2 items related to anxiety in the GAD-2 indicating that 84% of the total variance could be explained by these 2 factors, depression and anxiety (Kroenke et al., 2009). The PHQ-4 will be administered at baseline and post-intervention.

The Rosenberg Self-Esteem Scale (RSES; Rosenberg, 1965): The RSES is a 10-item self-report measure of global self-esteem. Participant agreement with positive and negative feelings about the self is rated using a 4-point Likert scale (0 = strongly disagree, 3 = strongly agree). Previous studies have found good internal consistency and test–retest reliability for this measure (Gray-Little et al., 1997; Robins et al., 2001). The RSES was administered at baseline and post-intervention.

The Perceived Stress Scale - 4 items (PSS-4; Cohen et al., 1983): The PSS-4 is a 4-item self-report questionnaire that assesses the degree to which recent life situations are appraised as stressful (Cohen et al., 1983). Degree to which participants have experienced specific thoughts and feelings over the preceding month is rated using a 5-point Likert scale (0 = never, 4 = very often). The PSS-4 shows good criterion validity and internal consistency (Cronbach's alpha above 0.70; Cohen et al., 1983). The PSS-4 will be administered at baseline and post-intervention.

Others measures

The Satisfaction with Treatment (SAT; Richards and Timulak, 2013): The SAT is a self-report measure of participants' positive and negative experiences with the internet-delivered intervention. Participant responses to statements are rated using a 5-point Likert scale (1 = disagree very strongly/not at all helpful, 5 = agree very strongly/very helpful). This measure also contains two open questions asking participants to describe what they liked most and least liked about the intervention. The SAT will be administered at post-intervention only to participants in the active intervention groups (intervention with human or automated support).

Engagement and usage measures

The online system will automatically record information about the programme usage. All user activity within the programme (i.e. clicking through the content, updating an activity, saving a journal entry) is recorded with a time stamp. Thus, a session is defined as any time that the user logs on the platform and the length of the session is determined by subtracting the time of the last time stamp of that session, to the time of the login. Total time spent in the programme is therefore calculated by adding the total time that the user spent in each session. The system also tracks the number of activities completed, the percentage of programme viewed and the number of reviews offered by the supporter (in the human support condition).

Ethical considerations

The study protocol, information on the study, informed consent and all related materials have been submitted to and subsequently approved by the research ethics committee at the School of Psychology, Trinity College Dublin (Approval ID: SPREC112018-12; 27th November 2018). Important protocol modifications, if any, will be communicated to relevant parties (i.e., trial participants, trial registries, journals, ethical committee, and researchers). Researchers of this current study will obtain informed consent from each participant before randomisation through the internet platform where participants first sign up to participate in the trial. Each participant will be given a unique username and password only known to themselves to log on and begin the intervention. Prior to giving informed consent participants will be made fully aware of what is involved in participation and in particular the importance of the wait-list control group. All participants will be informed they are free to withdraw at any stage if they no longer wish to take part in the trial. All participant data will be managed securely and confidentially; all relevant EU and Irish legislation on privacy will be observed and adhered to. All transferred data will be secured following the AES (Advanced Encryption/Standard) polynomial m(x) = ×8 + ×4 + ×3 + × + 1. A limited number of members on the research team will only have access to the final trial dataset. Should participants become distressed at any point during the study, they will be reminded that they will not have to continue participating and will be provided with a list of relevant support services.

Planned analysis

All analyses will be based on the intention-to-treat principle. All of the data will be prepared and analysed in SPSS 24.0. Analyses will be conducted following the CONSORT guidelines for reporting the results. Descriptive statistics using Chi-squared and t-tests will be used to examine demographic differences and clinical characteristics in the groups (Field, 2009). t-Tests will be conducted to explore differences in usage metrics and engagement between human and automated conditions.

Linear Mixed Model analyses will be executed to analyse the change on the reported measures from pre- to post-intervention (CD-RISC, PHI, BRS, PHQ-4, RSES, PSS-4) while accounting for missing data (Verbeke and Molenberghs, 2009). The magnitude of effects within and between the two groups will be established by Cohen's d statistic (Field, 2009). This will determine what is considered a small effect (d ≥ 0.2), a medium effect (d ≥ 0.5) and a large effect (d ≥ 0.8) (Cohen, 1988). In any case, the analytic plan will be revised once the data is collected based on the state of the art in order to apply the most up to date analytic procedures. Descriptive analysis will be used to analyse quantitative data from the SAT and their qualitative responses will be analysed using thematic analysis (Elliott and Timulak, 2005).

Discussion

This study protocol outlines a pilot RCT that seeks to investigate the preliminary efficacy and acceptability of an internet-delivered intervention, Space for Resilience, that seeks to promote well-being and resilience in a sample of college students. The intervention comprises elements from different theoretical backgrounds and it is framed within the positive psychology approach (Seligman, 2004). The present study will compare the effects of the intervention to a wait-list control group and acceptability will be examined through satisfaction with treatment. The element of support type will be preliminary explored via comparisons between two active intervention groups, one with human support and one with automated support provided. Similar to the automated support the human support is delivered asynchronously through reviews to accommodate equivalence of support delivery but also to expedite the support.

The adoption of a preventative approach with college students, who experience high levels of distress and demands, is presented as a way to address the high prevalence rates of mental health problems that this specific cohort present with at an early stage of their lives (Herrero et al., 2018). Furthermore, the positive focus of the intervention (i.e., promote well-being rather than fixing problems) is expected to address the low rates of help-seeking behaviours observed in this population (Bolier and Abello, 2014) In this sense, the Space of Resilience programme offers new learnings, skills and strategies aimed at building core strengths that will allow the individual to cope with stressors that can cause psychological strain.

Of importance, delivering these interventions in a digital, self-applied format contributes to making them more accessible and affordable than face-to-face training (Joyce et al., 2018). The review conducted by Joyce et al. (2018) outlined that few internet-delivered interventions for resilience are available and more research is needed to determine the potential benefits of these interventions. Furthermore, exploring different ways of providing support will help to build on the related literature of support type and how to make it more efficient and cost-effective. As a main limitation of this study, due to the pilot nature of the trial, sample size will not allow for the establishment of firm conclusions from the obtained results. However, it is expected that the trends observed will inform the development a future full-scale RCT.

Conclusion

Internet-delivered interventions have a substantial body of evidence supporting their efficacy, especially in the treatment of depressive and anxiety disorders (Andrews et al., 2018; Wright et al., 2019). However, moving beyond an emphasis on pathology and beginning to focus on human strengths and prevention is a worthwhile endeavour as the consequences could be substantial. On a large scale, this has the potential to reduce the burden of misery that characterises western society and foster greater happiness in greater numbers of people (Layard and Clark, 2014). The impact of such a change may be considerable in terms of increasing the prevalence of successful versus stressful life events (Andrews et al., 2018; Wright et al., 2019).

Authors contributions

AE and OM with DR conceptualised the trial design. Initial literature review and write-up was by AS and OM and this was further revised by AE and DR. SL and SF were both involved in accurate data collection and follow-up and contributed to the discussion, which was led by AE and OM, with further contributions by DR. Data analysis and interpretation will be led by OM and AE and will seek contributions from all other authors in time.

Funding

The trail was funded by joint resources provided from SilverCloud Health and Student Counselling Service, Trinity College Dublin.

Sponsor

University of Dublin, Trinity College Dublin (Ireland).

SilverCloud Health Ltd. (Ireland).

Declaration of Competing Interest

AE, SL, SF, and DR are employees of SilverCloud Health. AE and DR are members of the e-mental health research group, School of Psychology, Trinity College Dublin.