Adherence to e-mental health among youth: Considerations for intervention development and research design.

E-mental health programmes have great potential to provide young people with access to mental health support. However, it is commonly reported that adherence to these programmes is low. Low adherence can be problematic, particularly if young people do not receive the full benefits of a programme. In a research trial setting, non-adherence to treatment recommendations can prevent researchers from drawing strong conclusions about effectiveness. Although adherence has been recognised as an issue in need of attention, many of the reviews available are focused on adults and lack clear direction towards what strategies to employ. This paper presents a broad review of the adherence literature, focusing on factors associated with improving adherence to e-mental health among youth. Our view on the key elements to improve adherence identified from the existing literature are presented, and key recommendations for e-mental health intervention design are provided. These include: developing and communicating adherence guidelines based on individuals' needs and symptom severity, including customisable features to provide a tailored experience and promote a sense of agency, including engagement checks and adopting a user-centred approach by utilising strategies such as co-design. This paper provides guidance to intervention designers and researchers by outlining recommendations and considerations for intervention development and research design.

Adherence to e-mental health interventions

Mental illness can have significant negative impacts on the quality of life of young people (i.e. those aged 12–25 years).[1],[2] Despite this, more than three-quarters of youth with poor mental health do not seek professional care.[3] Delivering psychotherapy through e-mental health interventions can overcome many of the barriers that inhibit seeking help.[1],[4] These interventions typically provide support or brief therapy delivered through technological or digital platforms.[5] Digital cognitive–behavioural therapy has recently been recommended by national health bodies as a first-line treatment for young people with mild depression due to evidence of its effectiveness and potential to improve accessibility.[6] E-mental health programmes offer the opportunity for large-scale delivery of psychoeducation in schools[7] and facilitate the implementation of stepped models of care,[8] thereby reducing burden on service providers. Their capacity to be delivered at scale also offers an opportunity to reach more young people in the prevention of mental illness, and early detection and intervention.[9] While there is promising evidence to suggest e-mental health interventions can provide effective support for young people,[10] non-completion is commonly reported, and adherence to e-health programmes is low.[9] This is problematic, since as with all treatment, a certain degree of adherence is needed for an effect on outcomes to occur. Reports also suggest that adolescents’ adherence to medical treatment is challenging, with researchers proposing recommendations including education, communication considerations and motivational strategies involving extra care from health professionals or family involved in a young person’s treatment.[11-13] Young people have reported a myriad of factors negatively impacting their engagement and interaction with online programmes, including lack of time, access and technical issues, no perceived need for help, programme relevance, inappropriate content or repetitiveness, doubt regarding programme effectiveness, preferences for face-to-face help, concerns about privacy and anonymity, or perceiving a programme to be boring or activities laborious.[14-17] Adding to the complexity of this problem, non-adherence to e-health interventions may also indicate a poorly designed or delivered product, or that users’ needs have been met early. Despite the rapid development of digital mental health interventions,[18] ways to overcome many of these barriers are not well understood. This makes it challenging for researchers to develop e-mental health interventions that produce optimal adherence, thereby limiting the potential of these treatments to establish their effectiveness.

The purpose of this paper is to review the current literature pertaining to adherence to e-mental health interventions, and to provide a viewpoint on the association between adherence and intervention outcomes and factors associated with improved adherence. A snowball search strategy was utilised following an initial search of the literature in the PubMed, PsycInfo and Google Scholar databases. This method was chosen because it allowed the inclusion of all data pertaining to adherence (varied definitions of adherence are discussed below) to present a broad perspective and overview of the main themes. It also allowed for a review of the adult literature to fill gaps in the youth literature which would have been prevented in a more systematic review. Our views will be presented as a series of recommendations for those who design and evaluate e-mental health interventions. We have limited our review to interventions designed for treatment rather than prevention due to the limited research available. It is beyond the scope of this paper to discuss ways to engage healthy young people in e-mental health care aimed at prevention, who presumably have different user needs and motivations. More research is needed to help understand the challenges specific to prevention interventions, including how to motivate healthy young people who do not see a need to engage with e-mental health.

The concept of adherence and the impact on outcomes

The first major challenge in this field is the variation in the way adherence is defined and measured. The terms ‘adherence’, ‘usage’, ‘engagement’, ‘attrition’, ‘compliance’ and ‘drop-out’ are used interchangeably to describe users’ interaction with online programmes.[19-22] In many e-health interventions, log data are recorded automatically and can provide objective information about how users interact with a programme. Many of these programmes utilise cognitive–behavioural therapy strategies and are delivered in a structured format, grouping topics into approximately five or six modules designed to be accessed sequentially or to be explored freely as a user needs.[9] Frequency of logins, time spent online, number of modules or activities completed,[23] data entered, minimum number of modules required[20] as well as pages accessed and printed are common ways to measure adherence.[19],[24],[25] While these data may be collected, they are often not reported. These research design issues, including various methods to operationalise adherence, make it difficult to review the data systematically and compare the data meaningfully.

The evidence regarding the impact of adherence on the effectiveness of online interventions for mental health remains unclear. While many studies link the notion of adherence to greater symptom improvement in mental health outcomes,[7],[14],[26-29] others suggest the relationship may be more complex, finding weak effects between adherence and outcome.[30-32] In an adult sample, Donkin et al.[33] proposed that gradual programme use combined with active engagement may provide a more detailed understanding of the treatment process, as treatment effects were only found among those who completed more activities per login. However, in a sample of young adults using a self-directed online programme, Clarke et al.[34] found an inverse relationship between programme usage and outcome; that is, fewer page visits and less time online were associated with a greater reduction in depressive symptoms. These counterintuitive findings may be explained by a reduction in participants’ motivation to complete the programme due to symptom improvement. This study, along with another,[7] also found that the more severe the mental health symptoms, the greater the adherence. These findings highlight the necessity for researchers and programme developers to set adherence guidelines prospectively based on the anticipated needs and desired outcome for individuals. A recent systematic review of 69 studies measuring adherence to psychological and physical e-therapies among adults found that for psychological interventions, only two measures of adherence – module completion and a composite measure comprised of time online and number of activities completed – were related to positive treatment effects.[24] This suggests that the relationship between adherence and outcome is still largely unclear and may vary depending on the type of health condition targeted and how adherence is operationalised.

In addition to symptom severity, youth studies have found living rurally and high self-esteem[7] as well as higher academic achievement[17] were associated with greater adherence. There is also evidence to suggest that monitored settings, such as the school environment or primary-care settings, can increase adherence to online programmes for youth.[23],[35-37] Reasons for this may relate to participants having a dedicated time to complete the intervention,[17] participants feeling a sense of accountability to perform tasks in the school environment or the social presence of school staff.[38] Evidence suggests that guided interventions promote adherence in adults.[39] However, the degree of human support needed to produce the same effects for youth is unclear.[5] A pilot study examined whether the presence of a small online peer-support network that enabled adolescent participants to monitor peers’ activities and provide supportive accountability influenced adherence.[40] This study found a positive relationship between communication between participants and the mean time spent on the site. Users also reported feeling motivated to engage with the site in the first few weeks when site use was high, but this was not sustained, as site use dropped off in the later weeks. However, as this study had quite a small sample (N = 13), further research is needed to determine the potential of peer networks to promote adherence. Other factors are largely unknown.

Improving adherence in e-mental health for youth

Defining adherence

In face-to-face therapy, the degree to which a patient engages with therapy as rated by the therapist has been found to be a better predictor of treatment outcome than attendance alone.[41] Applying this concept to online therapies, simple usage metrics may fail to provide an accurate reflection of genuine treatment engagement. Sieverink et al.[25] argued that the amount of activity completed within a programme is simply a measure of attendance, whereas adherence should reflect compliance with the recommended and intended use of a programme. This is supported by Doherty et al.[42] who posited that adherence should focus explicitly on the engagement with treatment components rather than the technology, as it is simply the delivery tool. Therefore, we suggest researchers establish an upfront expectation of what is required from a user based on the level of engagement and use needed for a therapeutic benefit[43] and use this as their definition of adequate engagement. This can be estimated and evaluated if the exact therapeutic dose is unknown[25] and adjusted based on outcomes. Researchers should also have a clear understanding of the time needed for users to engage with content so that expectations and experiences are realistic and aligned. Relatedly, this definition of adherence should be made explicit to users early in programme engagement. To assist users in understanding their expectations for effective engagement, designers can consider embedding a section in the onboarding process that outlines the recommended use to experience a beneficial effect and asks users to agree to it or to input what they think is achievable.[33],[38],[44]

Programme flexibility/personalisation

Delivering personalised interventions includes tailoring content specific to individuals’ symptom profile[45],[46] or the individual’s level of severity,[8] allowing participants to select the order of module completion or the choice of session scheduling with customised reminders.[26] Incorporating features that enable personalisation can create a sense of ownership and agency over one’s treatment and content, allowing users to explore the modules that suit their needs and preferences.[28],[47] Batterham et al.[45] found some indication that a tailored intervention designed for youth was related to slightly better adherence compared to a static intervention, although these findings were not significant. Importantly, however, participants in this study were significantly more satisfied with the tailored programme, providing support for the creation of programmes designed to meet participants’ treatment needs. In line with this, Mohr et al.[38] suggested that in order to avoid the potential negative effects of rigid adherence guidelines, it may be useful to involve participants in the process of setting usage and interaction goals for their treatment. There has been support for this idea in a study with adults, which found that adherence rates improved by >20% after four adherence strategies were implemented – two of which related to providing participants with freedom to choose and tailor their treatment. However, as these changes to the delivery of the programme were made in unison, it is difficult to determine whether the combination of the four changes or individual adaptions led to the improved adherence rates.[26]

Understanding participants’ contact preferences in regards to the time and frequency of reminders and offering choice flexibility[48] may ensure that reminders and notifications are received at actionable times of the day and are not intrusive.[49] Programme designers might also consider enabling users to opt out of reminders and notifications,[48] especially giventhese features may not be useful to promote self-directed use in young people.[17] However, more research has been recommended in this area in order to understand whether reminders are beneficial.[50] The Persuasive Systems Design (PSD) framework has been applied to conceptualise the various components in online psychological therapies that might influence individuals’ attitudes or behaviour.[22] In this context, PSD features are often utilised in programmes to facilitate adherence. Two features commonly used in youth e-health interventions include tunnelling and reduction.[50] Tunnelling involves guiding users through therapeutic content in a structured and linear manner,[49] and reduction involves breaking down complex activities into simple steps.[51] Reduction strategies such as minimising the length and wordiness of modules are related to higher adherence.[50] However, evidence on the effectiveness of tunnelling is unclear. For example, Burckhardt et al.[16] found that after converting an unstructured free-to-use programme to a structured school-based programme, the positive effects on mental health outcomes were not observed. The authors of this study proposed that participants may have resented the pressure to use the programme due to the imposed usage recommendation in comparison to the free-to-use version of the programme. Supporting the idea of offering programme flexibility, Doherty and Coyle[47] found evidence to suggest that allowing young users to explore a programme and find treatment that meets their needs is useful to encourage adherence. Related to this, health research has highlighted that providing young people with a sense of control and agency through treatment decisions may be a developmentally appropriate way to improve adherence.[52] Other researchers have suggested that allowing users to choose modules freely may increase participants’ self-efficacy.[46] Further research measuring factors that might influence participants’ self-efficacy is needed, given its relationship with self-efficacy and adherence in health research.[53],[54] On the other hand, it is important to consider that many e-mental health interventions are purposely designed in a structured manner to mirror the delivery of face-to-face therapy.[49] Therefore, it may be necessary to weigh up these factors when planning the structure of an online programme.

Co-design/participatory design

Involving the end users in guiding the design and development of an intervention is important in order to ensure it is appropriately matched with a youth’s goals and needs.[55] A recent review examining the effectiveness of e-mental health interventions for young people identified that all programmes failed to use or report Participatory Design methods.[10] Ignoring the end user or assuming what they need may result in programme designers inadvertently including features that are: mismatched to a young person’s preferences, age inappropriate, patronising, tedious, repetitive or aesthetically unappealing.[56] On the other hand, tailoring the programme and its content to users’ needs and preferences[15] may help to increase adherence by ensuring interventions are credible, useful and engaging. It is necessary to apply user-centred design practices so that programmes include content and features that are relevant to young people, and are not directed by unfounded ideas and beliefs.[48],[57],[58] This co-design process is also important, as young people are digital natives and may have preferences that are different from those of adult users.[59] Finally, it is important for researchers to consider carefully the unique e-mental health needs and preferences of all young people and to ensure that disadvantaged and rural youth are represented in the design process.[60]

Supported use

Intervention designers may also consider whether it is appropriate to provide guidance to support and monitor participants’ adherence and reinforce participants’ accountability.[38],[40],[61] Digital treatments can be utilised in a number of different ways, including as a purely self-directed treatment programme, treatment supported by a non-therapist or a blended care model whereby the treatment is digitally led with therapist support or therapist led with digital support.[62] There is evidence to suggest that face-to-face or digital support may positively influence programme support and outcomes.[9] The supportive accountability model suggests that adherence to treatment increases when users’ actions are accountable to a respected and credible support person.[39],[63] Offering online or digital components of therapy as an adjunct to therapist-led support may offer an alternative for young people who prefer professional contact.[56] This method of support may also be cost-effective and efficient for consumers and therapists.[62],[63] However, more research has been recommended to understand how this method is implemented,[62] including considering the varying needs of different populations such as young people. Depending on the level of support required, this does not necessarily need to be provided by therapists, as research has demonstrated that automated support can provide the same benefits to adherence as human support,[27] and the qualification of those providing guidance may not be a significant influencer.[39] There is also great potential for artificial intelligence and advanced computational methods, including chatbot technologies, to be utilised in interventions as an adjunct or alternative to human support.[64] Employing data-driven methods can also help to ensure resources such as face-to-face help are utilised efficiently. An example of this is monitoring automatic usage data to recognise users not engaging with a programme and targeting these users with a tailored engagement strategy which may include face-to-face support.[65]

Further evaluations

To encourage active engagement, designers can embed a range of activities in e-mental health programmes, including multiple-choice quizzes[33] and case-enhanced learning strategies[44] that use educational stories to exemplify problem solving. These activities can also be used to evaluate knowledge and retention. Providing feedback on users’ answers can ensure they are achieving the desired outcomes. Engagement checks such as reviewing responses to online activities[29] or measuring the completion of the non-compulsory activities[7] can then help to ascertain whether users are indeed engaging with the intervention. Donkin et al.[24] recommended embedding a time-out function when a user is not actively engaged to help ensure this measure is accurately represented. Without this function, it is difficult to determine whether increased time online is due to actual engagement or simply a participant leaving a programme open. To ensure accurate representation further of time online as a measure of adherence, it is also important to consider how a user’s stage of development may impact their comprehension and processing speed.[33] To understand whether a young person is engaging with various treatment components both online and offline, researchers may also consider gathering information from key observers such as parents, teachers or siblings.

Adherence to treatment may be improved by implementing adaptive intervention designs such as randomised encouragement trials, which aim to deliver treatment that meets patients’ needs by determining intervention goals and doses individually.[66] It is also important for researchers to consider carefully the approach used to analyse data in a randomised control trial (RCT). Following an intention-to-treat analysis, a complier average causal effect (CACE) analysis has been recommended as an alternative to per-protocol or on-treatment analyses, as CACE analysis does not interfere with randomisation and takes into account the proportion of participants who comply with treatment.[67] CACE analysis estimates the treatment effects of the intervention group compared to the proportion of compliers in the control group, thereby improving the validity of the comparison.[68]

It is important for researchers to consider the factors that impact adherence when designing interventions for youth, but also important to measure, report and describe these clearly in study protocols. Asking participants to report reasons for non-use is also important for uncovering new factors and learning how to address these.[23] Much of what is currently understood regarding factors that may improve adherence is based on predictive modelling. However, experimental research is needed to examine whether design and service elements affect adherence rates[26] and to understand the effect of different doses of treatment.[69] To attempt to understand what the active elements are of online interventions, researchers can consider alternative study designs to the RCT such as the multiphase optimisation strategy (MOST) methodology or a fractional factorial design.[27]

Investigating young people’s treatment expectations and their desired outcomes of e-mental health interventions can also highlight factors that may be influencing their adherence patterns. Researchers can consider measuring the predictors of adherence and engagement in traditional therapy such as the therapeutic alliance, an individual’s level of motivation and need for treatment care.[31],[47] Measuring such factors can shed light on how the therapeutic dose may differ among participants, as well as what the unintended side effects may be. A user may experience outcomes that are different from those intended, or experience longer-term benefits that are not immediately recognisable or measured.[25] Importantly, this would help determine whether early treatment improvements do influence further programme usage.[20] Currently, research on the effects of alliance on adherence in e-mental health programmes is scarce.[70],[71] However, one study demonstrated a positive relationship between perceived emotional connection with the programme (myCompass) and number of logins, psychoeducation modules completed and frequency of self-monitoring.[72]

Conclusion

There is currently insufficient evidence to outline clearly how to increase adherence to e-mental health among youth. This is partially due to differences in the operationalisation and reporting of adherence, large variability in the delivery and design of online interventions and the heterogeneity of users across studies. Continued evaluation of youths’ adherence to online treatment is needed to develop this area and to validate researchers’ investment in modifying and creating interventions. The research reviewed in this paper highlights important considerations that can be applied to the design of research studies and online interventions for youth. Table 1 outlines a summary of recommendations for intervention and research design to improve adherence to e-mental health. It is important to consider how a young person’s stage of development, treatment needs and experience with technology might influence their interaction with, and sustained use of, an online intervention. It is recommended that future research continues to trial and report which strategies are effective, or not, at promoting engagement and adherence. More research is needed to understand the relationship between these individual variables, adherence and outcome. The current research highlights the complexity in understanding usage data, the importance of understanding why and how individuals with different needs adhere to programmes and how individual factors impact treatment benefits. Therefore, when designing interventions, determining guidelines for adherence and measuring how adherence relates to treatment outcomes, it is necessary for researchers to consider that users may desire different outcomes or goals from those intended by developers[43] and that partial completion of online programmes may be beneficial and satisfy some users’ needs.[69] The authors note that this paper was limited by its broad search strategy and cannot be considered a comprehensive review. This approach was taken due to the limitations outlined above, including the limited youth-focused adherence research published, differences in definitions and inconsistent reporting of programme usage and meaningful engagement. The authors aimed to summarise the main themes pertinent to readers interested in young people’s adherence to e-mental health.

Table 1.

Recommendations for intervention and research design to improve adherence to e-mental health.

Intervention design
Embed a terms and conditions section to which users agree
Outline expectations of programme usage at the outset
Include reminders with customisable scheduling
Build a time-out function
Include interactive activities and non-compulsory activities
Embed engagement checks and provide feedback on responses
Include the user in the design process to aid in the understanding of user needs
Consider whether persuasive design features may be incorporated

Click here for additional data file.

Supplemental material, sj-pdf-1-dhj-10.1177_2055207620926064 for Adherence to e-mental health among youth: Considerations for intervention development and research design by Melinda R Achilles, Melissa Anderson, Sophie H Li, Mirjana Subotic-Kerry, Belinda Parker and Bridianne O’Dea in Digital Health