A usability study of an internet-delivered behavioural intervention tailored for children with residual insomnia symptoms after obstructive sleep apnea treatment.

Better Nights, Better Days (BNBD) is a 5-session online intervention designed to treat insomnia in 1-10-year-old children (Corkum et al. 2016). Obstructive sleep apnea (OSA) and insomnia commonly occur in children and, after surgical treatment for OSA, it is estimated that up to 50% of children may continue to suffer from insomnia symptoms. Access to insomnia interventions following OSA treatment is limited as there are few programs available, few trained practitioners to deliver these programs, and limited recognition that these problems exist. The current study involved the usability testing of an internet-based parent-directed session of BNBD tailored towards the needs of children (ages 4-10 years) who experience residual insomnia symptoms after treatment of OSA. This new session was added to the BNBD program. Participants (n = 43) included 6 parents, 17 sleep experts, and 20 front-line healthcare providers who completed and provided feedback on the new session. Participants completed a feedback questionnaire, with both quantitative and qualitative questions, after reviewing the session. Quantitative responses analyzed via descriptive statistics suggested that the session was primarily viewed as helpful by most participants, and open-ended qualitative questions analyzed by content analyses generated a mix of positive and constructive feedback. The results provide insights on how to optimally tailor the BNBD program to meet the needs of the target population and suggest that testing the session on a larger scale would be beneficial.

Introduction

Insomnia has a significant impact on children's daily functioning and development (Curcio et al., 2006; Paavonen et al., 2000) and previous studies suggest that there are significant barriers to accessing pediatric behavioural sleep interventions (Boerner et al., 2013; Honaker & Meltzer, 2016). Better Nights, Better Days (BNBD) is an interactive 5-session, parent-directed eHealth program intended to share psychoeducation and behavioural strategies about insomnia in 1–10-year-old children (Corkum et al., 2016). Sessions focus on the importance of sleep and consequences of poor sleep (Session 1), healthy sleep practices (Session 2), falling asleep independently (Session 3), staying asleep through the night, reducing early morning awakenings, and ensuring adequate napping for younger children (Session 4), and a review of progress combined with future goal setting (Session 5). The focus of the current study was a usability test of a new session of BNBD tailored for the specific needs of parents of children ages 4–10 years who had previously been surgically treated for Obstructive Sleep Apnea (OSA).

OSA involves a narrowing or obstruction of the upper airway during sleep resulting in ventilatory disturbances, caused either by adenotonsillar hypertrophy (Capdevila et al., 2008) or obesity (Capdevila et al., 2008; Gaines et al., 2018; Patinkin et al., 2017), and results in sleep disturbance and daytime impairment (Rosen, 2010). Snoring, gasping, and choking are the most common nighttime symptoms of OSA for children (Dehlink and Tan, 2016; Trosman, 2013). The consequences of OSA are significant and can include psychosocial problems such as behavioural dysregulation (Blechner and Williamson, 2016; Owens, 2009), cognitive and school-related problems (Gozal, 2009; Hunter et al., 2016; Xanthopoulos et al., 2015), as well as increased risk of physical problems such as diabetes, cardiovascular disease, and obesity (Framnes and Arble, 2018).

OSA is often thought of as an adult disorder; however, the disorder is common among children, with a prevalence of 1–5% (Lumeng and Chervin, 2008; Meltzer et al., 2010). While the primary risk factor for adult OSA is male gender and obesity (Gaines, Vgontzas, Fernandez-Mendoza, & Bixler, 2018; Qaseem et al., 2014), the primary contributor to pediatric OSA is enlarged adenoids or tonsils (i.e., adenotonsillar hypertrophy) (Erler and Paditz, 2004; Rosen, 2010). Various other risk factors have been described including asthma, allergies, prematurity, ethnicity, and exposure to environmental tobacco smoke (Erler and Paditz, 2004; Redline et al., 1999; Rosen, 2010). Prevalence rates are highest when children are 3–6 years old, a period during which the tonsils and adenoids are largest relative to the size of the upper airway (Ahn, 2010; Li et al., 2016; Sheldon et al., 2014). Due to the risks associated with childhood OSA, early identification and treatment are essential (Marcus et al., 2012). Adenotonsillectomy (AT; surgical removal of adenoids and tonsils) is recommended as the first-line treatment for childhood OSA (Cielo and Gungor, 2016; Tan et al., 2016; Trosman, 2013).

Previous research indicates that many children continue to have sleep disturbances after adenotonsillectomy. This can in part be understood by the high comorbidity rate between OSA and insomnia (Al-Jawder and BaHammam, 2012; Lack and Sweetman, 2016). Insomnia involves problems initiating sleep, staying asleep, or waking too early (American Academy of Sleep Medicine, 2014). Preliminary research indicates a rate of comorbidity among children of approximately 30% (Kukwa et al., 2016; Owens et al., 1998); however, to date, little research has been conducted on this comorbidity in children (Byars et al., 2011). Children with comorbid OSA and insomnia, like children with just OSA, tend to undergo adenotonsillectomy as a first-line treatment for their sleep-related issues. While adenotonsillectomy may successfully treat OSA, it has been reported that >50% of children with comorbid insomnia and OSA may continue to have insomnia symptoms after recovering from surgery (i.e., six months post-OSA surgery) (Chervin et al., 2014), meaning that approximately 15% of all kids with OSA may continue to have insomnia after treatment for OSA. The cause of this is likely multifaceted, but one potential cause is that apnea-related episodes can result in difficulties falling asleep and contribute to night awakenings, establishing a behavioural pattern which continues even after adenotonsillectomy (Byars et al., 2011).

Even though it is common for insomnia to persist after OSA treatment, research on the effectiveness of treatments for residual insomnia is limited (Björnsdóttir et al., 2013; Manickam et al., 2015). Insomnia is less likely to be screened for in the context of OSA (Byars et al., 2011), and as such, insomnia may continue to present itself even after OSA has been treated. Thus, treatments should be explored for children with treated OSA who continue to suffer from insomnia after recovering from adenotonsillectomy.

Behavioural interventions are the recommended treatment for children with insomnia (Vriend and Corkum, 2011; Meltzer and Mindell, 2014; Morgenthaler et al., 2006; Taylor and Roane, 2010), however, in-person treatment is often difficult to access due to factors such as finances, transportation, or a general lack of available services (Speth et al., 2015). eHealth interventions (i.e., behavioural interventions delivered via the internet) have become increasingly popular over the last decade, as they improve accessibility. eHealth interventions designed for a wide range of children's mental and physical health issues have been shown to be an effective and cost-saving alternative to providing children with in-person treatment (Cushing and Steele, 2010). While there are no published eHealth interventions for insomnia in school-aged children, there is preliminary evidence demonstrating that distally provided behavioural interventions can improve sleep in children with insomnia (Corkum et al., 2016); however, such interventions are not tailored towards the unique needs of children with ongoing sleep problems after OSA treatment.

The primary research question of the current usability study was focused on determining the overall usability and general need of the OSA-tailored session by collecting feedback from relevant stakeholders. An additional goal of the current study was to collect feedback from parents, sleep experts and healthcare providers (HCPs) to help determine the general need for the tailored session. A randomized control trial (RCT) was recently completed to evaluate the effectiveness of BNBD (Corkum et al., 2018); therefore, it was not necessary to re-evaluate the whole program, but rather focus on the new OSA-tailored session. The OSA-tailored session was added to the front-end of BNBD for parents to complete before BNBD itself and includes psychoeducation about OSA, the reasons for the comorbidity between OSA and insomnia, and an elaboration of how insomnia symptoms can be problematic even after OSA treatment (Table 1). Feedback collected for the current usability study design was based on Peter Morville's “user experience honeycomb” model which postulates that multiple usability factors (i.e., usefulness, usability, desirability, valuableness, accessibility, and credibility) work together to create a meaningful online user experience (Morville and Sullenger, 2010). To test the OSA-tailored session, we collected qualitative and quantitative feedback from parents, sleep experts, and front-line HCPs. Based on the usability study feedback, we expected that qualitative and quantitative feedback would be primarily positive across all participant groups, and that feedback would provide useful information to guide modifications for the session.

Table 1

Overview of content for the OSA-Insomnia session and each of the original BNBD sessions.

Session name	Session content
OSA-Insomnia Pre-session: navigating insomnia after OSA treatment	• Education about OSA etiology, symptoms, consequences, and primary treatments • Understanding the relationship between OSA and insomnia • Investigating the behaviourally-based connection between the two disorders (e.g., how night wakings that arose during OSA may become habitual even after OSA treatment)
Session 1: introduction to Better Nights, Better Days (BNBD)	• Education about how sleep works and the consequences of poor sleep with a focus on insomnia • Introduction to the BNBD team
Session 2: healthy sleep practices	• Information about healthy sleep practices and the ways in which they can lead to better sleep
Session 3: settling to sleep	• Focus on settling independently at bedtime (i.e., self-soothing)
Session 4: going back to sleep	• Addresses what to do if a child wakes during the night or is up too early in the morning
Session 5: looking back and ahead	• Review the progress made over the previous sessions • Revisit goals and address how to maintain them in the future

Method

Participants

The sample of the current study consisted of stakeholders in the OSA-tailored session of BNBD (n = 43). Stakeholders included parents (n = 6), sleep experts (n = 17), and front-line HCPs (n = 20).

Parents

The parent group included primary caregivers of typically-developing children who had received successful surgical treatment for OSA. Children were included if they had undergone surgery 6 to 18 months prior to recruitment. The ages of children ranged from 4 to 10 years (M = 6.17). The timeframe of surgery relative to the beginning of the study was chosen to give children enough time to recover from the procedure, but not enough time for adenoids to regrow and have a recurrence of symptoms (which can happen in roughly 7% of children; Babademez et al., 2017). To meet inclusion criteria, parents and their children had to reside in Canada, the child had to have undergone successful removal of the adenoids and/or tonsils for OSA 6–18 months prior, OSA symptoms were resolved, and the child had to present with residual insomnia. Finally, all parents were expected to have ongoing internet and email access for the study duration, and to be comfortable with reading at a grade 8 level. Parents who self-reported cognitive or physical issues that could interfere with their online participation were excluded. Additionally, parents of children with developmental or neurodevelopmental problems, as reported by parents, were excluded.

Of the 16 parents who expressed interest in the study, seven were not eligible due to the child not meeting criteria for insomnia (n = 3), presenting with a neurodevelopmental disorder (n = 2), or presenting with residual OSA symptoms (n = 2). Additionally, three parents did not begin or did not finish their review of the session, resulting in a final sample of six parents who completed the study. Demographics are reported for the five parents who submitted demographic information (see Table 2). The sample was mostly comprised of mothers, with a high level of education and income, and who were employed full-time. The children had a mean age of six and half years, three of the five children were male, and most were living in two-parent homes with a sibling in an urban area. The parent sample was all Caucasian and English-speaking. See Table 2 for more information about the characteristics of the parent sample.

Table 2

Demographics for parents.

	Count (%)	Mean (SD)	Range
Parent characteristics
Age (in years)		38.4 (6.47)	27–43
Relationship to child
Mother	4 (80%)
Father	1 (20%)
Ethnicity
White/Caucasian	5 (100%)
Education
Master's	2 (40%)
Bachelor's	1 (20%)
Some university	1 (20%)
Some college	1 (20%)
Employment status
Full-time	3 (60%)
Homemaker	1 (20%)
Student	1 (20%)
Child characteristics
Age (in years)		6.2 (2.56)	4–10
Sex
Female	2 (40%)
Male	3 (60%)
Ethnicity
White/Caucasian	5 (100%)
Family characteristics
Children at home
One	1 (20%)
Two	3 (60%)
Three	1 (20%)
Adults at home
One	2 (40%)
Two	3 (60%)
Primary language
English	5 (100%)
Gross household income
$150,000+	1 (20%)
$100,000–$124,999	1 (20%)
$80,000–$99,999	2 (40%)
$10,000–$14,999	1 (20%)
Community description
City under 500,000 people	3 (60%)
Rural	2 (40%)

Note. One parent did not submit responses for the demographic form.

Sleep experts

The sleep experts group included registered HCPs in Canada who specialized in pediatric OSA or sleep disorders and had 50% or more of their clinical work or research pertaining to sleep medicine. They were also required to have ongoing internet and email access for study duration and could not report any cognitive or physical issues that would impede their online participation.

Of the 19 sleep experts who expressed interest in the study, one was not eligible due to not seeing children as part of their regular clinical practice. Additionally, one sleep expert did not finish reviewing the session, resulting in a final sample of 17 sleep experts who completed the study. Sleep experts were comprised of pediatricians (n = 7), psychiatrists (n = 2), respirologists/otolaryngologists (n = 2), nurses (n = 2), sleep technicians (n = 2), one psychologist, and one neurologist. Most of the sample was female, had medical or graduate training, were working in the field for over 10 years, and worked in a hospital setting. While there was variability in clinical practice, all had some of their practice dedicated to working with children with OSA. See Table 3 for further information about sleep expert and HCP characteristics.

Table 3

Demographics for sleep experts and HCPs.

	Sleep experts (n = 17)	HCPs (n = 20)
Personal characteristics
Sex
Female	14 (82.3%)	18 (90%)
Male	3 (17.7%)	2 (10%)
Highest level of education
Medical	10 (58.9%)	7 (35%)
Masters	2 (11.7%)	8 (40%)
Doctorate	3 (17.7%)	2 (10%)
Undergraduate	2 (11.7%)	3 (15%)
Primary profession
Pediatrician	7 (41.2%)	3 (15%)
Psychologist	1 (5.9%)	8 (40%)
Psychiatrist	2 (11.7%)	3 (15%)
Nurse	2 (11.7%)	2 (10%)
General practitioner	0	3 (15%)
Respirologist	2 (11.7%)	0
Technologist	2 (11.7%)	0
Neurologist	1 (5.9%)	0
Social worker	0	1 (5%)
Length of time as a healthcare professional
10+ years	16 (94.1%)	12 (60%)
7–10 years	1 (5.9%)	3 (15%)
4–6 years	0	3 (15%)
1–3 years	0	1 (5%)
<1 year	0	1 (5%)
Work with sleep disorders
Setting where children with OSA are seen
Hospital	11 (64.7%)	7 (35%)
Private Practice	4 (23.5%)	7 (35%)
School	0	4 (20%)
University	2 (11.7%)	1 (5%)
Community MH	0	1 (5%)
Percentage of practice devoted to children with any sleep disorder
91–100%	1 (5.9%)	0
81–90%	0	1 (5%)
71–80%	3 (17.6%)	0
61–70%	1 (5.9%)	0
51–60%	1 (5.9%)	0
41–50%	3 (17.6%)	2 (10%)
31–40%	3 (17.6%)	2 (10%)
21–30%	2 (11.8%)	2 (10%)
11–20%	1 (5.9%)	3 (15%)
1–10%	1 (5.9%)	10 (50%)
0%	1 (5.9%)	0
Percentage of practice devoted to children with OSA
91–100%	0	0
81–90%	0	0
71–80%	0	0
61–70%	1 (5.9%)	0
51–60%	1 (5.9%)	0
41–50%	3 (17.6%)	0
31–40%	3 (31–40%)	1 (5%)
21–30%	1 (5.9%)	0
11–20%	2 (11.8%)	2 (10%)
1–10%	4 (23.5%)	12 (60%)
0%	0	5 (25%)
Number of children worked with who have untreated OSA
40+	5 (29.4%)	1 (5%)
31–40	6 (35.3%)	0
21–30	2 (11.8%)	1 (5%)
11–20	2 (11.8%)	2 (10%)
1–10	1 (5.9%)	11 (55%)
0	1 (5.9%)	5 (25%)
Number of children worked with who have surgically treated OSA
40+	6 (35.3%)	1 (5%)
31–40	7 (41.2%)	0
21–30	0	0
11–20	1 (5.9%)	3 (15%)
1–10	2 (11.8%)	11 (55%)
0	1 (5.9%)	5 (25%)
In your best estimate, what percent of children have insomnia (e.g., trouble falling asleep, trouble staying asleep, waking too early, not getting enough sleep) that persists after surgical treatment of OSA?
0%	2 (11.8%)	1 (5%)
1–10%	7 (41.2%)	7 (35%)
11–20%	4 (23.5%)	4 (25%)
21–30%	1 (5.9%)	1 (5%)
31–40%	1 (5.9%)	3 (15%)
41–50%	0	1 (5%)
51–60%	1 (5.9%)	1 (5%)
61–70%	1 (5.9%)	0
71–80%	0	2 (10%)
81–90%	0	0
91–100%	0	0

Note. One HCP endorsed >50% of clinical work devoted to sleep disorders, however she also noted that sleep problems were not a primary focus of her practice. Sleep issues were typically assessed and managed in the context of other issues that were more pertinent to her clinical practice. As such, this participant was grouped as an HCP.

Note. One HCP endorsed no regular experience with any sleep disorders (0%); however, she noted having exposure to children with sleep disorders in the past. As such, she was included in the final sample.

Front-line HCPs

This group included HCPs in Canada who occupied various healthcare-related roles. This category included HCPs who saw children with a range of problems but who did not specialize in pediatric sleep disorders within their clinical practice. Their clinical work or research with sleep would have made up <50% of their professional activities. Like parents and sleep experts, HCPs were required to have ongoing internet and email access for the study duration and could not report any cognitive or physical issues that would impede their online participation.

Of the 24 HCPs who expressed interest in the study, one was not eligible due to not seeing children with sleep problems in regular clinical practice. Additionally, three HCPs did not complete the feedback questionnaire, resulting in 20 HCPs completing the study. HCPs were comprised of psychologists (n = 8), pediatricians (n = 3), general medical practitioners (n = 3), psychiatrists (n = 3), nurses (n = 2), and a social worker. Most HCPs were female, had medical or professional graduate training, worked for over 10 years in their respective fields, and saw children with sleep problems in either a hospital or private practice setting, and children with sleep problems only comprised 1–10% of their clinical practice. Most had worked with 10 or fewer children with treated or untreated OSA. See Table 3 for further information about sleep expert and HCP characteristics.

Measures

Screening and demographic measures

Screening Questionnaire - Parents (SQ-P)

The SQ-P is a 21-item questionnaire used to determine parents' eligibility for the study. It reviewed inclusion criteria such as ability to actively participate in the online components, ability to appropriately communicate in English, and a preliminary exclusion of developmental and neurodevelopmental disorders in parents' children.

Behavioural Insomnia Questionnaire (BIQ)

The BIQ is an author-created, 23-item questionnaire completed by parents which is used to confirm the presence of insomnia symptoms in their child. Questions pertained to bedtime, number of nighttime awakenings, morning wake up time, bedtime routines, and related issues with sleep. It focused on both the quality and quantity of sleep and was based on pediatric insomnia criteria proposed by Anders and Dahl (2007).

Health-Related Questionnaire - Modified (HRQ-M)

The parent-completed HRQ-M is an author-created, 37-item questionnaire about OSA, neurodevelopmental disorders, other mental health disorders, and sleep disorders. It was primarily used to identify the presence of disorders which would make the child ineligible for this study (e.g., neurodevelopmental disorders such as ADHD, mental health disorders such as depression, or sleep disorders such as narcolepsy).

Demographic Questionnaire – Parents

A 33-item questionnaire on background characteristics of parents such as employment, family composition, and personal characteristics such as sex and age of parents, children, and spouses.

Demographic Questionnaire - Sleep Experts & HCPs

An 11-item questionnaire on background characteristics of sleep experts and HCPs such as personal characteristics (e.g., age and sex), education, employment, and clinical experience with children who have sleep disorders.

Usability measures

Session Feedback Questionnaire - Parents (SFQ-P)

The SFQ-P is a 35-item questionnaire that consisted of 26 closed-ended and 9 open-ended questions about the usefulness, usability, desirability, value, accessibility, and credibility of the OSA-tailored session, as well as about participants' impressions of the videos and features (e.g., worksheets and supplemental materials) and the general readiness of the session for end users. The questionnaire was based on Peter Morville's user experience honeycomb (Morville and Sullenger, 2010) and adapted from questions used in a previous usability study evaluating the original BNBD intervention (Speth et al., 2015). The SFQ-P was completed at the end of the OSA-Insomnia session to evaluate participants' experience with the new session. Closed-ended questions were rated on a Likert scale from 1 to 5 (e.g., “Strongly Agree” to “Strongly Disagree”) or 1–3 (e.g., “Yes”, “Maybe”, “No”) to capture levels of agreement with each aspect of the session. Responses which agreed with the presented statements (i.e., “Agree”, “Strongly Agree”, and “Yes”) were considered positive responses. Open-ended questions offered participants the opportunity to elaborate more generally about each aspect of the session in text boxes with no word or character limit.

2.2.2.2. Session Feedback Questionnaire - Sleep Experts & HCPs (SFQ-SE & HCP)

The SFQ-SE & HCP are both 40-item questionnaires that are almost identical to the SFQ—P, but the wording of the questions focused on participants' thoughts about the OSA-tailored session for parents rather than for their own use of the session. This questionnaire included reworded versions of the 35-items from the SFQ—P, plus 4 additional open-ended questions to obtain feedback on potential additions, deletions, changes, and general readiness of the session from a professional/expert perspective. The questionnaire also added one closed-ended question asking participants to rate the general need for the session on a three-point scale (“Yes”, “Maybe”, “No”).

Procedure

Parents were recruited through direct mailing in collaboration with the Otolaryngology Department of a local children's hospital. Specifically, surgical records were accessed to identify 3-to-10-year-old children who had tonsils and/or adenoids removed for OSA in the past 6 to 18 months. Five-hundred potentially eligible parents were then mailed the study flyer. Sleep experts and HCPs were recruited through professional connections, BNBD social media (Facebook), and postings on websites of relevant organizations (e.g., the Canadian Sleep Society).

All potential participants expressed interest in the study by initiating email correspondence with the study investigators. Interested parents were then emailed a link to an online Screening Questionnaire. If eligible, parents progressed to a consent form detailing study procedures and requirements. Subsequently, they completed the BIQ and HRQ-M. Interested sleep experts and HCPs were also emailed a link directly to a consent form and a subsequent demographic form (eligibility was confirmed at this step). All screening and demographic measures were hosted on REDCap©, a secure online survey program. All participants were then emailed a link to the OSA-Insomnia session.

After finishing the OSA-Insomnia session, participants completed the Session Feedback Questionnaire which was housed on Opinio, an online survey program. Participants were emailed a $30.00 gift card to Amazon.ca as an honorarium following their participation in the study.

Data analysis

The responses on the Feedback Questionnaires resulted in both quantitative and qualitative data. Descriptive statistics (percentages) were calculated for quantitative questions, while qualitative data were analyzed and interpreted using directed content analysis (Assarroudi et al., 2018), which allows coding within an existing framework. Qualitative data was organized using Microsoft Word, and quantitative data was analyzed using IBM SPSS 24 (released 2016) and Microsoft Excel. Codes were created within Morville's ‘honeycomb’ categories (e.g., “usefulness”, “desirability”) (Morville and Sullenger, 2010) and other relevant elements of the session (features, readiness, general need). Coding units involved words, clauses, and complete sentences. Two coders independently coded the feedback in two rounds. The first round involved coding units into nine categories based on Morville's categories (Morville and Sullenger, 2010) and the second round involved placing coded units into two broader categories (strengths and challenges) for the purpose of analysis. Discrepancies were resolved by mutual agreement. Inter-rater reliability was calculated for both rounds by looking at percent agreement between raters. Agreement of 98.6% was obtained for the first round of coding and an agreement of 99.2% was obtained for the second round of coding.

Open-ended questions were included in the Feedback Questionnaires to help further understand the quantitative ratings. The open-ended questions resulted in 368 units of feedback provided about the OSA-Insomnia session. Within each of Morville's “honeycomb” categories and other areas of feedback, two categories were generated to describe feedback: strengths and challenges. Strengths referred to positive aspects of the session, and challenges pertained to practical barriers regarding the session's usability and need, as well as suggestions for improvement. Feedback provided by sleep experts and HCPs was included in the analysis if the same point (e.g., adding more videos to the session) was noted in >10% of the responses in a given response category (e.g., challenges/usefulness), after accounting for opposing feedback within the response category which nullified corresponding feedback (e.g., one sleep expert stating that more videos would be beneficial and one saying that less videos would be beneficial resulting in null data), and if the feedback was provided by more than one participant. Since parents were the primary stakeholders in the current study and a small sample of parents participated, all feedback from parents was included in the analysis.

Results

A summary of the survey results is provided below, with the quantitative results followed by the qualitative results. Additional details are presented in Table 4. Questions are grouped based on the concept they were designed to assess (e.g., usefulness, usability, desirability). The following sections also describe primary strengths and challenges mentioned by each group. There were 151 total units of feedback for strengths and 217 total units for challenges. Parents provided 7 units of feedback for strengths and 14 units for challenges resulting in 21 total units. Sleep experts provided 56 units of feedback for strengths and 89 units for challenges resulting in 145 total units. HCPs provided 88 units of feedback for strengths and 114 units for challenges resulting in 202 total units. Below is a summary of the qualitative feedback. Additional detailed feedback, including quotes from participants, is presented in a supplementary table (Table S5).

Table 4

Responses from feedback questionnaires for each of the usability factors.

Question	Strongly agree	Agree	Neither agree nor disagree	Disagree	Strongly disagree
Useful 1 – session provided information to help better understand child's ongoing sleep problems following treatment for child's obstructive sleep apnea (OSA)
Parents	2 (33.3%)	4 (66.7%)	–	–	–
Sleep experts	12 (70.6%)	4 (23.5%)	1 (5.9%)	–	–
HCPs	12 (60.0%)	7 (35.0%)	1 (5.0%)	–	–
Useful 2 – Session allowed understanding about why insomnia and OSA are related
Parents	2 (33.3%)	4 (66.7%)	–	–	–
Sleep experts	10 (58.8%)	6 (35.3%)	1 (5.9%)	–	–
HCPs	10 (50.0%)	8 (40.0%)	2 (10.0%)	–	–
Usable 1 – parents would find this session to be user-friendly
Parents	1 (16.7%)	3 (50.0%)	2 (33.3%)	–	–
Sleep experts	9 (52.9%)	7 (41.2%)	1 (5.9%)	–	–
HCPs	6 (30.0%)	12 (60.0%)	2 (10.0%)	–	–
Usable 2 – parents would be able to follow along and understand the session with ease
Parents	2 (33.3%)	3 (50.0%)	1 (16.7%)	–	–
Sleep experts	8 (47.1%)	9 (52.9%)	–	–	–
HCPs	5 (25.0%)	10 (50.0%)	4 (20.0%)	1 (5.0%)	–
Usable 3 – Session would take a reasonable amount of time to review
Parents	3 (50.0%)	3 (50.0%)	–	–	–
Sleep experts	8 (47.1%)	4 (23.5%)	5 (29.4%)	–	–
HCPs	8 (40.0%)	10 (50.0%)	–	2 (10.0%)	–
Desire 1 – parents would think the session is visually appealing and the organization of information on the screen is clear
Parents	1 (16.7%)	3 (50.0%)	2 (33.3%)	–	–
Sleep experts	8 (47.1%)	9 (52.9%)	–	–	–
HCPs	8 (40.0%)	8 (40.0%)	3 (15.0%)	1 (5.0%)	–
Access 1 – parents will think that this session is easy to follow and they will be able to absorb all the necessary information in this session
Parents	4 (66.7%)	2 (33.3%)	–	–	–
Sleep experts	7 (41.2%)	8 (47.1%)	2 (11.8%)	–	–
HCPs	7 (35.0%)	13 (65.0%)	–	–	–
Access 2 – parents will be able to easily navigate this session and find all the relevant information
Parents	N/A	N/A	N/A	N/A	N/A
Sleep experts	8 (47.1%)	6 (35.3%)	2 (11.8%)	1 (5.9%)	–
HCPs	6 (30.0%)	13 (65.0%)	1 (5.0%)	–	–
Features 1 – parents will find the videos in this session helpful
Parents	1 (16.7%)	4 (66.7%)	1 (16.7%)	–	–
Sleep experts	14 (82.4%)	3 (17.6%)	–	–	–
HCPs	12 (60.0%)	6 (30.0%)	2 (10.0%)	–	–
Features 2 – parents will find the activities in this session helpful
Parents	3 (50.0%)	2 (33.3%)	1 (16.7%)	–	–
Sleep experts	9 (52.9%)	7 (41.2%)	1 (5.9%)	–	–
HCPs	6 (30.0%)	11 (55.0%)	3 (15.0%)	–	–
Features 3 – parents will find the written summary of the session content helpful
Parents	2 (33.3%)	4 (66.7%)	–	–	–
Sleep experts	11 (64.7%)	5 (29.4%)	1 (5.9%)	–	–
HCPs	7 (35.0%)	12 (60.0%)	1 (5.0%)	–	–
Features 4 – parents will find additional videos from sleep and OSA specialists to be a helpful addition to the program
Parents	3 (50.0%)	1 (16.7%)	2 (33.3%)	–	–
Sleep experts	10 (58.8%)	–	5 (29.4%)	2 (11.8%)	–
HCPs	5 (25.0%)	10 (50.0%)	4 (20.0%)	1 (5.0%)	–
Valuable 1 – the information provided in this session would be valuable to parents
Parents	4 (66.7%)	2 (33.3%)	–	–	–
Sleep experts	13 (76.5%)	3 (17.6%)	1 (5.9%)	–	–
HCPs	15 (75.0%)	5 (25.0%)	–	–	–
Valuable 2 – this session will provide parents with a deeper understanding of why children continue to experience sleep problems after OSA treatment
Parents	3 (50.0%)	3 (50.0%)	–	–	–
Sleep experts	13 (76.5%)	3 (17.6%)	1 (5.9%)	–	–
HCPs	12 (60.0%)	5 (25.0%)	1 (5.0%)	2 (10.0%)	–
Valuable 3 – parents will find the review of what OSA is to be valuable
Parents	2 (33.3%)	4 (66.7%)	–	–	–
Sleep experts	11 (64.7%)	6 (35.3%)	–	–	–
HCPs	10 (50.0%)	9 (45.0%)	–	1 (5.0%)	–
Valuable 4 – parents will find the information about assessing and treating OSA to be valuable
Parents	2 (33.3%)	4 (66.7%)	–	–	–
Sleep experts	10 (58.8%)	5 (29.4%)	2 (11.8%)	–	–
HCPs	11 (55.0%)	6 (30.0%)	3 (15.0%)	–	–
Valuable 5 – parents will find the information about the connection between OSA and insomnia to be helpful
Parents	4 (66.7%)	2 (33.3%)	–	–	–
Sleep experts	11 (64.7%)	5 (29.4%)	1 (5.9%)	–	–
HCPs	12 (60.0%)	8 (40.0%)	–	–	–
Credible 1 – parents will believe that the information provided in this session comes from a reputable source
Parents	3 (50.0%)	3 (50.0%)	–	–	–
Sleep experts	10 (58.8%)	2 (11.8%)	5 (29.4%)	–	–
HCPs	11 (55.0%)	8 (40.0%)	–	1 (5.0%)	–

Note. “N/A” pertains to no response resulting from a question not being administered to that participant group.

Usefulness

When asked if the session provided information that would help parents better understand their child's ongoing sleep problems following OSA treatment, all parents (6/6), 94.1% (16/17) of sleep experts, and 90% (18/20) of HCPs responded positively. When asked if this session helps parents to understand why insomnia and OSA are related, all parents (6/6), 94.1% (16/17) of sleep experts, and 90% (18/20) of HCPs responded positively. All three participant groups reported that a primary strength of this session related to the description of how OSA and insomnia can be linked. Regarding challenges, all three groups suggested reducing the length of the session to highlight the most relevant information. Sleep experts and HCPs emphasized how the session can devote less time to the assessment and symptoms of OSA, given that the program is catering to parents of children whose OSA has already been treated. Additionally, all groups suggested specific useful additions to the session such as information about parental sleep issues. Furthermore, sleep experts and HCPs stressed the importance of further elucidating the connection between OSA and insomnia. Finally, sleep experts highlighted the need for parents to ensure OSA is diagnosed correctly, given limitations to overnight polysomnography assessments (e.g., waitlists, cost), and that the session content would be increasingly helpful if opened to families with current OSA as well. They suggested including this information in the session, alongside additional information that would help parents identify residual OSA symptoms (if present).

Usable

Most parents (66.7%; 4/6), sleep experts (94.1%; 16/17), and HCPs (90%; 18/20) responded positively when asked if parents would find this session to be user-friendly. Additionally, 83.3% (5/6) of parents, all sleep experts (17/17), and 75% (15/20) of HCPs responded positively when asked if parents would be able to follow along and understand this session with ease. Finally, all parents (6/6), 70.6% (12/17) of sleep experts, and 90% (18/20) of HCPs responded positively when asked if the session would take a reasonable amount of time for parents to review. All three groups provided strengths pertaining to how the information was user-friendly, well-organized, and generally easy to navigate. However, all three groups also provided helpful feedback about how to improve the usability of the session. Challenges noted by all groups involved technical problems, some of which create barriers for most online programs (e.g., mobile accessibility, browser compatibility, and speed) and some of which may represent more idiosyncratic issues inherent in the current session (e.g., pages randomly displaying error messages). Sleep experts and HCPs also provided suggestions about the layout of the session to enhance user-friendliness (e.g., including a large ‘menu’ button).

Desirability

66.7% (4/6) of parents, all sleep experts (17/17), and 80% (16/20) of HCPs responded positively when asked if the session is visually appealing and the organization of information on the screen was clear. For strengths, all three groups emphasized how the session provided information that was eye-catching and generally visually appealing, by highlighting different characteristics of the session that were brought together to create an enjoyable user experience. In particular, most of the sleep expert feedback was positive; they focused on the general visual appeal and logical progression of the session. Challenges pertained to certain preferences noted by each group that would optimize the visual appeal and progression of the session. Sleep experts and HCPs focused on the inclusion of more videos coupled with a reduction of written text, while parents provided a contrasting view involving a preference for written material.

Accessibility

All parents (6/6), 88.2% (15/17) of sleep experts, and all HCPs (20/20) responded positively when asked if parents will think this session will be easy to follow and if they will be able to absorb all the necessary information. Furthermore, 82.4% (14/17) of sleep experts and 95% (19/20) of HCPs responded positively when asked if parents will be able to easily navigate the session and find all the relevant information. All three groups noted strengths pertaining to the session being a helpful resource for parents. Further, all groups described how the method of delivery made the session particularly engaging for families. HCPs also focused on how the session would generally be able to increase accessibility for parents with limited time or resources. However, all groups also noted accessibility-related challenges. Notably, sleep experts and HCPs suggested scaling back the reading level and medical jargon embedded throughout the session. Sleep experts also noted how language options and additional accommodations, such as subtitles on videos, would make the session more accessible. Finally, certain sleep experts and HCPs expressed concerns that the length of the session may be a barrier to parents completing the full session and absorbing the relevant information. Parents seemed to think the session was an appropriate length, however one parent expressed interest in moving quickly through the OSA session to start learning about insomnia techniques.

Features

83.3% (5/6) of parents, all sleep experts (17/17), and 90% (18/20) of HCPs responded positively when asked if parents would find the videos in this session helpful. Moreover, 83.3% (5/6) of parents, 94.1% (16/17) of sleep experts, and 85% (17/20) of HCPs responded positively when asked if parents would find the activities in the session helpful. In addition, all parents (6/6), and 94.1% (16/17) of sleep experts and 95% (19/20) of HCPs responded positively when asked if parents would find the written summary of the content helpful. Finally, 66.7% (4/6) of parents, 58.8% (10/17) of sleep experts, and 75% (15/20) of HCPs responded positively when asked if parents would find additional videos from sleep and OSA specialists to be a helpful addition to the program. Feedback primarily pertained to which features were the most preferred. This feedback was mixed, but primary strengths related to the helpful and engaging combination of features (e.g., the mix of videos and written text), and how this array of features was able to increase understanding of the material. HCPs noted how features were specifically helpful with clarifying the connection between OSA and insomnia. Consistent challenges were noted across all groups such as technical issues with features, and a general desire for the session to re-balance its ratio of audio and visual features. All groups mentioned how they would like to see more information about certain topics in videos and other features (e.g., the insomnia/OSA connection), and they also addressed areas they believed could be reduced in videos and related features (e.g., assessment and symptoms of OSA).

Valuableness

All parents (6/6), 94.1% (16/17) of sleep experts, and all HCPs (20/20) responded positively when asked if the information provided in this session would be valuable to parents. In addition, all parents (6/6), 94.1% (16/17) of sleep experts, and 85% (17/20) of HCPs responded positively when asked if the session would provide parents with a deeper understanding of why children continue to experience insomnia after OSA treatment. As well, all parents (6/6), all sleep experts (17/17), and 95% (19/20) of HCPs responded positively when asked if parents would find the review of OSA to be valuable. Similarly, all parents (6/6), 88.2% (15/17) sleep experts, and 85% (17/20) HCPs responded positively when asked if parents will find the information about assessing and treating OSA to be valuable. Finally, all parents (6/6), 94.1% (16/17) of sleep experts, and all HCPs (20/20) responded positively when asked if parents would find the connection between OSA and insomnia to be helpful.

Regarding strengths, sleep experts and HCPs stated how the session would be valuable for clinicians to share with families and children. They addressed the general need for the session by noting that it would be valuable for other stakeholders, such as teachers, who regularly interact with children and parents about sleep difficulties. Parents also found that session content was particularly valuable when key points were repeated. However, all groups noted challenges that deserve attention. HCPs and parents suggested improving the value of the session by including interviews with physicians and testimonials from parents. Finally, certain sleep experts discussed how the value of the session may be diminished due to systemic issues within the healthcare and medical system that may delay identification of insomnia and the subsequent referral to behavioural treatment for this population (even in the case of an online intervention).

Credibility

All parents (6/6), 70.6% (12/17) of sleep experts, and 95% (19/20) of HCPs responded positively when asked if parents will believe that the information provided in the session comes from a reputable source. All three groups generated credibility feedback that was primarily positive. Particular noted strengths pertained to the reputability of the information, the experience of the researchers, and the lack of bias inherent in the session. However, for challenges, sleep experts and HCPs provided helpful suggestions on how to bolster the session's credibility, such as the inclusion of information at the outset of the session, or in videos, that explicitly describes the credentials and experience of the researchers. Parents mentioned no challenges for credibility.

General readiness

Half of parents (3/6), and 70.6% (12/17) of sleep experts and 75% (15/20) of HCPs responded with “Extremely Ready” or “Very Ready” when asked how ready the session is for use with parents of children with insomnia and treated OSA. When asked if there is a satisfactory amount of information about OSA in the session, 66.7% (4/6) of parents, 94.1% (16/17) of sleep experts, and 95% (19/20) of HCPs said “Yes”. When asked if there is a satisfactory amount of information about the link between OSA and insomnia in the session, 83.3% (5/6) of parents, 88.2% (15/17) of sleep experts, and 80% (16/20) of HCPs said “Yes”. When asked if anything should generally be added to the session, 33.3% (2/6) parents, 58.8% (10/17) sleep experts, and 15% (3/20) HCPs said “No”. Similarly, when asked if anything should generally be changed in the session, 5/6 parents (83.3%) said “No”, however only 47.1% (8/17) of sleep experts and 65% (13/20) of HCPs said “No”. When asked if anything should generally be deleted from the session, 66.7% (4/6) of parents, 70.6% (12/17) of sleep experts, and 65% (15/20) of HCPs said “No”. Most of the information about the session's General Readiness was addressed in previous sections of qualitative feedback (e.g., Usefulness, Usability, and Desirability). However, some comments still pertained to the general readiness of the session for use with parents such as “Could be used as [is] and be beneficial, but opportunities for improvement are present” (HCP) and “I think it's ready to go!” (HCP). Parents, approximately half of which reported that they felt the session was ready for end users, suggested that “hearing parent testimonies would be great”.

General need

Only sleep experts and HCPs were asked about the general need for program, as this question was added due to ongoing difficulties recruiting parents. When asked if there is a general need for the program, 94.1% (16/17) of sleep experts and 85% (17/20) of HCPs said “Yes”. Prior to reviewing the session, sleep experts were also asked to estimate the percentage of children who would have issues with insomnia after surgery for OSA. On average, sleep experts estimated that 11–20% (Mode = 1–10%, Range = 0–70%) and HCPs estimated that 21–30% (Mode = 1–10%, Range = 0–80%) of children would have residual insomnia. Most of the information about the session's General Need was addressed in previous sections of qualitative feedback (e.g., Valuableness). Feedback involved comments about the beneficial value of the session for various stakeholders, combined with Expert and HCP concerns about the small size of the target population and their own lack of related experience with the population. Particularly, sleep experts and HCPs believed that the session would only be valuable for a small and specialized sample.

Discussion

Based on this usability study, the OSA-tailored session was positively received by participants and useful information for future program development was obtained. The session was generally viewed as a well-tailored version of insomnia treatment in the context of treated OSA. The vast majority of participants responded positively when asked to rate their experience with all aspects of the program. Primary strengths of the OSA-Insomnia session were reported to be the usefulness of the information, the design and features of the session, the increased accessibility, and the general value for clinicians and parents alike. Examples of qualitative feedback indicating a generally positive appraisal of the session can be found in Table S5.

Feedback from sleep experts and HCPs also supports the general need for the session. While most participants endorsed a theoretical need for this program by suggesting that many children might have unrecognized residual insomnia (94.1% of sleep experts and 85% of HCPs said “Yes” when asked if there is a general need for the program), some provided qualitative feedback which expressed concerns about the small size of this specific population. Examples of qualitative feedback that raised questions about the session's need included “I am not sure how many kids with treated OSA have insomnia – rare in my clinics” (sleep expert), and “Applies only to a select sub-group of children with OSA" (sleep expert). However, comments like these are probably informed by participants' experience working with OSA patients, which may not involve follow-up after intervention.

In addition to the positive feedback, all groups provided constructive, design-focused feedback that will inform future development of the session and improve the extent to which BNBD can be tailored for OSA. Although there were some conflicting opinions between parents and sleep experts/HCPs, primary suggestions for improvement were to simplify language, clarify more difficult concepts, provide accommodations to increase accessibility (e.g., subtitles on videos), re-balancing the ratio of visual and audio information to enhance usability, reducing information that might not be relevant for the current sample or detract from usefulness and value of the information being presented, and bolstering the session's credibility by explicitly giving credentials of the developers. Although the suggestions pertaining to the session's design and features (e.g., the balance of ration of audio and visual components) echo those seen in previous evaluations of online behavioural interventions (Mohr et al., 2013), it is difficult to have an a priori understanding of how to balance the components of an intervention to meet users' needs (Mohr et al., 2013). As such, the feedback obtained from the current study is extremely valuable for further development of the session as it will allow for a more nuanced, well-tailored approach to intervention design.

Participants also highlighted a larger conceptual challenge with the OSA-tailored session in that they indicated that more information about the nature of the relationship between OSA and insomnia could enhance the usefulness of the session. Unfortunately, clarifying updated research on the OSA-insomnia link is challenging as emerging research has shown a high comorbidity between OSA and insomnia, as well as the persistence of insomnia symptoms after OSA treatment, but the exact mechanisms of these combined and residual sleep issues are still not fully understood (Al-Jawder and BaHammam, 2012). While multiple mechanisms have been proposed (e.g., Luyster et al., 2010), further research is required to clarify these pre-existing theories. However, being aware of this issue and how it impacts end users' perception of the session is useful as future development of the session could present theories and their potential treatment implications while clearly noting that additional evidence is needed.

Participants also cited technical issues as a consistent problem, which is a common issue with online intervention programs (e.g., Donker et al., 2013). During the current study, more technical problems than would be typically experienced were likely present because this was the first use of the BNBD program on a new platform. While these technical issues may have impeded optimal user engagement with the session, feedback will be used to identify core areas where technical personnel can invest further attention (e.g., lag during videos).

Parental recruitment problems raised questions about the prevalence of the target population. Our recruitment strategy originally involved online and print advertising through medical clinics, sleep organizations, and social media. While this approach was very successful in our other sleep intervention research, it resulted in zero uptake from parents for the current study. As such, flyers were mailed directly to 500 potentially eligible families whose children had their tonsils and/or adenoids removed for OSA. This form of targeted recruitment generated some interest, but still only led to a final sample of six parents (3.2% of the targeted group participated despite all meeting initial eligibility criteria).

There are several reasons why recruitment challenges and sleep expert's concerns about the need for this program might have been encountered: (1) Insomnia symptoms following surgery may not be as prominent as emerging research suggests, as adenotonsillectomy significantly increases children's sleep efficiency and time spent in deeper sleep stages (Lee et al., 2016). (2) Insomnia symptoms are not noticed or are accepted by parents due to a relative improvement in symptoms post-surgery. Due to the symptomatic overlap between OSA and insomnia, it may be easier for parents to identify general symptom improvements rather than noticing the unique consequences of insomnia, similar to what has been seen in other instances of comorbidities (e.g., anxiety disorders; Clark et al., 2017). (3) Given the common co-occurrence of sleep problems and mental health problems or neurodevelopmental disorders, our eligibility criteria likely resulted in the exclusion of many children who could benefit from such a session (Corkum et al., 2014). (4) Finally, children with insomnia and OSA typically have a long waiting period in pre-treatment phases of OSA management and insomnia screening is not common for children who have OSA (Byars et al., 2011), making insomnia less likely to be identified as a presenting problem among children OSA.

Strengths and limitations

A primary strength of the current study was the breadth of feedback. Although stakeholder groups did not always agree on aspects of the session (e.g., preference for more videos from HCPs and sleep experts but preference for more written text from parents), observing varying perspectives underscores the importance of garnering feedback from a variety of stakeholders and highlights the importance of conducting usability studies to obtain an understanding of the differing perspectives of end users and content experts. Another strength of the current study was the relatively robust sample size (despite significant recruitment challenges). Research suggests that about 85% of unique feedback/problems inherent in usability studies can be detected with only 5 participants (Virzi, 1992), suggesting that our sample of 43 participants is sufficient for obtaining useful insights into the usability of the session.

There were also several notable limitations to the current study. Firstly, given the conflicting information provided by emerging research, attempted recruitment, and study feedback, the true need for this session in its current form is still difficult to determine. However, this is important to know prior to potentially investing resources and recruiting new participants for an RCT. Additionally, sleep experts and HCPs endorsed a wide-range of clinical expertise and some mentioned that they were only able to provide a rough estimate of their experience working with sleep disorders. As such, it is possible that any results differentiating sleep experts and HCPs may be ambiguous due to significant overlap in scope of practice, resulting in a more consistency in perspectives.

Clinical implications and future research directions

While the results of this usability study are generally positive, there remains some question as to the need for this intervention given stakeholder feedback and recruitment challenges. Furthermore, the results from this study raise further questions about the prevalence and mechanisms of insomnia in the context of OSA. Additionally, although tailored interventions may provide a nuanced approach to a presenting problem, eHealth interventions like BNBD are already highly personalized and may not require additional tailoring to meet the needs of a target population.

Further research is needed to understand the degree to which insomnia may be under-recognized in children with past or current OSA and the extent to which parents and HCPs are aware of behavioural insomnia in the context of OSA. These estimates will help to inform future testing of behavioural insomnia interventions at various stages of OSA management, providing crucial information about the effectiveness and appropriate timing of sleep treatments among children facing multiple sleep disorders, such as whether the tailored session should be provided to patients immediately after surgery, after post-surgery follow-up, or to any patients with OSA or insomnia who may find it helpful. Future research should also aim to survey the degree to which insomnia is screened and treated among children with OSA, as preliminary research has suggested that OSA may overshadow behavioural sleep problems in clinical practice. Finally, future research should aim to better elucidate the prevalence of comorbid OSA and insomnia in children, as well as the prevalence of residual insomnia after OSA has been treated.

Conclusions

Insomnia is common among children with OSA (Byars et al., 2011). Even after surgical treatment for OSA, it is estimated that >50% of treated children may continue to suffer from insomnia (Chervin et al., 2014), underscoring the importance of an accessible insomnia intervention that contextualizes ongoing sleep problems within the context of past issues with OSA. Based on the results of this usability study of the OSA-tailored session, participant feedback generally suggested that the session would be valued and usable. Nonetheless, other feedback combined with parental recruitment difficulties created questions about the wide spread applicability of this session.

The following are the supplementary data related to this article.

Table S5

Examples of commonly occurring qualitative feedback for each group.

Declarations of interest: If the Better Nights, Better Days for OSA intervention proves to be effective after further testing, we may move towards commercialization.

Funding

This work was supported by the Canadian Sleep and Circadian Network [grant number CDT-142656]; and the Nova Scotia Health Research Foundation [grant number PSO-REDI-2016-867]; and Fondation Les Petits Trésors de l'Hôpital Rivière-des-Prairies [grant number 9100425].

Declaration of competing interest

We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.

We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us.

We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing we confirm that we have followed the regulations of our institutions concerning intellectual property.

We understand that the Corresponding Author is the sole contact for the Editorial process (including Editorial Manager and direct communications with the office). She is responsible for communicating with the other authors about progress, submissions of revisions and final approval of proofs. We confirm that we have provided a current, correct email address which is accessible by the Corresponding Author and which has been configured to accept email from Internet Interventions.