Prevention and early intervention of anxiety problems in young children: A pilot evaluation of Cool Little Kids Online.

Anxiety disorders are common, debilitating, and begin early in life. Early intervention to prevent anxiety disorders in children who are at risk could have long-term impact. The 'Cool Little Kids' parenting group program has previously been shown to be efficacious in preventing anxiety disorders in temperamentally inhibited young children. Wider dissemination of the program could be achieved with an internet-based delivery platform, affording greater accessibility and convenience for parents. The aim of this study was to evaluate 'Cool Little Kids Online', a newly developed online version of the existing parenting group program. Fifty-one parents of children aged 3-6 years were recruited to evaluate the online program's acceptability and preliminary efficacy in reducing inhibited young children's anxiety problems. Parents were randomized to receive either a clinician-supported version or an unsupported version of the program. Parents had 10 weeks to access the program and completed questionnaires at baseline and post-intervention. Both groups showed medium-to-large reductions in children's anxiety symptoms, emotional symptoms, number of child anxiety diagnoses, and improvements in life interference from anxiety. The effect of clinician support was inconsistent and difficult to interpret. Parents reported high levels of satisfaction with the program. These encouraging results indicate that the online version is acceptable and useful for parents with temperamentally inhibited young children. Cool Little Kids Online may be a promising direction for improving access to an evidence-based prevention and early intervention program for child anxiety problems. A large randomized trial is warranted to further evaluate efficacy.

RCT Entities:   Population Interventions Outcomes

Introduction

Anxiety is the most common mental health disorder in children and adolescents, with lifetime prevalence rates of 30% by age 18, and median age of onset of 6 years (Merikangas et al., 2010). Anxiety in childhood is a strong predictor of anxiety disorders during adolescence and early adulthood, as well as secondary problems such as depression (Rapee et al., 2009). There is also considerable stability in early childhood, as 50% of three year olds with an anxiety disorder also experience anxiety at six years of age (Bufferd et al., 2012). Childhood anxiety disorders cause substantial impairment in peer and family relationships and academic achievement (Drake and Ginsburg, 2012). Given anxiety's early onset and association with significant impairment, there is a clear rationale to intervene early with children at risk. Anxiety prevention programs for children and adolescents can be effective, especially when programs are targeted towards children at greater risk (Teubert and Pinquart, 2011).

Both genetic and environmental risk factors for anxiety disorders in children have been identified (Drake and Ginsburg, 2012). A clear risk factor in early childhood is temperamental inhibition, which has a strong genetic basis and refers to fearfulness and withdrawal in response to novel stimuli (Fox et al., 2005). Parenting practices also have an important influence and include overinvolved or overprotective parenting and harsh or negative interactions (Drake and Ginsburg, 2012). An inhibited temperament is thought to interact with overprotective parenting practices to contribute to the development of child anxiety disorders (Rubin et al., 2009). This occurs when a child's inhibited temperament elicits parental overprotection from anxiety-provoking situations, which inadvertently reinforces child avoidance and fear, and reduces child mastery and confidence in dealing with new situations.

The ‘Cool Little Kids’ parenting group program aims to intervene early in life to protect children at known risk for anxiety disorders during development (Rapee et al., 2005). The program targets preschool-aged children with high levels of temperamental inhibition. It is delivered to small groups of parents and addresses key factors in the development of anxiety problems in young children, such as overprotective parenting, children's avoidant coping styles and parental anxiety. Two randomized controlled trials (RCTs) have demonstrated its efficacy (Rapee, RM, et al., 2005, Kennedy, SJ, et al., 2009). In one study, significantly fewer anxiety diagnoses were detected among intervention than control children three years later (40% versus 69%) (Rapee et al., 2010), with some effects lasting into adolescence (Rapee, 2013). Anxiety symptoms also reduced significantly, with medium-sized between-group effects observed on child-report anxiety (d = 0.50) and maternal-report anxiety (d = 0.46) at the 3-year follow-up. A second study with a higher-risk sample and slightly more intensive intervention also found significantly fewer anxiety disorders in intervention children compared to controls at the 6-month follow-up (53% versus 93%) (Kennedy et al., 2009). These impressive results suggest that intervening early with a brief program could produce lasting mental health change in children.

A key issue for prevention programs is whether they can be sustainably delivered in the community. Programs should be easily accessible by participants to maximize uptake and retention, and be cost-effective in order to secure continual funding (Andrews and Erskine, 2001). An online version of the Cool Little Kids parenting group program was developed to widen and facilitate access for parents to the program's content. As 93% of Australian families with children have internet access at home (Australian Bureau of Statistics, 2011), the ‘Cool Little Kids Online' program could afford greater accessibility and convenience for parents who find it difficult to travel from their homes to attend parenting groups for a variety of reasons (e.g., childcare arrangements and physical health/mobility issues). It could also be a helpful resource for parents who live in areas with limited access to mental health services. Whilst early intervention parenting programs for child externalizing problems have been successfully adapted to online formats (e.g., Sanders et al., 2012), the equivalent online parenting programs for internalizing problems in young children are still rare.

The present study is a pilot evaluation of the Cool Little Kids Online parenting program, which aimed to explore its acceptability to parents and potential efficacy in reducing inhibited young children's anxiety problems. The study also tested two potential program formats (supported and unsupported) and the feasibility of online data collection. It was hypothesized that the program would be rated as useful by parents and associated with significant reductions in child anxiety and emotional symptoms, and improvements in life interference from anxiety.

Method

Design

This pilot study was an uncontrolled pre-post trial with random allocation to two active intervention arms: clinician support versus no clinician support. At the end of the baseline questionnaire a computer script randomly allocated parents to study arms in a 1:1 allocation (simple randomization). Participants were given 10 weeks to access the Cool Little Kids Online program, after which they were invited to complete the post-questionnaire online. The study was approved by the La Trobe University Human Ethics Committee (14-021) and was registered with the Australian and New Zealand Clinical Trials Registry (ACTRN12614000659606).

Participants and recruitment

Participants were eligible to take part in the study if they were a parent of a child between the ages of 3 and 6 years who was highly inhibited (see Section 2.4.1). Participants were excluded if they were not a resident of Australia or they reported that their child had cerebral palsy, an intellectual disability, or severe autism. Participants without access to a printer were also excluded, because we believed that program benefits would only be possible if participants could print out the program worksheets that are used in-between modules.

Participants were recruited from preschool services and advertisements on Facebook, Google, and parenting forums. Preschools within three local government areas in the state of Victoria, Australia (selected for a mix of socioeconomic disadvantage level) were asked to help distribute a study flyer to parents of children aged 3 to 6 years at their service. Interested participants visited the study website (www.coollittlekids.org.au) for more information about the program and how to join the study. Parents were screened online for eligibility, and if eligible were invited to participate. Parents who gave informed consent continued with the baseline questionnaire. Parents were able to access the Cool Little Kids Online program immediately after completing the baseline questionnaire by creating their own account with a unique username and password.

We aimed to recruit 40 participants (20 per condition) as this would reliably indicate the acceptability of the program and to what degree it is used by participants. This would also give adequate power to detect an effect size of d = 0.5 pre- to post-intervention on anxiety symptoms. Prior studies of group-based programs for parents of anxious young children show within-group effects of this size or greater over a similar time period (e.g., van der Sluis et al., 2012). Fig. 1 shows the flow of participants through the study. During the two month recruitment period, 171 parents were screened for eligibility, with 75 deemed ineligible, primarily due to low child inhibition scores. Eighty-five parents gave consent to participate (88.5%), but only 51 parents completed the baseline questionnaire and hence were fully enrolled in the study (20 randomly allocated to the clinician support condition and 31 to the unsupported condition). Participant characteristics were generally well balanced across the two groups (see Table 1).

Fig. 1

Flow of participants through the study.

Table 1

Baseline participant characteristics in the clinician supported and unsupported conditions.

	Clinician supported(n = 20)	Unsupported(n = 31)
Child age M (SD)	4.8 (0.8)	4.7 (0.8)
Child gender (female) (%)	50.0	48.4
Child inhibition M (SD)	35.9 (2.7)	34.9 (3.2)
One or more anxiety disorders (%)	85.0	83.9
Parent age M (SD)	35.7 (4.7)	36.2 (3.7)
Parent is child's birth mother (%)	90.0	96.8
Child lives with both birth parents (%)	90.0	93.5
Parent highest level of education (%)
Postgraduate degree	30.0	22.6
Tertiary degree	25.0	38.7
Technical diploma/certificate	40.0	19.4
High school	5.0	16.1
Other	0.0	3.2
Speaks mainly English at home (%)	95.0	100.0
Household income (%)
AUD 75,000 or more	70.0	64.5
Less than AUD 75,000	10.0	22.6
Don't know/missing	20.0	12.9
Financial difficulty (‘Health care card’) (%)	15.0	12.9
Weekly internet use (%)
< 10 h	40.0	51.6
10 h or more	60.0	48.4
Moderate or above on the Depression, Anxiety or Stress Scales (%)	35.0	29.0

Note. AUD = Australian dollars.

The mean age of parents in the sample was 36.0 years (SD = 4.7) and the majority were birth mothers (94.1%). The target children ranged in age from 3.5 to 6.8 (M = 4.7, SD = 0.8), 26 were boys and 25 were girls. The majority (92.2%) lived with both parents and four lived with their birth mother only. This is slightly higher than the Australian average where 84.9% of children aged 0 to 9 years live in two-parent families (Australian Bureau of Statistics, 2011). Of the 51 parents, 2% had a grade 10 education, 12% had a year 12 qualification, 27% had a technical diploma or certificate, 33% had a tertiary degree, and 25% had a postgraduate degree. The majority (98%) spoke mainly English at home. Twenty-one percent reported a household income less than the Australian median of AUD $75,000 (Australian Bureau of Statistics, 2013). A significant minority (14%) reported possessing a ‘Health Care Card’, indicating financial difficulty. Sixteen parents (31%) scored moderate or above on any subscale of the Depression Anxiety Stress Scales-21 (Lovibond and Lovibond, 1995). More than half (52.9%) used the internet for 10 or more hours a week.

Program description and development

Cool Little Kids Online contains the same anxiety prevention content as in the 6-session Cool Little Kids parenting group program developed by Rapee et al. (2005), but reorganized to suit an online format. There are 8 interactive online modules that contain a mix of written information, videos, audio narration, interactive worksheets and activities, and parent experiential stories (see Table 2 for a content overview of each module). Development of the online format was informed by formative research with clients of the Emotional Health Clinic, Macquarie University and feedback from participants in the population trial of Cool Little Kids conducted in Melbourne, Australia (Bayer et al., 2011). Online program development was also guided by research on persuasive design elements that maximize adherence (Fogg, BJ, 2003, Oinas-Kukkonen, H and Harjumaa, M, 2009) and the therapeutic alliance in internet-based interventions (Barazzone et al., 2012).

Table 2

Overview of Module Content in Cool Little Kids Online.

Module number and title	Content overview
1. Understanding anxiety	Teaches the nature of child anxiety and its development and the role of temperament. Gives an overview of program content and parents set goals for what they want to achieve.
2. Introducing stepladders	Teaches the principles and application of exposure hierarchies (stepladders).
3. Using rewards	Teaches the principles of using rewards effectively to reinforce child behavior.
4. Parenting an anxious child	Teaches the role of overprotection in child anxiety and alternative parental strategies including encouraging greater child independence.
5. Troubleshooting stepladders	Review of stepladder progress and troubleshooting difficulties that commonly occur.
6. Overcoming barriers	Review of stepladder progress and how to overcome barriers to stepladder practice. Introduction to cognitive restructuring for parents' own worries.
7. Managing your worries	Teaches cognitive restructuring for parent worries in more depth, particularly in the context of implementing exposure with their child. Review of stepladders.
8. Planning for the future	Review of progress and planning of strategies to use for future challenges or high-risk times such as starting school.

Formative research indicated that one of the most helpful aspects of the Cool Little Kids program reported by parents was hearing the experiences of other parents during their group sessions. To reproduce this in an online setting, the program included stories and examples from six real parents who had taken part in the group parenting program. In addition, two example families with an inhibited child were followed throughout the online program as they learned the program skills. These families were fictitious but faced similar issues to participating parents in order to maximize the similarity principle of persuasive system design (Oinas-Kukkonen and Harjumaa, 2009). These families' stories provided an opportunity to normalize setbacks or difficulties and show examples of completed worksheets.

Program material incorporated the principles of good e-learning instruction where possible (Clark, RC and Mayer, RE, 2008, Dirksen, J., 2012, Allen, MW, 2007). For example, techniques that are more difficult to learn, such as developing an exposure hierarchy and realistic thinking, were taught with several worked examples that faded out the amount of help provided to finish the task. For teaching exposure hierarchies, parents worked through identifying what was wrong with example hierarchies, to completing a partially-finished hierarchy, to finally developing their own from scratch. Difficult concepts were presented using a mix of media, and modules combined new content with a review and practice of previous skills. Text was written in short, simple sentences in an easy-to-read informal conversational style. The text from all modules had a Flesch-Kincaid Grade Level of 7.3, indicating suitability for individuals with a seventh or eighth grade education. The first two modules underwent usability testing with five parents of young children using the ‘think aloud’ protocol (Krug, 2010) to improve the navigation and content presentation.

Participants could access all modules upon registration (open access) but were encouraged to complete one module per week. Modules took approximately 30 to 60 min to complete, and included 25 web pages on average. In between modules, parents were encouraged to undertake practice tasks with their child, as occurs in the group parenting program. Parents were given the opportunity to self-monitor their progress with an online diary, and they could also view a chart of their child's fear symptoms over time, which they were prompted to rate weekly. These features are based on the persuasive system design model, which proposes that systems that incorporate self-monitoring of performance or status can support behavior change (Oinas-Kukkonen and Harjumaa, 2009). Parents received automated emails after completing each module, halfway through the study and one week before their program access ended. Participants also received an email reminder after 2 weeks of website inactivity, and then a further phone call after another week had elapsed to check whether they were having technical problems and to encourage them to log in.

Clinician support

Online programs for mental health problems in adults can be more effective when enhanced by guidance from a professional (Baumeister et al., 2014), although good effects are also possible without such support (e.g., Titov et al., 2009). Within online parenting programs specifically, beneficial outcomes have been achieved with no professional support, email consultations, and intensive telephone or videoconferencing support (Nieuwboer et al., 2013). Programs with professional support face the trade-off of requiring greater resources to deliver, which limits wide dissemination. This consideration is especially important in preventive programs which are typically designed to reach large numbers of people. To explore this issue in the context of the Cool Little Kids Online program, parents were randomly allocated to either a supported or unsupported program format. Participants in the supported format received telephone support from a clinician who had prior experience delivering the Cool Little Kids parenting group program and was undertaking an accredited psychology postgraduate degree. Based on our experience delivering the group program, and in order to test a model of clinician support that would be cost-effective to deliver, parents were phoned at two key points during the program when it was anticipated they would most need additional support. Up to two messages were left to facilitate each call if the parent could not be reached during their preferred contact time. Semi-structured scripts for the two calls were developed to ensure a consistent approach, and the clinician was able to examine parents' online worksheets and their progress through the program. During the calls the clinician reinforced program material, helped troubleshoot difficulties the parents were having and encouraged them to practice the program skills.

Measures

Short temperament scale for children (STSC) – Approach subscale (Prior et al., 1989)

The Approach subscale of the STSC consists of 7 items and is used to screen for temperamental inhibition in children aged 3 to 8 years. It has been validated in a large representative sample of children and has good internal consistency (α = 0.84). A score over 30 (> = 85th percentile) was the cut-point for study eligibility, which was consistent with meeting criteria for behavioral inhibition on laboratory assessment in previous research (Rapee et al., 2005).

The revised preschool anxiety scale (PAS-R) (Edwards et al., 2010)

The PAS-R assesses parent-reported anxiety symptoms in young children across four subscales: generalized anxiety, social phobia, separation anxiety, and specific phobias. The total score and subscales have good internal consistency (α = 0.72–0.92), correlate moderately with observer ratings of child anxiety, and are predictive of DSM-IV anxiety diagnoses assessed by semi structured interview. Cronbach's α in the present study were also acceptable: PAS-R total α = 0.86, generalized anxiety subscale α = 0.77, social phobia subscale α = 0.79, separation anxiety subscale α = 0.71, specific phobia subscale α = 0.80. The total score ranges from 0 to 112 and can differentiate between children with an anxiety disorder (M = 61) and without (M = 38) (Edwards et al., 2010).

Online assessment of preschool anxiety (OAPA)

This newly developed measure assesses the presence or absence of anxiety diagnoses in preschool aged children. It is adapted from the Online Assessment of Anxiety - Parent version (unpublished) developed by Dr. Lyneham and colleagues from Macquarie University. Parents complete the OAPA online and are asked screening questions for each child anxiety disorder (separation anxiety disorder, social phobia, generalized anxiety disorder, specific phobia). Automated rules determine whether the rest of the questions for that section are presented. Parents rate child anxiety symptoms and level of interference in closed questions, and also provide written descriptions of their child's behaviors and thoughts related to each anxiety problem and examples of life interference from their child's anxiety symptoms. Responses are automatically scored for the presence or absence of a disorder based on DSM-IV criteria. These provisional diagnoses are then reviewed by a psychologist to check whether parents' written descriptions are consistent with the disorder being assessed and whether the level of impairment described is clinically sufficient to warrant a diagnosis. As this new measure has not yet been validated, the present study was an initial test of its utility and feasibility in assessing anxiety disorders in young children.

Strengths and difficulties questionnaire-emotional symptoms subscale (SDQ-ES) (Goodman, 1997)

The SDQ is a widely used screening tool for psychosocial problems in children. The Emotional Symptoms subscale (SDQ-ES) was used in this study to measure internalizing symptoms in young children. The parent report version for children aged 4 to 10 years has five items and scores range between 0 and 10. It correlates highly with other measures of internalizing symptoms (rs 0.67–0.73) and can discriminate between children with and without psychosocial diagnoses (Stone et al., 2010). Cronbach's α in the current study was lower than other studies, α = 0.57.

Children's anxiety life interference scale – preschool version (CALIS-PV)

The CALIS assesses the impact of children's anxiety symptoms on their own life and their family's functioning. Its psychometric properties are sound with support found for its factor structure, reliability, and convergent and divergent validity (Lyneham et al., 2013). The CALIS-PV was adapted by Kennedy et al. (2009) for use with younger, preschool age children. It is a 20-item parent-report questionnaire with two subscales: child life interference from anxiety and family interference due to child anxiety. The CALIS-PV's total score has excellent internal consistency (α = 0.94) and is sensitive to change with anxiety treatment (Kennedy et al., 2009). Cronbach's α for this study was 0.82 for the child interference subscale and 0.90 for the family interference subscale.

Over-involved/protective parenting scale (OI/P) (Bayer et al., 2006)

The OI/P is an 8-item measure of overinvolved/protective parenting behaviors that discourage autonomy in young children (e.g. ‘I prevent my child getting involved in activities or tasks that he/she finds too difficult and may fail at’). Items are rated on a 4-point response scale and refer to specific behaviors rather than broad parenting statements to minimize social desirability bias. A 5-point version of the scale had good internal consistency (α = 0.81) and was significantly associated with child internalizing symptoms (r = 0.40) (Bayer et al., 2009).

Depression anxiety stress scales-21 (DASS-21) (Lovibond and Lovibond, 1995)

The DASS-21 measures psychological distress in adults and distinguishes between three separate factors: depression, anxiety (fear, panic), and stress (tension, agitation). It has excellent psychometric properties, and scores on the DASS-21 can be doubled to achieve comparable scores on the full-length DASS (Lovibond and Lovibond, 1995). Cut-points have been developed for mild/moderate/severe/extremely severe for each scale. The DASS-21 assessed parental mental health at baseline.

Program satisfaction and feedback

A mix of open-ended questions and Likert-scale questions were used to measure participant satisfaction and program feedback. Five questions assessed the usefulness of the program for learning about anxiety and how to manage it in their child, measured on a 5-point scale: not at all, a little, quite, very and extremely useful. Participants also rated website ease of use on a 5-point scale: easy, somewhat easy, neutral, somewhat hard, hard. The length of each module was assessed as Too short, About right, Too long, or Not sure/Didn't use.

Statistical analyses

Primary and secondary child mental health outcomes were evaluated using mixed models for repeated measures with a compound symmetry covariance structure. The mixed-models approach is consistent with intention-to-treat analytic approaches under the assumption that data is missing at random (Salim et al., 2008). Any participants who were randomized but withdrew from the study, or did not complete the intervention, were included in these analyses as randomized. Where the distribution of residuals showed substantial non-normality, the analyses were re-run on transformed data. Where the pattern of results did not change, the untransformed data were reported for ease of interpretation. For continuous measures, effect sizes (Cohen's d) were calculated for within-group effects based on the observed means and pooled standard deviation. Confidence intervals for d were calculated using Exploratory Software for Confidence Intervals (Cumming, 2013). For categorical variables, phi (ϕ) was used as a measure of the strength of association, and effect sizes can be interpreted as 0.1 = small, 0.3 = medium, 0.5 = large. In participants with baseline and post-intervention data, the amount of change on the PAS-R considered to be statistically reliable was calculated according to the procedures in Jacobson and Truax (1991). Observed data from this study were used in the calculation (Cronbach's α = 0.86 and SD = 16.1), making a change of 16.7 points up or down unlikely to have occurred by chance. The analysis of program usage was conducted on participants who did not withdraw, with independent-sample t-tests calculated for differences in the amount of usage between groups, and Fisher's exact test calculated for categorical variables. All analyses were conducted using SPSS 22.

Results

Program usage and clinician support calls

Website server logs showed that the mean number of modules accessed was 4.3 (SD = 3.2), with 12.8% not accessing any modules. Module 1 was used the most, by 87.2% of participants. The mean number of website logins over the 10-week evaluation was 5.9 (SD = 4.1), according to website server logs. Thirty-seven participants provided data post-intervention on program satisfaction or mental health outcomes (37/51, 72.5%). Of the 36 participants who reported on the number of modules completed at post-intervention, 38.9% completed all eight modules (M = 5.1, SD = 3.0). Self-reported module completion corresponded highly with website server logs of module use, r = 0.98, p < 0.001. The most common reason for not completing all eight modules was lack of time (95.2%), followed by child improved and no longer needed help (47.7%), parent sought help for their child from a professional instead (19.1%), parent experienced website technical problems (19.0%), and the program wasn't helping (14.3%).

For the 20 parents randomized to the supported arm, one parent withdrew before receiving the first call, and 12 parents were successfully contacted. A further three parents withdrew before their scheduled second support call, and 10 out of the remaining 16 parents successfully received the second support call. Excluding the four parents who withdrew, 81% received at least one support call and 44% received two calls. Calls averaged 18 min in length. Support calls were rated Very helpful by 64% and A little helpful by 36% of parents in the post-intervention questionnaire.

There was no clear pattern of difference in program usage between parents in the clinician supported and unsupported arms. There were fewer logins in the unsupported group (M = 5.2, SD = 3.8) than the supported group (M = 7.2, SD = 4.4), and the difference was medium in size, though not statistically significant, t(45) = − 1.62, p = 0.113, d = 0.51. The number of self-reported modules completed by parents in the unsupported arm (M = 5.2, SD = 3.2) was not significantly different from parents in the supported arm (M = 4.8, SD = 2.9), t(34) = 0.42, p = 0.676, d = 0.15. All eight modules were completed by 9/22 (40.9%) parents in the unsupported arm, and 5/14 (35.7%) parents in the supported arm, χ2 (1) = 0.097, p = 1.00, ϕ = 0.05.

Child mental health outcomes

Primary outcomes

Results for child mental health outcomes are presented in Table 3. The two groups were similar at baseline on most outcome measures, except for the PAS-R where the unsupported group had a somewhat higher mean total score than the supported group. There was a significant improvement over time on child anxiety symptoms measured with the PAS-R total score, F(1.34.7) = 22.7, p < 0.001, and all PAS-R subscales (Generalized Anxiety: F(1.37.2) = 13.6, p = 0.001; Social Phobia: F(1.36.1) = 13.8, p = 0.001; Separation Anxiety: F(1.34.2) = 20.5, p < 0.001; Specific Phobia: F(1.30.8) = 16.1, p < 0.001). Within-group effect sizes were generally medium in size (ds 0.4 to 0.8).

Table 3

Observed Means, Standard Deviations, and Effect Sizes for All Outcome Measures at Baseline and Post-Intervention.

Outcome measure	Group	Baseline M (SD)	PostM (SD)	Cohen's d (95% CI)
PAS-R Total Score	Total	58.0 (16.3)	46.0 (18.4)	0.69 (0.35 to 1.02)
	Unsupported	61.4 (15.8)	49.0 (19.9)	0.78 (0.32 to 1.22)
	Supported	55.9 (16.3)	40.3 (14.2)	0.59 (0.07 to 1.09)
Generalized Anxiety	Total	16.5 (5.5)	13.5 (5.9)	0.53 (0.19 to 0.85)
	Unsupported	16.9 (5.5)	14.5 (6.1)	0.52 (0.08 to 0.94)
	Supported	16.3 (4.9)	11.5 (5.3)	0.57 (0.03 to 1.09)
Social Phobia	Total	18.1 (4.8)	14.9 (5.7)	0.61 (0.22 to 0.99)
	Unsupported	19.5 (5.2)	14.8 (6.3)	0.69 (0.21 to 1.15)
	Supported	18.8 (5.3)	15.1 (4.6)	0.42 (0.24 to 1.07)
Separation Anxiety	Total	10.2 (4.5)	7.4 (4.6)	0.60 (0.32 to 0.88)
	Unsupported	10.3 (4.4)	7.6 (5.1)	0.69 (0.33 to 1.04)
	Supported	9.3 (3.9)	7.1 (3.9)	0.41 (0.04 to 0.84)
Specific Phobia	Total	13.2 (7.7)	10.2 (7.4)	0.40 (0.19 to 0.60)
	Unsupported	14.7 (8.7)	12.1 (8.0)	0.43 (0.15 to 0.71)
	Supported	11.6 (6.5)	6.6 (4.5)	0.45 (0.14 to 0.75)
OAPA No. of anxiety disorders	Total	2.1 (1.7)	1.1 (1.3)	0.64 (0.28 to 0.99)
	Unsupported	2.0 (1.8)	1.2 (1.2)	0.74 (0.27 to 1.19)
	Supported	2.2 (1.9)	1.0 (1.5)	0.47 (− 0.12 to 1.03)
SDQ-ES	Total	5.2 (2.2)	3.3 (2.1)	0.86 (0.45 to 1.25)
	Unsupported	5.5 (2.1)	3.6 (2.2)	0.93 (0.37 to 1.47)
	Supported	5.2 (2.2)	2.8 (1.9)	0.76 (0.21 to 1.28)
CALIS-PV Child	Total	22.6 (6.8)	16.3 (7.2)	0.90 (0.43 to 1.36)
	Unsupported	23.2 (7.8)	16.2 (8.1)	0.90 (0.33 to 1.46)
	Supported	22.7 (7.9)	16.5 (5.6)	0.85 (0.02 to 1.65)
CALIS-PV Family	Total	13.9 (7.9)	8.1 (6.7)	0.79 (0.34 to 1.23)
	Unsupported	14.3 (8.0)	8.7 (7.0)	0.84 (0.23 to 1.43)
	Supported	14.8 (9.6)	6.9 (6.4)	0.69 (0.03 to 1.31)
OI/P	Total	1.6 (0.3)	1.5 (0.3)	0.20 (− 0.18 to 0.57)
	Unsupported	1.6 (0.3)	1.6 (0.4)	0.09 (− 0.32 to 0.49)
	Supported	1.5 (0.4)	1.4 (0.1)	0.43 (− 0.43 to 1.28)

Note: At baseline N = 51 and post-intervention N = 32.

The group by time interaction was not significant for the PAS-R total score or any PAS-R subscale, suggesting that the clinician supported and unsupported groups experienced a similar amount of improvement. PAS-R total score: F(1.34.7) = 0.12, p = 0.730; Generalized Anxiety: F(1.37.2) = 0.40, p = 0.533; Social Phobia: F(1.36.1) = 0.59, p = 0.446; Separation Anxiety: F(1.34.2) = 1.4, p = 0.241; Specific Phobia: F(1.30.8) = 0.32, p = 0.579.

Out of 32 participants with data at both time points, 22 (68.8%) showed a reliable improvement on the PAS-R total score, nine (28.1%) showed no reliable change, and one (3.1%) showed a reliable deterioration. A greater proportion had reliably improved in the supported group (81.8%) than the unsupported group (61.9%), although this difference was not statistically significant, χ2 (1) = 1.3, p = 0.425, ϕ = 0.20.

For diagnostic outcomes measured on the OAPA, there was a reduction in the mean number of anxiety disorders per child over time, F(1.33.0) = 13.5, p = 0.001. The group by time interaction was not significant, F(1.33.0) = 0.001, p = 0.980. In participants with data at both time-points, 87.5% of children had an anxiety disorder at baseline, reducing to 56.3% at post-intervention, χ2 (1) = 5.88, p = 0.028, ϕ = 0.43. Rates of anxiety disorders in the two groups at baseline were 95.2% of the unsupported group and 72.7% of the supported group. At post-intervention, the rates were 66.7% for the unsupported group (a decrease of 30%) and 36.4% for the supported group (a decrease of 50%).

Secondary outcomes

Significant improvements over time were found for the SDQ-ES, F(1.37.1) = 26.3, p < 0.001; CALIS-PV Child Interference, F(1.34.8) = 18.0, p < 0.001, and CALIS-PV Family Interference, F(1.33.9) = 16.5, p < 0.001, with medium-to-large effect sizes (ds 0.7 to 0.9). Change in overinvolved/protective parenting (OI/P) from baseline to post-intervention was not significant, F(1.37.6) = 1.8, p = 0.190, and showed only a small effect size.

The difference between the clinician supported and unsupported groups in the amount of change over time was not significant for any of the secondary outcomes, and the pattern of effect did not consistently favor either group. SDQ-ES: F(1.37.1) = 0.001, p = 0.970; CALIS-PV Child Interference: F(1.34.8) = 0.09, p = 0.770; CALIS-PV Family Interference: F(1.33.9) = 0.16, p = 0.692; and OI/P: F(1.37.6) = 0.45, p = 0.508.

Intervention feedback

Usefulness and satisfaction

Most parents reported positive feedback on the usefulness of the program. Eighty-six percent rated the program as ‘quite’ to ‘extremely’ useful for understanding young children's shyness, inhibition and anxiety; 86% for knowing what leads to anxiety developing in young children; 84% for knowing how to encourage brave behavior in their child; 86% for knowing how to reduce anxious behavior in their child; and 73% for knowing how to change their own anxious and fearful thoughts. Most parents (89.2%) reported that they would probably or definitely recommend the program to others.

Ease of use

Website ease of use was satisfactory, with 86.2% of parents rating it as easy or somewhat easy to use (5.6% rated it hard or somewhat hard). The length of the eight modules was also appropriate, with all judged ‘about right’ by > 80% of parents who used them.

Best and worst aspects

Parents were asked to write open-ended feedback on what they thought was the best and worst aspects of the program and how it could be improved. These responses were content analyzed into themes. Parents thought the best things about the program were its flexible access and self-paced nature (36.3%), learning effective strategies to manage child anxiety (30.3%), great examples (18.2%), understanding shy/anxious behavior (15.2%), easy to understand (12.1%), and understanding parent behavior (6.1%). The worst things reported were finding time to use the program (30.3%), the short access period (27.3%), it was somewhat repetitive (12.1%), lack of contact with people (9.1%), and computer or technical issues (9.1%). Suggestions for improvements followed on from these, as the most common recommendation was extra time to complete the program (34.5%). Other suggestions were shortening the program (17.2%), including more reminders to use it (6.9%), and improvements to viewing on smaller devices (6.9%).

Discussion

The aim of this study was to explore the acceptability and potential efficacy of a newly developed online version of the Cool Little Kids parenting program. Results from this pilot showed clear improvements in children's mental health with generally medium-to-large effect sizes. Parents also provided positive feedback on the program, with most reporting it as useful, user-friendly, and that they would recommend it to others. These results are similar to the parent feedback on the group version of this parenting program delivered in the community (Beatson et al., 2014). Notwithstanding these positive findings, the amount of program use by parents was lower than anticipated, with a significant minority not completing any modules, and less than half completing the program within the study period. Motivating parents to engage in parenting programs is a significant issue more broadly (Axford et al., 2012), and some programs have explicitly incorporated motivational interviewing components to elicit parent motivation (Gill et al., 2008). For programs focused on prevention and early intervention of child anxiety problems, it is possible that some parents may lack motivation if they do not perceive their child as overly impaired. However, the most common reason given for not completing the program was lack of time, and a longer access period was the most frequent recommendation. The 10-week access period was chosen as it was analogous to the length of the group program, but feedback from this pilot study suggests this is not long enough for online delivery.

Whilst program use could be targeted for improvement, a common reason why parents didn't complete the program was that their child had improved. This could indicate that not all parents require the full program to receive benefit. In the group program, effects were demonstrated despite only 72% of mothers attending at least 5 of the 6 sessions (Rapee et al., 2005). Research on internet interventions also suggests that a linear relationship between program use and outcome may be too simplistic, and that module completion is a poor indicator of outcome (Donkin et al., 2013). Alternatively, because children were selected on the basis of high inhibition, some may have improved naturally with time regardless of their parents' program use. Effect sizes within control groups of other preschool anxiety RCTs indicate small improvements over 10 weeks (d = 0.30, Donovan and March, 2014), small to medium over 6 months (d = 0.27 and d = 0.51, Kennedy et al., 2009), and medium over 12 months (d = 0.62, Rapee et al., 2010). Although the effect sizes observed in this 10-week study (d = 0.69) tentatively suggest that the program conferred additional benefit, it will be necessary to conduct a controlled trial. An RCT with a longer follow-up would clarify whether changes in child anxiety are related to the intervention rather than natural improvement and whether any improvements persist.

Clinician supported and unsupported versions of the program were compared to explore the feasibility of providing clinician support and the effect of this support on adherence and outcomes. Effects were difficult to interpret due to the small sample size and lack of power. Whilst support was provided to a majority of parents in the clinician supported group, trying to reach parents by telephone was time consuming and not all parents seemed to need or desire this support. In the unsupported arm there were few complaints or suggestions from parents for more interpersonal support. Regarding child mental health outcomes, there was no clear evidence for better effects in the supported group, with both groups improving to a similar degree. The only indication of superiority for the supported group was in the rates of reliably improved and proportion of children without an anxiety disorder at post-intervention. As noted by Baumeister et al. (2014), the evidence base on the optimal way of providing guidance in internet interventions is scarce. This pilot study only tested one method of providing clinician support to parents, and more effective alternatives are possible, including more frequent support, support provided via a different medium, or only on demand.

Limitations of this study include the absence of a control group, the use of self-report parent measures, and no longer-term follow-up. The simple randomization strategy resulted in unequal groups and there was a higher than expected attrition rate. For parents who did not complete the post-questionnaire, program satisfaction is unknown, and this must temper the positive ratings by parents who did complete the questionnaire. Notwithstanding these limitations, the study provided valuable information about how to refine and improve the program before evaluation in a planned large RCT. Changes will include offering access to the program for a longer period of time, providing clinician support only when requested, and making it easier to log in and use the program on a smartphone or tablet. Also, modules will be released sequentially once a week in conjunction with automated emails notifying parents about the availability of each new module. Offering a fixed sequence of modules (a ‘tunneled’ approach) rather than ‘open’ module access (with greater user control) may increase intervention use (Crutzen et al., 2012). The emails about the new modules will serve as reminders which could also increase program use (Kelders et al., 2012).

In conclusion, the results from this initial evaluation of the Cool Little Kids Online program indicate that it is feasible to adapt an efficacious group parenting program into an online format. The online program was acceptable to parents with temperamentally inhibited young children and was associated with significant improvements in child anxiety. The findings support implementation of a large randomized trial to determine efficacy. Cool Little Kids Online may be a promising direction for improving access to an evidence-based prevention and early intervention program for child anxiety disorders.