Cluster Analysis of Child Externalizing and Prosocial Behaviors in a Randomized Effectiveness Trial of the Family-Check Up and Internet-Delivered Parent Training (iComet).

OBJECTIVE: To explore whether children with various externalizing/prosocial behavior profiles benefit differently from face-to-face training than from an internet-based parent management training (PMT) programme.
METHODS: A total of 231 families with children (aged 10 to 13 years) with externalizing behavior problems (EBP) were randomized to receive either the Family Check-Up, delivered by therapists in the community, or the internet-based PMT program (iComet). Person-oriented analysis was used for subtyping the children according to combinations of prosocial behavior and EBP.
RESULTS: The person-oriented analysis resulted in five significantly different clusters. There were no significant differences between the five clusters in relation to the total difficulties score of the Strengths and Difficulties Questionnaire, family warmth or family conflict, but the within-group effect sizes for the main outcome (total difficulties score) from baseline to post-treatment varied from Cohen's d of 0.52 to 2.56. There were no significant interaction effects between the clusters and type of intervention. However, for children high on symptoms of attention deficit hyperactivity disorder and moderate to high on oppositional defiant disorder, and low to relatively high on prosocial behaviors (Cluster 3 respectively 5), substantial residual EBP-symptomatology remained at post-treatment, although both interventions resulted in significant effects. The other three clusters were within the non-clinical EBP-range at post-intervention, irrespective of treatment condition. There were no significant differences between the clusters regarding treatment completion rate (ranging from 47.2% to 67.4%). This study illustrates the value of distinguishing between different profiles of children in the context of PMT for parents of children with EBP. © Person-Oriented Research.

Introduction

Children with externalizing behavior problems (EBP), that is, disruptive, oppositional, aggressive, and conduct problems, constitute a heterogeneous group of children. EBP may also include attention deficit hyperactivity disorder (ADHD) problems. Children with EBP have broad characteristics of behavior problems in common, but they also exhibit a variety of other qualities (Bloomquist & Schnell, 2002), which may indicate different needs in terms of treatment. To provide effective help to children with EBP and their parents, clinical assessments are needed to effectively analyse different areas of concern in order to tailor interventions accordingly (Van Ryzin & Dishion, 2012).

Oppositional defiant disorder (ODD) and conduct disorder (CD) are categorized under “Disruptive, impulse-and conduct disorders”, whereas ADHD is classified as a neurodevelopmental disorder in the Diagnostic and Statistical Manual of Mental disorders (DSM-5; American Psychiatric Association, 2013). Generally, ADHD without comorbid CD or ODD is not as strongly related to the development of later conduct problems and antisocial behavior in children as when ODD/CD is present in childhood (Barkley, 1997; Loeber et al., 2000), whereas aggressive or oppositional behaviors are predictors of later behavior problems for individuals both with and without ADHD (Patterson et al., 2000). This indicates that ODD/CD children without ADHD might require a different treatment plan compared to children with ODD/CD and ADHD (Schachar & Tannock, 1995; The MTA Cooperative Group, 1999). For children with disruptive disorders, parent management training (PMT) programs are commonly recommended (NICE, 2013; Thompson et al., 2009). In ADHD-specific PMT programs, parental psycho-education about ADHD and a focus on developing helpful strategies to compensate for the child’s deficits are often included, which is different from programs targeting children with ODD or CD.

DSM-based assessments are helpful in choosing the most efficacious treatments for children with a particular diagnosis. However, there are also other aspects, such as child characteristics, parental characteristics, parent-child interaction, family functioning, and peer factors, that should be evaluated in order to identify specific risk factors (e.g., callous-unemotional traits, lack of social skills, coercive parenting, low monitoring) with a possible direct, indirect or cumulative influence on the development and maintenance of a child’s problem behaviors (Dodge & Pettit, 2003; Hill, 2002), and a possible impact on treatment planning and outcome (Lundahl et al., 2006; NICE, 2013). The main objective of the present study was to investigate whether children can be categorized into distinct clusters based on a combination of difficulties and strengths, and whether these clusters show different patterns in terms of engagement in the treatment and outcome.

Many families with children who suffer from both ADHD and conduct problems are stuck in coercive parent-child interactions (Patterson et al., 2000; Patterson et al., 1998). Coercive parental behaviors include, for example, inconsistent and/or harsh discipline, low levels of warmth and frequent criticism directed at the child. Negative reinforcement, or escape conditioning, is the central feature of coercive parent-child interactions. The non-compliant child behavior is reinforced and in the long run this interactional pattern leads to frequent conflicts between the parent and the child (Patterson, 1982). Attending to and addressing conduct problems and ADHD-related difficulties of a child without considering and addressing potential patterns of coercive parenting, if present, might lead to a less optimal treatment outcome, and vice versa.

Another variable that can operate as either a protective factor or an additional risk factor is the level of prosocial behaviors in terms of being considerate of other people’s feelings, and being helpful if someone is hurt, upset or feeling ill (Goodman, 1997). The presence or lack of prosocial behaviors might make parenting easier or more difficult, and might also affect children’s interaction with peers (Andrade & Tannock, 2014). Andrade and Tannock (2014) found that prosocial skills and conduct problems partially mediated the relationship between hyperactivity/ inattention and peer functioning in different directions. Prosocial behavior might therefore be important to include as a specific variable in a potential profile or extended analysis of a child’s strength and difficulties.

We do not know with precision and scope which treatments might be best suited for children with combinations of different levels of behavior problems in conjunction with various risk factors. For instance, if a family with a child with symptoms of ODD and ADHD and characterized by having few prosocial behaviors is referred to a clinic, would it then be appropriate to recommend the family a regular PMT program? Is PMT sufficient or may more modalities be needed, such as a focus on developing the child’s prosocial skills and training the child in compensating for the core deficits of his or her ADHD?

Other major problems reported are access to treatment and high dropout rates among those who receive treatment. First, about 66% of people in need of treatment have trouble receiving it, whereas for those who have access to and start treatment the dropout rate is approximately 50% (Kazdin, 2008). Modern interventions need to have embedded features that enhance access and decrease dropout rate. One of the interventions examined in the present study, the Family Check-Up (FCU: Dishion & Stormshak, 2007), is designed to pay attention to families’ motivation to engage in the program, leading to fewer instances of dropping out. The other intervention evaluated in this study, the iComet (Enebrink et al., 2012), is internet-based and therefore more available to the families since they can access the program themselves whenever they choose.

The main purpose of the present study was to evaluate the effects of FCU and iComet for specific clusters of children (aged 10 to 13 years) with a variety of behavioral profiles. More specifically, the aims were to use a person-oriented approach to: (1) evaluate the plausibility of different subtypes/profiles derived from critical features of child behavior (i.e., levels of ADHD and ODD symptoms as well as prosocial behaviors); (2) examine possible differences in outcome (behavioural difficulties, family warmth and family conflict) between the established profiles and within the clusters over time; and (3) evaluate whether treatment engagement differs depending on the cluster profile of the child and the type of intervention.

Methods

Participants

Families with children aged 10 to 13 years, living in six different city areas of Gothenburg, were invited to participate in the study. Inclusion criterion was the presence of some degree of problem behavior, defined in terms of a cut-off of 3 points or more on the Conduct Problem subscale of the Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997, 1999) rated by the parent or child’s teacher. Exclusion criteria were simultaneous participation in another treatment for conduct problems or other interventions initiated by the Office of Social Services, and comorbid psychiatric diagnosis (e.g., depression, obsessive compulsive disorder, or autism). The characteristics of the participants are presented in Table 1. There were no significant differences in any of the background variables (marital status, educational level or income) between those randomized to FCU and those assigned to iComet (p > .05).

Table 1

Characteristics of the families in the FCU, the iComet, and the total sample

	FCU n = 122 Number (%)	iComet n = 109 Number (%)	Total sample N = 231 Number (%)
Mothers and fathers
Mother	104 (85.2)	99 (90.8)	203 (87.9)
Father	18 (14.8)	10 (9.2)	28 (12.1)
Parents’ marital status
Married	51 (41.8)	44 (40.4)	95 (41.1)
Single parents	35 (28.7)	32 (29.4)	67 (29.0)
Living together but not married	27 (22.1)	25 (22.9)	52 (22.5)
Other	9 (7.4)	8 (7.3)	17 (7.3)
Family income
Monthly income is not enough for our expenses	7 (5.7)	12 (11.0)	19 (8.2)
Monthly income can barely cover our expenses	28 (23.0)	33 (30.3)	61 (26.4)
Monthly income is enough for our expenses - we are not worried	71 (58.2)	57 (52.3)	128 (55.4)
Monthly income is good – we do not need to think about our expenses	16 (13.1)	7 (6.4)	23 (10.0)
Education
Elementary school or less	12 (9.8)	10 (9.2)	22 (9.5)
High school	45 (36.9)	52 (47.7)	97 (42.0)
College or university	65 (53.3)	47 (43.1)	112 (48.5)
Children’s sex
Girls	49 (40.2)	46 (42.2)	95 (41.1)
Boys	73 (59.8)	63 (57.8)	136 (58.9)
Children´s grade
4th grade	33 (27.0)	27 (24.8)	60 (26.0)
5th grad	21 (17.2)	25 (22.9)	46 (20.0)
6th grade	27 (22.1)	18 (16.5)	45 (19.5)
7th grade	41 (33.6)	39 (35.8)	80 (34.6)

Procedure

This study is based on data from a randomized effectiveness trial of the FCU and the iComet using a variable-oriented approach (Ghaderi et al., 2018), pre-registered at the primary clinical trial registry ISRCTN (http://www.isrctn.com/ISRCTN09352710). Due to lack of normative data from the general population for the instruments used we collected norms from a random sample of the target population in Sweden (for details of this study, see Björnsdotter et al., 2013). Families with children in the targeted ages in participating municipalities in Gothenburg were informed about the study in letters, bulletin board advertisements, as well as at parent meetings at the schools.

A flow chart illustrating recruitment and dropout is presented in Figure 1. Parents interested in participating sent in a letter or e-mail expressing their interest and were thereafter contacted by a research assistant, a graduate, or a doctoral student for more information. A screening interview was conducted by phone. If the family belonged to the target group the parents were informed that participation in the study was voluntary, and that they could choose to withdraw at any time. After obtaining written informed consent, the parents and the child’s teacher were asked to respond to the SDQ (Goodman, 1997). Children scoring above the cut-off (i.e., three points or more) on the SDQ Conduct Problems subscale (Goodman, 1997), as rated by parents or teacher, were included in the study (n = 231). The included parents were asked to provide more background information and respond to other questionnaires via Regional Ethical Review Board (dnr 2010/119). Post-intervention measures were completed via the Internet as a baseline measurement. If the parent or teacher preferred to fill out the questionnaires by hand, these were sent to them by regular mail with a pre-paid return envelope included.

Figure 1

CONSORT flow diagram depicting enrollment, allocation and post-measurement

The families were then randomly assigned to either FCU (n = 122) or iComet (n = 109) using an online randomization service (www.randomizer.org). The randomization was stratified according to children’s age and sex. A trained therapist for those assigned to FCU, or a research assistant/doctoral student for those assigned to iComet, contacted each family to start the intervention once the questionnaires were completed. The participants had the opportunity to choose between four different gifts after filling out the questionnaires (all with the same monetary value; approximately 30 USD). The project was approved by the Regional Ethical Review Board (dnr 2010/119). Post-intervention measures werecompleted via the Internet, 10 weeks after intervention started.

Interventions

The Family Check-Up

The FCU (Dishion & Stormshak, 2007; Dishion et al., 2012) is a family-centered intervention and a development of the Parent Management Training Oregon Model (Forgatch et al., 2005). Theoretically, the FCU has its base in social learning theory and family theory, and is grounded in the coercion model (Patterson, 1982). The FCU is a brief face-to-face intervention consisting of a three-session evaluation phase designed (1) to determine the presence of risk and protective factors known to have an impact on child psychological development, and (2) to enhance parent motivation to change. The three-session assessment is followed by an optional parent management training program, the Everyday Parenting Curriculum (EPC).

The FCU assessment starts with an initial contact with the parents, and an interview focusing on parents’ concerns and perceptions of their child’s behavior. The parents are also asked to complete questionnaires to gather information about family background, child behavior, and parenting practices. During the second session, a family observation is conducted, consisting of a videotaped home visit during which the families are asked to complete five interaction tasks of five minutes each. These tasks are videotaped in order to enable coding of parenting skills and parent-child interaction patterns. The video can then be used later in the feedback session. The cornerstone of the FCU is the third session, a structured feedback session that emphasizes the role of parenting, focuses on family strengths, and enhances parent motivation to change in areas where risk factors are present (Dishion & Stormshak, 2007). A unique aspect of the FCU is the ability to adapt and tailor the intervention based on the family’s specific needs and the parent’s level of motivation. The periodic and sometimes brief model of intervention is another unique quality of FCU as families are able to get help during challenging periods, such as different development phases or contextual transitions (Dishion et al., 2012). In addition, each family is followed up once a year for as long as there is a mutual agreement regarding the need for and benefit of yearly check-ups.

EPC consists of three different modules: positive behavior support, setting healthy limits and building family relationships and can vary from 1 to 12 sessions (Dishion et al., 2012). In the FCU model, families are usually offered a menu of intervention options such as parent skills training, parent groups, family therapy, child interventions, school interventions or other support based on available resources (Dishion et al., 2012). During the present study, parents were given the option of participating in up to 12 sessions of EPC after the feedback session. Of those families randomized to the FCU intervention, 100 families (82%) received 3 sessions or more, of which 22 families completed the FCU only, and 78 families continued with EPC sessions (see Figure 1). No other options from a broader menu of interventions were offered. The engagement in EPC sessions ranged from 1 to 12 sessions (M = 4 sessions).

A total of 16 therapists delivering the FCU and EPC were trained and supervised during the study. Five supervisors were also trained and supervised by a project researcher certified in FCU education and supervision. Every session was also videotaped in order to ensure program fidelity. All therapists had a university degree and between 3 and 23 years of working experience (M = 11.81 years).

The PMT intervention delivered through the internet, iComet

The iComet (Enebrink et al., 2012) is an internet-delivered PMT program originating from the Swedish PMT-program Comet (“COmmunication METhod”), which is based on principles from social learning theory and cognitive-behavioral therapy (Hassler Hallstedt et al., 2005; Kling et al, 2010). A few adaptations were made to the program for delivery via the internet such as a decrease from 11 to 7 sessions, although three of the sessions contained homework assignments to be completed over a 2-week time span, thus extending the duration of the intervention period to 10 weeks. The face-to-face Comet includes a meeting between the parent, the child’s schoolteacher and the therapist but this was not included in iComet. The content of iComet is similar to other PMT programs with a focus on parenting skills such as positive behavior support, communication, problem solving and parents’ management of their own dysfunctional emotional reactions (Högström et al., 2013).

The online sessions were composed of video vignettes, written material and illustrations. After each session parents answered multiple-choice questions about the session’s content and were given a direct response from the program as to the accuracy of their answers. The rationale for this was to reinforce correct statements and explanations in order to enhance parents’ learning of the material. Time spent on each session was estimated to be approximately 1.5 hours, followed by assignments for the week to practice the skills learned. In order to enhance adherence to the intervention, a program facilitator provided support and feedback via e-mail, and also assigned the next session. All facilitators were trained in iComet. Five students in the final semester of their education (MS in Psychology), three research assistants with various academic educations and one clinical psychologist acted as facilitators in the program. The facilitators were instructed to give feedback to reinforce engagement with and focus on the program. The parents were also able to communicate via e-mail with a facilitator if needed, to receive support for example with problem solving or clarifying the program content.

Following random assignment to the iComet, one of the facilitators contacted the family, described the intervention and scheduled a date for an introduction. During the first meeting parents were given an explanation of the rationale for iComet as well as instructions for how to use the website, along with a password.

Engagement in the iComet condition ranged from 0 to 7 sessions (M = 3.73 sessions). Of the included 109 families, 67 (61%) families started the intervention; of these 39 (58.2%) received 3 sessions or more and were considered to have completed treatment (see Figure 1). Parents were encouraged to complete one session per week but this was not a requirement for participation.

Measures

Externalizing behavior problems

The Disruptive Behavior Disorders (DBD) Rating scale (Pelham et al., 1992) covers the DSM-IV-based symptoms (APA, 2000) for all three disruptive behavior disorders: CD, ODD, and ADHD. Each item in the DBD (Pelham et al., 1992) is rated on a four-point scale (0 = Not at all, 1 = Just a little, 2 = Pretty much, and 3 = Very much). It is also possible to answer “Do not know” on the items, in which case the item is not included in the scoring procedure. The DBD (Pelham et al., 1992) includes 45 items, but after the revision of the DSM-III-R to the DSM-IV (APA, 1987, 2000), three of the items are no longer coded in the scoring (item 10, 14 and 21). Furthermore, item 5 (“Often initiates physical fights with other members of his or her household”) is not coded since it does not correspond to any criteria in the DSM-III-R or the DSM-IV (APA, 1987, 2000). Two different scoring procedures can be used to summarize the responses on the DBD: a symptom count or a composite score, of which the latter was used in the present study. The scale was therefore used dimensionally with different levels of ADHD and ODD behaviors. A composite score was calculated by adding the scores of the items within each subscale and then dividing this sum by the total number of responses (Pelham et al., 1992).

Two subscales (ODD and ADHD) of the DBD (Pelham et al., 1992) were used in the person-oriented analysis. The subscale ODD consists of eight items, such as “Often argues with adults”. The ADHD subscale consists of nine items assessing attention difficulties and nine questions assessing hyperactivity. “Often has difficulty organizing tasks and activities” and “Often has difficulty awaiting turn” are typical examples from the respective constructs of the ADHD subscale. There are no reverse questions on these subscales; therefore, higher scores correspond to a greater degree of difficulty within the measured area. Good internal consistency (Cronbach’s alpha) was found for the subscales at baseline assessment, .86 (ODD; n = 205) and .93 (ADHD; n = 166). These numbers are slightly lower than earlier reported figures, where internal consistency of ADHD was .96 and the corresponding figure for ODD was .95 (Pelham et al., 1992). One possible explanation for these differences is that the latter study was based on teacher ratings whereas the present study relied on parent ratings.

Prosocial behavior

Another variable included in the person-oriented cluster analysis is the Prosocial Behavior subscale from the Strengths and Difficulties Questionnaire, SDQ (Goodman, 1997, 1999, 2001). The SDQ is a well-validated questionnaire to which both parent and teacher can respond (Smedje et al., 1999). The subscale Prosocial Behavior consists of five questions, such as “Considerate of other people's feelings”. A three-point Likert-type scale is employed to indicate the degree to which each attribute applies to the target child (0 = Not true, 1 = Somewhat true, and 2 = Certainly true). Higher scores reflect a greater degree of social solicitude. The internal consistency (Cronbach’s alpha) at baseline-measurement was .77, n = 231.

General difficulties

As a treatment-outcome measure, the summary subscale Total Difficulties from the SDQ (Goodman, 1997) was used. This summary score consists of four of the five SDQ subscales (i.e., Emotional Problems, Peer Problems, ADHD and Conduct Problems). A total of 20 items are used, five items per subscale. Five items are reversed, and these have all been converted in the analysis. A higher Total Difficulties score indicates a greater severity of problems. The internal consistencies (Cronbach´s alphas) in the present study were .76 at baseline and .78 at the post-measurement.

Family climate

The five items measuring Family Warmth are taken from the Adult-Child Relationship Scale (ACRS: Criss & Shaw, 2005), which is an adaptation of the School-based Student-Teacher Relationship Scale (Pianta & Nimetz, 1991). The items on this subscale reflect the degree of warmth between the parent and the child. An example of an item on this scale is “If upset, this child seeks comfort from me”. Items are answered on a five-point scale ranging from “Definitely not” to “Definitely”.

Family Conflict is measured with four questions, adapted from the PAL2 project by the Child and Family Center, University of Oregon, USA. These items are broad and encompass conflicts between all family members. The Family Conflict subscale consists of questions such as “We got angry at each other,” which are answered on a seven-point Likert scale from “Never” to “More than 7 times during the last month.” The internal consistencies (Cronbach’s alphas) of both subscales were good (.82 and .81 at baseline and post-measurement for Family Warmth). The corresponding figures for Family Conflict were .72 at baseline and .74 at post-treatment measurement.

Dropout

There were no significant differences in demographic characteristics (i.e., marital status, educational level, income or psychological distress) or the extent of child symptoms (i.e., scores from the subscales of the DBD), between those who completed the post-measurement and those who did not.

The FCU intervention

Of the 122 families who were randomized to the FCU, sixteen (13.1%) families chose not to participate in the study before starting the intervention, resulting in 106 families actually starting the FCU. The assessment phase of the FCU intervention, ending with the feedback session, was completed by 100 families (82.0%). Of all families randomized to FCU, 78 (63.9%) families also chose to participate in EPC (ranging from 1-12 sessions). A total of 82 (67.2%) families responded to the post-treatment measurement, 12 of which received the FCU assessment only and 70 of which received the FCU assessment plus EPC. Of those randomized to FCU, 82 (67.2%) families submitted post-treatment data. Since all of them had started the intervention they were all included in the outcome analyses.

The iComet intervention

Of the 109 families who were randomized to the iComet, 24 (22%) families chose not to participate in the study after being informed about the result of the randomization. Some families, 18 (17%) never logged into the iComet and were thus excluded in the outcome analysis. A total of 39 (35.8% of those originally randomized) families in the iComet condition completed three sessions or more (out of seven sessions) and were considered treatment completers. The rationale for choosing three sessions was to enable comparison with earlier studies that have utilized the same division between completers and non-completers (Enebrink et al., 2012; Högström et al., 2013). This also facilitated comparison to the FCU intervention, which has a three-session completion criterion. Fifty-six (51.4%) families submitted post-treatment data, of which 48 completed at least one session and were therefore included in outcome analyses.

Statistical analyses

Pearson’s chi-square for categorical variables and t-test for continuous variables were used to explore differences in background and baseline variables between the intervention and profile subgroups, and in baseline measurements for those who provided post-treatment data and those who did not. Cronbach’s alpha was used to determine the reliability of the different subscales included in the study.

In order to analyze the effectiveness of the interventions for different subgroups of children a person-oriented cluster analysis (Bergman et al., 2003) was performed. The statistical software used were ROPstat (Vargha et al., 2015) and IBM SPSS statistics 23. The cluster analysis (Bergman et al., 2003) was conducted in order to group the children into profiles based on critical variables from the baseline measurement. In this study the variables in focus were baseline levels of ADHD, ODD and Prosocial Behavior. The cluster analysis followed the LICUR procedure (Bergman et al., 2003). First, outliers were identified by the residue procedure. Secondly, Ward's hierarchal cluster method was applied. When selecting the cluster solution, the criteria suggested by Bergman et al. (2003) were used: the solution should be theoretically motivated, with preferably no fewer than five clusters (or more than 15 clusters), and ideally the solution should account for at least 67 % of the total explained error sum of squares, and if not at least exceed 50% of the total explained error sum of squares. Major changes between different cluster solutions were also analyzed to ensure that two clusters were not grouped together in an inappropriate manner. Euclidean distance was used, even though it is sometimes criticized for being unsuitable, especially when the variables have different units of measurement and different variances, or when the variables correlate highly with each other. To address this problem, data were standardized based on norm data (see Table 2) before calculating distances. A data simulation was conducted to explore whether the proposed cluster solution accounted for significantly larger percentage of the total explained error sum of squares than expected by chance.

Table 2

Profiles of the clusters based on the baseline measurements subscale scores

Cluster name (cluster size)	Prosocial Behavior M (SD)	ADHD M (SD)	ODD M (SD)
1. High on Prosocial Behavior, low on ADHD and medium on ODD (n = 54)	8.76 (0.97)	0.82 (0.37)	1.04 (0.43)
2. Low on Prosocial Behavior and low on ADHD and ODD (n = 36)	6.25 (0.77)	0.74 (0.44)	0.75 (0.25)
3. Very low on Prosocial Behavior and very high on ADHD and ODD (n = 43)	3.35 (1.27)	1.89 (0.50)	2.09 (0.54)
4. Low on Prosocial Behavior, medium on ADHD and high on ODD (n = 56)	5.96 (0.89)	1.15 (0.41)	1.75 (0.33)
5. High on Prosocial Behavior, very high on ADHD and high on ODD (n = 42)	8.45 (1.15)	1.98 (0.46)	1.90 (0.49)
Total sample (N = 231)	6.63 (2.20)	1.30 (0.66)	1.52 (0.65)
Norms from the general population*	8.35 (1.74)N = 1437	0.31 (0.38)N =1390	0.38 (0.40)N = 1407

Note: *The norms used are unpublished raw data collected in the same way as in Björnsdotter et al. (2013)

A series of one-way ANOVAs including the Tukey post-hoc test, with cluster categorization as the independent variable, and the subscales Prosocial Behavior, ADHD and ODD as dependent variables, were then conducted to understand in what ways the clusters were different from each other on these subscales.

The intra-group effect sizes (ES), taking the correlation between the measurements into account, are presented as Cohen’s d. A Cohen’s d of 0.8 was considered a large effect, 0.5 a medium effect and 0.2 a small effect (Cohen, 1988). The effects of the interventions, depending on cluster subtype, were evaluated with ANCOVA. The independent variable was FCU or iComet (i.e., the intervention) and the dependent variables were the outcome as measured by SDQ Total Difficulties, Family Warmth and Family Conflict, controlling for the baseline measurements of these scales.

Results

Cluster solution

The cluster solutions were derived from the randomized families’ baseline measurements (n = 231). After the residue procedure was conducted, a limit was set for the Average Squared Euclidian to 0.7 (and the number of twins required = 1). There were no outliers within this sample (the participant furthest away was at .69).

In Ward's method of hierarchical analysis, several cluster solutions were obtained. Based on the criterion for selecting a cluster solution (see Statistical analyses) a five-cluster solution was derived, explaining 65.39% of the total Error Sum of Squares. In Table 2, the profiles are presented with the mean and standard deviations for each subscale in the baseline measurement.

A more illustrative way of comparing the different cluster profiles is to use z-scores for each subscale. Z-scores are calculated by taking the difference between the cluster mean and the mean for the norm group and dividing by the SD of the norm group. The profiles are illustrated in Figure 2, where each cluster has been plotted based on the z-score for each variable as compared to norms for the population.

Figure 2

The emerging profiles based on z-scores of ODD, ADHD and Prosocial Behavior in each cluster

There were significant differences between the profiles, and the largest differences were found between Clusters 1 and 3. Cluster 1 had a relatively high degree of prosocial behavior paired with low levels of behavioral problems both in terms of ODD and ADHD. For Cluster 3, the opposite was found, that is, in addition to being high on the two behavior problem scales (i.e., ODD and ADHD), the children in this profile exhibited a relatively low degree of prosocial behavior. Cluster 5 showed the most pronounced problems in terms of ADHD and some degree of defiance behavior, paired with a high degree of prosocial behavior. Cluster 4 was characterized by the presence of behavioral problems and a low level of prosocial behavior. Participants in Cluster 2 had few ODD and ADHD behaviors, as in Cluster 1, but showed a lower degree of prosocial behavior.

In summary, Clusters 1 and 2 were characterized by fewer behavior problems, and levels of prosocial behavior within the normal range for Cluster 1, although below average for Cluster 2. Further, clusters 3 and 5 had the highest ratings of behavior problems, but Cluster 5 was high in prosocial behaviour, while Cluster 3 was very low. Finally, Cluster 4 had medium levels on ADHD and prosocial behavior ratings compared to the other clusters featured in this sample and rather high ratings on ODD behaviors even though there were not as high as for Cluster 3 and 5.

A data simulation showed that the cluster solution accounted for a significantly larger percentage of the total explained error sum of squares than expected by chance (t(9) = 17.64, p < .001).

A series of one-way ANOVA showed significant differences between the clusters on each of the dependent variables (Prosocial Behavior F(4,226) = 209.38, p = < .0001; ADHD F(4,226) = 80.22, p = < .0001 and ODD F(4,226) = 80.47, p < .001). Tukey’s post-hoc tests showed that Clusters 1 and 5 were significantly higher on prosocial behaviors than the other clusters and Cluster 3 had a significantly lower degree of prosocial behaviors than the other clusters. Clusters 3 and 5 were significantly higher on both the ADHD-subscale and the ODD-subscale than the other three clusters. Cluster 4 was also significantly higher on ODD behaviors than Clusters 1 and 2.

Outcome of interventions for each profile

To test whether various cluster profiles had significantly different outcomes (regardless of the type of intervention) three ANCOVAs were conducted separately. The independent variable was cluster membership, and the dependent variables were the post-measurements of SDQ Total Difficulties score, Family Warmth and Family Conflict scales. The covariate variables were the baseline measurements of each corresponding scale. There were no significant differences between the clusters on any of the scales, Total Difficulties (F[4, 129] = 2.19, p = .075), Family Warmth (F[4, 128]) = 0.47, p = .755) or Family Conflict (F[4, 128] = 1.59, p = .185).

Three clusters were relatively low in ADHD-symptoms (Clusters 1, 2 and 4) and these only differed somewhat with regard to Prosocial Behavior and level of ODD behaviors. Figure 3 shows the baseline measurement and post-treatment measurements for the SDQ Total Difficulties subscale for these three clusters per intervention.

Figure 3

Baseline and post-treatment scoreson Total Difficulties per intervention in Cluster 1, 2 and 4 visually compared to clinical boarder

Figure 4 shows the baseline and post-treatment scores on the SDQ Total Difficulties subscale for Clusters 3 and 5 per intervention.

Figure 4

Baseline and post-treatment scoreson Total Difficulties per intervention in Cluster 3 and 5 visually compared to clinical boarder.

Table 3 presents the means and standard deviations for the pre-post changes of our main outcome measure (SDQ Total Difficulties), as well as the effect size (Cohen’s d) for each cluster and intervention. The effect sizes for the main outcome measure (SDQ Total Difficulties) ranged from d = 0.52 to 2.56, indicating medium to large effects. Large effect sizes were noted for those who received FCU in Cluster 3, 4 and 5, and those who received iComet in Cluster 1, 2, 3, and 5.

Table 3

Mean, standard deviation and effect sizes per cluster and intervention for Total Difficulties scale

Cluster	Intervention (n)	Baseline measurement M (sd)	Post-intervention measurement M (sd)	Effect size Cohen’s d
Cluster 1.	FCU (n = 19)iComet (n = 9)	13.37 (5.09)12.44 (4.93)	11.16 (5.55)8.89 (5.01)	-0.52-1.12
Cluster 2.	FCU (n = 14)iComet (n = 4)	11.64 (4.60)18.00 (8.64)	8.79 (5.92)12.00 (4.55)	-0.75-1.43
Cluster 3.	FCU (n = 15)iComet (n = 11)	21.87 (4.44)20.45 (5.37)	16.33 (5.02)14.27 (5.68	-1.14-1.10
Cluster 4.	FCU (n = 23)iComet (n = 14)	16.30 (4.32)15.71 (3.85)	11.78 (3.81)12.14 (4.19)	-1.21-0.70
Cluster 5.	FCU (n = 11)iComet (n = 10)	19.64 (4.84)22.00 (3.20)	16.27 (4.29)19.40 (4.74)	-2.56-0.97

In terms of Family Warmth, the magnitude of effect across time for different conditions in the various clusters varied greatly (d = - 0.71 to 1.18). For Cluster 5 participants within the FCU, the level of Family Warmth decreased with a medium effect size from pre- to post- intervention, d = - 0.71 (i.e., reversed direction of hypothesis), while for Cluster 2 and 3 FCU participants a large improvement was observed (d = 1.01 and 1.18, respectively). For iComet the clusters showed less difference with small to large effect sizes, d = 0.33 for Cluster 4 to d = 1.03 for Cluster 2.

When Family Conflicts was evaluated, effect sizes ranged from d = - 0.03 to 1.23. Participants in FCU reported small to medium effect sizes (d = 0.22 to d = 0.63). In the iComet intervention most clusters showed small to non-existent effect sizes (d = -0.03 to d = 0.26). Cluster 3 differed from the other clusters as they experienced a large decrease in family conflicts after the interventions (d = 1.23).

To examine how the interventions worked for each specific cluster, two-way factorial ANCOVAs were employed with intervention (i.e., FCU or iComet), and the clusters as independent variables. The dependent variables were the post-treatment SDQ Total Difficulties scores, and Family Warmth and Family Conflict ratings in separate analyses, controlling for each scale’s baseline scores. No significant interaction effects emerged between cluster and type of intervention: SDQ Total Difficulties (F[4, 129] = 0.733, p = .571), Family Warmth (F[4, 128] = 1.867, p = .121), and Family Conflict (F[4, 128] = 0.215, p = .930).

Dropout and engagement in treatment

Families that completed three or more sessions of the iComet intervention (i.e., between three and seven sessions) or the assessment phase of FCU were considered treatment completers. The total proportion of completers in each cluster (including both interventions) was relatively even, ranging from 47.2% (Cluster 2) to 67.4% (Cluster 3). The corresponding figure for Cluster 1 was 57.4%, for Cluster 4 66.0%, and for Cluster 5 59.5%. However, there was a considerable difference between the proportions of completers within the clusters depending on whether they received the FCU or the iComet. In the FCU, 71.9% to 90.3% of the families in the clusters attended at least three sessions, that is, finished the first phase of FCU. The corresponding figures for the clusters receiving iComet were 11.8% to 47.4%. Cluster 2 with only 11.8% completers for iComet differed from those receiving FCU in two ways. First, only a few families (n = 10) assigned to iComet belonged to this cluster. Second, the baseline value of the SDQ Total Difficulties for the Cluster 2 receiving iComet was higher (M = 18.00) compared to those receiving FCU in Cluster 2 (M = 11.64).

Discussion

Research investigating the effectiveness of PMT interventions in “real life settings” is essential in order to evaluate if these interventions really work (Costin & Chambers, 2007). This study examined two different interventions for families with children with EBP in a community-based setting.

By conducting a person-oriented cluster analysis, based on the critical variables (i.e., subscales score from ADHD, ODD and Prosocial Behavior), five distinct profiles emerged. This implies that this group of children with EBP is a heterogeneous group, within which significantly different subgroups can be identified.

Interestingly, Cluster 2 with FCU as well as Cluster 1 with iComet and FCU as interventions, were already within the non-clinical range at baseline measurement (see Figure 3). Clusters 1 and 2 thus had less EBP initially, and Cluster 1 also exhibited prosocial behaviors, as shown by their cluster profile (see Figure 2). Clusters 1, 2 and 4 were all within the non-clinical range after intervention, regardless of whether the families received FCU or iComet, with moderate to large effect sizes for all clusters and both interventions. This result indicates that FCU and iComet function well, and may be sufficient for these profiles as early interventions.

The group of children at risk for continuing EBP are those with comorbid problems with very high levels of ADHD (four standard deviations above the mean for the general population) and ODD behaviors, who comprise Cluster 3 and 5 in this study. There is an important difference though, between these two burdened clusters. Cluster 3 is almost three standard deviations below the mean for the general population regarding Prosocial Behavior, indicating that this specific cluster may have problems with social skills and interrelationships (see Figure 4).

The results for Cluster 3 showed a large effect size with regard to SDQ Total Difficulties score for both FCU and iComet (d = 1.14 and d = 1.10, respectively), but both groups’ scores were still in the clinical range following the interventions (see Figure 4). The problem behaviors might continue to decrease over time, since the starting point was high and the trend is toward improvement. If their problems are not reduced over time, they might be at high risk for further EBP (Barkley, 1997). These behaviors might interact negatively with the lack of prosocial behavior and might be accompanied by peer problems or other social maladjustments. At this point however, we argue that neither the FCU nor iComet were sufficient for the children in this cluster. Since children with profiles such as those in Cluster 3 represent a high-risk group, more effort should be put into identifying children with such profiles in order to provide efficacious interventions.

Cluster 5 is characterized by having severe difficulties relating to ADHD problems and ODD behaviors, and differs from Cluster 3 most importantly in terms of the presence of prosocial behavior. The outcome analysis revealed quite different effect sizes depending on the type of intervention given to families with children belonging to this profile. FCU resulted in a large effect size (d = 2.56) compared to d = 0.97 in iComet. However, these differences should be interpreted cautiously as the main reason for the large difference between these effect sizes is the large correlation in the observations from baseline to post-treatment in the FCU.

One of the theories behind both interventions, the coercive family process (Patterson, 1982) postulates that child EBP decrease as a consequence of improved parenting skills. Family warmth and conflicts might be important contextual factors for the association between parental skills and child EBP. The outcome measurements of change in family warmth were mixed. In one cluster, the family warmth decreased at post-intervention and for the others the family warmth increased. No clear pattern of change with regard to family warmth and conflict emerged, even though child EBP decreased for all clusters. When Family Conflicts were evaluated, most clusters showed small to non-existent effect sizes and also in both directions (i.e., increase as well as decrease). Therefore, a focus on positive behavior support practice does not seem to affect the family climate in a consistent way when children are grouped into different clusters. The way the Family Warmth scale has been operationalized (e.g., “If upset, this child seeks comfort from me”, “This child likes telling me about him/herself”, “It is easy to be in tune with what this child is feeling”, “This child is open with me about sharing feelings and telling me how things are” and “Dealing with this child makes me feel good about how I am handling things”) may be insufficient as an outcome measure at short term follow-up. The same goes for the Family Conflict scale, which is about conflict in general within the family and not specifically conflicts between parent and child. However, when analyzing the whole sample together the direction of decrease in Family Conflict and increase in Family Warmth is reinsuring. Therefore, the mixed result in our study may be partly due to the small sample sizes of treatment conditions within each cluster.

Neither of the interventions offered in this study have included sessions relating to social skills training or a specific focus on ADHD. This could indicate an area in need of development for the next generation of parent management training programs and interventions. It might be difficult to motivate parents of children presenting with these profiles to continue with another intervention focusing on ADHD or social skills training after completion of a regular PMT program. An alternative is then to identify children belonging to these clusters and offer a more tailored intervention at once. One way to develop future PMT programs could therefore be to include optional modules targeting areas such as core ADHD-symptoms and prosocial behavior skills, when needed. Future research is needed to investigate whether adding or changing treatment modules can accomplish clinically significant improvement for children with profiles such as those in Cluster 3 and 5.

Another aim of this study was to evaluate whether treatment engagement differs depending on cluster profile and intervention offered. The results show that a larger number of families that were randomly assigned to the iComet than those randomized to the FCU chose not to start the intervention (39% versus 13%), but this was not associated with cluster membership. This indicates that many families randomized to iComet were not exposed to the intervention at all. Perhaps, many of the families that applied to the study were expecting a face-to-face intervention and therefore discontinued when they received iComet. In another study where the participants knew beforehand that the intervention given were through the internet, all participants started the intervention (i.e., 100%) and 83% were intervention completers (Enebrink et al., 2012; Högström et al., 2013). This can be compared to 36% intervention completers for iComet in this study. Another reason for the low rate of the completers in the iComet intervention might be inadequate presentation of its potential to be a helpful intervention. Lack of standardized text in the process of describing the treatment alternatives, combined with the context in which the study was conducted (i.e., mainly presenting professionals engaged in the FCU) probably contributed to a less optimal perception of the iComet by the families. In FCU 82% of those randomized to the intervention were considered treatment completers. A strength of FCU is that the first session of the intervention focuses on creating an alliance with parents, and the perception of the parents is that the treatment has started. Other variables that might influence parental engagement are that face-to-face meetings are booked in FCU, they have a “family therapist” in contact with them, and the parents have the impression that they work together with a therapist, compared to the fairly independent parental work that is expected in iComet.

Limitations

There are methodological constraints to this study that should be considered when interpreting the results. Despite intensive efforts, the drop-out rates were high. However, we did not find any significant differences in either demographic variables or baseline measurements between those who provided post-intervention data and those who chose not to. Assumptions such as missing at random (RAM) and extensive drop-out analysis can at best strengthen the internal validity of the study, and groups based on person-oriented analysis have not been randomly assigned to treatments. Thus, conclusions based on all of the analysis should be interpreted with caution, and in light of limited external validity of the study.

Throughout the analysis, only parental reports were used. Child and teacher reports might have been valuable for validating the parent reports, although these sources of information also have certain limitations. When analyzing the outcome of interventions (i.e., decrease in child problems and improved family climate) post-measurement data was used only from those families who completed at least one session of intervention. Including families that never started any of the treatments, into the analyses, would not inform us about the effect of the interventions specifically, but how the outcome looks for those who we intended to treat. This is basically what we found in the ITT analyses. The same pattern of results emerged when the analyses were re-run including all the subjects who were randomized (Intention to treat analysis: ITT). Precautions are also necessary when drawing conclusions about the effect sizes for different clusters and interventions, since the cluster sizes were very small due to the fact that five distinct clusters emerged when conducting the person-oriented cluster-analysis.

Conclusions

This study illustrates the value of distinguishing between different profiles of children. For most of the families included in this study both FCU and iComet were sufficient interventions and the child problem behaviors after intervention were within the non-clinical range. Two of the profiles however (Cluster 3 and 5) seemed to need further adaptations of interventions and/or supplemental interventions, in order to achieve sufficient results (i.e., to bring the post-treatment scores within normal range). Both of these two clusters had a high level of EBP at baseline. Cluster 3 children were also characterized by very low scores of Prosocial Behavior and thus might have made more improvement if social skills training had been part of the intervention. Common to both Clusters 3 and 5 were the high level of ADHD symptoms, indicating that a more specific module about ADHD or ADHD adjustments would have been appropriate to add for both these clusters.

All results should be interpreted with caution since there are several limitations with this study. Most importantly drop-out rates were high, many families were not exposed to all the sessions offered within each PMT-program and only parent reports were used. The analyzed clusters were also small (ranging from n = 4 to n = 23).

Author contributions

AG and PE designed the study and organized the data collection. AB participated in the data collection. AB drafted the first version of the manuscript. AB and AG carried out the statistical analyses. All the authors were actively involved in revising the manuscript. All authors also read and approved the final manuscript.

Declaration of interests

The authors’ declare that they have no conflict of interest.