A tailored cognitive behavioral program for juvenile justice-referred females at risk of substance use and delinquency: A pilot quasi-experimental trial.

This pilot quasi-experimental trial tested a gender-responsive cognitive behavioral group intervention with 87 court-involved female adolescents (5 juvenile courts) who were at indicated risk for substance use disorder. Participants in the intervention (n = 57) received twice weekly group sessions for 10 weeks (20 sessions) focused on building emotional, thought and behavior regulation skills and generalizing these skills to relationally-based scenarios (GOAL: Girls Only Active Learning). Youth in the control condition (n = 30) received services as usual, which included non-gender-specific aggression management training, individual counseling and no services. The GOAL program was found to be acceptable to youth and parents and feasible to implement within a juvenile court setting using skilled facilitators. Compared to services as usual, the program significantly and meaningfully reduced self-reported delinquent behavior (β = 0.84, p < 0.05) over 6 months, and exhibited trend level effects for reduced substance use (β = 0.40, p = 0.07). The program had mixed or no effects on family conflict and emotion regulation skills. These findings are discussed in light of treatment mechanisms and gender-responsive services.

RCT Entities:   Population Interventions Outcomes

Introduction

Sustained substance use for females is associated with increased risk for HIV infection, violent victimization and downstream risk for birth defects [1-3]. Preventing substance use disorders with effective programs for selective and indicated prevention populations can yield substantial savings in avoided medical, system and human costs and are a good investment for social service systems [4]. To date, prevention programs show mixed effects for females and males, and there are, as yet, no clear guidelines for what prevention program components are the most effective by gender. However, a number of female-specific programs shown to be effective at promoting health outcomes often include elements that emphasize empowerment and relationship skills. The current pilot study presents the feasibility and preliminary effectiveness of a female-specific prevention program that integrates empowerment, relational and cognitive behavioral skills. The program was developed for an indicated prevention population of female youth in contact with the justice system at moderate or high risk of recidivism, and was designed to work smoothly with court operations to promote sustainability.

Adolescence as critical intervention point in preventing substance use

Adolescence is a significant turning point for developing sustained, problematic substance use [5-7]. While most adolescents will not go on to develop substance use disorders, those who begin using substances early and show patterns of moderate to heavy use during this time are at higher risk for developing adult substance use disorders [8]. The prevalence of use jumps sharply as youth transition to adulthood, between the ages of 13 and 18 [6]. Efforts to prevent substance use disorders are also likely to prevent general delinquency [9, 10] and aggression [11], making these services a good investment for both health and justice systems [11, 12].

Cognitive behavioral interventions and gender

Cognitive behavioral approaches employ a shared framework for teaching skills related to managing emotions, challenging negative thoughts and problem-solving [13]. This approach has the strongest evidence of effectiveness for preventing and reducing substance use disorders [14-15], reducing adolescent aggression [16] and preventing adolescent delinquency [17]. In a meta-analytic review of substance use treatment programs for adolescents, Vaughn & Howard [14] rated cognitive behavioral therapy groups in the highest category for evidence (effect sizes greater than 0.20 in highly controlled designs). However, a nine-year follow-up study of substance use prevention programs using a CBT skills-based approach found that long term effects from these programs were only sustained for males [18]. Further, an earlier review of 47 general prevention programs by Fagan & Lindsey [19] found that 51% of programs varied in outcomes by gender. However, no consistent patterns emerged by program type that could account for these differences. A review of the delinquency intervention literature [20] found similarly mixed effects between genders with inconsistent patterns. To date, little is known about why programs for youth involved in the juvenile justice system demonstrate different outcomes by gender, but more effective approaches are clearly needed [19, 21].

Female-specific programming

Researchers and advocates have called for female-specific programs that blend both relational and cognitive behavioral approaches [22] given the strong evidence for CBT but inconsistent results for females. Differences in the etiology of substance use disorders also support the need for gender-specific models. Compared to males, females’ relationship with parents and conflict in the home is more strongly linked to substance use [23, 24]; aggressive behavior [25-27]; and delinquency [28]. Poor outcomes associated with family conflict are especially likely when females have higher sensitivity to emotional cues [29-33] and less assertive coping styles [23].

The predictors of substance use disorder for adolescent females suggest patterns of risk that may result from poor self-concept and low self-efficacy [34]. Consequently, effects of prevention programs for females are likely to be enhanced if they included content focused on building up perceived and actual efficacy. Indeed, examples of successful female-specific programming for other health targets suggest relationship-based programs focused on empowerment, referred to as social empowerment programs (SEP), are particularly well-received by females and are becoming increasingly popular in domestic and global health [35, 36]. These programs typically include educational sessions on physical and emotional safety, relationship building, social awareness, and may include mentoring, problem-solving and assertiveness skills. SEPs can be effective in improving sexual health, HIV prevention and reduced domestic violence for adult females [37-39], and so there is reason to expect this approach would benefit adolescents as well. However, little is known about the effect of a standalone SEP approach for adolescent substance use prevention. Some mixed findings in the literature, particularly null effects in high needs populations [35], suggest SEP may not be effective alone and a blend of empowerment and CBT in program design may yield more robust effects for indicated prevention programs.

Implementing prevention programs with court-based staff

Training juvenile probation officers to deliver prevention programming presents possible challenges. It is not clear that probation officers, hired and trained for a traditionally compliance-oriented job, can be feasibly trained to deliver second tier prevention services. The approach also raises some larger concerns about the wisdom of centering therapeutic services within courts, including the risk of retaining youth in court services who could otherwise be diverted into community services [40], and subjecting youth who are noncompliant with program services to higher levels of scrutiny [41]. Approaches using probation as the facilitators of prevention programming would need to be thoughtful about separating those roles. If done successfully, however, youth could be expected to benefit from a probation workforce more attuned to principles of positive youth development and therapeutic skill building [42]. Most studies of interventions involving court-referred youth study the impact of interventions delivered by providers external to court operations (e.g., [43, 44]). Very few published studies examine the feasibility of training internal court services staff (e.g., probation officers) to deliver interventions unless these approaches are framed as enhancing supervision services [45]. However, probation officers trained to conduct prevention services may be more likely to incorporate this framework into other areas of work, including their probation supervision approach [46, 47]. Further, using internal staff to provide prevention services may save costs by reducing the need for external contracting and increasing access to services.

The current study

Accordingly, we present feasibility and preliminary outcomes of a pilot study that included a controlled trial of a female-specific substance use disorder and delinquency prevention program. The program was designed to work sustainably within a juvenile court setting using existing staff (juvenile probation officers and contracted community providers) as the facilitators. To support usability, the program design team included probation officers along with a sexual abuse and trauma expert, a clinical psychologist, a justice programs quality assurance manager, and a youth behavioral health implementation researcher.

The program development process used a co-design, participatory approach [48]. Co-design is a method of program development that engages the intended systems in the development process to ensure fit and sustainability. The process views sustainability to be equal in value to effectiveness. The development process thus attempts to engage all relevant local expertise in order to build capacity around existing systems. Consequently, in the present study, the development process began with a review of an existing CBT program already operating in the local juvenile courts [17, 49] to determine which elements aligned with the principles of female development and empowerment and where adaptations should be made. The decision to include probation officers as facilitators was largely guided by the courts’ prior success in using probation officers as facilitators for a different CBT-based program [17]. The state quality assurance manager for this program served as a lead along with the first author in bringing together courts to discuss these adaptations, subsequently adding more content area expertise as the project evolved. The resulting program, Girls Only Active Learning (GOAL), looks significantly different from the original CBT program but has earned significant buy in and support from the state as a result of the development process (see Fig 1).

Fig 1

Program components.

GOAL is a 20 session, 10-week program for female youth that includes a parent education component. The CBT elements are introduced early in the program with sessions 1–10 focused on learning and practicing emotion identification and coping skills, cognitive restructuring, goal-setting, and assertive and problem-solving skills. Empowerment language is integrated throughout with an emphasis on “personal power.” This is particularly used in the latter half of the sessions (10–19) where participants discuss and practice using skills in relationship-based scenarios related to family, peer, and romantic relationship contexts. The theory of change behind the program is based on a social development model, theorizing that enhancing the capacity of adolescents to adopt healthy responses to relational conflict in these domains while promoting a future orientation will lead to fewer opportunities for conflict and substance use [50]. The home environment of the participants is addressed through weekly text messages to parents (or current guardian) that describe the weekly program topic along with brief psychoeducation about healthy female development and suggestions for reinforcing skills being learned in group.

The goal of this study was two-fold. First, we examined the feasibility of co-designing and embedding a new prevention program within juvenile courts and assessed feasibility by tracking the number of implemented programs, the individual attendance rates, and acceptability using qualitative feedback from facilitators, program participants, and parents. Second, we examined the preliminary effectiveness of the program as compared to usual care through 3- and 6-month assessments of primary and secondary health and behavior outcomes.

Method

Participants

Eligible participants included 136 females referred from juvenile probation programs from five counties between September 2014 and December 2015. Of the eligible sample, n = 101 provided consent and n = 87 completed at least one survey over the three data collection waves. All participants were female, with the following racial and ethnic groupings: White (47%), mixed ethnicity (26%), Black (8%), Latina (7%), American Indian (2%) or other (9%), which is roughly equivalent to the distribution of ethnicity of youth involved in juvenile courts in Washington State. Participants were between 12–17 years old; M[SD] = 15.15[1.72]. Inclusion criteria were: identified as female, were on probation, and had at least one of the following risks: moderate family strain/conflict; exposure to violence; poor school achievement; or antisocial peer involvement as measured by the court risk and needs assessment used in all juvenile courts in the state (Positive Achievement Change Tool, PACT) [51]. Youth were excluded if they had a diagnosable substance use disorder or serious emotional disturbance as these needs require more intensive interventions.

Procedures

This study employed a quasi-experimental design to assign participants to GOAL or services as usual by offering GOAL a limited number of times in each site and referring all eligible participants within this service window to GOAL and all eligible participants outside of this service window to usual programs, as described in more detail below (also see Fig 2). To work smoothly with the operations of the juvenile court, youth were not required to enroll in the research study in order to access GOAL. All eligible youth were referred to GOAL automatically when the program was available and to services as usual when it was not. Research recruitment occurred after enrollment in GOAL or services as usual.

Fig 2

Consort diagram.

Research eligibility was determined by the court probation counselors at each site who approached eligible participants about the study using a script and information sheet. Each site also designated a study liaison who reviewed eligibility and reached out to Probation Officers when an eligible youth was on their caseload. Once a probation officer received permission to share contact information from the youth, they securely faxed this information to the research group who followed up with caregivers and youth to explain the study and solicit assent/consent over the phone. Youth provided assent for their own participation and parents provided verbal consent for their child to participate and to provide feedback on weekly parent information messages. Participants had the option of receiving surveys over email, text, or through regular mail. The procedures were approved by the University of Washington Institutional Review Board.

Youth were assigned to either GOAL or services as usual based on the availability of the GOAL program which was strategically only offered twice a year in each court. Courts were instructed to refer all study eligible youth to GOAL during the period of GOAL enrollment. Over the study time period, the courts referred n = 123 youth to GOAL. Youth receiving treatment as usual were referred to services based on eligibility for other court and community programs already available to court-involved youth. For both GOAL and usual services, referral from probation officers was considered part of the court order and the services was considered completed if a youth did not miss more than three sessions. We did not ask courts to provide the study with information about the total number of eligible youth receiving services as usual.

Youth were surveyed at three points during the study: baseline, 3 months, and 6 months from study enrollment. These timeframes were established to assess pre-program, immediate post-program and sustained effects for those participating in GOAL compared to usual services. Parents with a daughter enrolled in GOAL were surveyed once after the program ended (second wave) to assess acceptability. Youth and parents were given the option to complete the surveys online or by mail. Youth received $10, $15 and $20 for returned surveys. Parents received $10 for returned surveys.

GOAL implementation

Group facilitators were probation officers (50%), contracted-counselors (25%) and community youth workers (25%) trained by the study team to conduct the GOAL groups. Each site recruited individuals who fit the criteria below, and were typically currently delivering programs through the court or with court-referred youth in the community. A minimum of two facilitators were trained for each juvenile court. To be eligible, facilitators had to have either previous experience leading skills-based treatment groups with justice-involved youth, or previous experience working in therapeutic environments with adolescent youth. All probation officers in the study had previous experience leading an existing court-based therapeutic group, Aggression Replacement Training (ART). Other facilitators had a mix of experience with either facilitating ART or other therapeutic groups or services. A master’s degree or two years post-bachelor equivalent experience was required to facilitate groups. In half of the sites (3), programs were conducted in rooms in the juvenile court facilities, in the other half (3), programs were conducted offsite in community settings.

Training was provided by two PhD-level, trained mental health clinicians with experience conducting therapeutic groups with adolescents. The training included 10 hours of active instruction and role play over two days and weekly consultation calls through the first round of facilitating the program (10 consultation calls). Throughout implementation of GOAL, facilitators completed self-assessment forms for each session on at least weekly basis. The forms assessed adherence to the intervention manual, and challenges and successes in implementation. The consultation calls focused on reviewing self-assessment forms, problem-solving issues that arose in implementation, and planning for upcoming sessions. Finally, facilitators were asked to videotape two sessions, which were reviewed by the trainers.

Measures

Measures of feasibility

Feasibility was measured by rate of program completion and facilitator fidelity. Program completion was compared to a benchmark of program completion for other court-based services and the number of youth enrolled for each group. Fidelity was measured via facilitator self-report. For each session, one of the facilitators reported on “how closely [they followed] the written curriculum (the education, activities, order/sequencing, discussion prompts, etc.)” on a scale of 1 = Exactly, 2 = Mostly, missed a couple of things, 3 = Only covered a few areas, and 4 = Didn’t really follow it at all. They were also asked to describe and explain any modifications that were made via open-ended responses.

Measures of acceptability

Acceptability was measured with a study-developed satisfaction survey with Likert scale items and open-ended responses for youth participating in GOAL and their parents. Parent satisfaction was measured with six scaled items developed by the researchers with a range of 1–5 (strongly agree) to measure parents’ sense of being informed about the program and whether the program helped their child. Youth satisfaction was assessed with six scaled items with a range of 1–7 (strongly agree) developed by the researchers which included items about general satisfaction, whether the respondent learned new skills and whether the respondent felt treated with respect, alpha = 0.72. Satisfaction surveys were obtained from parents and youth at 3 months.

Substance use and consequences

Substance use and consequences were measured with items taken from the Drug Involvement Scale for Adolescents (DISA) [52]. DISA was developed to assess the context and consequences of substance use. Use is measured on a 6-point scale (Not at all–Every day) for the last month and includes 12 controlled substances. Consequences of substance use is measured on a 7-point scale (Not at all–Six or more), including missing an assignment in class, being arrested, stealing, damaging property, late for work/class. Items were summed to create a frequency score across all substance types. Item reliability was good for baseline and 6-month follow-up (alphas of 0.87 and 0.83, respectively).

Risk behavior

Risk behavior was measured with the Risky Behaviors subscale of the High School Questionnaire [53], a reliable measure of youth offending. Youth responded to four questions on a 7-point scale (Not at all–Six or more) for the past month. Items include getting in a physical fight, threatening to hurt others, shoplifting, and hitting something. The risk behaviors were summed to create a frequency score. In the current study, item reliability was acceptable for baseline and 6 months (alphas of 0.70 and 0.73, respectively).

Family climate

Family climate was measured with the Communities that Care Youth Survey (CTCYS) [54]. This 8-item survey includes questions about house rules, parental support, and conflict among family members on a 5-point scale (Almost never–All of the time). Family climate in the home is predictive of later problem substance use and delinquency. In the current study, item reliability was acceptable for baseline and 6 months (alphas of 0.84 and 0.73, respectively).

Emotion dysregulation

Emotion regulation was measured with the Difficulties in Emotional Regulation Scale (DERS) [55]. The DERS assesses perception and beliefs about difficulties in emotional regulation and beliefs about emotional control. It is scored on a 5-point scale (Almost never–Almost always). In the current study, item reliability for the total scale was high for baseline and 6 months (alphas of 0.93 and 0.94, respectively).

Quantitative data analyses and missing data

Missing data within surveys was low, but there was notable missingness between waves of data collection, with only 26 youth (30%) completing all three waves. All but 7 youth completed one survey at baseline and/or 3 months (92%). Missingness was highest at the third wave (54%), compared to wave 1 (24%) and wave 2 (32%). The youth completing data only at 6 months were youth the study team had the most difficulty engaging and it cannot be assumed that these 7 youth had data missing at random. Consequently, these youth were removed from the outcome analyses, resulting in an analytic sample of n = 80. Scores were then analyzed with linear regression models using full information maximum-likelihood estimation using Mplus7 to estimate over missing data [56].

Results

Descriptive information

At the time of enrollment in the study, 57% of the sample had used an illicit substance in the past 30 days and 41% reported some negative consequence as a result of use. Emotion dysregulation was comparable to a population-based sample of high school students, M[SD] = 2.74[0.65]. Nearly half (45%) of youth reported having family conflict during “at least half” of their parent interactions. Participants in the comparison condition (i.e., services as usual) accessed a variety of services in the treatment as usual (TAU) condition. Twelve (50%) participants declined to respond to the question. Of the respondents, 4 received no services (14%), 5 received substance use disorder treatment (17%), 2 received ART (7%), 2 received mental health counseling (7%), and one received a youth development program (3%). Participants in the two conditions were balanced in age, race/ethnicity and measured baseline characteristics, confirming the design for condition assignment was adequate in balancing observed characteristics.

Feasibility

Courts were largely able to meet the target group size with an average of 8 youth enrolled per group (range 6–13). Of those referred to GOAL and the study (n = 66), 9 youth (14%) did not complete the program, which outperforms the benchmark rate for youths’ typical attendance in court-based group programming in Washington State (36% non-completion). A one proportion test of significance found this to be a significant improvement in retention when compared to the existing rate, z = 3.89, p < .0001.

Across all sessions and facilitators, the mean self-reported therapist fidelity was 1.64 (range = 1 to 2), indicating that facilitators followed the manual “exactly” or closely. A review of the qualitative descriptions and explanations for modifications suggested that the modifications were minor, and consisted primarily of adaptations (e.g., changing examples to enhance relevance for females), or omission of a session component (e.g., journaling).

Acceptability

Parent surveys

Parent acceptability of the weekly messages and of the program was high overall. However, the response rate for the parent surveys was 43% (25/57), so responses may indicate some bias towards extreme or favorable views. Responses from the six questions ranged from a mean of 3.86 to a high of 4.50 on a 5-point scale. On average, parents strongly agreed that they would recommend the program to other parents, M[SD] = 4.5[0.63]. The updates were uniformly seen as helpful, M[SD] = 4.21[0.75]. Parents also noted the value of the program in open-ended responses: “The weekly updates as to what would be addressed so I could speak to my teen about it. The updates were very thorough.” “The concepts presented are directly relevant to the challenges [my daughter] is facing.”

Youth surveys

Just over half of the youth referred to GOAL completed satisfaction surveys, n = 33 (58%). Those who responded had slightly lower reported substance use than youth who did not complete satisfaction surveys, M[SD] = 14.31[3.44]; M[SD] = 16.71[4.34] as well as lower reported risk behaviors, M[SD] = 7.56[4.79]; M[SD] = 8.52[6.12]. The average response to each of the six questions assessing GOAL satisfaction were “agree” or higher. The item with the most variation in response was whether the program helped, M[SD] = 5.52[1.50], with one youth reporting “strongly disagree” and another reporting “somewhat disagree” in addition to 75% reporting “somewhat agree” or higher. Responses included: “Getting to know my emotions more and getting to know how other people react to things and getting to know their personality!” “How we got to do the talking, not just the instructors.” “I wasn’t judged negatively.” “I liked talking to other youth with similar problems.” When asked what could be improved, youth reported nothing (n = 8), listening more (n = 5), and adding more skills “because the skills helped a lot.”

Substance use

The linear model predicting substance use at 3 months allowed age to correlate with baseline substance use and included age and baseline substance use as covariates, CFI/TLI = 0.99/0.99, SRMR = 0.06, RMSEA = 0.03. GOAL was negatively associated with substance use at 3 months, M[SE] = -1.98[1.03], p = 0.06, with a small effect, β = 0.15, but did not reach statistical significance (see Table 1).

Table 1

Estimated means and outcome effects for GOAL and TAU at 3 and 6 months.

	Baseline			3 Months		6 Months		Intervention Effects (3 mos)			Intervention Effects (6 mos)
	N	Mean	SD	Mean	SD	Mean	SD	B	SE	β	B	SE	β
Substance Use
GOAL	54.00	15.75	0.70	14.77	0.63	14.83	0.69	-1.98	1.03	0.15	-2.28	1.27	0.40
TAU	26.00	16.66	0.85	17.15	1.00	18.56	0.90
Behavior
GOAL	54.00	7.58	0.64	6.69	0.66	5.37	0.53	-1.08	1.06	0.48	-1.99	0.80	0.84
TAU	26.00	8.62	1.20	7.90	0.98	8.49	1.04
Dysregulation
GOAL	54.00	2.74	0.09	2.63	0.11	2.72	0.14	0.00	0.14	0.18	-0.01	0.19	0.10
TAU	26.00	2.62	0.17	2.59	0.16	2.69	0.20
Family climate
GOAL	54.00	2.78	0.13	2.48	0.11	2.49	0.14	-0.28	0.19	0.31	-0.15	0.50	0.10
TAU	26.00	2.89	0.23	2.78	0.19	2.65	0.18

Notes. Bold = p < .05. For parsimony, estimated baseline means are taken from the three month models. Baseline means for the six month models differed negligibly from the three month models and are available from the authors. Effect sizes were calculated as the mean difference of the change by condition divided by the standard deviation of the dependent variable for each model.

The best fitting model for predicting substance use at 6 months included age and substance use at baseline as covariates predicting substance use at 6 months, CFI/TLI = 0.92/0.86, SRMR = 0.08, RMSEA = 0.11. GOAL was negatively associated with substance use, M[SE] = -2.28[1.27], p = 0.07, with a medium effect, β = 0.40, and did not reach statistical significance.

Risky behavior

The model predicting behavior at 3 months allowed age to correlate with behaviors at baseline and included age and baseline behavior, CFI/TLI = 0.91/0.87, SRMR = 0.06, RMSEA = 0.06. GOAL was negatively but not significantly associated with behavior at 3 months, M[SE] = -1.08[1.06], p = ns, with a medium effect, β = 0.48.

The model predicting risky behavior at 6 months allowed risky behavior at baseline and 3 months to correlate, along with age as covariates in the model, CFI/TLI = 0.97/0.92, SRMR = 0.06, RMSEA = 0.08. GOAL was negatively and significantly associated with risky behavior at 6 months, M[SE] = -1.99[0.80], p < .05, with a strong effect, β = 0.84.

The model predicting emotion dysregulation (ED) at 3 months allowed age to correlate with ED at baseline and included age and ED at baseline as covariates, CFI/TLI = 0.99/0.99, SRMR = 0.05, RMSEA = 0.04. GOAL was not significantly related to ED at 3 months, M[SE] = 0.003[0.14], p = ns, and had a small effect, β = 0.18.

The model predicting ED at 6 months allowed age and ED at baseline to correlate and included age and ED at baseline as covariates, CFI/TLI = 0.99/0.99, SRMR = 0.05, RMSEA = 0.04. GOAL was not significantly associated with ED at 6 months, M[SE] = -0.01[0.19], p = ns, and had a small effect, β = 0.10.

The model predicting family conflict (FC) at 3 months included baseline FC as a covariate, CFI/TLI = 1.0/1.0, SRMR = 0.02, RMSEA = 0.00. GOAL was negatively but not significantly associated with family climate at 3 months, M[SE] = -0.28[0.19], p = 0.14, and had a medium effect, β = 0.31.

The model predicting family climate at 6 months allowed age to correlate with family climate at baseline and included age and baseline family climate as covariates, CFI/TLI = 1.0/1.12, SRMR = 0.004, RMSEA = 0.00. GOAL was negatively but not significantly associated with family climate at 6 months, M[SE] = -0.15[0.19], ES = 0.10, p = ns, and had a small effect, β = 0.10.

Discussion

The purpose of the study was to assess the acceptability, feasibility, and preliminary effectiveness of a female-specific, CBT program for reducing substance use risk and delinquency for youth in contact with the justice system. Taken together, results from this quasi-experimental pilot study suggest that GOAL is a promising intervention for targeting the indicated treatment needs of this group.

Study findings suggest the program can be feasibly implemented by probation and court-contracted group facilitators who have previous experience running therapeutic groups. The fidelity scores indicated high adherence to the program with only minor adaptations and attendance exceeded female attendance rates for other court-based programming. Feedback from parents and youth who responded to the program satisfaction surveys indicated very high acceptability but these appear to underrepresent youth at higher risk for substance use and delinquency. Consequently, it is not clear that higher risk youth viewed the programs as helpful so results can only be assumed to hold for females at moderate, but perhaps, not highest risk for ongoing substance use. GOAL was also very well-received by a little over half of the youth responding the study survey, which represents just under 30% of all of the youth who participated in GOAL during the study timeframe. Missing responses did not appear to be random, as youth not responding to the satisfaction survey had more risk factors at the baseline assessment. Consequently, the program appears to be a good fit for a number of youth but this may not hold as youths’ needs increase.

Taken together, the female-specific program seems well-suited for courts with either court-contracted staff or probation officers who are already familiar with running therapeutic skills groups. The model presumes a baseline level of therapeutic knowledge and skill that may not be as readily present in other juvenile court systems. Results regarding the preliminary effectiveness of GOAL were promising, with notable findings for reducing risk behaviors associated with delinquency. Specifically, there was a statistically significant, large effect for reduced risk behaviors at the 6-month follow up for the GOAL group compared to controls. For 3- and 6-month substance use, the mean differences were in the expected direction, and the results showed trend-level effects (p < .10) for GOAL compared to TAU. For 3-month risky behavior, 3- and 6-month family climate, and 3- and 6-month emotional dysregulation, no significant differences were observed between the intervention conditions. The findings suggest that GOAL may be effective in reducing the primary targets of delinquency and substance use but the lack of observed change in emotion regulation suggests the mechanisms of change may be different than hypothesized.

The mechanisms of change for substance use and delinquency prevention interventions are not well-understood. In the social development model, the presumed effects lie in the success of the model in changing the social and internal incentives to participate in prosocial rather than antisocial activities [51]. The theorized mechanism of effect of a CBT approach is the expected increase in consequential thinking in order to help youth understand that short term choices have long term consequences. This is then expected to build youth motivation to participate in prosocial activities and mitigate the influence peers will otherwise have on decision-making in the middle adolescent years. Consequently, the model assumes that success in the intervention effects should be mediated by the youth’s successful use of these skills but this was not observed in our study. We did not measure problem-solving skills with a standardized measure, although qualitative responses from youth indicated some support for practicing problem-solving skills in role plays. This provides some qualitative support that the rehearsal and building of cognitive and behavioral skills had some effect on outcomes.

An alternative model of substance use and delinquency intervention using mindfulness-based intervention (MBI) proposes that decisions to engage in risk behaviors can be mitigated by an increased ability to resist distressing emotions [57]. Mindfulness-based interventions (MBI) attempt to reduce negative thoughts and emotions by helping the individual focus on present sensation [58, 59]. Interventions employing mindfulness in isolation or as part of a multicomponent curriculum are based on the theory that individuals are building their ability to manage distress through direct manipulation of their physical states, which leads to less perceived urgency and impulsivity [60]. This capacity is also observed to affect other areas of health and behavior as individuals gain more insight into the relationship between mind and body functioning [61]. We would expect to see some change in emotion regulation skills through this mechanism of action in GOAL, which did include skills training around mindfulness and emotion regulation. The lack of an observed effect within or between groups in the study could be due to insufficient dosage, lack of adequate skills transfer or lack of specificity in measurement. Also, the scale used to measure emotion regulation, the Difficulties in Emotional Regulation Scale (DERS), is a multicomponent scale. While the total score demonstrates the best predictive capacity in psychological studies [62], it is possible that the intervention exerted effects on more specific skill areas than detected by the total DERS score. Our sample was not sufficiently large to power analyses to test this theory and would need to be replicated by a larger study.

Limitations

This pilot study was limited by a relatively small sample size for detecting effects in a sample in which the majority of youth in the comparison condition were participating in other therapeutic programs. As a pilot, the study had limited resources to track down youth who did not respond to the surveys after email and phone contact. Because the study was focused on rolling out a codesigned program across multiple sites, we focused resources on maximizing the reach of the program rather than improving the survey response rate. While the sampling can be considered missing at random, and robust estimators were used to estimate over missingness, it is not possible to completely control for bias that may have impacted the outcomes.

The current evaluation will need to be replicated with a larger sample. A larger study will also provide sufficient power to model nested effects for youth outcomes within facilitator characteristics. Youth in the study were assigned to GOAL or control conditions using a staggered model that appeared to be successful in balancing group differences but was not random. Further, youth only came to the attention of the study team following referral from the courts and potential differences in referred and non-referred youth are not available to confirm the generalizability of the findings to all potentially eligible participants. The study recruited fewer youth from the treatment as usual group compared to GOAL, and we cannot be sure that this did not introduce some bias into the sampling.

While parent acceptability was high, the results are limited to parents being willing to respond to the parent survey. Parents and youth not responding to the survey may have been less motivated to respond because they found less value in the program. Replication of parent acceptability is needed with a higher participant response.

Finally, our measure of fidelity only assessed adherence to intervention elements via facilitator self-report. Previous studies have shown that front-line providers are accurate reporters of their adherence, and self-reported adherence significantly predicts response to intervention [63, 64]. Nevertheless, future research should incorporate other dimensions of fidelity that may impact participant outcomes, such as facilitator competence. Similarly, self-reported delinquent behavior and substance use outcomes, while shown to be reliable indicators in other studies [62, 65], may underestimate the true incidence of these behaviors and could reflect bias in reporting.

Missingness between waves of data collection was notable and while full information maximum likelihood is robust up to 50% missingness [66], replication is needed to ensure study results are not due to biases introduced by the estimated models.

Conclusions

Overall, this pilot demonstrated the feasibility and acceptability of a substance use and delinquency prevention program for females involved with the justice system. The pilot additionally demonstrated promising results for reduced delinquency compared to other, non gender-specific services and trend level effects for reduced substance use. The study failed to find effects on reducing overall emotion regulation capacity, one of the hypothesized mechanisms of action. Additional study is necessary to identify whether the effects were limited to lower risk youth and if the program theory of change is mis-specified or if the measures used in the study were not sensitive enough to detect the hypothesized mechanisms of change.

12 Aug 2019

PONE-D-19-16985

A tailored cognitive behavioral program for juvenile justice-referred females at risk of substance use and delinquency: A pilot quasi-experimental trial

PLOS ONE

Dear Dr. Walker,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We would appreciate receiving your revised manuscript by Sep 26 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Thach Duc Tran, M.Sc., Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

1. Thank you for including your competing interests statement; "I have read the journal's policy and the authors of this manuscript have the following competing interests: S.C.W., L.B., C.H. and M.D. are listed as co-developers on a University based license for the distribution of the GOAL curriculum (patent pending)."

We note that you have a patent relating to material pertinent to this article. Please provide an amended statement of Competing Interests to declare this patent (with details including name and number), along with any other relevant declarations relating to employment, consultancy, patents, products in development or modified products etc. Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

This information should be included in your cover letter; we will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

2. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Authors should describe the effect size results, especially since this is a small sample and the effect size results help interpret the effectiveness of the intervention.

Authors should review more literature relevant to female specific interventions--this is a growing field and placing this study within the existing literature can be done better.

Reviewer #2: This is a review of “A tailored cognitive behavioral program for juvenile justice-referred females at risk of substance use and delinquency: A pilot quasi-experimental trial.” As a licensed clinical psychologist who contracts to supervise psychology graduate students delivering an intervention in a youth court system I was initially very interested in this project. I think the topic of trying to test what is both effective and feasible for high at-risk youths like those in juvenile justice is an important goal but felt that this study as it is now presented fell short of providing information that is practically useful. I recognize that it is a pilot study, but nonetheless I have some concerns about both the way the information is presented in the manuscript and how the results would really provide actionable information to other researchers or court personnel. My comments below are mostly in order of the document.

The authors should consider updating terms like “substance abuse” and “dependence”.

This intervention seems like a lot of extra work for POs and they are not typically appropriately trained for this kind of thing. One also wonders about dual relationships. I think it really benefits the kids I work with to have a therapist who is not also dealing with their probation compliance. I actually argued against having a PO with a master’s degree in counseling provide the clinical services for this exact reason- you can’t expect to wear both hats and have it go well. For what it’s worth, the immediate reaction of every (psychology) colleague I mentioned the counselor-PO thing to was essentially “oh my God, no” so I wonder about the wisdom of this intervention, even though the authors were careful to select qualified people and had buy-in from the local system.

How many kids were in actual parental custody?

On p. 5 there is a citation listed as (author, year) rather than the numbering scheme used elsewhere.

Is there a citation for “probation officers who are familiar with principles of prevention are likely to incorporate this framework into other areas of work, including their probation supervision approach and participation in organizational policymaking” (p. 5)?

The authors claim that concerns about whether POs can or should deliver the intervention are “outside the scope of the current study” and I would insist that they are not. Both the competence of these non-mental health professionals, the effects of their having an apparent dual relationship with the participants (counselor and PO), and how well this worked on a macro level (did they have time to do this and the rest of their job well?) seem like pretty important points as far as the utility of this study.

The n of 87 doesn’t seem to match the CONSORT chart. I guess that is everyone who did at least one survey? I wonder if the authors should be more clear about how many youths did (or did not) do which aspects of the study up front.

How were emotion dysregulation and family conflict measured (p. 8)? Means are given before measures info.

Does “the court risk and needs assessment” have a name (p. 8)?

Why were so many more girls referred to GOALS than TAU?

The “participant self-report at 3and 6 months follow up” on p. 9 is 1) confusing as described and 2) there is a space needed between 3 and ‘and’ . 12 participants is 50% of what, exactly? And what questions were being asked?

Why the mix of group leaders? The intro is written like it will be POs but then it is only half POs. And I know this is a small sample but were there any effects of leader type? And POs who have 2 years of groups/MH experience and/or a master’s degree seems like a pretty special bunch. How does this program translate to any other jurisdiction?

The way feasibility was measured here is not really in line with the spirit of determining actual feasibility. People completed the program is certainly a component of that, but so are things like whether it was overly burdensome on staff, etc. Was group leader satisfaction assessed?

Why was there so much missing data? This is a tough population, but I’m still surprised at how high it is. Do the authors have an explanation?

How was the survey given to the participants? There isn’t really a procedure section for the surveys, and the GOAL group itself could be explained in more detail in a paper ostensibly about how its implementation went. Was acceptability collected at the same time as other data? How many youths actually provided this information? It’s less than half of parents and general follow-up with youths has a lot of non-responses so this should be specified.

Were kids mandated to attend treatment? What were the consequences of not attending or completing? What were to reasons for non-completion of poor attendance?

p.15- “each of the six question” question should be plural.

The youth response that an improvement would be “more food” suggests that food was provided but this is not mentioned anywhere.

I am wondering if this is an initial paper reporting some early results of a larger project, but in any case one wonders about statistical power in this small of a sample. Was there enough power to detect effects in the linear regressions with multiple IVs?

Something like a latent growth model might make more sense than separate linear models for two time points, though I suppose the sample is too small (and has too much missing data) for that to make sense.

The discussion overstates the findings a bit given the amount of missing data. The intervention doesn’t appear to have been unacceptable to the people who provided feedback, but that’s the minority of youths and parents so it’s hard to say how acceptable it really is.

In the discussion the authors should probably focus on effect sizes and more clearly discuss issues of sample size and statistical power rather than focus on significance.

If the authors are going to propose problem-solving as a potential mechanism, they should probably provide those (qualitative) data.

Similarly, suggesting that the facets of the DERS may have had more information than the mean of for all items (is that what the score is? Because that’s unusual, the measure has scoring guidelines) begs the question or why don’t the authors look at the facets?

The self-report nature of the outcome variables (for information that might easily be reported in a biased manner for a number of reasons) should also be mentioned as a limitation.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

26 Sep 2019

Response to Reviewers

Ref: PONE-D-19-16985 entitled “A tailored cognitive behavioral program for juvenile justice-referred females at risk of substance use and delinquency: A pilot quasi-experimental trial”

We thank the editor and both reviewers for the thorough read and helpful suggestions. We have made revisions to the manuscript to address each concern and provide details below explaining where changes can be found in the revised version.

Editor:

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at: http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf

Response:

We have made formatting edits to comply with PLOS ONE’s style requirements. Thank you.

2. We note that you have a patent relating to material pertinent to this article. Please provide an amended statement of Competing Interests to declare this patent (with details including name and number), along with any other relevant declarations relating to employment, consultancy, patents, products in development or modified products etc. Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared. This information should be included in your cover letter; we will change the online submission form on your behalf. Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

Response: We revised our patent statement. We have a statement of invention filed with the National Institute of Drug Abuse and a license with our University but not a patent. We added a statement to the cover letter indicating which authors are listed as inventors for the federal grant and in the license.

3. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

Response: We will upload and obtain a DOI for making the data available if the paper is accepted, thank you.

Reviewer #1:

4. Authors should describe the effect size results, especially since this is a small sample and the effect size results help interpret the effectiveness of the intervention.

Response:

Thank you. The effect sizes were reported and identified in the study as ES (Effect Size) statistics which are standardized regression coefficients (standardized beta) and interpreted the same as Cohen d (<.20 small, .30-.70 medium, >.80 large). We have now relabeled them so they can be more easily recognized as standardized beta coefficients (pp. 17-19).

5. Authors should review more literature relevant to female specific interventions--this is a growing field and placing this study within the existing literature can be done better.

Response:

We appreciate this comment and added more description and references of female-specific programs relevant to our area of study in this section (pp. 5).

Reviewer #2:

6. The authors should consider updating terms like “substance abuse” and “dependence”

Response: The terms “substance abuse” and “dependence” have been changed to “substance use disorder” or “problem substance use” in the revised manuscript. (e.g. pp. 2).

7. This intervention seems like a lot of extra work for POs and they are not typically appropriately trained for this kind of thing. One also wonders about dual relationships. I think it really benefits the kids I work with to have a therapist who is not also dealing with their probation compliance. I actually argued against having a PO with a master’s degree in counseling provide the clinical services for this exact reason- you can’t expect to wear both hats and have it go well. For what it’s worth, the immediate reaction of every (psychology) colleague I mentioned the counselor-PO thing to was essentially “oh my God, no” so I wonder about the wisdom of this intervention, even though the authors were careful to select qualified people and had buy-in from the local system.

Response:

We appreciate the challenges involved with having Probation Officers deliver therapeutically oriented services and have attempted to capture issues of acceptability and feasibility in the study. In the sites involved in the study, many of the PO’s were already providing group-based prevention programming. Using PO’s was one of the decisions made in the design process by the participatory team, which included system leaders. We have added this information to clarify (pp. 7). In addition, care was taken to ensure youth were not assigned to groups facilitated by their own PO and the intent was not to train PO as clinicians for their own probationers. We add this description in the text (pp. 6).

8. How many kids were in actual parental custody?

Response:

All of the youth were in parental custody during the course of the study.

9. On p. 5 there is a citation listed as (author, year) rather than the numbering scheme used elsewhere.

Response:

This was corrected, thank you (pp. 5).

10. Is there a citation for “probation officers who are familiar with principles of prevention are likely to incorporate this framework into other areas of work, including their probation supervision approach and participation in organizational policymaking” (p. 5)?

Response:

Thank you for pointing out that this section could benefit from more literature support. We added two citations for this sentence: Charles Schwalbe’s 2012 study of the different approaches probation officers take in supervision with youth (social work oriented vs. compliance oriented) and Katherine Schwartz’s 2017 study looking at the use of motivational strategies to encourage youth success while on probation (pp. 6).

11. The authors claim that concerns about whether POs can or should deliver the intervention are “outside the scope of the current study” and I would insist that they are not. Both the competence of these non-mental health professionals, the effects of their having an apparent dual relationship with the participants (counselor and PO), and how well this worked on a macro level (did they have time to do this and the rest of their job well?) seem like pretty important points as far as the utility of this study.

Response:

We clarified that the scope of the study was to assess acceptability and feasibility of the program, which we think bears on whether POs can be feasibly trained to deliver the intervention. (pp. 8).We added some language in the discussion to note that this does not mean this should be the first option for all or most youth. (pp. 20).

12. The n of 87 doesn’t seem to match the CONSORT chart. I guess that is everyone who did at least one survey? I wonder if the authors should be more clear about how many youths did (or did not) do which aspects of the study up front.

Response:

Thank you, we clarify that the sample included 136 girls of which 87 ended up completing at least one survey (pp. 8).

13. How were emotion dysregulation and family conflict measured (p. 8)? Means are given before measures info

Response:

Thank you, we moved the description of baseline needs to the first paragraph in the Results section (pp. 15).

14. Does “the court risk and needs assessment” have a name (p. 8)?

Response:

We added the title of the assessment, the Positive Achievement Change Tool (PACT) to the description (pp. 9).

15. Why were so many more girls referred to GOALS than TAU?

Response:

We added language to clarify that as a real world research design, participants were not referred to an intervention after enrollment in the study. Youth were referred to programming and then recruited to be in the study. We add language to suggest that GOAL recruitment was likely higher because both the youth’s PO and GOAL facilitator was likely to give them info about the study, whereas only PO’s referred youth in the SAU condition (pp. 9).

16. The “participant self-report at 3and 6 months follow up” on p. 9 is 1) confusing as described and 2) there is a space needed between 3 and ‘and’ . 12 participants is 50% of what, exactly? And what questions were being asked?

Response:

Thank you. In rereading this section, we agree that including information about the breakdown of services received in SAU is confusing as the section is intending to describe the group assignment process. We moved this information to the result section, and have corrected the typo (pp. 15).

17. Why the mix of group leaders? The intro is written like it will be POs but then it is only half POs. And I know this is a small sample but were there any effects of leader type? And POs who have 2 years of groups/MH experience and/or a master’s degree seems like a pretty special bunch. How does this program translate to any other jurisdiction?

Response:

We added language to clarify that each site independently recruited the staff to be trained to implement the program. The staff needed to meet the criteria and were typically individuals already delivering therapeutic programming through the court or in the community with court referred youth. (pp. 11).

18. The way feasibility was measured here is not really in line with the spirit of determining actual feasibility. People completed the program is certainly a component of that, but so are things like whether it was overly burdensome on staff, etc. Was group leader satisfaction assessed?

Response:

To provide a more complete picture of feasibility, we moved the self-reported facilitator fidelity into this section. (pp. 12, 15). Our team also did an in-depth qualitative study of facilitator feedback on the curriculum, but we do not have the space to report those results in this paper and they were consistent with the fidelity scores (e.g., the facilitator were able to implement and found the program, overall, to be helpful and useful). The reviewer comments are helping us recognize that readers of the paper may have a number of questions about the implementation of this program using probation officers. We attempt to address this in a number of places to clarify the limitations and possibilities of this approach (e.g. pp. 20).

19. Why was there so much missing data? This is a tough population, but I’m still surprised at how high it is. Do the authors have an explanation?

Response:

The missing data largely reflects the funding level (R21) of the project as a pilot. We had limited funds to follow up with youth who did not complete the surveys after email/phone contact. Our goal was to obtain enough data to examine promising effects, but would need replication in order to confirm efficacy. We added a statement to this effect in the limitations section (pp. 23).

20. How was the survey given to the participants? There isn’t really a procedure section for the surveys, and the GOAL group itself could be explained in more detail in a paper ostensibly about how its implementation went. Was acceptability collected at the same time as other data? How many youths actually provided this information? It’s less than half of parents and general follow-up with youths has a lot of non-responses so this should be specified.

Response:

Thank you for these prompts. We added more information on survey procedures and timing in the methods section under procedures (pp. 9-10) and measures (pp. 13).

21. Were kids mandated to attend treatment? What were the consequences of not attending or completing? What were to reasons for non-completion of poor attendance?

Response:

We added language in the GOAL implementation section to clarify that referral to GOAL was similar to other programs referred through probation in that it was considered part of the court order (pp. 11).

22. p.15- “each of the six question” question should be plural.

Response:

Thank you, this was changed. (pp. 17).

23. The youth response that an improvement would be “more food” suggests that food was provided but this is not mentioned anywhere.

Response:

Some facilitators of GOAL groups provided light snacks, but this was not considered essential to the program model. To reduce confusion, we omitted this feedback in the results (pp. 17).

24. I am wondering if this is an initial paper reporting some early results of a larger project, but in any case one wonders about statistical power in this small of a sample. Was there enough power to detect effects in the linear regressions with multiple IVs?

Response:

Thank you. As per the other reviewer comments as well, we make the reporting of effect sizes more clear and note the meaningfully large effects (e.g. pp. 17-19). We also more clearly note that this was a pilot study. (pp. 6).

25. Something like a latent growth model might make more sense than separate linear models for two time points, though I suppose the sample is too small (and has too much missing data) for that to make sense.

Response:

Yes, we agree, in this case the sample is much too small for latent growth modeling.

26. The discussion overstates the findings a bit given the amount of missing data. The intervention doesn’t appear to have been unacceptable to the people who provided feedback, but that’s the minority of youths and parents so it’s hard to say how acceptable it really is.

Response:

Thank you, we edited the language to reflect the nuance in the findings, particularly in regards to the missing satisfaction data for youth (pp. 20).

27. In the discussion the authors should probably focus on effect sizes and more clearly discuss issues of sample size and statistical power rather than focus on significance.

Response:

We agree and added that information (pp. 21, 22).

28. If the authors are going to propose problem-solving as a potential mechanism, they should probably provide those (qualitative) data.

Response: We reported some of the qualitative feedback from the youth on the value of learning and practicing skills in the results section (pp. 17). We changed the term “broad support” to “some support” to more directly reflect the data reported in this paper (pp. 21).

29. Similarly, suggesting that the facets of the DERS may have had more information than the mean of for all items (is that what the score is? Because that’s unusual, the measure has scoring guidelines) begs the question or why don’t the authors look at the facets

Response:

Our understanding (e.g., Hallion et al., 2018) is that the total score has the best psychometric properties for predicting psychological disorders. Our point in the discussion was intended to convey that the effect of the intervention may have had more specific effects not detected by the total score. We modified the language to clarify (pp. 22).

30. The self-report nature of the outcome variables (for information that might easily be reported in a biased manner for a number of reasons) should also be mentioned as a limitation.

Response:We agree and have added language accordingly (pp. 24).

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

14 Oct 2019

A tailored cognitive behavioral program for juvenile justice-referred females at risk of substance use and delinquency: A pilot quasi-experimental trial

PONE-D-19-16985R1

Dear Dr. Walker,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Thach Duc Tran, M.Sc., Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: (No Response)

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: (No Response)

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: (No Response)

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: (No Response)

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

29 Oct 2019

PONE-D-19-16985R1

A tailored cognitive behavioral program for juvenile justice-referred females at risk of substance use and delinquency: A pilot quasi-experimental trial

Dear Dr. Walker:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Thach Duc Tran

Academic Editor

PLOS ONE