The Impact of Juvenile Drug Treatment Courts on Substance Use, Mental Health, and Recidivism: Results from a Multisite Experimental Evaluation.

Juvenile Drug Treatment Courts (JDTC) emerged in the mid-1990s as a potential solution to concern about substance use among youth in the juvenile justice system (JJS). Despite substantial research, findings on the JDTC effectiveness for reducing recidivism and substance use remain inconsistent, hampered by methodological problems. In 2016, the Office of Juvenile Justice and Delinquency Prevention published research-based JDTC Guidelines for best practices, and funded technical assistance for implementation and a multisite national outcomes study among JDTCs implementing the Guidelines. Ten sites were originally selected for this study, with a JDTC and Traditional Juvenile Court (TJC) participating. In two sites, moderate- to high-risk youth were randomized to JDTC or TJC, and in eight sites, a regression discontinuity design assigned moderate- to high-risk youth to JDTC, and other youth to TJC. Findings from four sites with sufficient cases and follow-up rates indicated that JDTCs reduced cannabis use, increased access to mental health services, and reduced recidivism. However, the effects were small to moderate, with positive impacts mainly observed among high-risk youth. The impacts of JDTCs may have been attenuated because Guidelines implementation was inconsistent across courts, and some TJCs implemented elements of the Guidelines, blurring the distinction between JDTCs and TJCs.

Introduction

The National Survey on Drug Use and Health[1] indicates that approximately 6.3% of adolescents between 12 and 17 self-reported criteria consistent with a substance use disorder (SUD), including 4.1% for cannabis, 2.8% for alcohol, and 1.1% for other drugs. Young people with a SUD have higher rates of juvenile delinquency and involvement in the child welfare system, are more likely to drop out of school, and have higher rates of physical and sexual victimization.[2-5] Adolescents who have been involved with the juvenile justice system (JJS) are approximately twice as likely to be diagnosed with a mental health disorder as youth in the general population.[4]

Juvenile courts faced rising drug cases in the early 1990s, limited treatment availability, weak family participation, and minimal ability to motivate delinquent youth to accept treatment.[6] To address substance use and related problems of justice-involved youth, the first Juvenile Drug Treatment Court (JDTC) was implemented in 1995 in Visalia, CA.[7,8] Adult and juvenile drug treatment courts follow a therapeutic jurisprudence model that seeks to use the authority of the court to address underlying needs contributing to criminal behavior.[9,10] Early on, JDTCs adapted the adult drug court model with minimal modification.[5,11] But given that JDTCs had different needs from adult drug courts, the Bureau of Justice Assistance of the U.S. Department of Justice published the “16 strategies”— guidelines on how to design JDTCs to achieve positive outcomes.[12]

JDTCs grew quickly in the latter part of the 1990s,[13] peaking at 476 programs in 2009;[14] an estimated 382 programs are currently operational.[15] Generally positive findings from early JDTC evaluations, despite weak methodologies,[8,16] may have helped fuel their rapid expansion, but findings from subsequent evaluations and meta-analyses were not uniformly positive.[17-21] A study of six JDTCs found substantial variation in effectiveness, with some JDTCs reducing recidivism and others increasing recidivism.[22]

It was not until 2004 that the first national multisite study of JDTCs was implemented,[23] concluding that there was considerable variation among JDTCs in structure and operations that would need to be accounted for in future multisite studies. A randomized experiment compared the effectiveness of JDTCs and community-based treatment services in reducing both substance use and criminal involvement in juveniles and found JDTCs to be more effective.[24] A multisite study concluded that JDTC participants were less likely to be arrested for new offenses.[25] In a comparison of justice-involved youth in substance use treatment with and without involvement in JDTC, youth with JDTC received significantly more substance use treatment and reported greater reductions in substance use and emotional problems.[26]

Despite this research base, there remain many unanswered questions about impacts of JDTCs.[11] The majority of studies contained insufficient information on JDTC implementation,[21,27,28] despite scholars’ suggestion that JDTCs use the 16 strategies across jurisdictions to ensure that all are implemented consistently.[6,11] Cognitive and behavioral issues were commonly not addressed during treatment; treatment was not often tailored to the individual and varied in quality across programs. Enrollment sizes in JDTCs are commonly small, which can present a challenge for program evaluation.[29] Participant retention was also an issue due to a lack of youth motivation or negative attitudes, and problematic linkages with treatment services.

Finally, JDTC studies often include a small number of courts, making it difficult to generalize findings.[11] A recent meta-analysis found that some studies had methodologically inferior designs for causal inference (e.g., pre- to post-test change only, administrative outcomes without youth self-report or urine test results, qualitative results).[20] These results prompted the Office of Juvenile Justice and Delinquency Prevention (OJJDP) in 2014 to launch a 6-year plan to further explore current JDTC research, develop new evidence-based guidelines, and reevaluate the effectiveness of the new guidelines using randomized or strong quasi-experimental designs.[7]

The new evidence-based guidelines comprise 7 objectives related to court philosophy and focus; equitable treatment; engaging the team and following fair processes; comprehensive needs assessment; implementation of effective strategies for contingency management, case management, and community supervision; use of evidence-based substance use treatment; and monitoring program progress and completion.[30] Each objective contains 2 to 6 guidelines (31 overall). For example, Objective 1 emphasizes that JDTC philosophy and practice should focus on addressing substance use and criminogenic needs to improve outcomes. The specific guidelines under this objective include having a team committed to these philosophies and practices, with each team member having clearly articulated roles, and ensuring school systems are represented. Also under Objective 1, guidelines call for a deliberate effort to involve the family of participants throughout the court process and to provide licensed onsite interpreters to family members with limited English proficiency.

Following development of the guidelines, the OJJDP funded the National Cross-Site Evaluation of JDTC Guidelines project.[31,32] Implementation of the Guidelines was assessed via the repeated administration of the Court Self-Assessment (focused on JDTCs) and a Traditional Juvenile Court (TJC) self-assessment, and a multi-day site visit by researchers conducted between these survey administrations.[31,32] The data for the current study come from this project.

This paper describes the main outcomes for the multisite evaluation. The primary research question was: Do youth experience more positive outcomes if assigned to a JDTC vs. TJC? The authors hypothesized that youth assigned to the JDTCs would have, at follow-up, (1) reduced cannabis use (the vast majority of youth in the sample using substances used cannabis), (2) reduced mental health concerns, and (3) lower rearrest likelihood, compared with youth in TJC.

Methods

Design

The cross-site evaluation involved two parallel studies across 10 sites, defined as a county or other court jurisdiction with a participating JDTC and a participating TJC. In two sites, youth who were eligible for JDTC and TJC (moderate to high risk for recidivism and substance use) were randomly assigned (RA) to JDTC vs. TJC. In the remaining eight sites, youth who were eligible for JDTC or TJC were assigned to the most appropriate court using a regression discontinuity (RD) assignment rule based on recidivism risk and substance use severity. Per the JDTC Guidelines, youth at moderate to high risk of recidivism and with a mild to severe SUD were assigned to JDTC and the rest to TJC. Because these two groups are by definition different in their recidivism and substance use risk, to meaningfully compare outcomes, their “expected” outcome was adjusted based on baseline risk, by subtracting expected from actual recidivism. This calculation is similar to a pre-post change score and is particularly useful when focused on modifiable outcomes.[33] Figure 1 displays the case flow (CONSORT diagram) for both research designs, with the details discussed further in the narrative below.

Fig. 1

CONSORT diagram

Court sample

Three sites were selected through a national competition by OJJDP and provided with grant funding for 5 years, and seven sites through a national competition held by the National Council of Juvenile and Family Court Judges (NCJFCJ) with smaller 3-year grants and training/technical assistance. The 10 sites were from nine states, representing jurisdictions with a JDTC and TJC that expected to admit at least 150 youth in 3 years (none met this goal, only in part due to the significant disruptions caused by the COVID-19 pandemic). All agreed to use either RA or RD assignment, collaborate with the evaluation team, and have their JDTC implement the new guidelines with help from technical assistance providers. To maintain anonymity, all sites were designated with letters A through J.

Court self-assessments conducted for the larger evaluation showed considerable overlap between JDTCs and TJCs in the level of overall guidelines achievement at both baseline (70% vs. 51%) and follow-up (86% vs. 56%).[34] On average, JDTCs were higher on guidelines achieved at both times and improved more. However, the best TJC showed more practices consistent with the new guidelines than the “average” JDTC and its corresponding local JDTC at baseline. However, after describing the characteristics of the full sample, the outcome analyses focus on a subset of four sites with at least 75% follow-up at 6 months and at least 10 youth in each court type (one RA and three RD sites).

Youth sample

To be eligible for the study, youth had to be aged between 14 and 17 and enter the jurisdiction’s court dockets during the recruitment window (November 2017 to December 2020). Youth were excluded if previously adjudicated delinquent for a violent offense and/or if the local justice system needed to control their assignment to one of the participating courts or another court (9.4% of cases). To be eligible for JDTC or TJC in the RA sites, youth also had to be moderate to high severity on the crime/violence screener and moderate to high severity on the substance disorder screener (see below). For RD sites, youth who met the above criteria were assigned to JDTC and those who did not were assigned to TJC. Participation was voluntary and required both youth assent and parent/guardian consent. Participating youth received gift cards for completing each survey at baseline ($5), 6 months ($15), and 12 months ($20). The study was conducted under the supervision of American Institutes for Research’s Institutional Review Board and an OJJDP Privacy Certificate. Site staff obtained parent/guardian consent and youth assent.

Data sources

The youth data primarily came from abstraction of their court records and surveys at baseline, and 6- and 12-month follow-ups. Administrative record abstraction included prior arrests/convictions/charges; current arrest/charges; court assignment/movement; urine test results; new arrests, adjudications, and charges post-baseline; and substance use treatment type, initiation, engagement, and continuing care. Data were abstracted by a local court evaluation liaison at each site and transmitted to the evaluation team in de-identified form. The surveys were adapted from the Global Appraisal of Individual Needs Quick Screener version 3 (GAIN-Q3),[35,36] which included sections on demographics, health, HIV risk behaviors, victimization, internalizing and externalizing mental health, substance use, and crime/violence. Each section included standardized screeners for problem severity (e.g., symptoms) and recency that were highly correlated (~ 0.9) with their longer versions in the full GAIN, as well as frequency questions on behaviors (e.g., days of illegal activity, days of substance use) and service utilization (e.g., days in detention, days in substance use treatment). The GAIN Q3’s crime/violence and substance disorder screeners were used for court eligibility/placement. The survey was supplemented with other short, standardized measures related to well-being,[37,38] family effectiveness,[39] having a very important adult,[40] out-of-school structured activity,[41] and social environment.[42] The full versions of the record abstraction tool and surveys are available from the authors.[34]

Outcome measures

The study’s primary outcome measures include (1) number of days of cannabis use and percentage of youth with no use of any drugs other than cannabis in the 90 days prior to the 6-month survey, (2) changes in mental health symptoms and services participation during the 6-month follow-up, and (3) recidivism at 12 months post court assignment based on court records, defined as a subsequent arrest for a criminal offense (status offenses were excluded). Mental health measures included symptoms from the GAIN-Q3’s Internalizing Disorder Screener related to depression, anxiety, trauma, suicidal thoughts, and hallucinations, and the number of days they were bothered by these problems; and history of physical, sexual, or emotional victimization.

Assignment mechanism

Court assignment for both designs was generated by computer after eligibility, consent, and baseline surveys were completed. For RA sites, assignment was blocked on time to ensure that every 4 cases the assignment to condition was balanced, to prevent a run of more than 4 cases to one court type. The local evaluation liaison notified the youth and their parents/guardians immediately after assignment. Although courts could override assignment in response to other factors, this only happened in 39 out of 415 total cases (four in the RA sites and 35 in the RD sites, with 30 of the latter occurring in two of those sites).

Procedures

Local court liaisons trained and certified by the evaluation team performed the eligibility screening, obtained parent or guardian consent and youth assent, completed the follow-up locator form, and administered baseline and follow-up surveys. Surveys were digitally recorded and used to provide feedback/coaching, following the GAIN certification model.[43] Court staff were also trained on strategies for tracking participants and obtaining follow-up surveys. Initially, surveys were primarily conducted in person, with some follow-up surveys completed by phone. In response to lower-than-expected follow-up rates and then COVID-19 restrictions, the follow-up surveys were shortened to focus on just substance use and crime outcomes, and youth were given the option of completing the survey online. All data were cleaned quarterly, and feedback was provided to sites to improve future data submissions.

Recruitment case flow and attrition

Overall, 415 youth were recruited into the study and completed baseline surveys: 78 in the two RA sites (39 JDTC, 39 TJC) and 337 in the eight RD sites (201 JDTC, 136 TJC). Six-month surveys were completed for 61.5% across both RA sites, and 70.9% across all RD sites; 12-month follow-ups were completed for 59.0% in RA sites and 53.7% in RD sites (Fig. 1). Given the low 12-month follow-up rates, the focus is on 6-month outcomes for the survey data.

There was substantial variation in attrition across sites. One RA site had a very low 6-month follow-up rate (17%), and collected minimal administrative data, so was excluded from the analyses. One RD site withdrew from the study before follow-up data collection began. In addition, survey follow-up rates in four RD sites were either relatively low overall (57% with 6-month follow-ups) or skewed toward following up JDTC youth (fewer than five 6-month follow-ups for youth in the TJC condition). These sites also enrolled relatively few youth (13.9% of the youth in all the original RD sites). Accordingly, only the remaining three RD sites were included in the outcome analyses along with the remaining RA site. The percentages of youth with 6- and 12-month follow-up surveys were 88% and 80%, respectively, in the RA site, and 84% and 63%, respectively, in the three RD sites. Administrative records were available on all youth in all four sites used in the outcome analysis over the 12-month follow-up period.

Examination of differences between youth with and without follow-up surveys in the RD sites indicated that youth with follow-ups tended to be somewhat higher risk in terms of recidivism risk and substance use compared with those without follow-ups. This result is contrary to many outcome studies with delinquent and substance-using youth samples that find higher risk among youth lost to follow-up.

Analytic methods

All analyses were conducted using SPSS version 27. Recidivism outcomes from administrative records were analyzed by “expected” risk based on the GAIN Q3’s crime/violence and substance disorder screeners, court type, and their interaction. For the expected risk variable, each screener was scored as low (0 symptoms in the past year), moderate (1–2 symptoms), or high (3–5 symptoms), then combined to make 9 levels of risk of recidivism using prior data.[34] In data from 9399 youth from 141 U.S. juvenile justice sites before the 2016 JDTC Guidelines were issued, these 9 levels had monotonically increasing 6-month recidivism rates (25%, 29%, 34%, 29%, 45%, 51%, 56%, 62%, 67%) and odds ratios of 1.0, 1.2, 1.6, 1.9, 2.4, 3.1, 3.8, 4.9, 6.1. For 12-month recidivism, the expected rate was increased by a factor of 1.38 to reflect typical growth in cumulative recidivism rates between 6 and 12 months.

Analyses of youth survey data were based on youth who had both a baseline and 6-month follow-up survey, due to the lower follow-up rates for the 12-month surveys and the shortened 12-month survey administered to some of the youth. Recidivism analyses were based on administrative records and were available for the full 12-month follow-up period. Descriptive analyses compared the baseline and outcome values of the key primary and secondary measures, controlling for the baseline value where appropriate. Ordinary least squares regression models were estimated for the RD sites for the primary outcome variable of days used cannabis. The small number of cases in the RA sample precluded additional multivariate analyses, and the small number of sites meant that multilevel modeling was not appropriate. Missing data for specific variables was not an issue for either the youth survey or administrative data.

Because the court assignment rule in the RD was based on risk/need, there were significant differences in the expected risk of recidivism at baseline. Within and across the 4 sites, both t-tests (parametric) and Mann–Whitney Rank order tests (non-parametric) were used. The latter was necessary due to the unequal Ns by site/court type, a bimodal distribution, and an unexpected interaction between initial expected risk and the amount of change. The RD analytic model assumes that the effect of the baseline covariate can be modeled with a difference in the a-intercept and no difference in the slope (i.e., two parallel lines). However, there was a significant impact of court type on slope as well.

Results

Baseline youth characteristics

Table 1 displays the baseline characteristics of the full sample.

Table 1

Sample characteristics — all study sites (% unless otherwise noted)

	Random assignment		Regression discontinuity		Total
	JDTC (n = 39)	TJC (n = 39)	JDTC (n = 201)	TJC (n = 136)	N = 415
Gender
Female	5.1	5.1	28.4	30.1	24.3
Male	94.9	94.9	71.6	69.9	75.4
Race/Ethnicity
Black	43.6	33.3	30.0**	41.9	35.5
White	5.1	5.1	49.5	30.1	34.8
Hispanic	28.2	35.9	3.5	5.9	9.7
Mixed	20.5	23.1	16.0	21.3	18.8
Another race/ethnicity	2.6	2.6	1.0	0.7	1.1
Mean age	15.8	15.5	15.2	15.4	15.5
Mean last grade completed	8.9	8.5	8.7	8.8	8.8
Delinquency history
Mean no. of times arrested and charged lifetime	2.6	4.2	3.0***	1.5	2.6
Ever pushed, grabbed, shoved someone	87.2	76.9	89.1***	51.5	75.5
Ever taken something from store w/o paying	69.2	64.1	65.7***	30.6	54.5
Ever sold/distributed/made illegal drugs	48.7	56.4	35.3***	7.5	29.5
Ever destroyed property	43.6	46.2	46.3***	20.1	27.5
Recidivism risk
Low	0.0	0.0	0.0***	25.4	8.2
Moderate	33.3	25.6	36.3	67.9	45.3
High	28.2	20.5	29.9	6.7	21.3
Very high	38.5	53.8	33.8	0.0	25.2
Substance use (past 90 days)
% cannabis use	97.4*	84.6	91.5***	40.4	74.7
Avg. no. of days cannabis use (SD)	55.0 (28.3)*	42.2 (28.1)	39.4 (30.4)***	9.5 (21.0)	31.6 (31.5)
% alcohol or non-cannabis drug use	46.2	33.3	47.8***	11.2	34.7
% ever received AOD treatment	43.6	35.9	21.9**	7.5	20.6
Mental health indicators
Bothered by MH problems past 90 days	64.1	59.0	69.3*	43.3	59.4

*p < .05, **p < .01, ***p < .001

Random Assignment

In the two RA sites, demographic characteristics for the JDTC and TJC groups were quite similar. Most were male and self-identified as Black, mixed race, or Hispanic. JDTC youth were arrested a mean of 2.6 times in their lifetimes, compared with 4.2 for the TJC group (NS). Substantial percentages of both groups had engaged in various forms of delinquent behavior as well. As expected with randomization, there were no significant differences in recidivism risk at baseline between the JDTC and TJC youth, with about 30% of youth assessed as moderate risk, 24% high risk, and 46% very high risk.

JDTC youth reported very high rates of cannabis use and high frequencies of cannabis use in the past 90 days that were greater than youth randomized to the TJC. Just under half of JDTC youth reported any other illicit or prescription drug misuse in the past 90 days, or any lifetime history of drug treatment. Youth in both groups reported high prevalence of mental health problems, with about 60% being bothered by mental health problems in the past 90 days.

Regression Discontinuity

Because assignment was based on recidivism risk level and substance use, it was expected that the JDTC youth would show a higher risk profile. Thus, JDTC youth had a mean of 3.0 prior arrests at baseline compared with 1.5 for TJC youth (P < 0.001; Table 1) and reported significantly higher likelihood of prior delinquent behaviors. In addition, 29.9% were in the high recidivism risk category and 33.8% were in the very high risk category, while in the TJC condition, 25.4% were low risk and 67.9% were moderate risk (P < 0.001). JDTC youth also had a significantly higher mean number of days using cannabis in the 90 days prior to baseline, greater misuse of other drugs, more frequent prior substance use treatment, and significantly higher likelihood of being bothered by mental health problems.

The outcomes analyses in the remainder of this paper focus on the subset of 1 RA and 3 RD sites with sufficient numbers of cases.

Youth survey outcomes

Substance Use

Table 2 displays the changes in cannabis and other drug use from baseline to 6-month follow-up for the two assignment mechanisms. In the RA site, there was a greater decrease in the percentage of JDTC youth reporting cannabis use than TJC youth (33.3 vs. 17.4 percentage point decrease) although the difference was not significant due to low statistical power. Similarly, the mean number of days using cannabis in the past 90 days decreased to a greater extent in the JDTC condition (reduction of 29.1 days vs. 12.1; t =  − 1.71, P < 0.10). Use of alcohol or drugs other than cannabis was much less common in the JDTC group, and there were no significant differences between JDTC and TJC in the changes in other drug use at follow-up. Similar results were found for the RD sites. Due to the RD assignment criterion, JDTC youth were at higher risk for substance use at baseline, and there was a greater reduction in cannabis use compared with the TJC youth, reflected in both the percentage reporting no cannabis use (P < 0.001) and mean number of days used (mean reduction of 18.6 days vs. 1.4 for TJC; t =  − 3.73, P < 0.001).

Table 2

Substance use outcomes

Random assignment	JDTC (n = 21)			TJC (n = 23)
	Baseline	6 months f/u	Change	Baseline	6 months f/u	Change
% reporting cannabis use, past 90 days	100.0	66.7	− 33.3	100.0	82.6	− 17.4
Avg. days cannabis use, past 90 days (SD) (including zero)	63.7 (SD 21.7)	34.7 (SD 38.9)	− 29.0	52.0 (SD 22.4)	39.9 (SD 31.7)	− 12.1
% youth no AOD use other than cannabis, past 90 days	9.5	14.3	+ 4.8	17.4	21.7	+ 4.3
Regression discontinuity	JDTC (n = 130)			TJC (n = 82)
	Baseline	6 months f/u	Change	Baseline	6 months f/u	Change
% youth reporting cannabis use, past 90 days	96.2	64.6	− 31.6	34.6	22.0	− 12.6
Avg. days cannabis use, past 90 days (SD) (including zero)	42.15 (SD 30.41)	23.55 (SD 29.65)	− 18.6***	10.17 (SD 22.77)	8.65 (SD 20.50)	− 1.42
% youth reporting AOD use other than cannabis, past 90 days	48.5	36.2	− 12.3	13.4	11.0	− 2.4

*p < .05, **p < .01, ***p < .001

Table 3 shows the results of a stepwise OLS regression model on the number of days used cannabis in the past 90 days prior to follow-up in the RD sites. Criteria to enter and remove were set at ≤ 0.050 and ≥ 0.100, respectively. Study condition (JDTC vs. TJC) was not a significant predictor of cannabis use at 6-month follow-up, controlling for baseline cannabis use and other variables. The final model included days of cannabis use in the 90 days prior to baseline assessment (P < 0.05) and number of lifetime prior arrests at baseline (P = 0.06). That study condition was not significant is likely due to the RD assignment mechanism, which included many of the same factors used to predict cannabis use at follow-up.

Table 3

Effect of court type (JDTC versus TJC) on days of cannabis use at 6-month follow-up for the three RD sites combined

Backward stepwise regressiona	b	SE	95% LCL	95% UCL	β	t-ratio	Sig
Variables in model
Constant	13.115	4.201	4.814	21.416		3.122	.002
Baseline days MJ use in past 90	.171	.076	.019	.322	.179	2.231	.027
Baseline lifetime arrests	1.358	.709	− .043	2.759	.153	1.916	.057
Variables excluded
Court type (JDTC vs. TJC)	.036					.424	.672
Baseline substance use problems, lifetime (L, M, H)b	− .044					− .532	.595
Initiated treatment	− .040					− .471	.638
Gender	− .048					− .596	.552
Mental health problems, past 90 days at 6 months	.071					.885	.377
Final model	R2 = .061, F = 4.841, SE = 28.770

aListwise missing exclusion

bGAIN Substance Disorder Screener, grouped (low = 0, moderate = 1–2, high = 3–5)

Mental Health

Table 4 summarizes changes in mental health over time. In the RA site, JDTC and TJC youth showed increased access to mental health services: the percentage of youth receiving outpatient mental health services and medications increased in both groups. The percentage reporting being bothered by mental health problems in the past 90 days increased from 69.6 to 81.8% for TJC youth, but only increased slightly for JDTC youth. There was no significant change in the mean number of symptoms.

Table 4

Mental health outcomes, past 90 days

a. Random assignment (%)
	JDTC (n = 21)			TJC (n = 23)
	Baseline	6 month f/u	Change	Baseline	6 month f/u	Change
Received outpatient MH treatment	9.5	23.8	+ 14.3	8.7	27.3	+ 20.3
On MH medication	0.0	9.5	+ 9.5	4.3	22.7	+ 18.4
Bothered by MH problems	61.9	66.7	+ 4.8	69.6	81.8	+ 12.2
Mean no. of MH symptoms	2.6	2.7	+ 0.1	2.7	3.1	+ 0.4
b. Regression discontinuity (%)
	JDTC (n = 130)			TJC (n = 82)
	Baseline	6 month f/u	Change	Baseline	6 month f/u	Change
Received outpatient MH treatment	21.7	60.5	+ 38.8***	17.1	22.4	+ 5.3
On MH medication	13.8	19.4	+ 5.6	14.6	14.3	− 0.3
Bothered by MH problems	76.6	71.2	− 5.4***	50.0	51.9	+ 1.9
Mean no. of MH symptoms	3.2*	3.5**	+ 0.3	2.4	2.4	0.0

*p < .05, **p < .01, ***p < .001

In the RD sites, the percentage of JDTC youth receiving outpatient mental health services in the past 90 days increased substantially from 21.7 to 60.5% (P < 0.001 compared with TJC youth), and the percentage receiving medications increased from 13.8 to 19.4% (NS). There were slight changes in both outcomes in the TJC group compared to baseline. The percentage of youth reporting being bothered by mental health problems in the past 90 days decreased slightly from 76.6 to 71.2% for JDTC youth and remained about the same for TJC youth (P < 0.001). The mean number of symptoms reported was significantly higher for JDTC youth at both baseline and 6-month follow-up and increased slightly from 3.2 to 3.5 for JDTC youth during follow-up.

Rearrest based on administrative records

In the RA site, relative to TJC, youth in the JDTC were significantly less likely to recidivate during the 12-month follow-up (60% vs 32%, 2(50) = 3.95, P < 0.05, OR 0.31 [95% confidence interval 0.10–1.00]); the JDTC group was about one-third as likely to be rearrested.

Table 5 shows the expected (determined by baseline risk), observed, and difference in the 12-month rearrest percentage by design, court type, and site. By design, the RA site has nearly identical risk distributions for JDTC and TJC, while in the RD sites, the JDTCs have higher expected risk than the TJCs. Overall, 57.9% of the JDTC youth and 28.3% of the TJC in the three RD sites were rearrested within 12 months of baseline (P < 0.001, OR 3.17). Table 5 also displays the results of t-tests (differences in the mean) and Mann–Whitney rank order tests (differences in the rank and overall distribution). For the mean percentile rank, 1% is the most reduced re-rearrests and 100% is the most increased re-arrest rate (i.e., lower numbers are good).

Table 5

Rearrests in 12 months after assignment by design, site, and court typea

Design	Random Assignment			Regression discontinuity
Site	Site A			Site C			Site H			Site I			Across 4 sites
Court type (number of participants)	TJC (n = 30)	JDTC (n = 49)	TJC (n = 30)		JDTC (n = 49)	TJC (n = 23)			JDTC (n = 69)	TJC (n = 58)		JDTC (n = 24)	TJC (n = 136)	JDTC (n = 167)
Expected rearrest rateb	74%	72%	52%		72%	50%			72%	51%		69%	55%	72%
Observed rearrest rate	60%	32%	27%		59%	61%			68%	16%		25%	34%	54%
Difference (Obs.–Exp.)	− 14%	− 40%	− 25%		− 13%	+ 11%			− 4%	− 35%		− 44%	− 21%	− 18%
Relative difference (JDTC-TJC)	− 26%			+ 12%				− 15%			− 9		+ 9.10
Cohen’s D	− 0.49			+ 0.25				− 0.31			− 0.22		+ 0.19
T-test	− 3.91			+ 2.26				− 2.94			− 1.99		+ 2.76
d.f	49			78				91			81		302
P (t)	.001			0.027				0.004			0.050		0.90
More reduction in	JDTC			TJC				JDTC			JDTC		–
Mean rank % (low is good)	57% v 45%			54% v 49%				67% vs 45%			58% vs 32%		55% vs. 47%
Mann–Whitney based Z-rank	− 1.4			− 0.7				− 3.1			− 3.8		− 2.4
P (Z)	0.172			0.484				0.002			< 0.001		0.016

aBold means that the probability of the t-test or Z-rank values is less than 0.05 and/or that the Cohen’s effect size is greater than − 0.20 (i.e., JDTC reduced recidivism from expected more than TJC)

bBased on national data

In the RA site, relative to the difference (observed minus expected) in the TJC, the JDTC reduced recidivism significantly more (− 14% vs. − 40%, t(49) =  − 3.91, Cohen’s d =  − 0.49). Because of bimodal distributions of differences, the Mann Whitney rank order test goes in the same direction but does not reach significance (mean rank 57% vs. 47%, Zrank =  − 1.4, P = 0.172).

In the RD sites, relative to the expected rates, the observed rearrest rates were lower (i.e., negative difference scores) in the TJC in 2 of the 3 sites and in JDTC in all three sites. The reductions in observed-expected difference scores were significantly larger for JDTC than TJC in 2 of the 3 sites: Site H (+ 11% TJC vs − 4% JDTC, t(91) =  − 2.94, P < 0.05) and Site I (− 35% TJC vs − 44% JDTC, t(81) =  − 3.8, P < 0.05). In the third RD site, C, the reduction was significantly larger for TJC than JDTC (− 25% TJC vs − 13% JDTC, t(78) =  + 2.26, P < 0.05). As noted earlier, these t-tests focus on the differences in the mean by court type but assume a similar distribution. Since the distributions differ in more complex ways, the last three rows of Table 5 also report the results of Mann–Whitney rank order tests. There remain significant reductions in site H (Zrank =  − 3.1, P < 0.05) and site I (Zrank =  − 3.8, P < 0.05). However, in Site C, the relative difference in court type is no longer significant and has actually flipped in direction (Zrank =  − 0.7, P = 0.484).

The last column of Table 5 shows the results aggregated across all 4 sites. There is no significant difference by t-test, but the rank order tests show that JDTCs had greater reductions in mean rearrest rate percentiles (55% TJC vs 47%, Zrank =  − 2.4, P < 0.05). Figure 2 illustrates the bimodal distributions and how they differ by court type across sites. TJC outcomes were more diverse than JDTC outcomes and virtually no cases are at the “mean” score (i.e., an ecological fallacy). Although not shown directly in Fig. 2, there also appeared to be an interaction of initial risk severity with the degree of court effectiveness within JDTCs. Specifically, JDTC effectiveness mostly reflects outcomes among youth at higher levels of recidivism risk.

Fig. 2

Differences between predicted and observed probability of 12-month recidivism by court type (combined RA and RD sites). Rearrest probability differences are binned. Greater reductions in re-arrest probability (i.e., more negative observed-minus-predicted differences) were observed in JDTCs (mean rate percentile = 47%) than in TJCs (55%), Zrank =  − 2.4, p < .05

Discussion

The current study explored the effectiveness of the JDTC Guidelines using random assignment (RA) and regression discontinuity (RD) designs. Baseline results from all 10 sites confirmed that the study sample comprised relatively high-risk and high-need youth. Youth were primarily male and identified as Black, Hispanic, or multiple races. They had multiple prior arrests and self-reported involvement in delinquent behaviors, substantial prior cannabis use, and co-occurring mental health issues. Youth who were “eligible” for JDTC (both groups in the RA design, only JDTC in the RD design), reported significantly higher risk of recidivism, need for substance use treatment, and a range of co-occurring mental health problems.

Outcome analyses from the subset of 4 study sites suggest that JDTCs help to reduce substance use, increase access to mental health services, decrease self-reported mental health symptoms, and reduce recidivism. However, the effects are small to moderate, and lower severity youth do better in TJC, moderate severity youth fare similarly in both, and higher severity youth do better in JDTC. In the RA site, JDTC youth had relatively larger decreases in self-reported cannabis use and significantly lower rearrest rates over a 12-month follow-up (32% for JDTC compared to 60% for TJC). In the RD sites, JDTC youth—at higher risk—reported a larger reduction in cannabis use than TJC youth. The JDTC youth had significantly lower than expected recidivism than TJC youth overall across sites and in 2 of the 3 RD sites. Overall, JDTC effects were largely driven by youth in the higher risk groups, and may reflect increased access to mental health and substance use treatment, or the case management and family-related services provided by JDTCs.

In another study component, courts completed self-report assessments of the extent to which they had implemented the JDTC Guidelines.[34] The RA site, with the lower JDTC recidivism rate, had the highest rate of Guidelines implementation of the four outcome study sites (86% at baseline, 92% at follow-up), indicating it more completely represented a jurisdiction that was following the JDTC model. The three RD sites also had much higher implementation of the Guidelines than the local TJCs (74% vs. 49% at baseline; 88% vs. 55% at follow-up). This finding reinforces the importance of JDTCs (and the JJS overall) adopting research-based practices and providing appropriate services, particularly when working with higher-risk and higher-need youth.

Limitations

The largest limitation of the current study is the smaller than planned sample sizes. In part due to overall reductions in juvenile crime and changes in JJSs that provide a range of diversion and intervention options, JDTC referrals have declined in recent years. The anticipated sample numbers were further reduced by the onset of the COVID-19 pandemic and resulting court closures and stay-at-home-orders (which also affected survey follow-up rates). The small samples decreased the number of viable outcome study sites to 4, limiting statistical power.

While the study involved a broad representation of jurisdictions, in terms of size, region of the country, and demographic characteristics, the sites agreed to participate either because of their interest in the JDTC Guidelines and their goal of improving their local JDTC, or because they received funding and/or training/technical assistance as part of their participation. Accordingly, the results from these sites are not necessarily generalizable to other JDTCs.

Data from the youth survey were the sole source of information about substance use and mental health and were not validated by administrative data. However, youth self-reports of cannabis and other commonly used substances have been found to be reliable measures of this information.[44] The youth self-reported arrest prevalence was consistent with findings from official youth records from the juvenile courts.

The final data did not meet the assumptions of the original RD design in that there were both mean and distribution effects (i.e., lines were not parallel). Attempts to improve the model for predicting recidivism, transform the variables, and/or use analysis of covariance did not improve the results. With only 4 sites, there was insufficient power to do some of the originally planned moderator, mediator, and/or multi-level analysis.

Finally, it is important to note that JDTCs varied considerably in their implementation of the JDTC Guidelines, and that some TJCs had implemented some JDTC Guidelines as well. Therefore, at minimum, TJCs represented only partial comparisons (not a “no treatment” comparison), potentially attenuating some of the impact that might otherwise have been found. However, the overlap with TJCs may be compounded by the heterogeneity within JDTCs — rendering the two conditions less distinct than would be preferable for comparison and raising questions about the nature and extent of JDTC’s unique traits.

Future directions and conclusions

This study confirms the potential benefit of JDTCs as an effective intervention for youth with moderate to high risk of recidivism and substance use. Given the mixed results from prior JDTC research, this study provides timely data that suggest the importance to JDTCs of incorporating the Guidelines. When JJSs develop specialized dockets with a trained, collaborative multidisciplinary team of professionals, and access to treatment services, youth and the court systems benefit. However, given the additional resources required to implement JDTCs, it would also be important to conduct rigorous cost-effectiveness or benefit–cost analyses of JDTCs to assess whether the positive outcomes are also economically viable.

Future analyses will further explore the effects of mediator and moderator variables, such as program, court, and community characteristics. New studies are also needed to assess the degree to which JDTCs have implemented the Guidelines, and whether a certain configuration of Guidelines is significantly related to improved outcomes. Larger sample sizes and additional study sites would provide statistical power for additional analyses. Longer follow-ups could provide more opportunity for training/technical assistance, to maximize the degree of implementation of JDTC Guidelines and assessment of their impact. In the current study, technical assistance occurred during the study, so programs were still in the process of improving. If sites underwent this improvement process first, studies could assess the impact of the JDTC model more fully. The current broader study utilized mixed methods, which provided for rich and useful information to contextualize and interpret the outcomes. It is suggested that future studies continue to incorporate mixed methods, including qualitative interviews with JDTC staff and participants.

Implications for Behavioral Health

The findings from this study have a number of implications for improving the delivery of behavioral health services to youth in the JJS. First, the results suggest that intensive programs such as JDTCs are more effective if they serve higher-risk youth, while TJCs may be able to adequately address the behavioral health service needs of low- to moderate-risk youth.

Second, co-occurring substance use and mental health problems are common among justice-involved youth, and services, assessment, and treatment linkage must adequately address these issues if positive youth outcomes are to be achieved. Third, the case management and family-oriented services in JDTCs may serve to address the youths’ behavioral health needs and thereby reduce substance use and recidivism in this vulnerable population.

Fourth, the findings suggest that greater implementation of JDTC Guidelines may be associated with improved behavioral health outcomes. Both JDTCs and TJCs could benefit from incorporating as many of these Guidelines as possible and integrating evidence-based practices into their assessment, referral, and service delivery models.

Finally, the findings suggest that community behavioral health services are an important adjunct to both traditional and specialty juvenile courts. Improved interagency collaboration and communication, mission alignment, and matching behavioral health services to the needs of moderate- and high risk justice-involved youth are likely to lead to improved youth outcomes, including reduced substance use, mental health problems, and recidivism.