The Missing Link(age): Multilevel Contributors to Service Uptake Failure Among Youths on Community Justice Supervision

Each year, 1.7 million U.S. youths are referred to juvenile justice systems (1). Youths who have contact with the justice system have more significant behavioral health concerns, involving troublesome peer and family relationships (2), adverse childhood experiences (3), and mental health problems (4, 5) compared with the general youth population. Many justice system–involved youths report frequent use of alcohol and marijuana (6–8) and prescription drug misuse (9), and 34% meet criteria for a diagnosis of a substance use disorder, across a range of justice settings (4). Juveniles with substance use disorders are nine times more likely than youths without such disorders to be involved in the justice system (10). Early intervention can break the cycle of continued justice involvement, because treatment for behavioral health disorders lowers recidivism risk (11, 12). Screening and assessment of service needs are recommended in judicial and justice processing (13) to facilitate service receipt.

Most youths (75%−95%) in contact with the juvenile justice system are handled by community supervision agencies at some point in case processing (9, 14). Approximately 85% of justice agencies do not directly provide behavioral health services (9). Although justice agencies commonly screen youths for behavioral health needs and delinquency risk, their mission does not include conducting clinical assessments to disentangle the factors that promote delinquency.

Juvenile justice administrators note that fragmentation in local service delivery systems creates barriers to obtaining behavioral health services (9) and report difficulty working collaboratively with local service providers (15, 16). A large majority of justice-involved youths do not access services, even after it has been established that they have a need for such services (17). Failure to access those services exacerbates continued substance use and mental health concerns, contributing to further involvement in the juvenile or adult justice systems (10, 18–20). The high screening rate, coupled with a low treatment initiation rate, warrants closer inspection of which youths are referred to services.

To navigate between justice and behavioral health service sectors, youths must pass through a series of steps, ranging from screening, to referral, and to service initiation, engagement, and continuing care, described as the behavioral health services cascade (referred to as “cascade” in the following) (13). We lack an understanding of the degree to which this model accurately captures the justice system’s approach to young persons’ behavioral health needs or the point(s) at which juveniles may depart from the cascade. Without these data, targeted system- or policy-level changes that address service gaps and missed treatment opportunities cannot be implemented.

To better understand the effects of multilevel factors on the steps in the cascade, we assessed the contributions of youth, staff, agency, and county characteristics to these steps. We did so in 33 discrete justice settings, involving a multilevel approach to understand these complex service systems. Youths are supervised by justice staff whose agencies operate within a system that includes behavioral health providers, all of which are further influenced by larger community contexts (21).

Methods

JJ-TRIALS Research Cooperative

The Juvenile Justice–Translational Research on Interventions for Adolescents in the Legal System (JJ-TRIALS) cooperative agreement, which was funded by the National Institute on Drug Abuse, includes six research centers, each working with four to six local justice agencies providing community justice supervision (e.g., probation) in Florida, Georgia, Kentucky, Mississippi, New York, Pennsylvania, and Texas. Its primary aim was to test the effectiveness of a locally customized intervention to improve delivery of services addressing substance use needs for justice-involved youths by working with justice agencies and their community-based behavioral health partners. We analyzed data from youths and staff in 33 county-level sites during the baseline study phase, before researchers implemented a systems-level intervention to improve interagency service delivery. Research centers received study approval from their institutional review boards (21). Justice staff provided informed consent for surveys, and a waiver of consent was granted to review deidentified youth records. By design, justice sites (27 intake settings and six youth courts) provided community-based justice services to juveniles whose behavioral health needs were typically addressed by other local providers. Sites varied in sociodemographic characteristics and service availability.

Procedures and Measures

Youth case records.

Data from all new intakes (N=8,307) into the justice system between March 2014 and August 2015 were captured to provide a baseline sample (22). The baseline period covered 9–18 months (six of the seven states contributed 11 or more months of data) wherein youths underwent intake evaluations and case disposition and were then either diverted to other services or placed on community supervision. Justice staff routinely enter information on youths and case management events into agency information systems; through study agreements, information from these systems was made available to researchers, generally by automatic record abstraction. Additional details have been published elsewhere (22).

Cascade outcomes were derived from official case records reporting whether a juvenile had been screened for substance use treatment needs; had been found to require substance use services because of a positive screening or urine test result, clinical assessment, alcohol or other substance charges, or court-ordered substance use treatment; or had been referred to a local service provider by justice staff, or had initiated those services, engaged in treatment (for at least 6 weeks), and received care for ≥90 days. Juvenile justice agencies used a variety of evidence-based and not evidence-based screening instruments, which screened for substance use (and could have also screened for broader behavioral health constructs) (13, 21).

In total, 67% of the sites (22 of 33) relied on evidence-based instruments such as the Massachusetts Youth Screening Instrument (MAYSI), Youth Assessment and Screening Instrument (YASI), Positive Achievement Change Tool (PACT), Substance Abuse Subtle Screening Inventory (SASSI), CRAFFT (Car, Relax, Alone, Forget, Friends, Trouble), and Global Appraisal of Individual Needs (GAIN). Often both substance use and mental health services were provided by a single agency, or substance use services were first accessed via mental health service providers. Therefore, we generically describe these services (and the corresponding service referrals and initiations) as behavioral health services. In short, youths were screened for substance use problems, identified as needing substance use services, and were then referred to local behavioral health providers.

Agency records did not consistently document cascade steps beyond initiation: 23 (70%) sites had information on treatment initiation for >95% of cases; only 20 (61%) had data on treatment engagement and 19 (58%) on continuing care. Accordingly, we restricted reporting to all 33 sites through referral and 23 sites through initiation. Because swift identification and service linkage are key to good clinical practice (23) for those youth whose administrative records contained dates of occurrence, we examined whether youths were screened within 30 days of intake and whether those needing services were referred within 30 days of the screening.

Youth demographic and offense characteristics were extracted from justice case management records and included gender, race (White or non-White), Hispanic ethnicity, age, and type of supervision and whether the current offense or offenses included alcohol- or other drug-related charges. A higher supervision level included ongoing formal oversight by probation, parole, or juvenile drug court authorities; a lower level reflected diversion to community programs or placement on informal community supervision (24). (The dichotomy “formal-informal,” which is frequently used in criminal justice research, does not precisely capture what is meant by “high and low supervision” in this study.) Decisions regarding the needed supervision level generally reflect offense severity or chronicity (24).

Justice agency staff and leadership surveys.

All staff carrying caseloads and their supervisors (N=621) were invited to complete surveys (71% response rate, N=441) (25), for which informed consent was obtained onsite. Agency leaders completed questionnaires regarding site practices for substance use services and shared practices and information about collaboration with local providers. Survey responses were aggregated to calculate agency-level measures. Years of experience and caseload size were averaged across respondents within sites. Perceived competency items assessed how well respondents felt they could identify youths’ behavioral health concerns and could link youths to providers (26). Indicators of organizational characteristics included quality of intra-agency communication (six items, Cronbach’s α=0.8) and job-related stress (four items, Cronbach’s α=0.86) (27). All items were rated on 5-point Likert scales; scale scores were multiplied by 10 for ease of interpretation, and responses were averaged (Cronbach’s α=0.85). Possible scores ranged from 10 to 50. Higher scores on each scale represent a higher level of the attribute being represented by the scale. Information about collaborative practices with service providers was based on sharing 12 activities (e.g., pooled funding or joint staffing) with external providers (28).

County characteristics.

County-level measures were obtained from census data (29), including the percentage of families with children ages <18 years living in poverty, percentage of children ages <18 who were without health insurance, and percentage of residents in urban areas. Information on available service providers per 100,000 residents came from the Centers for Disease Control and Prevention’s children’s mental health portal (30).

Analysis plan.

First, we examined independent and dependent (i.e., cascade outcomes) variables for the overall sample and by site. We examined four cascade steps (screening, in need of treatment, treatment referral, and treatment initiation) and two time-related outcomes (screened within 30 days of intake and referred within 30 days of screening). Overall rates were calculated relative to all youths in the intake cohort. On the other hand, “within-step” rates depended on an event occurring in the previous step—for example, we examined referral only for those identified as “in need” in the previous step. We next examined correlations among independent variables, to ensure that multicollinearity would not compromise the stability of regression analyses and assessed bivariate associations for each independent variable (youth as well as agency- and county-level measures, described above and listed in Table 1) and cascade outcome.

Multivariable, multilevel regression analyses used the same predictors for each outcome, although the contribution of alcohol- and other drug-related charges was examined only for screening, because it was included in the definition of “in need” and therefore included in subsequent conditional cascade steps. Measures included in each step were selected a priori to examine associations with cascade outcomes (rather than data-driven modeling). In order, we entered the characteristics of youths and justice staff, followed by those of the organization of agencies and counties. Because of the data’s hierarchical nature (youth data nested within county), we used SAS PROC GLIMMIX (31) for the nested data structures, separating within-county and within-person variance from between-county and between-person variance (32, 33).

Missing data.

In a related JJ-TRIALS evaluation of 31,308 youth records from these sites (22), data were available for a median of 49 of 72 (68%) core items that justice partners all reported as being available in their records (e.g., demographic factors and offense-related and cascade events). Data were coded as “missing” if they were not collected, not coded similarly, or inaccessible. Cascade events were interpreted as “yes” versus “other,” whereby “other” included all “missing/no indication of event” data, providing conservative estimates of retention rates at each cascade step.

Results

Contributing Characteristics and Outcomes

Youth characteristics.

In the sample of 8,307 youths, most were male (74.2%), and approximately half (49.4%) were White (Table 1). About one-fifth (18.5%) were facing alcohol- or drug-related charges, and one-quarter (26.4%) had violence-related charges. Slightly more than half (55.4%) received a higher level of justice supervision.

Justice agency staff and organizational characteristics.

The mean number of years of justice staff member experience was 15.3, and caseloads averaged about 16 youths at the time staff completed the survey (Table 1). The mean score on perceived competency to identify youths’ behavioral health needs and link juveniles to services was above the midpoint (i.e., 38 of 50 points). Intra-agency communication scores averaged slightly above the midpoint (30 of 50 points; possible scores ranged from 0 to 50), reflecting a perceived need for better communication. Staff reported relatively high job stress (35 of 50 points). Finally, agency leadership reported collaborating with behavioral health partners in approximately five (of 12) different activities, most commonly “sharing information on youth needs for treatment services” (91%) and “joint staffing/case reporting consultations” (89%).

County characteristics.

Most justice agencies (87%) were in urban counties (Table 1). Almost one-fifth (18%) of families with children in these counties lived below the poverty line; most (93%) children in these counties had health insurance, including Medicaid. Finally, counties had a mean of 37.4 behavioral health service providers for every 100,000 persons.

Cascade outcomes.

As shown in Table 2, rates at each cascade step were highly variable across the 33 sites. Just under three-quarters of youths were screened (range across sites 7%−100%), and more than half of the 8,307 youths in the sample were identified as requiring treatment (range 13%−95%). Among those in need, about one-fifth were referred to treatment (range 4%−24%). If referred to treatment, rates for initiation and engagement were relatively high: two-thirds of those referred initiated treatment (68%, range 8%−100%), almost half of those who initiated treatment engaged in services (range 6%−94%), and more than one-quarter who initiated treatment remained in continuing care (range=10%−75%). On average, youths were screened within 104 days of justice system intake. For those who received referrals, referral occurred within an average of 26 days.

Predicting Cascade Steps

Screening.

About 43% of the youths were screened with an evidence-based instrument: 19% with the MAYSI, 10% with the YASI, 8% with the PACT, 3% with the SASSI, 2% with the CRAFFT, and 1% with the GAIN. Youths with alcohol- or drug-related charges were more likely than those with other charges to be screened (adjusted odds ratio [aOR]=1.37, p=0.002) (Table 3). Youths receiving higher levels of justice supervision were almost three times as likely to be screened as were those who were diverted or informally supervised (aOR=2.92, p<0.001). Finally, youths living in counties where fewer children had health insurance were also more likely to be screened (aOR=1.62, p=0.001).

In need.

Youth characteristics predicted service need most consistently. Older youths were more likely to have substance use service needs (aOR=1.30, p<0.001) (Table 3). Males were more likely than females (aOR=1.39, p<0.001), and White youths were more likely than non-White youths (aOR=1.40, p<0.001), to have substance use service needs. Those receiving higher levels of supervision were more than 2.5 times as likely to be identified as being in need of services as those receiving lower levels (aOR=2.58, p<0.001). The sole remaining significant contributor to identification of service needs was caseload size: youths under supervision of staff with larger caseloads had slightly lower rates of identified need for substance use services (aOR=0.93, p=0.042).

Referral.

Youths under higher levels of supervision were nearly five times as likely to be referred to treatment (aOR=4.89, p<0.001) as those who were diverted or informally supervised (Table 3). Juveniles from agencies where staff had more years of experience were less likely to receive referrals (aOR=0.80, p=0.002), as were those supervised in agencies with larger caseloads (aOR=0.95, p=0.036).

Initiation.

Significant contributors to service initiation were confined to agency organizational characteristics. Referred youths were less likely to initiate treatment in counties where staff reported greater levels of intra-agency communication (aOR=0.69, p=0.011), greater levels of job stress (aOR=0.74, p=0.031), and more collaborative practices with service providers (aOR=0.61, p=0.001).

Predicting Days to Screening and Referral

Youths more likely to be screened within a month of intake were older (aOR=1.09, p=0.024) and were receiving higher levels of supervision (aOR=1.33, p=0.033) (Table 3). Juveniles were less likely to be screened within 30 days of intake if they were seen in agencies where staff reported greater levels of both intra-agency communication (aOR=0.51, p=0.002) and job stress (aOR=0.58, p=0.001) and if youths lived in counties where more families were living in poverty (aOR=0.84, p=0.015). Youths more likely to receive service referrals within 30 days of screening were older (aOR=1.51, p=0.002). Those receiving a higher level of supervision were less likely to receive service referrals within 30 days of screening than were those receiving less supervision (aOR=0.35, p=0.028). In addition, youths were less likely to be referred within 30 days if they were supervised in agencies whose staff had larger caseloads (aOR=0.89, p=0.035).

Discussion

The findings from this large, multisite study corroborate and extend previous research, which has consistently shown that justice-involved youths do not receive necessary and appropriate behavioral health services (34). More than 70% of youths were screened at intake for behavioral health concerns, and more than half of those screened required further evaluation or treatment. However, only one in five juveniles with an identified treatment need was referred to further clinical assessment or treatment. By examining outcomes step by step across the cascade, our investigation has documented where service gaps occurred in this process and has identified critical intervention targets to better facilitate cross-system linkage regardless of youths’ level of recidivism risk.

Across all sites, screening of youths was not universal; the unadjusted rate ranged from 7% to 100%. The two strongest contributors to screening were a youth being charged with a substance-related offense and receiving a higher level of justice supervision. Many justice systems select those youths for screening who present with an arrest for a substance-related offense or who have been adjudicated for a crime, both of which may be viewed by agencies as proxies for behavioral health problems. Agency policies do not embrace utilization of screening to identify “hidden populations” of juveniles with behavioral health problems.

Even after screening identified behavioral health needs, justice sites had low rates of referral to community services, and many youths received referrals after a long time in the system. Only about 20% of those identified as being in need of services were referred to providers; of these, fewer than half were referred within 30 days. In adjusted analyses, referral rates were almost five times higher for those in need who were receiving higher levels of supervision, compared with youths receiving lower supervision levels, consistent with longitudinal studies of juvenile detainees (35). Although those at higher supervision levels were more likely to receive service referrals, these referrals occurred less swiftly than for those at lower supervision levels. The length of time between screening and referral could be influenced by court processes, because a referral may not be provided until it is included in a formal court probation service plan.

Staff practices resulting in a greater likelihood of referrals for youths at higher levels of supervision (or resulting in decisions to screen them) appear to be consistent with the risk-need-responsivity (RNR) model (16, 36). The RNR framework directs intensive services to individuals at higher risk of serious recidivism, offering fewer interventions to lower-risk individuals (presuming that these individuals may desist from future offending without more intensive justice involvement). Staff are likely well aware that failure of youths to comply with a service referral could result in increased supervision or more restrictive placement for these juveniles at lower risk for recidivism (37, 38). In a justice agency that both screens for behavioral health needs and follows RNR principles, it is mostly individuals who have high behavioral health need and high recidivism risk who are likely referred to local providers, although all those with screening-identified needs require such services, even if their justice supervision status is considered “low.” Although considerable evidence supports RNR-based policies for targeting the values, attitudes, and family factors contributing to criminal activity among higher-risk individuals, the relevance of this approach for service referral of individuals with behavioral health needs may not be supported (39).

The RNR-informed case management model seems to be at odds with the public health approach underlying the cascade (13). This distinction may follow from the different ways health and justice agencies view the outcomes they address. Identification of most health outcomes presumes that treatment is viewed positively by clients, so that few social or legal consequences for failure to seek treatment are needed. The views on health outcomes may change when negative outcomes affect the larger community’s well-being because of contagion. For health conditions that are seen as contributing to criminal activity, such as substance use or certain mental health problems (11, 18, 19), the justice system is likely to apply punitive consequences for failure to follow through with treatment.

A large proportion of youths in contact with community justice agencies are diverted to other programs, or their cases are handled informally. Of 1,368,200 estimated delinquency cases nationwide in 2010, 68% were either not processed under court oversight or placed on informal supervision (24). Providing access to behavioral health services for juveniles with identified needs can successfully prevent further offending (11, 12). Addressing service needs of youths while avoiding possible justice consequences for treatment nonattendance requires balancing health and justice perspectives for lower-risk youths. For these juveniles, referrals can be offered and supported without punitive legal consequences for noncompliance. Policies to support prevention can further community needs as well as justice agency goals (i.e., preventing reoffending).

In developing a coordination plan, justice and behavioral health agencies should consider the principles underlying each partner’s decisions to refer or treat specific individuals. Justice agency policies that result in sending to local providers only those youths whose profiles most seriously point to possible future criminal activity (i.e., those with a history of more serious offending, antisocial attitudes or peers, or fewer family supports) have implications for clinical practice. These youths are less likely to initiate and engage in offered services (40), and their multiple problems compound the challenges for provider agencies in addressing their diagnostic concerns. Once they engage in behavioral health services, justice-involved youths often drop out or experience service gaps (35). Both lack of parental buy-in and family dysfunction are likely to be associated with lower adolescent behavioral health service use (15, 41, 42). Challenges such as these strongly underscore the need for interagency coordination to support service linkage (15, 35).

This study contributes to our understanding of how staff- and community-level features underlie practices along the cascade. For example, even when the analyses held other features constant, youths with identified behavioral health needs were more likely to receive service referrals if they were seen at supervision agencies with smaller caseloads and when supervised by staff with fewer years of experience in their job. Effectively managing the service referral process, addressing potential barriers, and engaging in practices that support service initiation (e.g., confirming that appointments are kept [17, 25]) require considerable staff effort. More recent hires may have been exposed to training that stresses the importance of understanding the behavioral health needs of youths in the justice system (43).

Although we were unable to pinpoint directionality, some associations likely reflected the consequences of lower rates of service utilization. After referral, for juveniles who did not initiate treatment (compared with those who did), staff reported greater levels of intra-agency communication, more job stress, and more collaborative practices with local providers, perhaps as they struggled to address nonattendance. Although contributors to optimal agency practice deserve further study, advance interagency planning that clarifies which youths will be referred, to which partner agencies, and how attendance will be monitored may actually reduce effort as agencies spend fewer resources determining how to resolve similar problems in the future.

As is the case with most descriptive investigations of agency services, this study had several limitations. Chief among these might be the high level of missing data in administrative records and the nonrandom selection of participating states and local sites. Our conservative approach to estimation (22) was applied to address the former, and despite the latter, participating sites displayed wide ranges on most measures, highlighting the great variability in these measures across such systems nationwide.

Conclusions

In this multisite study, we found that justice agencies identified youths’ treatment needs but fell short in making referrals that should have resulted from that identification. The strongest predictor of youths’ participation in behavioral health services was their level of supervision. This finding suggests that justice agencies may utilize a model for service referral (i.e., RNR) that may not support individuals with behavioral health needs, focusing instead on those with a serious risk for reoffending, although both systematic screening and referral are possible within these settings. The failure to provide juveniles at lower levels of supervision with needed treatment referrals is worrisome, given their high numbers and the value of these services in preventing further justice system involvement. Increased cross-agency planning should facilitate systematic referral of those whose needs have been identified, reducing unmet youth service needs and, in turn, enhancing both public safety and public health.