Toward Evidence-Based Practice for Probationers and Parolees Mandated to Mental Health Treatment

Individuals with mental illness and substance use disorders are grossly overrepresented in the criminal justice system. The six-month prevalence rate of serious mental illness among jail detainees is nearly four times as high for men and more than eight times as high for women than in the general population (1,2,3). Of detainees with mental illness, 75 percent have a co-occurring substance use disorder (2). Recently, the number of people under correctional supervision reached an all-time high of 6.9 million individuals (4). Three out of four of these people are supervised in the community on probation or parole; the vast majority (93 percent) are on probation (4). Mental illness and co-occurring substance use disorders are at least as prevalent among probationers as among jail detainees (5). Together, these factors suggest that at least half a million individuals with mental illness are placed on probation or parole each year, which constitutes 15 percent of the total population under community supervision.

Compared with probationers without mental illness, those with mental illness are highly likely to fail on supervision—that is, to have their probation or parole revoked for violating its terms or committing a new offense. Porporino and Motiuk (6) followed 36 pairs of parolees with and without mental illness for two years after their release from prison. Parolee pairs were matched in age, offense, and sentence. Parolees with mental illness were twice as likely as those without mental illness to have their parole suspended (65 percent compared with 30 percent) and were significantly more likely to have their parole revoked without committing a new offense. Probationers with mental illness are also at risk of short-term and long-term failure. In a study of an unmatched sample of 613 probationers who were followed for three years, Dauphinot (7) found that probationers with mental illness were significantly more likely to have probation revoked than those without mental illness (37 percent compared with 24 percent). Moreover, the rate of rearrest for probationers with mental illness was nearly double that of the comparison group (54 percent compared with 30 percent).

The enormity of this problem prompted the Council of State Governments (8) to recommend that individuals with mental illness be assisted in complying with the conditions of probation and parole, in part by having agencies adopt specialized mental health caseloads. Specialized agencies assign offenders with mental illness to officers with some training in mental health who have relatively small caseloads, rather than to general officers with large, mixed caseloads. The assumption is that specialty caseloads provide officers with the resources to gain better access to mental health services, respond to minor violations with intermediate sanctions, and promote supervisees' reentry to the community.

The assumption about specialty caseloads has yet to be examined rigorously. Research has only begun to focus on probationers and parolees with mental illness, despite the fact that there are more people with mental illness under community supervision than anywhere else in the correctional system. This article reviews this small but growing body of research.

As discussed below, investigators have made steps toward understanding why individuals with mental illness are at greater risk of supervision failure than those without mental illness. Investigators have also begun to describe and evaluate existing practices meant to address this problem. The literature is in too early a stage of development to make definitive statements about evidence-based practice in probation and parole, but it provides a sound foundation for outlining a research agenda to advance the field toward evidence-based practices. With this goal in sight, this review is organized to address four questions.

First, what is the problem? To provide a foundation for evidence-based practice, we must identify the reasons that individuals with mental illness are at greater risk of failure in community supervision. Risk factors for failure that can be changed are appropriate targets for intervention.

Second, what interventions have been tried? With an imperfect understanding of the problem, communities have responded by developing specialty mental health agencies. A description of these agencies and their differences from traditional agencies is needed to identify appropriate agencies to compare as well as to disseminate specialty practices, if they are shown to be effective.

Third, how effective are these interventions? Available research describes opinions about the effectiveness of specialty agencies, and researchers have begun to evaluate them. To qualify as an evidence-based practice, these agencies must be more rigorously evaluated.

Fourth, what general principles of practice hold promise? To promote positive outcomes across supervision contexts (probation and parole agencies with varying resources), we must look beyond "trademarked packages" or programs to identify the principles of change that may underlie them (9).

As we review research that addresses each of these questions, we provide recommendations for studies designed to identify evidence-based practices. This article places an emphasis on understanding the role of mandated treatment in outcomes, because participating in mental health treatment as a special condition of probation (10) or release on parole (11) is frequently required by judges and parole boards, because providing mandated treatment with close monitoring has been cast as a crucial aspect of integrated treatment for individuals with co-occurring disorders who are involved in the criminal justice system (12), and because monitoring and enforcing treatment compliance are viewed as officers' most fundamental and challenging task in supervising probationers and parolees with mental illness (13,14,15). The manner in which officers implement treatment mandates may facilitate or block goals of treatment engagement and reentry.

Methods

Research reports were identified by performing a keyword search that combined community supervision (probation, parole, community corrections) with mental illness (mental illness, mental disorder). Several databases were searched for articles in English published from January 1975 to April 2005. The databases included PsycINFO, Criminal Justice Abstracts, Social Services Abstracts, Sociological Abstracts, ERIC, MedLine, the National Institute of Corrections, and the American Probation and Parole Association. All studies that focused on adult probationers or parolees with mental illness were included. Emphasis was placed on articles that described the effectiveness of supervision or intervention for probationers or parolees. Because there were few such articles, reports from investigators known to have conducted systematic evaluations of specialty agencies were included.

Results

What is the problem?

Reasons for supervision failure. Although research consistently indicates that individuals with mental illness are at double the risk of supervision failure, few investigators have examined why this is the case. Addressing this issue is a crucial step toward developing empirically based interventions for these individuals, given that a chief goal is to reduce risk of supervision failure (16). To reduce risk, these interventions must target dynamic risk factors that "when changed, are associated with changes in the probability of [supervision failure and] recidivism" (17). Risk factors for supervision failure may be intrinsic to an individual (dispositional, clinical, or historical), may be environmental (contextual), or both.

Mental illness and supervision failure. The link between mental illness and supervision failure can be characterized as direct, indirect, or spurious. Mental illness constitutes a clinical risk factor of primary interest. Mental illness may be related to supervision failure in several ways. First, mental illness may directly cause supervision failure. A probationer with active psychosis who shouts obscenities at neighbors and makes threatening gestures may be arrested. A parolee with mental illness who is vulnerable to drug use will violate the terms of parole when he or she uses heroin and could also be arrested. It is important to remember that "mental illness" in the context of community supervision typically means co-occurring disorders, given that three out of four offenders with mental illness also have a substance use disorder. The disorders interact, in that individuals with mental illness are particularly vulnerable to the effects of drugs and experience more severe symptoms and disturbances in behavior (18). A direct causal relationship between mental illness and supervision failure would suggest that the target of intervention should be mental illness or co-occurring disorders. Increasing participation in evidence-based practices for these disorders would directly reduce the risk of supervision failure.

Second, mental illness may cause supervision failure through an indirect effect on other factors that heighten the risk of failure. For example, mental illness is associated with an increased potential for violence, particularly when it co-occurs with a substance use disorder (19) and treatment noncompliance (20). Aggressive behavior, in turn, heightens the risk of supervision failure. In addition, mental illness may be associated with functional impairment that renders a probationer of parolee unable to comply with such standard conditions as maintaining employment, paying supervision fees, or completing community service (21). If functional impairment (or some other factor) completely mediated the relationship between mental illness and supervision failure, it would be the appropriate target of intervention. If the indirect effect was weaker, both functional impairment and mental illness would be appropriate treatment targets.

Third, the relationship between mental illness and supervision failure may be a spurious one caused by some other influence that relates to both. For example, poverty, living in a disadvantaged neighborhood, and having poor social networks are risk factors for both mental illness and antisocial behavior (22). Moreover, individuals with mental illness and with criminal tendencies are relatively closely monitored by agents of social control. Intensive monitoring, in turn, is associated with increased discovery and punishment of poor behavior (23). Given the triple stigma of mental disorder, substance abuse, and criminal justice involvement, probationers and parolees with co-occurring disorders may be perceived as "mad, bad, and dangerous" and watched especially closely. In short, those with mental illness may often fail supervision not because of their disorder but because they tend to have more of the "usual" risk factors for failure, or they are monitored more closely by treatment and correctional personnel. In this case, strategies that focus on intensifying "treatment as usual" for mental illness will not be effective. Instead, interventions should focus on the spurious factor—changing social resources or the process of social control.

Glimpses of the reasons for failure. Three studies relevant to assessing the nature of the link between mental illness and supervision failure have been conducted. First, Dauphinot (7) studied whether the reasons for revocation of probation varied as a function of mental illness. She compared the reasons cited for revocation of probation of 43 individuals with mental illness and 326 probationers randomly selected from an annual database of revocations who may or may not have had a mental disorder. This study must be viewed as exploratory, because no effort was made to ensure that the two samples did not overlap or to code a primary reason for revocation in each case.

For the sake of simplicity, we describe differences that held for both felons and misdemeanants and present data only from the subset of felons in each group (33 in the group with mental illness and 243 in the comparison group). The results indicate that the reasons for revocation for those with mental illness were less likely to include a new arrest (3 percent compared with 12 percent for the comparison group), equally likely to include a new felony conviction (36 percent compared with 35 percent), and more likely to include a new misdemeanor conviction (24 percent compared with 5 percent). With respect to technical violations of the conditions of probation, the reasons cited for revoking probation for both groups were equally likely to include failure to report to the probation officer (58 percent compared with 65 percent) and positive urinalysis (36 percent compared with 35 percent). However, for those with mental illness, reasons for revocation were more likely to include failure to pay fines or fees (79 percent compared with 60 percent) and "other" violations, such as failure to work (36 percent compared with 0 percent). These results suggest that persons with mental illness have functional impairments that complicate their ability to follow such standard conditions of probation as paying fees (an indirect causal relationship between mental illness and failure). These results also suggest that probation officers and judges may set lower (or different) revocation thresholds for those with mental illness (a spurious relationship).

The latter possibility is consistent with the results of a second study. In this study, Solomon and colleagues (23) followed 250 individuals for one year who were supervised by specialty mental health probation and parole officers to identify risk factors for incarceration. They interviewed the probationer and his or her officer every three months or until the client was incarcerated for a technical violation (16 percent) or new offense (18 percent). The probationers' characteristics (clinical, criminal, and demographic) and treatment motivation were considered as potential risk factors, as were officers' supervision strategies and perceptions of the client. The strongest risk factors for incarceration on a new offense included probationers' belief that medications were not helpful (odds ratio [OR]=4.9) and officers' perception of poor treatment motivation (OR=2.7) and high risk of violence to others (OR=2.7). The strongest risk factors for incarceration on a technical violation were similar but also included use of psychiatric medication during the follow-up period (OR=9.5). A comparison of probationers incarcerated for new charges and probationers incarcerated for technical violations revealed that the latter group was six times as likely to have received intensive case management services and more likely to have had high scores on the hostility scale of the Brief Psychiatric Rating Scale. Additional analyses suggested that officers who collaborated with intensive case managers were 12 times as likely to threaten probationers with incarceration (24).

These results are consistent with Solomon and Draine's (25) previous observation that case managers often sought reincarceration on a technical violation to secure jail-based treatment for probationers and parolees who were perceived to be noncompliant with treatment and decompensating. More generally, the results are less consistent with a direct causal relationship between mental illness (for example, diagnosis and symptom severity) and supervision failure than with a spurious relationship that reflects increased monitoring and control of individuals with low motivation and poor attitudes. Notably, the failure to find a direct relationship may be partly attributable to the sample's restriction in range, given that all clients in this study had serious mental disorders. Moreover, it is unclear whether these risk factors will generalize to traditional probation and parole settings, in which caseloads are larger and fewer resources for monitoring and control are available or to more prototypic specialty settings in which the "treater-turned-monitor" phenomenon is explicitly avoided (15,26).

Skeem and colleagues (15) conducted a series of five focus groups in three states to help understand the range of influences that probationers and probation officers viewed as influencing probationers' outcomes, including supervision failure. Participants were 32 probationers with mental illness and their probation officers from specialized or traditional probation agencies. Although probationers and their officers participated in separate groups, their perceptions were consistent in suggesting that three factors contribute to poor outcomes. The first factor was officers' use of negative pressures, such as threats of incarceration, as a strategy for ensuring compliance with the conditions of probation. As observed by one officer, "What happens is you create more anxiety when you're threatening to send them to jail. They don't want to go to jail—they're not stupid—they're a little bit crazy. And then they'll stop coming in because they're afraid.'' The second factor was probationer-officer relationships that were uncaring, unfair, and disrespectful; such relationships were perceived as negative influences that colored every interaction and that often led to supervision failure. Although the tide may be changing with the "reinventing of probation," officers tend to focus on their surveillance role (control) nearly to the exclusion of their therapeutic one (care) (27). As noted by Clear (28), when probationers fail to follow orders, "it is not merely a breach of good mental fitness, it is a violation of the law. Stern responses often ensue."

A third factor that was perceived as harmful to these probationers was the limited resources and goals of traditional probation agencies. For decades, probation and parole officers have "struggled with unreasonable caseload numbers and expanding workload expectations" (29), with the essential goal of protecting public safety. Probationers with mental illness had pronounced needs for treatment and other social services, such as housing and disability benefits, that fell outside the range of officers' ordinary duties and that were difficult to address. Traditional officers were provided with no resources for supervising probationers with mental illness any differently than other probationers. Generally, they appeared ill at ease with supervising probationers with mental illness. They often perceived these probationers as potentially dangerous entities to be watched carefully until supervision could be ended through transfer to another officer, revocation, or completion of probation. The results of this study suggest an indirect relationship between mental illness and supervision failure in which individual risk factors (needs for services needs, functional impairment, and reduced coping resources) interact with contextual risk factors (limited resources in traditional supervision, probationer-officer relationships, and officers' strategies to ensure compliance). The results also lend some support to a spurious relationship that involves increased monitoring.

In summary, research has only begun to investigate the nature of the problem that underlies the link between mental illness and supervision failure. Preliminary findings provide little support for the premise that the link is a direct causal one. Instead, the link seems to involve both factors associated with mental illness that heighten risk of failure (an indirect effect) and the interaction of these factors with the typical context of community supervision (a spurious effect).

Recommendations for better understanding the link. To better understand the causal chains that lead to supervision failure among probationers and parolees with mental illness, longitudinal studies with repeated observations are needed (30). For example, to understand probation revocation (rather than recidivism), one could follow a matched sample of probationers with and without mental illness from the initiation of probation through the period of greatest risk of revocation. Matching variables would be those related to risk of revocation, such as age and offense type, and probationers who had a history of revocation would be excluded. At baseline and subsequently at regular intervals, promising individual and contextual risk factors for revocation would be assessed. The occurrence, timing, and type of revocation (a new offense versus a technical violation) would be assessed as the outcome. This design would permit investigators to identify causal dynamic risk factors—factors that change over time and increase risk of revocation. These are the most appropriate targets of evidence-based practice.

What interventions have been tried?

As is the case in other fields of intervention, scientific understanding lags behind practice in community supervision. Research has yet to pinpoint the problem that underlies the link between mental illness and supervision failure. Nevertheless, communities have responded by developing specialty mental health caseloads in probation and parole (8). Describing the nature of these specialty agencies is an essential step toward selecting appropriate agencies for empirical comparison and, ultimately, disseminating evidence-based practices. To date, specialty agencies have been described in more detail for probation than for parole.

Describing specialty probation. Skeem and colleagues (15) conducted a national survey to characterize supervision practices for probationers with mental illness in traditional and specialty agencies and to assess the degree of heterogeneity among specialty agencies in their structure, philosophy, and practices. Of interest was whether specialty agencies shared enough features to define a single, prototypic model. The study involved three stages. First, all specialty agencies in the United States were identified, chiefly by contacting probation executives at all levels of oversight—state, regional, and local. Through these efforts, 137 probation agencies with at least one mental health caseload were identified, representing 5 percent of an estimated 2,600 agencies (31). Agencies with single mental health caseloads (where the practices of one officer define the agency) were screened out, as were agencies with mixed mental health caseloads (for example, those that included sex offenders). Seventy-three eligible specialty agencies remained. Of these, 66 (90 percent) participated. These agencies were concentrated in large urban areas, and they had been operating for a mean±SD of 9±6 years.

In the second stage of the study, a relatively small sample of traditional agencies that matched the specialty agencies in geographic region and population size was identified. Of the 26 identified traditional agencies, 25 (95 percent) participated. In the third stage the survey was conducted with supervisors of the 66 specialty and 25 traditional agencies. The survey consisted of a 45-minute telephone interview and follow-up questionnaire. Use of a case vignette helped to structure questions about strategies for enforcing treatment mandates.

Multivariate analyses of specialty agencies' structure, case management style, and enforcement strategies indicated that there was a single specialty model. Five features were most defining of this prototypic model. They were shared by most specialty agencies and made them maximally different from the traditional model.

The first feature is exclusive mental health caseloads. Officers who supervise probationers with mental illness supervise only this type of probationer. Mixed caseloads that include general probationers or sex offenders dilute resources for—and the focus on—mental illness (32).

The second feature is meaningfully reduced caseloads. On average, specialty caseloads consist of 45 probationers, which is approximately one-third the size of traditional caseloads. These smaller caseloads permit officers time to address probationers' risk and needs. The third feature is sustained officer training. Officers with interest or experience in mental health receive sustained training in relevant issues. Prototypic agencies provide 20 to 40 hours of mental health training annually to enhance officers' knowledge base.

The fourth feature is integration of internal and external resources. Officers intervene directly with probationers and actively coordinate with probationers' external service providers. Officers work as teams with treatment providers, attend treatment team meetings, and advocate to secure such appropriate treatment and social services as Supplemental Security Income and housing. Officers are not merely referral sources and monitors. The fifth feature is the use of problem-solving strategies. Officers are likely to address treatment noncompliance by talking with the probationer to identify any obstacles to compliance, resolve these problems, and agree on a compliance plan. If a probationer is reluctant to take medication, the officer might have a fair and respectful talk with the probationer about the problem, learn that the probationer was experiencing serious side effects, make plans with the probationer to seek a medication change, and agree on compliance with the new prescription. Specialty officers are unlikely to merely remind probationers of the rules or threaten them with incarceration.

Specialty agencies that deviated from these prototypic features tended toward the practices of traditional agencies. One of the most common deviations took the form of elevated caseload size. In about one-fifth of specialty agencies, officers were supervising 30 or more cases above the limit set by agency policy. Practices in these large-caseload specialty agencies were significantly less likely than those in lower-caseload specialty agencies to involve problem-solving strategies (r=-.20) and significantly more likely to involve revocation and jail as a strategy (r=.26). The few specialty agencies that presented jail as a viable strategy for addressing noncompliance described this option as a means of obtaining stabilization on medication or providing a "wake up call." This finding contradicts most specialty agencies' use of jail as the last resort, as well as the Council of State Government's (8) assertion that specialty probation officers "are much more likely [than traditional officers] to seek out and arrange revised treatment options … in lieu of issuing a warrant and instituting violation proceedings that would likely result in re-incarceration."

As part of the survey, supervisors rated the usefulness, practicality, and effectiveness of specialty agencies on a 3-point scale (not at all, somewhat, and very). Virtually all supervisors (87 percent) perceived specialty agencies as very useful. Compared with specialty supervisors, traditional supervisors were less likely to view specialty agencies as practical; a majority (56 percent) described reduced caseloads as not at all practical. Specialty supervisors were significantly more likely than traditional supervisors to view their agency as very effective in improving probationers' well-being (62 percent compared with 4 percent) and reducing their risk of probation violation (20 percent compared with 53 percent). No significant differences were found in perceived effectiveness in reducing the risk of longer-term reoffense.

Describing specialty parole. Relatively little research has been done on mental illness and parole, perhaps because prisoners with mental illness seldom qualify for parole (32) or parole agencies rarely attend to mental health issues. A decade ago, Boone (34) analyzed questionnaire data and archival data supplied by state administrators who were deemed most knowledgeable about mental illness in the probation and parole population. Of the 41 state representatives who responded (82 percent), ten (24 percent) reported that their state had at least one specialty caseload for offenders with mental illness. However, it is unclear how many of these caseloads were for parolees and how many were for probationers. In addition, the sampling procedure was unclear, leaving open the possibility that the sample does not represent the thousands of local probation and parole agencies across the nation.

The little available information suggests that specialty caseloads in parole are similar to those in probation—for example, with specially trained officers (35,36). The overlap between specialty programs is apparent in that at least one prototypic program with the features described above included both probationers and parolees (26). Some defining features of specialty parole may include exclusive mental health clinics for parolees, as offered in California (26), or collaboration between parole officers and treatment providers (36). Clearly, however, systematic research is needed to define the prevalence and properties of specialty parole interventions.

How effective are the interventions that have been tried?

Although research has begun to describe practices in the prototypic specialty agency, few evaluations of these agencies (prototypic or not) have been conducted. The available evidence consists of stakeholders' opinions on the effectiveness of these agencies, which is described above, and several outcome evaluations. Although this literature is not yet well developed, it provides a foundation for developing evidence-based practices. In this section, we define "evidence-based practice," review research on the effectiveness of specialty agencies, and outline an agenda for further validating these agencies.

Defining evidence-based practice. Few concepts have had a deeper impact on the fields of mental health (34) and community corrections (37) than the notion that the best research evidence can be applied to promote the best practice. These movements may become blended. In October 2004 President Bush signed the Mentally Ill Criminal Offender Treatment and Crime Reduction Act (PL 108-414) into law. The program included funds to summarize "best practices" for offenders with mental illness. In November 2005, the Senate approved a five million dollar appropriation for this program, which currently awaits the President's signature.

The concept of evidence-based practice most applicable to probationers and parolees is broader than that traditionally used in medicine or psychology (38). In clinical psychology, for example, application of the most commonly used rules for defining "empirically supported treatments" have resulted in a list of interventions in which the prototype is highly manualized, focused on a narrow diagnostic group, and supported by at least two randomized experiments in which the control condition was either another bona fide treatment or psychotherapy placebo (that is, a psychological intervention without the theoretically defined active ingredient of the proposed psychotherapy) (39).

A cogent criticism of such narrowly defined evidence-based practices is that they may be efficacious in the research laboratory but not effective in real-world practice (39). This issue is critical for probationers and parolees. Rather than fitting neat diagnostic categories, these individuals suffer a range of mental disorders (7). Moreover, these individuals are treated and supervised in overburdened public systems. Providing clinicians and officers with training, an intervention manual, and monitoring for adherence is a laudable goal. However, it is a lofty one. Instead, we seek the "optimal intervention … the least extensive, intensive, intrusive, and costly intervention capable of successfully addressing the presenting problem" (40).

Specialty caseloads present an opportunity for intervention that can affect the outcomes of probationers and parolees with mental illness. In this article, we apply the definition of evidence-based practice used in the Psychiatric Services series on evidence-based practices in 2001: "an intervention for which there is strong research demonstrating effectiveness in assisting consumers to achieve outcomes" (12,27). We do not apply an explicit rule for defining strong research evidence, given the breadth of research strategies that can generate valuable evidence (41) and the arbitrariness of such rules. Instead, we review relevant evidence with respect to an evidence hierarchy and explain our reasoning in reaching conclusions about the strength of the evidence. We place studies that randomly assign individuals to specialty and traditional supervision at the top of the hierarchy, because these trials are the standard for drawing causal inferences about the effects of an intervention. We highlight studies of prototypic specialty agencies and traditional agencies as those best suited for drawing general inferences about the effectiveness of the specialty model—not an "enhanced" or "weak" version of it. Case studies are useful for generating hypotheses, but they are placed at the bottom of the evidence hierarchy.

The effectiveness of specialty probation. Findings from the focus group study and national survey described above indicate that probationers, officers, and supervisors perceive specialty agencies to be more effective than traditional agencies for supervising probationers with mental illness. These perceptions are consistent with direct data. Roskes and Feldman (36) studied 16 probationers with mental illness who were supervised and treated in their own specialty agency. Probationers' rate of violation of probation or parole before entering the program was markedly higher than their rate during specialty supervision (56 percent compared with 19 percent). Although this study was not controlled, involved a small sample, and had a nonuniform follow-up, its results suggest that specialty agencies hold promise.

This promise is bolstered by projects recently completed as part of California's Mentally Ill Offender Crime Reduction Grant. Grantees were provided with funds to implement and evaluate an intervention for these offenders. Although the formal report is pending, findings are available on two of three projects that prioritized specialty probation, involved multiple specialty caseloads, and involved true random assignment.

In an evaluation of the Connections program in San Diego, Burke and Keaton (42) randomly assigned probationers with mental illness to specialty probation and case management (N=225) or traditional probation with "treatment as usual" (N=224). The specialty condition was an expensive, nonprototypic variant of the specialty model, in which specialty supervision was combined with intensive case management by designated social workers (staff to client ratio of 1 to 10), availability of the treatment and supervision team around the clock, and early intervention by a separate psychiatric emergency team. In the comparison condition, individuals were followed through whatever traditional supervision and standard treatment they happened to receive.

The Connection program's retention rate was limited; only 58 percent completed the program. Probationers who were classified as "noncompleters" chose to leave the program (16 percent), committed a new offense or were sentenced to prison (16 percent), or were dismissed for unacceptable behavior (7 percent). These individuals had more previous jail bookings and convictions and were more likely to be homeless than those who completed the program. These rules of retention were not applied to identify "noncompleters" in the comparison condition and to exclude them from the analyses. Compared with all the comparison probationers (who received traditional supervision and treatment as usual), probationers who completed Connections obtained significantly more mental health services during supervision and were less likely to be charged with a new offense (10 percent compared with 2 percent) or to be booked into jail (26 percent compared with 12 percent) during a six-month follow-up. These probationers also demonstrated significant improvement in functioning—for example, higher scores on the Global Assessment of Functioning and on a measure of substance abuse.

For three reasons, these results provide little support for specialty probation supervision. First, the evaluation compared the highest-functioning probationers in the Connections program with all probationers who participated in the comparison condition. Excluding from analyses probationers who failed to complete the Connections program—but not excluding those who failed to complete treatment as usual—amounts to "stacking the deck" in favor of Connections. Second, the Connections program represents an expensive program that more closely approximates the assertive community treatment (43) model on which it is based (42) than the specialty supervision model (15). The prototypic specialty program does not involve automatic access to a predefined, intensive case management program (15). It seems unlikely that the results for the Connections program would generalize to the more prototypic specialty context in which officers with reduced caseloads leverage services from the traditional mental health care system. Third, the program excluded probationers who were mandated to treatment, who may represent a majority of probationers with serious mental illness (7).

Many of these issues were addressed in the evaluation of the IMPACT program of Orange County, California, which was conducted by an independent research firm unaffiliated with correctional, mental health, and government agencies involved with the program. Investigators randomly assigned 800 offenders with mental illness who were eligible for probation to one of four groups: a control group not on probation, a group that was not on probation but received treatment (case managers provided jail aftercare), a control group on traditional probation, and a group on specialty probation (case managers and specialty officers provided services and supervision). Although case managers were hired for the project, intensive case management was rarely provided and specialty officers often acted as case managers toward the end of supervision. Because the program is more representative of the prototypic specialty model (15) it permits a better assessment of the effect of specialty supervision independent of increased mental health services. Probationers mandated to treatment were not excluded.

The results indicated that specialty probationers received significantly more mental health services and filled more prescriptions than any other group. However, there were no significant differences among the groups in their rates of booking back into the local jail during the follow-up (personal communications, Cunningham J, July 14, 2004, and April 25, 2005).

This study suggests that specialty probation works better than both traditional probation and case management alone in providing access to resources, but these increased resources do not translate into fewer jail bookings.

A multisite evaluation of jail diversion programs also indicated that rates of service use were unrelated to rates of rearrest (44). There are two potential explanations for the lack of an inverse relationship between treatment services and recidivism rates. First, the services may be poor-quality treatment that fails to reduce symptoms of mental illness and substance use disorders. Assuming that these symptoms are causally related to the risk of recidivism, receiving greater amounts of ineffective treatment would not reduce recidivism rates. Second, these services may simply fail to target risk factors for crime and violence that would be necessary for reducing rates of recidivism (45). Future evaluations should attend to measuring not only the quantity but also the quality and nature of treatment services received.

In summary, two studies—one focus group study and one national survey—suggest that probationers with mental illness, probation officers, and probation supervisors perceive specialty caseloads as more effective than traditional caseloads. Three additional studies—two randomized controlled trials and one uncontrolled cohort study—suggest that specialty agencies are more effective than traditional agencies in linking probationers with treatment services, improving their well-being, and reducing their risk of probation violation. Evidence is mixed on whether specialty agencies reduce probationers' longer-term risk of rearrest. To date, no controlled comparisons of prototypic specialty and traditional agencies have been published in peer-reviewed journals. Given the strength of this evidence, specialty agencies qualify as a "promising practice" for supervising probationers with mental illness (46).

The effectiveness of specialty parole. As described above, Roskes and Feldman (36) found that parolees' rate of violations during supervision in a specialty agency was lower than these individuals' previous violation rate. Garrow and colleagues (47) conducted detailed interviews with seven clinicians and parole officers associated with a multidisciplinary, collaborative reentry program that included both specialty and traditional caseloads. The authors found that only the specialty officer combined therapeutic and correctional philosophies, used intermediate sanctions appropriate to parolees with mental illness, and expressed confidence about outcomes. Moreover, Wiederanders and Sprinkman (48) evaluated California's Mentally Disordered Offenders program, a small branch of the CONREP program. In this program, parolees with mental illness and violent offenses who were judged to be at risk of future violence were mandated to treatment as a condition of parole. The authors found that only 8 percent of these individuals were rearrested during supervision, although 40 percent were rearrested during an average two-year follow-up. Although no appropriate high-risk comparison group is available to evaluate the rate of 40 percent, it is considerably lower than found in a study of California parolees (70 percent) who were followed for only 1.5 years (49).

In summary, three studies suggest that specialty agencies are effective in reducing parolees' short-term risk of parole violation, but none of these studies used control groups. At present, specialty parole agencies are, at best, on the cusp of qualifying as a "promising practice" in the supervision of parolees with mental illness.

Recommendations for evaluating specialty agencies. To elevate specialty probation and parole caseloads to the status of evidence-based practice, more studies and more rigorous designs are needed. Investigations that are designed, funded, conducted, and reported independent of government oversight agencies are needed, given the pressure that these agencies are often under to demonstrate that resources have been allocated wisely. Controlled, peer-reviewed comparisons that include standardized measures of outcomes are also needed. Given the size of the population of offenders with mental illness under community supervision and their risk of failure, a systematic line of research in this area is overdue.

Investigators may find it difficult to conduct truly randomized controlled trials in real-world settings that involve the court, correctional, and mental health systems. However, rigorous matched trials may be conducted in their stead. Ideally, the agencies selected for inclusion in these trials would be located in jurisdictions with similar mental health resources. These agencies would also approximate the prototypic specialty and traditional agency. (For parolees, the prototypic specialty agency must first be defined.) Additional studies of resource-intensive programs that combine specialty supervision with intensive case management or other nonprototypic features will not be helpful. These studies yield findings about a model that most jurisdictions do not have and will find infeasible to implement.

As we assess how well specialty caseloads "work," we should also assess how they work. Dismantling designs that compare the effectiveness of the prototypic specialty agency with specialty agencies in which only one feature is manipulated, such as reduced caseload size, could isolate the necessary ingredients of specialty caseloads and describe their parameters, such as the minimum caseload size.

Promising general principles of practice

Ultimately, specialty agencies are supervision programs that include such broad features as reduced caseload size. To promote positive outcomes across supervision contexts, we must look within such programs to identify the principles of change that may underlie them (9). The question of real interest is, "What principles, requiring what resources, applied in what context, produce the most meaningful change?" (50). If dismantling designs indicate that caseload size is an important ingredient of the specialty package, is it because officers are doing something more or different as a function of reduced caseloads that affects outcomes? If so, what is that something?

Process-outcome studies are particularly valuable for identifying mechanisms of change (41). General practice principles that build on these mechanisms may cross probation and parole contexts, given that judges have begun sentencing more serious offenders to probation (51) and the emphasis on treatment mandates "and the techniques for monitoring probationers and parolees are very similar" (52). Here, we summarize promising practice principles for supervising probationers and parolees with mental illness. These principles are based on research findings and are presented as provisional mechanisms for further study.

Systemic. It appears that dwindling resources must be creatively maximized to address the pronounced needs of probationers and parolees with mental illness. A necessary condition for effective supervision of these individuals may be adequate time. As shown in the national survey described above, specialty probation agencies without meaningfully reduced caseloads often operate no differently than traditional ones (15). One reason for this finding is that large caseloads may prevent probation officers from functioning as "boundary spanners" (53) who develop knowledge about mental health and community resources, establish and maintain relationships with clinicians, advocate for services, and actively supervise these individuals.

Although reduced caseloads may be dismissed as impractical, they may create administrative efficiencies. In programs with smaller caseloads, the supervisee with mental illness is no longer seen as a "problem to the system" but as a typical case that requires routine procedures, which could save resources (31). Cost-effectiveness studies should address this issue. Nevertheless, we should find an efficient way to intervene, given the evidence that this group is particularly likely to fail under traditional supervision.

Relational. Probation and parole officers have both "a legalistic, or surveillance, role," and a "helping, therapeutic, or problem-solving role" (54). Probation officers and probationers with mental illness believe that the quality of their relationship colors every interaction to strongly influence outcomes (14). On the basis of this finding, we conducted a study to develop a sound measure of the quality of relationships in mandated treatment or "dual role" contexts (unpublished manuscript, Skeem J, Eno Louden J, Emke-Francis P, et al, 2005). To develop this measure, we coded audiotaped meetings of a sample of 90 officer-probationer pairs drawn from a specialty mental health agency and asked participants to complete a battery of relevant measures. The data suggest that the instrument developed is reliable and valid and taps three components of relationship quality: caring and fairness, trust, and toughness. These components were related to officers' confrontation and probationers' resistance and cooperation during officer-probationer meetings. Negative aspects of relationship quality (toughness) were associated with greater probation violations during a two-month follow-up. Positive aspects of relationship quality (trust along with caring and fairness) were associated with fewer probation violations.

In psychotherapy research, the quality of the relationship explains substantially more variance in patient outcomes than the specific therapy techniques applied (55,56). In the search for evidence-based practices in community supervision, it will be important to assess relationship quality as a potential mechanism of change.

Strategic. Nontraditional strategies for addressing noncompliance also hold promise as a component of effective practice in the supervision of individuals with mental illness. The problem-solving strategies described above involve collaboratively resolving issues of noncompliance and are perceived as helpful by probation officers and probationers with mental illness (14). Traditional strategies of issuing rule reminders and threatening incarceration are unlikely to resolve noncompliance, particularly if noncompliance is based on inability or contextual barriers.

These relatively quick and easy strategies may be favored by professionals in environments that are resource impoverished.

Improving practice through research and implementation

In this article, we have made research recommendations for better defining the problem by examining the assumption that untreated mental illness directly underlies probationers' and parolees' greater likelihood of supervision failure. We have described what has been done to address the problem by articulating the features of specialty probation and parole agencies. We have made recommendations about determining whether current practice packages "work" by further evaluating specialty caseloads. We have also isolated the potentially active ingredients of effective supervision.

Although such research is necessary to move the field toward evidence-based practice, action is also needed. A large number of individuals with mental illness are floundering in large caseloads in overburdened systems. These individuals do not fit the correctional system well, and they often fail supervision. The research reviewed here suggests that specialty caseloads improve their functioning, enhance access to services, and reduce the risk of probation violation. In short, existing practices hold promise. Implementing these practices may prevent offenders with mental illness from becoming more deeply entrenched in the criminal justice system, which would better realize the supervision goal of facilitating their reentry to the community.

Acknowledgments

This work was supported by the John D. and Catherine T. MacArthur Research Network on Mandated Community Treatment. The authors thank Paula Emke-Francis and Jacqueline Camp for their assistance.

The authors are affiliated with the department of psychology and social behavior at the University of California, Irvine, 3311 Social Ecology II, Irvine, California 92697-7085 (e-mail, [email protected]).