Assessing the New York City Involuntary Outpatient Commitment Pilot Program

Although involuntary outpatient commitment is widely permitted throughout the United States (1), its advisability and effectiveness remain contested (2,3,4,5). Studies have provided little evidence of its effectiveness (6,7,8,9,10,11,12,13,14,15,16), with one exception (16). However, interpretation of the results of that study is complicated by a host of confounding problems. For example, eligibility criteria were poorly specified, leaving open the issue of selection bias. Effects were poorly monitored, making it unclear how many persons complied with ordered treatment. In addition, several uncontrolled variables, such as enhanced services, secure housing, and competent case management, may rival legal coercion in accounting for positive effects.

In 1994 the New York state legislature authorized a three-year pilot program in outpatient commitment to be conducted at Bellevue Hospital and independently evaluated. The required design called for random assignment of eligible persons referred to the program either to court-ordered treatment, including enhanced services, or the enhanced-service package alone. Enhanced services included inpatient assessment, a comprehensive postdischarge treatment plan in which the patient participated, arrangements for ongoing case management, and continued oversight of the patient by the outpatient commitment coordinating team. For the group that received court-ordered treatment, the outpatient treatment plan was formalized by a court proceeding and an explicit judicial order.

Under the law (chapter 560 of the state's Mental Hygiene Law, section 9.61.4), to be eligible for outpatient commitment, a current Bellevue inpatient must be 18 or older, with at least two involuntary hospitalizations in the past 18 months because of noncompliance with treatment. Eligible persons also must be unlikely to comply with services once discharged and thus can be considered in need of involuntary treatment to prevent relapse and likely to benefit from such a program.

It is important to note from the outset that our results must be interpreted cautiously, for two reasons. First, no pick-up order procedures for noncompliant subjects in the court-ordered group were implemented, which compromised the differences between the program conditions of the two study groups. Such procedures allow the New York City police to transport noncompliant subjects and bring them to the hospital. Second, persons with a history of violence were excluded from the program.

Methods

Sample size and randomization

From January 6, 1996, through February 28, 1998, a total of 567 patients were referred by Bellevue Hospital staff to the outpatient commitment program. The Bellevue outpatient commitment coordinating team found that 315 of these patients (56 percent) were eligible for outpatient commitment. Of the eligible patients, 112 were discharged or eloped from the hospital or the treating psychiatrist withdrew referral before the patient entered the outpatient commitment program.

When the coordinating team completed the treatment plan, they called the research team, who checked the random number list to see whether the client was to be assigned to the group receiving court-ordered treatment or the comparison group. Of the 203 individuals who were approached, 175 (86 percent) signed a consent form indicating their willingness to participate in the study; the 28 who refused remained eligible for outpatient commitment but not for random assignment in the study. Reasons for refusal to participate ranged from unwillingness to discuss personal history to a fear that participation would alter the treatment plan or delay discharge. Refusal rates were similar for men and women.

Of the 175 persons who consented to participate in the study, 23 eloped or were discharged before assignment to study groups. Of the remaining 152 potential subjects, 142 completed the baseline interview. The ten persons who did not complete the interview either refused to do so or were discharged before an interview could be conducted. Seventy-eight persons were in the experimental group—that is, went to court—and 64 were in the control group. In the experimental group, 39 were committed with a medication order, and 38 were committed with no medication order. Because no results differed by whether or not the person had a medication order, this issue is not addressed further. The court rejected the team's recommendation of outpatient commitment for one person. The slightly uneven distribution of subjects in the two groups resulted by chance from the randomization procedures.

Between 57 and 68 percent of the subjects were interviewed at each follow-up, at one, five, and 11 months after hospital discharge. Losses to follow-up occurred for several reasons. After participating in the baseline interview, some subjects refused to participate in the follow-up interviews—7 percent (N=10), 7 percent (N=10), and 14 percent (N=17) at the one, five, and 11 month follow-ups, respectively. Some were too impaired to participate in an interview—9 percent (N=12), 7 percent (N=10), and 11 percent (N=13), respectively. Others had moved out of the New York City area—2 percent (N=3), 9 percent (N=12), and 10 percent (N=12), respectively. Across all three follow-ups, more of the experimental subjects were interviewed, because fewer had left the area. A logistic regression analysis showed no bias across the two study groups as a result of differential attrition.

Comparability of groups

The two study groups were quite comparable. They did not differ significantly in gender, race, age, and median length of stay. Fifty-four patients (69 percent) in the experimental group were men, compared with 40 (62 percent) in the control group. The numbers by race were 25 (32 percent) and 27 (42 percent) Caucasian, respectively; 30 (39 percent) and 23 (36 percent) African American; and 16 (21 percent) and nine (14 percent) Latino. The mean±SD ages were 41± 11 years for the experimental group and 41±12 years for the control group. The median lengths of stay for the index hospitalization were 53 days and 51 days, respectively.

The two groups were similar in terms of whether they were receiving entitlements—63 subjects (81 percent) in the experimental group and 50 (78 percent) in the control group. The groups had similar employment rates, 17 subjects (22 percent) and ten subjects (16 percent), respectively. The only significant difference was that subjects in the control group were more likely than those in the experimental group to have been homeless at the time of their index hospitalization (33 percent versus 10 percent; χ²=11.0, df=1, Fisher's exact p=.001).

Because outpatient commitment patients had to be scheduled for a court appearance and the opportunity for the court appearance occurred on only one specific day each week, there was some concern that outpatient commitment might artificially extend the length of hospital stay. However, length of stay was about the same for both groups—a median of 53 days for the experimental group and 51 days for the control group.

Psychosocial and other characteristics of the two groups were also similar. The majority of participants in both groups had a psychotic disorder; 56 patients (72 percent) in the experimental group had a diagnosis of schizophrenia, schizoaffective disorder, or other psychotic disorder, compared with 50 (78 percent) in the control group. A significantly larger proportion of participants in the experimental group had a co-occurring substance use disorder—44 (56 percent) versus 25 subjects (39 percent) (χ²= 4.24, df=1, Fisher's exact p= .04). The Global Assessment of Functioning score for both groups at baseline was 43, placing these patients at a highly symptomatic level.

The MacArthur Perceived Coercion Scale (17), adapted for use in this study, indicated that the majority of subjects in both groups reported a high level of coercion—54 subjects (72 percent) in the experimental group and 38 (63 percent) in the control group.

Baseline and follow-up interviews

The initial interview occurred as soon as possible after consent was given. The interview assessed subjects' perceptions of illness severity and susceptibility, the benefits of and barriers to treatment, and cues to action related to compliance. Also assessed were current functioning and symptomatology, quality of life, perceived level of coercion, social supports, and recent life experiences. Subjects were asked to give their opinion about outpatient commitment.

Subjects were contacted in the community, in hospitals, or at other clinical facilities for follow-up interviews at one month, five months, and 11 months after the baseline interviews. These time periods were selected to coincide with key points in the outpatient commitment program: initial linkage to community supervision, the conclusion of the first 180-day order, and the conclusion of the second outpatient commitment order if one had been issued. The core measures at all three follow-ups were the same as those at baseline; some additional questions about community-based treatment and experiences were included.

Discharge plans and use of services

Treatment plans for patients in both groups typically included case management services, psychiatric aftercare, and residential placement. Case management services at discharge included supported case management for 69 patients (47 percent), intensive case management for 42 (30 percent), and assertive community treatment for 31 (22 percent). Half of those in the court-ordered group had a medication order (N=39). The outpatient commitment order was renewed at six months for 51 subjects (65 percent).

Residential placement at discharge varied considerably. Of the 142 patients in the sample, 54 (38 percent) were discharged to independent living, that is, alone or with family or friends. Forty-seven subjects (33 percent) were placed in structured housing programs, such as residences for persons with co-occurring mental health and substance use disorders. Most of these initial placements were intended to be temporary, with more appropriate housing as a long-term goal. Twenty-five persons (18 percent) went to less structured or semi-independent supported housing programs, and 16 (11 percent) went to single-room-occupancy hotels.

The psychiatric aftercare arranged for study participants ranged from hospital-based day programs to individualized treatment at an outpatient clinic. The treatment plans for most of the 142 participants (86, or 60 percent) specified highly structured types of treatment environments—day treatment that required consistent attendance or residential programs, such as those for persons with co-occurring substance use disorders—in which treatment is integrated in the daily routine of the residence.

Results

Outcome data for both the experimental and control groups were obtained from official records. Data on hospitalizations were from the New York City Health and Hospitals Corporation, and information about arrests was from the New York State Division of Criminal Justice Services. Data were also obtained from a review of the Bellevue Hospital charts and from patients' self-reports during the follow-up interviews.

Rehospitalization

No statistically significant differences were found between the experimental and control groups in psychiatric hospitalization in acute facilities or state hospitals. As Table 1 shows, the proportion of subjects rehospitalized in the 11-month follow-up period was not significantly different between the groups, nor was the number of days subjects were hospitalized. The median number of days hospitalized during the period was 43 for the experimental group and 101 for the control group. Although this difference is substantial, a standard nonparametric statistical test of significance (the Wilcoxon two-sample test) showed that the two distributions were not significantly different. If we had recruited twice as many subjects for the study, this difference in hospital days would have attained the .05 level of statistical significance. Thus, although on inspection the difference between the two groups looks like an important finding, it does not meet the conventional scientific standard to rule out the possibility that it occurred by chance.

It could be argued that the fundamental question from a services standpoint is demand for and use of services—or, more bluntly, costs. That is, from the perspective of a mental health system, we want to know whether it is less costly in terms of hospitalizations to mount an "assisted outpatient treatment" program—the addition being the court order—than simply to provide the services. To answer that question, a simple comparison of mean days rehospitalized ought to suffice. That comparison should model the phenomenon that a majority of cases were not hospitalized. In our statistical approach, we modeled zero days hospitalized binomially and the duration of hospitalization as a Weibull distribution. Using that method, we concluded that there was no statistically significant difference in the average rehospitalization times of the two groups (18).

The data presented in Table 2 point to a key finding of this research. Although no differences between the experimental and control groups were found on any major outcome measure, significant differences were noted in the court-ordered group in the proportions of subjects hospitalized in the 12 months before the index admission and in the first 11 months in the community under the commitment order. As noted, the law requires that persons eligible for outpatient commitment have at least two involuntary hospitalizations in the previous 18 months. Eighty-four percent of all study participants had at least one hospitalization in the previous year, and 35 percent had two or more in that year. During the 11 months of participation in the outpatient commitment program, 47 percent of all subjects were rehospitalized once, and 21 percent were rehospitalized two or more times. Both differences were found to be statistically significant. Both the experimental and control subjects had significantly fewer rehospitalizations during the 11-month period in the program. However, the number of multiple rehospitalizations was significantly reduced for the control subjects only.

Arrests

Arrests during the follow-up period are summarized in Table 3. No subjects were arrested for a violent offense, and relatively few were arrested overall—16 percent of the control group and 18 percent of the experimental group. No between-group differences were found on indicators for any arrest, multiple arrests, number of arrests, or most serious charge.

Quality of life and symptom reduction

Tables 4 and 5 present results on other outcome measures. As Table 4 shows, differences between the experimental and control groups were not significant on any quality-of-life or symptomatology outcome measures. Also, both groups changed very little on these outcome measures from baseline through the follow-up period. Table 5 summarizes data on other outcome measures. One important and desired outcome of the outpatient commitment program was successful engagement in services. Such engagement, it was hoped, would lead to more favorable outcomes in the community. For all participants, we measured self-reports of compliance, conducted interviews with community providers of treatment, and compiled a measure of treatment discontinuation. The proportion of persons in both groups who discontinued treatment was similar—27 percent for the experimental group and 26 percent for the control group.

Treatment discontinuation varied by the type of residential placement. Subjects in programs for persons with co-occurring substance use disorders were twice as likely as other subjects to discontinue treatment (χ²=6.31, df=1, Fisher's exact p= .018). Those engaged in treatment with an assertive community treatment team had the lowest dropout rate—9.7 percent. The dropout rate was significantly lower than the rates for other treatment groups (χ²=5.14, df=1, Fisher's exact p=.034). Residential placement and treatment type had no association with other outcome measures, such as rehospitalization and arrest.

Other factors, including diagnosis, existence of a medication order, case management type, and services, were examined to probe further for differences between the experimental and control groups on the outcome measures, but remarkably few differences were found. The primary difference was between those with a diagnosis of substance abuse or dependence and those without. A significantly higher proportion of subjects with a substance use disorder were rehospitalized: 40 with a substance use disorder (58 percent) were rehospitalized compared with 27 with no substance use disorder (37 percent) (χ²=6.27, df=1, Fisher's exact p=.018). The difference in rehospitalization rates was significant whether or not subjects had a co-occurring diagnosis of psychosis. A logistic regression entering the main effects for the diagnosis variable and group membership (experimental or control) confirmed a significant interaction between substance abuse or dependence and group membership in the explanation of rehospitalization rates.

In summary, the court-ordered group and the control group were remarkably similar on most outcome measures over the follow-up period. These findings may be due to the overrepresentation in the court-ordered group of persons with a diagnosis of substance abuse or dependence, which was significantly correlated with a higher rate of rehospitalization. However, that conclusion is not clear from these data.

Discussion and conclusions

A number of contextual factors constrain the conclusions that can be drawn from these data. First, the pilot outpatient commitment program at Bellevue Hospital was in its start-up and early phases, and many aspects of the program were in flux during data collection. It is possible that results would have been different for a mature program.

One major missing ingredient was the anticipated special enforcement mechanism, specifically under section 9.61, through which the New York City Police Department could have been called for pick-up orders for noncompliance. Because of the Police Department's misgivings, such a mechanism did not materialize until after the study was completed. During the pilot program, emergency mental health services for both study groups were available under existing civil Mental Hygiene codes. Specifically, under existing regulations, mobile crisis staff, assertive community treatment teams, and similar programs were able to call 911 to summon police when emergency transport was necessary. Even though the enforcement mechanism was not in place, service providers sometimes warned court-ordered participants that their noncompliance would result in return to Bellevue Hospital; the implication was that it was especially easy to rehospitalize patients because of the special court order under section 9.61.

Another limitation of our study was the modest size of our sample—142 subjects. Although the sample was large enough to allow valid comparison of the two groups on all major outcome variables, it was too small to address the subtler questions requiring subgroup analyses.

On all major outcome measures, no statistically significant differences were found between the court-ordered group and the control group. Neither group had a single arrest for a violent crime. A modest proportion of both groups—18 percent and 16 percent, respectively—were arrested at least once. The percentage of both groups rehospitalized over the 11-month follow-up period was about the same, 51 percent and 42 percent, respectively. The total number of days hospitalized during the follow-up period did not differ significantly between the two groups. From the subjects' perspective at all three follow-ups, their quality of life and the level of coercion they perceived were about the same. No evidence was found that the outpatient commitment order increased perceived coercion. From the community service providers' perspective, the two groups were similarly compliant with their treatment plans.

Notably, in both the experimental and the control groups, a significantly smaller proportion of subjects were hospitalized in the 11 months after program enrollment than in the year before the index admission. This finding suggests that the package of enhanced services and vigilant coordination after discharge yielded real benefits. Clinicians and administrators in the outpatient commitment program strongly believe that the court order was crucial for ensuring compliance and reducing negative consequences for a small group of subjects, perhaps 5 to 10 percent. The limited sample size in our study did not permit us to test this belief or to describe the characteristics of this putative group.

Still, it is clear that under the auspices of the pilot outpatient commitment program, the Bellevue coordinating team played a powerful role in marshaling resources and cooperation among providers of mental health services, substance abuse treatment, housing, and social services for the court-ordered participants. Community providers who had been quite reluctant at the beginning of the program proved to be faithful collaborators throughout. In effect, the pilot program "committed the system" as much or more than it did any individual program participants.

Finally, it is important to recognize that all the results included in this report must be understood to apply to a select group of persons. The sample consisted of persons with two involuntary hospitalizations in the 18 months before the index admission to Bellevue Hospital and a history of noncompliance with treatment associated with decompensation in the community who agreed to participate in outpatient commitment as a condition of discharge from the hospital. From the outset of the pilot program, it was explicitly recognized by program administrators that people with a history of violence who were considered at high risk of future violence would not be included. Thus the results of this study may not be generalizable to other outpatient commitment programs that include such people.

Acknowledgments

The authors thank Howard Telson, M.D. They also thank Steven Banks, Ph.D., Roumen Vesselinov, M.A., Jeffrey Swanson, Ph.D., and Joseph Wanderling, M.A., for their contributions to the data analysis.

Dr. Steadman, Dr. Gounis, Dr. Dennis, Dr. Roche, and Dr. Robbins are with Policy Research Associates, Inc., in Delmar, New York. Dr. Hopper is with the Nathan Kline Research Institute in Orangeburg, New York. Dr. Swartz is affiliated with Duke University Medical Center in Durham, North Carolina. Send correspondence to Dr. Steadman at Policy Research Associates, 345 Delaware Avenue, Delmar, New York 12054 (e-mail, [email protected]). This paper is part of a special section on outpatient commitment.

Enlarge table

Enlarge table

Enlarge table

Enlarge table

Enlarge table