An Examination of Eligibility Decisions in New York State’s Assisted Outpatient Treatment

Kendra’s Law was enacted in New York State in 1999 after the murder of Kendra Webdale by Andrew Goldstein, an individual diagnosed as having schizophrenia who had reportedly sought treatment but had been discharged due to medication noncompliance (1). One of the main provisions of Kendra’s Law was the implementation of assisted outpatient treatment (AOT), described as a tool “for certain people with mental illness who, in view of their treatment history and present circumstances, are unlikely to survive safely in the community without supervision” (2). AOT involves a court mandate for individuals to receive outpatient treatment services, and noncompliance may result in sanctions. According to the New York State Office of Mental Health (3), to successfully petition for an AOT order, the potential client must be at least 18 years old, have a mental illness, be unlikely by clinical determination to survive safely in the community without supervision, and have a history of lack of compliance with treatment for mental illness that has been a significant factor leading to hospitalization or service receipt in a forensic mental health unit at least twice within the past 36 months, not including the current hospitalization or incarceration or any others within the past six months, or that has resulted in one or more acts of serious violent behavior or threats or attempts toward self or others within the past 48 months. The client must also be determined to be unlikely to voluntarily participate in outpatient treatment that would enable him or her to live safely in the community, to be in need of AOT to prevent a relapse or deterioration likely to result in serious harm to self or others, and to be likely to benefit from AOT.

Once an AOT court order has been petitioned, it is evaluated at a court hearing and either dismissed or approved. If the order is approved, the subject of the petition is then referred to outpatient treatment; in New York State, the resulting AOT often consists of either intensive case management or assertive community treatment (ACT) services (4).

Since the implementation of AOT, the literature supporting its effectiveness has grown. Comparisons of individuals assigned and not assigned to AOT have found mandated individuals to have lower rates of perpetration of serious violence and suicide risk (5), lower rates of arrest (6), and reduced likelihood of psychiatric rehospitalization (7). However, studies that have employed random assignment have yielded mixed results for outpatient commitment (OC), a related form of mandated treatment; one study found no significant differences between groups in measurable outcomes (8), whereas another found that individuals assigned to OC had fewer hospital admissions, lower violence and victimization rates, and higher adherence to community treatment, but only when the duration of commitment was considered in analyses (9). In addition, AOT is not without controversy, as some have raised the concern that it comes close to forced medication compliance and “blurs the relationship between civil and criminal mental disability law” (10).

Although outcome studies of AOT have found positive results for mandated individuals, considering AOT’s potentially coercive nature, a key question concerns whether the process of assigning some individuals and not others to mandated status is equitable and consistent with the initial intent of Kendra’s Law. In order to examine whether AOT in practice is targeting its intended population, this study reviewed charts of AOT and non-AOT individuals served by three ACT teams in New York City. In addition, we incorporated a structured violence risk assessment measure into our review. We hypothesized that individuals assigned to AOT versus those not assigned to AOT would more closely meet eligibility requirements and would demonstrate higher scores on an empirical measure of violence risk.

Methods

Charts were reviewed from three ACT teams within one treatment agency in New York City. Inclusion criteria for eligibility included being at least 18 years old, a diagnosed mental disorder as determined with DSM-IV-TR criteria, and available referral information.

Institutional review board approval was obtained before data collection. We conducted retrospective chart reviews from October 2013 through June 2014. According to referral date (within six to eight months), we matched individuals assigned to ACT through AOT with individuals assigned without AOT at each of the three agency sites. Charts were coded for demographic information and AOT eligibility criteria and evaluated for risk of future violence at the time of referral by using forms completed by the referring agency. The principal investigator and a trained graduate-level research assistant completed chart reviews. To ensure interrater reliability, assessments for the same charts were completed and compared by both raters at random throughout data collection.

Demographic information was first collected from each chart. Information regarding AOT eligibility was then coded by using criteria that are empirically defined: age, number of hospitalizations in the past 36 months, number of violent acts in the past 48 months, and treatment compliance. An additional variable, “history of suicide attempts,” was included to provide more information about danger toward the self, given that most charts did not include record of suicide risk evaluations.

Structured assessments of charts were also conducted with the Historical Clinical Risk Management−20, version 3 (HCR-20V3) (11). The HCR-20V3 is a structured professional judgment instrument used by clinicians to assess violence risk. It includes 20 empirically supported risk factors divided into three sections: historical (H) consists of ten risk factors related to past problems that are considered more stable, clinical (C) consists of five risk factors related to more current functioning that are more dynamic, and risk management (R) consists of five dynamic risk factors related to future plans. These 20 items are all scored on a 3-point scale: 0, not present; 1, possibly or partially present; and 2, present. The HCR-20V3 also includes three global judgments to be made on the basis of the pattern of risk factors identified: risk of future violence or case prioritization, risk that such future violence would include serious physical harm, and risk that such violence would be imminent. These judgments are also scored on a 3-point scale, from low risk, 0, to high risk, 2. Prior research has deemed file review to be acceptable for research purposes (12).

Results

We first examined demographic characteristics of the sample (Table 1). There were no significant differences between the two groups in diagnosis, gender, or race-ethnicity. AOT participants were significantly younger than non-AOT participants and slightly more educated, although this difference was not significant. Although we did not code charts for comorbid substance use disorder, history of problems with substance use was accounted for in the HCR-20V3 and is therefore likely to be more thorough because it concerns any reported substance use. A significant difference was found in problematic substance use history between groups, with a greater number of AOT charts indicating history of substance use problems. Overall, sample characteristics were very similar to the characteristics of persons with AOT status in New York City in 2013 (2).

We next evaluated the difference between AOT and non-AOT charts according to measurable AOT criteria (Table 1). No significant differences were found between groups in any of the AOT eligibility criteria, although there was a trend for AOT participants to have more incidents of violence in the past four years (however, 67% of the AOT sample had no recorded violent incidents). We then evaluated AOT and non-AOT differences according to HCR-20V3 scores (Table 1) on the global judgment scale of future violence/case prioritization. There was a significant difference between groups in risk of future violence, with AOT charts scoring at a higher risk than non-AOT charts. However, most charts (three-fourths of non-AOT charts and almost half of AOT charts) were coded as low risk.

We also evaluated violence risk using the HCR-20V3 as an actuarial measure, as used in previous research (12,13). We took the sum of all H, C, and R items to yield an HCR-20V3 total score, which we then evaluated for AOT and non-AOT charts. As shown in Table 1, a significant difference was found between summed scores for HCR-20V3 items between groups, with AOT charts receiving higher HCR-20V3 total scores than non-AOT charts.

History of suicide attempts was dichotomized according to homogeneity in the sample (0, no recorded history; 1, history of one or more suicide attempts). A significant difference was found between groups in history of suicide attempts (χ²=4.56, df=1, N=129, p<.05), with more non-AOT charts than AOT charts indicating a history of suicide attempts (non-AOT, 14 charts, 11% of sample; AOT, six charts, 5% of sample).

Discussion and Conclusions

As hypothesized, individuals referred to the agency through AOT were more likely to receive high ratings for future violence risk at the time of referral compared with those referred without AOT. However, the finding that there were no differences between groups in number of prior hospitalizations, number of prior violent incidents, or treatment compliance did not support the hypothesis that AOT-referred individuals would be more likely to meet AOT eligibility criteria. Unexpectedly, we also found that more non-AOT charts than AOT charts indicated a history of at least one prior suicide attempt.

On the basis of our findings, the stipulated criterion of potential danger to others living in the community was most closely related to the AOT referral process. However, although individuals referred through AOT were rated as being at higher risk of violence compared with those without AOT, 67% of the AOT sample had committed no violent acts within the four years before referral, and half of the AOT sample were rated as low risk of future violence, according to the HCR-20V3. In addition, only a small number of AOT charts indicated evidence of suicide risk. These findings raise the question of whether other, unknown factors were involved in the decision to make the referral if risk to self or others was not evident in the referral process.

If danger to the self or others is not the only consideration in AOT assignment, other potential referral factors should be explored. Although we cannot determine from the findings what other reasons might exist for referral, future research should consider other factors that might be driving the decision to seek a referral to AOT. For example, some individuals might be referred to AOT because they will be guaranteed services regardless of ability to pay, providing a means to gain access to treatment they otherwise might not receive. Other factors, such as the impact of “grave disability” or the role of family advocacy for AOT, also should be considered and explored.

This study included several important limitations. In that this was a retrospective chart review study, information was limited to what was available in the chart. When evaluating AOT eligibility, we examined only criteria that were able to be empirically coded and thus could not evaluate those that were based on the clinical expertise of clinicians involved with AOT assignment. Although this is considered a limitation of our study, it can also generalize as a limitation of AOT implementation in that clinical determinations are not standardized and easily measured. In addition, charts were reviewed from one treatment agency located in the Harlem area of New York City. Although AOT referrals and eligibility decisions were made through New York’s centralized referral system before reaching the treating agency, the eligibility decision-making process may not necessarily generalize to other agencies located in other areas of the city. Finally, charts were coded from samples including both AOT and non-AOT clients, and therefore it was not possible for individual raters to be blind to AOT status. Although this is a limitation, all coded items were based on documented information provided in charts which would be highly unlikely to be influenced by rater bias.

Study findings suggest that, despite AOT’s being celebrated in the media as succeeding in reducing incidents of violence from “dangerous” individuals with mental illness (14), determinations for AOT referrals in New York City use more narrow criteria than may have been originally intended in Kendra’s Law. Future research should evaluate the referral process by examining decision making of referring entities to further explore reasons why some individuals, but not others, are referred to this form of mandated treatment.