New York State's assisted outpatient treatment (AOT) program (1) is a form of involuntary outpatient civil commitment (2) that places certain individuals with serious mental illness under a court order to receive enhanced community-based mental health services. The goal of the policy is to ensure that persons with serious mental illness who meet legal and clinical criteria receive the treatment they need to remain in the community, forestalling a costly and deleterious pattern of revolving-door hospital admissions (3).
AOT is controversial, due mainly to concerns about coercion (4). This study addressed a possible unintended negative consequence of the privileged access created by AOT: How does the policy affect persons with serious mental illness who do not receive AOT but share a service system with those who have AOT orders?
To the extent that service capacity is constrained in a public system of care, a "zero-sum game" may mean that an increase in services for a court-mandated population comes at the expense of individuals who are voluntarily seeking care. Might this situation amount to a de facto denial of needed services to persons with serious mental illness who do not qualify for AOT? Did the mental health system serve some people through the AOT program at the cost of failing to serve others? Did the AOT program's impact on the system differ in the short and long run?
We addressed these questions empirically using longitudinal data on enhanced service utilization by voluntary and involuntary service recipients with serious mental illness in New York during the eight years after AOT was enacted in 1999.
A key premise of New York's AOT policy from its inception was that the program would require investment of new resources. There was no expectation that the mental health system with its existing service capacity would be able to absorb a potentially large new pool of outpatients with intensive service needs. Thus the state legislature appropriated substantial new funding to increase the availability of enhanced services to coincide with AOT's implementation (5). Whether these new resources were distributed in a way to protect access for individuals who did not have AOT orders is unknown.
The most common forms of case management services required and delivered under the AOT program are intensive case management and assertive community treatment (ACT). These enhanced service models were originally developed to address the complex care needs of individuals with serious mental illness who were entering the community after deinstitutionalization (6). Both models share a number of common characteristics: low staff-to-consumer ratio, intensive outreach, 24-hour coverage, low consumer input, some skills training, frequent consumer contact based in the community, high treatment integration, and intensive direct service provision (7). The ACT model uses an interdisciplinary, community-based team to provide a full range of medical, psychosocial, and rehabilitative services (8). ACT teams use a variety of approaches and leverage to encourage adherence to prescribed treatment (9). In the intensive case management model, a single intensive case manager has primary accountability and coordinates other services as necessary.
Although there is limited research evidence that intensive case management and ACT directly improve psychosocial functioning (7), studies have found that these services effectively increase medication adherence, decrease the likelihood of psychiatric hospitalization and arrest, and can improve quality of life in the long run (10).
ACT and intensive case management have formed the core of most court-ordered treatment plans under AOT in New York State, but they are also often the cornerstones of voluntary treatment for people with serious mental illness who are not under AOT. Availability of ACT and intensive case management services is limited, and priority for intensive services is given to AOT clients (5). Thus it is important to consider whether individuals who voluntarily seek treatment may experience diminished access to these community-based services as an unintended consequence of the AOT program. It is also important to examine the AOT program's overall impact on the mental health system's resources over time.
We addressed these issues by analyzing trends in the volume of paid Medicaid claims for ACT and intensive case management services over an eight-year period after initiation of AOT (1999—2007). The claims examined were for services that were delivered to court-ordered individuals with serious mental illness in the AOT program and to their counterparts without a court order.
During the AOT implementation period, the number of ACT teams in New York was increasing, effectively replacing intensive case management for many recipients after 2001. To assess the AOT program's impact on service recipients who did not have an AOT order, we combined ACT and intensive case management into one category of intensive case coordination services and examined trends in receipt of these services over time. We addressed several questions: To what extent did the implementation of AOT coincide with an increase in the volume of enhanced services being delivered to persons with serious mental illness in New York State and in the number of persons receiving these services overall? Did individuals with serious mental illness who were not AOT clients experience similar trends in utilization of enhanced services over time as their counterparts with AOT orders? Did AOT offset, or divert, enhanced services from individuals with serious mental illness who were not AOT clients? What were the short-term and longer-term consequences of AOT for persons who were not AOT clients?
This project was approved by the institutional review boards of Duke University, Policy Research Associates, the New York State Office of Mental Health, and Biomedical Research Alliance of New York.
We used administrative data from the New York State Office of Mental Health and Medicaid claims data to identify a group of individuals with serious mental illness who were voluntary recipients of enhanced services and who were otherwise clinically comparable to the AOT population. Several selection criteria for this non-AOT sample were used: current service user with a record of receiving outpatient mental health services on or after July 1, 2006; diagnosis of schizophrenia, bipolar disorder, or major depression as a billing diagnosis for a hospital admission since 1999; two or more psychiatric inpatient admissions in any year since 1999; total of 14 or more inpatient days in any single year; and use of intensive outpatient services (that is, ACT or intensive case management) at any time since 1999. Application of these criteria produced a sample of 3,295 non-AOT service recipients with an aggregate clinical profile comparable to that of the AOT population. Medicaid service claims records for eligible participants were then extracted for dates of service between January 1, 1999, and March 14, 2007. This resulted in approximately four million claims records.
Dependent variables. Two binary dependent variables were constructed for multivariable analyses of person-month observations: initiating ACT or intensive case management and discontinuing or no longer receiving ACT or intensive case management. Individuals with a first-time Medicaid claim for ACT or intensive case management services in a given month were coded as initiating ACT or intensive case management. These individuals had no previous claim for enhanced services during the period. Individuals who had been receiving ACT or intensive case management services at any previous time but were no longer receiving them in a given month were coded as having discontinued enhanced services; this group included persons not receiving services for any reason, if they had received them in the past.
Independent variables. Time was measured with seven dummy variables representing each year of data available, with the first year (1999) serving as the reference group. Categorical coding allowed a nonlinear representation of the main effect of time on enhanced service initiation and discontinuation.
The number of people with existing AOT orders during each month was collapsed initially into three categories: 1—200, 201—400, and 401 or more. For a period-specific analysis comparing the start-up phase of the AOT program to the later, steady-state period, the number of AOT orders was coded into additional intervals for the later period: 500—700, 701—900, and 901 or greater. Again, categorical coding allowed for a nonlinear specification of the effect of increasing numbers of AOT clients coming into the mental health service system. Exploratory analysis of the data suggested that this approach provided a better fit with the form of the association between AOT case volume and services to non-AOT individuals; minor fluctuations in AOT caseload from month to month did not tend to have an appreciable impact on services, but above certain points of "critical mass," an effect was observed. In the overall and start-up period analyses, the category of 1—200 AOT cases in a month served as the reference group for the other categories. For the analysis of the later period, the category of 500—700 AOT cases per month (the minimum after four years) was used as the reference.
Region was indicated by five dummy codes: Central, Hudson River, Long Island, and Western regions were each compared with the New York City region.
Race-ethnicity, sex, and age were all represented by a series of dummy variables in the analysis. Hispanic ethnicity, African American or black, Asian or Pacific Islander, and a composite "other" category were compared with non-Hispanic whites. Men were compared with women, and older individuals (at or above age 43, the sample median) were compared with younger individuals.
Diagnosis was indicated by the most frequent primary diagnosis reported in the Medicaid claims for a given individual. Specifically, we extracted the primary diagnosis from each claim and then grouped diagnoses into four categories: schizophrenia spectrum disorders, bipolar disorder, major depressive disorder, and other. The fourth category thus represented persons who had a historical diagnosis qualifying them for the study—schizophrenia, bipolar disorder, or major depression—but who also had another diagnosis (for example, anxiety disorder) recorded with greater frequency in Medicaid claims. We used a hierarchical coding scheme to count the number of claims with each diagnosis and assigned the most frequent diagnosis type over the study period as the primary diagnosis.
Medicare eligibility status indicated whether individuals were Medicare eligible in any given month. Medicare eligibility for persons under 65 is an indication of long-term disability, and thus this variable may capture otherwise unmeasured severity of disorder (11) as well as coinsurance.
The analytic data file contained multiple observations per person. Specifically, a vertical data shell was constructed in which each individual had 88 rows of data, one record for each month of the study period from 1999 to 2007. Within this data structure, each month was represented by a separate record, and each type of service event or status, such as receiving enhanced services or being on AOT or not, was indicated by a separate variable.
The analysis was designed to look for two kinds of patterns in the data, either of which might suggest that service capacity was being diverted, in effect, from voluntary service seekers to AOT recipients. The first pattern would be that voluntary service seekers were less likely to initiate enhanced services during months of high AOT service delivery. The second pattern would be that voluntary service recipients who had been receiving enhanced services were more likely to discontinue them—or to no longer be receiving them—during periods of high AOT service delivery. Both of these findings would suggest that nonrecipients of AOT were less able to initially obtain and to consistently maintain access to enhanced services because of a "queue-jumping" effect of AOT. It would be consistent with services being assigned to AOT recipients first, making services less available to those without court orders, although it would not necessarily prove this effect.
Accordingly, multivariable models were developed to estimate the likelihood of initiating intensive services and of discontinuing or failing to receive previously utilized intensive services in any given month between 1999 and 2007 as a function of the number of AOT orders in the system that month; these analyses controlled for time and relevant covariates found in the Medicaid data. A second set of models was estimated to examine these effects separately for the start-up period and the later, steady-state period of the AOT program.
It should be noted that the AOT petition process and system response begin sometime before the actual initiation of an AOT order, partly because a detailed treatment plan (typically including ACT or intensive case management placement) must be prepared before the scheduled court hearing. Also, psychiatrists must examine the respondent and prepare testimony for the hearing, to the effect that the individual meets criteria for AOT. Thus key actors in the service system necessarily become aware of, and may respond to, an anticipated increase in the number of AOT cases before these cases officially become active after a court hearing. Thus there is a short built-in lag between changes in the AOT caseload and any observed system changes during the month an AOT order is formally initiated, giving some confidence in the proper temporal ordering of the tested relationship.
Generalized estimating equations for logistic regression were used to obtain unbiased parameter estimates with multiple observations for each individual. The unit of analysis was the person-month. All analyses were conducted with SAS, version 9.1.
Table 1 displays aggregate characteristics of the AOT recipients and the non-AOT service recipients. The non-AOT sample resembled the AOT population in its basic demographic and clinical characteristics. (To be clear, the characteristics of the non-AOT sample are of primary interest for this study; AOT recipients were excluded from the main analysis except insofar as the aggregate AOT caseload was incorporated as an independent variable.)
Over two-thirds of the non-AOT sample were located in New York City (69%); Long Island was the next most prevalent non-AOT treatment region with 16% of the cases. The remaining three regions had substantially fewer non-AOT service recipients. The average age of the non-AOT sample was 43.5 years, and over two-thirds (68%) were men. Forty-three percent of the sample were African American, 38% were non-Hispanic white, and 12% were Hispanic. About four-fifths of the sample (83%) had a diagnosis of schizophrenia. The next most prevalent diagnoses were bipolar disorder (9%), major depression (6%), and other (2%). Finally, 18% of the sample were also Medicare enrollees.
Trends in AOT and non-AOT service claims
Figure 1—in essence, a longitudinal stacked bar graph—displays a proportional area plot of the allocation of services over time, showing the distribution of paid Medicaid claims for ACT or intensive case management allocated between four groups in any given month: those who never received AOT orders, those who would receive AOT orders in the future but did not yet have them in a given month, those currently under an AOT order, and those with completed AOT orders and still receiving services. The figure suggests a steep upward trend in the total number of monthly paid Medicaid claims for ACT or intensive case management, with overall service volume increasing about 400% between 1999 and 2007. While more individuals were being served, many more services were also being provided to each individual on average. However, important differences emerged in the comparison of trends for AOT recipients (before, during, and after AOT) and those who never received an AOT order.
The volume of ACT or intensive case management services doubled in the first three years after AOT began, but virtually all of that increase appears to have gone to AOT recipients. As Figure 1 shows, there was no increase in ACT or intensive case management services for those who did not have an AOT order as a group during the first three years of AOT implementation. However, between 2002 and 2007, the trend shifted as non-AOT service recipients saw an increase in ACT or intensive case management services that paralleled that for AOT recipients.
After the first three years of AOT implementation, ACT or intensive case management services increased both for post-AOT recipients and for those who never received an AOT order. As a result, by 2007 the volume of ACT or intensive case management monthly claims was almost evenly distributed between service recipients who had completed their AOT orders and those who had never received an AOT order, with a small and diminishing share of the services going to current AOT participants.
Further analysis was conducted to examine the proportion of monthly Medicaid claims for enhanced services delivered to individuals who were not on AOT orders. Between 1999 and 2003, the non-AOT share of monthly Medicaid claims for ACT or intensive case management services was cut in half—from 100% to less than 50%. Some of this shift reflects movement of individuals from non-AOT to AOT status. After 2003 the share of all claims for ACT and intensive case management, which had declined for the non-AOT group, stabilized and gradually reversed. By 2007 three-quarters of ACT or intensive case management services were going to non-AOT service recipients—the same proportion as in 2001. The increase in ACT or intensive case management services to nonrecipients of AOT coincided with the decrease in new AOT orders (5).
Table 2 presents the adjusted odds ratios (AORs) for the likelihood that persons in the non-AOT sample initiated and discontinued (or were no longer receiving) services in a given month. In statewide analysis, we found that an increase in the number of AOT orders in the system was significantly associated with a decrease in the likelihood that individuals who were not on an AOT order would initiate receipt of ACT or intensive case management. Specifically, during months when the number of AOT orders exceeded 200, the odds were approximately cut in half that a non-AOT service seeker would initiate ACT or intensive case management services.
In terms of discontinuation and nonreceipt of services (Table 2), we found that an increase of AOT orders in the system was significantly associated with increased odds of non-AOT service recipients forgoing ACT or intensive case management services. Specifically, during months when the number of AOT orders rose to between 201 and 400, the odds of non-AOT service recipients' no longer receiving ACT or intensive case management was 1.45 compared with months in which fewer AOT orders (1—200 orders) were in the system. When the number of AOT orders in the system exceeded 400 in a given month, the odds of discontinuing or no longer receiving ACT or intensive case management for non-AOT service recipients was substantially higher—2.22—compared with months in which there were fewer orders (1—200).
Given that New York City may be qualitatively distinct from other areas of the state, we repeated these analyses, comparing New York City with all other regions of the state. We found a similar pattern in New York City and in other regions with two exceptions. First, the main effect of time on discontinuation of ACT or intensive case management services for nonrecipients of AOT recipients was delayed in the New York City region, emerging in the sixth year after AOT started, whereas outside New York City the effect was seen in the third year. Second, an increase in AOT orders from 1—200 to 201—400 had a somewhat sharper impact on discontinuation or nonreceipt of services outside the New York City region, as indicated by a higher odds ratio (AOR=1.67, 95% confidence interval [CI]=1.12—2.50 compared with AOR=1.36, CI=1.07—1.72).
Was the apparent AOT crowd-out effect confined to the start-up period, or did it continue even after the program reached a steady state? To address this important question, we tested the AOT program's net effect on non-AOT clients' initiation and discontinuation of ACT or intensive case management separately for the early and later periods, months 1—48 and months 49—88, respectively (not shown).
In the model of the start-up period, we found that ACT or intensive case management initiation among non-AOT service seekers decreased significantly as the AOT caseload increased: from 1—200 to 201—400 AOT cases per month, AOR=.47 (CI=.40—.55) and from 201—400 to 401 or more cases per month, AOR=.46 (CI=.37—.57). We also found in the start-up model that discontinuation or nonreceipt of ACT or intensive case management increased significantly among non-AOT service recipients as AOT cases increased: from 1—200 to 201—400 AOT cases, AOR=1.53 (CI=1.25—1.87) and from 201—400 to 401 or more AOT cases, AOR=2.42 (CI=1.94—3.03). These models controlled for time, region, race, sex, age, and diagnosis (data not shown).
In contrast, in the later-period model (not shown), we found that initiation of ACT or intensive case management among non-AOT service seekers did not continue to decrease significantly as AOT cases increased from 500—700 to 701—900 AOT cases per month (AOR=.94, CI=.85—1.03) and decreased only slightly when AOT cases exceeded 900 (AOR=.87, CI=.78—.98). We also found in the later-period model that discontinuation of ACT or intensive case management among non-AOT service recipients did not increase but actually decreased slightly as AOT cases increased from 500—700 to 701—900 AOT cases per month (AOR=.95, CI=.90—1.01) and to more than 900 AOT cases per month (AOR=.94, CI=.88—1.01). These models also controlled for time, region, race, sex, age, and diagnosis.
In a review of cost-effectiveness studies of services for homeless persons with mental illness, Rosenheck (12) observed that the apparent success of augmented service programs must be tempered by considering their queue jumping function—whether giving priority access to one targeted population may simply displace others who also need services: "Since each of these programs had negotiated special access to housing resources … they did not actually increase the available pool of resources. Rather, they helped their clients jump to the head of the line, displacing others who were probably just as deserving" (12).
Rosenheck went on to suggest that dissemination of programs whose success is shown to be the result of queue jumping should be accompanied by increasing resources to make the services available to those in need. The broader implication of this point is that service programs should be evaluated not only for their impact on a primary target population but also for their indirect and unintended effects on other individuals—those who are not selected for the program but who may be considered competitors for the resources that the program encumbers (13,14).
Did AOT recipients benefit from queue-jumping to the detriment of others for whom services were delayed or who were displaced from services? Arguably, the appropriation of new funding for services that accompanied AOT's implementation (5) should have prevented queue-jumping. The policy was enacted in tandem with a large increase in the provision of enhanced community-based services for persons with serious mental illness, not limited to those under court-ordered treatment. However, in the process of implementing the AOT program, it appears that priority in allocation of ACT and intensive case management was initially given to persons with a new AOT order. This article presents empirical evidence that suggests that the increasing number of AOT cases in the system significantly affected the pattern of ACT and intensive case management service delivery to service seekers who did not have an AOT order. The findings are consistent with, although do not prove, the hypothesis that AOT delayed others' initiation of ACT or intensive case management and accelerated their discontinuation of these services.
The apparent impact of AOT in diverting services from non-AOT clients was concentrated mostly in the first three years of AOT implementation, between 2000 and 2003. During those years, there was essentially no expansion in ACT or intensive case management services to non-AOT clients. However, after 2003, as the number of new AOT orders stabilized and then declined, a steady expansion in enhanced services to non-AOT service seekers is seen. This pattern is consistent with the notion that new service capacity accompanied AOT and was eventually made available to other individuals who needed these services, irrespective of an AOT court order. Thus, after the initial ramp-up of the AOT programs throughout the state, intensive community-based services increased for all individuals with serious mental illness—those on an AOT order and those without an order.
One limitation of the study is that the inclusion of current service recipients in 2007 may have excluded early dropouts or those who discontinued services for any reason and were no longer in the available Medicaid data by 2007. This sampling strategy was necessary to meet the conditions of access to Medicaid data from the New York State Department of Health via the Office of Mental Health. However, the design may have inclined the sample generally in the direction of showing increasing services over time. Because the number of AOT orders also generally increased over time, at least for the first several years of the program, there was some reason to expect an artifactual correlation between increasing services for the non-AOT sample and increasing number of AOT orders. Notwithstanding this expectation, the study found no increase in enhanced services to the non-AOT sample in the aggregate during the first three years of AOT, at a time when services to AOT recipients were markedly increasing. Moreover, stratified regression analyses revealed that during the program's ramp-up period, increased AOT caseload was associated with lower adjusted odds of initiating enhanced services for non-AOT service seekers in any given month.
Because of the possibility of sample bias, the study cannot rule out other interpretations of the service utilization patterns found among non-AOT service recipients. However, several factors may mitigate this concern. First, ACT and intensive case management tend to be long-term modes of service for people with serious mental illness; these services have relatively low rates of permanent discontinuation within a time frame the length of this study's period of observation. Many temporary dropouts from ACT or intensive case management services are eventually rehospitalized; when this happens in New York, patients are routinely referred back to Office of Mental Health outpatient services through discharge planning. This would have returned some dropouts to sample eligibility. Even if the Medicaid sample underrepresented people who discontinued services during the early period of the time series, there is little reason to think this would have systematically affected the observed association between AOT caseload and the likelihood of starting or discontinuing ACT or intensive case management for people not on AOT orders. More specifically, if the increasing volume of AOT cases initially crowded out some non-AOT service seekers who were otherwise "queued up" to start ACT or intensive case management, it logically would have affected those in the queue who would eventually drop out, as well as those who would be retained.
The primary limitation of the study is lack of information about individuals' specific level of need for enhanced services at any given time. Such information could have illuminated reasons for initiating or discontinuing services, as well as differences in acute clinical need between AOT recipients and nonrecipients of AOT in the analysis. Whereas some individuals may discontinue intensive services because they are successfully recovering and no longer need the services, others may discontinue because they refuse treatment or experience other barriers to care, notwithstanding their continuing need; these two scenarios would obviously have different implications. We know that during the initial years of AOT, providers were encouraged to give service priority to persons with an AOT order (5). In so doing, perhaps providers reviewed existing non-AOT service recipients to identify those who might be dischargeable; they might also have focused less attention on retaining some non-AOT service recipients or may even have denied or delayed service initiation for some voluntary service seekers. In any event, because of this limitation in the data, we can say only that our evidence suggests that the AOT program initially affected the pattern of service initiation and discontinuation for those who did not have an AOT order. Whether the altered pattern of service receipt was definitely attributable to AOT, and AOT alone, or actually detrimental to these individuals is unknown.
Key features of the system of care in which AOT was implemented—the capacity to provide enhanced community-based mental health services, resource allocation, and patterns of service utilization—may have been altered by AOT, at least initially, in ways that indirectly affected other persons with serious mental illness who did not receive an AOT order. In the short run, services to individuals who were not under AOT may have been delayed or some individuals may have been displaced from services by the influx of new court-ordered persons in the system of care. However, the findings of this study suggest that in the long run AOT is a program that came bearing new services and has left in its wake a system of care with greater capacity to serve all persons with serious mental illness, voluntary patients no less than involuntary ones.
The study is funded by the New York State Office of Mental Health, with additional support from the John D. and Catherine T. MacArthur Foundation Research Network on Mandated Community Treatment. The authors gratefully acknowledge the contribution of numerous individuals in collecting, synthesizing, and reporting the data for this effort. At Policy Research Associates, they thank Pamela Clark Robbins, B.A., Henry J. Steadman, Ph.D., Karli J. Keator, B.A., Wendy Vogel, M.P.A., Roumen Vesselinov, Ph.D., Jody Zabel, Steven Hornsby, L.M.S.W., and Amy Thompson, M.S.W. They also acknowledge the extensive reviews and critical feedback provided by the MacArthur Research Network on Mandated Community Treatment, which served as an internal advisory group to the study. Although the findings of the study are solely the responsibility of the authors, they gratefully acknowledge the support and assistance of the New York State Office of Mental Health in completing the report, including Steve Huz, Ph.D., Chip Felton, M.S.W., Peter Lannon, Susan Shilling, J.D., L.C.S.W., Qingxian Chen, Michael F. Hogan, Ph.D., Bruce E. Feig, and Lloyd I. Sederer, M.D.
The authors report no competing interests.