The Effectiveness of Recovery-Oriented ACT in Reducing Hospital Use: Do Effects Vary Over Time?

From its inception in the early 1970s, assertive community treatment (ACT) has been viewed as a time-unlimited treatment program for persons with severe mental illness (1,2). This “ACT-for-life” rationale was based on several considerations, including level of disability, the episodic nature of the illness, and the intensity of the services required to help people to live in the community outside institutional settings. Empirical support came from the original clinical trial of ACT (3), in which individuals discharged after 18 months of services because of the end of grant funding returned to their previous levels of hospital use in the following 14 months (4).

As the model was disseminated across the country, however, state mental health authorities and program managers began to question the unlimited-duration assumption because of the per-capita costs of ACT and the capacity constraints associated with its deployment. Under an “ACT-for-life” mandate, once an ACT team recruited its full caseload of 60–100 consumers, the only way it was able to accommodate any new referrals was through death of enrolled consumers or their move to another area. This was in marked contrast to most other mental health services that dealt with a regular flow of admissions and discharges. Whereas a psychiatric inpatient unit or clinic, for example, might treat thousands of individuals over the course of a year, an ACT team with an unlimited-duration policy would be able to care only for its initial caseload of 60–100 enrollees.

Adherence to the unlimited-duration precept began to be questioned when both clinicians and advocates recognized that some consumers who were enrolled in ACT achieved improved functioning over time and no longer required the intensity of services that ACT routinely provided. For these consumers, indefinite retention may not be consistent with principles of recovery (5). These concerns about cost and personal growth have stimulated a number of studies about the opportunities and outcomes associated with ACT graduation, transitioning, or step-down to less intensive services (6–8). In fact, states such as New Mexico have provided two “tracks” for ACT services, at different levels of intensity and funding (9).

The related issue of how much exposure to ACT is required to achieve positive outcomes has received less attention. One uncontrolled study of ACT found that the biggest drop in days of psychiatric hospitalization occurred after the first year (7). A report from the ACCESS study of homeless individuals with mental illness, in which participants were enrolled in ACT services for only one year and then transitioned to other community services, found that selected participants were able to transition without subsequent loss of gains in mental health status, substance abuse, housing, or employment (8). But there have been few controlled efforts to model the timing and trajectory of consumer outcomes after ACT enrollment. Interest has recently been increasing in “critical time interventions” after psychiatric hospitalization (10–12) and prison release (13), but the critical time idea is largely foreign to ACT research because of the intensity of ACT services and assumptions about indefinite enrollment.

Our study of Washington State’s PACT, a recovery-oriented ACT initiative, found large differences in the use of state psychiatric hospitals over a 36-month period in contrast to matched controls, especially among participants with high levels of prior psychiatric hospital use (14). This study expanded on those findings to determine the timing of PACT effects over a three-year period.

Methods

Sample

This study examined data for individuals who participated in one of ten PACT teams established by Washington’s State Mental Health Authority (15). The first clients were enrolled in July 2007. Details about the PACT teams, state psychiatric hospitals, and the data sources in this study are described in a companion article (14) and are only briefly summarized below.

Of the 636 PACT participants with a valid match in the administrative data assembled for this study, we selected the 450 participants with a history of state psychiatric hospital utilization and matched them on the basis of propensity scores with 450 participants who also had prior state hospital use but did not receive PACT (control group).

Data and variables

Statewide, linked administrative data were obtained from the Research and Data Analysis Division, Washington State Department of Social and Health Services (DSHS). The database included DSHS client service contacts since 2000, measures derived from Medicaid claims data, demographic characteristics, diagnostic information, and service costs (16). Comprehensive data on outpatient costs were not available for our study.

All data were collapsed to the person-month level, with each observation reflecting use of services during that calendar month. A balanced panel of 111 monthly observations (January 2001 to March 2010) was included for each of the 900 study participants, with up to 33 monthly observations in the postenrollment period (post period), depending on the timing of PACT enrollment. That is, the first PACT enrollees began receiving PACT services in July 2007 and had 33 months of post period data and 6.5 years of pre-PACT data, whereas subsequent PACT enrollees contributed more pre-PACT observations and fewer observations in the post period.

Study design and analysis methods

We used an observational study design to compare PACT enrollees and propensity score–matched consumers in the control group. Consumers in the control group were randomly assigned a start date for their post period, based on the distribution of actual start dates from the PACT sample (17). The propensity score model, balance, and matching process are described in a companion article (14). In short, we used a set of covariates in the propensity models describing demographic characteristics, diagnoses, and patterns of use of a variety of services offered in Washington State up to four years before PACT enrollment. The propensity (logit) model passed a number of specification and balancing tests (18). Control group participants were selected by using nearest-neighbor techniques. Because the results from the previous study were quite similar regardless of whether nearest neighbors in the control group were required to have an estimated propensity score within a fixed radius or caliper, in this article we report results only for the full PACT sample (N=450) and the propensity score–matched consumers in the control group (N=450).

Outcome measures included state hospital costs alone and a composite measure of psychiatric stay costs obtained by multiplying days spent in a local psychiatric hospital and days spent in local crisis stabilization units by costs approximating per diem reimbursement rates ($1,166 and $600, respectively [14]) and adding these monthly costs to state psychiatric hospital costs. Cost measures were estimated with generalized estimating equations with an AR1 correlation structure. We ran separate models of both cost measures, which allowed for a separate effect of PACT for participants with a substantial history of state psychiatric hospitalization (defined as ≥96 days in the previous two years [19]).

Our primary interest was in measuring whether the effect of PACT varied over time since enrollment. We therefore estimated a time-varying effect of PACT by creating quarterly indicators of time since PACT enrollment—or the start of the post period for consumers in the control group—and interacted these quarterly time periods with the PACT indicator variable. From these models, we generated the average marginal effect of PACT enrollment for each time period. [The full set of model estimates is available from the authors.]

We also examined whether the PACT teams exhibited learning. Learning would have occurred if PACT participants enrolled later in the study period had better outcomes (lower costs) than earlier PACT enrollees, with the analysis controlling for pre-PACT characteristics. Alternatively, enrollees of PACT teams may have exhibited proportionately worse outcomes over time if, as time went on, the teams selected participants less suitable for PACT. We therefore included a variable reflecting time since the first PACT participants were enrolled and interacted this variable with the PACT indicator; the direction of the marginal effect of time for PACT teams allowed us to determine which of the two effects (learning or differential selection) had a larger effect, if any. In all analyses, we controlled for an array of pre-ACT inpatient and outpatient service utilization measures, demographic characteristics, diagnoses, and region of Washington State (east or west). The research was conducted with the approval of Institutional Review Boards at University of North Carolina and the Washington State Department of Social and Health Services.

Results

As shown in Table 1, control group participants were similar to PACT enrollees on all observable baseline characteristics and service utilization indicators. Both groups had a mean age of 40, and about 60% were male. In the overall sample (PACT and control group participants), approximately 77% were white, 13% were African American, and just over 2% were Latino. The vast majority (over 97%) had a diagnosis of schizophrenia. Almost three-fourths also had a diagnosis of an affective disorder, and just over half had a substance use diagnosis, indicating a high level of comorbidity in this population.

At baseline, approximately one-fifth of the overall sample had used a state hospital in any month in the two previous years, and 58% met the criterion of “high users” (an average of ≥96 days in the previous two years). At baseline, average monthly state hospital costs for a PACT enrollee were $2,805. After PACT enrollment, the level of state hospital costs dropped to $1,458, whereas for consumers in the control group, costs increased to an average monthly costs of $2,838 (p<.01).

The effect of PACT on state hospital costs over the three-year follow-up period was previously estimated to be a reduction of $12,700 per person per year (or $1,058 per month), compared with the costs for consumers in the control group (14). The term “reduction” as used in this article reflects the difference in costs between PACT participants and the matched control group participants—not a reduction from baseline costs. As shown in Figure 1, the estimated effect of PACT actually varied substantially over time. The average monthly cost reductions during the initial quarter of PACT enrollment were almost $3,500 per person, nearly triple the three-year average. By the second quarter after PACT enrollment (months 4–6), the differential costs decreased to $2,790 per person per month. For each subsequent quarter of PACT enrollment, a decline in differential state hospital costs was observed for the PACT participants compared with the control group. By the seventh quarter (months 19–21), the difference between PACT and control participants had flattened out, but PACT participants continued to have lower state hospital costs than control group participants. Only in the last quarters (months 28–33) were the state hospital costs virtually indistinguishable between the two groups.

We examined whether the time trends in state hospital costs varied according to baseline measures of use. As shown in Figure 2, state hospital cost reductions were much larger for PACT participants with high baseline levels of state hospital use, and these differences persisted for the full study period. In contrast, PACT participants who did not have at least 96 days in the state hospital before PACT enrollment showed a reduction in state hospital costs only for the first three quarters; beginning in the fourth quarter (month 10 postenrollment), state hospital costs for PACT participants did not differ significantly from state hospital costs for the control group participants.

State hospital costs accounted for the bulk of composite psychiatric stay costs, because of the low use of local hospitals and crisis stabilization units. Therefore, trends in psychiatric stay costs (Figures 3 and 4) were similar to those for the state hospital, except that the lack of difference between PACT and control group participants occurred earlier (third quarter, or month 7) for those with lower state hospital use at baseline.

We also investigated learning among ACT teams during the 33-month postenrollment period. We found no difference in state hospital or composite psychiatric stay costs by time enrolled in PACT (data not shown).

Discussion

Consistent with our previous findings (14), this study indicated that the timing of PACT effects on state psychiatric hospital costs depended on the PACT participant’s level of state hospital use at baseline. For high users, the biggest effect occurred in the first quarter after PACT enrollment; however, reductions were observed throughout the entire postenrollment period. In contrast, for low users at baseline, no reductions occurred after the third quarter. These results raise a number of programmatic and policy issues. Results for a composite measure of psychiatric stay costs were largely consistent with results for state hospital costs, indicating that large offsets were not occurring from increases in psychiatric stays at local hospitals or crisis stabilization units.

These results indicate that the largest gain from PACT occurred initially and tapered off over time. Reductions in state hospital use occurred largely among consumers with a history of high levels of state hospital use, which is consistent with other literature. In our study, the reductions in state hospital costs for PACT participants compared with control group participants leveled out at just over $1,500 per month. It remains a limitation of our analysis that we were unable to determine the effect of PACT on outpatient costs or other related outcomes. Currently, Washington State spends $10.4 million per year for up to 800 consumers who receive PACT team services, averaging approximately $1,100 per person per month. For participants with high state hospital use, PACT is clearly paying for itself, at least in terms of reductions in the types of costs considered in this study. The lack of cost data on outpatient services limits us from offering a more precise estimate of total cost reductions over consumers in the control group who did not receive PACT services. However, the high intensity of service use within PACT teams suggests that any costs associated with external outpatient services would likely be nominal. For PACT enrollees with low baseline levels of state hospital use, reductions in composite psychiatric costs occurred only for the first six months on PACT.

Our results suggest that it may be useful to consider a much shorter enrollment period in PACT instead of the traditional “ACT-for-life” standard. Further extensions of PACT services could then be reauthorized in time-limited increments. Although the original ACT model allowed for titrated services depending on need, it did not consider adjusted reimbursements. One approach to adjusted reimbursements, which has taken hold in Holland (20,21), is to enroll consumers with severe mental illness in regular case management services and then ramp up the intensity of services to the level of ACT when a person requires it. Yet another option is exemplified by New Mexico’s Medicaid program, which authorizes a two-tiered reimbursement structure for ACT teams—a higher rate for consumers needing intensive services and a lower rate for those needing less frequent staff contact (9).

The optimum time on an ACT team caseload, however, needs to be considered in the context of outcomes other than decreased hospitalization. Research must address the question of whether the benefits that accrue from receiving ACT services change over time, with initial benefits from decreased hospitalization and later benefits from improvements in functional and quality-of-life outcomes. The lack of information about functional and quality-of-life outcomes is a limitation of the administrative data used in this study that should be noted.

Further research that focuses on the occurrence and timing of ACT’s effects on promoting community adjustment and recovery—rather than just avoiding negative outcomes such as frequent or prolonged hospitalization—can help to clarify these issues. Comparative studies of recovery-oriented ACT and traditional ACT would help to determine whether the rapid timing of reductions in state hospital use, as reported here for Washington State’s PACT teams, can be attributed to recovery-oriented practices and associated opportunities for consumers of personal growth and independence. The measurement and timing of recovery and whether ACT teams have any particular advantage over other service interventions should also receive priority attention in future studies.

Conclusions

Washington State’s PACT teams had their greatest effect in reducing state and local psychiatric hospital use immediately after consumer enrollment, and these effects tapered off over time. Further research is needed to address other positive functions of ACT and their contributions to the timing and sustainability of consumer recovery. In this sense, it would be premature to conclude that ACT treatment should be terminated when the cost reductions from decreases in hospitalization become negligible.