The Effectiveness of Assertive Community Treatment for Homeless Populations With Severe Mental Illness: A Meta-Analysis

Several evidence-based interventions have demonstrated efficacy in treating severe mental illness (1) . The most aggressively studied model of case management is assertive community treatment, derived from the work of Stein and Test (2) . Assertive community treatment is distinguished from traditional approaches by the following features: a multidisciplinary team, low client/staff caseloads that enable more intensive contact, community-based services that are directly provided rather than brokered to other organizations, and 24-hour coverage by the treatment team (1 – 4) . The superiority of assertive community treatment compared with other case management models is well documented. A number of studies, including several meta-analyses, demonstrate statistically significant advantages of assertive community treatment in substantially reducing the length and frequency of hospitalization and increasing independent living while moderately improving psychiatric symptoms and quality of life for persons with severe mental illness (4 – 9) . While assertive community treatment is more costly to administer than other case management treatments, studies have found that it is more cost-effective because of a reduced utilization of hospitalization and emergency services (4 , 10) .

While evidence supports the effectiveness of assertive community treatment in treating persons with severe mental illness, less is known about its effects in specific subpopulations that present particular challenges for engagement or recovery (8) . One such subgroup is the homeless mentally ill. The homeless population in all cities of the United States exceeds 200,000, and an estimated 14 million Americans experience at least one episode of homelessness during their lifetime (11 , 12) . Among these individuals, 20% to 35% suffer from severe mental illness (13) . They also experience higher rates of substance abuse and criminal justice involvement (14) . As a result, this is a particularly challenging group for the mental health service system to engage and assist.

Identifying a best practice may help to improve the quality of care for the homeless mentally ill. There is no consensus regarding the best treatment; therefore, we sought to determine whether current evidence supports the use of assertive community treatment over other case management models in this population. We conducted a meta-analysis to test the hypothesis that assertive community treatment is more effective than other case management models in reducing homelessness, hospitalization, and symptom severity outcomes in homeless persons with severe mental illness.

Method

Study Selection

We performed a standardized search of abstracts in MEDLINE (1966–2003), PubMed (1950–2003), the Cochrane Database of Systematic Reviews, and PsycINFO (1974–2003) databases. The search involved the intersection of the following three topic areas: assertive community treatment (keywords: assertive community treatment or case management or intensive case management), severe mental illness (keywords: severe mental illness or community mental health services or psychotic disorders or schizophrenia), and homelessness (keywords: homeless persons or homeless or homelessness). Citations from two substantial reviews of assertive community treatment or community programs for homeless persons were also examined (8 , 15) . Several principal investigators were contacted regarding unpublished data. We examined studies for the following inclusion criteria: experimental or observational design; subjects limited to homeless persons with a severe mental illness, but not limited to addictive disorders alone; use of assertive community treatment or an assertive community treatment-based treatment; and report of the outcomes of interest (housing plus hospitalization and/or symptom severity). We set no limits on cohort size or length of follow-up.

Data Abstraction

Authors independently used a standardized data abstraction instrument to extract data from the included studies. Abstracted data consisted of the study design, cohort size, treatment versus comparison groups, measurement methods, and effect size with variance for the three outcome variables. Because of heterogeneity of measures across studies, we performed basic data transformation in order to create linearly equatable outcome measures. For example, in creating a homelessness measure, studies reporting days in stable housing during a follow-up period were transformed into days of homelessness by subtracting the published result from the total days in follow-up. When multiple symptom rating measures were available, we selected a measure of positive psychotic symptoms as the best representation of severe mental illness. For one study, we contacted authors to provide supplementary data (16) .

Data Synthesis

For experimental trials, we compared outcomes of subjects receiving assertive community treatment versus subjects receiving other case management treatments. For observational studies, we compared post-assertive community treatment versus pre-assertive community treatment assessments. In trials with three treatment groups (e.g., assertive community treatment versus two non-assertive community treatment comparison groups), results were aggregated into assertive community treatment and non-assertive community treatment outcomes by calculating a weighted mean and pooled standard deviation from the published data (17 – 20) .

We first calculated study-level raw effect differences with 95% confidence intervals (CIs). For binary outcomes, we calculated risk differences with standard error using methods published by Deeks (21) . For continuous outcomes, we used Hedges’ adjusted g to calculate standardized mean differences with standard error (21) . Effect differences were illustrated on a continuous scale that compared assertive community treatment subjects with comparison subjects. A study-level difference of 0% indicated no difference in outcome between groups. A positive-effect difference indicated the degree to which the assertive community treatment outcome surpassed the comparison outcome and vice versa.

In order to maximize homogeneity for statistical synthesis (22) , results were segregated by study type (randomized controlled trials and observational studies) and outcome (homelessness, hospitalization, and psychiatric symptom rating). This resulted in six subgroups for synthesis.

Assuming that the published measures within each subgroup remained different yet linearly equatable, unbiased estimators of pooled effect size and variance for each subgroup were calculated using fixed effects methods (21) . The heterogeneity of studies within each subgroup was then assessed using chi square test of the Q statistic (22) . When significant intragroup heterogeneity was discovered, we recalculated summary effects using random effects methods as described by DerSimonian and Laird (21 , 23) . All calculations were performed using Excel 2000 software.

Publication bias is a potential confounder of any meta-analysis. Following methods described by Petitti (22) , we assessed the likelihood of publication bias using a funnel plot displaying study-level effect difference versus cohort size. Because precision to a true effect difference increases with cohort size, a “funnel” with a wide base and narrow vertex is expected. When negative studies go unpublished, one corner of the funnel will be missing.

Results

The standardized search identified 52 abstracts for review. Ten studies met inclusion criteria; they were six randomized controlled trials comparing assertive community treatment with standard case management (3 , 18 – 20 , 24 , 25) and four observational studies comparing pre- and post-assertive community treatment outcomes (17 , 26 – 28) . We excluded publications that were reviews or descriptive accounts that did not include an assertive community treatment-based treatment or that did not report the outcomes of interest (29 – 43) . Study characteristics are shown in Table 1 . We note a number of potential limitations in the quality of included studies. Among randomized controlled trials, there is evidence of a gender bias in recruitment (3) as well as differential attrition between treatment groups (3 , 19) . Elsewhere, an intent-to-treat analysis is not documented (18 – 19) . We are unable to assess the presence or impact of systematic bias in the individual studies created by these limitations.

Enlarge table

Assertive community treatment subjects experienced significantly greater success in reducing homelessness in eight out of 10 studies and four out of six randomized trials ( Table 2 ). The summary effect across randomized trials was 37% (95% CI=18%–55%, Z=3.85, p=0.0001), signifying that assertive community treatment subjects, on average, experienced a 37% greater reduction in homelessness compared with standard case management subjects. Across observational studies, subjects averaged more substantial improvement ([104%] 95% CI=67%–141%, Z=5.50, p<0.0001) when compared with baseline housing levels.

Enlarge table

On average, assertive community treatment subjects had better hospitalization outcomes; however, this was statistically significant in only one of four randomized trials ( Table 3 ). The summary effect across randomized trials revealed no significant difference in hospitalization between assertive community treatment and standard case management ([10%] 95% CI= -7%–27%, Z=1.17, p=0.24). A single observational study demonstrated better hospitalization outcomes after assertive community treatment ([69%] 95% CI=60%–78%, Z=5.21, p<0.0001).

Enlarge table

Assertive community treatment subjects had significant reductions in psychiatric symptom severity beyond that experienced by comparison subjects ( Table 4 ). Study-level effect differences were significant in four out of six studies, including two out of three randomized controlled trials. When combined, assertive community treatment subjects averaged a 26% (95% CI=7%–44%, Z=2.76, p=0.006) further symptom improvement in randomized trials and a borderline significant 62% (95% CI=0%–124%, Z=1.96, p=0.05) greater symptom improvement in observational studies.

Enlarge table

Data for the analysis of publication bias is shown in Figure 1 . We see a narrowing range of study-level effect differences as cohort size increases. Both negative and positive results are reported in the included studies.

Discussion

Relative to standard case management or comparison treatments, assertive community treatment is associated with significant improvements in rates of homelessness and levels of psychiatric symptom severity in the homeless mentally ill. The evidence shows that assertive community treatment was statistically equivalent to standard case management in reducing hospitalization.

As previously mentioned, prior meta-analyses examined the effectiveness of assertive community treatment versus other case management in severely mentally ill subjects without a specified housing status. These studies showed robust improvement in housing stability for assertive community treatment subjects. It is not surprising that our study, which specifically examines homeless subjects, replicates this significant advantage. It is likely that the key processes of assertive community treatment provide real advantages in engaging the severely mentally ill and providing social supports that correlate with stable housing (44) .

The earlier meta-analyses also showed that assertive community treatment led to better hospitalization and symptom severity outcomes for severely mentally ill subjects. In contrast, we found that assertive community treatment led homeless mentally ill subjects toward statistically significant symptom severity reduction but not hospitalization reduction when compared with standard case management.

We believe that the difference in hospitalization findings can be explained by the heterogeneity of the hospitalization measures used in our meta-analysis. As can be seen in Table 3 , included studies reporting “days hospitalized” demonstrated more positive findings, while studies reporting “percent hospitalized” demonstrated more negative findings. This suggests that assertive community treatment subjects were more likely to be hospitalized but, simultaneously, to spend less time in the hospital than comparison subjects.

Because the homeless mentally ill are a population difficult to engage (45) , it is possible that a higher hospitalization rate using assertive community treatment could be considered a success in treatment. Fewer days in the hospital may indicate a benefit of assertive community treatment in facilitating shorter stays or reducing rehospitalization. A “percent hospitalized” measure (18 , 25) is not capable of making these distinctions. Their inclusion, therefore, limits the interpretation of our hospitalization outcome. In the future, we recommend the use of “days hospitalized” as a more appropriate measure to assess homeless mentally ill subjects.

Our study is affected initially by the limitations of the included studies ( Table 1 ). Four out of 10 studies were observational and reported greater effect differences. Evidence of impaired randomization or interpolation of missing data could bias in favor of assertive community treatment. In contrast, documentation of low fidelity to assertive community treatment or availability of assertive community treatment to comparison subjects from agencies outside of the study could contribute to a type II error. While each of these limitations may bias the results of an individual study, we do not detect a consistent bias across studies. We conclude that there is a low likelihood of systematic bias in the meta-analysis. Regarding a possible publication bias, our meta-analysis includes smaller studies that exhibit both large and small and positive and negative effect differences ( Figure 1 ). We therefore conclude that there is a low likelihood that publication bias affects the meta-analysis.

The meta-analysis also has limitations. The cohort of included studies was small. In order to avoid combining dissimilar entities, studies were then segregated by design and outcome, further limiting the power of the meta-analysis. Regarding the symptom severity outcome measure, studies reported a wide variety of instruments, and an assumption was made that these were linearly equatable. Generalizability must also be interpreted with caution. All subjects had a severe mental illness not limited to an addictive disorder, but variation in diagnostic case-mix was not consistently reported across studies.

Despite these limitations, evidence supports our conclusion that assertive community treatment offers significant advantages over standard case management programs in the care of homeless persons with severe mental illness. The use of assertive community treatment leads to greater improvement in housing stability and symptom reduction early in treatment. While hospitalization appears similar in assertive community treatment and standard case management, differences in hospitalization rate and duration require further study. These findings provide support for policy makers and community program directors to institute assertive community treatment as a best available practice to improve outcomes for the homeless mentally ill.