Full-Service Partnerships Among Adults With Serious Mental Illness in California: Impact on Utilization and Costs

The lack of safe, affordable, and integrated housing remains a significant barrier to participation in both health care and community life for persons with serious mental illness (1). Chronically homeless individuals may spend years, or even decades, living on the streets and in shelters and cycling through emergency rooms, inpatient and crisis facilities, jails, and mental health and substance use programs (2). As a result, these individuals incur high medical and social service costs, including increased use of inpatient and emergency services and justice system resources (3–5). Although the multiple service systems used by homeless persons with serious mental illness provide various opportunities for engagement, research has shown that homeless persons with serious mental illness are more likely to engage in a subset of treatment programs that are more responsive to their needs (6).

Supportive housing models have emerged as an effective approach to improving housing and health service outcomes, while reducing the costs of caring for chronically homeless persons (7,8). These models typically provide immediate housing and access to either intensive case management or a multidisciplinary treatment team, as well as community supports that provide flexible, consumer-driven services (1). Studies of supportive housing models have found that they result in reduced costs for diverse target populations, including persons with serious mental illness, severe alcohol problems, and chronic medical conditions (9–14). However, these studies have focused on relatively small populations, often within a single city. It is possible that program savings result from a smaller group of individuals who are higher users of expensive services (9). A recent meta-analysis suggests that cost savings might not be realized among populations with less severe illness (15). Despite this uncertainty and a national focus on supportive housing, there has not been an evaluation of a large-scale implementation of supportive housing programs (16).

A recent policy experiment in California provided an opportunity to address this limitation. On November 2, 2004, California voters approved Proposition 63, the Mental Health Services Act (MHSA), which applies a tax of 1% on incomes over $1 million to fund public mental health services (17). The cornerstone of the MHSA was the implementation of full-service partnerships (FSPs) (11). In this study, we examined changes in health service utilization and costs associated with California’s FSPs.

Methods

FSPs implemented under the MHSA

The FSP programs in California provide individuals with serious mental illness who are homeless or at risk of homelessness with subsidized permanent housing and multidisciplinary team–based services with a focus on rehabilitation and recovery. FSP services typically follow either an intensive case management model or a modified assertive community treatment model. Clients are recruited through outreach and referrals from psychiatric hospitals, emergency departments, other mental health programs, county agencies, jails, shelters, rescue missions, and the street. FSPs deliver services to clients in various settings: in their homes, workplaces, and other places in the community chosen by the client or that FSP staff has deemed to be of therapeutic value. Crisis intervention services are available 24 hours a day, seven days a week. A detailed description of FSP program practices, including their fidelity in comparison with the Housing First model, is provided elsewhere (18).

FSP study sample and propensity score matching

We used data from the California Department of Mental Health (DMH) Data Collection and Reporting (DCR) system to identify FSP participants and their initial participation date: 11,402 adult clients aged 25 to 59 were enrolled in FSPs between January 1, 2005, and June 30, 2009; 1,044 (9%) were excluded because they did not receive a diagnosis of serious mental illness (defined as schizophrenia, bipolar disorder, or major depressive disorder), and 127 were excluded (1%) because they died prior to 425 days after enrollment in the FSP. The final FSP sample included 10,231 adults.

Propensity score methods were used to identify a comparison group of clients with serious mental illness with demographic characteristics, clinical characteristics, and health service use profiles similar to those of the FSP clients (19,20). We used multiple administrative data sets from DMH (described below) to identify approximately 400,000 persons with serious mental illness who were receiving outpatient mental health services during the study period. Because these individuals did not have participation dates corresponding to those of the FSP clients, we randomly selected a participation date from days on which an outpatient mental health service was received. Using logistic regression, we estimated a propensity score of FSP participation based on service utilization in the prior year, participation date, and age, gender, race-ethnicity, clinical diagnosis, comorbid substance use disorder, Medicaid coverage, and county of residence. We identified a matched comparison group with nearest-neighbor matching (21). We have used this approach to identify comparison groups for studies of supportive housing programs in San Diego County (11,22).

Health service use and costs

Data on mental health service utilization and costs were derived from three administrative data sets: the Client and Services Information (CSI) system, Medi-Cal Short Doyle, and Inpatient Fee-for-Service (FFS) Consolidation. The CSI system is an encounter-based data system that is used to track state- and county-funded mental health services in California. CSI provided data on non-inpatient mental health service utilization for insured and uninsured persons as well as their demographic characteristics (including age, gender, and race-ethnicity) and clinical diagnoses. Short Doyle is a claims-based data system for non-inpatient mental health services that are reimbursed by Medi-Cal (California’s Medicaid program). Medi-Cal Short Doyle provided amounts paid for services and captured some services that were not tracked in CSI; for example, some psychiatrist services were billed directly to Medi-Cal and do not appear in CSI. The Inpatient FFS Consolidation database provided data on admissions to FFS hospitals, including the amounts paid for inpatient stays.

These administrative data were merged with the Inpatient Hospital Discharge and Emergency Department Encounter databases provided by California’s Office of Statewide Health Planning and Development (OSHPD) and with the vital statistics data provided by the California Department of Public Health. The OSHPD data were used to identify all inpatient admissions (including both psychiatric admissions and admissions for general medical conditions) and all emergency department admissions (also both mentally and medically related) that occurred in the state of California. Previous research has shown that admissions for psychiatric and general medical conditions may be correlated (23). The vital statistics data were used to limit the data to persons who remained alive during the study period; this restriction was applied so that the cost estimates would not be influenced by mortality (24). Merging these data involved matching specific services across multiple files in order to remove the overlap in reporting and avoid double-counting of services. The resulting data set captured all psychiatric services (except for state hospitals) as well as nonpsychiatric inpatient and emergency department admissions.

A limitation of encounter-based systems such as OSHPD and CSI is that they do not include information on the amounts paid for services. We used the Medi-Cal Short Doyle and Inpatient FFS Consolidation claims data to calculate service costs based on the amounts paid by Medi-Cal in 2010. We estimated the costs for these services with the actual amounts paid when available (for example, when the services were reimbursed by Medi-Cal). When these were not available (such as when the services were delivered to uninsured persons), we calculated service costs by multiplying the units of each service by the average amount paid by Medi-Cal per unit of service during 2010. We have used a similar approach to study the costs of mental health services in San Diego County (25).

Service use and costs were calculated for one year preenrollment and one year postenrollment in the FSP. We calculated costs for the following categories: inpatient services, crisis and residential services (including services provided by crisis residential facilities, psychiatric health facilities, residential facilities, emergency departments, and institutions for treatment of mental illness), and mental health outpatient services (including assessment, case management, crisis intervention, medication management, rehabilitation, and therapy). Utilization and cost data were available from January 1, 2004, through June 30, 2010. Thus clients had a full year of exposure to services in their pre- and postenrollment periods. Finally, we used data from the DCR to estimate days spent in a state hospital or in a justice system facility in the pre- and postenrollment periods among FSP clients. These data were not included in the main analyses because they were not available for the control group, but they are provided here to give an indication of the potential effect of missing data.

Study design and statistical analysis

Health service use and costs were analyzed with a quasi-experimental, pre-post, intent-to-treat design with a contemporaneous control group. This is also known as a difference-in-difference design (26). An estimator calculates the treatment effect by estimating the pre-post difference and accounting for possible confounding time trends by subtracting the observed pre-post difference from that of a control group. The design is “intent to treat” because all FSP participants were included even if they were discharged from the program during the follow-up period; we examined this approach with sensitivity analyses. Propensity score matching helps to ensure the validity of the key assumption of the difference-in-difference design: comparable time trends between the FSP and an otherwise comparable control group.

We used zero-inflated negative binomial regression models to analyze the numbers of inpatient days and outpatient visits (27–29). We used two-part models to analyze costs of inpatient and crisis and residential services. The two-part model is commonly used to estimate health care costs when the dependent variable is nonnegative and when its distribution is noticeably skewed and kurtotic (with a heavy right-hand tail) (30). We used logistic regression to estimate the probability of any health service use, and a generalized linear model (GLM), based on a gamma family with a log-link function, to estimate costs conditional on use of services. We used a single GLM to estimate mental health outpatient and total costs, because almost all individuals were users of these services. We selected these specific distributions based on standard tests for assessing alternative GLM and transformed models (31–33). We assessed goodness of fit by using a modified Hosmer-Lemeshow test and a Pregibon’s link test (34,35). We conducted analyses stratified by county to investigate whether the estimated effects of FSPs on costs varied across geographically distinct delivery systems. These analyses were limited to four counties with sufficient sample sizes to support county-level analyses.

In all models, we included age, gender, race-ethnicity, clinical diagnosis, comorbid substance use disorder, and Medi-Cal coverage as additional control covariates. For the postenrollment period, indicator variables were included for participation in the FSP and for the interaction between the FSP and the postenrollment period. Incremental effects associated with the FSP were standardized to the underlying population characteristics; in the two-part models, these effects were calculated over both parts of the model. We computed three sets of estimates from these regressions: pre-post estimates for FSP clients, pre-post estimates for the clients in the control group, and the difference between these estimated pre-post differences (difference-in-difference estimate). Standard errors were calculated with the nonparametric bootstrap, and p values were computed with the percentile method from the empirical distributions of the results from 1,000 replications (36). All analyses were conducted in Stata, version 12 (37).

The University of California, San Diego, Human Research Protections Program, the State of California Committee for the Protection of Human Subjects, and the OSHPD approved the use of these data for the purpose of this study in accordance with the Privacy Rule of the Health Insurance Portability and Accountability Act of 1996.

Results

Study sample characteristics are shown in Table 1. Among FSP clients, the mean age was 42±10, and 46% were female; 31% were non-Latino white, 10% were African American, 10% were Latino, 3% were Asian, and 46% were of other or unknown race-ethnicity; 62% had a diagnosis of schizophrenia, 20% bipolar disorder, and 18% major depressive disorder; 41% received a diagnosis of substance use disorder; and 58% had Medicaid coverage before enrollment in the FSP. There were no statistically significant differences in demographic or clinical characteristics between FSP clients and the propensity score–matched control group.

Estimates of annual standardized utilization are shown in Table 2. Inpatient days declined about equally in the FSP and control groups; as a result, the difference-in-difference estimate was small and statistically insignificant. In contrast, mental health outpatient visits increased substantially among FSP clients but remained relatively stable in the control group; the resulting difference-in-difference estimate of visits associated with FSP enrollment was a mean increase of 55.5 visits (p<.001).

Estimates of annual standardized costs are shown in Table 3. Consistent with the estimated changes in utilization, there was no significant difference in difference in costs of inpatient stays or costs for crisis and residential services between the FSP and control groups. Mental health outpatient costs increased by a mean of $11,725 (p<.001) for FSP clients compared with the control group, and total service costs were $12,056 (p<.001) higher for clients of FSPs compared with the control group.

Table 4 shows standardized difference-in-difference cost estimates stratified by county. Inpatient costs associated with FSP enrollment declined by a mean of $2,828 in San Diego County and by $1,166 in Los Angeles County (p<.05 each). In contrast, there was a significant increase of $3,957 in inpatient costs associated with FSP enrollment in Orange County (p<.001). There were significant increases in mental health outpatient costs in all four counties. Total costs ranged from $7,379 in San Diego County to $16,124 in Orange County (p<.001 each).

Changes in days spent in a state hospital and days spent in the justice system were examined before and after enrollment among FSP clients in order to provide information on the potential effect of missing data. Days in a state hospital declined a mean±SE of 3.0±.4 days, from 3.2±.4 to .2±.1 days, and days in the justice system declined by 12.5±1.0 days, from 34.1±.9 to 11.6±.6 days among FSP clients (p<.001 each, using paired t tests). Comparable data were not available for clients in the control group.

Discussion

We found that compared with a propensity score–matched control group, participation in FSPs was related to increases in mental health outpatient utilization and costs. Our findings contrast with previous studies that have found that the costs for more intensive services and subsidized housing are mostly or entirely offset by reductions in inpatient, emergency, and justice system costs (9–14). It is possible that the previously demonstrated cost offsets cannot be replicated beyond model programs. The leadership and support staff in model programs may be more fully committed to the program’s goals and to adhering to a high standard of program fidelity. Once these programs are expanded, there may be a risk that the leadership or staff members in the more recently created programs are less experienced or less committed to the supported housing model. It may also be that clients who benefit most from supportive housing programs are recruited into the model programs but significant expansions of the model result in the recruitment of clients who benefit less from these programs or who need less intensive supportive services.

We found declines in inpatient costs in two counties and an increase of inpatient costs in another county. Our previous work has demonstrated substantial variation in the implementation of FSPs. It is possible that variations in FSP program characteristics or in delivery system design at the county level moderate the effect of the FSPs on inpatient utilization. FSP program outreach or county policies regarding FSP eligibility may also affect the population targeted for enrollment and the resulting effect of FSP participation on inpatient utilization. Overall, our findings suggest that the favorable cost profile that has been associated with specific supportive housing programs may vary across large-scale implementation depending on program design and populations targeted for engagement.

Our study had a number of strengths and limitations. We were able to study a large-scale implementation of supportive housing programs using detailed data on service utilization and costs. The difference-in-difference study design that we used is one of the strongest of the observational study designs and is gaining wider use to evaluate interventions and natural policy experiments where results from randomized controlled trials are often costly and take considerable time to obtain (12,22,26). The difference-in-difference design removes both unobserved, time-invariant differences between two groups as well as unobserved, time-varying system-level changes between the pre- and postenrollment periods of the FSPs. The critical assumption of this design is that the two groups being compared have similar time trends. This assumption would be violated if the trajectory in illness severity or engagement in the system differed across the two groups in ways that were unrelated to the intervention. For example, in this study, we matched to potential clients in a control group on the basis of demographic characteristics, clinical characteristics, and service utilization in a preenrollment period. In a previous study, we matched to potential control group clients who were homeless at admission to an outpatient program (11). The differential service trajectories of homeless clients who are seriously mentally ill and seriously mentally ill clients in general may have affected our results and may account for some of the differences in the findings between these two studies.

The analyses relied on data derived from complex administrative databases. Although some of these databases overlapped, thus allowing verification (for example, the Medi-Cal Short Doyle and OSHPD inpatient data), it is possible that some mental health services were not captured in the data.

We did not have data on several types of services, including state hospitalization, residential and outpatient substance abuse treatment programs, general medical outpatient services or mental health services provided in the justice system. We did not have pharmacy data. We did not have data on social costs, including costs of crimes committed or resulting arrests and incarcerations. It is possible that cost offsets in these areas would reduce the total cost of the FSP. We also did not have measures of mental health recovery, quality of life, or emotional health, such as anxiety, stress, confusion, or depression. A previous study of FSPs in San Diego County showed that participation in the FSP was associated with improvements in several common dimensions of quality of life (11).

Conclusions

Our study provides timely analysis of an ongoing policy experiment that can inform policy makers, in California and nationwide, about the effectiveness and cost of FSPs and other supportive housing programs. FSPs are a major component of the MHSA and have been controversial because of their expense and the opportunity cost of services that might be delivered more widely (albeit less intensively). FSPs are similar in structure and operation to other large supportive housing initiatives under way across the country. Further research is needed to determine which FSP practices and policies are the most important predictors of costs. As supportive housing programs are implemented nationally and on a large scale, it is likely that these programs will need to be more effectively designed and targeted in order to achieve reductions in costly acute services.