Permanent Supportive Housing for Transition-Age Youths: Service Costs and Fidelity to the Housing First Model

Permanent supportive housing (PSH) has been demonstrated to be an effective approach to reducing homelessness among persons with serious mental illness and engaging them in treatment (1). These models typically provide immediate housing and access to either intensive case management or a multidisciplinary treatment team, as well as community supports that provide flexible, consumer-driven services (2). PSH has been shown to result in increased housing stability and reduced costs for diverse target populations, including persons with serious mental illness, severe alcohol problems, and chronic general medical conditions (3–9). However, less evidence exists for the effectiveness of supported housing for nonadult populations, such as transition-age youths—that is, individuals ages 18 to 24 who are making the transition to adulthood.

PSH may be an effective approach to engaging transition-age youths with mental illness, who have higher rates of homelessness, poverty, substance use disorders, and criminal justice involvement and lower rates of education and employment than their peers without mental illness (10–14). The challenges inherent in the transition to adulthood are often more difficult among youths with mental illness as a result of emancipation among foster care youths, a lack of natural mentors, and mental health service needs related to life transitions that are not adequately met by the mental health service system (15–17). A qualitative study of PSH programs found that youth-oriented programs differed from adult programs in their referral sources and their housing and treatment strategies (18). Self-referrals were more common among youths, housing was more often considered transitional and involved roommates, and services focused to a greater extent on education geared toward youths’ gaining meaningful employment. The focus of PSH on education, employment, and the development of independent living skills addresses commonly cited service needs among transition-age youths (19,20).

A policy experiment in California provided an opportunity to examine PSH programs for transition-age youths. On November 2, 2004, California voters approved Proposition 63, the Mental Health Services Act (MHSA), which applies a tax of 1% on incomes over $1 million to fund public mental health services (21). The cornerstone of the MHSA is the implementation of full-service partnerships: team-based PSH programs that do “whatever it takes” to improve housing and recovery outcomes among persons with serious mental illness who are homeless or at risk of homelessness (5). The MHSA also supports a diverse array of programs, including outreach and treatment for underserved populations, prevention and early intervention programs, and innovative approaches to service delivery. In this study, I estimated changes in health service costs associated with youth participation in California’s FSPs and examined the relationship between fidelity to the Housing First model and health service outcomes.

Methods

PSH Programs Implemented Under the MHSA

The PSH programs in California provide individuals with serious mental illness who are homeless or at risk of homelessness with subsidized permanent housing and multidisciplinary team–based services with a focus on rehabilitation and recovery. PSH services follow either an intensive case management model or a multidisciplinary treatment team model (22). Clients are recruited through outreach and referrals from psychiatric hospitals, emergency rooms, other mental health programs, county agencies, jails, shelters, rescue missions, and the street. Most PSH programs deliver services to clients in real-world settings: in their homes, workplaces, and other places in the community chosen by the client or deemed of therapeutic value by staff. Crisis intervention services are available 24 hours a day, seven days a week.

PSH Study Sample and Propensity Score Matching

I used administrative data (described below) to identify 2,609 youths ages 18 to 24 with a diagnosis of serious mental illness (defined as schizophrenia, bipolar disorder, or major depressive disorder) who enrolled in PSH programs between January 1, 2005, and June 30, 2009. Propensity score methods were used to identify a comparison group of youths with serious mental illness with demographic and clinical characteristics and health service use profiles similar to those of PSH clients (23,24). I identified 42,112 youths with serious mental illness who were receiving outpatient mental health services during the same time frame as youths enrolling in PSH programs. Because these youths did not have participation dates corresponding to youths in PSH programs, I randomly selected a participation date from the days on which an outpatient mental health service was received. Using logistic regression, I estimated a propensity score of PSH participation based on prior-year service utilization and participation month, as well as on age, gender, race-ethnicity, clinical diagnosis, comorbid substance use disorder, Medicaid coverage, and county of residence. I identified a matched comparison group by using nearest-neighbor matching (25). This approach has been used to identify comparison groups for studies of PSH programs for adults (5,26,27).

Health Service Costs and Service Outcomes

Data on mental health service utilization and costs were derived from multiple administrative data sets from the State of California. The Department of Mental Health provided data on PSH enrollment, demographic characteristics, clinical diagnoses, and mental health service utilization. The Office of Statewide Health Planning and Development provided data on inpatient admissions (including both psychiatric admissions and those for general medical conditions) and emergency department admissions (also related to both mental and general medical conditions). The Department of Health Care Services provided information on service costs. Merging these data involved matching specific services across multiple files in order to remove the overlap in reporting and avoid double-counting of services. The resulting data set captured all psychiatric services and their costs (except for state hospitals and jails) as well as nonpsychiatric inpatient and emergency department admissions.

Service use and costs were calculated for one year before and one year after enrollment in the PSH program. I calculated costs for the following categories of service: inpatient, crisis and residential services (including services provided by crisis residential facilities, psychiatric health facilities, residential facilities, emergency departments, and institutions of mental disease), and mental health outpatient (including assessment, medication management, rehabilitation, and therapy). Utilization and cost data were available from January 1, 2004, through June 30, 2010. Thus clients had a full year of exposure to services in their pre- and postentry periods.

Fidelity to the Housing First Model Among PSH Programs

Fidelity was measured at a point in time, from June through November 2010, with the self-report Housing First Fidelity Survey (28). Among the 58 counties in California, 53 (91%) implemented PSH programs under the MHSA, and of these counties, 23 (43%) participated in survey. Among participating counties, 93 of 135 FSP programs (69%) responded to the survey. The survey measures fidelity to the Housing First model across two factors and five domains. One factor measures fidelity with respect to housing choice and structure, separation of housing and services, and service philosophy. A second factor measures fidelity with respect to service array and team structure. Factor scores were used to rank programs by fidelity: the top 20% of programs were designated as high fidelity and the bottom 20% were designated as low fidelity; the remaining were mid-fidelity (29).

Study Design and Statistical Analysis

Health service costs were analyzed by using a quasi-experimental, intent-to-treat, difference-in-differences (DID) design with a propensity score–matched contemporaneous control group (30). A DID estimator calculates the treatment effect by estimating the pre-post difference and accounting for possible confounding time trends by subtracting the observed pre-post difference values from a control group. The intent-to-treat design included all PSH participants even if they were discharged from the program during the follow-up period. Propensity score matching helped to ensure the validity of the key assumption of the DID design: comparable time trends between the PSH participants and an otherwise comparable control group. Health service use by level of fidelity to the Housing First model was analyzed using a pre-post design among youths enrolled in PSH programs that responded to the Housing First Fidelity Survey.

I used generalized linear models to estimate health service costs and service outcomes. I used two-part models to analyze inpatient and crisis and residential service costs. The two-part model is commonly used to estimate health care costs when the dependent variable is nonnegative and when its distribution is noticeably skewed and kurtotic (with a heavy right-hand tail) (31). I used logistic regression to estimate the probability of any use and a model based on a gamma family with a log link function to estimate costs conditional on use of services. I used a single model based on a Poisson family with a log link function to estimate mental health outpatient costs. Total costs were estimated as the sum of the estimated component costs. I used zero-inflated negative binomial regression models to estimate the number of outpatient visits (32–34). I selected these specific distributions on the basis of standard tests for assessing alternative generalized linear and transformed models (35–37). I assessed goodness of fit by using a modified Hosmer-Lemeshow test and a Pregibon’s link test (38,39).

The primary variables of interest in the models comparing PSH participants with the propensity score–matched control group were indicator variables for participation in the PSH program for the postperiod and for the interaction between the PSH and the postperiod. The primary independent variables of interest in the models comparing PSH participants by level of fidelity to the Housing First model were indicator variables for levels of fidelity and for interactions between levels of fidelity and the postperiod. In all models, I included age, gender, race-ethnicity, clinical diagnosis, comorbid substance use disorders, and Medi-Cal coverage as additional control covariates.

Incremental effects associated with PSH and level of fidelity were standardized to the underlying population characteristics. For the models comparing PSH participants with the propensity score–matched control group, I calculated pre- and postperiod estimates for PSH clients, pre- and postperiod estimates for clients in the control group, and the difference between these estimated pre-post differences (DID estimate). For the models comparing PSH participants by level of fidelity, I computed standardized preperiod, postperiod, and difference estimates by level of fidelity. These estimates were based on the assumption that a program had the same level of fidelity for each of the two factors of fidelity to Housing First. Standard errors were calculated by using the nonparametric bootstrap with clustering at the program level, and p values were computed by using the percentile method from the empirical distributions of the results from 1,000 replicates (40). All analyses were conducted in Stata, version 13 (41).

The University of California, San Diego, Human Research Protections Program, the State of California Committee for the Protection of Human Subjects, and the Office of Statewide Health Planning and Development approved the use of these data for the purpose of this study in accordance with the privacy rule of HIPAA.

Results

Study sample characteristics are shown in Table 1. Among youths in PSH programs, the mean age was 21±2, and 1,034 were female. A total of 818 were non-Latino white, 280 were African American, 569 were Latino, 93 were Asian, and 849 were of other or unknown race-ethnicity. Diagnoses were as follows: schizophrenia, 1,240; bipolar disorder, 739; major depressive disorder, 630; and substance use disorder, 1,213. Prior to enrollment in PSH programs, 1,529 had Medicaid coverage. There were no statistically significant differences in demographic or clinical characteristics between youths in PSH programs and youths in the propensity score–matched control group.

PSH Program Characteristics

Factor scores were used to rank PSH programs by fidelity for each of the two factors. Of the 93 PSH programs surveyed, 77 (83%) were located in counties that provided service utilization data and of these, 63 (82%) enrolled transition-age youths. Table 2 shows PSH program characteristics of these 63 programs at low, medium, and high levels of fidelity to the Housing First model. By design, high-fidelity programs were more likely than mid- or low-fidelity programs to meet fidelity thresholds for most items in the five domains. Exceptions included the two items in the housing choice and structure domain and opportunities for community employment and volunteering and program provision of three core social integration services in the service array domain. Low-fidelity programs were less likely than mid- and high-fidelity programs combined to offer these services (p<.05 for each, data not shown).

Health Service Utilization and Costs

Estimates of annual standardized costs are shown in Table 3. Inpatient costs and costs for crisis and residential services declined to a greater extent for the control group than for youths in PSH programs. As a result, the DID estimates show that inpatient costs increased by $1,088 (p=.046), costs for crisis and residential services increased by $1,271 (p=.036), and mental health outpatient costs increased by $10,979 (p<.001) for youths in PSH programs compared with the control group; as a result, total service costs were $13,337 (p<.001) higher for youths in PSH programs compared with the control group.

Table 4 shows the standardized probability of inpatient admission and the standardized number of outpatient mental health visits by level of program fidelity. There were no significant differences in the probability of inpatient admission by level of program fidelity in the preperiod. The probability of admission decreased by 9.2 percentage points (p=.004) among youths in mid-fidelity programs and by 14.9 percentage points (p<.001) among youths in high-fidelity programs. Youths in low-fidelity programs had more outpatient mental health visits in the preperiod than youths in high- and mid-fidelity programs (p<.001 for between group differences, not shown). Patients in high-fidelity programs had the largest increase in the number of visits, followed by clients in mid- and low-fidelity programs: 63.8 versus 43.7 and 26.9, respectively (p<.001 for between group differences, not shown).

Table 5 shows standardized total costs by level of program fidelity. Patients in high-fidelity programs had the largest increase in costs, followed by clients in mid- and low-fidelity programs: $17,610 versus $7,224 and $4,575, respectively (p<.001 for between-group differences, not shown).

Discussion

This study found that compared with a propensity score–matched control group, youths participating in PSH programs had increased inpatient, crisis residential, and mental health outpatient costs. Our findings contrast with those of previous studies that have found that among adults, the costs for more intensive services are mostly or entirely offset by reductions in inpatient, emergency, and justice system costs (3–9). I believe this is the first study to show potentially increasing inpatient costs associated with PSH participation. PSH programs may provide greater access to specialized inpatient services for youths, or the increased inpatient admissions may reflect a positive strategy for some youths. Alternatively, as discussed below, youths who enroll in PSH programs may be experiencing different service trajectories compared with youths who do not enroll.

This study also found that youths in PSH programs with higher fidelity to the Housing First model had greater declines in the probability of inpatient admission and greater increases in outpatient visits in the year after enrollment, compared with youths in lower-fidelity programs. This suggests that higher-fidelity programs may be more effective than lower-fidelity programs at engaging youths in appropriate treatment modalities. In higher-fidelity programs, the service philosophy may be more welcoming, the service array may be more youth specific or may provide more opportunities for youths (for example, in supported education), or the team structure may be more effective at engaging youths in treatment. The finding of lower rates of inpatient admission among youths in high-fidelity programs suggests that it may be particularly important for youth-oriented PSH programs to adhere to the Housing First model.

This study had a number of strengths and limitations. The study examined a large-scale implementation of PSH programs for youths by using detailed data on service utilization and costs. The DID study design is widely used to evaluate natural policy experiments (6,26,30). The DID design removes both unobserved, time-invariant differences between groups as well as unobserved, time-varying system-level changes. The critical assumption of a DID design is that the groups being compared have similar time trends. This assumption would be violated if, for example, PSH participants were experiencing a worse trajectory in their illness than the matched controls (as opposed to being simply more severely ill). In this case, the DID estimator could provide an estimate that is biased against PSH programs.

Fidelity was measured at a point in time by using a self-administered survey. This approach offers an expeditious way of obtaining information on a critical array of practices across a wide range of programs. However, using a point-in-time estimate does not capture any changes in fidelity over time. Also, a survey measure of fidelity lacks depth and objectivity compared with a site visit, and some providers may be overstating the fidelity of their programs. These limitations could weaken the estimates of the effect of fidelity on outcomes. Participation in the survey was voluntary, and participating programs may be different than nonparticipating programs. This study had limited measures of program effectiveness and did not include measures of mental health recovery, education, employment, quality of life, or emotional health, such as anxiety, stress, confusion, or depression. A previous study of PSH in San Diego County showed that participation was associated with increases in several common dimensions of quality of life (5).

The findings suggest that PSH programs may not be optimized for transition-age youths. The finding that youths in higher-fidelity programs experienced a greater reduction in admissions suggests that PSH program practices can be modified to improve youth outcomes. Additional work is required to determine which program practices are most effective at supporting transition-age youths. To the extent that the most effective practice can be identified, it may be desirable to disseminate these practices into less intensive outpatient programs where they can reach a larger number of youths at a lower cost. The large increase in outpatient mental health services suggests that implementation of PSH programs for youths appears to be a promising first step toward providing youths with intensive team-based services. However, continuing efforts to improve the model may result in improved outcomes for transition-age youths.

Conclusions

This study suggests that PSH programs may not be adequately designed or implemented to meet the needs of transition-age youths who are at high risk of inpatient admissions and that higher-fidelity PSH programs are more effective than lower-fidelity programs at improving health service use outcomes among transition-age youths. Additional research is necessary to determine the specific program practices that most effectively support transition-age youths in their recovery from mental illness.