Fidelity to the Housing First Model and Variation in Health Service Use Within Permanent Supportive Housing

Permanent supportive housing has been recognized by the federal government as the main strategy to address chronic homelessness among persons who have a serious mental illness and other disabilities (1). It has been shown to result in increased housing stability and reduced costs for diverse target populations, including persons with serious mental illness, severe alcohol problems, and chronic general medical conditions (2–7). Permanent supportive housing has been credited with decreasing chronic homelessness by 25% from 2006 to 2013 through federal funding initiatives such as Shelter Plus Care and through local and state investment (8–10). However, substantial variation exists in the implementation of permanent supportive housing (11,12). Although there are well-articulated models for it (13), the definition of permanent supportive housing beyond the notion of providing permanent housing with ongoing supportive services remains underspecified (14).

Previous studies have found that permanent supportive housing is associated with improvements in residential outcomes, reductions in the use of inpatient and emergency services, and increases in the use of outpatient services (4,15). These studies suggest that permanent supportive housing may result in more appropriate service use, steering individuals away from inpatient and emergency services and toward community-based care. Results regarding the cost-effectiveness of permanent supportive housing vary, but studies tend to demonstrate cost savings among clients with high inpatient service utilization, extensive incarceration, or extensive use of homeless shelters (2,15). However, few studies consider how differences in programs’ practices may affect changes in service utilization patterns.

An important component of permanent supportive housing, as advocated for by the federal government, is the use of a Housing First approach (1). The Housing First model provides homeless individuals with immediate access to housing and either intensive case management or a multidisciplinary treatment team without readiness requirements or requirements for treatment participation, and it provides community supports that offer flexible, client-driven services (16). The development of fidelity scales has facilitated the study of the components of Housing First (17–19). Permanent supportive housing programs with greater fidelity to the Housing First model rather than less have been shown to result in greater improvements in residential outcomes, by enrolling clients with longer histories of homelessness and placing most of these individuals in apartments (20). However, to date there has not been a systematic investigation of the relationship between fidelity to Housing First and health service utilization among persons in permanent supportive housing programs.

A recent policy experiment in California provided an opportunity to examine the relationship between fidelity to the Housing First model and health service utilization by studying a large-scale implementation of permanent supportive housing programs. On November 2, 2004, California voters approved the Mental Health Services Act (MHSA), which applied a tax of 1% on incomes over $1 million to fund public mental health services (21). The cornerstone of the MHSA consists of full-service partnerships (FSPs), calling for permanent supportive housing programs to do “whatever it takes” to improve residential stability and mental health outcomes among persons with serious mental illness who are homeless or at risk of homelessness (4). FSPs share goals, vision, and structure similar to those of the Housing First model (20). The MHSA’s emphasis on a whatever-it-takes vision of recovery-oriented care, flexibility in funding, and influence of stakeholders, combined with a lack of specificity and oversight regarding expected FSP practices, led to the implementation of a diverse set of FSP programs (22). In this study, we leveraged the variation in the implementation of the FSPs to study the relationship between fidelity to the Housing First model and use of health services among program participants.

Methods

FSPs Implemented Under the MHSA

The FSP programs in California provide individuals with serious mental illness who are homeless or at risk of homelessness with subsidized permanent housing and multidisciplinary team-based services with a focus on rehabilitation and recovery. FSP services follow either an intensive case management model or a multidisciplinary treatment team model (23). Clients are recruited through outreach and referrals from psychiatric hospitals, emergency departments, other mental health programs, county agencies, jails, shelters, rescue missions, and the street. Most FSPs deliver services to clients in settings such as their homes, workplaces, and other places in the community either chosen by the client or deemed of therapeutic value by staff. Crisis intervention services are available 24 hours a day, seven days a week.

Fidelity to the Housing First Model Among FSP Programs

Fidelity was measured at a point in time, from June through November 2010, with the use of the Housing First Fidelity Survey (18). We recruited 93 FSP programs to participate in the survey. The survey instructions requested that FSP staff review the survey and respond as a team that included the FSP program manager, several staff members, and at least one—but preferably two—client representatives. These instructions were aimed at increasing the range of staff input and including the client’s voice in survey responses.

The survey measures fidelity to the Housing First model across two factors and five domains. One factor measures fidelity with respect to the domains of housing choice and structure, separation of housing and services, and service philosophy. This factor considers client choice in housing, use of scattered-site housing, affordability, and privacy; the extent to which obtaining and retaining housing is independent of participation in treatment; and client choice in treatment and the recovery orientation of the program. A second factor measures fidelity with respect to the domains of service array and team structure. This factor considers the types of services available and the organization and coordination of services, staffing, and client representation.

Three domains of fidelity in particular may be expected to affect service use. The service philosophy of high-fidelity programs emphasizes client choice and employs a harm reduction approach, which may encourage increased treatment participation. High-fidelity programs provide a wide array of services that may be more attractive to clients than lower-fidelity programs or that may provide more opportunities for treatment participation. The team structure of high-fidelity programs may more effectively engage and retain clients in treatment.

Factor scores were used to rank programs on fidelity to the Housing First model according to each factor. With our knowledge of the programs and examination of natural cut points, we designated the top 20% of programs as having high fidelity to the Housing First model, the bottom 20% as having low fidelity, and the remaining programs as having mid-fidelity.

FSP Participant Study Sample

We used data from the California Department of Mental Health (DMH) Data Collection and Reporting system to identify FSP participants and their initial participation date. Data on mental health service utilization were derived from the DMH Client and Services Information (CSI) system. The CSI system is an encounter-based data system that is used to track state- and county-funded mental health services in California. CSI provided data on mental health service utilization for insured and uninsured persons who were not inpatients, as well as their demographic information (including age, gender, and race-ethnicity) and clinical diagnoses. These administrative data were merged to the Inpatient Hospital Discharge and Emergency Department Encounter Databases provided by California’s Office of Statewide Health Planning and Development (OSHPD). The OSHPD data were used to identify all inpatient admissions (including psychiatric admissions and admissions for general health conditions) and all emergency department admissions (also concerning mental health and general health) that occurred in the state of California. The resulting data set captured all psychiatric services (except for those of state hospitals) as well as nonpsychiatric inpatient and emergency department admissions.

Service use was calculated for one year preenrollment and one year postenrollment in the FSP for the following categories of service: inpatient admissions, days using crisis and residential services (including services provided by crisis residential facilities, psychiatric health facilities, residential facilities, emergency departments, and institutions of mental disease), and mental health outpatient visits (including for assessment, collateral, crisis intervention, medication management, rehabilitation, and therapy). Utilization data were available from January 1, 2004, through June 30, 2010. Thus clients had a full year of exposure to services in their pre and post periods.

Study Design and Statistical Analysis

Health service use was analyzed with a pre-post design. We used logistic regression models to estimate the probability of any use of inpatient and crisis or residential services. We used zero-inflated negative binomial regression models to estimate the number of outpatient visits (24–26). The primary independent variables of interest were indicator variables for levels of fidelity to the Housing First model and for the interactions between levels of fidelity and the post period. In all models, we included age, gender, race-ethnicity, clinical diagnosis, comorbid substance use disorder, and Medicaid coverage as additional control covariates.

We computed standardized pre, post, and difference estimates for each outcome by level of fidelity. These estimates were conditional on a program’s having the same level of fidelity for each of the two factors of fidelity to Housing First. Standard errors were calculated with the nonparametric bootstrap with clustering at the program level, and p values were computed with the percentile method from the empirical distributions of the results from 1,000 replications (27). All analyses were conducted in Stata, version 12 (28).

The University of California, San Diego, Human Research Protections Program, the State of California Committee for the Protection of Human Subjects, and the Office of Statewide Health Planning and Development approved the use of these data for the purpose of this study in accordance with the 1996 HIPAA Privacy Rule.

Results

Client Characteristics

Demographic and clinical characteristics of 5,067 FSP clients are shown in Table 1. Overall, the mean age was 40±14; 45% of the sample were female; 65% had a diagnosis of schizophrenia, 19% had bipolar disorder, and 15% had major depressive disorder; 54% received a diagnosis of substance use disorder.

FSP Program Characteristics

Table 2 shows FSP program characteristics at low, medium, and high levels of fidelity to the Housing First model. Of the 93 FSP programs surveyed, 77 (83%) were located in counties that provided service utilization data. By design, high-fidelity programs were more likely than mid- or low-fidelity programs to meet fidelity thresholds for most items in the five domains. Exceptions included the two items in the housing choice and structure domain, and two items in the service array domain—opportunities for community-based employment and volunteering. For these items, low-fidelity programs were less likely than mid- and high-fidelity programs combined to offer these services (p<.05 each, data not shown).

Use of Mental Health Services, by Level of Program Fidelity

Table 3 shows the standardized probability of inpatient admission by level of program fidelity. Clients in high- and mid-fidelity programs were more likely than clients in low-fidelity programs to have an inpatient admission in the pre period (p<.001). The probability of admission between pre and post periods decreased across all levels of fidelity. Mid-fidelity programs had larger decreases in inpatient admissions than both low- and high-fidelity programs (p<.001 each); there was no difference in the change in inpatient admission between the low- and high-fidelity programs (p=.632).

Table 4 shows the standardized probability of using crisis or residential services by level of program fidelity. Similarly to the inpatient findings, clients in high- and mid-fidelity programs were more likely than clients in low-fidelity programs to use crisis or residential services in the pre period (p<.001). Probability of use declined across all three groups, but there was no statistically significant difference in the change in the probability of use across the three groups.

Table 5 shows the standardized number of outpatient mental health visits, by level of program fidelity. Clients in high- and mid-fidelity programs had fewer visits in the pre period than clients in low-fidelity programs had (p<.001). Patients in high-fidelity programs had the largest increase in the number of visits, followed by clients in mid-fidelity and clients in low-fidelity programs: 71.6 versus 48.2 and 29.0, respectively (p<.001).

Discussion

This study found that clients in FSP programs with higher fidelity to the Housing First model had greater increases in outpatient visits in the year after enrollment compared with clients in lower-fidelity programs. This suggests that higher-fidelity programs may be more effective than lower-fidelity programs at increasing outpatient service utilization. In higher-fidelity programs, the service philosophy may be more welcoming, the service array may be more attractive or provide more opportunities, or the team structure may be more effective at engaging clients in treatment. Higher-fidelity programs also enrolled clients who used fewer mental health outpatient services in the year before enrollment. This finding suggests that higher-fidelity programs may be more effective than lower-fidelity programs in their outreach to and engagement of clients who are not appropriately served by the public mental health system. The concept of high-fidelity programs conducting more outreach and engagement is supported by previous qualitative work that found that high-fidelity programs reported conducting outreach to target individuals in the community who were not being served by the existing system (29).

An alternative explanation is that because the programs studied here are designed to serve a certain number of clients at a certain level of service, the greater increase among high-fidelity programs simply reflects the lower service use in the year before enrollment. That is, these programs may be designed to serve, for example, 100 clients over a year who will make an average of 70–80 visits per year, and this may not vary significantly with illness severity of the clients. Multidisciplinary teams are often expected to maintain a certain consistent level of client contact across their caseload to meet program standards.

Finally, the greater increase in use of outpatient services among high-fidelity FSPs may have been a response to the potentially more complex needs of the population served by these programs, as indicated by their clients’ higher use of inpatient, crisis, and residential services before enrollment. The fact that mid- and high-fidelity programs enrolled participants who had higher levels of these acute care services in the pre period suggests that more intense services were required in order to see rates of reduction in those services similar to those seen with low-fidelity programs. It is not clear why clients in mid-fidelity programs had greater declines in inpatient service use compared with clients in high- or low-fidelity programs.

Permanent supportive housing programs involve both the provision of housing and the delivery of support services. The results of this study suggest that providing a wide array of services operationalized within a strong team structure may result in increased engagement in community-based services but does not result in sufficiently large reductions in either inpatient admissions or the use of crisis and residential services to potentially offset the cost increases associated with increased use of outpatient mental health services. Although from a primarily financial viewpoint this may create a disincentive to provide higher-fidelity services, we note that these findings do not consider the context of the service delivery system or health and social outcomes. The FSP programs were charged to enroll clients who were unserved or underserved. Higher-fidelity programs may be more successful at meeting these goals and may therefore be viewed as worthy of the additional resources.

In addition, providing more intensive support services may be justified if they result in improved housing and health outcomes, such as increased rates of mental health recovery or improved quality of life, or improved socioeconomic outcomes, such as increased education, employment, or social connectedness. One could interpret these findings as consistent with the evidence-based models of permanent supportive housing that underscore the need for flexible support services that match the variable needs of residents (13).

The fact that high-, medium-, and low-fidelity programs did not differ in terms of employment and volunteer opportunities shows that the variation in practices was not exhaustive. It may be that more intensive employment or volunteer services could have a profound effect on a population that experiences high rates of unemployment (30,31), which could result in lower inpatient admission and use of crisis services. Future research should consider different types of support services in addition to the intensity or quality of those services, as well as explore the role of client choice in determining service intensity.

This study had several limitations. Fidelity was measured with a self-administered survey. This approach offers an expeditious way of obtaining information on a critical array of practices across a wide range of programs but lacks some depth and detail in measurement compared with site visits. Participation in the survey was voluntary, and not all FSPs participated. We did not have data on several types of services, including state hospitals, residential and outpatient substance abuse treatment programs, general medical outpatient services, medication treatment, or mental health services provided in the justice system. We also did not have measures of mental health recovery, quality of life, or emotional health, such as anxiety, stress, confusion, or depression; a previous study of FSPs in San Diego County showed that participation in the FSP was associated with increases in several common dimensions of quality of life (4).

The effectiveness of permanent supportive housing lies in both the provision of housing and the delivery of support services. Permanent supportive housing programs have been shown to vary in their implementation in these areas, with implications for residential outcomes and service use. The cost-effectiveness of these programs may depend, in part, on program practices, including programs’ emphasis on outreach and engagement. Additional data on mental health recovery and quality of life are needed in order to develop better guidelines for programs about the optimal configuration and intensity of support services.

Conclusions

This study suggests that higher-fidelity permanent supportive housing programs are more effective than lower-fidelity programs at increasing outpatient service utilization as well as outreach to and engagement of clients who are not appropriately served by the public mental health system. Administrators and policy makers who place a high value on reaching clients who were unserved or underserved may want to consider implementing policies and procedures that encourage and support the development of high-fidelity programs. Additional research is necessary to determine the extent to which fidelity to the Housing First model is associated with improvements in mental health recovery and quality of life.