The American Psychiatric Association (APA) has updated its Privacy Policy and Terms of Use, including with new information specifically addressed to individuals in the European Economic Area. As described in the Privacy Policy and Terms of Use, this website utilizes cookies, including for the purpose of offering an optimal online experience and services tailored to your preferences.

Please read the entire Privacy Policy and Terms of Use. By closing this message, browsing this website, continuing the navigation, or otherwise continuing to use the APA's websites, you confirm that you understand and accept the terms of the Privacy Policy and Terms of Use, including the utilization of cookies.

×
ArticlesFull Access

Housing Programs for Homeless Individuals With Mental Illness: Effects on Housing and Mental Health Outcomes

Published Online:https://doi.org/10.1176/appi.ps.201400294

Abstract

Objective:

This systematic review analyzed the best available research in the United States on permanent supportive housing programs for homeless individuals with mental illness and the effect of these programs on housing status and mental health. It updates older and broader reviews that included weaker studies or those that did not analyze permanent housing as an input and housing and mental health as primary outcomes.

Methods:

The literature search (1980–2013) yielded 14 studies (randomized controlled trials and quasi-experimental studies).

Results:

The studies found that a majority of participants placed in experimental housing programs with case management support remained in housing for at least one year or experienced more days housed than homeless relative to a comparison group. Although this finding is in line with previous literature reviews on permanent supportive housing, this analysis found limitations in each of the 14 reviewed studies, such as attrition, selection and response bias, imprecise definitions and implementation of housing programs, and a lack of appropriate controls. Only three of the reviewed studies reported using a housing fidelity assessment tool to test whether the housing intervention was faithful to theoretical standards, and conceptions and implementation of housing varied widely across studies, threatening internal and external validity.

Conclusions:

Pitfalls in the best available studies on permanent supportive housing programs in the United States limit the ability of research to inform the policy goal of ending chronic homelessness and demonstrate a need for further experimental research upon which to make funding and policy decisions, especially in light of prioritized federal funds.

Homelessness among individuals with severe mental illness has been called “the most pervasive manifestation of the failure of public policy” (1). The U.S. Department of Housing and Urban Development (HUD) estimated that 610,042 people were homeless on a single night in January 2013. More than seven million poor Americans—among the more than 45 million Americans living in poverty—are at risk by living “doubled up” with friends or family (2,3). The rate of mental illness in the homeless population is higher than that in the general population (4). A 2008 meta-analysis found that in the homeless populations of western countries rates of psychosis ranged from 3% to 42%; the upper range for alcohol or drug dependence was as high as 50% (5).

In recent years, federal officials and treatment experts have touted permanent supportive housing as the solution to ending chronic homelessness among individuals with severe mental illness (6). Permanent supportive housing is loosely defined as subsidized housing coupled with supportive services. It has been implemented in a variety of ways, from scatter-site apartments visited by case managers to clustered housing with onsite staff.

No standardized model for housing homeless individuals with mental illness exists to guide research or policy (1,7). Supportive housing programs vary widely, in part because of financial and infrastructure constraints, such as a lack of affordable housing, and philosophical differences that have persisted for decades about how best to house and treat this population (1,8). Housing studies involving this population are plagued by vague or conflicting definitions and conceptualizations of supported housing, hampering comparative analysis (9,10). Despite such limitations, permanent supportive housing has overtaken so-called “transitional” or continuum of care (CoC) housing programs as the preferred public policy strategy for housing individuals who are chronically homeless (11). As the primary funding source for homeless services nationwide, HUD requested that communities applying for CoC grants during the 2013–2014 funding cycle prioritize individuals who were chronically homeless for permanent supportive housing that is based on a Housing First approach, because research has shown that this model is more effective and less costly than other approaches (12).

In CoC or transitional housing, progress from short-term housing into independent living in permanent supportive housing is predicated on participation in mandatory psychiatric treatment and sobriety goals established by service providers (13). The only major codified difference between transitional and permanent supportive housing concerns timing: HUD-funded transitional housing is limited to 24 months, whereas permanent supportive housing has no limitation.

Although practical differences may be more difficult to determine, philosophical differences between transitional housing and the Housing First approach run deep. Unlike transitional housing programs, Housing First asserts a right to housing without a requirement that homeless individuals take medication, receive psychiatric treatment, or abstain from using drugs and alcohol as a condition of receiving housing—requirements widely criticized as ineffective by homeless individuals and service providers (13,14). In announcing the agency’s decision to prioritize funding for Housing First programs, HUD in 2013 defined the model as “housing assistance that is offered without preconditions (such as sobriety or a minimum income threshold) or service participation requirements” with “rapid placement and stabilization in permanent housing” as primary goals (12). Permanent supportive housing programs do not necessarily operate with the degree of tenant autonomy ascribed to the Housing First model pioneered by Pathways to Housing in New York, in part because of disagreement over how much independence is appropriate for this population (10). Still, policy makers have partly attributed double-digit declines in chronic homelessness in recent years to the success of permanent supportive housing, especially Housing First–type programs (11,15).

The policy emphasis on permanent supportive housing is not based on a large or definitive base of research and has been heavily influenced by political and social considerations (1). The landmark McKinney-Vento Homeless Assistance Act of 1987 included funding for permanent supportive housing and largely predated experimental research on such programs, which employ a variety of housing and supports (1,10).

Although positive research outcomes helped build national support for the Housing First model, it also gained legitimacy because researchers and policy makers framed chronic homelessness as an economic problem with a market-based solution. By defining the problem as an affliction among individuals with mental illness who are frequent and expensive users of public services, advocates spread the message that Housing First could address a public eyesore while saving communities money (16). Advocates claim that housing is cheaper than emergency care, although experimental research has yielded modest savings among programs that do not cover housing costs or among programs that inflate estimated savings in public service expenses by calculating average costs rather than marginal costs (1719). A recent quasi-experimental study in California found that mental health costs and outpatient visits increased among participants in permanent supportive housing programs, which may have resulted from increased contact with service providers (20).

Several literature reviews have analyzed research on the effects of housing programs on homelessness (1,8,2127). These broader reviews include methodologically weaker studies and studies that analyzed inputs other than permanent housing. This article updates these reviews by summarizing findings from the most recent and credible literature and restricting results to randomized controlled trials and quasi-experimental studies in the United States that analyzed the effect of supportive housing programs on housing or mental health outcomes.

Methods

This analysis reviewed results of the best available research in the United States on permanent housing programs for homeless individuals with mental illness and the effect of these programs on treatment outcomes, including housing status and mental health. The review was limited to studies in which permanent housing was offered as an intervention component and research input, including subsidized housing offered through housing vouchers. A search of indexed literature from January 1980 through December 2013 using the terms “homeless,” “experimental,” “randomized,” “housing,” “supported housing,” and “supportive housing” was conducted, limiting findings to randomized controlled trials and quasi-experimental studies. These searches were cross-referenced against multiple databases, including PsycINFO and Thomson Reuters Web of Knowledge. The search yielded 12 primary studies (7,8,13,17,2835), and two secondary analyses (36,37), involving more than 7,400 homeless participants with mental illness.

The review excluded research that did not include permanent housing, such as studies of transitional housing, inpatient treatment of substance use disorders, respite care, day treatment, case management, drop-in centers, and outreach programs. Also excluded were studies comparing housing programs that were too similar to warrant analysis of differences in housing outcomes and those examining the contribution of housing to outcomes other than housing and mental health, such as cost-effectiveness, hospitalization, and social service use. Studies that examined housing as an output, such as the effect of substance use disorders or mental illness on housing stability, were excluded, as well as studies involving homeless individuals who were not mentally ill or individuals with mental illness who were not homeless. This body of research thwarts a meta-analysis because it lacks sufficient data to compute effect sizes and because it adopts incompatible interventions or weak methodologies that make comparative analysis difficult. Findings are analyzed in narrative form.

Results

The studies found that a majority of participants placed in experimental housing programs with case management support remained in housing for at least one year or experienced more days housed than homeless relative to a comparison group. This finding is in line with literature reviews concluding that most homeless individuals with mental illness stay in housing regardless of housing program type (24), although results are mixed when programs are compared with each other (1,25).

Twelve of the 14 studies reported housing-related outcomes, and 11 of the 12 reported statistically significant results supporting a hypothesis that the preferred housing intervention outperformed a control condition (7,8,13,17,2935). The remaining study set out to settle the debate between the Housing First and CoC models and did not articulate a hypothesis to be tested (28). It found no difference in housing tenure between experimental and control conditions.

Of the 11 studies with statistically significant results supporting a hypothesis in favor of a preferred housing intervention, two did not guarantee housing for participants in the experimental group, thwarting comparisons to the majority of studies in this review that did guarantee housing (7,29). When interventions were compared with control conditions, the intervention in one of these studies did not reduce homelessness (29). In a third study, the intervention did not reduce homelessness among individuals with low to moderate psychiatric and substance use disorder symptoms (30). A fourth study employed experimental and control conditions that were so difficult to distinguish as to yield results of limited practical value for policy makers (8). The remaining seven studies showed significant differences in housing stability favoring experimental over control conditions, but, like the others, they suffered from methodological challenges (13,17,3135).

The studies defined housing outcomes in various ways, limiting external validity. Seven reported as a primary outcome the percentage of participants housed at the end of the study period, although some also measured other housing indicators, such as length of time to achieve stable housing (8,13,28,29,3335). Three studies reported the proportion of time spent in stable housing versus homeless (3032). Two studies reported the number of days in which participants were housed versus homeless (7,17). Studies used a variety of methods to obtain data on housing status.

Seven of the 14 studies reported mixed clinical and substance use outcomes (7,8,3133,36,37). One of these studies found support for a hypothesis that the experimental housing condition outperformed a comparison or control condition by reducing psychiatric symptoms (8). Another study reported that the experimental housing condition was associated with a reduction in substance use (36). Another found no difference in substance use between persons in experimental and control conditions because substance use declined in both conditions (37). Four studies found that the preferred housing condition did not yield any advantage in clinical outcomes over the comparison or control condition, either because no improvements were found (7,32,33) or because both experimental and comparison groups showed similar gains (31). Most of the studies used self-report instruments to determine psychiatric diagnoses and substance use at baseline.

Two of the 14 studies also analyzed case management as a separate input from housing (7,29). Both found that more intensive case management had no effect on housing; neither study could support a hypothesis that higher levels of case management support would result in improved housing outcomes. Eight of the 14 studies analyzed well-being or other self-reported psychological states as an outcome in addition to housing or clinical outcomes (7,8,13,17,28,30,31,33). These outcomes, obtained by using a variety of self-report tools, also were mixed.

This analysis of interventions and outcomes found limitations in each of the 14 studies, such as attrition, selection and response bias, imprecise definitions and implementation of housing programs, and a lack of appropriate controls. These pitfalls limit the ability of the research to inform the policy goal of ending homelessness and demonstrate the need for further experimental research to inform funding and policy decisions. The body of research is unable to answer fundamental questions about what type of housing program works best for homeless individuals with mental illness.

Discussion

Sampling and Selection Bias

All but one (33) of the 14 studies had original sample sizes of more than 100 individuals, with a range from 49 to 3,811 and a mean of 613 (Table 1). Results were skewed by two quasi-experimental studies that used large, unmatched control groups (13,34). Excluding these studies, the range of participants was 411 and the mean was 217. All of the reviewed studies followed participants for at least one year. Four studies followed participants for up to three years (7,13,29,35), and one followed participants for five years (34).

TABLE 1. Information about 14 studies of outcomes of supported housing among homeless persons with mental illnessa

StudySample sizebMaleAge (years)Psychotic disordersSubstance useExperimental vs. control group outcome
HousingClinicalCase management
Lipton et al., 1988 (33)49Experimental, unknown; control, not reportedExperimental, unknown; control, not reportedExperimental, unknown; control, not reportedExperimental, unknown; control, not reportedMore housed by end of studyHousing not associated with reduction in psychiatric symptoms
Goldfinger et al., 1999 (35)118Experimental, 72%; control, 41%M=3862%50%More housed by end of study
McHugo et al., 2004 (8)121Experimental, 8%; control, 83%M=4073%31%–54%cMore housed by end of studyContinuum housing associated with reduction in psychiatric symptoms; no change in substance use across groupsd
Siegel et al., 2006 (28)e139Experimental, 57%–72%; control, 26%–54%35–48Experimental, unknown; control, not reportedExperimental, unknown; control, not reportedSimilar number housed by end of study
Clark and Rich, 2003 (30)152Experimental, 52%; control, 20%M=38Experimental, unknown; control, not reportedExperimental, unknown; control, not reportedGreater number with severe psychiatric symptoms housed; fewer with less severe psychiatric symptoms housed
Padgett et al., 2006 (37)173Experimental, 77%; control, 40%M=4290%No difference in substance use
Greenwood et al., 2005 (31)197Experimental, 76%; control, 29%M=4254%Experimental, unknown; control, not reportedGreater proportion of time housedSimilar reduction in psychiatric symptoms across groups
Tsemberis et al., 2004 (32)206Experimental, 79%; control, 41%59% were between ages 31 and 50 53%90%Greater proportion of time housedHousing not associated with reduction in psychiatric symptoms or substance use
Hurlburt et al., 1996 (29)361Experimental, 67%; control, 63%42% were between ages 30 and 3955%53%More living in independent housing, but experimental condition did not reduce homelessnessCase management not associated with positive housing outcome
Gilmer et al., 2010 (17)363Experimental, 56%; control, 25%M=4460%Experimental, unknown; control, not reportedFewer days homeless and more days housed
Rosenheck et al., 2003 (7)460Experimental, 96%; control, not reportedM=42Experimental, unknown; control, not reported50%More days housedHousing not associated with reduction in psychiatric symptomsCase management not associated with positive housing outcome
Cheng et al., 2007 (36)460Experimental, 96%; control, not reportedM=42Experimental, unknown; control, not reported50%More days housedHousing associated with reduction in substance use
Tsemberis and Eisenberg, 2000 (34)f1,841Experimental, 67%; control, 73%M=41Experimental, 60%; control, 46%Experimental, 58%; control, 49%More housed by end of study
Tsemberis, 1999 (13)g3,950Experimental, 74%; control, 66%Experimental, 58% were between ages 30 and 49; control, 69% were between ages 30 and 49Experimental, 59%; control, 55%Experimental, 62%; control, 49%More housed by end of study

aDemographic data reflect baseline characteristics.

bMean sample size, N=584. The mean excludes the two studies that were secondary analyses (36,37).

cAggregates four groups of data, by group condition and by alcohol and illicit drug use

dThe study did not hypothesize an outcome but compared two experimental groups, one adhering more to an independent housing model and the other to the continuum of care model.

eTo mimic randomization, the study divided participant data into strata on the basis of the probability that each individual would need a higher intensity of supported housing. The study reported demographic information by strata, resulting in a range of values for each variable.

fExperimental group, N=241; nonrandomized, unmatched comparison group, N=1,600

gExperimental group, N=139; nonrandomized, unmatched comparison group, N=3,811

TABLE 1. Information about 14 studies of outcomes of supported housing among homeless persons with mental illnessa

Enlarge table

Study participants were mostly male, with mean ages in the late 30s to mid-40s. Characteristics such as race-ethnicity, psychiatric diagnosis, and substance use differed across studies, although a majority of participants in most studies had psychotic disorders and a co-occurring substance use disorder. All study participants were homeless or at immediate risk of homelessness and had a serious mental illness, such as major depression, bipolar disorder, or schizophrenia, at baseline. Each study used different criteria for homelessness to establish eligibility, ranging from self-reported information of less than one month spent homeless to many months. Studies either used clinical records or interviewed participants with diagnostic tools to establish mental illness.

Thirteen of the 14 studies used a screening process to exclude participants from the study (7,8,13,2837), although most contained few details on the process. The two studies that used Section 8 vouchers to house participants screened out individuals with histories of violence and criminal records (7,29), while another excluded those believed to be a danger to self or others (35). Two reported the number of individuals screened for eligibility relative to participants who gave informed consent (7 Three reported eligibility criteria for experimental-housing participants, with less detail on control groups (31,32,37). One study claimed not to screen out any participants from the study sample (17).

The studies obtained research participants from a mix of referral sources, threatening the external validity of housing and clinical outcomes. Some participants were recruited from a single source or type of source, such as one hospital, multiple hospitals, and multiple shelters, or from a combination of sources, including hospitals, shelters, and streets. Some studies relied on social services agencies to recruit participants who were not already receiving services, and other studies recruited clients or outreach workers aligned with existing programs. Such procedures may have positively influenced outcomes if referring sources referred individuals with less severe disabilities. Individuals discharged from hospitals and stabilized on medication, as well as those receiving services in shelters, may have better outcomes in housing than those placed directly from the streets (33,35). Two studies reported directing service providers to refer individuals who were capable of participating in housing programs or who were willing participants (29,32).

Attrition

Researchers have used various methods of calculating and reporting attrition, making comparisons difficult (38). The 12 primary studies, excluding the secondary analyses (36,37) reported attrition in a variety of ways (7,8,13,17,2835). Seven studies reported attrition as the percentage of participants housed at the end of the study period (8,13,28,29,3335), two reported attrition as the percentage of participants completing follow-up interviews (7,30), two reported attrition as the proportion of time spent homeless (31,32), and one reported days spent homeless (17).

Table 2 includes the percentage data from the nine studies that either analyzed the percentage of participants housed at the end of the study period or those completing follow-up interviews (7,8,13,2830,3335), excluding two studies that reported attrition as the proportion of time spent housed versus homeless (31,32) and excluding the secondary analyses (36,37). Results from one study were averaged with results from a secondary analysis that reported original outcomes that were slightly different from the outcomes of the primary study (29). Attrition rates were averaged for a third study in which attrition rates were reported as a range of percentage values (28). For a study with an experimental group and two control groups, data from the group that received case management only, which received more intensive services than the “standard care” group, were included (7).

TABLE 2. Attrition in studies of outcomes of supported housing among homeless persons with mental illnessa

Attrition rate and studyStudy periodExperimental group or group hypothesized to outperform comparison group (%)bControl group or group hypothesized to underperform experimental group (%)cAverage attrition across experimental and control groups
0%–29% (medium or high statistical power)
 McHugo et al., 2004 (8)18 months153224
 Goldfinger et al., 1999 (35)3 years232424
 Tsemberis, 1999 (13)3 years164028
30%–49% (low or medium statistical power)
 Tsemberis and Eisenberg, 2000 (34)5 years125333
 Siegel et al., 2006 (28)18 months284637
 Rosenheck et al., 2003 (7)3 years235238
 Clark and Rich, 2003 (30)1 year246444
≥50% (very low statistical power)
 Lipton et al., 1988 (33)1 year317051
 Hurlburt et al., 1996 (29)3 years427056

aAttrition was measured either as the percentage of participants who were housed by the end of the study (8,13,28,29,3335) or as the percentage who completed follow-up interviews (7,30). Studies that measured homelessness as the proportion of time spent homeless (31,32) or days spent homeless (17) are not included in this table.

bMean=24%

cMean=50%

TABLE 2. Attrition in studies of outcomes of supported housing among homeless persons with mental illnessa

Enlarge table

Six of the nine studies had mean attrition rates of more than 30% across experimental and control groups (7,2830,33,34). These rates are similar to previously reported attrition rates involving homelessness research (30), but they are large enough to threaten statistical power. Attrition is common in longitudinal research on homeless individuals with mental illness (39). High attrition rates lower statistical power by reducing sample sizes and threatening internal and external validity (38). Large numbers of dropouts make it more difficult to attribute outcomes to the intervention and raise the question of whether differences are attributable to underlying differences between groups, such as the relative severity of mental or general medical illness.

All of the studies reporting attrition included dropouts in their statistical calculations of attrition rates and housing outcomes. Most of the studies used various statistical methods to correct for potential bias caused by attrition, including inputting missing data on the basis of assumed values, weighting adjustments for nonrespondents, and analyzing relationships between baseline scores and participant characteristics for those who stayed in the study and those who dropped out. Such methods can smooth out data points, which can suggest a lack of potential bias, such as a lack of association between dropouts and high levels of psychiatric symptoms that might indicate that attrition was not influenced by mental illness. Statistical corrections rely on untestable assumptions about chance events that if wrong could lead to inaccurate or misleading conclusions (40). Such methods do not protect against the potential risk that dropouts did not exit at random, were doing less well than their participating counterparts, or exited because of some program feature.

Only four of the 12 original studies reported the whereabouts of study participants during the program, at the end of the study period, or thereafter (17,31,33,35). None of the studies had specific explanations for why participants dropped out of the study or what precipitated their exit from housing, although some reported dropouts as “lost,” refusing follow-up interviews, leaving against staff advice, or not wanting to continue the program. Lack of detail on the reasons for attrition or unavailability for interviews is unsatisfactory for empirical research (38).

Response Bias

The studies determined housing status by interviewing study participants, although some of the studies attempted confirmation with management information systems or visits by case managers or research staff. Self-report interviews, especially of transient individuals with mental illness and substance use disorders, are subject to response error and misunderstanding even when formal tools are used (41).

Five of the 14 studies used a tested assessment tool developed by the Dartmouth Psychiatric Research Center called the Residential Time-Line Follow-Back Inventory to determine homelessness and residential status, although most of the reviewed studies used a variety of instruments (8,28,3032). All but one (17) of the 14 studies reported the use of various self-report tools to determine mental health status or substance use. Studies involving homeless individuals with both mental illness and substance use disorders have shown a high rate of discrepancy between self-reported information on clinical status and observed data (41,42). In the absence of toxicology tests to determine substance use, the reliability of self-report information is questionable given the presence of mental illness and the risk of recall error, refusal to disclose information, intentional misrepresentation, or the provision of socially acceptable responses in the presence of authority figures.

Five of the 12 primary studies reported paying study participants for completing interviews about housing or mental health status (7,2931,35). Although research compensation is common, paying individuals for participation may have biased results by encouraging socially acceptable answers or increasing greater participation in housing programs than would otherwise have been the case. Research reactivity may also have biased outcomes. The informed consent process prevents participants from being blinded to study conditions and, by extension, to broader goals and policy implications.

The reviewed studies may have been subject to demonstration effects because self-aware study participants may have performed better or responded more favorably to interview questions. Only one of the studies explicitly addressed this concern (35). The federal funds that financed many of the reviewed studies may also have contributed to more robust services and more motivated workers than would exist under nonresearch conditions.

Design and Implementation of Housing and Support Services

The reviewed studies defined, designed, and implemented supportive housing in a variety of ways. Most of the studies did not specifically refer to their experimental housing conditions as “supportive housing,” instead using a variety of terms, such as Housing First (13,31,32,34,37), comprehensive housing (30), evolving consumer households (35), integrated housing (8), and full-service partnerships (17).

Twelve of the 14 studies lacked detail or offered vague descriptions of experimental or control conditions (7,8,13,17,2934,36,37); only two of the 14 studies reported sufficient detail to enable comparative analysis (28,35). All but one (33) of the 14 studies used regression analyses or other statistical methods to analyze associations between variables that may have affected housing or clinical outcomes. None of the studies could pinpoint what drove positive outcomes, however. Imprecise definitions and implementation of housing programs limit the usefulness of statistical analysis and threaten internal and external validity because individual variables, such as substance use, can become subject to influence by many other mediating variables that are not controlled for or excluded in experimental conditions.

Only three of the 14 studies reported using a housing fidelity assessment tool to test whether the housing intervention was faithful to theoretical standards (8,28,30). Few studies on supported housing have incorporated fidelity assessments, which are important in determining whether interventions are implemented as planned and whether outcomes can be attributed to the interventions as implemented (10,23). Research on supportive housing has suffered from vague housing models and lack of fidelity to these models (10). For example, the Pathways studies described the principles, policies, and procedures of their Housing First program but did not include a quantitative assessment of whether the model was consistently implemented (13,31,32,34,37). In studies that incorporated fidelity instruments, ratings showed that experimental programs were mostly faithful to their program models, strengthening internal validity. External validity was limited, however, because conceptions and implementation of housing varied widely across studies.

All of the studies included case management as part of their experimental housing programs, reflecting a long-standing view that case management is essential to helping individuals with mental illness live stably in the community (4345). The studies generally included more information on the case management components of their programs, such as staffing ratios and time spent with clients, than on the housing settings. They fell short in terms of detailing how case management was implemented across study conditions and especially in describing differences in the quantity, variety, intensity, and quality of services in control versus experimental groups.

Three of the 12 primary studies contained information on case management staffing ratios (7,8,29), although only one of the three included ratios for both experimental and control groups (29). The three studies used various staffing ratios to quantify higher-intensity case management services, making comparisons of care standards difficult. Case management programs were not otherwise quantified.

The studies defined and implemented case management in a variety of ways. Seven studies used ACT teams (13,17,28,31,32,34,37) and others used services that included “intensive case management” (7,8,36), “intensive clinical case management” (35), and “comprehensive case management” alongside “traditional case management” (29). Two studies reported applying fidelity assessment tools that were either homegrown (30) or based on the Dartmouth Assertive Community Treatment Fidelity Scale (8); one study employed a “full fidelity” assertive community treatment team that met at least monthly with Housing First participants (17).

Six of the 12 primary studies stated how frequently case managers met with participants, although confirmation methods were unclear. Visits ranged from at least one per week (7,28,35) to at least two times per month (13,31,34). Five of the 14 studies analyzed the degree to which participants used services, and the results varied across studies as experimental groups used more or less than control groups (7,8,17,32,37).

How programs handled drug and alcohol use among participants also was unclear because only two studies elaborated on crisis response procedures (13,28). One Pathways study explained that tenants who relapsed did not lose housing and that case managers intensified their work with tenants to manage the problem (13). In the other study, supportive housing tenants were prescreened for evidence of six months of clean and sober behavior and could be asked to leave if they were disruptive (28). Lack of information raises questions about how Housing First programs are implemented, especially given rules prohibiting drug use in federally subsidized housing.

Randomization and Control Group Design

None of the reviewed studies included a no-treatment control group that lacked housing or other forms of assistance. Eight of the 14 studies compared two types of housing programs with each other, placing participants in some form of guaranteed housing (8,13,28,31,32,34,35,37); these studies suffered from imprecise descriptions of the housing interventions. Without control groups that lacked a housing component, it is difficult to estimate the degree to which housing contributed to reductions in homelessness or psychiatric symptoms.

Three studies contained an experimental condition in which priority access to Section 8 vouchers, but not housing itself, was guaranteed (7,29,36). These studies contained multiple control conditions that varied the intensity of case management assistance but did not guarantee housing. The three remaining studies compared a program that offered guaranteed housing with a condition in which participants were not offered guaranteed housing; instead of guaranteed housing participants were offered “active outreach and engagement, some onsite counseling, medication and medication management, assistance obtaining housing and linkages to other psychosocial services” (30), “routine discharge planning” from a hospital (33), or outpatient public mental health services (17).

The six studies that did not guarantee housing for all participants lacked detail on standard care control conditions (7,17,29,30,33,36). Outcome relevance is limited without more information about what “usual care” entails and whether it resulted in housing or temporary shelter for participants or no shelter at all. Ambiguous distinctions between experimental and comparison groups in housing and case management assistance limit overall credibility.

Five of the 14 studies were quasi-experimental in that they did not randomly assign participants into experimental and control conditions (13,17,28,30,34). Quasi-experimental studies are problematic in that outcomes may be result from differences between groups and unmeasured variables rather than to the interventions. “Readiness to change,” for example, could be one of several confounding variables influencing outcomes (17). Four of the five studies had experimental and comparison groups that were significantly different in several categories including gender, ethnicity, substance use, clinical diagnosis, and history of homelessness (13,28,30,34). Three of the five quasi-experimental studies used propensity scoring to mimic randomization and minimize bias (17,28,30). One assigned propensity scores to participants to match a control group against an experimental group along demographic indicators, excluding an unknown number of participants (17). None of the studies reported selecting research participants at random.

One Pathways study attempted to minimize bias by establishing a comparison group of participants referred from sources similar to the experimental Housing First group (34). Another Pathways study did not report taking such steps, although the author stated that bias was minimized because the experimental group was more disabled than the control group at baseline (13).

Two quasi-experimental studies used data obtained from two different sources, resulting in significant differences in characteristics between groups and in control groups that were more than six times (34) and more than 20 times (13) larger than experimental Housing First groups (Table 1).

Conclusions

This review of the best studies on permanent supportive housing identified a small base of research with limited usefulness for decision makers seeking empirical evidence to justify policy choices. The research cannot yet pinpoint which factors drive positive housing and clinical outcomes. Research problems involving attrition, lack of detail on housing conditions and supports, selection bias, and lack of standardized program models and definitions limit internal validity, the ability to generalize findings, and efforts to replicate research conditions. With the possible exception of the Pathways studies, the reviewed studies are unique to their environments.

Supportive housing research relies on external and limited funding that may lead to program modifications (10). Many supportive housing studies do not employ randomization because of ethical concerns about assigning vulnerable individuals to control groups that withhold services (8,24). The At Home/Chez Soi demonstration project is the world’s largest Housing First randomized controlled trial, involving approximately 2,500 participants in five Canadian cities; it reported significant opposition to randomization, among other challenges (4649). Ongoing results from this study are expected to make a significant contribution to the literature. Given the scope of services and participant needs subjected to study, including multiple degrees of disability across a broad demographic sample, results from the research may address many of the concerns raised in this article. Results should be addressed in the context of Canadian policy priorities and a more generous safety net than exists in the United States. At present, the best research—lacking a verifiable standard for the quality of housing and supports against which to test program variations—fails to detail the specific therapeutic benefits of supported housing beyond having a roof overhead.

Problems defining and conceptualizing variables may reflect overly broad applications of services such as case management, a treatment concept “used so broadly as to have no specific meaning at all” (50). The two studies measuring case management as an input independent of housing found that the more service-intensive condition did not influence housing outcomes, raising questions for future research about the therapeutic benefits of case management (7,29). Moreover, the studies in this review did not control for placebo affects arising from expected benefits. Assertive community treatment teams and intensive case management models have been modified over time, with no single standard of care for homeless individuals with mental illness (51). Casework is subject to variation with regard to house rules, crisis management, and treatment participation. Services may vary despite attempts to standardize them across experimental and control groups (7). Case managers are not disinterested observers and relate to clients on the basis of personality type, moral imperative, and institutional policy.

Recent fidelity models have identified similar features, such as immediate placement in housing and the lack of a mandate for treatment participation, as key factors distinguishing the Housing First formula from other programs (5254). Faithful implementation remains in doubt, especially in light of provider preferences for treatment mandates and sober living programs (54). A qualitative study of permanent housing programs in California found significant variation in fidelity to the Housing First model, driven largely by the priorities and values of program directors (55). Future research could be strengthened by use of fidelity models that encompass every stage of the treatment process, from outreach and screening to follow-up care, as well as a range of supportive housing types with various degrees of psychiatric and case management involvement. Unraveling the effects of housing and support services presents a methodological challenge likely requiring multiple experimental and control groups (25).

In contrast to the reviewed research, recent qualitative studies have documented in detail how Housing First programs are implemented in practice, such as particular casework approaches to assisting substance users after housing placement (6,5660). These studies generally conclude that “harm reduction” policies incorporating flexible housing rules improve housing stability. In the best available experimental research, fidelity to the supportive housing model remains a concern, as well as a challenge to the implementation and evaluation of standardized treatment methods.

Most of the reviewed studies did not analyze the effect of housing on mental health outcomes, whereas those that did were inconclusive. The chronic yet fluctuating nature of mental illness makes clinical outcomes difficult to interpret, because study participants may experience “regression to the mean” (8). More exploration of how housing and casework affect mental health is needed to analyze program effectiveness.

As service providers face a HUD mandate to end chronic homelessness by 2015, questions persist about the type of housing this population needs. The economic justification needed to fulfill society’s moral obligation to house homeless individuals with mental illness should be built on rigorous experimental research, which is expensive and time consuming but necessary to answer fundamental questions about efficacy.

Ms. Benston is with the State of Nevada Department of Health and Human Services, Las Vegas (e-mail: ).

The opinions and conclusions in this article are solely those of the author and not necessarily those of her employer.

The author reports no financial relationships with commercial interests.

References

1 Newman S, Goldman H: Housing policy for persons with severe mental illness. Policy Studies Journal 37:299–324, 2009CrossrefGoogle Scholar

2 The State of Homelessness in America. Washington, DC, National Alliance to End Homelessness, 2014Google Scholar

3 DeNavas-Walt C, Proctor BD: Income and Poverty in the United States: 2013. Current Population Reports, P60-24. Washington, DC, US Census Bureau, 2014Google Scholar

4 Montgomery AE, Metraux S, Culhane, D: Rethinking homelessness prevention among persons with serious mental illness. Social Issues and Policy Review 7:58-82, 2013CrossrefGoogle Scholar

5 Fazel S, Khosla V, Doll H, et al.: The prevalence of mental disorders among the homeless in western countries: systematic review and meta-regression analysis. PLoS Medicine 5:e225, 2008Crossref, MedlineGoogle Scholar

6 Pearson CL, Locke G, McDonald WR: The Applicability of Housing First Models to Homeless Persons With Serious Mental Illness. Washington, DC, US Department of Housing and Urban Development Office of Policy Development and Research, 2007Google Scholar

7 Rosenheck R, Kasprow W, Frisman L, et al.: Cost-effectiveness of supported housing for homeless persons with mental illness. Archives of General Psychiatry 60:940–951, 2003Crossref, MedlineGoogle Scholar

8 McHugo GJ, Bebout RR, Harris M, et al.: A randomized controlled trial of integrated versus parallel housing services for homeless adults with severe mental illness. Schizophrenia Bulletin 30:969–982, 2004Crossref, MedlineGoogle Scholar

9 Fakhoury WKH, Murray A, Shepherd G, et al.: Research in supported housing. Social Psychiatry and Psychiatric Epidemiology 37:301–315, 2002Crossref, MedlineGoogle Scholar

10 Tabol C, Drebing C, Rosenheck R: Studies of “supported” and “supportive” housing: a comprehensive review of model descriptions and measurement. Evaluation and Program Planning 33:446–456, 2010Crossref, MedlineGoogle Scholar

11 Opening Doors: Federal Strategic Plan to Prevent and End Homelessness. Update 2013. Washington, DC, US Interagency Council on Homelessness, 2013. Available at usich.gov/resources/uploads/asset_library/USICH_Annual_Update_2013.pdfGoogle Scholar

12 Notice of Funding Availability (NOFA) for the Fiscal Years 2013 and 2014: Continuum of Care Program Competition. Washington, DC, US Department of Housing and Urban Development, 2013. Available at www.onecpd.info/resources/documents/FY2013-2014CoCProgramNOFA.pdfGoogle Scholar

13 Tsemberis S: From streets to homes: an innovative approach to supported housing for homeless adults with psychiatric disabilities. Journal of Community Psychology 27:225–241, 1999CrossrefGoogle Scholar

14 Ridgway P, Zipple AM: The paradigm shift in residential services: from the linear continuum to supported housing approaches. Psychosocial Rehabilitation Journal 13:11–31, 1990CrossrefGoogle Scholar

15 HUD Reports Continued Decline in US Homelessness Since 2010. Washington, DC, US Department of Housing and Urban Development, 2013. Available at portal.hud.gov/hudportal/HUD?src=/press/press_releases_media_advisories/2013/HUDNo.13-173Google Scholar

16 Stanhope V, Dunn K: The curious case of Housing First: the limits of evidence based policy. International Journal of Law and Psychiatry 34:275–282, 2011MedlineGoogle Scholar

17 Gilmer TP, Stefancic A, Ettner SL, et al.: Effect of full-service partnerships on homelessness, use and costs of mental health services, and quality of life among adults with serious mental illness. Archives of General Psychiatry 67:645–652, 2010Crossref, MedlineGoogle Scholar

18 Culhane DP, Metraux S, Hadley T: Public service reductions associated with placement of homeless persons with severe mental illness in supportive housing. Housing Policy Debate 13:107–163, 2002CrossrefGoogle Scholar

19 Larimer ME, Malone DK, Garner MD, et al.: Health care and public service use and costs before and after provision of housing for chronically homeless persons with severe alcohol problems. JAMA 301:1349–1357, 2009Crossref, MedlineGoogle Scholar

20 Gilmer TP, Stefancic A, Tsemberis S, et al.: Full-service partnerships among adults with serious mental illness in California: impact on utilization and costs. Psychiatric Services 65:1120–1125, 2014LinkGoogle Scholar

21 Fitzpatrick-Lewis D, Ganann R, Krishnaratne S, et al.: Effectiveness of interventions to improve the health and housing status of homeless people: a rapid systematic review. BMC Public Health 11:638, 2011Crossref, MedlineGoogle Scholar

22 Kyle T, Dunn JR: Effects of housing circumstances on health, quality of life and healthcare use for people with severe mental illness: a review. Health and Social Care in the Community 16:1–15, 2008Crossref, MedlineGoogle Scholar

23 Nelson G, Aubry T, Lafrance A: A review of the literature on the effectiveness of housing and support, assertive community treatment, and intensive case management interventions for persons with mental illness who have been homeless. American Journal of Orthopsychiatry 77:350–361, 2007Crossref, MedlineGoogle Scholar

24 Rog DJ: The evidence on supported housing. Psychiatric Rehabilitation Journal 27:334–344, 2004Crossref, MedlineGoogle Scholar

25 Newman SJ: Housing attributes and serious mental illness: implications for research and practice. Psychiatric Services 52:1309–1317, 2001LinkGoogle Scholar

26 Rosenheck R: Cost-effectiveness of services for mentally ill homeless people: the application of research to policy and practice. American Journal of Psychiatry 157:1563–1570, 2000LinkGoogle Scholar

27 Rog DJ, Marshall T, Dougherty RH, et al.: et al.. Psychiatric Services 65:287–294, 2014LinkGoogle Scholar

28 Siegel CE, Samuels J, Tang DI, et al.: Tenant outcomes in supported housing and community residences in New York City. Psychiatric Services 57:982–991, 2006LinkGoogle Scholar

29 Hurlburt MS, Wood PA, Hough RL: Providing independent housing for the homeless mentally ill: a novel approach to evaluating long-term longitudinal housing patterns. Journal of Community Psychology 24:291–310, 1996CrossrefGoogle Scholar

30 Clark C, Rich AR: Outcomes of homeless adults with mental illness in a housing program and in case management only. Psychiatric Services 54:78–83, 2003LinkGoogle Scholar

31 Greenwood RM, Schaefer-McDaniel NJ, Winkel G, et al.: Decreasing psychiatric symptoms by increasing choice in services for adults with histories of homelessness. American Journal of Community Psychology 36:223–238, 2005Crossref, MedlineGoogle Scholar

32 Tsemberis S, Gulcur L, Nakae M: Housing First, consumer choice, and harm reduction for homeless individuals with a dual diagnosis. American Journal of Public Health 94:651–656, 2004Crossref, MedlineGoogle Scholar

33 Lipton FR, Nutt S, Sabatini A: Housing the homeless mentally ill: a longitudinal study of a treatment approach. Hospital and Community Psychiatry 39:40–45, 1988AbstractGoogle Scholar

34 Tsemberis S, Eisenberg RF: Pathways to Housing: supported housing for street-dwelling homeless individuals with psychiatric disabilities. Psychiatric Services 51:487–493, 2000LinkGoogle Scholar

35 Goldfinger SM, Schutt RK, Tolomiczenko GS, et al.: Housing placement and subsequent days homeless among formerly homeless adults with mental illness. Psychiatric Services 50:674–679, 1999LinkGoogle Scholar

36 Cheng AL, Lin H, Kasprow W, et al.: Impact of supported housing on clinical outcomes: analysis of a randomized trial using multiple imputation technique. Journal of Nervous and Mental Disease 195:83–88, 2007Crossref, MedlineGoogle Scholar

37 Padgett DK, Gulcur L, Tsemberis S: Housing First services for people who are homeless with co-occurring serious mental illness and substance abuse. Research on Social Work Practice 16:74–83, 2006CrossrefGoogle Scholar

38 Cohen EH, Mowbray CT, Bybee D, et al.: Tracking and follow-up methods for research on homelessness. Evaluation Review 17:331–352, 1993CrossrefGoogle Scholar

39 Kalton G, Kasprzyk D: The treatment of missing survey data. Survey Methodology 12:1–16, 1996Google Scholar

40 Ribisl KM, Walton MA, Mowbray CT et al.: Minimizing participant attrition in panel studies through the use of effective retention and tracking strategies: review and recommendations. Evaluation and Program Planning 19: 1–25, 1996CrossrefGoogle Scholar

41 Calsyn RJ, Morse GA, Klinkenberg WD, et al.: Reliability and validity of self-report data of homeless mentally ill individuals. Evaluation and Program Planning 20:47–54, 1997CrossrefGoogle Scholar

42 Goldfinger SM, Schutt RK, Seidman LJ, et al.: Self-report and observer measures of substance abuse among homeless mentally ill persons in the cross-section and over time. Journal of Nervous and Mental Disease 184:667–672, 1996Crossref, MedlineGoogle Scholar

43 Stein LI, Test MA: Alternative to mental hospital treatment: I. conceptual model, treatment program, and clinical evaluation. Archives of General Psychiatry 37:392–397, 1980Crossref, MedlineGoogle Scholar

44 Bond GR, Witheridge TF, Dincin J, et al.: Assertive community treatment for frequent users of psychiatric hospitals in a large city: a controlled study. American Journal of Community Psychology 18:865–891, 1990Crossref, MedlineGoogle Scholar

45 Bond GR, McGrew JH, Fekete DM: Assertive outreach for frequent users of psychiatric hospitals: a meta-analysis. Journal of Mental Health Administration 22:4–16, 1995Crossref, MedlineGoogle Scholar

46 Goering PN, Streiner DL, Adair C, et al.: The At Home/Chez Soi trial protocol: a pragmatic, multi-site, randomised controlled trial of a Housing First intervention for homeless individuals with mental illness in five Canadian cities. BMJ Open 1:e000323, 2011Crossref, MedlineGoogle Scholar

47 Hwang SW, Stergiopoulos V, O’Campo P, et al.: Ending homelessness among people with mental illness: the At Home/Chez Soi randomized trial of a Housing First intervention in Toronto. BMC Public Health 12:787, 2012Crossref, MedlineGoogle Scholar

48 Aubry T, Tsemberis S, Adair CE, et al.: One-year outcomes of a randomized controlled trial of Housing First with ACT in five Canadian cities. Psychiatric Services (Epub ahead of print, February 2, 2015)LinkGoogle Scholar

49 Nelson G, Patterson M, Kirst M, et al.: Life changes among homeless persons with mental illness: a longitudinal study of Housing First and usual treatment. Psychiatric Services (Epub ahead of print, February 17, 2015)LinkGoogle Scholar

50 Mechanic D: Correcting misconceptions in mental health policy: strategies for improved care of the seriously mentally ill. Milbank Quarterly 65:203–230, 1987Crossref, MedlineGoogle Scholar

51 Coldwell CM, Bender WS: The effectiveness of assertive community treatment for homeless populations with severe mental illness: a meta-analysis. American Journal of Psychiatry 164:393–399, 2007LinkGoogle Scholar

52 Tsemberis S: Housing First: The Pathways Model to End Homelessness for People With Mental Illness and Addiction. Center City, Minn, Hazelden, 2010Google Scholar

53 Gilmer TP, Stefancic A, Sklar M, et al.: Development and validation of a Housing First fidelity survey. Psychiatric Services 64:911–914, 2013LinkGoogle Scholar

54 Watson DP, Orwat J, Wagner DE, et al.: The Housing First Model (HFM) Fidelity Index: designing and testing a tool for measuring integrity of housing programs that serve active substance abusers. Substance Abuse Treatment, Prevention, and Policy, 2013 (doi 10.1186/1747-597X-8-16 )Crossref, MedlineGoogle Scholar

55 Gilmer TP, Katz ML, Stefancic A, et al.: Variation in the implementation of California’s Full Service Partnerships for persons with serious mental illness. Health Services Research 48:2245–2267, 2013Crossref, MedlineGoogle Scholar

56 Watson DP, Wagner DE, Rivers M: Understanding the critical ingredients for facilitating consumer change in Housing First programming: a case study approach. Journal of Behavioral Health Services and Research 40:169–179, 2013Crossref, MedlineGoogle Scholar

57 Tiderington E, Stanhope V, Henwood BF: A qualitative analysis of case managers’ use of harm reduction in practice. Journal of Substance Abuse Treatment 44:71–77, 2013Crossref, MedlineGoogle Scholar

58 Collins SE, Clifasefi SL, Dana EA, et al.: Where harm reduction meets Housing First: exploring alcohol’s role in a project-based housing first setting. International Journal on Drug Policy 23:111–119, 2012Crossref, MedlineGoogle Scholar

59 Henwood BF, Stanhope V, Padgett DK: The role of housing: a comparison of front-line provider views in Housing First and traditional programs. Administration and Policy in Mental Health and Mental Health Services Research 38:77–85, 2011Crossref, MedlineGoogle Scholar

60 Henwood BF, Padgett DK, Tiderington E: Provider views of harm reduction versus abstinence policies within homeless services for dually diagnosed adults. Journal of Behavioral Health Services and Research 41:80–89, 2014Crossref, MedlineGoogle Scholar