A Multiple-City RCT of Housing First With Assertive Community Treatment for Homeless Canadians With Serious Mental Illness

Urban homelessness has long been a seemingly intractable social problem in Western countries (1,2). Among the homeless population, a substantial proportion has been found to have a serious mental illness (2–5). The most common interventions for people with a serious mental illness experiencing homelessness target stabilizing functioning before moving into regular housing. Unfortunately, this approach has had limited success (6–9). In the 1990s, Housing First emerged as an alternative response to homelessness (10). One variant of this approach, the “Pathways to Housing” Housing First model, provides participants with scattered-site apartments along with assertive community treatment (ACT) or intensive case management (ICM) (11,12).

To date, two randomized controlled trials (RCTs) and a small number of quasi-experimental studies have examined the effectiveness of Housing First. A recent review of research on Housing First concluded that there was a “moderate level” of evidence of its effectiveness (8). This research has found that Housing First with ACT decreases homelessness, increases housing tenure, and reduces hospitalization stays compared with standard care (11,13–23). Notable methodological limitations include inconsistent definitions of Housing First, a lack of assessment of fidelity, small study samples, evaluation of a narrow range of outcomes, and research being conducted mostly by the developers of Housing First (6–9).

This article presents the findings of a large multisite RCT that compared the effectiveness of Housing First with ACT and treatment as usual over a two-year period. The trial addressed previous methodological shortcomings by implementing a standardized model of Housing First, delivering Housing First tailored to the needs of diverse populations, conducting fidelity assessments, having lead investigators who were not involved in the development of Housing First, and examining a wide range of outcomes (24). A previous article in Psychiatric Services presented one-year findings from this study (25).

Methods

Study Design

The study design was a parallel-group RCT conducted in Vancouver, Winnipeg, Toronto, Montreal, and Moncton comparing individuals receiving Housing First with individuals receiving treatment as usual. With results from the baseline interview, we stratified participants into “high need” or “moderate need” groups before randomization.

Participants were referred to the study by health and social service agencies. In the four largest cities, high-need participants were randomly assigned to receive either Housing First with ACT or treatment as usual, and moderate-need participants received either Housing First with ICM or treatment as usual. In Moncton, the small sample size did not allow for stratification, and participants with either a high level of need or a moderate level of need received Housing First with ACT or treatment as usual. This article focuses on the effectiveness findings for participants who received Housing First and ACT. The findings for those receiving Housing First with ICM have been reported separately (26).

Ethical approval of the study was received from the institutional review boards of 11 institutions.

Study Population

Participants met the following eligibility criteria: age 18 or older (age 19 in Vancouver); either absolutely homeless or precariously housed (such as lived in a rooming house, single-room occupancy unit, or hotel or motel room and had two episodes or more of homelessness in the past year); had a current mental disorder, as determined on the Mini-International Neuropsychiatric Interview, version 6.0 (MINI [27]), or by recent written diagnosis; not receiving ACT or ICM; and legal status as a Canadian citizen, landed immigrant, or refugee claimant.

We identified people as having a high need for treatment if they had a bipolar disorder or psychotic disorder, scored less than 62 on the Multnomah Community Ability Scale (MCAS) (28,29), and met one or more of the following criteria: were hospitalized twice in any one-year period in the past five years, had substance abuse or dependence (as assessed on the MINI or by a reported diagnosis from the referral source), or had been arrested or incarcerated in the past six months.

Study Interventions

Housing First.

The Housing First programs developed for the study were based on the Pathways Housing First Model (11,12). Participants contributed 30% of their income toward rent, and subsidies covered the difference. Housing units consisted mostly of private-market scattered-site units. Clients were assisted to choose among available units and furnish and move into them. Study participants had to agree to observe the terms of their lease and to be available for at least one weekly visit by ACT staff. Two assessments of program fidelity conducted during the study found good fidelity overall, with 78% of the 38 fidelity scale items rated higher than 3 on a 4-point scale on the second fidelity assessment, 24 to 29 months after the start of the programs (30).

Treatment as usual.

People assigned to treatment as usual had access to the existing programs available in their communities. Specifically, they could receive any housing and community support services other than from the Housing First program. None of the cities offered Pathways Housing First before the demonstration project.

Randomization

Using a 1:1 allocation ratio, we randomly assigned eligible participants into the two groups. Randomization allocations were done by a central data collection system that used an adaptive randomization algorithm. Assignments were revealed to participants at the end of the first interview. The nature of the project did not allow for blinding.

Study Outcomes

To collect outcome data, we interviewed study participants in person at baseline and every six months for a period of 21 or 24 months. The original protocol planned 24 months of follow-up, but final interviews for 172 (18%) participants were moved up to 21 months due to time and resource constraints. We conducted a brief interview every three months to stay in touch with participants and to collect data on their recent housing history. Interviewers recorded their impressions of the validity of data for each interview. For 100 interviews (2%), the interviewers reported having “no confidence” in the veracity of reporting. These records were excluded from the analysis.

The primary outcomes, defined in a published study protocol, were housing stability and community functioning (24). Secondary outcomes were self-rated health status, mental health symptoms, physical and psychological integration, substance use, quality of life, arrests, time hospitalized, and emergency department visits.

For housing outcomes, we examined time to first move into stable housing, percentage of days housed over the study period, and number of days consecutively housed at the final interview. We used data from the Residential Time-Line Follow-Back Inventory (RTLFB) (31) to calculate these outcomes. We measured perceived housing quality with the Perceived Housing Quality measure (32). Using the MCAS, the interviewers rated the community functioning of participants at the end of each interview (28,29).

Quality of life was assessed with the Quality of Life Interview (33); psychological integration, with a measure of sense of belonging in the community (34); and physical integration, with a measure of participation in activities outside the home (35). Health status was measured with the EQ-5D (36); severity of mental health symptoms, with the Colorado Symptom Index (37,38); and substance-related problems in the past month, with the Global Assessment of Individual Needs Short Screener–Substance Problem Scale (39,40). The RTLFB provided a count of nights hospitalized. We adapted existing measures (41–44) to develop the Health, Social, and Justice Service Use Inventory to document service use, including emergency room visits and arrests.

Sample Size and Statistical Analysis

The targeted sample size was set at 100 individuals per group in each site. Site-level studies were powered to have a minimum of 65 per group to allow for the detection of a moderate effect size (ES=.5) with α=.05 and β=.20 and anticipating up to 35% attrition.

We calculated group- and time-specific means for all outcomes and used bootstrapping with 2,000 replications to obtain 95% confidence intervals (CIs) around these values. We report bias-corrected and accelerated intervals (45).

Using linear regression models, we analyzed time to housing and length of stay in housing. For time-varying outcomes, we used mixed-effects models. We fit linear models for continuous outcomes, logistic models for binary outcomes, and negative binomial models for counts of events. In all cases, we included as covariates age, sex, site, ethnoracial status, and aboriginal status.

We calculated two-level random-intercept mixed-effects models, with time points nested within participants. We treated time as a categorical variable. This involved dummy-coding events and including an interaction between each event and group membership. We evaluated two measures of group differences on change from baseline: difference between groups at the final time point and the average difference over all postbaseline events (reflecting differences over the study period as a whole). To calculate the latter, we used postestimation tests (“lincom” process in Stata, version 13).

To examine site differences in intervention effects, we fit models that treated baseline levels as a covariate (to simplify the task of isolating site differences) and included group × site interaction terms. We tested the overall effect of the interactions using a postestimation contrast.

Finally, to investigate potential problems resulting from missing data, we performed a sensitivity analysis using a set of 40 imputed data sets developed with sequential regression multivariate imputation (46). We performed this analysis for the MCAS. Effects for the other primary outcome, stable housing, were too large for missing data to seriously threaten inferences.

Results

Study Participants

The total sample at baseline (N=950) represented 95% of the planned enrollment. [A CONSORT diagram for the study is available in an online supplement to this article.] Baseline interviews began in October 2009 and ended in August 2011, with the last follow-up interviews conducted in June 2013. A total of 780 (82%) participants completed the final interview: 369 of 481 in treatment as usual (77%) and 411 of 469 (88%) in Housing First. Table 1 provides the characteristics of participants.

Primary Outcomes

Table 2 presents the primary outcomes. The mean proportion of time spent in stable housing over the 24-month period was 71% for Housing First participants compared with 29% of treatment-as-usual participants (adjusted absolute difference [AAD]=42%, 95% confidence interval [CI]=38%−45%, p<.01). As shown in Figure 1, differences between the groups were present at all sites.

At the final interview, 273 of 369 Housing First participants (74%, CI=69%−78%) and 138 of 337 treatment-as-usual participants (41%, CI=35%–46%) were in stable housing. The mean length of stay for these individuals was 401.9 days (CI=372.2–430.2) for Housing First participants and 281.2 days (CI=251.2–318.6) for treatment-as-usual participants (p<.001). Of participants who achieved stable housing at any time during the study, those in Housing First moved into housing more rapidly than treatment-as-usual participants did (72.9 versus 219.7 days, AAD=146.4, CI=118.0–174.9, p<.001). Taking into account all study participants, we found that Housing First participants also had longer tenures than treatment-as-usual participants at the study end point (280.7 versus 115.3 days AAD=161.8, CI=82.5–241.1, p<.01). Compared with treatment-as-usual participants, Housing First participants rated their housing as being of significantly better quality (adjusted standardized mean difference [ASMD]=.17, p<.01, CI=.06–.28).

Housing First participants showed more rapid improvement in the first year of the study and had greater improvement in community functioning over the course of the study as a whole (average ASMD over all follow-ups=.18, p<.01, CI=.05–.31). However, this group difference was attenuated by the end of the study as a result of continued improvements in the treatment-as-usual group in the second year of the study (ASMD=.12, CI=–.04 to .30, p=.15). Both groups improved substantially over the course of the study (pooled standardized mean difference [SMD] for change from baseline to 24 months=1.05). Results from the multiple imputation analysis did not change the findings.

Secondary Outcomes

Table 3 presents the secondary outcomes for the two groups. Both groups of participants reported similar improvements in their health status (pooled SMD=.34), mental health symptoms (pooled SMD=.70), as well as a reduction in substance use problems (pooled decrease in mean symptom count=30%). For mental health symptoms, a small group difference favoring treatment as usual emerged at the final follow-up (ASMD=.17, CI=.05–.30, p=.01). There were no significant changes in physical integration for both groups. Both groups reported significant increases in psychological integration (pooled SMD=.53).

Differences in quality of life showed a pattern similar to community functioning, with Housing First participants improving more rapidly in the first year and having higher average scores over the study period (ASMD=.15, p<.01, CI=.04–.24) but with the gap subsequently narrowing over time (ASMD at final interview=.05, CI=–.08 to .18, p=.43). A moderate to large effect in improvements of quality of life over time was present for the two groups (pooled SMD=.76).

Both groups reported similar decreases in the number of days hospitalized (pooled decrease=62%), emergency department visits (pooled decrease=53%), and arrests (pooled decrease=60%). Housing First participants showed an initial greater decrease in emergency department visits (incidence rate ratio [IRR] at the six-month follow-up=.68, CI =.52–.90, p=.007), but the difference for the study period as a whole fell short of our significance threshold (IRR=.80, CI=.65–1.00, p=.05). There were otherwise no significant group differences for these outcomes.

We found a significant difference across sites in the effect of the intervention on perceived housing quality, with larger advantages for Housing First in Vancouver and Moncton and no group difference in Montreal. For other outcomes, intervention effects did not vary across sites.

Costs

On average, Housing First with ACT services cost $22,257 (Canadian dollars) per participant annually. Taking into account use of health, social, and justice services, Housing First produced an average net cost offset of $21,367 Canadian per participant per year, or 96% of the cost of the intervention. The most important cost offsets were office visits, hospitalizations for general medical conditions, emergency shelter visits, home visits, and incarceration.

Discussion

Our study extends previous research (11,13–23) by demonstrating the effectiveness of Housing First with ACT in helping individuals to rapidly exit homelessness and achieve housing stability in different Canadian cities. Our housing results are clearly in favor of Housing First in contexts in which it had not been previously researched. They demonstrate that in the absence of providing housing, health and social services yield slower exits from homelessness and less housing stability even in the context of the universal health care available in Canada.

Housing First also improved community functioning and quality of life more rapidly than treatment as usual. However, the difference between the two groups narrowed, with plateaus in Housing First after the first year and continued improvement in treatment as usual over the two years of the study. Community functioning operationalized by the MCAS has not been previously examined in related research (11,13–23). A previous study comparing individuals receiving Housing First with ACT to individuals receiving outpatient mental health services found the Housing First group reporting a better quality of life after 12 months (20). We also found differences in quality of life also favoring Housing First with ACT after 12 months but with these differences attenuating in the second year. A plausible interpretation is that the earlier stabilization in housing combined with ACT serves as a catalyst for the more rapid improvements in community functioning and quality of life.

Both groups improved substantially on all nonhousing outcomes except physical integration. These improvements probably reflect in part the effect of services received by both groups, but they are also likely to result from regression to the mean. Given the nature of the study, it is probable that participants tended to be referred when they were experiencing significant difficulties. Participants’ histories often reflected intermittent homelessness, and many psychiatric crises are episodic. The typical course of illness is therefore likely a fluctuating one, which tends to increase regression to the mean (47). Community programs were responsible for referring participants; thus treatment-as-usual participants were likely to be receiving care at baseline.

We found no intervention effect on mental health symptoms or on substance use–related problems. For mental health symptoms, there was a small difference in improvement favoring the treatment-as-usual group at the final follow-up that could reflect chance variation. Previous research comparing Housing First to usual care found no change in the severity of psychiatric symptoms or level of alcohol or drug use in both groups over 48 months (15–17).

Our findings also show notable and comparable decreases in number of days hospitalized, number of emergency department visits, and number of arrests for both groups. These decreases are consistent with improvements in health status and community functioning. Previous research conducted in the United States has reported decreased number of days in psychiatric hospitals favoring Housing First compared with a residential continuum or treatment-as-usual approach (14) and decreased use of inpatient, emergency, and justice system services also favoring Housing First compared with use of outpatient mental health services (20). The difference in treatment services and other social services available in Canada compared with the United States may be a consideration when interpreting the absence of group differences in changes in health status and service use outcomes, given that all participants had, at least in principle, access to universal general medical and mental health care.

Our study had several strengths. These include the size and diversity of the sample, the low attrition rate, the frequency of data collection, fidelity assessments, and the wide range of examined outcomes. Limitations include the early stage of development of the Housing First programs, the broad range of treatment-as-usual services that were available, and the nonblinding in the study. It is also possible that our measures were not sensitive enough to capture the very positive life changes reported by stably housed recipients of Housing First in qualitative interviews (48).

Conclusions

The findings of our trial extend previous research into a Canadian context and demonstrate that Housing First with ACT yielded significant benefits to individuals with high levels of need, notably helping them to exit homelessness as well as experience rapid gains in community functioning and quality of life. In comparison, individuals receiving treatment as usual experienced poorer housing outcomes but similar nonhousing outcomes. From a policy perspective, the choice becomes to either implement Housing First and significantly reduce homelessness while having a modest effect on mental health and addiction or to provide treatment first, then housing, with similar clinical outcomes but inferior housing outcomes. The Canadian federal government has used the study findings to prioritize the development of Housing First programs in its national homelessness initiative (49).