Outcomes of Homeless Adults With Mental Illness in a Housing Program and in Case Management Only

In the 1980s, research on homeless persons with mental illness focused on understanding the extent of the problem, the relationship between mental illness and homelessness, and the system and service needs of this population (1). Review of this research suggests that the task of future research named as most important was to identify effective interventions and determine which program components would be effective for which subgroups and contexts (1).

In the 1990s, studies suggested that homeless people with severe mental illnesses would use accessible, relevant services and, furthermore, that programs tailored to the needs of this population, such as housing with a range of supportive services or specialized case management, were effective in assisting these individuals to move into and maintain stable housing (2,3,4). Certain factors, especially substance abuse, were shown to affect homeless persons' ability to maintain housing (5,6,7,8).

Given competing demands for limited service dollars, it is of great interest to policy makers, providers, and consumers to determine what types and levels of homeless services are effective and for whom. What is needed, then, from current and future research is studies comparing the effectiveness of various evidence-based services for this population (9).

In a prospective, quasi-experimental research design, with measures administered at baseline and six months and 12 months after participants entered the programs, this study compared the effectiveness of two types of programs in achieving positive housing, mental health, and substance use outcomes for a sample of severely mentally ill homeless persons. The comparison was made between a pair of comprehensive housing programs (for this study, considered as one program or condition) offering guaranteed access to housing, housing support services, and case management and a single program offering only specialized outreach and case management services.

Methods

Characteristics of the programs

Comprehensive housing programs. Two programs were chosen as representative of comprehensive housing and housing support services programs. A large agency based on psychosocial rehabilitation principles—Boley Centers for Behavioral Healthcare in Pinellas County, Florida—has long served individuals with serious mental illness. In 1988, Boley Centers developed a program specifically to prevent and reduce homelessness in this population. The program features guaranteed access to housing and housing support services, case management, and priority access to everything from medication management to vocational services (10). Project Return in Tampa, Florida, also provides comprehensive housing services to homeless persons with severe mental illness, including guaranteed access to housing, housing support services, and case management.

Case management only. Suncoast Center for Community Mental Health, a large community mental health center also in Pinellas County, Florida, has developed a homeless outreach and support team (HOST) to provide short-term case management services for homeless individuals with severe mental illness. The activities of this blended case management program (11) include active outreach and engagement, some on-site counseling, medication and medication management, assistance with obtaining housing, and linkages to other psychosocial services.

Process evaluation and fidelity assessment

For the process evaluation of this study, the model developed was based on the key components of the Boley Homelessness Prevention Project (10): guaranteed access to housing, housing-related support services, active linkages to other services and resources, a supportive organizational climate, and strong staff-consumer relationships. Each component's relevant dimensions were operationally defined. Criteria and objective anchors were established for scoring each dimension on a 5-point scale as a measure of fidelity to the comprehensive housing program model. Multiple data sources were used for each rating, including management information system reports, self-report of clients and staff, objective measures, and ethnographic interviews.

The results of these ratings confirmed that the comprehensive housing programs, at the midpoint of data collection for the project, were operating with a high degree of fidelity to their program models. Boley Centers and Project Return had a high degree of concordance on the critical elements of a comprehensive housing program. The Suncoast HOST program had linkages to services similar to those of the other two programs but did not have the guaranteed housing and housing services aspects, validating its use as a program of case management only. Furthermore, the two types of programs do not differ greatly in other agency characteristics such as organizational climate and staff-consumer relationships, suggesting that any differential effect would be due to housing and housing-related components.

Sample characteristics

Participants were individuals entering the three programs from December 1997 through April 1999. All participants were homeless or at immediate risk of homelessness and had diagnoses of serious mental illness. Research personnel contacted those who agreed to participate, explained the study in detail, and obtained written informed consent. The institutional review board of the University of South Florida approved the consent form and other research procedures. Participants were paid $20 for each interview. A total of 152 individuals agreed to participate—83 from the comprehensive housing programs and 69 from the program that provided case management only. Of these, 108 (71 percent) were available for at least one follow-up interview.

The sample consisted of 79 males (52 percent) and 73 females (48 percent). A total of 117 (77 percent) were white, 30 (20 percent) were black, and five (3 percent) were other races. In addition, three participants reported Hispanic ethnicity. The mean age was 38.4±8.7 years, with a range of 19 to 58. The modal marital status was never married (73 participants, or 48 percent). A total of 50 participants (33 percent) had some college or were college graduates, 58 (38 percent) were high school graduates only, and 44 (29 percent) did not finish high school; 19 (12.5 percent) were veterans, and 33 (22 percent) were employed. The mean monthly income from all sources was $403.47±$454.63. For lifetime homelessness, the modal number of episodes was three to five; 138 participants (91 percent) had a history of previous homelessness. Chart reviews suggest that all participants had a current axis I diagnosis; about half had a psychotic disorder as the primary diagnosis, and mood disorder diagnoses accounted for another 45 percent. About half of all participants also had a secondary diagnosis of a substance use disorder.

Table 1 presents a summary of the demographic composition of the sample by type of program. The participant groups in both types of programs were comparable in each characteristic except for number of times previously homeless. The group receiving only case management included a significantly greater percentage of participants who had been homeless more than once.

Outcome measures

Housing outcomes. The six-month retrospective residential measure was adapted from the Residential Follow-Back Calendar (12). Participants indicated, for all the places where they had resided during the 180 days before entering the study, the location, duration, reasons for moving in, and reasons for moving out. Three related housing outcomes were used for the analysis: proportion of time in stable housing, proportion of time literally homeless, and proportion of time functionally homeless. Scores for stable housing and literal homelessness were developed directly by summing the number of days spent in settings considered to be stable and the number of days spent with no place to live and dividing each by the number of days in the assessment period.

The scoring of functional homelessness was more complex, because it took both prior and later housing status into account. For example, if a consumer was homeless, then went into an institutional setting such as an inpatient hospital or a detoxification facility, then returned to being homeless, the entire period would be considered one of functional homelessness. In contrast, if the prior setting was stable housing and the consumer entered a facility, then became homeless after discharge from the facility, functional homelessness would be calculated from the time of discharge.

Psychiatric symptoms. The Modified Colorado Symptom Index (13), adapted from the Colorado Symptom Index (14), was administered. Participants answered questions about the frequency of 15 psychiatric symptoms on a 5-point scale, from 1, not at all, to 5, every day. The score is the sum of the responses.

Substance abuse. Participants retrospectively reported the number of days of use of 19 different drugs and three types of alcoholic beverages over the previous six months; for responses, they used the Drug/Alcohol 6-Month Follow-Back Calendar (15). Data were analyzed in terms of days of alcohol use and days of illegal drug use.

Design and analysis

The outcome assessment used a quasi-experimental or nonrandom-assignment design to evaluate whether two evidence-based homelessness programs produced differential changes among the target population in mental health, substance abuse, and housing stability. The relative importance of aspects of the intervention was determined by comparing the outcomes of individuals in programs that included housing and case management with the outcomes of individuals receiving specialized case management only.

Longitudinal research of this type is difficult because of missing data and because participants are assessed at different time points. Therefore, a random regression model was used as the primary method of analysis. Random regression models analyze changes over time for both the population and the individual. As in an intent-to-treat research model, participants remained in the group in which they were originally being served regardless of subsequent use of services.

Results

Comparability of the intervention groups at baseline

To determine how comparable the groups were before the interventions, one-way analyses of variance were conducted on all baseline dependent measures. As mentioned, the groups were comparable in all demographic variables except for the number of times of previous homelessness. The two groups did, however, differ significantly in psychiatric symptoms and days of alcohol and illegal drug use at baseline. They also differed significantly in baseline housing status. Participants in the program that provided only case management were more impaired in these areas.

Because of the noncomparability of the groups in psychiatric and substance abuse status, a conventional approach to statistical analysis was not feasible. Instead, a propensity score subclassification was used to reduce the biases associated with pretreatment difference between the groups (16). The propensity score balances all observed covariates (16) and reduces the pretreatment differences between the groups to a single score that appropriately summarizes those differences. To develop the propensity scores, scores from all pretreatment measures except housing outcome measures were entered into a logistical regression analysis. Similar to a discriminant function analysis, this analysis estimates the probability of each person's being in the treatment group by comparing observed scores with predicted ones. Among the participant sample who entered the programs (N=152), three significant discriminator variables emerged: psychiatric symptoms (χ²=15.64, df=1, p<.001), days of illegal drug use (χ²=11.52, df=1, p<.001), and alcohol use days (χ²=4.37, df=1, p=.04).

On the basis of this analysis, each participant had a propensity score assigned, which was derived from baseline levels of mental health symptoms, days of alcohol use, and days of illegal drug use, or "level of psychiatric and substance abuse impairment." The distribution of propensity scores was divided into thirds to form low-, medium-, and high-impairment subgroups within the two types of interventions. Level of impairment, type of intervention, and time were the independent variables for the main analyses.

Attrition analysis

Of the 152 participants who began the study, 99 (65 percent) were available at six-month follow-up assessment and 88 (58 percent) were available at 12 months. The comprehensive housing program group had 83 participants at baseline, 66 (80 percent) at six months, and 63 (76 percent) at 12 months. The group that received only case management had 69 participants at baseline, 33 (48 percent) at six months, and 25 (36 percent) at 12 months. The attrition rate in the group receiving only case management was significant and was probably due to the time-limited nature of these services.

A number of participants received the 12-month but not the six-month assessment, or vice versa. We therefore combined the six-month group and the 12-month group to increase the number of observations per cell and the power of the analyses (17). Each participant retained in the analysis had at least one follow-up score, and scores were averaged for those with two follow-up assessments. The numbers arrived at through this procedure were as follows: the comprehensive housing group had 69 participants available at follow-up (83 percent retention), whereas the group receiving only case management had 39 participants (56 percent retention). Hence, a total of 108 participants completed at least one follow-up interview, resulting in an overall retention rate of 71 percent. This rate is consistent with that of other follow-up studies of homeless persons, which found retention rates ranging from 33 percent to 86 percent (18).

Analyses of variance were used to analyze the differences in baseline scores on the dependent measures and the age and income variables between the participants who remained in the study and those who dropped out at either six or 12 months; the results are shown in Table 2. There were no significant interactions of program by retention status on any measure, indicating that the baseline scores and background characteristics of those who stayed in the study and those who dropped out were similar across the two conditions. Importantly, the results for the variables that made up the impairment scores suggest that attrition was not related to levels of psychiatric symptoms, illegal drug use, or alcohol use. Chi square analyses revealed no significant differences in dropout rates as a function of demographic characteristics or as a function of past history of homelessness.

In summary, the baseline scores of the participants who remained in the study and those who dropped out were generally comparable.

Outcome analysis

The means and standard deviations of housing outcomes for the two types of interventions by propensity score subclassification are presented in Table 3. The random regression analysis using the autoregressive maximum likelihood method of estimation revealed a significant triple interaction among type of intervention, propensity score subclassification, and time in stable housing (p=.004). The triple interaction was also significant for time functionally homeless (p<.001). Although the results for literal homelessness were in the same direction, they were not statistically significant. High-impairment participants showed less gain in stable housing and less reduction in functional homelessness in the program providing case management only than did high-impairment participants in the comprehensive housing program. However, the participants with low and medium impairment in the program providing only case management did just as well on housing outcomes as did those in comprehensive housing programs.

The analyses of change in psychiatric symptoms, days of alcohol use, and days of drug use were evaluated through 3 × 2 × 2 random regression model analyses. Although improvements were observed over time for each of these variables, no significant differences were found on any measure as a function of impairment and type of program.

Discussion

Persons with high psychiatric symptom severity and high alcohol and drug use achieved better housing outcomes with housing and housing support services plus case management than they did with case management alone. However, persons with low and medium symptom severity and low alcohol and drug use did just as well with case management alone. Substance use and psychiatric status at baseline appear to moderate the effects of the programs for all three housing outcome variables.

The dramatic nature of this differential effect can be seen when the outcomes are considered in terms of actual days of stable housing. Participants with low impairment who received only case management increased their time in stable housing by an average of 109 days during the 180-day period. However, those with high impairment who received the same intervention showed an average increase of only 52 days. Highly impaired participants in the comprehensive housing programs showed an average increase of 106 days in stable housing. In other words, for the highly impaired participants, the type of program made an average difference of 54 more days in stable housing—an increase of more than 100 percent.

The results of this study suggest a new direction in homelessness research. The homeless population is diverse in terms of gender, ethnicity, age, education, pathways to homelessness, severity of psychiatric symptoms, incidence of co-occurring drug and alcohol problems, problem-solving resources, and degree of social support. This study suggests that effectiveness rates can be increased by careful matching of interventions and consumer characteristics.

For example, a young mother who is experiencing a severe major depressive episode, who is homeless because of domestic violence, and who has basic job skills may need an intervention different from the one needed by a single male with chronic schizophrenia and a co-occurring substance abuse disorder who has repeatedly failed at independent living. A brief intervention that includes housing assistance and linkages for mental health and family services may suffice for the first person, whereas a comprehensive, multifaceted housing program will probably be required for the second (19). Rosenheck and colleagues (20) demonstrated that each treatment element in a program for a similar population had its own cost-effectiveness ratio.

These findings are limited by the fact that participants were not randomly assigned to conditions. Although the attrition analysis showed that those who dropped out and those who remained in the study were comparable on most dimensions, and although the propensity score subclassification removed observed pretreatment bias, various unmeasured participant characteristics could have influenced the results. The demographic characteristics of the sample reflected those of the local area, but they may not be characteristic of the national homeless population. This sample had a large proportion of whites, for example. It is not clear that minorities, women with children, or residents of larger urban centers, even those with less substance abuse and fewer mental health problems, would be able to secure stable housing with case management services alone (6,19,21). Finally, a longer period of follow-up is needed to determine whether these results would be maintained over time.

Conclusions

The answer to the question of which is more effective, comprehensive housing programs or case management alone, appears to be "it depends." This study suggests that for those who have relatively low levels of psychiatric symptoms and substance use, less expensive case management services are just as effective. The study also suggests, however, that those with high levels of psychiatric symptoms and substance use benefit most from the comprehensive services of guaranteed access to housing, housing services, and the assistance of and connection to a case manager. Judging from the striking reductions in homelessness, such programs appear to be well worth the investment.

These findings are especially compelling in light of a recent comprehensive five-year study that found that homelessness among mentally ill people imposes a surprisingly high cost on taxpayers and that independent housing linked to comprehensive health support and employment services provides major reductions in this cost (22).

The results of the study reported here also argue against a one-size-fits-all approach to services for this diverse population. As policy makers, funders, and providers review the eligibility criteria and performance standards for such programs, consideration should be given to providing for the most impaired individuals the greatest access to comprehensive housing programs.

Finally, this study is a first step in a new direction for research with this population, one that emphasizes matching interventions to specific characteristics of the consumers and determining what works best for whom.

Acknowledgments

This study is part of the cooperative agreement study of the Center for Mental Health Services and Center for Substance Abuse Treatment Program to Prevent Homelessness and was funded by grant S UD9 SM1965-03 from the Substance Abuse and Mental Health Services Administration. The authors acknowledge the contributions of Steven Banks, Ph.D., in designing the analysis strategies and of Gregory Teague, Ph.D.

Dr. Clark and Dr. Rich are affiliated with the department of mental health law and policy at the Louis de la Parte Florida Mental Health Institute at the University of South Florida, 13301 Bruce B. Downs Boulevard, MHC 1345, Tampa, Florida 33612-3807 (email, [email protected]).

Enlarge table

Enlarge table

Enlarge table