Service Use Over Time and Achievement of Stable Housing in a Mentally Ill Homeless Population

The literature on service provision for the homeless population with serious mental illness provides few examples of empirically validated treatment models. Evidence on the effectiveness of case management (1,2,3,4,5,6,7) and transitional housing (8,9,10,11,12,13) for this population is mixed.

The relative superiority of outcomes for other, more intensive models of service provision, such as assertive community treatment, has been documented (14,15,16). Morse and colleagues (14) reported that participants in a traditional assertive community treatment program experienced significantly fewer days homeless and fewer psychiatric symptoms than did participants in a drop-in center or in traditional outpatient treatment interventions. In a separate study, Morse and colleagues (16) demonstrated the superiority of assertive community treatment over brokered case management along similar dimensions. Meisler and colleagues (15) also found evidence supporting the effectiveness of higher-intensity services for a homeless population with co-occurring severe psychiatric and substance use disorders.

One consistent finding in homelessness services research is a positive association between the amount of service use and the achievement of positive outcomes (17,18,19,20,21,22,23). Pollio and colleagues (17) reported that the volume and type of service use predicted housing outcomes. Similarly, Cohen and colleagues (18) found that the number of service encounters was associated with both improved housing and sobriety outcomes.

The timing of the intervention also appears to be a key factor in achieving good outcomes (7,17,24,25,26). Susser and colleagues (24) reported that implementation of a "critical timing" intervention after discharge from an intensive program for homeless persons with mental illness yielded dramatic improvements in housing stability 18 months after completion of the intervention. In a substance abuse program for homeless persons, Wright and Devine (25) found that clients who met a certain threshold of service use at three months had more positive outcomes than those who did not. Sosin and colleagues (7) demonstrated that clients in a program with progressive diminution of service intensity over time achieved better outcomes than those in a control group.

A second key factor associated with achieving positive outcomes among homeless mentally ill persons is the development of a personal relationship between the service provider and the client and a mutual understanding of the client's specific needs. Erickson and colleagues (27) empirically documented the positive impact on treatment outcomes of clients' "willingness" to engage in treatment. This willingness construct encompassed motivation, readiness, and suitability. Central to Susser and colleagues' (24,28) critical timing intervention was the importance of developing relationships during periods of adjustment to transitions.

Pollio and Kasden (29) reported that homeless persons affirmed the critical importance of the match between the needs perceived by the client and those assessed by the service provider. North and Smith (30) found substantial differences between service needs as perceived subjectively by homeless persons and assessed objectively by researchers. Other research has also identified the importance of incorporating the presenting problem in service recommendations, both as perceived by the client and assessed by the worker (18,31).

These observations of the optimal timing for interventions and the willingness of clients to accept services have led us to hypothesize that homeless persons with mental illness go through five distinct stages of readiness in the achievement of positive outcomes. The first stage is pre-engagement, in which clients' willingness to engage in the change process is minimal. The second stage is contemplation, in which the potential for change is first considered. This stage is followed by engagement, in which the client and the agency negotiate mutually acceptable service use. The fourth stage is the "strategic moment" when change is achieved. The final stage is consolidation, in which the change is strengthened by behaviors employed to successfully maintain the positive outcomes.

This conceptual model of achieving change over time maps directly onto the transtheoretical model of change (32,33,34). Although the transtheoretical model has not been tested in a homeless population, its utility has been demonstrated in a variety of studies across different populations—perhaps most of all with populations coping with addictions (33,35,36,37,38,39). The transtheoretical model has been criticized for applying arbitrary stages to what is in reality a continuum and for presenting as a progression a process that does not often behave in an orderly progressive fashion (40,41).

From a service perspective, the conceptual model proposed here suggests that people exhibit different patterns of service use according to their stage in the change process. Those who are entering the engagement and contemplation stages demonstrate increasing use of services, which will reach greatest intensity around the strategic moment—the achievement of change—and immediately afterward, and then decrease markedly as the individual consolidates the resulting changes.

The purpose of this study was to test two elements of this model of behavior change by examining the relationship between the rate of service use and the single outcome of achieving stable housing among participants in an agency serving persons who are homeless and have severe mental illness. The two stages examined were the strategic-moment stage—or the achievement of housing—and the consolidation stage, which occurs afterward. On the basis of the conceptual model, we predicted that homeless persons with mental illness who achieve stable housing would use the greatest number of services in the period when they obtained housing. We also hypothesized that service use would decrease over time as stable housing was maintained, with the greatest decreases occurring immediately after housing was obtained. We predicted that those who did not achieve stable housing would use fewer services consistently and show a progressive linear decrease in service use over time. The alternative hypothesis was that all service use followed a similar pattern over time regardless of the achievement of positive outcomes.

Methods

Data collection

Data were gathered from St. Patrick Center, a multiservice agency serving the homeless mentally ill population in St. Louis, Missouri. The center provides a broad base of services, including drop-in services, a variety of counseling and case management services, housing services, employment and vocational training, referrals to outside agencies, and substance abuse and dual diagnosis programs. Because the agency provides a wide variety of services, it does not screen out homeless persons who do not have severe mental illness, and thus it serves subpopulations of persons with primary substance abuse and with economic problems.

In the initial phase of data collection, information was obtained from chart review and from the management information system on age, gender, race, marital status, military history, psychiatric diagnosis, and presenting problem, along with cross-sectional data on service use. Results of analyses of the relationships among these variables for clients who did and did not obtain stable housing has been reported elsewhere (17). After the publication of the initial report, however, it proved possible to disaggregate the cross-sectional service use data into monthly increments. Therefore, longitudinal service use data were analyzed, and the results are presented here.

Sample

Two groups of service participants were identified for this study. The housed group consisted of 58 individuals who had maintained stable housing for at least 24 continuous months before data collection began in October 1995. Use of the 24-month criterion was intended to produce a housed sample with a low risk of returning to homelessness. Persons using residential services, such as transitional housing, in the 24 previous months were not counted as having achieved stable housing, although the management information system listed them as such.

The unhoused group was identified from the center's rosters on the basis of date of service entry. For each housed client in the program, the next unhoused client who participated in the service system beyond intake for at least one month was chosen as a match. This comparison sample was selected to represent the remainder of the homeless population who continued to be consistently or cyclically homeless; this population constitutes the vast majority of users of the agency's services. The one-month requirement for service system participation controlled for immediate dropouts, of which there were eight. For three housed cases, no matched subject for the unhoused group was available for the same month of entry, and so three fewer individuals were included in the unhoused group, and data were analyzed using an unmatched design. Thus the unhoused group had only 55 subjects, for a total sample size of 113.

Variables

The primary comparison in this analysis was service use between the housed and unhoused groups. Service use was separated into eight mutually exclusive categories: drop-in center; counseling, which included any counseling not directly related to psychiatric treatment of mental illness, such as insight-oriented therapy; health services, which included direct health services and provision of health education services; transportation services, which included transportation by staff and receipt of bus passes; vocational counseling, job training, and assistance seeking employment; substance abuse services, which primarily consisted of attendance at Alcoholics Anonymous meetings; psychiatric services directly related to a mental illness, including medication management; and financial support, including help in applying for entitlements and emergency support and help with budgeting.

For all service types, face-to-face contacts were aggregated on a monthly basis using data from the management information system that were input from daily activity forms completed by all staff.

Service data were obtained for each individual starting at service entry. Because the period between service entry and obtaining housing was different for each client, the service data proved too varied to analyze the entire service process. Thus the analysis focused on a more limited test of the hypothesized model. We examined service use beginning in the month before the clients in the housed group achieved stable housing, including the month housing was achieved, and ending 24 months after this event. For individuals in the unhoused group, data for the same 26 months were collected. An aggregate monthly service use variable was derived from the sum of service contacts of all types for each month.

To address skew in distribution, the data were examined for outliers. Months during which an individual had more than 300 service contacts were eliminated for that person. For the aggregate service use variable, this procedure resulted in eliminating three months of data, one month for each of three subjects from the entire sample of 113 clients (113 individuals × 26 months). To compensate for this elimination, two months of service use data were eliminated for clients in the unhoused group who used the drop-in center.

Data analysis

To determine the most effective analytic strategy for testing the study hypothesis, service use by time was initially plotted for the housed group only. A time plot of service for clients in the housed group revealed a dramatic drop in services used during the first few months after housing was obtained, followed by a more gradual drop beyond that point. Because the decrease in service use was so pronounced, it seemed parsimonious to model the change in terms of two intersecting straight lines rather than by a quadratic or higher-degree polynomial, which could not capture the exact point at which the change took place. This type of model is called a linear spline model (42). In general, splines are polynomial curves joined together at points called knots. Linear splines are first-degree polynomials (straight lines). While a single straight line can be fitted by ordinary linear regression, fitting splines, even linear ones, requires use of a specialized algorithm.

All models were fitted to the mean utilization, using the bootstrap method to assess the standard errors and covariances of the estimates. Because of dependence of values within subjects, the bootstrap sampling was performed at the subject level. Bootstrap sampling consists of sampling at random with replacement to generate artificial samples of the same size as the original (43). Because of sampling with replacement, in any given bootstrap sample some subjects do not appear, and others are included more than once. In these analyses, 1,000 bootstrap samples were drawn to generate each set of estimates.

The specialized algorithm used in this analysis was created using SAS 6.12 (44). For each bootstrap replication, ordinary least-square models were used to generate paired lines for all possible break points, and the pair with the smallest sum of squares was selected. Coordinates for the intersection were then calculated from the equations of the two lines. Standard errors were combined with covariances to test the null hypothesis that the lines are identical in slope and intercept, indicating that service use by the housed and unhoused groups was similar for the two time periods. The slope and intercept for the paired lines and their standard errors along with results of significance tests are reported in Tables 1 and 2.

Data from months 2 through 26 were included in the analysis. Month 1 was excluded because housing was achieved at month 2 for the housed group, and the study's purpose was to test differences in patterns of utilization between those obtaining housing and those not obtaining housing from the time the housed subject obtained housing. Service use during the month before housing was achieved would theoretically occur at lower rates—because service use increased with engagement—which would tend to obscure the significance test of the hypothesized relationship in our analysis. (Visual inspection revealed that this actually was the situation.) The same two-line model was fitted to those in the unhoused group to examine the alternative hypothesis that individuals in the unhoused group would exhibit the same sharp decrease in service use.

Finally, differences between the first sets of intercepts generated were tested for significance to examine the hypothesis that subjects who attained stable housing would use more services than those in the unhoused group during the month that they achieved stable housing.

An additional analysis was conducted to test the hypothesis that service use was different for the two stages of change for each service type; the same statistics and bootstrap sampling techniques were used. Initially, this analysis was planned for both housed and unhoused groups. The results from the first analysis, which are discussed below, indicated the need for a secondary analysis only for members of the housed group. The purpose of the separate analyses was to determine whether the hypothesized service use pattern existed for overall service use or only for a subset of service types.

Results

Descriptive statistics

Clients in the sample were relatively young. The mean±SD age of the 113 clients was 39.9±9.7 years. Most were men (88 clients, or 78 percent), and most were African American (86 clients, or 76 percent). Nearly two-thirds of the sample had never been married (72 clients, or 64 percent). More than half (67 clients, or 59 percent) had either completed high school or received a graduate equivalent diploma. Two-thirds of the sample (76 clients, or 67 percent) had a chart diagnosis of a substance use disorder; smaller proportions had a diagnosis of schizophrenia (19 clients, or 17 percent), bipolar or unipolar affective disorder (14 clients, or 12 percent), and personality disorder (seven clients, or 6 percent).

Presenting problems listed by the intake worker included mental illness (44 clients, or 39 percent), substance abuse (34 clients, or 30 percent), dual diagnosis (25 clients, or 22 percent), and subsistence needs (10 clients, or 9 percent). The only significant demographic difference between the housed and unhoused groups was that the housed group included a higher proportion of men (86 percent versus 71 percent; χ²=4.10, df=2, p=.04).

For the eight service categories, the 113 clients each used a mean±SD aggregate of 455±751 units of service over the 26 months. The most frequently used services were the drop-in center (mean±SD=296±457 units for the 26 months) and counseling (83±175) units. Cross-sectional analyses indicated that in seven of the eight use categories, the housed clients had significantly higher rates of service use (in either the volume or the proportion used) than the unhoused group; the exception was psychiatry-related services, for which no significant difference was found between groups. Details about the rates of service use in these groups were published previously (17).

Hypothesis testing

The results of analyses comparing service use by the housed and unhoused groups are reported in Table 1. The first two rows report the intercept and slope of the steeply descending initial stage, when clients' service use was decreasing after they obtained housing. The next two numbers describe the intercept and slope of the more gradually descending second line, when clients were consolidating the change they had made. The last two rows report the x and y coordinates of the intersection of the two lines.

The null hypothesis that there would be a single line for the entire period (equal intercepts and equal slopes), which would indicate that there was no change in rates of service use over time, was rejected for the housed group (χ²=20.94, df=2, p<.001) but not for the unhoused group (χ²=4.20, df=2, p=.122). A test for differences between the initial intercepts of the two samples was significant (z=2.68, p=.007). This dramatic difference in test results provides strong evidence of fundamental differences in service utilization patterns between clients who achieved housing and those who did not.

Figure 1 presents a graphic representation of the hypothesis for both groups. The actual monthly means for total service use by both groups are shown as well as the two lines fitted by the two-stage solution from the data analysis for the housed group.

Because a two-stage solution emerged only for the housed group, analyses by service use category were conducted only for the housed group. Of the eight services, three were used consistently enough across time to warrant separate study. They were the drop-in center, counseling, and health services. Two-line models were fitted to each of these three service variables.

Table 2 presents robust evidence that use of the drop-in center and counseling services followed the hypothesized two-line models for the housed group. Use of the drop-in center during the first stage was significantly different from use in the second (χ²=19.20, df=2, p<.001), as was use of counseling services (χ²=26.93, df=2, p<.001). The difference in use of health services for the two stages fell just short of statistical significance (p=.062).

The two-line models must be considered approximations over the range of months (months 2 through 26) for which data were available. None of the model estimates predicted negative service utilization within that range. However, given the negative slopes, all the lines will eventually drop below zero at some point after month 26. To fit a model with curvature, or perhaps a second knot point beyond which there would be steady use of services, would require collection of data over a much longer period.

Discussion and conclusions

In a previous cross-sectional analysis of this dataset examining service use between the housed and unhoused groups, we speculated that "moments may exist when homeless individuals can be motivated to enter service systems" (17). The study reported here was able to address the limitation we noted that cross-sectional "data could not trace the path of specific services through which homeless persons become housed" (17). Accumulation of longitudinal data on service use over 26 months allowed direct testing of hypotheses predicting service use patterns for achievement of stable housing—the single most important outcome variable for homeless persons.

For total service use, the patterns closely followed those hypothesized. Service involvement for clients in the housed group was greatest in the period immediately before and after they achieved housing, and use of services decreased sharply during the first few months after they became housed. We could directly test only the final two stages in the readiness model—the strategic moment and consolidation of change. However, the evidence we found supporting these two stages of change among clients who obtained housing lends credibility to the five-stage conceptual model of achievement of stable housing.

This study also examined the alternative hypothesis that all service use follows the same pattern over time, regardless of the achievement of outcomes. This hypothesis was not supported by the data, as the two-stage service use solution was not significant for the unhoused group. Rather, members of the unhoused group demonstrated a linear decrease in service use over time. This finding suggests the possibility that the two different patterns of decreases in service use by these groups may emerge from totally different experiences of the services provided. The decrease in service use by the housed group appears to reflect successful negotiation of the strategic moment, whereas the unhoused group's decrease in service use may be the result of dwindling activity over time after clients failed to seize the potential strategic moment.

This interpretation leads to a refinement of the conceptual model of change to incorporate two discrete pathways of service use among clients, depending on their success in obtaining housing. Individuals can either engage in services, negotiate strategic moments to achieve outcomes, and then consolidate these gains—stages 3, 4, and 5 of the conceptual model of change—or engage in services but fail to negotiate a strategic moment, and thus not achieve desired outcomes, after which they gradually disengage from services and return to the pre-engagement stage. Figure 2 illustrates these discrete pathways.

In a third possibility that was not tested here, individuals might engage in services and negotiate strategic moments to achieve outcomes but then not consolidate their gains. Figure 2 also illustrates this pathway.

Although the data did not allow direct testing of the reasons for the particular path of service use followed, previous research has suggested the importance of clients' willingness to negotiate with providers about services appropriate to desired outcomes (27). A possible explanation suggested by previous research by members of this team would be that a motivated client negotiates an appropriate match between the services offered by the agency and the services that the client perceives as necessary (29). A good match would be expected to significantly increase the client's likelihood of achieving lasting outcomes. Individuals who fail to establish a successful match for whatever reason—lack of motivation, lack of available appropriate services, or inability to negotiate a mutually acceptable service plan—would then gradually disengage from services over time.

The comparatively low rate of use of several service types allowed us to test only three of the eight services—the drop-in center, counseling, and health services. However, the results of these analyses support the validity of the conceptual model for at least a subset of the services offered. For clients in the housed group, use of the drop-in center and counseling services followed the hypothesized service patterns. Use of these two types of services may conform to the model because both services are based primarily on relationships rather than on provision of a product. Therefore, successfully developing relationships and using them to negotiate participation would appear to be directly linked to achieving housing outcomes.

Two possible explanations can be entertained for the failure of the third service type, health services, to conform to the two-stage model. First, although it was possible to develop a statistical model, the relatively small number of service units used may have provided insufficient variability to clarify the differences. Certainly the fact that the two-stage solution approached statistical significance (p=.06) is not inconsistent with the explanation offered here. In this explanation, service use would be expected to conform to the conceptual model, regardless of the type of service—at least for the single outcome of homelessness. Intuitively, however, this hypothesis appears to be somewhat simplistic.

A second explanation is that health services constitutes a class of services different from the other two. Health services are required regardless of housing status, whereas clients use the drop-in center and counseling services because they are homeless. Although the need may remain for counseling and support at other service settings for issues not directly related to homelessness, these services are offered in the direct context of addressing a person's homelessness. Because homeless persons have a greater risk of health problems, the nearly significant difference between the slopes of the two lines may represent an incidental trend related to greater access to these services rather than a pivotal moment in the achievement of stable housing. Individuals would have more access to health services at the time of greatest service engagement, and then after their general health needs were met, their use of health services would be expected to continue at a relatively constant rate. The almost level slope (−.03 services per month) supports this explanation for the second stage of the regression analysis for this variable.

The findings of this study have implications for service provision. First, the findings suggest the utility of a multiphase intervention, with lower intensity in the initial stages when relationships are being established, high intensity as soon as the match is achieved, and then a return to a lower intensity of service provision when outcomes are consolidated. Several programs described in the literature follow this pattern of service provision (7,24,26). In particular, Susser and colleagues' (24) robust findings for their "critical timing intervention" during transition to stable housing appear to provide validation for this pattern of service use.

A second implication of the model described here derives from the association between counseling and other relationship-based services and the achievement of housing outcomes. These findings belie the "just give the homeless housing" argument so prevalent in the media. The data are limited, allowing only speculation about whether relationship-based services are more or less important than housing services. Minimally, the data suggest the potential utility of including counseling and drop-in services in a service package designed to facilitate achievement of stable housing.

Finally, the conceptual model suggests that one common path to achieving housing outcomes may consist of repeated episodes of service use over time. Ideally, therefore, the provision of services should consist of multiple opportunities for engagement with a system that is sufficiently flexible to address the needs of a diverse and heterogeneous population (45). This approach is supported by strong evidence that homelessness often consists of multiple episodes (46,47,48).

As an initial attempt to test our modification of the transtheoretical model of change as it applies to homeless persons with mental illness, this study is not without important limitations. First, because the sample consisted of persons with mental illness, it was not experimentally randomized. It also was limited to a single service agency. Therefore the findings are not strictly generalizable either to the general homeless population or across all types of service providers. Second, we did not measure the variability in the amount of time spent in services used, both for the different types of services and within each service type. For example, a particular drop-in center session might last from 15 minutes to several hours. A further limitation of the study was its inability to provide separate analyses for all of the eight service types. Also, because the data on service use were limited to the month immediately before clients obtained housing, we could examine only two stages of the conceptual model—the strategic moment and consolidation. Service use patterns for the first two stages of the model, pre-engagement and contemplation, must still be tested. Finally, this study was limited to examination of only a single outcome, albeit the most critical outcome for homeless persons—obtaining stable housing.

Future quantitative research in this topic would benefit by the incorporation of some other features, such as random sampling from a defined geographical area; more precise measurement of service use; inclusion of data on multiple types of services across multiple providers; observation of individuals across the entire course of homelessness, from entry to final exit; and examination of other outcomes. Future research might include qualitative studies of individuals who achieve stable housing to specify in greater detail the mechanisms and evolution of the "strategic moment." Such qualitative research might then lead to additional hypotheses to be tested in quantitative research.

Acknowledgments

The authors thank Leo Paradis, M.A., and William Wyman, M.S.W., for their participation. The study was funded by grant MH-50857-03 from the Center for Mental Health Services Research of the National Institute of Mental Health.

Dr. Pollio and Mr. Foster are affiliated with the George Warren Brown School of Social Work at Washington University, Campus Box 1196, St. Louis, Missouri 63130-4899 (e-mail, [email protected]). Dr. Spitznagel is with the department of mathematics and biostatistics at the university, and Dr. North is with the department of psychiatry at the university's School of Medicine. Dr. Thompson is affiliated with the School of Social Work of the State University of New York at Buffalo.

Enlarge table

Enlarge table