Assessing the Impact of Community-Based Mobile Crisis Services on Preventing Hospitalization

Mobile crisis services are widely considered to be an integral part of an effective emergency mental health services system (1,2). Such services are thought to reduce hospitalization rates by diverting patients from hospital admission into community-based treatment (3).

Although the modern concept of mobile crisis services is about a quarter century old and services are widely available throughout the United States, little effort has been directed to formal evaluation of mobile crisis programs (4,5). Most published studies are descriptions of individual programs or surveys of programs at the state or national level.

Programs described in the literature have varied on several dimensions. Some were community based, and others were operated out of hospitals. Services provided ranged from crisis intervention to medication administration and multiple follow-up visits. Service hours ranged from normal business hours to around the clock. Services were provided by various combinations of mental health workers, including psychiatrists, social workers, nurses, case managers, and "indigenous homemakers."

Most of these descriptive studies discussed advantages or positive results of mobile crisis services. Using results of a national survey of state mental health agency directors, Geller and colleagues (4) divided the purported effects of mobile crisis services into three categories of benefits according to the locus of impact: consumers and families, service providers, and the mental health system.

Several studies have examined the degree to which mobile crisis programs reduced consumers' hospitalization rates (6,7,8,9,10,11). The studies used a variety of methodologies, including examining a mobile crisis treatment group only (6,7,8), time-series comparisons of hospital admissions before and after the introduction of the mobile crisis program (10,11), and a comparison of hospitalization rates in catchment areas with and without mobile crisis service programs (9). No published study has compared the effects of different types of interventions—such as community-based versus hospital-based programs—on hospital admissions, and none has investigated which consumer characteristics are associated with a greater risk of hospitalization.

Hence two major shortcomings in the literature need to be addressed. First, although mobile crisis services are widely assumed to reduce hospital admissions, evidence about the effectiveness of such programs is limited by the methodological shortcomings of previous studies. Either they lacked a rigorous study design that controlled for extraneous factors, or they employed qualitative approaches based on experts' observations and case studies. Second, little is known about the characteristics of mobile crisis services consumers who are more likely to be hospitalized.

To help address these shortcomings, this study examined the effectiveness of community-based mobile crisis intervention as compared with hospital-based intervention in preventing hospital admissions. Because an experimental design randomly assigning study subjects into different interventions could not be undertaken, a quasi-experimental design with an ex post matched control group was used to approximate random assignment (12,13). A second purpose of the study was to determine the characteristics of mental health services consumers most likely to be hospitalized by using a multivariate statistical analysis focusing on the timing of hospitalization.

Methods

Setting

On July 1, 1996, the county mental health authority covering a large Midwestern industrial metropolitan area instituted a centralized approach to the delivery of psychiatric emergency services. The new approach included a hotline, a community-based mobile crisis program, and authorization for state hospitalization under the administration of a single agency. Before mobile services were implemented, people experiencing a mental health crisis went to a hospital-based psychiatric emergency room. The emergency room had authority to approve admission to the state hospital. The change in service delivery was accomplished by shifting funds from the emergency room to the mobile crisis program, with no change in cost to the system.

Mobile crisis services were provided by an interdisciplinary team composed of crisis intervention specialists, registered nurses, and psychiatrists. The team was managed by a community mental health agency that contracted with the county mental health authority. The crisis intervention specialists were licensed mental health professionals who provided diagnostic assessment, prehospital screening, and case management services in addition to their crisis intervention duties. Psychiatric and nursing staff provided medical services and conducted diagnostic assessments. The psychiatrists also provided supervision and screened patients for possible hospitalization. When a call came in, the mobile crisis team would review the case, send out team members to see the person, attempt to stabilize the crisis, recommend appropriate services, and provide follow-up. The recommended services could include referral and linkage to additional community services or hospitalization.

Study design

To determine whether adult consumers who used the community-based mobile crisis intervention services were more or less likely to be hospitalized than those who used hospital-based crisis services, we compared the rate and timing of hospitalization of two cohorts of crisis service consumers. The hospital-based intervention cohort (N=4,372) included all consumers who received face-to-face, hospital-based crisis services during the first 11 months of fiscal year 1995. The community-based intervention cohort (N=1,757) included all consumers who received community-based mobile crisis services during the first 11 months of fiscal year 1997. Each consumer was tracked for 30 days after the initial crisis service to determine whether a hospitalization occurred during that period, and if so, when. Consumer characteristics were examined in relation to hospitalization to determine what factors were associated with a higher likelihood of hospitalization.

To control for self-selection we used a quasi-experimental research design with an ex post matched control group (13). In this design causal inference is drawn by matching the two groups on relevant characteristics and assessing the net difference between the groups on the outcome variable. If the matching variables are appropriately chosen, the differences in the outcome variable can be attributed to the program intervention with a high degree of confidence.

After deleting a total of 327 cases with missing data (266 cases, or 6.1 percent, from the hospital-based cohort and 61 cases, or 3.5 percent, from the mobile community-based cohort), we matched 1,696 consumers from the mobile services cohort with 4,106 consumers from the hospital-based intervention cohort on seven variables: gender, race, age at the time of crisis service (in five-year intervals), primary diagnostic group, recency of prior service use, indication of substance abuse, and certification of severe mental disability. Certification of severe mental disability is based on a diagnosis of a severe illness, the duration of the illness, and the degree to which the illness interferes with the person's functioning.

The matching variables were chosen on the basis of prior knowledge and theoretical understanding of factors affecting mental health service use. Other than the introduction of mobile crisis services, no changes occurred in the service system configuration between 1995 and 1997 that would be expected to influence the diversion of cases from psychiatric hospitalization.

A match-merge process was used to combine the data for the two cohorts. Both data sets were sorted and matched on the seven variables listed above. Whenever a consumer from the community-based intervention group was matched with more than one consumer from the hospital-based intervention group, we randomly selected one case from the multiple matches to create the comparison group. This procedure generated a data set with 1,100 persons in each cohort.

The data for these analyses were drawn from the administrative databases available to the county's mental health authority. Demographic and crisis service utilization data were derived from the community service reporting system. Hospitalization data came from state hospital records and from Medicaid records for persons served by private hospitals.

Statistical analysis

We employed a Cox proportional hazards model to analyze the rate and timing of hospitalization (14). This model is well suited for analyzing a data set that contains censored cases. The outcome variable we evaluated was the hazard rate of hospitalization within the 30-day window. Those who were not hospitalized within 30 days of initial contact constituted the censored cases.

The Cox proportional hazards model defines an unobserved variable, commonly known as a hazard rate, as the dependent variable. The hazard rate translates the length of time it takes an event to occur—in this study, hospitalization—into a rate. The longer a consumer remains out of the hospital, the lower the hazard rate.

Like a multiple regression analysis, the Cox proportional hazards model examines the relationship between an independent variable and the hazard rate while controlling for other independent variables. By including the group membership variable in the proportional hazards equation, we were able to assess the degree to which use of the community-based mobile crisis intervention program prevented hospital admissions. In addition to group membership, we included in the analysis the covariates gender, age at the initial crisis contact, race, living arrangements, primary income source, referral source, primary presenting problem, primary diagnostic group, recency of prior service use, indication of substance abuse, and severe mental disability certification status. These covariates were selected on the basis of prior studies.

Risk ratios were calculated to express the relative risk of hospitalization for each independent variable. Thus the risk ratio for group membership is the relative risk of hospitalization for the community-based mobile services cohort compared with the hospital-based cohort, controlling for all other independent variables. We also used the model to predict the proportions of people in each cohort who remained out of the hospital at each point during the 30 days. We used the PHREG procedure of SAS software (release 6.12) to conduct the analyses (15).

Subjects

The study included 2,200 matched subjects, with 50 percent in each cohort, as described above. All subjects had valid information on all study variables. The overall sample consisted of more males than females and more African Americans than Caucasians, with a mean age of about 36 years at initial contact (see Table 1).

The two cohorts did differ in some characteristics. As a result, only 1,100 of the 1,696 potential subjects in the community-based mobile crisis intervention cohort (65 percent) had matches in the hospital-based comparison group. As in previous studies that found differences between users of the two approaches, consumers for whom matches were found were younger (16,17) and less likely to be certified as severely mentally disabled (8,16,18). In addition, the match group contained fewer males and fewer individuals who were neither African American nor Caucasian than the community-based cohort. Since the primary concern of the study was the comparison of the two approaches, it was necessary to sacrifice some of the study's external validity through the matching process to maximize internal validity.

Results

Impact on hospitalization

Results show that 32.6 percent of the matched hospital-based treatment intervention group were hospitalized within 30 days after an initial-crisis contact, as compared with 24.2 percent of the community-based intervention treatment group. The Cox proportional hazards model fits the data well (χ²=263.23, df=33, p<.001). When all other independent variables were controlled for, a consumer in the hospital-based intervention group was 51 percent more likely to be hospitalized than a consumer in the community-based mobile crisis intervention group (p<.001). A hierarchical proportional hazards analysis that entered covariates before group membership indicated that group membership was a strong predictor of whether or not a person would be hospitalized.

To investigate the timing of hospitalization more fully, the model was used to generate the predicted proportions of consumers not hospitalized on each day within the 30-day window for each cohort. These predictions were obtained by inserting appropriate values of the group membership variable into the model and holding all other independent variables constant at their mean values.

Most hospitalizations occurred in the first two days for both groups, with some 26 percent of the hospital-based comparison group and 18 percent of the community-based intervention group being hospitalized by day 2 (Figure 1). The differences between the two groups in the slopes of proportions of persons not hospitalized were statistically significant (p<.001). In both groups the crises were typically resolved or a disposition of the case was made during the two-day period.

A standard measure of the quality of crisis services is an analysis of the number of cases in which the person ends up being hospitalized after an initial decision not to hospitalize. About 4 percent of subjects in both the mobile community-based and the hospital-based groups were hospitalized during days 3 through 30. Thus, although the mobile crisis approach was less likely to hospitalize consumers, the decision not to hospitalize had no adverse consequences.

To evaluate whether the declining use of hospitalization during the 1990s acted as a historical effect to threaten the internal validity of the study, we analyzed the hospitalization trends of both public and private psychiatric hospitals. This analysis showed that although hospitalization episodes declined from 1992 through 1995, the period when the mental health system under study used a hospital-based psychiatric emergency service, these reductions amounted to less than 2 percent per year. In 1996 and 1997, after the community-based mobile crisis services program was implemented, hospital episodes decreased at a rate of more than 8 percent per year. Projection from the regression equation for utilization from 1992 through 1995 indicated more than 500 fewer hospital episodes in 1997 than would have been predicted from the historical trend.

Consumer characteristics related to hospitalization

Next we examined variables associated with the likelihood of being hospitalized, again using risk ratios. The significance of the effect of each independent variable on hospitalization was examined by controlling for all other independent variables.

Several variables were significant at a p value of less than .05. Consumers who were referred by the legal system were nearly twice as likely as self-referred consumers to be hospitalized, and those referred by a psychiatric hospital or other treatment facility were more than twice as likely as self-referred consumers to be hospitalized. Consumers whose primary diagnosis was schizophrenia, affective psychosis, or other psychosis were more than twice as likely to be hospitalized as those whose primary diagnosis was an alcohol or drug disorder.

Consumers whose primary presenting problem was a suicidal gesture or attempt or agitation or anxiety were nearly twice as likely to be hospitalized as those with substance abuse as the primary problem, and those whose primary presenting problem was delusions or hallucinations were more than twice as likely to be hospitalized. The risk ratio was also high for those with depressed mood.

Consumers with an indication of substance abuse and those with severe mental disability status were more likely than those without to be hospitalized. Those with no source of income or whose primary source of income was specified as other were more likely to be hospitalized than employed consumers, and those whose living arrangements were specified as other were less likely to be hospitalized than those who were homeless.

Discussion and conclusions

The study found that a matched sample of consumers who used hospital-based crisis services were 51 percent more likely to be hospitalized, after other variables had been controlled for, than users of community-based mobile crisis services. The study thus provides empirical evidence supporting the widespread belief that mobile crisis services reduce hospital admissions. This result is consistent with those of five of six published studies that evaluated the degree to which mobile crisis programs reduced hospital use (6,7,8,10,11). However, these studies did not compare alternative approaches, as we did in this quasi-experimental investigation. The program under study included characteristics that had previously been found to be critical to reduced hospitalization, including the ability to prescribe medication (7,11) and the active participation of a psychiatrist (10). After implementation of the mobile crisis intervention program, 24 percent of those who received the service were hospitalized, which is below the range of 27 percent to 34 percent found in previous research (7,8,19,20,21).

Reduction in hospitalizations is a frequently cited goal of crisis intervention programs. However, because in many cases hospitalization is the best outcome, it must be remembered that the goal should actually be to reduce unnecessary hospitalizations. One quality assurance measure of crisis intervention services is to determine the percentage of consumers who were not hospitalized at the point of the crisis event but who required hospitalization within a prescribed follow-up period—typically 30 days. The analysis of hospitalizations in days 3 through 30 revealed that the lower likelihood of hospitalization during the first two days for the mobile community-based approach did not increase the risk of a subsequent hospitalization. Thus, although it appears that this approach was successful in preventing hospitalization of persons whose crises could be effectively resolved through community-based treatment, a causal relationship cannot be ascertained because of the limitations of the quasi-experimental design.

The study found that some consumer characteristics were related to risk of hospitalization. Consumers more likely to be hospitalized were young and homeless; had acute psychiatric problems; were referred by psychiatric hospitals, the legal system, or other treatment facilities; had a substance abuse indication; had no income; and were severely mentally disabled.

Evaluating a large-scale program intervention is challenging. A quasi-experimental design partially resolves problems threatening internal validity but nevertheless has other limitations. First, since consumers were not randomly assigned to treatment or control conditions, comparability between groups was created by matching subjects on the basis of variables thought to be related to hospital use. Adding or dropping control variables in the matching process could alter the findings.

Second, this study employed mental health administrative data, which did not include all of the variables that might have been included had the study employed primary data collection. The acuity of the crises was not measured, for example. Acuity can only be inferred from diagnoses and presenting problems. Future studies evaluating mobile crisis intervention need to include these variables.

Acknowledgments

This study was supported by the Cuyahoga County community mental health board. The authors thank Sam MacMaster, Ph.D., and Meeyoung Oh Min, Ph.D., for their assistance in literature review and data analyses.

Dr. Guo is associate professor of social work in the College of Social Work at the University of Tennessee in Memphis. Dr. Biegel is Henry L. Zucker professor of social work in the Mandel School of Applied Social Sciences of Case Western Reserve University in Cleveland. Dr. Johnsen is director of research and evaluation and Dr. Dyches is grants coordinator of the Cuyahoga County Community Mental Health Board in Cleveland. Send correspondence to Dr. Biegel, Mandel School of Applied Social Sciences, Case Western Reserve University, 10900 Euclid Avenue, Cleveland, Ohio 44106 (e-mail, [email protected]).

Enlarge table