Psychiatric emergency visits lead to suboptimal treatment outcomes compared with admissions not associated with urgent care (1,2), impede others from receiving needed emergency services (3,4), and impose a substantial economic burden on the health care system (5). In the United States, mental health-related complaints account for 5.3 million annual emergency department visits. Researchers have argued that persons with a mental disorder could avoid psychiatric emergency treatment if they instead received proper outpatient or community-based care (6,7). Moreover, a study of patients with nonurgent conditions but seen in the emergency setting indicates that improved access to a primary care physician could reduce the number of unnecessary emergency visits (8). Taken together, the literature implies that changes over time in the incidence of emergency psychiatric admissions may sensitively gauge the overall functioning of the mental health care system.

In 2004 California voters approved the ballot measure Proposition 63 (Mental Health Services Act, referred to as MHSA) to expand public mental health funding and services in all counties. A 1% tax on adjusted gross income over $1 million, which affects approximately 30,000 taxpayers, provides the source of funding (9). By law, MHSA prevents counties from offsetting mental health budgets by reducing or supplanting mental health funding from previous levels. This landmark legislation intended to expand innovative recovery-oriented and client-driven services to individuals with a severe mental illness or a serious emotional disturbance (9). As of the end of fiscal year (FY) 2006–2007, the MHSA had distributed approximately $647.7 million in new dollars to augment county mental health programs (10). These funds represent about a 10% increase in county mental health budgets (11).

The State of California disbursed MHSA funds with the condition that counties allocate at least 51% toward addressing new and expanded services through full-service partnerships (FSPs) (11). These partnerships are modified assertive community treatment programs that provide intensive case management and services such as housing, employment, education, peer support, treatment of co-occurring mental and substance use disorders, and outreach. The California Department of Mental Health (DMH) stipulates that only persons at high risk (“at risk”) can gain admission into FSPs. DMH defines this high-risk category for adults as meeting the three following criteria: qualifies for public assistance; diagnosed as having a severe mental illness; and viewed by county officials as homeless or at risk of becoming homeless, involved with or at risk of involvement with the criminal justice system, a frequent user of mental health-related emergency services, or at risk of being institutionalized (additional criteria also apply depending on age) (12). Clients who qualify for FSPs represent high-risk and often homeless persons with disproportionately high levels of need (for a complete list of the criteria used to determine eligibility for FSPs, see the California Code of Regulations) (12).

In Los Angeles County, FSP clients have a point of contact, available at all hours, to provide services (13). These services involve clinic and in-home care, peer support programs, and housing and employment assistance. The allocation of MHSA funds to clients with high levels of need adheres to the strategy of targeting resources to a distinctly identifiable high-risk minority so that resources accrue the greatest societal and health benefits (14,15).

A recent evaluation of voluntary use of emergency psychiatric services after MHSA funds were disbursed focused on the subset of persons (>6% of total population served by MHSA) enrolled in FSPs (16). The authors used a longitudinal data set of FSP clients for seven California counties and applied a conditional logistic regression approach. They reported that after one year in the FSP, clients showed a significantly reduced likelihood of mental health-related emergency visits. Longer membership in the FSP further reduced the likelihood of emergency visits. This finding indicates that MHSA's programmatic focus on a high-risk population may channel these clients toward more effective outpatient and community-based treatment.

Brown and colleagues' (16) evaluation, although encouraging, did not address the important policy question of whether the population-level rate of mental health-related emergencies fell after the implementation of MHSA. No change, or even an increase, in the population incidence of psychiatric emergencies could occur after disbursement of MHSA funds. This circumstance could arise from the “crowd-out” problem (17), whereby focused benefits for a small group at high risk of emergency care inadvertently shift programmatic resources away from the larger population at low or medium risk of emergency care (that is, the remaining 94% served by MHSA). Whereas high-risk FSP clients may benefit from the MHSA with reduced emergency visits, a stagnant or even slightly increasing rate of visits among the low-risk population may result in a relatively stable population rate of emergency visits. This response can occur even if all counties adhere to the MHSA's restriction on reducing or supplanting care. Knowing the net effect of MHSA on psychiatric emergency visits holds implications for economic and health policy given that public policy decisions often turn on net-effects considerations (18).

We analyzed whether the overall rate of psychiatric emergencies has fallen since the implementation of MHSA. We examined Los Angeles County, the most populous county in California and in the United States. We used time-series methodology and controlled for potentially confounding variables that could induce spurious associations. Results may hold policy implications for reducing the demand for emergency mental health care.

Variables and data

We retrieved psychiatric emergency visit data from Medi-Cal, California's version of the federal Medicaid program. California's DMH provided us specialty mental health Short-Doyle Medi-Cal (SD/MC) claims files for 96 consecutive months beginning in July 2000 and ending in June 2008. These data constituted the longest series with consistent collection and reporting methodology available at the time of our study.

The SD/MC system consists of a group of related local assistance programs and activities associated with Medi-Cal behavioral health claims. Statewide, the SD/MC system processes approximately one million specialty mental health care claims per month for Medi-Cal-eligible persons aged 18 to 64 years. Counties and health care providers file these mental health claims directly with the state DMH. Because these claims are reimbursable to the county or the provider or both, reporting of emergency visits in this population is believed to be almost 100% complete. The data set includes all persons with psychiatric diagnoses (including schizophrenia, bipolar disorder, substance use disorder, depression, anxiety disorder, personality disorder, and other mental disorders) who seek care under the auspices of Medi-Cal insurance. We used deidentified, aggregate-level data; therefore, informed consent was not required. The State of California, Committee for the Protection of Human Subjects, approved use of this data set for our study (protocol 06-10-59).

SD/MC data do not contain records of coerced psychiatric emergencies (that is, involuntary civil commitments). We therefore examined only voluntary emergency admissions. We defined an emergency service on the basis of the “Crisis Stabilization—Emergency Room” function listed under the SD/MC cost reporting code. We derived the monthly odds of psychiatric emergency visits by dividing the incidence (monthly proportion of Medi-Cal emergency visits among all persons using any Medi-Cal mental health service) by 1 - incidence.

We focused on Los Angeles County for three reasons. First, Los Angeles has the largest public mental health system of any county in the United States. Its size yields generally stable measures over time of the odds of psychiatric emergency visits—an important consideration for our analytic procedures (described below). Second, Los Angeles received a relatively large initial MHSA disbursement (>$69 million) within the range of years for which we have data, which could permit detection of a service response at the population level. Third, unlike the vast majority of California counties, Los Angeles made no apparent administrative change in the reporting of voluntary emergencies over our test period. Such administrative changes, although often legitimate, could induce a sharp decline or rise in the rate of visits identified as emergencies in the SD/MC data. Our focus on Los Angeles precluded the possibility that administrative shifts in coding could induce a spurious association between emergency visits and disbursement of MHSA funds. We note that our focus on Los Angeles in no way reflects unfavorably on the quality of MHSA programs in other counties.

California approved and funded Los Angeles' MHSA application in February 2006. We classified the MHSA variable as a binary indicator, coded as 1 in February 2006 and 0 for all other months. We, as described below, used previous literature to guide the temporal lag in which we specified the hypothesized relation between MHSA funds and psychiatric emergencies.

Analysis

Support for our hypothesis turned on whether voluntary psychiatric emergency visits in Los Angeles fell below expected levels after the disbursement of MHSA funds. Emergency visits in Los Angeles, however, may exhibit temporal patterns (such as trend) from 2000 to 2008 (19,20). Temporal patterns, collectively referred to as autocorrelation, could complicate our analysis because the expected value of emergency visits in late 2006 (that is, after MHSA funding) may not equal the mean value of previous months.

We applied autoregressive, integrated, moving-average (ARIMA) time-series methods, developed by Box and colleagues (21), to address this autocorrelation issue. Time-series routines identify and remove autocorrelation in the dependent variable before examining the effect of the independent variable. This process yields a time series of residual, or unexpected, components whose monthly values have no temporal pattern and are statistically independent of one another. In regard to our analysis, we applied ARIMA methods to the history of emergency visits to arrive at the expected value of visits in FY 2006–2007.

We used the routines devised by Box and colleagues (21) to identify and remove autocorrelation in the monthly counts of psychiatric emergency visits for the 96 months beginning in July 2000 and ending in June 2008. These routines could detect nonstationarity or seasonality or both among the voluntary psychiatric emergency visits. The routines were used to model these patterns as well as the tendency of a series to remain elevated or depressed after high or low values. The time-series methods modeled trends by examining differences within a series (in other words, by subtracting the values of each month from those of the next month). Similarly, if we detected seasonal cycles, we used a differencing operator of 12 (for example, subtracting values for each month from those 12 months later). The Box and colleagues approach uses autoregressive and moving-average parameters to model other forms of autocorrelation. Autoregressive parameters best describe patterns that persist for relatively long periods, whereas moving-average parameters parsimoniously describe less persistent patterns.

A key statistical assumption of our method involves constant error variance of the time series. To ensure that emergency visits met this assumption, we log-transformed the monthly values. Log transformation, as with other power transformations, tends to harmonize error variance (22). This process also confers the benefit of interpreting the antilog of the MHSA coefficient as a familiar “effect on odds” (or odds ratio) metric.

Two evaluations of emergency services in California after MHSA was enacted appear in the literature. The first discovered reduced emergency visits among a high-risk group (6% of all clients) as of eight months post-MHSA funding (16). The second report showed fewer involuntary 14-day holds between three and 12 months after MHSA's implementation (23). Consistent with these findings, we specified a delayed effect of three to 12 months between MHSA implementation and a reduction in voluntary emergency visits.

We attempted to minimize confounding bias by including several control variables. We first included the county's capacity to provide emergency care. Inadequate funding for hospitals that provide emergency care may lead to closure of such hospitals, which may result in changes in visitation patterns if Medi-Cal clients seek care outside Los Angeles. To control for this possibility, we included the count of hospitals with an emergency medical treatment station (24). In addition, rainfall may interfere with access to care, which could either increase or decrease emergency visits (25). We created a “high precipitation” variable, consistent with the literature, by scoring months with values greater than the 75th percentile of rain over the entire study period (that is, 1.38 inches) by inches of precipitation. We scored all months with less than the 75th percentile value of rain over the entire study period as 0 (26).

Regional economic contraction reportedly precedes an increase in psychiatric emergency treatment (26,27). Failure to account for economic change could bias results if sudden gains in employment coincided with the disbursement of MHSA funds. We therefore included as a control variable the monthly number of unemployed persons in Los Angeles over the 96-month period (28). Changes in unemployment over time gauge the extent to which regional economies stagnate or contract.

The analysis proceeded as follows. First, we identified and removed autocorrelation in emergency visits over the test period. Second, we added the three control time-series variables (count of emergency hospitals, high precipitation, and unemployment) and the binary MHSA variable to the model. Third, we estimated the MHSA variable's association with emergency visits three to 12 months after the disbursement of MHSA funds. Fourth, we inspected the unexpected (residual) components of emergency visits to ensure that they displayed no temporal patterns.

As a robustness check, we also examined whether outliers in emergency visits distorted our results (29). Outliers in the time series could artificially inflate standard errors and lead to a type II error (false acceptance of the null). We therefore used outlier detection and correction routines as recommended in the time-series literature. We used software from Scientific Computing Associates (30) to apply the outlier detection and correction method, proposed by Chang and colleagues (29), to the regression model. This method iteratively adds binary variables for each year to find any that, if added to the equation, would have coefficients with t values greater than 3.5. The method also adjusts the ARIMA parameters as outliers are added. Using the full equation (with covariates of unemployment, high precipitation, and available hospital emergency departments), we specified that the outlier routines not control for the negative outliers we hypothesized a priori (three through 12 months after disbursement of MHSA funds).

Over the eight-year period, Los Angeles reported an average of 61,450 clients per month with a mental health-related Medi-Cal claim. Figure 1 shows the raw count of mental health-related emergency visits in Los Angeles County over the 96 months. We identified a mean of 493 visits per month as psychiatric emergencies. Figure 2 shows the odds of psychiatric emergency visits for the same period. Less than 1% of Medi-Cal clients who used mental health care sought psychiatric emergency care (mean odds=.008). The three lowest monthly odds of emergency visits over the entire test period occurred eight, ten, and 12 months after disbursement of MHSA funds (that is, at months 76, 78, and 80).

Time-series routines did not detect nonstationarity or seasonality in monthly emergency visits; therefore, we required no differencing operator. However, we identified patterns in visits best characterized by autoregressive parameters at lags of one and two months. This pattern implies that high or low values of emergency visits were followed by similarly high or low values one and two months later but in diminishing amounts (autoregression 1 coefficient=.386, SE=.097; autoregression 2 coefficient=.344, SE=.096).

Table 1 shows the results of the time-series models that included the MHSA as well as control variables. Consistent with earlier reports, the initial model showed that unemployment varied positively with emergency psychiatric visits, whereas high precipitation varied negatively with these visits. Three of the ten months after MHSA funding (eight, ten, and 12 months after MHSA implementation) were negative and statistically significant. The odds ratios (antilog) of the significant coefficients ranged from .81 to .83 (not displayed).

We then proceeded with the routines for outlier correction (29). The routines detected five outliers; results that adjusted for these outliers are shown in Table 1. Control of outliers strengthened statistical inference for coefficients at eight, ten, and 12 months after MHSA funding relative to the original analysis and uncovered a negative relation 11 months after MHSA funding (coefficient=−.123, SE=.053, p<.05). Inference for all other MHSA coefficients did not change.

To give the reader a sense of the magnitude of the discovered effects, we estimated from the outlier-adjusted results the reduced number of emergency psychiatric visits statistically attributable to the MHSA. Applying the discovered coefficients at eight, ten, 11, and 12 months after MHSA funding to the population served by Medi-Cal mental health services resulted in an estimated 412, 410, 437, and 415 emergency visits for these respective months. Subtracting these counts from the monthly mean count of 493 visits yielded a total of 306 fewer psychiatric emergency visits statistically attributable to MHSA funds. Multiplication of these averted visits by the average medical inflation-adjusted cost of an emergency visit ($554 in current dollars) yielded an estimated cost savings of $169,524 (31).

We examined whether results appeared sensitive to the choice of the natural log-odds scale for emergency visits. We converted the dependent variable back to its original incidence measure and repeated the analysis. Other than the metric of the coefficient, results remained essentially the same.

Although the research literature compelled us to focus our analysis on the three to 12 months after disbursement of MHSA funds, it remains possible that emergency visits may have begun to decline earlier than three months or continued to remain low after 12 months. We explored this possibility by examining delays zero through three months and (separately) 13 through 18 months. None of these coefficients differed significantly from the null. In addition, we assessed whether psychiatric emergencies exhibited a “step” function such that the mean odds for all 28 months after disbursement of MHSA funds fell below the mean odds for the 68 months before MHSA funds. Time-series results did not support such a stepwise decline (MHSA coefficient=.02, SE=.04, p=.61).

The MHSA endeavors to transform the system of mental health care in California to a patient-centered, recovery-oriented model. One of the several indicators of the overall functioning of the system of care involves the rate of emergency psychiatric visits. We used data from Los Angeles County to analyze whether MHSA funds preceded a decrease in voluntary emergency psychiatric visits. Time-series analyses support the hypothesis in that psychiatric emergencies among Medi-Cal patients fell below expected levels in the months after disbursement of MHSA funds. However, we did not observe a sustained decline one year after disbursement began. Findings offer partial support for the argument that an infusion of public funds devoted to mental health services improves the overall system of public mental health care.

Strengths of our analysis include control for county-level variables known to affect the monthly incidence of emergency department visits (26). Our time-series methods, moreover, preclude spurious associations between MHSA and emergency visits as a result of shared trends or other temporal patterns because we identified and removed all autocorrelation in emergency visits before examining the effect of MHSA. Examination of the incidence, rather than the raw counts, of psychiatric emergencies among persons insured by Medi-Cal also controlled for the increase from 2000 to 2008 in the size of the population seeking services under the auspices of Medi-Cal.

Whereas previous research has indicated a reduction in emergency department visits among high-risk FSP clients (16), a key concern of this strategy involves a potential commensurate reduction in the quality or quantity of mental health resources for the remaining 94% of clients (14). Our results, which are based on voluntary psychiatric emergency visits as a gauge of the overall functioning of the system of care, indicate no such tradeoff. We observed, for the entire Medi-Cal population seeking mental health care, a modest net reduction in emergency visits.

We discovered a delayed, rather than an immediate, decline in emergency visits after MHSA was in place. Discussions with staff at the Los Angeles DMH suggest a post hoc explanation for this delay. The initial MHSA disbursement mobilized mental health worker volunteers and promoted organizational change in designing integrated services. These efforts, however, required extensive training and staff engagement, such that the volunteers did not implement these changes “in the field” until late 2006—approximately eight months after disbursement of MHSA funds.

Whereas the MHSA may have, in the short term, lowered the population-level incidence of emergency department visits, this rate returned to its long-term mean after 12 months. This shift could have arisen from several potential factors. First, outreach to homeless clients with severe mental illness, as part of the MHSA program or through independent efforts, may have slightly increased the demand for emergency care. Second, because excess emergency department use serves as one criterion for entry into the FSP program, some individuals may have used more emergency services than they otherwise would have to increase the likelihood of gaining entry into an FSP (16). Third, individuals with severe mental illness and living in surrounding counties may have sought high-quality mental health care in Los Angeles, which implemented its integrative care team earlier than did adjacent areas. Fourth, budget realignments that coincided with economic recessions may have augmented prevention and early-intervention services but lowered the level of resources devoted to treatment of clients with severe mental illness. We caution the reader that these claims represent informed speculation. We encourage closer investigation of Los Angeles' system so that policy makers may learn how to devise strategies that lead to sustained declines in emergency visits into the future.

Data limitations did not allow us to examine mental health-related emergency visits among the populations not insured by Medi-Cal. We caution the reader against generalizing our results to the uninsured, or privately insured, population. We also could not compare emergency responses in California counties outside Los Angeles after MHSA funding owing to inconsistencies in data collection or administrative changes from 2000 to 2008 in the classification of these visits. Lack of a comparison population from another region in our analysis also allows for possible bias resulting from unmeasured confounding. These limitations imply that our results pertain to the Medi-Cal population in Los Angeles and may not represent the state-level response to MHSA in regard to psychiatric emergencies.

Accumulating evidence indicates beneficial treatment outcomes and system-level improvements within one year of MHSA funding (16,23,32). Although the cost-savings estimate due to averted emergency visits appears modest, we await comprehensive cost-effectiveness analyses of all population benefits that will inform the policy debate regarding the economic and societal value of MHSA's implementation. In addition, we encourage long-term evaluation of the public mental health system to assess whether the initial benefit of MHSA continues well into the future.

This study was supported by contract 08-78106-000 from the State of California Department of Mental Health. The authors thank Sun-Soon Choi, Ph.D., for her assistance with data preparation and management. The authors also gratefully acknowledge Marvin Southard, D.S.W., Debbie Innes-Gomberg, Ph.D., and Tony Beliz, Ph.D., at the Los Angeles Department of Mental Health for their insight regarding the interpretation of the findings.

The authors report no competing interests.