Assessing the Long-Term Effectiveness of a Behavioral Health Home for Adults With Bipolar and Psychotic Disorders

Adults with serious mental illness face a 20- to 30-year mortality gap (1–3) driven by poor access to high-quality primary care (4–10), metabolic side effects of antipsychotic medications (11–13), and other factors (14–16) that increase risk for cardiometabolic disease (17, 18). Behavioral health homes (BHHs) help mitigate these mortality disparities by integrating primary care and wellness services into behavioral health clinics and enhancing care coordination via care management (1–3, 16, 19–27).

A 2018 systematic review found that BHH models improved cardiometabolic screening but that health impacts were limited (28). Some studies reported improvement in general medical health (GMH) outcomes (23), and others found improved GMH process measures but not outcomes (24, 29). Maryland’s Medicaid Medical Home model showed reduction in emergency department (ED) visits for GMH (30), but the study had small effect sizes and showed no observed effect on inpatient admissions. Because early BHH demonstrations were promising (9, 23, 31–33), adoption of these models has increased. As clinic and hospital systems adopt value-based payment models and shift to population health approaches for designing clinical care, interest has increased in understanding BHH’s impacts on health and service utilization.

Few longitudinal evaluations have been conducted on BHH implementation. Many studies also lack a comparison group or emphasize vulnerable subpopulations (23). To close these literature gaps and understand the BHH’s potential to improve health for individuals with serious mental illness, we analyzed longer-term (3.5 years) impacts on health care utilization among all participants and vulnerable subgroups as well as metabolic screening and cardiometabolic rates. A prior 1-year evaluation found that BHH participants had improvements in hemoglobin A1c (HbA1c) screening rates and marginal improvement in lipid monitoring rates and had fewer psychiatric inpatient and total ED visits (34).

Methods

Setting and Intervention

A BHH was established in September 2015 at a community-based behavioral health clinic in an urban, safety-net health system caring for about 140,000 patients annually. Patients with primary psychotic or bipolar disorders (plus any antipsychotic prescription) receiving care at the clinic were automatically enrolled in the BHH on September 1, 2015. Referrals within the health system were accepted on the basis of set criteria for psychiatric diagnosis, medical risk, and care coordination needs. The BHH sought to increase access to primary care through intensive referrals and service colocation, but neither primary care visits, nor any other kind of visit for GMH care, were required for participants. (For key BHH components, see part 1 in an online supplement to this article.)

Study Period and Sample

The BHH implementation start date was September 1, 2015; our observation period included service encounters 1 year preintervention (September 1, 2014–August 31, 2015) and 3.5 years postintervention (September 1, 2015–February 28, 2019).

Our study sample consisted of all patients receiving care in the health system during the observation period before October 31, 2015, with diagnoses of either primary psychotic disorders or bipolar disorder accompanied by an antipsychotic medication. Data from patients whose BHH enrollment was documented in the first 2 months of enrollment were included in this analysis (N=413). Those enrolled after this date (N=304) were included in a sensitivity analysis (see part 2 in the online supplement). Our control group consisted of patients from the same health system diagnosed as having serious mental illness who never enrolled in the BHH and who had at least one specialty behavioral health visit in the pre- and postintervention periods.

Electronic health record data were accessed for individual-level demographic characteristics (age, sex, race, language spoken at home, marital status), medical information (diagnosis, prescriptions, health insurance type), and health encounter information (date and type, lab values). Area-level variables (percentage foreign born, percentage living in poverty, percentage of female-headed households, and percentage with less than a high school education) were sourced from census data on individual zip codes or were imputed for patients with missing zip codes by using the “mi impute” command in Stata (35).

Statistical Analysis

We employed multigroup interrupted time series analysis (ITSA), which compares pre- and postintervention trends from longitudinal data between the treatment sample and a representative control sample. The control sample adjusted for exogenous shocks that could influence outcomes of interest.

To adjust for differences between samples, we used inverse propensity weighting (IPW) to weight individuals from the control sample based on how they resembled the BHH sample (36). Propensity scores were calculated by using individual and area-level characteristics. We did not include service use variables in the propensity score to avoid potential bias due to regression to the mean (37). Control group members with scores lower than the lowest value in the BHH sample were dropped. To estimate average treatment effect on the treated sample, standardized mortality ratio weights (38) were generated from scores and applied to individual observations for both BHH patients and control group members.

Autocorrelation was assessed with the Cumby-Huizinga test. When first-order autocorrelation was detected, Newey-West standard errors were replaced with Prais-Winsten standard errors. Newey-West standard errors were used to account for heteroskedasticity and autocorrelation. Monthly indicators were added to our models to control for seasonal variation. All analyses were conducted in StataMP, version 15.1 (39).

Primary and Secondary Outcomes

Our ITSA followed Linden (40). We applied individual propensity scores to weight the number of monthly service encounters for each individual. The sample average of these values (the sum of individual weighted values divided by the sample size) was the dependent variable in the ITSA regression model.

We considered six visit types as dependent variables: primary care, ED, GMH inpatient, GMH outpatient (i.e., other types of specialty GMH care), behavioral health inpatient, and behavioral health outpatient. Our primary outcomes of interest were, first, the difference in predicted monthly visits immediately after the intervention start date (“level shift”). Level shifts represent abrupt changes in patient behavior that occurred as soon as the intervention began. Second, we considered how the rate of monthly visit counts changed (i.e., whether counts were falling or rising, and by how much) over the 3.5-year follow-up period compared with the preintervention period (“trend shift”). A negative trend shift would occur, for instance, if utilization declined at a faster rate than in the preintervention period. Level shifts represent immediate intervention impacts, whereas trend shifts capture the intervention’s impact on outcome trajectories over time. Our ITSA model analyzed primary outcomes through a difference-in-difference (DID) framework, comparing shifts between BHH enrollees versus control group members.

Secondary outcomes included the average individual-level change in cardiometabolic measures between the pre- and postintervention period for the two samples. We conducted this analysis for all individuals with at least one value recorded in the pre- and postintervention period and, as a sensitivity analysis, among only those who had a categorically high value recorded in the preintervention period (greater than 5.6% for HbA1c and greater than 190 mg/dL for low-density lipoprotein [LDL] cholesterol).

Exploratory Subgroup Analyses

Subgroup analyses assessed how the intervention’s impact on the assessed variables varied by race-ethnicity and language spoken at home, both between the BHH and control groups and within the BHH group (no IPW applied). This study was approved by the institutional review board of the Cambridge Health Alliance.

Results

Study Sample

The demographic and clinical characteristics of 413 patients who enrolled in the BHH between September 1 and October 31, 2015, were compared with those of 1,929 control group members. Table 1 shows differences by sample before IPW. More BHH patients than control group members had primary psychotic diagnoses (86% vs. 51%), and fewer had bipolar diagnoses (31% vs. 61%). BHH patients were also more likely to identify as non-Hispanic Black (23% vs. 16%) and more likely to be a native English speaker (91% vs. 80%).

After we conducted IPW, all characteristics were balanced across the two samples (Figure 1 and part 3 of the online supplement). For the primary analyses, 61 control group members were excluded because their propensity scores were outside the range of that for BHH patients (i.e., scores were not good for comparison).

Service Utilization DID

Figure 2 displays data from the ITSA for our primary outcomes. Table 2 presents the multigroup ITSA estimates for utilization shifts between the pre- and postintervention period among the treatment and control groups. (All preintervention values are available in part 4 of the online supplement.)

In the year before BHH enrollment, average primary care utilization was 0.36 visits per month (falling at rate of 0.0057 fewer visits per month) for eventual BHH enrollees and 0.29 visits per month (falling at a rate of 0.0022 fewer visits per month) for the control sample. Postimplementation, BHH patients experienced an immediate level shift of 0.20 more primary care visits per month, compared with 0.0037 more visits per month among the control sample, resulting in a level shift DID of 0.18 more primary care visits per month (p<0.01) for those enrolled in BHH. BHH enrollment was also associated with an immediate decrease in ED visits (DID level shift=–0.031, p<0.01) and increase in specialty GMH outpatient visits (DID level shift=+0.055, p<0.01). Although both BHH and non-BHH participants showed a drop in behavioral health inpatient visits after BHH program implementation, BHH participants showed a shallower drop, suggesting a relative increase (DID level shift=+0.018, p<0.05). If level shifts were applied equally across the first year (not just the first month, as estimated by our model), BHH participation would be associated with 2.16 more primary care, 0.36 fewer ED, 0.66 more GMH outpatient, and 0.036 more behavioral health inpatient visits during that year.

When considering the intervention’s impact on the trajectory of outcomes (trend shifts), the rate of change among BHH patients for behavioral health outpatient visits was 0.024 fewer during the postintervention period compared with the preintervention period; this measure was estimated at 0.0090 fewer in the control sample. Within the DID model, BHH enrollment was associated with 0.016 fewer behavioral health outpatient visits every month after the intervention began (DID trend shift, p<0.01). Although GMH inpatient and outpatient visits were decreasing over time for both groups, relative to the control group, BHH enrollment was also associated with small increases in the monthly rate of GMH inpatient visits (DID trend shift=+0.00074, p<0.01) and GMH outpatient visits (DID trend shift=+0.0044, p<0.01). If trend shifts were applied equally across the 3-year postintervention period, BHH participation would be associated with 0.19 fewer behavioral health outpatient, 0.009 more GMH inpatient, and 0.05 more GMH outpatient visits for every year of observation. Shifts not listed here were not statistically significant and are presented in Table 2.

Subgroup Analyses

When restricting the sample to non-White patients and those with English as a second language, intervention effects were generally smaller or nonsignificant. The DID level shift in primary care visits for BHH enrollees after the intervention was 0.184 in the entire sample, but this value decreased to 0.167 when the sample was restricted to non-White patients (BHH patients, N=139; control group members, N=698); no associated level shift in primary care visits was detected when the entire sample was restricted to non-English speakers (BHH patients, N=39; control group members, N=382).

Compared with non-White BHH enrollees (N=139), White enrollees (N=274) experienced minor differences only for behavioral health outpatient visits. Among BHH enrollees, native English speakers (N=374) experienced a greater immediate increase in primary care visits and fewer ED visits (both immediately and over time) than did nonnative English speakers (N=39). Results for the subgroup analyses were exploratory and should be interpreted with caution because of small sample sizes (see part 5 of the online supplement).

Cardiometabolic Measures

To assess how BHH enrollment affected cardiometabolic measures, we restricted our sample to participants who had at least one laboratory test value in the pre- and postintervention period (N=186 BHH patients and 498 control group members for HbA1c; N=174 BHH patients and 475 control group members for LDL cholesterol). When comparing average laboratory test values from the pre- and postintervention periods between BHH patients and control group members, we detected no statistically significant change in LDL levels, but BHH participation was associated with 0.29 fewer percentage points for HbA1c (p<0.05). When restricted to those with high values in the preintervention period (504 control group members and 148 BHH patients for HbA1c; 10 control group members and six BHH patients for LDL), BHH participation was associated with 0.47 fewer percentage points for HbA1c (p<0.05). Data from all 10 control group members in the subsample with high LDL values were dropped because the propensity scores of these members were too low. An exploratory analysis suggested higher cardiometabolic screening rates among BHH participants compared with the control population (see part 6 of the online supplement).

Discussion

The findings of this study suggest that BHH enrollment altered health care utilization patterns. Relative to control group members, BHH patients had an immediate increase in primary care visits, which held during the 3.5-year follow-up, and an immediate decrease in ED visits. Behavioral health outpatient visits were increasing for BHH patients before program implementation but fell postimplementation at a rate significantly faster than for control group members. GMH inpatient and outpatient visits were decreasing over time for both groups but decreasing more slowly for BHH patients postimplementation. Although behavioral health inpatient visits decreased for both groups postimplementation, this initial drop was larger for the control group. Our model indicated small immediate shifts toward increased GMH subspecialty and GMH inpatient visits among the BHH patients. Although these shifts were small and qualitatively subtle, as seen in Figure 2, they may indicate BHH participants’ increasing engagement with emergent needs. Because our models estimated the average treatment effect on those receiving treatment, results correspond to the patients who were ultimately selected into the BHH program and are generalizable to patients who resemble our patient population (i.e., those who may enroll into such programs in the future).

Our results align with two recent reviews of BHHs (28, 41), which found that most BHHs have increased utilization of primary care visits with mixed effects on other service utilization. This literature includes a randomized controlled trial of a similar BHH model (e.g., integrated nurse practitioner and care manager), for which results are more robust than in other studies. Our findings are more consistent with BHHs that share such comprehensive features; future analyses should better describe program components to facilitate cross-study comparisons (see part 1 of the online supplement).

An increase in primary care visits occurring alongside a decrease in outpatient behavioral health visits suggests that before BHH implementation, some behavioral health utilization may have addressed GMH needs that were later met by the BHH’s primary care services.

An exploratory analysis suggested that HbA1c decreased for BHH participants with higher screening rates, especially if HbA1c values were high before the intervention. Other BHH evaluations detected no such association between BHH participation and HbA1c lab values. Collectively, the results suggest that BHH programs can improve health outcomes for those with serious mental illness, but only if these programs move beyond ensuring timely screening to providing targeted interventions that mitigate the risk that newly identified risk factors present. Failing to target the whole person will achieve minimal success (28). Evaluating the features of the BHH in this analysis that led to improved cardiometabolic values would be a fruitful research question.

Ours is one of few studies to consider heterogeneity in BHH outcomes by patients’ race-ethnicity and language. New initiatives should acknowledge health equity by building in robust data collection to understand disparities in enrollment and engagement. Mixed-methods studies that address unmet needs for patients with serious mental illness from diverse backgrounds are encouraged.

There is growing interest in developing value-based payment models that facilitate the integration of general medical and behavioral health care (42, 43), which have been shown to be cost-effective (44–46). The results of this study suggest that BHHs may increase primary care visit utilization without increasing visits to specialty behavioral health outpatient care. Although we did not consider economic outcomes, future studies using claims data may consider the long-term economic consequences of BHH programs. This is critical for decision makers who seek to improve care coordination in the context of value-based payment models.

Study Strengths

The follow-up period of our analysis (3.5 years) was longer than other analyses on BHHs and allowed us to distinguish between immediate and long-term effects of the intervention (level vs. trend shifts). In relation to a prior analysis of this BHH based on a 1-year follow-up (34), the present analysis confirmed that the increase in primary care visits was sustained for the 3.5-year follow-up period and that behavioral health outpatient visits began to decrease over time. The estimates produced from our study benefited from the quasi-experimental techniques of IPW and multigroup ITSA. We incorporated area-level socioeconomic characteristics into our propensity score model, which helps estimate the BHH program impact by accounting for variation in outcomes that may be due to area-level factors.

Study Limitations

This study had several limitations. First, all individuals needed at least one encounter in the health system in the pre- and postintervention periods. For the control group, this encounter was for a behavioral health appointment. Individuals may have recorded this encounter early in the postintervention period before leaving the health system’s catchment area. However, results of qualitative sensitivity analyses in which individuals in the control groups required an encounter in 3 separate years postintervention did not influence our primary conclusions (see part 7 of the online supplement). Results were similar in a sensitivity analysis that included later BHH enrollees, although BHH participation for later enrollees was associated with smaller shifts in primary care and behavioral health outpatient utilization compared with shifts among early enrollees (see part 2 of the online supplement). Second, our study mimicked an intent-to-treat analysis. BHH participants were included only if enrolled when implementation began. We did not capture differences for patients enrolled later, and we included in our analyses patients who might have disengaged from the program. Third, subgroup analyses were exploratory because of smaller sample sizes. Fourth, our propensity score model did not control for potential selection bias based on preintervention levels of behavioral health inpatient and outpatient visits. However, not including preintervention levels in the model prevents bias from regression to the mean, which may be strong precisely because of preintervention level differences. Finally, we acknowledge an abrupt decrease in behavioral health inpatient visits for the control population when the intervention began. This decrease may reflect general efforts across the health system to reduce behavioral health hospitalizations in the population. Unexpected changes in real-world data highlight the need for multigroup comparisons when evaluating complex interventions in real-world settings.

Conclusions

This study considered long-term impacts of BHH enrollment on service utilization and cardiometabolic measures and demonstrated increased use of primary care. We found mixed results for long-term health outcomes such as hospitalizations and cardiometabolic values. Future studies should consider barriers to successful BHH implementation among vulnerable subpopulations.