General Medical Outcomes From the Primary and Behavioral Health Care Integration Grant Program

Excessive morbidity and mortality among persons with serious mental illnesses is a public health crisis (1–6). Drivers of this disparity include medication side effects, poor self-care, substance abuse comorbidity, unhealthy lifestyles, and socioeconomic disadvantage (7–9). Fragmentation of the behavioral and general medical health care sectors also affects the health of populations with serious mental illness, limiting access to general medical care and reducing the quality of the care received (9–12). Integrated care has the potential to improve both general medical and behavioral health outcomes. Behavioral health care may help consumers better manage general medical conditions, for example, by encouraging them to take medications regularly or adhere to specialized diets. Reciprocally, improvements in general medical health may lead to improved quality of life and overall well-being.

The Substance Abuse and Mental Health Services Administration (SAMHSA) began the Primary and Behavioral Health Care Integration (PBHCI) grants program in 2009 to address issues related to access to and quality of general medical service by supporting the integration of primary care services into community mental health settings where adults with serious mental illness already receive care. The first three cohorts of PBHCI grantees (awarded in 2009 and 2010) were required to implement four core program features: screening and referral for general medical illness prevention and treatment, registry and tracking systems for general medical needs and outcomes, care management, and prevention and wellness services. A portion of the grant could be used to develop infrastructure and performance measurement for improving integrated care delivery and to implement optional program features, such as colocating primary care providers in behavioral health settings and embedding nurse care managers within primary care teams. There are now 187 organizations that have received PBHCI grants.

In 2010, an evaluation of the PBHCI program was initiated and included the first 56 grantees, which were funded in 2009 and 2010. The evaluation was arranged by SAMHSA and the Office of the Assistant Secretary for Planning and Evaluation (principal advisor to the Secretary of the Department of Health and Human Services) and was based on a structure-process-outcomes framework (13,14). The evaluation included a process evaluation to identify the range of structural and clinical approaches to integrating care through PBHCI, an outcomes evaluation to determine whether PBHCI led to improved consumer general medical health, and a model features evaluation to explore which components of integrated care were associated with better general medical outcomes.

This article presents the results of the outcomes evaluation for consumers with risk factors for general medical illness upon enrollment in PBHCI. We used a quasi-experimental, difference-in-differences design to compare change in general medical health indicators over approximately one year for consumers served at three PBHCI clinics and consumers served at three matched behavioral health clinics that were selected as control sites. We tested the hypothesis that consumers at PBHCI clinics who were at risk of general medical illness would show greater improvements in general medical health compared with their counterparts at control clinics.

Methods

All study procedures were approved by the RAND Corporation’s Human Subjects Protections committee and the federal Office of Management and Budget.

Study Site Selection

We selected three PBHCI sites for the quasi-experiment from the subset of grantees that were expected to serve sufficient consumers during the study period for adequate study power (750 consumers per site). Anticipated volume was based on early utilization data and SAMHSA project officer assessments. We applied additional criteria to include a sample of PBHCI sites with diversity in geographical location (for example, different states and urban and rural areas), consumer demographic characteristics, and approach to implementing PBHCI (for example, sites with and without a formal health clinic partner). PBHCI sites were included only if we could also engage a behavioral health clinic to serve as a matched control—a clinic located in the same state that served a similar consumer population but provided no or very limited primary care services.

The final sample of PBHCI sites included one program funded in 2009 and two funded in 2010. Service utilization data from all 56 grantees showed that the PBHCI sites included in the quasi-experiment had higher-than-average rates of primary care provider contacts and general medical screenings but generally did not differ dramatically from the set of unselected sites. [Tables comparing characteristics of the sites that were or were not selected for the quasi-experiment are available as an online supplement to this article.]

Consumers

Participants were at least 18 years of age; had a primary psychiatric diagnosis of schizophrenia or other psychotic disorder, bipolar disorder, or severe depression; and were enrolled in a PBHCI program or received behavioral health care at a matched control site. Individuals were excluded if they were unwilling or unable to provide written informed consent or complete study questionnaires in English or Spanish (with assistance, if requested).

Consumers in the PBHCI programs enrolled in the program during (approximately) the second year of grant funding (August 2010–July 2011 for the 2009-funded grantee and August 2011–July 2012 for the 2010-funded grantees). These enrollment windows allowed us to evaluate the effect of participating in a PBHCI program for a full year (±6 months) within the time frame of the evaluation contract, while allowing for program start-up time and keeping the number of years of PBHCI funding consistent across sites.

Grantees proposed various criteria for enrolling consumers in the PBHCI program, as described previously (15); however, because of challenges meeting enrollment targets, clinics typically relaxed criteria to include all adults with serious mental illness. Enrollment criteria originally proposed by the quasi-experimental sites were inadequate connection to primary care and psychotropic medication use (site 1), inadequate connection to primary care and high-risk clinical factors (site 2), and all adults with serious mental illness (site 3).

Data Collection

Baseline data for PBHCI clinics were collected by clinic staff at the time consumers enrolled in integrated care. We drew these data from SAMHSA’s online data repository (Transformation Accountability system). PBHCI follow-up data were collected by a biometric screening contractor during a weeklong data collection event held approximately one year (±6 months) after individual consumers enrolled. Baseline and follow-up data for the control sites were also collected by the contractor during weeklong data collection events held approximately one year apart. We used clinic-collected (as opposed to contractor-collected) data for baseline measurements for PBHCI participants to minimize participant burden, given considerable program- and consumer-level data requirements for all grantees (15). Clinic-collected data were expected to be more variable than contractor-collected data (thus potentially limiting power) but were not expected to be systematically biased, given that baseline data were collected prior to clinics’ being selected for the study.

Measures

Indicators of general medical health included body mass index (BMI), blood pressure (systolic and diastolic), and self-reported smoking status. Blood samples were collected for plasma glucose or hemoglobin A1c (HbA1c) and lipids, including cholesterol—total, high-density lipoprotein (HDL), and low-density lipoprotein (LDL)—and triglycerides. Blood samples were drawn from individuals who were fasting or nonfasting; fasting status was accounted for in analyses.

In lieu of using general medical diagnoses, we identified consumers as “at risk” of general medical illness if their baseline health indicators exceeded standard normal ranges (systolic blood pressure, ≥130 mmHg; diastolic blood pressure, ≥85 mmHg (16); BMI ≥25 kg/m² (17); total cholesterol, ≥240 mg/dL; HDL cholesterol, <40 mg/dL; LDL cholesterol, ≥130 mg/dL, triglycerides, ≥150 mg/dL (18); fasting plasma glucose, ≥100 mg/dL; nonfasting plasma glucose, ≥140 mg/dL, HbA1c, ≥5.7% (19), and self-reported current tobacco smoking).

Data about participants’ demographic and functioning variables were collected by using an abbreviated version of SAMHSA’s National Outcomes Measures (20), including measures of education, employment, social connectedness, housing stability, substance use, and arrests (21). Clinics provided participants’ primary DSM-IV psychiatric diagnosis.

Analytic Approach

We conducted difference-in-differences analyses to estimate the average treatment effect of PBHCI services on general medical outcomes (22–25). We used propensity-score weighting to account for the nonrandom assignment of consumers to treatment and to adjust for differences between consumers at the PBHCI and control clinics in the demographic and functioning variables described above. Each consumer was assigned a propensity weight according to the following model: the inverse of the estimated probability of receiving the treatment actually received (PBHCI or control), given individual demographic and functioning variable values. This approach is similar to survey weighting methods that adjust samples to be representative of target populations (25). Balance diagnostics were performed to assess the adequacy of the propensity-weight model described above (26).

For each general medical outcome of interest, analyses were limited to consumers whose baseline values for that outcome were in the “at risk” range. For example, the sample for estimating the effect of PBHCI on BMI included only consumers with a baseline BMI over 25 kg/m². These restrictions were used to identify individuals who were in need of intervention and for whom a change in clinical indicators was desired.

Difference-in-differences analyses were conducted for consumers with both baseline and follow-up general medical data. Sensitivity analyses were performed to compare these results to a cross-sectional approach that also included consumers with only baseline or only follow-up data (24). All analyses were conducted by using SAS, version 9.2, software.

Results

Participants

PBHCI and control clinics each aimed to recruit 300 consumers for the study. Across the three control sites, 793 consumers participated at baseline and 492 completed the one-year follow-up, yielding a 62% follow-up rate. Among the control-clinic consumers retained at follow-up, 469 (95%) had complete data for propensity-weight calculation and were included in difference-in-differences analyses. No directly comparable retention metric was available for intervention sites because baseline data for PBHCI consumers were retrospectively drawn from consumers who agreed to participate in the study at the follow-up data collection event. Across the three intervention sites, 1,049 consumers enrolled in PBHCI during the study baseline period (one year [±6 months] prior to follow-up), and 343 (33%) agreed to participate in the research study (in addition to receiving clinical services) and attend the follow-up, contractor-administered data collection event. Among these PBHCI participants, 322 (94%) had complete data for propensity-weight calculation and were included in outcome analyses. Sensitivity analyses revealed similar results for cross-sectional and repeated-measures analyses, suggesting neither selection into the PBHCI group nor attrition in the control group significantly affected outcome results; we present results for the subset of consumers with both baseline and follow-up data.

Baseline characteristics of the sample are shown in Table 1. Among PBHCI consumers, the mean±SD age was 42±12, and 59% were female; 71% were white, 16% black, 5% Hispanic/Latino, and 8% other race-ethnicity. The most common primary psychiatric diagnoses among PBHCI consumers were schizophrenia (28%), major depressive disorder (26%), and bipolar disorder (25%); diagnoses were similarly distributed among consumers in the control group.

Compared with the PBHCI sites, the control sites had more white and fewer black participants (p<.001). Consumers at the control site were also older (p<.001) than PBHCI consumers, and they were more likely to have completed high school (p<.01), to be employed or enrolled in school or job training (p<.05), to report social connectedness (p<.05) and housing stability (p<.001), and to report recent binge drinking at baseline (p<.001) but less likely to report illegal substance use (p<.001).

To adjust for baseline case-mix differences, we computed propensity-score weights using all variables shown in Table 1 to estimate treatment probabilities. After adjusting comparisons for propensity weights, we found no significant differences between consumers at PBHCI and control sites for any variables listed in Table 1, indicating successful balance across groups.

Consumer Health at Baseline

Table 2 shows mean values and proportions of consumers exceeding standard thresholds for health indicators at baseline. Compared with consumers at the control sites, PBHCI consumers were more likely to have elevated rates of diastolic blood pressure (30% versus 22%, p<.05), LDL cholesterol (28% versus 21%, p<.05), and tobacco smoking (62% versus 54%, p<.05). Consumers at the control site were more likely to have a high BMI (81% versus 74%, p<.05) and elevated rates of triglycerides (52% versus 38%, p<.001). [These variables were not included in case-mix adjustments because the difference-in-differences analyses accounted for group differences in baseline outcomes.]

General Medical Outcomes

Change in general medical outcomes for each group and the difference in differences between consumers in each group are shown in Table 3. Over the study period, consumers served at PBHCI clinics had statistically significant improvements in all general medical outcomes for which they were at risk, except BMI and HbA1c. Among control clinics, consumers had statistically significant improvements in all outcomes for which they were at risk, except LDL cholesterol and HbA1c.

After the analyses adjusted for case mix, the mean reduction in total cholesterol among consumers with cholesterol-related risk factors was greater by 36 mg/dL among PBHCI consumers compared with consumers at the control clinics (p<.01). The adjusted mean reduction in LDL cholesterol was greater by 35 mg/dL among PBHCI consumers compared with consumers at the control clinics (p<.001), and the increase in HDL cholesterol was greater by 3 mg/dL among PBHCI consumers compared with consumers at the control clinics (p<.05). No other differences between PBHCI and consumers at the control sites were observed.

Discussion

We found that consumers at risk for general medical illness had greater improvements in measures of cholesterol after being treated for approximately one year at PBHCI clinics versus control clinics. Cholesterol levels are a risk indicator for coronary heart disease, heart attack, and stroke. PBHCI effects were not observed for any other general medical health indicators measured.

This mixed set of PBHCI outcomes was not unexpected; particularly in the early stages of implementation, health care delivery changes rarely result in consistent improvements across all health outcomes examined. As with similar reforms (27–29), issues related to program implementation, quality of care, and patient engagement or adherence to treatment may have mitigated the effects of early PBHCI implementation on general medical outcomes (15).

Nonetheless, it is useful to compare the outcomes of the PBHCI program with those of other related interventions for populations with serious mental illness, particularly as the PBHCI grants program continues to grow. These results may also be of interest to those involved in other initiatives, such as health homes, that also aim to foster linkages between community mental health and medical providers.

Among general adult populations, diet modification and statins are effective treatments for dyslipidemia (30); however, because of the effects of psychotropic medications on lipid levels, lipid management among adults with serious mental illness is particularly important and may involve unique protocols (31). In this study, PBHCI was associated with greater improvements in total, HDL, and LDL cholesterol compared with treatment at control sites, and the effect sizes of these differences were large enough to result in clinical improvements. Studies have shown that each 10-mg/dL reduction in LDL cholesterol is associated with an approximately 10% reduction in cardiovascular risk (32). Among consumers with cholesterol-related risk factors, the mean reduction in LDL cholesterol was 35 mg/dL greater among PBHCI consumers compared with consumers in the control group, suggesting a cardiovascular risk reduction of up to 35%. This effect size is consistent with other published trials of cholesterol management (33–36).

Individuals with serious mental illness are at risk for hypertension because of a sedentary lifestyle, smoking, and antipsychotic medication complications (31). In this analysis, PBHCI did not significantly reduce diastolic or systolic blood pressure. Few studies have directly tested the effects of hypertension treatment among adults with serious mental illness. One randomized controlled trial investigated integrated hypertension and depression pharmacotherapy among older patients (37). Among the treated consumers, systolic blood pressure was lower by 14 mmHg and diastolic blood pressure was lower by 10 mmHg compared with the control group, larger than the effects on systolic and diastolic blood pressure (differences of 2.0 and 1.2 mmHg, respectively) we observed for PBHCI participants compared with participants at the control sites.

Diabetes poses a significant risk to adults with serious mental illness because of lifestyle factors and psychotropic medication use (31). We found no PBHCI-related improvements in diabetes risk. In recent reviews, the few studies showing any treatment benefit for diabetes outcomes among adults with serious mental illness included pharmacological interventions (33,38,39). We did not have data on consumers’ medication use; subsequent evaluations of PBHCI may prioritize medication monitoring to better address how consumer outcomes may be improved.

Serious mental illness is associated with increased risk of obesity because of lifestyle factors, poverty, limited access to healthy foods or opportunities for exercise, and psychotropic medication use (31). PBHCI did not reduce BMI-measured obesity compared with treatment at control sites, a disappointing result in light of results from the published literature (31,36). A recent meta-analysis of more than 30 studies of weight control interventions for adults with serious mental illness showed that the net effect of these interventions was typically positive, albeit small (weight loss of about 3 kg) and potentially short-lived beyond the intervention period (36).

High rates of cigarette smoking among adults with serious mental illness are attributable in part to enhanced dopamine reinforcement, metabolic effects of antipsychotic medication, and reduced opportunities for other rewards (40). We found no PBHCI effect on smoking outcomes. Until recently, research had not identified effective treatments for smoking among populations with serious mental illness. Some studies now show that oral, prescription-only interventions (bupropion and varenicline) can improve outcomes (41–43); however, most providers are still reticent to use these interventions because of fears that these medications might worsen psychiatric symptoms and increase suicide risk (41–43). SAMHSA currently provides technical assistance to behavioral health providers to improve smoking cessation interventions.

This study had several limitations. Quasi-experimental sites were not randomly selected and results may not generalize to other grantees. Similarly, individual participants were not randomly assigned to PBHCI, and results may not generalize to all adults with serious mental illness treated in community mental health centers. Our results reflect early implementation of PBHCI and of individual grantee programs, both of which may mature and improve over time.

The validity of difference-in-differences designs to account for nonrandom assignment of consumers to treatment depends on assumptions of similar time trends across intervention and control sites. Group differences in the trajectory of general medical health that were unrelated to the intervention would violate such assumptions, for example, if trajectories differed on the basis of extant physical illness, use of psychotropic medications, or other variables that clinics may have used to selectively target consumers for PBHCI. If PBHCI consumers included in this study were more motivated than control-clinic consumers, or otherwise were more likely to achieve general medical improvements regardless of the treatment effect, our findings would represent an overestimate of the true effect of PBHCI.

Although the control sites reported no formal plans to provide or coordinate primary care for consumers, we could not account for access to primary care by participants in the control groups during the study period. Increased access to primary care among participating control clinics compared with other non-PBHCI clinics may have limited our ability to detect PBHCI treatment effects. Study power was limited because of smaller-than-anticipated sample sizes. The small number of PBHCI sites precluded identification of site-level variables that may have moderated treatment effects. We did not have access to important consumer-level variables of interest, such as medication use.

Finally, although a one-year follow-up period may be sufficient to detect a change in conditions that respond quickly to medication, such as high blood pressure, or to other interventions, other conditions that require intensive lifestyle modification, such as smoking cessation or weight loss, may take longer to improve.

Conclusions

This small-scale quasi-experiment assessing the effect of early PBHCI implementation on general medical outcomes among consumers yielded modest results. Approximately one year of PBHCI treatment resulted in statistically and potentially clinically significant improvements in cholesterol levels but not in other indicators of general medical health. With further support from SAMHSA and the technical assistance center, quality improvement efforts, and other strategies to ensure rigorous program implementation, PBHCI programs may further improve health for adults with serious mental illness.