Comparing Preventable Hospitalizations Among Veterans With and Without Mental Illnesses Before and After Implementation of PACT

Mental illnesses are highly prevalent among primary care patients and are frequently managed within that setting (1–4). These patients may have difficulty obtaining high-quality primary care and adhering to recommendations because of their symptoms and limited economic, social, and physical resources (5). Consequently, the presence of a mental illness is associated with a higher risk of developing chronic illnesses and related complications, and the integration of mental health treatment into primary care has been proposed to lessen these risks (6–9).

Another approach is the patient-centered medical home (PCMH), which is designed to improve access to high-quality care and increase care coordination by using interdisciplinary teams. The Veterans Health Administration of the Department of Veterans Affairs (VA) deployed a PCMH model in 2010 called Patient Aligned Care Teams (PACT) to provide primary care to over 5 million veterans (10). PACT is a team-based model that aims to improve partnership between patients and their providers through improved access and care coordination. Through PACT, patients have many modalities to access their providers, and PACT teams aim to proactively reach the patients. Although these changes could benefit all primary care patients, they could be particularly beneficial for patients with mental illness, who may have unique challenges as a result of their illness—for instance, low motivation among those with depression. PACT teams were designed to incorporate behavioral specialists as part of regular care, in addition to providers who are a part of the VA primary care mental health integration.

A common metric to assess the quality of primary care is to evaluate the rate of preventable hospitalizations, defined as those attributed to ambulatory care–sensitive conditions (ACSCs). ACSCs are chronic conditions, such as diabetes and ischemic heart disease, that are thought to be the most responsive to high-quality primary care. Improving the quality of primary care through improved care coordination has been shown to reduce hospitalizations attributable to ACSCs (11). Several studies also have shown that patients with mental illness are at higher risk of ACSC-related hospitalizations than those without mental illness (12–14). Improvements in care coordination may be particularly advantageous for patients with mental illness because the primary care team can emphasize outreach and access to these vulnerable patients (15,16).

We sought to determine whether there were changes in the rates of hospital admissions attributed to ACSCs for patients with and without mental illness before and after PACT implementation. We hypothesized that veterans with mental illness would have a lower rate of ACSC-related hospitalizations following the implementation of PACT. We expected the lower rates to be the most pronounced among veterans with depression and anxiety, because these mental illnesses are commonly managed in primary care settings.

Methods

Setting, Study Design, and Participants

The analyses were conducted as part of a comprehensive evaluation of the nationwide implementation of the VA PACT initiative. We conducted an observational retrospective analysis of all primary care patients enrolled in the VA at any time between October 1, 2003, and March 31, 2015. Veterans were included if they were assigned to a primary care provider at any time during the study period. Veterans who disenrolled or who died during the study period were dropped from the analysis in subsequent quarters. [A figure depicting how the study sample was constructed, using a hypothetical clinic 1, is included in an online supplement to this article.] This study was deemed exempt from institutional review board approval because it was nonresearch and analyzed nonidentifiable data.

We grouped mental illnesses into depression, anxiety, posttraumatic stress disorder (PTSD), substance use disorder, and serious mental illness. Serious mental illness included patients with bipolar disorder or schizophrenia. We required that patients be assigned a diagnosis of depression, anxiety, serious mental illness, or PTSD during at least two outpatient visits or one inpatient stay during the previous year or that they received a diagnosis of a substance use disorder at one outpatient visit or during one hospitalization (1). These categories were not mutually exclusive; veterans with more than one mental illness were included in all relevant categories.

Data Sources

We used the VA’s Primary Care Management Module to identify veterans’ assigned primary care team and clinic and the Corporate Data Warehouse to retrieve demographic and diagnostic information. We abstracted unemployment rates from the National Bureau of Labor Statistics. Vital status was determined by using the Health Eligibility Center enrollment data.

Statistical Analyses

Because of the large sample of veterans who have received VA primary care, we modeled clinic-level rates of ACSCs, with each clinic contributing two observations to the analysis in each quarter—the clinic’s patients with and without a given diagnosis—and we adjusted for confounders by creating corresponding risk factors for each of these cells (including proportion of patients with a service-connected disability, proportion of male patients, average age, and average comorbid risk score). This empirical strategy of estimating models on aggregated cells that differ based on combinations of characteristics has been applied in prior research (17).

ACSC-related hospitalization rates, our main outcome, were calculated at the facility level. This was defined as the average number of ACSC-related hospitalizations among primary care clinics’ assigned patients. We included hospitalizations at any VA clinic for at least one of 13 ACSC conditions (excluding low birth weight) over the prior 12 months, as defined by the Prevention Quality Indicators of the Agency for Healthcare Research and Quality (18).

We attributed the hospitalizations to the clinic at which patients received their primary care. We included hospitalizations outside the VA only if covered by the VA. We determined age at the beginning of a quarter. We used a weighted index of Elixhauser/Charlson conditions to determine comorbidity, modified to exclude the diagnoses of depression, drug abuse, and psychosis (19).

We conducted longitudinal analyses and included a person-quarter observation for patients by using a 12-month retrospective window and used this look-back to determine mental illness group. To move out from a mental illness group, a patient was required to not have that diagnosis in the prior 12 months. We stratified all analyses by age group (<65 and ≥65) because of potential differences between patients who were or were not Medicare eligible (20). We calculated adjusted and unadjusted ACSC-related hospitalization rates (per 1,000 patients), stratified for mental illness categories and by age group.

Modeling

We performed interrupted time-series analysis to estimate how trends in facility-level ACSC-related hospitalization rates changed after PACT implementation and to assess whether these trends differed among the mental illness categories. We modeled ACSC-related hospitalization rates (per 1,000) for veterans with (or without) mental illness by using mixed-effects negative binomial regression models. To estimate trend deviations in ACSC-related hospitalization rates, we included overall intercept terms corresponding to the years prior to PACT, one to two years after PACT implementation (years 1–2), and three to five years after PACT implementation (years 3–5). These two time frames were selected to detect early and distal trends after the deployment of PACT. We applied an interrupted time-series approach that modeled quarterly trends in ACSC-related hospitalization rates over the period 2003 Q1 through 2014 Q4. To allow for changes in trends after PACT implementation in 2010 Q3, we included an interaction between the PACT indicator variable and the linear time trend. Using coefficient estimates and recycled predictions, we then calculated the difference in ACSC-related hospitalization rates with and without PACT, respectively (that is, holding PACT=0 and PACT=1).

We also calculated predicted ACSC-related hospitalization rates on the basis of historical trends prior to the implementation of PACT. We hypothesized that differences between observed and predicted rates reflected effects of PACT. We computed the changes in the rate of ACSC-related hospitalizations attributable to PACT, calculated as the difference in ACSC-related hospitalizations between two sets of predictors (PACT=1 and PACT=0) for each quarter following the implementation of PACT. ACSC-related hospitalizations were averaged across each quarter within the two time periods (that is, years 1–2 and years 3–5). We then conducted exploratory analyses to calculate the difference in PACT-attributable changes in ACSC-related hospitalization rates (that is, the difference between the predicted ACSC-related rate with and without PACT, respectively) between each mental illness group (for example, depression) and the corresponding group without the mental illness (for example, no depression). Significance tests were based on differences between the two post-PACT intercepts and the pre-PACT intercept.

All models were adjusted for clinic-level patient demographic factors and comorbidity score from the four prior quarters. Clinic-level covariates were calculated as within-facility means. To account for a temporal trend, we included separate linear time terms from October 1, 2003, to March 31, 2015, for patients with and without mental illnesses. To adjust for seasonality, we included quarter-of-year fixed effects. We also adjusted for whether a clinic was a hospital-based or community-based outpatient clinic because of differential primary care staffing. We adjusted for time-varying unemployment rate in each clinic’s county to account for changes in health care utilization attributable to the state of the economy. The model included the log of the number of patients in an observational unit as an offset term.

Determination of the final statistical model was based on a combination of conceptual background of the subject matter (that is, a one-time systemwide change in primary care delivery via PACT implementation), prior research examining health care utilization changes attributable to PACT implementation, and model parsimony to avoid overfit.

Reporting of Annual Versus Quarterly ACSC-Related Hospitalization Rates

Regarding the annualized rates presented, the negative binomial models allowed the estimation of the predicted average number of ACSC-related hospitalizations for a single patient in a quarter adjusting for covariates. To create an annualized ACSC-related hospitalization rate per 1,000 patients, this estimate was multiplied by 4,000. Varying clinic sizes were accounted for by the exposure term in the negative binomial models.

We used simple tests of significance of the PACT indicator variable, adjusting for age and clinic type. We then then tested linear combinations of the PACT years 1–2 and PACT years 3–5 indicator variables and the indicator variable for whether a clinic observation represented patients with or without a mental health condition. These were done by using SAS ESTIMATE statements. We conducted all analyses using PROC GLIMMIX in SAS Enterprise Guide 6.1.

Results

Descriptive Statistics

We included 9,206,017 veterans in this study. Participants were predominantly male and white (Table 1). Across all time periods, the proportion of veterans diagnosed as having a mental illness increased over time across all diagnoses and both age groups. For instance, 26% of all veterans had at least one mental illness in 2015, up from 17% in 2004. Use of primary care and primary care mental health integration stayed relatively stable, whereas use of specialty mental health services increased, especially among younger veterans. The average number of specialty mental health visits was higher for veterans <65, compared with those ≥65.

Pre-Post Changes in ACSC-Related Hospitalization Rates

Table 2 presents a comparison of ACSC-related hospitalization rates per 1,000 patients before and after PACT was implemented, with models stratified by age group and adjusted for clinic type. After PACT implementation, there were slightly more ACSC-related hospitalizations across all primary care patients. The rate of ACSC-related hospitalizations among veterans <65 was 6.0 per 1,000 patients prior to PACT, which increased to 6.4 following PACT (p<.001). The rate among veterans ≥65 was 6.2 per 1,000 patients, which increased to 7.6 per 1,000 following PACT (p<.001).

Among patients with mental illness, pre-PACT rates of ACSC-related hospitalizations were higher—11.1 per 1,000 patients for the younger group and 18.6 per 1,000 for the older group. Younger veterans with serious mental illness had a higher rate of ACSC-related hospitalizations in the post-PACT period, compared with the years prior to PACT (12.6 versus 11.5, p<.001). Older veterans had lower ACSC-related hospitalizations following PACT implementation if they had PTSD (12.8 versus 16.3, p<.05) or a substance use disorder (14.9 versus 17.3, p<.01).

We examined the relative difference in ACSC-related hospitalization rates in the various mental illness groups pre- and post-PACT. Younger veterans diagnosed as having anxiety had 1.4 more ACSC-related hospitalizations (per 1,000 patients) in years 3–5 than predicted. Older veterans with any mental illness had a lower-than-predicted rate of ACSC-related hospitalization at years 1–2. The predicted rates of hospitalization among older veterans with PTSD were lower at both years 1–2 (10.9 versus 10.8) and years 3–5 (10.8 versus 10.0); for those with a substance use disorder, the predicted rates were lower only for years 3–5 (30.5 versus 28.7). At years 3–5, older veterans with depression had seven fewer ACSC-related hospitalizations (per 1,000), compared with those who were not depressed. The relative differences are shown in Table 3. [Figures showing rates by age group and diagnosis are included in the online supplement.]

Discussion and Conclusions

The rate of diagnosis of mental illness increased in the five-year period after PACT implementation, compared with the six-year period before PACT. This increase was especially striking among younger veterans: one in three younger veterans was diagnosed as having a mental illness in 2015, compared with approximately one in four in 2004. We noted a substantial increase in receipt of a diagnosis of PTSD among older veterans. The rates of ACSC-related hospitalizations after PACT was implemented were slightly higher across both age groups and across all primary care patients. Future studies may examine whether the extent of PACT implementation improves the accuracy of diagnosis of mental illnesses.

Compared with the pre-PACT period, the post-PACT period had lower ACSC-related hospitalization rates for older veterans with any mental illness in years 3–5 after implementation of PACT, which might reflect the difficulty of rapidly implementing a major new model of care delivery in a large and complex system. We observed the opposite trend among younger veterans, for whom the rates were slightly higher after PACT. Among older veterans with depression, we observed seven fewer ACSC-related hospitalizations (per 1,000 patients) in years 3–5 after PACT was implemented, compared with the period before PACT. Therefore, our key findings were that in the post-PACT period, better outcomes were demonstrated among older veterans but not among younger veterans.

It is possible that the PACT model of care simultaneously improves coordination of management of chronic illness and enhances access to mental health services both within primary care and integrated mental health services. However, this may also have led to bias in our analyses. One consequence of improving diagnosis is that providers improve their ability to recognize mental illnesses of lower severity, thereby lowering the average severity of illness in the sample in the post-PACT period. On the other hand, we observed slight but significantly higher rates of ACSC-related hospitalizations among younger veterans with anxiety after PACT. In this scenario, excessive anxiety may have prompted patients either to avoid care for chronic conditions or to seek excessive care, either of which circumstance could lead to preventable hospitalizations. We observed no differential trends in ACSC-related hospitalizations among patients with serious mental illness or PTSD compared with their counterparts without these conditions. These analyses should be considered hypothesis generating because they were designed to capitalize on the power of our data set while recognizing the biases inherent in comparisons by mental illness groups.

Our analyses had several strengths, including the large population of primary care patients within VA and the high prevalence of well-documented diagnoses. The presence of time-varying confounders is an inherent limitation of conducting interrupted time-series analyses, especially over a long period. Cohorts that we considered homogeneous may have varied in important ways across time—in ways that the covariates may not have fully controlled. In the 11-year period that defined our study, the United States experienced the Great Recession and Housing Bubble of 2007, the “silver tsunami” of aging Baby Boomers, growing conflict in Central Asia that resulted in an influx of younger veterans, and publication of an updated DSM (DSM-5). Unfortunately, there was no control group to help address this endogeneity. Nevertheless, because an experiment across 9 million veterans was not possible, the interrupted time-series methodology represents the most effective tool for understanding the effects of systemwide change in primary care clinical practice and policy.

Other limitations were largely those inherent to associational studies. Administrative data may not have full captured mental illness diagnoses, and we were missing important metrics of mental illness, such as disease severity, change in symptoms, and response to treatment. Because we lacked full information about care received outside of the VA, we could not assess whether our observations extended to hospitalizations in those settings. Our models relied on extrapolating ACSC-related hospitalization rates prior to PACT, which may have evolved over time as a result of unobserved influences outside PACT. For instance, PACT was undertaken as part of a national transformational initiative, which included improvements in telemedicine and other aspects of care. Furthermore, if mental health diagnosis patterns shifted suddenly at the same time as PACT implementation, that component may have been correlated with changes in ACSC-related hospitalization rates. These are all challenges inherent to longitudinal observational studies and cannot be fully controlled for via covariates.

We note two important analytic decisions. First, our models used observations measured at quarterly intervals. These estimates may have been different if they were measured at an annual interval. The rationale for this modeling decision was to generate a larger number of observations within a clinic in order to provide a more reliable extrapolation of ACSC-related hospitalization rates post-PACT implementation. Second, we used a four-quarter rolling window of diagnosis codes (that is, a 12-month look-back) to ascertain patients’ mental illness status. This decision resulted in an inclusive definition of potential limitations. Using occurrence of mental health diagnosis codes (for example, two outpatient or one inpatient diagnosis in the past 12 months for depression) implies that the groups with and without a mental illness were imprecisely determined in any given quarter. For example, a patient successfully treated for depression would remain in the depression cohort for up to one year because of prior diagnosis codes. In this event, if there was a smaller reduction in the risk of an ACSC-related hospitalization attributable to PACT implementation for patients without depression, then inclusion of this patient in the depression cohort would understate the differential associations identified. The second potential issue with the 12-month look-back is related to the correlation of random-effect estimates, particularly the pre-PACT random intercept and years 1–2 post-PACT random intercept. For this reason, we focused on the presentation of results from the post-PACT years 3–5, which should have provided a long enough wash-out period to ensure that the findings were not correlated with the pre-PACT period.

Our study represents the largest evaluation of the effect of PCMHs on primary care patients with mental illnesses. It found that PCMH implementation resulted in a greater number of veterans with a diagnosis of mental illness and fewer ACSC-related hospitalizations among older veterans with mental illness. Future studies should explore the mechanisms underlying these findings.