Medical Records Flag for Suicide Risk: Predictors and Subsequent Use of Care Among Veterans With Substance Use Disorders

Despite representing only 8% of the U.S. adult population (1), veterans account for 18% of all U.S. deaths by suicide and are at 21% greater risk of death by suicide compared with members of the general population (2). In addition, suicide attempts are on the rise among veterans, increasing from approximately 600 per month in May 2012 to approximately 900 per month in August 2014 (3). A prior suicide attempt is a robust predictor of future suicide (4), and suicide attempts are associated with additional health care needs (5), follow-up care (6), safety planning, and stress for both patients and providers (7).

Individuals with substance use disorders are at particularly high risk of suicide. The risk of suicide is 7.5 times higher for males and 11.7 times higher for females with substance use disorders or psychiatric disorders compared with individuals without either disorder (8). The rate of suicide among veterans with a substance use disorder in 2014 was approximately 89 per 100,000 (2), the third-highest suicide rate among psychiatric disorders. Veterans with opioid use disorders are at even greater risk, with a suicide rate of approximately 140 per 100,000 (2). Alcohol misuse is also associated with an increased risk of suicide (approximately 77 cases per 100,000) (9).

The U.S. Secretary of Veterans Affairs (VA) has identified suicide prevention as the VA’s top clinical priority (10), and efforts have been ongoing for the past decade to identify and respond to veterans at high risk of suicide. Electronic medical record (EMR) systems provide an opportunity to improve suicide prevention. Electronic flags and triggers have been used to alert providers to a variety of clinical needs and prevention opportunities (11–14). The VA has implemented such tools in a number of areas, including alerting providers to veterans’ suicide risk through patient record flags (PRFs) indicating a high risk of suicide (15). By policy, the placement of a PRF is a clinical judgment based on an evaluation of risk factors, protective factors, and warning signs. However, the policy includes five “indicators that a veteran may be considered high risk” to improve uniform implementation (for example, verified suicide attempt or hospitalization for suicidal ideation) (15). When a flag is in effect, providers are alerted immediately upon entry into the EMR that the patient has been identified as being at high risk of suicide. In addition, mental health or substance use disorder treatment providers are expected to have contact with flagged veterans at least weekly in the month following PRF activation (16). Monthly clinical contact is recommended thereafter for the duration of the PRF, which is typically three months, pending reevaluation. During the time period under consideration in this study, the expectation of six clinical contacts was included in VA facility-level accountability metrics. Although other integrated health care systems are using EMR data to flag patients for suicide interventions (17), to our knowledge, no study has evaluated the impact of such policies on patient care, particularly among veterans with documented substance use disorders.

This study examined new PRF activation among veterans with documented substance use disorders in VA nationally. Specifically, the aims were to identify demographic, clinical, and service utilization predictors of new PRF activation and to describe changes in short-term (three months) and longer-term (one year) utilization of outpatient and inpatient services before and after new PRF activation among veterans with substance use disorders.

Methods

Source of Data and Study Population

This study used administrative medical records data from the VA Informatics and Computing Infrastructure, a national data repository that includes patient-level data on VA service use, as well as information on demographic characteristics and clinical diagnoses. VA patients ages 18 or older with a documented primary or secondary diagnosis of a substance use disorder (excluding tobacco) from an outpatient or inpatient contact at a VA facility between October 1, 2011, and September 30, 2012 (fiscal year 2012 [FY 2012]), were eligible for study inclusion (N=485,394). Diagnoses were identified by using ICD-9-CM codes for alcohol, opioid, cocaine, amphetamine, cannabis, sedative, and other substance use disorders.

Patients were classified as having a PRF if a new PRF was placed in their EMR in the first year after their initial substance use disorder diagnosis (hereafter referred to as the index year). Patients with PRFs in their EMR in the year prior to their initial substance use disorder diagnosis (hereafter referred to as the baseline year) were excluded (N=10,448). Patients who died in the index year (N=14,541; no PRF, N=14,366; PRF, N=175) were included in predictors of PRF initiation analyses but not in utilization analyses, given that utilization in their index year would be truncated.

Study approval was obtained from the VA Puget Sound Institutional Review Board.

Predictors of PRF Initiation

Predictors of PRF initiation were identified from administrative data in the year prior to patients’ initial substance use disorder diagnosis, rather than the PRF initiation date, to ensure equivalent comparison periods between veterans with and without PRFs.

Demographic characteristics included age, race-ethnicity, marital status, engagement in Operation Enduring Freedom/Operation Iraqi Freedom (OEF/OIF), homelessness status, and VA service-connected disability rating ≥50% (that is, injury or illness incurred or aggravated during active military service, which determines VA health care eligibility and benefits).

Clinical characteristics included substance use disorder diagnoses (listed above), psychiatric disorder diagnoses, suicide-related diagnoses, pain diagnoses, and medical comorbidity. Psychiatric disorder diagnostic categories were identified by using ICD-9-CM diagnostic codes and included posttraumatic stress disorder (PTSD) and anxiety, depressive, bipolar, and psychotic disorders. A suicide attempt–related diagnosis was determined by one or more ICD-9-CM code (E95.x). The presence of a pain diagnosis was determined by at least one ICD-9-CM chronic pain diagnostic code (18). Medical comorbidity was calculated from ICD-9-CM codes by using the modified (19) Charlson Comorbidity Index (CCI) (20). CCI scores were categorized into three groups: 0, 1, ≥2, with higher scores reflecting greater comorbidity severity.

Any use of VA outpatient services in the baseline year was determined by outpatient clinic codes representing mental health, substance use disorder, primary care, and emergency department (ED) visits. VA inpatient service utilization was measured by any admission to acute inpatient general medical, mental health, and substance use disorder–related (for example, detoxification) services, as determined by inpatient bed codes. General medical admissions included specialty medical stays (e.g., cardiology).

Service Use Before and After PRF Initiation

Because the initial activation period for suicide risk PRFs is three months, we were primarily interested in changes in service use in the three months preceding versus following PRF activation. To understand the impact of PRFs on longer-term utilization, changes in the one year preceding and the one year following PRF activation were also examined. Outpatient utilization was measured by counts of visit days in mental health, substance use disorder, primary care, and ED clinics. Inpatient utilization was measured by total days spent (based on admission and discharge dates) in inpatient acute general medical, mental health, and substance use disorder services. Outpatient visits and inpatient stays that included the PRF initiation date were excluded from visit counts and total inpatient days, respectively, because we could not determine whether utilization was the result or the cause of PRF activation. Veterans without active PRFs or who died in the index year were excluded from these analyses.

To assess the proportion of patients with a PRF who met visit targets per VA policy (16) we created a binary variable indicating whether patients received mental health or substance use disorder care on four or more visit days in month 1 and on one or more visit days in each of months 2 and 3.

Data Analysis

Demographic characteristics, baseline clinical characteristics, and baseline utilization among veterans with and without a suicide risk PRF are presented as frequencies and percentages and were compared by using chi-square tests. Multivariable logistic regression was used to identify characteristics associated with PRF initiation and to estimate adjusted odds ratios and 95% confidence intervals (CIs) in the full sample. The model included all predictors of PRF initiation mentioned above and was estimated with robust variance estimates to account for correlation between observations at the VA facility level.

Among patients with activated PRFs, the number of inpatient days (general medical, psychiatric, and substance use disorder–related) and outpatient visit days (mental health, substance use disorder, primary care, and ED) in the three months and one year preceding and following PRF initiation were compared by using unadjusted negative binomial regression models and estimated with incidence rate ratios and CIs. In exploratory analyses, we used multivariable logistic regression models to identify factors associated with meeting visit targets (four or more mental health or substance use disorder visit days in month 1 and one or more visit days in both months 2 and 3 after PRF initiation); factors included demographic and clinical characteristics indicated above in the year prior to PRF initiation. To account for multiple comparisons, we adopted a p value threshold of p<.001. All analyses were performed in Stata, version 14.0.

Results

Patient Characteristics

Among veterans with a substance use disorder in FY 2012 (N=474,946), 8,913 (1.9%) had a suicide risk PRF initiated in the index year (Table 1). In the full sample, most veterans were men, age 45 or older, white, and of non–Hispanic-Latino ethnicity. The most common substance use disorder and psychiatric disorder were alcohol use disorder and depressive disorder, respectively, and most veterans in the sample had a pain-related diagnosis.

Predictors of PRF Initiation

Significant baseline-year predictors of PRF initiation included being less than age 35, white, and homeless; having a service-connected disability rating <50%; and having served in OEF/OIF (Table 2). Any suicide attempt–related diagnosis in the baseline year was predictive of PRF initiation. Substance use disorder diagnoses that predicted suicide PRF initiation included cocaine, opioid, and sedative use disorders, and psychiatric disorder diagnoses that predicted suicide PRF initiation included PTSD and psychotic, bipolar, and depressive disorders. Any inpatient or outpatient mental health contact or ED visit predicted PRF initiation. Factors that protected against PRF initiation included any primary care or substance use disorder outpatient visit. [A table showing predictors of PRF initiation from an analysis run separately for men and women is included in an online supplement to this article.]

Service Use Before and After PRF Initiation

With the exceptions of ED visits and medical inpatient admissions, service use increased during the three months following PRF initiation, compared with the prior three months (Table 3). Patients with a suicide risk PRF (N=8,738; excludes 175 veterans with new PRFs who died in the index year) averaged 1.3 times more primary care visit days, 2.3 times more mental health visit days, and 4.1 times more substance use disorder visit days in the three months following PRF initiation, compared with the three months prior. Increases in days hospitalized were more modest; patients averaged 1.4 times more hospitalized days for psychiatric disorders and 1.3 times more hospitalized days for substance use disorders in the three months following PRF initiation, compared with the prior three months. ED visits decreased significantly, with patients averaging approximately one-half the number of ED visits in the three months after the PRF initiation, compared with the three months prior. Medical inpatient admissions remained largely unchanged.

Results for comparisons of the one-year periods prior to and following PRF initiation were similar to the three-month results. Figure 1 shows an increase in outpatient mental health and substance use disorder treatment contacts immediately prior to PRF initiation, with visits continuing to rise sharply in month 1 following PRF initiation, and subsequently decreasing but remaining elevated in month 2, followed by a gradual decline over months 3 to 12. Of note, patients with PRFs averaged 12 contacts in the two months following PRF initiation, and utilization remained higher than the baseline months for up to one year after PRF initiation.

Overall, 82.5% (CI=81.7%−83.3%) of veterans attended six or more mental health or substance use clinic visits in months 1 to 3 after PRF initiation, and 61.7% (CI=60.7%−62.7%) met specific VA visit targets (four or more mental health or substance use disorder treatment contacts in month 1 or one or more treatment contacts in each of months 2 and 3), with an additional 14.3% (CI=13.6%−15.0%) meeting targets in month 1 only. Homelessness, bipolar disorder diagnosis, and age 45 to 54 (versus under age 35) were associated with greater likelihood of meeting visit targets (Table 4).

Discussion

To our knowledge this is the first study to examine VA service use before and after initiation of a suicide risk PRF. According to VA policy, patients with new suicide risk PRFs are expected to have weekly clinical contacts during the first month after PRF initiation, with monthly visits encouraged thereafter. Consistent with this policy, 62% of patients with new PRFs attended the recommended number of visits in months 1 to 3, with an additional 14% meeting recommended targets in month 1 only. Furthermore, outpatient contacts in mental health and substance use disorder clinics increased 2.3 and 4.1 times, respectively, over the three months after PRF initiation, with mean contacts for these services exceeding the minimum requirement of one contact per week in month 1. In contrast, ED visits decreased by 45% in the three months following initiation of a PRF.

Although this study was not able to assess the impact of increased clinical contacts on subsequent suicide-related behaviors, reductions in suicide-related behaviors have been reported by studies of similar aftercare interventions. Interventions that aim to engage individuals after a suicide attempt by using weekly to semimonthly contacts have shown reductions in both suicide attempts and suicides (21,22), and health care systems implementing suicide prevention policies that increased assessment and outreach have seen decreased suicide rates (23). Additional research is needed to determine whether increased use of mental health services following initiation of PRFs is associated with decreased suicide behaviors and other adverse outcomes in the VA health care system.

Of patients with suicide risk PRFs, 17% received fewer than six clinical contacts and 38% failed to meet specific VA visit targets in months 1 to 3 following PRF initiation (16). Several possibilities may account for this finding, including patients not attending follow-up visits, difficulty accessing care in rural areas (24), relocating out of the area, unwillingness to participate in aftercare, incapacitating illness, or other barriers to care (for example, transportation difficulties and incarceration) (22). It is also possible that some veterans who initially received PRFs improved rapidly or were determined to be at lower risk and in need of fewer and less frequent contacts. It is notable that patients meeting visit targets were more likely to have a bipolar disorder diagnosis or to be homeless, suggesting that providers allocated additional resources to those with significant psychiatric or psychosocial challenges.

Several predictors of suicide risk PRFs observed in this study have been identified as risk factors for suicide in the literature. White race and younger age are associated with suicide among veterans (3). Studies have reported an increased risk of suicide behaviors among those with social disadvantages, such as lack of education, poverty, and unemployment (25–27), and suicide rates are elevated among homeless veterans (28). Our finding that alcohol use disorders did not predict initiation of suicide risk PRFs was surprising given the significant body of research indicating that having an alcohol use disorder is an important risk factor for suicide (19–21). Our results may be due to limiting our cohort to patients with substance use disorders. Consistent with prior research on suicides and suicide behavior (4,29), prior suicide attempts and psychiatric disorders such as PTSD and depressive, bipolar, and psychotic disorders were predictive of suicide risk PRF initiation. Taken together, most of the correlates of PRF initiation align with known risk factors for suicide and reinforce the importance of prevention strategies among groups with financial problems, prior suicide-related behavior, and psychiatric and substance use disorders.

Implications

Overall, study findings suggest that implementation of suicide risk PRFs in an EMR and subsequent follow-up is feasible even in health care systems as diverse as the VA, which may be encouraging to other health care systems interested in implementing a similar approach. Providers appeared to make prudent decisions regarding new PRF activations, because approximately 2% of patients with substance use disorders were flagged as being at high risk of suicide. Furthermore, most patients with new PRFs received care compliant with VA policy.

Limitations

These analyses had several important limitations, primary among them that PRF activation was a subjective, clinical decision and may have varied regionally or by individuals initiating such flags. Our data did not allow us to determine the specialty or type of provider who initiated the PRF; thus we cannot comment on whether particular provider groups or clinics responded differently to this VA initiative. Data on suicides or suicide attempts, as well as on PRF continuation or removal following PRF activation, were unavailable for analysis, preventing examination of the impact of increased health care utilization on these specific outcomes. In addition, use of administrative data limited the variables included in the predictive models, and potential differences resulting from unmeasured variables (for example, substance use disorder severity and pain severity) may have affected study results. We did not have access to data on the quality of the health care visits. Our sample consisted of VA patients, and thus results may not generalize to nonveterans or veterans who receive care in the community. We did not include use of non-VA services, and thus patients’ use of services may be higher than reported here. In addition, these analyses focused on veterans with substance use disorders, a high-risk population with specialized care needs, and thus these results may not generalize to other veteran or nonveteran populations. The number of women was small, and results may not generalize to this population. Because this was an observational study, changes in service use before and after PRF initiation may have resulted from clinical procedures unrelated to PRF activation.

Conclusions

PRFs indicating a high risk of suicide were implemented to identify and encourage the provision of additional care to veterans perceived as being at increased risk of suicide. Results indicate that among veterans with substance use disorders, the use of PRFs was associated with increases in clinical contacts with both outpatient and inpatient mental health and substance use disorder services, suggesting that once PRFs were activated, veterans significantly increased their service use within the VA health care system. Further research is needed on the effects of PRF activation and increased care on clinical outcomes, such as suicide behaviors. In addition, the research should be expanded to examine clinical contacts among veterans without substance use disorders. Although more work is needed, these encouraging results support the use of PRFs for the important goal of suicide prevention among veterans.