Self-Reported Suicidal Ideation as a Predictor of Suicidal Behavior Among Outpatients With Diagnoses of Psychotic Disorders

Suicidal behavior is especially common among individuals with psychotic disorders (1); approximately 5% die by suicide, and over 25% attempt suicide at some point (2–4). Risk is especially high after the first onset of symptoms (3, 5, 6) and soon after psychiatric hospitalization (7, 8). Many risk factors for suicidal behavior among individuals with psychotic disorders are similar to those in the general population: depressive symptoms, substance use disorder, and a history of suicidal behavior (2, 4, 8–11). A meta-analysis by Chapman and colleagues (12) found a stronger association between suicidal ideation and subsequent suicide death among persons with psychotic disorders than among those with mood disorders. Some risk factors, however, are specific to psychosis, such as higher educational attainment or premorbid intelligence (4, 8) and lower levels of negative symptoms (13). Consequently, assessing suicide risk among persons with psychotic disorders may be more challenging (14, 15).

Standard questionnaires administered at outpatient mental health visits can accurately identify increased risk of a subsequent suicide attempt and suicide death (16–19). Considering this evidence, both the Joint Commission (20) and the National Action Alliance for Suicide Prevention (21) have recommended systematic assessment of suicide risk. None of this evidence (16–19), however, has examined accuracy of questionnaires or other risk prediction tools among individuals with psychotic disorders.

In this study, we used data from seven large integrated health systems to examine the relationship between self-reported suicidal ideation and subsequent suicidal behavior among adult outpatients with diagnoses of schizophrenia spectrum psychosis or unspecified psychotic disorder. We address three related questions: What are the prevalence and correlates of self-reported suicidal ideation as assessed by item 9 of the Patient Health Questionnaire (PHQ-9)? How well does response to PHQ-9 item 9 at an outpatient visit predict suicide attempt and suicide death over the following 90 days? Does including responses to PHQ-9 item 9 at previous visits improve identification of risk?

Methods

The seven health systems participating in this research (HealthPartners; Henry Ford Health System; and the Colorado, Hawaii, Northwest, Southern California, and Washington regions of Kaiser Permanente) serve a combined population of approximately eight million members in nine states. Each system provides insurance coverage and comprehensive health care (including general medical and specialty mental health care) to a defined population. Members are enrolled through employer-sponsored insurance, individual insurance plans, capitated Medicaid or Medicare plans, and subsidized programs for low-income residents. Each health system’s members are generally representative of the service area in terms of age, race-ethnicity, and socioeconomic status (22, 23). Participating systems recommend use of the PHQ-9 at all mental health specialty visits and primary care visits for depression, but implementation was still in progress during the study period (24).

As members of the Mental Health Research Network, each health system maintains a research data warehouse organized according to the Health Care Systems Research Network Virtual Data Warehouse model (25). This resource combines data from insurance enrollment records, electronic health records (EHRs), insurance claims, pharmacy dispensing, state mortality records, and census-derived neighborhood characteristics. Responsible institutional review boards for each health system approved use of these deidentified data for this research.

The study sample included outpatient visits by members ages 18 or older during which a billing or encounter diagnosis of a psychotic disorder was recorded and a response to item 9 of the PHQ-9 was entered in the EHR (item 9: “Over the past 2 weeks, how often have you been bothered by thoughts that you would be better off dead or of hurting yourself in some way?”) Eligible psychotic disorder diagnoses included schizophrenia spectrum psychosis (ICD-9-CM codes 295.0, 295.1–295.3, 295.6, 295.8, and 295.9), schizoaffective disorder (ICD-9-CM code 295.7), and unspecified psychosis (ICD-9-CM code 298.9). All visits meeting these criteria from January 1, 2009, through June 30, 2015, were included, except at Henry Ford where only visits after implementation of a new EHR system on December 1, 2012 were included. Each patient could contribute multiple visits, and study outcomes (suicide attempt and suicide death) were observed for 30- and 90-day windows following each visit. Consequently, each suicide attempt or suicide death could be linked to more than one prior visit. This approach avoids informative censoring by using only information available at the time of the selected visit. Because recording of diagnoses may vary from visit to visit, sensitivity analyses included a broader sample of visits by individuals with a psychotic disorder diagnosis recorded at any time in the past 5 years.

For each visit, probable suicide attempts over the following 90 days were ascertained from all injury or poisoning diagnoses recorded in EHRs and insurance claims accompanied by an ICD-9-CM cause-of-injury code indicating intentional self-harm (E950–E958) or undetermined intent (E980–E988). Data from these health systems (17, 26–28) support both the sensitivity and positive predictive value of this definition (including both definite self-harm and “undetermined intent” diagnoses) for identification of probable suicide attempts. [A table in an online supplement to this article presents data supporting use of ICD-9 coding for identifying probable suicide attempts.] Visits were excluded from analyses regarding suicide attempts if the patient disenrolled from the participating health plan during the 90-day follow-up period, because suicide attempts after disenrollment would not be captured by health system records.

Suicide deaths were ascertained from state mortality records. On the basis of common recommendations (29–32), all deaths with any ICD-10 cause-of-death diagnosis of definite self-inflicted injury (X6–X84) or injury-poisoning with undetermined intent (Y10–Y34) were classified as probable suicide deaths.

Other potential correlates of suicidal behavior extracted from health system EHRs and insurance claims data included age, sex, race, ethnicity, responses to the PHQ-9 at prior visits, recent use of emergency department or inpatient mental health care, and recent diagnosis of self-inflicted injury or poisoning.

Initial descriptive analyses examined characteristics of patient visits according to response to PHQ-9 item 9. Primary analyses examined rates of suicide attempt and suicide death over 90 days following each visit according to response to PHQ-9 item 9 at the selected visit and according to response to item 9 at any time during the prior year. Secondary analyses examined rates of suicide attempt and suicide death during a 30-day risk period. Heterogeneity of suicide attempt or death rates across item 9 response categories was first evaluated by using chi-square statistics. Logistic regression models were used to compare odds of suicide attempt and suicide death across item 9 response categories, with and without adjustment for other potential predictors (recent suicide attempt or recent acute care mental health treatment). To account for possible correlation among multiple visits by a single patient, logistic models were estimated by using generalized estimating equations with the robust sandwich estimator (working independence covariance matrix).

Results

The criteria described above identified 32,982 eligible visits by 5,947 patients; 30,091 visits (91%) in mental health specialty clinics and 2,891 (9%) in primary care clinics. A total of 89 visits (.3%) were excluded because of fewer than 90 days of health system enrollment after the eligible visit. Of the 5,947 patients contributing to the sample, 2,239 (38%) contributed one visit, 912 (16%) two visits, 498 (8%) three visits, 345 (6%) four visits, 865 (15%) five to nine visits, and 833 (15%) 10 or more visits. Response to item 9 of the PHQ-9 regarding thoughts of death or self-harm was “not at all” in 24,654 visits (75%), “several days” in 4,635 (14%), “more than half the days” in 1,926 (6%), and “nearly every day” in 1,767 (5%). Duration of health system enrollment prior to the visit (so that complete data were available) was greater than 1 year for 28,035 visits (85%) and greater than 5 years for 20,119 (61%).

Table 1 displays characteristics of eligible visits according to response to PHQ-9 item 9. Patients reporting more frequent suicidal thoughts were younger. A greater proportion of those reporting more frequent suicidal thoughts were female, compared with male; black, Native American, or Native Hawaiian, compared with non-Hispanic whites; and more likely to have had recent mental health emergency department or inpatient care. Compared with non-Hispanic whites, the proportion of Asians reporting more frequent suicidal thoughts was smaller. However, all of these associations were modest in magnitude. Reporting more frequent suicidal thoughts was strongly associated with a diagnosis of self-harm in the past year and with a higher total PHQ-9 score.

Health system records identified 145 unique suicide attempts within 90 days of any visit, and 386 visits were followed by one of these events within 90 days. State mortality records identified 10 unique suicide deaths within 90 days of an eligible visit, and 19 visits were followed by one of these deaths within 90 days.

Rates of probable suicide attempt and suicide death over 30 and 90 days according to response to PHQ-9 item 9 are shown in the top portion of Table 2. Likelihood of suicide attempt within 90 days was strongly associated with response to item 9, increasing from .8% for those responding “not at all” to 3.5% for those responding “nearly every day.” A similar pattern of consistently increasing risk was seen for suicide attempts within 30 days. Likelihood of suicide death within 90 days did not vary significantly according to item 9 response. Although likelihood of suicide death within 30 days varied significantly across item 9 response categories (that is, significant chi-square test for heterogeneity), the pattern was neither consistently increasing nor decreasing.

A total of 28,728 visits (87%) visits were preceded by at least one prior PHQ-9 item 9 response. In the bottom portion of Table 2, rates of suicide attempt are compared according to maximum response to item 9 in the prior year (including the index visit). The pattern is similar to that seen with item 9 response at the sampled visit: a strong and consistent association with likelihood of suicide attempt and no clear relationship to likelihood of suicide death.

Comparison across rows in Table 2 also illustrates the performance of PHQ-9 item 9 in identifying risk of suicide attempt following visits with psychotic disorder diagnoses. For example, any response other than “not at all” to item 9 during a specific visit would have a sensitivity of 47% (180/386) to identify probable suicide attempt in the following 90 days. Positive predictive value or the probability that patients with a positive screen (a response other than “not at all”) would have a suicide attempt would increase from 1.6% for a response of “several days” to 3.5% for a response of “nearly every day.” When maximum response to item 9 in the past year was considered, any response other than “not at all” would have a sensitivity of 59% (228/386) to identify probable suicide attempt in the following 90 days. Positive predictive value would increase from 1.0% for a response of “several days” to 3.0% for a response of “nearly every day.”

Results of logistic regression models are shown in Table 3. Unadjusted odds of a suicide attempt within 90 days were approximately four times as great following a visit in which a patient responded “nearly every day,” compared with a visit with a response of “not at all” (model 1). This association was slightly reduced after adjustment for history of psychiatric hospitalization, psychiatric emergency department visit, or suicide attempt in the prior year (model 2). Analyses regarding maximum response to item 9 in the prior year (models 3 and 4) showed the same pattern. Parallel logistic models predicting risk of suicide attempt within 30 days yielded similar results, but confidence limits were wider, reflecting the smaller number of events.

Because recorded diagnoses may vary from visit to visit, sensitivity analyses included a broader sample of visits (N=97,106) by individual patients (N=17,778) with a recorded diagnosis of psychotic disorder at the index visit or at any encounter (outpatient, emergency department, or inpatient) up to 5 years previously. The results were not meaningfully different from those of the primary analyses. [Tables presenting these results are included in the online supplement.]

Discussion and Conclusions

In a sample of nearly 33,000 outpatient visits by almost 6,000 individual patients from seven large health systems, we found that response to item 9 of the PHQ-9 was a strong predictor of suicide attempt over the following 90 days among individuals who received diagnoses of psychotic disorders. Likelihood of a suicide attempt was strongly associated with both response to item 9 at the index visit and with the maximum response at visits during the prior year.

Among visits with diagnoses of psychotic disorder, the epidemiology of suicidal ideation (Table 1) was generally similar to that among mental health visits overall (16). Prevalence of self-reported suicidal ideation was moderately higher at visits with psychotic disorder diagnoses, compared with all mental health visits (16). As expected, reporting frequent thoughts of death or self-harm was strongly associated with more severe symptoms of depression and with a history of suicide attempt.

Limitations

Given the small number of suicide deaths in this sample, our data could not accurately assess the relationship between item 9 response and suicide mortality. Larger samples are needed to address this question.

Our identification of individuals with psychotic disorders relied on diagnoses assigned by treating mental health and primary care providers. Our previous research in these health plans regarding first-appearing diagnoses of psychotic disorders suggested that over 80% of these diagnoses would be confirmed by review of full-text medical records (33). Nevertheless, these findings should apply to patients receiving “usual care” diagnoses rather than to those undergoing research assessments.

Interpretation of these findings should consider false-positive and false-negative errors in ascertainment of probable suicide attempts and deaths. Previous research has indicated that attribution of death to suicide by medical examiners (32) has false-positive rates near zero and that documentation of definite or possible self-inflicted injury in records from these health systems (26, 27, 34) has false-positive rates of 10%−20% [see table in online supplement]. False negatives, however, may be more common. Up to one quarter of suicide deaths may not be identified by medical examiners (35, 36). Random misclassification of suicidal behavior (either false positives or false negatives) would introduce a conservative bias. Health system records may not capture suicide attempts, either because people do not seek care or because providers do not recognize or document self-harm. Our findings might not apply to suicidal behavior unrecognized by medical examiners or health care providers.

Treating providers were aware of PHQ-9 responses and may have changed or intensified treatment for patients reporting thoughts of death or self-harm. The elevated risk of a suicide attempt that we observed occurred despite any such treatment changes.

Our data did not allow us to examine mental state or severity of psychotic symptoms at the time of the outpatient visit. Self-reported thoughts of death or self-harm could be a less accurate indicator of risk among patients who are experiencing significant disorganization, as might occur in emergency department or inpatient settings.

Data from health system records apply only to patients engaged in care. Although more than half of persons attempting suicide or dying by suicide have had some outpatient health care contact in the preceding 90 days (37, 38), the remainder would be missed by visit-based screening with the PHQ-9 or any other assessment.

Context

The PHQ-9 was not designed as a measure of suicidal ideation or as a screening tool to identify suicide risk. We have previously reported that response to item 9 is a more accurate predictor of suicidal behavior than are responses to other PHQ-9 items assessing overall depression severity (17). Nevertheless, use of more detailed measures specifically designed to assess suicidal ideation or predict suicide risk (19, 39, 40) could yield improvements in both sensitivity and positive predictive value. Given that the PHQ-9 is widely used in outpatient mental health care settings, these findings should have practical utility.

Identification of suicide risk should not be limited to self-report questionnaires and information typically recorded in EHRs. Providers must also consider important social risk factors for suicidal behavior, such as job loss, bereavement, or relationship disruption. Because those risk factors are not systematically assessed or recorded in standardized fashion, we could not consider them in our analyses.

Comparison of these findings with previously reported findings in the broader population reveals important similarities and differences. Among individuals with psychotic disorders, thoughts of death or self-harm were strongly associated with severity of depressive symptoms as measured by overall PHQ-9 score. Similar to findings at visits for general outpatient mental health care and at primary care visits for depression (16), response to item 9 was a strong predictor of subsequent suicide attempt in this study. The relative increases in risk of suicide attempt associated with item 9 response in this sample, however, were smaller than those seen in broader outpatient samples. For example, the fourfold increase in likelihood associated with a response of “nearly every day” following a visit with a psychotic disorder diagnosis (Table 3) compares to an eightfold increase following a general outpatient mental health care visit or primary care visit for depression with the same item 9 response (16). Absolute risk differences associated with item 9 response, however, were larger in this sample. For example, we observed an absolute risk difference of 2.7% over 90 days between highest and lowest responses to item 9 in this sample, compared with the absolute difference of approximately 1% previously seen among all outpatient visits (16). These two findings regarding response to item 9 (smaller relative risk difference and larger absolute risk difference) are not in conflict; they both result from the higher baseline risk of suicide attempt among patients with psychotic disorders. We also observed that almost half of suicide attempts following completion of the PHQ-9 at visits with a psychotic disorder diagnosis occurred among patients responding “not at all” to item 9. This proportion is similar to that previously reported in general samples of mental health and primary care outpatients (16, 17).

Our data suggest that providers should consider responses to item 9 during past visits, rather than focusing only on the current response. Considering item 9 response only during the current visit (top portion of Table 2) would fail to identify more than half of the patients attempting suicide in the following 90 days. Expanding consideration to all item 9 responses in the prior year (bottom portion of Table 2) would increase sensitivity to approximately 60% while only modestly decreasing positive predictive value. These findings are consistent with the observation in broader outpatient samples that response to item 9 indicates elevated risk of a suicide attempt lasting for at least 12 to 18 months (16).

Implications

We believe that these findings should apply to outpatient mental health and primary care practices across the United States. The study included approximately 33,000 visits made by nearly 6,000 adults receiving outpatient treatment for a psychotic disorder in seven health systems across nine states. Participating health systems serve diverse populations, including large numbers from minority racial and ethnic groups and large numbers enrolled via Medicare, Medicaid, and other subsidized insurance programs for low-income individuals.

These findings have important implications for clinicians caring for people diagnosed as having psychotic disorders. First, our findings indicate that individuals with psychotic disorders provide clinically useful responses to simple self-report questions about thoughts of death or self-harm. Second, these results provide practical guidance to clinicians regarding the implications of response to PHQ-9 item 9. A response of “not at all” certainly does not imply absence of risk; over 40% of suicide attempts in this sample occurred among patients who did not endorse suicidal thoughts during the prior year. But endorsement of frequent thoughts of death or self-harm certainly indicates an increased risk of self-harm and a need for more detailed assessment and appropriate care planning. Of patients reporting thoughts of death or self-harm nearly every day, approximately one in 30 received care for self-harm or probable suicide attempt during the following 90 days. Our findings demonstrate that simple self-report measures such as the PHQ-9 can be useful in addressing the high risk of suicidal behavior among individuals with psychotic disorders.