Identifying Adolescent Suicide Risk via Depression Screening in Pediatric Primary Care: An Electronic Health Record Review

With the rollout of depression screening programs in pediatric primary care (1), and in accordance with recommendations to conduct universal adolescent depression screening (2), suicidality items on these screeners will help detect an increasing number of adolescents at risk for suicide. Furthermore, the rising rate of youth suicide in the United States (3) highlights a need for effective suicide assessment and prevention practices. Adolescence is a particularly crucial time to evaluate and target suicide risk given the increased prevalence of suicidal ideation, plans, and attempts during this period (4). To address this public health crisis, efforts are under way to implement youth suicide screening and prevention in primary care (5, 6). Although there is uncertainty as to whether widespread screening for suicide should take place in primary care (7), such screening is likely to occur as long as universal adolescent depression screening remains part of routine care. Thus, it is critical to determine how to optimize adolescent suicide risk identification and management in the context of depression screening.

This study provides an opportunity to understand suicide risk detected via routine adolescent depression screening in pediatric primary care and the types of follow-up care that adolescents receive in the year following an endorsement of suicidality. Moreover, this study enables the examination of an amalgamation of suicide risk identification approaches in the literature (8, 9) because suicidality was first determined by depression screening, and further assessment was then conducted with designated follow-up questions. Understanding the current suicide assessment and prevention landscape in primary care, including the proportion of adolescents at risk for suicide identified through depression screening, and the follow-up services that these adolescents receive will be crucial for informing best practices in this setting.

Suicide Assessment and Follow-Up in Primary Care

Primary care provides an opportunity to deliver screening and prevention services where individuals at risk for suicide are likely to present (10). The U.S. Preventive Services Task Force released a statement in 2014 reporting that “the current evidence is insufficient to assess the balance of benefits and harms of screening for suicide risk . . . in primary care” (7). Nonetheless, identification of suicide risk is often a natural consequence of adolescent depression screening, which is endorsed by the U.S. Preventive Services Task Force (2). Understanding suicide risk rates and follow-up procedures in the context of depression screening is critical for elucidating necessary modifications to make system-wide adolescent suicide screening suitable and sustainable.

After a suicide risk is identified, the literature recommends several follow-up actions that primary care providers (PCPs) may use (11). Research demonstrates that data extraction from electronic health records (EHRs) may underestimate rates of follow-up actions after a positive suicide screen (5). Thus, we used a combination of EHR extraction and manual chart review in this study to gain an in-depth understanding of follow-up actions implemented when suicide risk was detected via depression screening.

Study Goals

We explored 1-year follow-up for adolescents in pediatric primary care who endorsed suicide risk on a depression screener. The aims of this study were to describe levels of adolescent suicide risk detected via depression screening in a large primary care network, understand PCPs’ fidelity to the system’s suicide risk assessment procedures, and examine follow-up care for adolescents at risk for suicidality in the year after a risk was detected. Using these findings, we aimed to elucidate strengths and areas for improvement when depression screening is used as a springboard for adolescent suicide assessment and follow-up to guide refinements in such practices across pediatric primary care.

Methods

Setting and Screening Procedures

In this study, we utilized retrospective EHR data from a large pediatric primary care network (i.e., 31 practices) that is part of the Children’s Hospital of Philadelphia. The data spanned the 2014–2016 period, when an institutional recommendation advised to screen for depression at age-16 well-visits. Many PCPs elected to screen adolescents for depression at other ages. In 2017, the health care system shifted its depression screening guideline to include all well-visits for adolescents ages ≥12 years. Adolescents completed the Patient Health Questionnaire–Modified for Teens (PHQ-9-M) on a tablet or at a kiosk in the waiting room before their visit. The PHQ-9-M was developed on the basis of the Patient Health Questionnaire–9 (PHQ-9) (12), the Revised PHQ-9 for Adolescents (13), and the Columbia Diagnostic Interview Schedule for Children–Depression Scale (14). The PHQ-9-M retains the nine core items from the PHQ-9, with slight modifications to enhance its relevance to youth depression. The PHQ-9-M includes two supplemental items that assess suicidality (“Has there been a time in the past month when you had serious thoughts about ending your life?” and “Have you ever, in your whole life, tried to kill yourself or made a suicide attempt?”). These items were used to define suicide risk in this study given concerns that item 9 (i.e., “Thoughts that you would be better off dead, or of hurting yourself”) “combines a large range of severity of suicidal ideation, blurring and eliminating the distinction between higher and lower risk ideation indicators” (15). Total PHQ-9-M scores of 0–4 were considered within the normal range, 5–10 were flagged as subthreshold (i.e., mild symptoms of depression), and 11–27 were marked as threshold (i.e., moderate-to-severe symptoms of depression) (8). Cronbach’s alpha for the nine core items was 0.82.

The PHQ-9-M results populated the EHR for PCPs to review. The procedures for addressing suicidality endorsement are shown in Figure 1. This study was approved by the institutional review board of Children’s Hospital of Philadelphia.

Participants

Between September 1, 2014, and August 31, 2016, 12,690 adolescents completed the PHQ-9-M during a well-visit: 6,923 at the age-16 well-visit and 5,767 at other well-visits. In line with the institutional recommendation to screen all adolescents at the age-16 well-visit, 76% (N=6,981) of adolescents were administered the PHQ-9-M as recommended at age 16; 6,923 of these adolescents had complete PHQ-9-M data, which were included in these analyses (16).

Suicide risk rates were comparable for adolescents screened during their age-16 well-visit (thoughts of death or self-harm, N=388 [5.6%]; serious ideation in the past month, N=193 [2.8%]; and lifetime attempt, N=280 [4.0%]) and for those screened at other ages (thoughts of death or self-harm, N=259 [4.5%]; serious ideation in the past month, N=110 [1.9%]; and lifetime attempt, N=170 [3.0%]). Therefore, to obtain a comprehensive understanding of risk identification and follow-up procedures, we combined all age groups for our subsequent analyses.

In this study, we focused on the 643 adolescents (5.1% of the 12,690 screened) who endorsed a lifetime suicide attempt, serious suicidal ideation in the past month, or both. To gather detailed information on follow-up care for at least 20% of these 643 adolescents, we selected a stratified random subset of 150 adolescents (23%) for manual chart review. This subsample was composed of equal numbers of adolescents with PHQ-9-M scores in the normal (N=50), subthreshold (N=50), and threshold (N=50) ranges because suicide risk is present across depression symptom levels (17).

EHR Data Extraction and Manual EHR Review

An information analyst extracted EHR data to gather demographic information, PHQ-9-M results, and responses to the suicide risk follow-up questions. Manual chart review was crucial for gaining an in-depth understanding of the nature of follow-up that would have been missed in a discrete EHR data pull. Chart review codes captured various recommended follow-up actions for identifying and managing suicide risk (11): emergency department visits for psychiatric concerns, psychiatric hospitalizations, behavioral health referrals, consultation (i.e., communication regarding topics such as a patient’s health status) between PCPs and behavioral health providers, safety planning, providing crisis line information, therapy, and antidepressant prescriptions (determined by the U.S. Food and Drug Administration’s List of Antidepressant Drugs With Medication Guides) (18). The research team developed a codebook operationalizing each variable.

Two research team members (M.D. and V.R.) completed the manual chart review and coding, which included a review of encounters, notes (except mental health notes because of the institutional review board’s determination that a waiver of consent could not be granted for those notes), problem lists, medication lists, orders, and uploaded files within the EHR for the 13 months after the initial PHQ-9-M screening. This time frame made it possible to gather data from multiple primary care visits. For the 150 patient files, 20% (N=30) were doubly coded to establish interrater reliability. The mean interclass correlation coefficient across variables for the doubly coded reliability files was 0.88 (95% confidence interval=0.81–0.95), corresponding to good interrater reliability (19). For double-coded reliability files, coders met to discuss discrepancies and reconcile differences; consensus codes were included in the analyses. The other 120 files were coded independently by one of the coders (M.D. or V.R.).

Results

Suicide Risk Rates

The demographic characteristics of the adolescents are shown in Table 1. Among the 12,690 adolescents screened for depression with the PHQ-9-M, 5.1% (N=647) endorsed thoughts of death or self-harm, and 5.1% (N=643) endorsed lifetime suicide attempts, serious suicidal ideation, or both in the past month. Specifically, 3.6% (N=450) endorsed lifetime suicide attempts, 2.4% (N=303) endorsed serious ideation in the past month, and 0.9% (N=110) endorsed both.

Fidelity to the Suicidality Follow-Up Questions

For the 643 adolescents who endorsed serious suicidal ideation in the past month, a lifetime suicide attempt, or both, the minimum five follow-up suicidality questions were asked and documented in most cases (73%; N=472). When adolescents provided responses indicating increased risk to at least one of the first five follow-up questions (N=272), all eight questions were usually asked (87%; N=236).

Follow-Up in the 13 Months After PHQ-9-M Screening

Frequencies for each follow-up action identified in the manual EHR Review for the subset of 150 adolescents endorsing suicidality are presented in Table 2. The main findings from this chart review, which spanned the year after screening, were the following.

Monitoring and care coordination.

Provision of a suicide crisis line number was documented for 11% (N=17) of these adolescents, and 10% (N=15) had documented safety planning; 2% (N=3) had evidence of both. Most safety planning (93%; N=14) and documentation of crisis number provision (65%; N=11) occurred on the day of the PHQ-9-M screening. PCPs consulted with behavioral health providers for 5% (N=8) of the adolescents.

Referrals.

Behavioral health referrals were documented for 34% (N=51) of the adolescents on the day of the PHQ-9-M screening. Fewer of these adolescents (16%; N=24) were referred to behavioral health providers by PCPs in the following year. Of the adolescents who received a same-day behavioral health referral, 18% (N=9) also had evidence of referrals later in the 13-month window.

Intervention.

Documentation of therapy (46%; N=69) in the year after screening was more common than antidepressant treatment (19%; N=28). Most adolescents who received these treatments had services in place before their PHQ-9-M screening (therapy, 87% [N=60] and antidepressants, 82% [N=23]). Additionally, 7% (N=10) of adolescents presented to the emergency department for psychiatric reasons, and 5% (N=7) had a documented psychiatric hospitalization.

Discussion

The findings of this study yield important information on suicide risk levels and on immediate assessment and short-term follow-up procedures by PCPs and other health care providers for adolescents endorsing suicidality in the context of depression screening in pediatric primary care. The rate of lifetime suicide attempts was similar to the prevalence reported in epidemiological research with adolescents, whereas the prevalence of suicidal ideation was lower in our sample (4), likely because of our focus on serious ideation in the past month. Few adolescents endorsed suicidality in the past week. Thus, the rate and severity of suicide risk detected via adolescent depression screening in primary care seem relatively manageable. Additionally, our findings reaffirm that suicidality occurs across the depression spectrum. Therefore, detailed assessment of suicide risk is advisable even in the absence of elevated depression symptoms. These results highlight the utility of including suicidality questions within depression screeners, in addition to providing PCPs with structured follow-up assessment tools, demonstrating how merging existing approaches with suicide risk identification (8, 9) can further prepare pediatric PCPs to identify and assess suicide risk that may otherwise be overlooked. Additionally, efforts to enhance depression screening rates are likely to have an important ripple effect for improved identification of suicide risk.

PCPs conducted the follow-up suicide risk questions with a high degree of fidelity, suggesting the feasibility of implementing suicide assessment procedures in a large health system and the benefits of doing so within the EHR context. Yet, for about a quarter of adolescents, documentation of any of the follow-up questions was missing. Numerous factors (e.g., time and provider knowledge and comfort) likely affected implementation of suicide assessment and follow-up procedures. Further research is needed to systematically identify barriers and facilitators to implementing suicide assessment and follow-up care protocols in primary care.

Beyond the suicide assessment questions, suicide-specific follow-up actions were relatively sparse in the year after PHQ-9-M screening per a retrospective manual chart review. The benefits of widespread suicide assessment cannot be fully realized without such assessment being converted to consistent, appropriate follow-up care. Perhaps the low frequencies of certain follow-up actions can be attributed to adolescents’ limited endorsement of suicidality in the past week. Nonetheless, only a small subset of adolescents had documented safety planning or were given a crisis number, and even fewer received both. Of note, PCPs had access to a handout that listed behavioral health resources (e.g., the National Suicide Prevention Lifeline). Thus, our manual chart review likely underestimates follow-up actions that were covered in the handout but not necessarily documented in patients’ charts.

Behavioral health referrals were recorded for a subset of the sample of adolescents. It is possible that for some adolescents, particularly those exhibiting low depression symptoms and more remote suicide risk, referrals were not indicated. Nonetheless, from a prevention standpoint, especially given what is known about the recurrence and escalation of suicidality among adolescents (20), it would be ideal for all adolescents to receive referrals at the first sign of suicide risk should they require future services.

Most adolescents participating in therapy were enrolled in treatment before their PHQ-9-M screening. The exact nature of these services was often unknown. Moreover, consultation between PCPs and adolescents’ behavioral health providers was rare and, even when it occurred, may not have related to suicidality. We conclude that some behavioral health providers might have remained unaware of adolescents’ suicide risk, particularly if the PHQ-9-M was their first disclosure of suicidality. Our results underscore the importance of enhancing communication between PCPs and behavioral health providers to ensure comprehensive, high-quality care.

Although the findings of this study can help refine youth suicide assessment and prevention efforts in primary care, particularly within the context of existing depression screening efforts, we acknowledge some limitations. The manual chart review subsample included only 23% of adolescents at risk for suicide on the basis of their responses to screener items and, therefore, represents preliminary information on the types of follow-up care that adolescents received following endorsement of suicidality. Replication with a larger sample, for example, across hospital systems, will aid understanding of the generalizability of these findings. Nonetheless, we note that our manual chart review sample size was consistent with, or larger than, those in several published studies that used retrospective chart review (21, 22). Although suicide risk rates were similar for adolescents screened at their age-16 well-visit and for those of other ages, we cannot rule out the possibility that selective screening occurred for adolescents younger or older than 16 years.

Additionally, data on actual suicide attempts were unavailable; these data will be important to include in upcoming studies as a means to understand the effectiveness of primary care–based suicide screening and follow-up for reducing suicidal behaviors. An additional limitation is that lack of documentation in the EHR could not be distinguished from lack of follow-up (1). Furthermore, without access to behavioral health notes, the rates of certain follow-up actions are likely underestimates. In future studies, researchers may benefit from using a multimethod approach (e.g., direct observations and qualitative interviews) to gain a comprehensive understanding of current practices and areas for improvement. Although reassuring from a patient safety perspective, the low rate of suicide risk in the week before screening hindered our ability to assess fidelity to the system’s emergency procedures. Future work that overselects for adolescents endorsing imminent suicide risk will be important for understanding follow-up actions taken in these higher-risk situations.

Conclusions

The findings from this study offer valuable insights into the feasibility of incorporating suicide follow-up questions within depression screening programs in pediatric primary care and highlight ways to enhance provider- and system-level practices that can mitigate the risk of adolescent death by suicide. Given the level of fidelity to the suicide follow-up questions, it seems likely that, if similarly incorporated into the EHR, evidence-based assessments such as the Columbia–Suicide Severity Rating Scale (23) could replace existing procedures. Evidence-based suicide assessment provides an important guide for determining risk that may not be adequately captured through depression screening alone. Additionally, collaborative and integrated care models in which behavioral health providers, PCPs, families, and patients work together closely to address patient needs (24) will be critical to handle the volume of suicide risk likely to be detected as adolescent depression screening programs continue to expand. Relatedly, a care navigator (i.e., someone who provides support in the initiation and maintenance of care) can help connect adolescents to mental health services and facilitate information sharing between providers and families (25).

Building additional resources into the EHR system (e.g., safety planning templates) can also facilitate tailored referrals and intervention strategies for adolescents at risk for suicide, minimize bottlenecks in workflow, and enable EHR data extraction. Similarly, incorporating prepopulated text options into EHR note templates (e.g., SmartLists and SmartPhrases available within EHR systems such as Epic) can simplify documentation, clarify which suicide assessment and follow-up steps were implemented, and capture important outcomes. Electronic reminders to assess suicide risk and provide follow-up, along with functions that prohibit providers from closing a progress note until such actions are documented, may also be helpful. Taken together, refinements in suicide assessment, documentation, and linkages to care guided by the current findings may make a meaningful difference in the lives of adolescents, their families, and the providers who care for them.