Effectiveness in Regular Practice of Collaborative Care for Depression Among Adolescents: A Retrospective Cohort Study

Depression in adolescent populations is prevalent and harmful, but access to services remains limited. In the United States, 14% of those ages 13–18 meet criteria for a mood disorder (1). A similar percentage (12.5%) report having a major depressive episode in the past year (2); those episodes typically cause severe problems with youths’ functioning. Thus depression is interlinked with important aspects of adolescent life, including academic achievement and substance use (2,3).

The United States Preventive Services Task Force recommends screening for adolescent depression with “adequate systems in place to ensure accurate diagnosis, effective treatment, and appropriate follow-up” (4). However, although receipt of mental health services can improve depression, adolescents’ access to these services remains limited (5). Services are available in some areas via the National Network of Child Psychiatry Access Programs (NNCPAP) (6), although these vary geographically and are not integrated into every practice. Approximately 40% of adolescents with depressive disorders receive no treatment (7).

Collaborative care is one model used to integrate services for depression into primary care. Although varying in practice, collaborative care may be understood to include four essential elements: team driven (incorporating primary care providers, care managers, and psychiatric specialists), population focused (employing registries and systematic screening), measurement guided (emphasizing patient-reported outcomes); and evidence based (8). Collaborative care has shown effectiveness among adults in randomized studies (9,10) and among adolescents, although the evidence is more limited (11). However, studies of real-world applications of these models are rare, which may be related to poor reimbursement for this type of practice. Some evidence suggests effectiveness in regular practice for depressive outcomes and functioning (12), but these findings are based on adult populations. Establishing effectiveness of collaborative care for depression among adolescents outside formal research trials may motivate additional effort toward improved mental health services and help make the case for improved funding to sustain effective treatments for this group.

The goal of this retrospective study was to critically examine the primary clinical outcomes (six-month response and remission rates) of an ongoing collaborative care program for depressed adolescents embedded in a busy primary care practice in the Midwest.

Methods

This was a retrospective cohort study applying an intent-to-treat (ITT) approach to secondary, deidentified data with no possibility of reidentification during analysis. The study had human subjects research approval as a minimal risk study from the Mayo Clinic Institutional Review Board.

Intervention

The intervention was a quality improvement project that became standard practice in a community pediatric clinic, with eventual spread to family medicine clinics. The project, termed EMERALD (Early Management and Evidence-based Recognition of Adolescents Living with Depression), was an adaptation of an adult collaborative care model (12). Some patients might have found alternate routes to behavioral health, such as the NNCPAP-affiliated Psychiatric Access Line available in the same state (13). However, EMERALD was designed as a stand-alone set of integrated services.

Pediatric and family medicine patients with a depression diagnosis—a score of ≥10 on the nine-item Patient Health Questionnaire Modified for Adolescents (PHQ-9A) (14)—and without a diagnosis of bipolar disorder were eligible. Beginning in 2014, the PHQ-9A was routinely administered in well-child visits for patients ages 12–17. Prior to that, patients who came in for limited exams with a chief complaint of a mood-related, sleep, or school concern and those who were taking an antidepressant were systematically given the PHQ-9A; it was also administered to other patients at the discretion of the provider or care team. Recent public reporting data indicated a 71% rate of mental health screening at well-child visits among this system’s local sites in 2016 (15); EMERALD sites had rates up to 79%.

A primary care provider (PCP) made the initial diagnosis of depression; if clarification was needed, the patient could see a child and adolescent psychiatrist (CAP) or another member of the integrated behavioral health team. Patients who verbally agreed were then referred, often directly via “warm hand-off,” to a registered nurse care coordinator (RNCC) embedded in a pediatric practice who was trained in depression management and motivational interviewing. Adolescents ages 12–17 required patient and parental consent; those age18 (eligible if in high school) could provide consent themselves.

The RNCC met with the patient and one or both parents (or caregivers), explained the program, answered questions, and scheduled enrollment interviews with both the patient and one or both parents. This initial meeting included a review of confidentiality; the RNCC explained that what the adolescent said was confidential, but because of mandated reporting, questions of safety or abuse would necessitate telling both parents. Addressing this issue in the beginning allowed for questions and was intended to provide reassurance for both adolescents and parents.

Every patient enrolled into EMERALD was administered case-finding tools for depression (PHQ-9A), anxiety (16,17), substance abuse (18), attention-deficit hyperactivity disorder (19), and bipolar disorder (20) to identify comorbidities or exclusions. The RNCC interviewed patients and parents separately and together. Enrollment interviews also covered current stressors; social, family, and medical history; school performance and other activities; and psychiatric history, including medications, counseling, or hospitalization. The RNCC established behavioral activation goals, assisted with internal and external referrals, and started to develop a relapse prevention plan in collaboration with the patient.

In weekly case review meetings with the RNCC, the CAP made recommendations regarding medication, therapy, or a consult with the CAP. The RNCC then passed on this input (for example, drug and dosing recommendations and diagnostic clarifications) to the PCP, who made the final decision. The RNCC then contacted the patient and parents with recommendations.

The RNCC also had weekly or biweekly contact with the adolescent, either by telephone or face to face, and contacted one or both parents biweekly. Fidelity to these contacts was generally good. We lacked data on successful and unsuccessful contacts for this cohort, but unpublished early data on EMERALD indicated that 61% of contacts were successful. The frequency of visits depended on the severity of depression; the RNCC established the time frame with both the adolescent and parents. Motivational interviewing was a core skill of the RNCC, and thus it was used in essentially every interaction. Its use, along with behavioral activation, was key to engaging the adolescent in care coordination, including addressing the willingness of the adolescent to engage and identifying his or her specific concerns and goals for each visit and the next.

Psychotherapy (primarily cognitive-behavioral therapy) and other services and referrals via a licensed clinical social worker were available to all patients and into the primary care setting. Patients who had significant concerns regarding comorbidities or patients who were not improving were seen directly by the CAP or referred on for higher-level services, or both. The CAP was available in the clinic during designated times; providers were encouraged to discuss concerns then. The CAP made medication and treatment recommendations and diagnostic clarifications and saw patients when necessary.

Patient outcomes were tracked via a clinical registry. The primary discharge criterion was sustained depression remission (PHQ-9A score of <5 for three consecutive months). For patients who did not enter EMERALD, other support included usual care with a PCP or provision of a list of community providers (for example, for cognitive-behavioral therapy) to the patient or the PCP. Other referrals could be made to social work, CAPs, and individual and group therapy. PCPs for non-EMERALD patients could also attend a weekly huddle session for pediatric integrated behavioral health or could contact the RNCC to discuss specific cases.

There was no additional unique billing or payment for collaborative care—only what was already present in fee-for-service billing (for example, therapy and PCP visits). Almost 50% of the patients in the practice were self-insured; combined with the promise of a better model of adolescent depression care and existing challenges to accessing the services of a psychiatrist, this was sufficient support to obtain permission from clinic leadership and move forward without additional fees.

Data and Sample

Data come from electronic health records for individuals eligible for EMERALD (see below) during the observation period (January 2010 to December 2015) at a large multispecialty practice in the U.S. Midwest. The sample included adolescents (ages 12–17 or age 18 and still in high school) from pediatric and family medicine practices who had a PHQ-9A score of ≥10 and a diagnosis of major depressive disorder or dysthymia (ICD-10 diagnosis of persistent depressive disorder) during the observation period and who were at least six months past baseline at the time of data extraction (this allowed for follow-up assessment). Because eligibility required a PHQ-9A of ≥10, the sample was limited to those with nonmissing baseline scores. Baseline was defined as the date of eligibility—that is, the date with a qualifying PHQ-9A and diagnosis during the period when EMERALD was in place. Individuals were excluded if they elected not to have their clinical data used for research. The final sample was 162 EMERALD and 499 comparison patients (total N=661).

Measures

Our two primary outcomes were both based on PHQ-9A scores (the PHQ-9A was rolled out during the observation period, resulting in some regular PHQ-9 scores in the data [see sensitivity analyses below]). Outcomes were six-month remission of depression (six-month score of <5) and six-month treatment response (>50% reduction from baseline). Attrition-related missingness was assumed nonignorable, that is, missing not at random (21), because missing depression scores were likely associated with the missing values themselves or unobserved confounders. Therefore, we used last observation carried forward (LOCF) for those without a score during the follow-up window to render the most conservative comparisons (this was borne out in sensitivity analyses [see below]). We measured two follow-up windows because of changes in quality reporting that happened midstudy: 150–210 days and 120–240 days. We chose the wider window because it reflected quality measurement at the time of writing (see sensitivity analyses).

Our primary independent variable was participation in EMERALD. Comparison patients included those eligible for EMERALD but for whom EMERALD was not yet available at their clinical site (N=360) and those who opted out of EMERALD because of patient, parent, or provider choice for the patient not to participate (N=139). Characteristics were largely similar between different sets of comparison patients. [A table in an online supplement to this article presents data on baseline and follow-up differences for the samples.] We also conducted sensitivity analyses regarding different comparison groups.

Potential confounders added as covariates included demographic characteristics—patient age, sex, race-ethnicity (coded as white non-Hispanic versus other; some groups were very small, creating problems with perfect prediction in models), and use of an interpreter.

We created covariates for clinical adjustments, including baseline PHQ-9A score. In addition, because EMERALD participation potentially entailed more observations or time observed, we controlled for the number of depression score measurements in the extracted data and the number of days from baseline to the latest measure in in the extracted data. Because these variables were based on total data extracted (rather than the six-month follow-up window), the variables could count observations and days beyond the six-month window to offer the broadest possible look at patterns of individual contact. We also measured whether individuals had a depression measure in the six-month follow-up window (versus requiring LOCF for the outcome).

Main Analysis

Analyses were conducted with Stata 14 (22). Our main analyses consisted of adjustment by using logistic regression and propensity score matching, both with robust standard errors. First, we used logistic regression models controlling for all covariates to produce marginal adjusted predictions (adjusted rates) of outcomes and contrast tests of those estimates. Next, for propensity score matching, we used Stata’s teffects psmatch command for a 2:1 nearest-neighbor match of comparison with EMERALD patients, using matching with replacement and a caliper (maximum difference in propensity score) of .1, to obtain two or more matches within the caliper. Matching with replacement increases the set of possible matches, minimizing bias, and allows matching for intervention and comparison patients to approximate the population average treatment effect (23). Matching in psmatch also tests for overlap between groups. To ensure balance, we produced box and density plots of propensity score balance [see figure in online supplement]. We found that age and baseline depression score had worse individual balance after matching. After removing them, overall balance improved, and results were slightly more conservative for treatment response; thus the main results omit age and baseline PHQ-9A. We obtained average treatment effects as the difference in potential outcomes between EMERALD and comparisons.

Sensitivity Analyses

We conducted several sensitivity analyses to assess robustness (not shown). All sensitivity analyses described here produced effect sizes similar to or less conservative than the main results. For simplicity, we used logistic regression to test multiple variations on data and sample, including a narrower follow-up window, using PHQ-9 and PHQ-9A scores, and limiting comparisons to those for whom EMERALD was not available. We also tested inverse probability weighting with regression adjustment and nearest-neighbor Mahalobis distance matching. Finally, other approaches to attrition included complete case analysis; multiple imputation; and a Heckman model (Stata command heckprobit), which incorporates attrition by using a selection model. In addition, we performed pattern mixture models for nonignorable missing data (user-written Stata command rctmiss [24]). Although rctmiss is designed for experimental data, it allows the same logistic regression adjustment as in our main models while also comparing effect sizes across plausible values for missing data. Our pattern-mixture models tested assumptions in which missing EMERALD or comparison patients had remission or response rates that were either similar to their own observed rate or similar to the other group’s observed rate. Pattern-mixture model results were consistent with main results. The most conservative results from the pattern-mixture models—those that assumed that patients with missing values in one group would have had an outcome rate similar to the observed rate in the opposite group—were closest in effect size to the LOCF findings.

Results

Table 1 presents data for the EMERALD and comparison patients. Most were female and white non-Hispanic; the mean age was approximately 15 years. A larger proportion of EMERALD patients were female and nonwhite. EMERALD patients had more observation occasions and more days of observation postbaseline than the comparison patients, indicating the need to adjust for these characteristics.

Adjusted rates of outcomes and differences between groups, based on logistic regression and propensity score matching, are shown in Table 2. In both sets of models, EMERALD patients had treatment response rates of 43%−44%, versus 28%−30% for the comparison patients. Remission rates in both sets of models were 29%−31% for EMERALD patients, versus 19%−20% for comparison patients. Differences in treatment response (14–15 percentage points, p<.01, in both models) and remission (10–11 percentage points, p<.05, in both models) indicated better outcomes for EMERALD patients.

Discussion

Our findings are consistent with literature on the benefits of mental health services for adolescents in improving depression (11,25). We conducted multiple sensitivity analyses regarding adjustment for confounders, bias, and measurement issues within our ITT design to assess robustness. Our main results were as conservative as or more conservative than estimates from all other adjustments and specifications we tested. Moreover, balance was good in the propensity score analysis. We are thus relatively assured that these findings are robust within the limitations of observational designs.

Our results support the modest-to-moderate benefit in trials of collaborative care for adolescents (11) and the limited evidence for depression specifically. Whereas Clarke and colleagues (26) found a modest, nonsignificant effect, other trials indicated significant benefit across continuous and binary outcomes (27,28). Richardson and colleagues (28) found collaborative care treatment response and remission rates approximately 30 percentage points higher than those for usual care. The smaller differences found in this study could be explained in several ways. Because both our study and the study by Richardson and colleagues found approximately 20% remission in usual care, differences in usual care are likely not a key factor. Instead, reasons might include a less effective intervention; lower exclusivity owing to our practice-based, nontrial setting; or other methodological decisions (for example, Richardson and colleagues assumed ignorable missing data for attrition). Further studies in regular practice may help tease out these contributing factors.

The primary limitation of our study was nonrandomization. Although we adjusted for observable variables, potentially unobserved confounders could have been responsible for at least part of the treatment effect size. Triage in entering EMERALD may select for adolescents most likely to participate and improve. We attempted to control for differences in follow-up, contact-observation, and attrition between the two treatment groups. However, we cannot rule out residual bias attributable to these factors. Other possible limitations are related to changes in systematic collection of PHQ-9A scores, because the setting was a regular practice environment without dedicated research personnel. As such, variations in practice in this regard may have biased the sample. Moreover, number of contacts and measurements was a key source of bias. As shown in full regression results, having more contacts was associated with greater odds of improvement [see table in online supplement]. Although we controlled for it, contact-observation was notably different between groups and was thus a limitation. In addition, we lacked data on use of treatment or therapy external to this health system; a difference in use of such services between groups is a possible source of bias. Also, although contacts with the RNCC were scheduled calls and the model of care was not primarily centered on office visits, our lack of data on no-shows is a limitation.

This study reflects a regular practice setting. Usual care was not controlled in any way and may have involved use of specialty care, although any such use might have biased results more conservatively. Also, this population of mainly white non-Hispanic patients had access to care sufficient to detect depression; results should be generalized with caution. Because of the setting, self-selection and engagement among patients are important considerations. Anecdotally (on the basis of internal, non–peer-reviewed data), both adolescents and parents commented that warm hand-offs were important first contacts with the RNCC. Adolescents and parents also commented that it helped to have RNCC calls scheduled around their other commitments. Further testing of elements that improve engagement would be a useful next step, especially considering underrepresentation of males. We can only speculate about the reasons for the underrepresentation of males (lower rates of recognition of depression, help seeking, or diagnosis among males because of clinical, social, or cultural factors). However, better addressing adolescent male depression would improve practice and EMERALD.

Maintenance of this model in practice despite any reimbursable codes for care coordination suggests benefits beyond the outcomes addressed here. For instance, internal surveys show, anecdotally, greater PCP comfort in diagnosing and treating adolescent depression, higher perceived value of care coordination, and decreased difficulty with referral. However, such purported benefits require rigorous study. The maintenance of EMERALD in regular practice also suggests that payment models allowing for reimbursement may be worthwhile.

Conclusions

We found that collaborative care for adolescent depression within a busy pediatric and family medicine outpatient practice was associated with higher rates of improved mood at six months, compared with usual care. Practices may want to consider implementation of an analogous model of care to improve depression outcomes among adolescents. Future research on predictors of participation may help with targeting for better results. Payment for this approach based on outcomes seems warranted on the basis of results in trials and practice.