Comprehensive, multifaceted disease management programs can improve outcomes among depressed patients (1–10). Yet cost, staffing requirements, lack of trained providers, and other barriers limit their use. Care management by telephone can improve efficiency and reach more people (11,12). However, it requires considerable resources and has built-in inefficiencies (such as unproductive attempts to reach patients). Internet-delivered depression care, if effective, could circumvent these barriers. Randomized trials of online cognitive-behavioral therapy (CBT), counseling, and psychoeducation have shown promising results (13–28).
We developed and evaluated a patient-centered, interactive care management and patient self-management program, delivered through the Internet, to reduce the overall burden of recurrent or chronic depression and bipolar disorder. It is integrated with ongoing depression care, provides most services online, links to patients’ electronic medical records, and provides clinicians with clinical decision and panel management support. This article presents results for participants with recurrent or chronic depression.
We conducted an open, randomized clinical trial with blinded interviewers to answer the question: Can Internet-delivered care management have a sustained favorable impact on the burden of recurrent or chronic depression among patients receiving outpatient specialty mental health care? The main outcome was depression severity over 24 months. The intervention was available in year 1 but not in year 2.
Patients were recruited from two Kaiser Permanente specialty mental health clinics in Northern California. Study participants were receiving antidepressants, psychotherapy, or both for recurrent or chronic depression.
Institutional review boards of the Kaiser Foundation Research Institute and the University of California, San Francisco, approved the study. Participants gave written consent after study procedures and risks were described.
A list of eligible individuals was created by using electronic medical records. Eligible individuals were 18 years or older and not currently hospitalized; they had a diagnosis of recurrent or chronic depression (DSM-IV 296.3 or 296.35) (29) that was recorded between September 2001 and April 2003, with no recorded diagnosis of bipolar disorder.
Participants were recruited between April 2003 and October 2004. All candidates meeting eligibility criteria received a letter signed by their treating clinician introducing the study and an opt-out postcard. Telephone screeners confirmed eligibility. Using questions from the Structured Clinical Interview for DSM-IV (SCID) (30), screeners confirmed that the individual had at least two lifetime episodes of depression or chronic depression and no history of manic or hypomanic episodes, was fluent in English, used the Internet at home, and would get his or her psychiatric care during the coming year at the same clinic. Baseline interviews for eligible candidates were scheduled at their clinic.
A statistician blind to candidates’ identities prepared random treatment assignments in blocked sets of four (two randomized to the intervention and two to usual care). Treatment assignment was revealed at the end of the interview.
Both treatment groups received usual specialty mental health care, which is similar across psychiatry departments of Kaiser Permanente of Northern California. Care begins with phone screening by a licensed clinician, assignment of a primary clinician, and an intake visit. The primary clinician designs an individual treatment plan combining medication and case management; group, family, and individual psychotherapy; and psychoeducation, as appropriate. Intensive outpatient programs, residential crisis care, and full hospitalization are available when needed. Treatment is comparable to standard community care, except that it places more emphasis on case management, provision of a continuum of care, and group rather than individual psychotherapy.
eCare for Moods, a secure, password-protected Web site that was developed by clinicians, patients, and researchers (31,32), is based on established care management (2,6) and behavior therapy strategies (33–37) for depression. eCare was available 24 hours a day, seven days a week, for 12 months. At least twice daily during clinic hours the psychiatric nurse care manager (eCare manager) assigned to each participant monitored the site. eCare offered participants personalized self-monitoring, secure messaging with their eCare manager, depression education grounded in established CBT interventions emphasizing behavioral activation and social support, a monitored discussion group, problem-specific advice, a personal database, task lists, and an appointment calendar. It gave eCare managers and other mental health providers clinical decision and panel management support and regularly downloaded information from participants’ electronic medical records for use by both the participants and the providers.
These online functions were designed to help participants self-manage their depression and receive clinical intervention as needed. To self-monitor their depression, participants created personal depression profiles online, including their DSM-IV symptoms, personal behavioral indicators of depression, and early warning signs. They also tracked their disability, medication adherence, side effects, and alcohol and drug use online. Graphs in eCare displayed all monitoring information.
Eight education modules emphasized the effects of thoughts, actions, sociability, and productivity on mood and provided general health education. Supplementing the modules were interactive worksheets and links to Web sites that were vetted for accuracy and objectivity.
Participants could enlist a “care partner” (family member or friend) for emotional support and company during pleasurable activities. With permission, care partners could contact eCare managers directly if they observed worsening symptoms or suicidality. Care partners had their own Web site, which included evidenced-based guidelines for helping depressed individuals improve their mood.
eCare managers followed a panel of participants by reviewing their Web site activity, using secure messaging, and responding to alerts automatically generated by participants’ answers to monitoring questions—an important aspect of the eCare managers’ role. Alerts were displayed on eCare managers’ screens in order of urgency (emergency alerts first, then urgent alerts, and then notifications). Depending on the alert, the eCare manager notified a crisis team, telephoned the participant, or sent a secure message.
To ensure full integration of eCare with usual care, eCare managers knew each participant’s treatment plan, consulted with their treating clinicians, and followed clinic policies and guidelines. Treating clinicians could access eCare data through the Web site.
A clinical nurse specialist trained participants and care partners to use eCare in a one-hour, in-clinic session. Some participants were trained by telephone or instant messaging. The nurse specialist and a psychologist supervised the eCare managers. [An online data supplement to this article describes intervention details.]
In-person baseline interviews were conducted in the participant’s clinic. Telephone interviewers blind to the participant’s treatment group conducted follow-up interviews six, 12, 18, and 24 months after enrollment. Participants received $25 per interview.
All interviews included an assessment of depressive symptoms experienced over the previous two weeks with questions adapted from the SCID (30). This provided a psychiatric status rating (PSR) of the severity of current depression on a 6-point scale derived from the research diagnostic criteria (38): 1, no symptoms; 2, mild depression; 3, moderate impairment; 4, marked impairment but below DSM-IV criteria for an episode; 5, meets criteria for an episode; and 6, meets criteria for a severe episode. A PSR of ≥3 indicated that depression was definitely present.
At each follow-up, interviewers obtained a current PSR. Using the previous PSR and current PSR as anchors, interviewers used the semistructured Longitudinal Interval Follow-up Evaluation (LIFE) (39) to create a week-by-week timeline of PSRs for the six months before each follow-up. The LIFE employs association techniques to prompt participants to remember their mood at various points in the past six months. It provides reasonably accurate recall for depression with no evidence that accuracy deteriorates with longer recall intervals (40).
All interviews included the Sheehan Disability Scale (41); the mental health, general health, vitality, role-emotional (ability to accomplish tasks unimpaired by mood), and social functioning scales from the 36-item Short-Form Health Survey (SF-36) (42); questions from the Alcohol Use Disorders Identification Test (43,44); and questions about use of psychotropic medications during the previous three months, available social support over the previous two weeks, and employment and use of sick leave over the previous six months. Available social support was measured by a six-item scale consisting of questions about participants’ connections to family and friends.
To measure satisfaction with care, we developed new questions and conducted confirmatory factor analyses (45) to group the questions them into five scales: general satisfaction with psychiatric care, satisfaction with the personal and empathic nature of care, satisfaction with learning new coping skills, confidence in coping with the mood disorder, and complaints about access to care. The factor analyses yielded a Bentler index of .95 (46), indicating excellent fit to this subscale structure.
Kaiser Permanente databases provided information on prescriptions filled and health service utilization. Cost estimates for eCare included direct staff costs for monitoring, training, and supervision, plus organizational overhead. Nurse logs of time spent on eCare activities helped determine staffing costs. One-time development and technical support costs were excluded. Other costs were obtained from Kaiser Permanente’s Cost Management Information System, which allocates overhead and other indirect costs into outpatient visits, hospitalizations, laboratory, and other clinical services.
The primary hypothesis was that eCare would be associated with more favorable change over time than usual care in weekly measures of depression severity. For the primary test of this hypothesis, we focused on the time-by-treatment interaction effect, which was included in a repeated-measures model of depression severity. We fit the repeated-measures model to the series of 105 weekly depression severity measurements, starting with the baseline measure and including each subsequent weekly measure throughout the 24-month study period.
Secondary outcomes (current depression severity, depression presence, Sheehan Disability Scale, and SF-36 scales) were measured at baseline and at six, 12, 18, and 24 months. Again, we used repeated-measures analyses and focused on time-by-treatment interaction effects. In each repeated-measures analysis, the time-by-treatment interaction effect was interpreted as the effect of eCare on change in the outcome over time. All repeated-measures analyses of outcomes were implemented with generalized estimating equations (SAS, version 9.2, procedure GENMOD), which accommodate missing values.
Analyses of satisfaction scales focused on satisfaction levels reported in the four follow-up interviews. Satisfaction was not measured at baseline. The main effect of treatment on satisfaction was adjusted for baseline variables that correlated strongly with satisfaction.
Cohen’s d, a standardized measure of effect size, was calculated to compare our findings with those of other studies (47). It measured the standardized mean difference in depression severity attributable to the eCare intervention. Values of .2 suggest a small effect, .5 a medium effect, and .8 a large effect (47).
The number needed to treat (NNT) to obtain one additional participant free of definite depression (PSR <3) was computed at 24 months.
We compared health care utilization and costs during two years before and after enrollment, using t tests for the pre-post difference in differences—that is, the pre–post differences for eCare participants minus the pre-post differences for usual-care participants.
Analyses of baseline characteristics, days’ supply of medication, health service use, and cost included all participants randomly assigned to a treatment group. Analyses of interview data omitted three participants with no postbaseline data. All significance levels are two-sided.
Screeners assessed 304 patients for eligibility: 178 were ineligible, 23 refused to participate, and 103 consented. Consenting patients were assessed at baseline and randomly assigned to a treatment group—51 to eCare and 52 to usual care. Follow-up interviewers blind to treatment group assessed outcomes for 97% of participants (N=100) six months after baseline, 95% (N=98) at 12 months, 73% (N=75) at 18 months, and 83% (N=86) at 24 months [see the online data supplement for a flowchart]. Attrition was greatest in year 2, and somewhat higher among intervention participants, although not significantly. Participants who were not interviewed at 18 or 24 months did not differ significantly on key baseline variables (depression severity, sociodemographic characteristics, and antidepressant use) from those who were interviewed at these follow-up points.
Table 1 presents data on participants’ characteristics. Mean depression severity did not differ significantly by treatment group, but 22 of the eCare participants met criteria for being in an episode of depression (PSR ≥5), compared with 15 participants in usual care. Treatment groups did not differ significantly in baseline characteristics except for marital status.
Of the 51 eCare participants, 46 (90%) received the one-hour training. During the year that eCare was available, the 51 participants made, on average, 60 visits to the Web site (median 34; interquartile range 9–84).
Participants’ responses to monitoring questionnaires generated 17 emergency alerts, 93 urgent alerts, and 1,134 notifications. One participant’s emergency alert resulted in an emergency consultation; this participant made a subsequent suicide attempt.
On average, eCare participants completed 4.6 of the eight education modules (median 5); 22 participants (43%) completed all eight. Participants received an average of 10.7 messages (median 8) from their eCare manager and sent an average of 8.4 messages (median 4). Participants averaged 10.5 visits to the discussion group, which was available only to those who completed all eight education modules. Nine care partners made, on average, six visits to their separate Web site (median 5, maximum 20).
Figure 1 shows the percentages of participants who visited eCare in the first and second six months of the program and the program components that they used. Usage declined over time, except for participation in the discussion group, which increased in the second six months.
Postbaseline interviews were conducted with 100 participants (49 eCare and 51 usual care). Figure 2 shows the weekly course of depression severity over two years. In the first year, depression severity decreased steadily in the eCare group, and by one year, the mean PSR was about 1 point below baseline. This reduction persisted throughout the second year, when eCare was no longer available. Usual-care participants improved about half a point in the first five months and little thereafter. Overall, on the 6-point PSR scale, depression severity among eCare participants decreased nearly three-quarters of a point more, on average, than among usual-care participants (time-by-treatment estimate=–.74 over two years; 95% confidence interval [CI]=–1.38 to –.09, p=.025).
Adjustment for the level of depression severity at baseline yielded a similar estimate of eCare’s effect on the trend in depression severity: the time-by-treatment estimate was –.76 (CI=–1.39 to –.13, p=.018). Adjustment for marital status (the only baseline variable that differed significantly across the two treatment groups) yielded a similar estimate of eCare’s effect on the course of depression severity.
Assessments of the trend in depression severity, as measured every six months rather than weekly, indicated greater improvement among eCare participants (estimate=–.52 over two years; CI=–1.02 to –.01, p=.047) (Table 2). The proportion of participants who continued to have definite depression (PSR ≥3) was lower among eCare participants than among usual-care participants (estimate=–.24 over two years; CI=–.46 to –.03, p=.026).
Table 2 also presents data for six other outcomes. Improvement in general mental health (SF-36) was significantly greater among eCare participants (p=.002). Trends toward favorable treatment effects were seen for eCare in level of disability (Sheehan Disability Scale) (p=.073), vitality (SF-36) (p=.079), and role-emotional (SF-36) (p=.081).
Table 3 presents pooled main effects for five satisfaction scales. eCare participants reported more satisfaction with their psychiatric care (p=.003), more satisfaction with learning new coping skills (p<.001), and more confidence in coping with their mood disorder (p=.006).
The Cohen’s d for the standardized effect of eCare (time-by-treatment effect over two years) on depression severity was .60. The NNT was 8.3 at 24 months, when 43% of eCare participants were depression free, compared with 30% in usual care.
Among eCare participants, completion of more education modules was positively related to less severe depression at 24 months (p=.036). Other measures of Web site use were not associated with depression severity at 24 months.
+
Medications, services, and cost
At baseline, nearly all participants (99%) reported taking antidepressants for at least one month during the previous three months. Among interview respondents at 12 months, self-reported antidepressant use decreased to 88% (N=45) in the usual-care group and 87% (N=41) among eCare participants. Among those interviewed at 18 months, antidepressant use was 85% (N=34) among usual-care participants and 89% (N=31) among eCare participants. At 24 months, the proportion using antidepressants was lower in the eCare group—80% (N=32) versus 96% (N=44) (p=.04). Prescription fill data indicated significantly less antidepressant use among eCare participants than usual-care participants at 24 months (p=.046).
Data on use and cost of specialty mental health and other health services for all 103 participants were analyzed for two years before and two years after enrollment. [The online data supplement to this article includes two tables presenting these data.] Both groups had fewer specialty mental health visits after enrollment than before. The estimated cost to deliver eCare for one year was $345 (2009 USD) per participant. When use during the two years before and the two years after enrollment was compared, no significant differences were found between treatment groups in changes in mean use of specialty mental health or other health services or associated costs (including the cost of eCare). Nurse logs indicated that a nurse could manage 200 patients in ten hours a week.
eCare for Moods is among the first comprehensive online depression care management programs supporting both patients and clinicians to be tested in specialty care. Compared with patients randomly assigned to usual specialty mental health care, patients randomly assigned to eCare improved significantly more on the primary outcome—weekly depression severity over two years. Furthermore, eCare participants were depressed less often, experienced better overall mental health, were more satisfied with their mental health care and coping skills, and felt more confident handling their depression. The eCare program’s favorable effects were sustained over two years, suggesting that eCare has lasting benefit. Because the proportion of eCare participants taking antidepressants did not increase over time, its effectiveness cannot be attributed to increased use of medication.
The treatment effects of eCare are comparable to those obtained in randomized clinical trials of care management (48) and Internet-based programs (15) that target depression. A meta-analysis of 37 collaborative care trials found the overall standardized effect size of the nine studies that reported 24-month follow-up data was .15 (CI=–.03 to .34) (48). The standardized effect size in our study was .60. It should be noted that programs in the meta-analysis were delivered in primary care, where the control groups probably received less intensive treatment than the usual-care participants in our study.
A meta-analysis of 12 randomized trials of Internet-based depression programs found that those that involved support from a therapist and those with wait-list comparison groups were the most successful (average standardized effect sizes of .61 and .56, respectively) (15). Interventions without the support of a therapist and those with usual-care comparison groups had small effect sizes (.25 and .23, respectively). Other studies show a similar pattern (24–27,49,50).
This study had important strengths. It was a long-term, randomized trial with interviewers blind to treatment condition that was conducted in real-world specialty mental health settings. Extensive information on important outcomes was gathered over two years, including one year after the intervention ended. It demonstrated that an Internet-delivered program that gives depressed patients self-management tools and provides almost continuous access to the care delivery system and care management can produce better outcomes than usual care among patients being treated by mental health specialists. These findings offer hope to patients for whom depression is a lifelong burden. Given the cost-efficiencies of delivering care online, our findings could lead to greater access to care for people with mood disorders.
This study had limitations. First, the small sample limits the precision of our findings. Second, some eCare participants may have reported fewer symptoms than usual-care participants because they wanted to emphasize improvement, or they may have reported more symptoms because of heightened self-awareness. Third, the ability to generalize the results is limited by the inclusion of only English-speaking, Internet-using patients from two Kaiser Permanente clinics. Fourth, the p values reported are from multiple tests of secondary outcomes and should be interpreted cautiously.
Replicating this study with larger and more varied samples in other practice settings, especially primary care, would clarify the clinical effects and costs of the intervention and the generalizability of the findings. Similar programs could be beneficial for conditions such as schizophrenia, chemical dependency, diabetes, and asthma. Self-care, continuous care management, and clinical decision support—all integrated and delivered online—can improve the lives of patients with recurrent or chronic depression.
The eCare for Moods study was funded by Eli Lilly and Company and the Kaiser Permanente Garfield Memorial Fund. The Permanente Medical Group, Inc., of Northern California provided in-kind support. The Kaiser Foundation Health Plan provided usual care and medications to study participants according to health plan benefits. The study sponsors and funders had no role in the design of the study; in collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit it for publication. The authors thank the study participants. They gratefully acknowledge Paul Wilson, M.D., Ph.D., and Adele Adams, R.N., and the staff of the Psychiatry Department, Kaiser Permanente Redwood City Medical Center; and Mark McCormick, M.D., Anne Klevay, R.N., Sheri Ertzner-Groo, R.N., Lori Verne, R.N., and the staff of the Psychiatry Department, Kaiser Permanente Santa Clara Medical Center. Gwen Hebert, Jennifer Groebe, L.C.S.W, and Shella Chazen, M.S., provided outstanding operational support. The Depression Bipolar Support Alliance provided invaluable assistance in developing and testing the eCare program. The authors thank their technology development partner, the Dr. Global organization, especially Grant Goodman and Gregory Lynne. They are also grateful to Joseph Selby, M. D., for scientific insights and encouragement and to Donald Mordecai, M.D., Mason Turner, M.D., and Beth Streeter, M.P.H., for thoughtful comments on the manuscript. They thank the Permanente Medical Group leadership, including Patricia Conolly, M.D., Donald Dyson, M.D., Philip Madvig, M.D., James Chang, M.D., and Joseph J. Houska, Ph.D. They also acknowledge Robert Pearl, M.D., for his vision of the promise of online care.
Dr. Bauer has received royalties from Springer Publishing and New Harbinger Publishing for manuals related to collaborative care of mood disorders. The Permanente Medical Group, Inc., has submitted two patent applications related to online patient management systems and methods. Neither of the two authors (Ms. Hunkeler and Dr. Terdiman) who are named on the applications as inventors nor any other author stand to gain financially from the existence of these patent applications or patents should they be granted.