0
Get Alert
Please Wait... Processing your request... Please Wait.
You must sign in to sign-up for alerts.

Please confirm that your email address is correct, so you can successfully receive this alert.

1
Articles   |    
A Web-Delivered Care Management and Patient Self-Management Program for Recurrent Depression: A Randomized Trial
Enid M. Hunkeler, M.A.; William A. Hargreaves, Ph.D.; Bruce Fireman, M.A.; Joseph Terdiman, M.D., Ph.D.; Joel F. Meresman, Ph.D.; Yvonnne Porterfield, R.N., M.S.N.; Janelle Lee, Dr.P.H.; Robin Dea, M.D.; Gregory E. Simon, M.D.; Mark S. Bauer, M.D.; Jürgen Unützer, M.D., M.P.H.; C. Barr Taylor, M.D.
Psychiatric Services 2012; doi: 10.1176/appi.ps.005332011
View Author and Article Information

Ms. Hunkeler, Mr. Fireman, Dr. Terdiman, Ms. Porterfield, Dr. Lee, and Dr. Dea are affiliated with the Division of Research, The Permanente Medical Group, Inc., 2000 Broadway, Second Floor, Oakland, CA 94612 (e-mail: enid.hunkeler@kp.org).
Dr. Hargreaves is with the Department of Psychiatry, University of California, San Francisco.
Dr. Meresman is with the Department of Psychiatry, The Permanente Medical Group, Inc., Santa Clara, California.
Dr. Simon is with the Center for Health Studies, Group Health Cooperative, Seattle.
Dr. Bauer is with the Department of Psychiatry, Harvard Medical School, Boston.
Dr. Unützer is with the Department of Psychiatry and Behavioral Sciences, University of Washington Medical Center, Seattle.
Dr. Taylor is with the Department of Psychiatry, Stanford University School of Medicine, Stanford, California.
Some preliminary findings from this trial were presented at the National Institute of Mental Health’s Mental Health Services Conference, “Increasing the Efficiency of Research and Mental Health Services,” Washington, D.C., July 20–21, 2009.

Objective  This study assessed the impact of an Internet-delivered care management and patient self-management program, eCare for Moods, on patients treated for recurrent or chronic depression.

Methods  Patients with recurrent or chronic depression were randomly assigned to eCare (N=51) or usual specialty mental health care (N=52). The 12-month eCare program integrates with ongoing depression care, links to patients’ electronic medical records, and provides clinicians with panel management and decision support. Participants were interviewed at baseline and six, 12, 18, and 24 months after enrollment. Telephone interviewers blind to treatment used a timeline follow-back method to estimate depression severity on a 6-point scale for each of the 105 study weeks (including the baseline). Differences between groups in weekly severity over two years were examined by generalized estimating equations.

Results  Participants in eCare experienced more reduction in depressive symptoms (estimate=–.74 on the 6-point scale over two years; 95% confidence interval [CI]=–1.38 to –.09, p=.025) and were less often depressed (–.24 over two years; CI=–.46 to –.03, p=.026). At 24 months, 43% of eCare and 30% of usual-care participants were depression free; the number needed to treat to attain one additional depression-free participant was 8. eCare participants had other favorable outcomes: improved general mental health (p=.002), greater satisfaction with specialty care (p=.003) and with learning new coping skills (p<.001), and more confidence in managing depression (p=.006).

Conclusions  Internet-delivered care management can help improve outcomes of patients treated for recurrent or chronic depression.

Abstract Teaser
Figures in this Article

Comprehensive, multifaceted disease management programs can improve outcomes among depressed patients (110). Yet cost, staffing requirements, lack of trained providers, and other barriers limit their use. Care management by telephone can improve efficiency and reach more people (11,12). However, it requires considerable resources and has built-in inefficiencies (such as unproductive attempts to reach patients). Internet-delivered depression care, if effective, could circumvent these barriers. Randomized trials of online cognitive-behavioral therapy (CBT), counseling, and psychoeducation have shown promising results (1328).

We developed and evaluated a patient-centered, interactive care management and patient self-management program, delivered through the Internet, to reduce the overall burden of recurrent or chronic depression and bipolar disorder. It is integrated with ongoing depression care, provides most services online, links to patients’ electronic medical records, and provides clinicians with clinical decision and panel management support. This article presents results for participants with recurrent or chronic depression.

We conducted an open, randomized clinical trial with blinded interviewers to answer the question: Can Internet-delivered care management have a sustained favorable impact on the burden of recurrent or chronic depression among patients receiving outpatient specialty mental health care? The main outcome was depression severity over 24 months. The intervention was available in year 1 but not in year 2.

+

Participants

Patients were recruited from two Kaiser Permanente specialty mental health clinics in Northern California. Study participants were receiving antidepressants, psychotherapy, or both for recurrent or chronic depression.

Institutional review boards of the Kaiser Foundation Research Institute and the University of California, San Francisco, approved the study. Participants gave written consent after study procedures and risks were described.

A list of eligible individuals was created by using electronic medical records. Eligible individuals were 18 years or older and not currently hospitalized; they had a diagnosis of recurrent or chronic depression (DSM-IV 296.3 or 296.35) (29) that was recorded between September 2001 and April 2003, with no recorded diagnosis of bipolar disorder.

Participants were recruited between April 2003 and October 2004. All candidates meeting eligibility criteria received a letter signed by their treating clinician introducing the study and an opt-out postcard. Telephone screeners confirmed eligibility. Using questions from the Structured Clinical Interview for DSM-IV (SCID) (30), screeners confirmed that the individual had at least two lifetime episodes of depression or chronic depression and no history of manic or hypomanic episodes, was fluent in English, used the Internet at home, and would get his or her psychiatric care during the coming year at the same clinic. Baseline interviews for eligible candidates were scheduled at their clinic.

A statistician blind to candidates’ identities prepared random treatment assignments in blocked sets of four (two randomized to the intervention and two to usual care). Treatment assignment was revealed at the end of the interview.

+

Usual care

Both treatment groups received usual specialty mental health care, which is similar across psychiatry departments of Kaiser Permanente of Northern California. Care begins with phone screening by a licensed clinician, assignment of a primary clinician, and an intake visit. The primary clinician designs an individual treatment plan combining medication and case management; group, family, and individual psychotherapy; and psychoeducation, as appropriate. Intensive outpatient programs, residential crisis care, and full hospitalization are available when needed. Treatment is comparable to standard community care, except that it places more emphasis on case management, provision of a continuum of care, and group rather than individual psychotherapy.

+

Intervention

eCare for Moods, a secure, password-protected Web site that was developed by clinicians, patients, and researchers (31,32), is based on established care management (2,6) and behavior therapy strategies (3337) for depression. eCare was available 24 hours a day, seven days a week, for 12 months. At least twice daily during clinic hours the psychiatric nurse care manager (eCare manager) assigned to each participant monitored the site. eCare offered participants personalized self-monitoring, secure messaging with their eCare manager, depression education grounded in established CBT interventions emphasizing behavioral activation and social support, a monitored discussion group, problem-specific advice, a personal database, task lists, and an appointment calendar. It gave eCare managers and other mental health providers clinical decision and panel management support and regularly downloaded information from participants’ electronic medical records for use by both the participants and the providers.

These online functions were designed to help participants self-manage their depression and receive clinical intervention as needed. To self-monitor their depression, participants created personal depression profiles online, including their DSM-IV symptoms, personal behavioral indicators of depression, and early warning signs. They also tracked their disability, medication adherence, side effects, and alcohol and drug use online. Graphs in eCare displayed all monitoring information.

Eight education modules emphasized the effects of thoughts, actions, sociability, and productivity on mood and provided general health education. Supplementing the modules were interactive worksheets and links to Web sites that were vetted for accuracy and objectivity.

Participants could enlist a “care partner” (family member or friend) for emotional support and company during pleasurable activities. With permission, care partners could contact eCare managers directly if they observed worsening symptoms or suicidality. Care partners had their own Web site, which included evidenced-based guidelines for helping depressed individuals improve their mood.

eCare managers followed a panel of participants by reviewing their Web site activity, using secure messaging, and responding to alerts automatically generated by participants’ answers to monitoring questions—an important aspect of the eCare managers’ role. Alerts were displayed on eCare managers’ screens in order of urgency (emergency alerts first, then urgent alerts, and then notifications). Depending on the alert, the eCare manager notified a crisis team, telephoned the participant, or sent a secure message.

To ensure full integration of eCare with usual care, eCare managers knew each participant’s treatment plan, consulted with their treating clinicians, and followed clinic policies and guidelines. Treating clinicians could access eCare data through the Web site.

A clinical nurse specialist trained participants and care partners to use eCare in a one-hour, in-clinic session. Some participants were trained by telephone or instant messaging. The nurse specialist and a psychologist supervised the eCare managers. [An online data supplement to this article describes intervention details.]

+

Assessments

In-person baseline interviews were conducted in the participant’s clinic. Telephone interviewers blind to the participant’s treatment group conducted follow-up interviews six, 12, 18, and 24 months after enrollment. Participants received $25 per interview.

All interviews included an assessment of depressive symptoms experienced over the previous two weeks with questions adapted from the SCID (30). This provided a psychiatric status rating (PSR) of the severity of current depression on a 6-point scale derived from the research diagnostic criteria (38): 1, no symptoms; 2, mild depression; 3, moderate impairment; 4, marked impairment but below DSM-IV criteria for an episode; 5, meets criteria for an episode; and 6, meets criteria for a severe episode. A PSR of ≥3 indicated that depression was definitely present.

At each follow-up, interviewers obtained a current PSR. Using the previous PSR and current PSR as anchors, interviewers used the semistructured Longitudinal Interval Follow-up Evaluation (LIFE) (39) to create a week-by-week timeline of PSRs for the six months before each follow-up. The LIFE employs association techniques to prompt participants to remember their mood at various points in the past six months. It provides reasonably accurate recall for depression with no evidence that accuracy deteriorates with longer recall intervals (40).

All interviews included the Sheehan Disability Scale (41); the mental health, general health, vitality, role-emotional (ability to accomplish tasks unimpaired by mood), and social functioning scales from the 36-item Short-Form Health Survey (SF-36) (42); questions from the Alcohol Use Disorders Identification Test (43,44); and questions about use of psychotropic medications during the previous three months, available social support over the previous two weeks, and employment and use of sick leave over the previous six months. Available social support was measured by a six-item scale consisting of questions about participants’ connections to family and friends.

To measure satisfaction with care, we developed new questions and conducted confirmatory factor analyses (45) to group the questions them into five scales: general satisfaction with psychiatric care, satisfaction with the personal and empathic nature of care, satisfaction with learning new coping skills, confidence in coping with the mood disorder, and complaints about access to care. The factor analyses yielded a Bentler index of .95 (46), indicating excellent fit to this subscale structure.

Kaiser Permanente databases provided information on prescriptions filled and health service utilization. Cost estimates for eCare included direct staff costs for monitoring, training, and supervision, plus organizational overhead. Nurse logs of time spent on eCare activities helped determine staffing costs. One-time development and technical support costs were excluded. Other costs were obtained from Kaiser Permanente’s Cost Management Information System, which allocates overhead and other indirect costs into outpatient visits, hospitalizations, laboratory, and other clinical services.

+

Statistical methods

The primary hypothesis was that eCare would be associated with more favorable change over time than usual care in weekly measures of depression severity. For the primary test of this hypothesis, we focused on the time-by-treatment interaction effect, which was included in a repeated-measures model of depression severity. We fit the repeated-measures model to the series of 105 weekly depression severity measurements, starting with the baseline measure and including each subsequent weekly measure throughout the 24-month study period.

Secondary outcomes (current depression severity, depression presence, Sheehan Disability Scale, and SF-36 scales) were measured at baseline and at six, 12, 18, and 24 months. Again, we used repeated-measures analyses and focused on time-by-treatment interaction effects. In each repeated-measures analysis, the time-by-treatment interaction effect was interpreted as the effect of eCare on change in the outcome over time. All repeated-measures analyses of outcomes were implemented with generalized estimating equations (SAS, version 9.2, procedure GENMOD), which accommodate missing values.

Analyses of satisfaction scales focused on satisfaction levels reported in the four follow-up interviews. Satisfaction was not measured at baseline. The main effect of treatment on satisfaction was adjusted for baseline variables that correlated strongly with satisfaction.

Cohen’s d, a standardized measure of effect size, was calculated to compare our findings with those of other studies (47). It measured the standardized mean difference in depression severity attributable to the eCare intervention. Values of .2 suggest a small effect, .5 a medium effect, and .8 a large effect (47).

The number needed to treat (NNT) to obtain one additional participant free of definite depression (PSR <3) was computed at 24 months.

We compared health care utilization and costs during two years before and after enrollment, using t tests for the pre-post difference in differences—that is, the pre–post differences for eCare participants minus the pre-post differences for usual-care participants.

Analyses of baseline characteristics, days’ supply of medication, health service use, and cost included all participants randomly assigned to a treatment group. Analyses of interview data omitted three participants with no postbaseline data. All significance levels are two-sided.

+

Sample characteristics

Screeners assessed 304 patients for eligibility: 178 were ineligible, 23 refused to participate, and 103 consented. Consenting patients were assessed at baseline and randomly assigned to a treatment group—51 to eCare and 52 to usual care. Follow-up interviewers blind to treatment group assessed outcomes for 97% of participants (N=100) six months after baseline, 95% (N=98) at 12 months, 73% (N=75) at 18 months, and 83% (N=86) at 24 months [see the online data supplement for a flowchart]. Attrition was greatest in year 2, and somewhat higher among intervention participants, although not significantly. Participants who were not interviewed at 18 or 24 months did not differ significantly on key baseline variables (depression severity, sociodemographic characteristics, and antidepressant use) from those who were interviewed at these follow-up points.

Table 1 presents data on participants’ characteristics. Mean depression severity did not differ significantly by treatment group, but 22 of the eCare participants met criteria for being in an episode of depression (PSR ≥5), compared with 15 participants in usual care. Treatment groups did not differ significantly in baseline characteristics except for marital status.

 
Anchor for Jump
Table 1Baseline characteristics of participants in eCare and usual care
Table Footer Note

a N varies between 52 and 51 because of missing data.

Table Footer Note

b Pearson χ2 test, df=1, for dichotomous variables; t test for age and the remaining continuous variables

Table Footer Note

c Range 26 to 76 years

Table Footer Note

d Possible scores range from 1, free of depression, to 6, severe depressive episode.

Table Footer Note

e Scores on the 36-item Short-Form Health Survey are standardized to a mean±SD of 50±10; higher scores indicate better health, more vitality, better social functioning, and a greater ability to accomplish tasks unimpaired by mood (role-emotional).

Table Footer Note

f Possible scores range from 1, no disability, to 10, severe disability.

Table Footer Note

g Possible scores on the Alcohol Use Disorders Identification Test range from 0, abstinence, to 40, severe alcoholism.

Table Footer Note

h Range of lost work days, 0 to 130. A total of 28 eCare and 30 usual-care participants had been employed in the past 6 months.

Table Footer Note

i Possible scores on a six-item scale range from 0 to 24, with higher scores indicating more support.

+

Web site use

Of the 51 eCare participants, 46 (90%) received the one-hour training. During the year that eCare was available, the 51 participants made, on average, 60 visits to the Web site (median 34; interquartile range 9–84).

Participants’ responses to monitoring questionnaires generated 17 emergency alerts, 93 urgent alerts, and 1,134 notifications. One participant’s emergency alert resulted in an emergency consultation; this participant made a subsequent suicide attempt.

On average, eCare participants completed 4.6 of the eight education modules (median 5); 22 participants (43%) completed all eight. Participants received an average of 10.7 messages (median 8) from their eCare manager and sent an average of 8.4 messages (median 4). Participants averaged 10.5 visits to the discussion group, which was available only to those who completed all eight education modules. Nine care partners made, on average, six visits to their separate Web site (median 5, maximum 20).

Figure 1 shows the percentages of participants who visited eCare in the first and second six months of the program and the program components that they used. Usage declined over time, except for participation in the discussion group, which increased in the second six months.

 
Anchor for JumpAnchor for Jump
Figure 1Percentage of 51 eCare participants using the eCare Web site and each program component, by six-month interval
+

Treatment effects

Postbaseline interviews were conducted with 100 participants (49 eCare and 51 usual care). Figure 2 shows the weekly course of depression severity over two years. In the first year, depression severity decreased steadily in the eCare group, and by one year, the mean PSR was about 1 point below baseline. This reduction persisted throughout the second year, when eCare was no longer available. Usual-care participants improved about half a point in the first five months and little thereafter. Overall, on the 6-point PSR scale, depression severity among eCare participants decreased nearly three-quarters of a point more, on average, than among usual-care participants (time-by-treatment estimate=–.74 over two years; 95% confidence interval [CI]=–1.38 to –.09, p=.025).

 
Anchor for JumpAnchor for Jump
Figure 2Mean depression severity over two years among 49 eCare and 51 usual-care participantsa

aPsychiatric status rating: 1, no symptoms; 2, mild depression; 3, moderate impairment; 4, marked impairment but below DSM-IV criteria for an episode; 5, meets criteria for an episode; and 6, meets criteria for a severe episode. For the difference between groups in depression severity trend, a repeated-measures model with a time-by-treatment interaction effect was fit. The trend over two years was more favorable for eCare participants by an average of –.74 (CI= –1.38 to –.09, p=.025).

Adjustment for the level of depression severity at baseline yielded a similar estimate of eCare’s effect on the trend in depression severity: the time-by-treatment estimate was –.76 (CI=–1.39 to –.13, p=.018). Adjustment for marital status (the only baseline variable that differed significantly across the two treatment groups) yielded a similar estimate of eCare’s effect on the course of depression severity.

Assessments of the trend in depression severity, as measured every six months rather than weekly, indicated greater improvement among eCare participants (estimate=–.52 over two years; CI=–1.02 to –.01, p=.047) (Table 2). The proportion of participants who continued to have definite depression (PSR ≥3) was lower among eCare participants than among usual-care participants (estimate=–.24 over two years; CI=–.46 to –.03, p=.026).

 
Anchor for Jump
Table 2Depression severity and health-related secondary outcomes at baseline and follow-up interviews, by treatment group
Table Footer Note

a Reflects a time-by-treatment interaction in a repeated-measures model

Table Footer Note

b Possible scores range from 1, free of depression, to 6, severe depressive episode.

Table Footer Note

c Possible scores range from 0, free of depression, to 1, depression is present.

Table Footer Note

d Possible scores range from 0, no disability, to 30, severe disability.

Table Footer Note

e Scores on the 36-item Short-Form Health Survey are standardized to a mean±SD of 50±10; higher scores indicate better health, more vitality, better social functioning, and a greater ability to accomplish tasks unimpaired by mood (role-emotional).

Table 2 also presents data for six other outcomes. Improvement in general mental health (SF-36) was significantly greater among eCare participants (p=.002). Trends toward favorable treatment effects were seen for eCare in level of disability (Sheehan Disability Scale) (p=.073), vitality (SF-36) (p=.079), and role-emotional (SF-36) (p=.081).

Table 3 presents pooled main effects for five satisfaction scales. eCare participants reported more satisfaction with their psychiatric care (p=.003), more satisfaction with learning new coping skills (p<.001), and more confidence in coping with their mood disorder (p=.006).

 
Anchor for Jump
Table 3Satisfaction and self-efficacy outcomes at four follow-up interviews, by treatment groupa
Table Footer Note

a Sample sizes varied because of missing data.

Table Footer Note

b Reflects the eCare effect, on average, across the four follow-up interviews

Table Footer Note

c Possible scores range from 1, most unsatisfied, to 5, most satisfied.

Table Footer Note

d Possible scores range from 1, strongly disagree, to 5, strongly agree, that they learned new coping skills or gained confidence.

Table Footer Note

e Possible scores range from 1, no complaints, to 5, strong complaints.

The Cohen’s d for the standardized effect of eCare (time-by-treatment effect over two years) on depression severity was .60. The NNT was 8.3 at 24 months, when 43% of eCare participants were depression free, compared with 30% in usual care.

Among eCare participants, completion of more education modules was positively related to less severe depression at 24 months (p=.036). Other measures of Web site use were not associated with depression severity at 24 months.

+

Medications, services, and cost

At baseline, nearly all participants (99%) reported taking antidepressants for at least one month during the previous three months. Among interview respondents at 12 months, self-reported antidepressant use decreased to 88% (N=45) in the usual-care group and 87% (N=41) among eCare participants. Among those interviewed at 18 months, antidepressant use was 85% (N=34) among usual-care participants and 89% (N=31) among eCare participants. At 24 months, the proportion using antidepressants was lower in the eCare group—80% (N=32) versus 96% (N=44) (p=.04). Prescription fill data indicated significantly less antidepressant use among eCare participants than usual-care participants at 24 months (p=.046).

Data on use and cost of specialty mental health and other health services for all 103 participants were analyzed for two years before and two years after enrollment. [The online data supplement to this article includes two tables presenting these data.] Both groups had fewer specialty mental health visits after enrollment than before. The estimated cost to deliver eCare for one year was $345 (2009 USD) per participant. When use during the two years before and the two years after enrollment was compared, no significant differences were found between treatment groups in changes in mean use of specialty mental health or other health services or associated costs (including the cost of eCare). Nurse logs indicated that a nurse could manage 200 patients in ten hours a week.

eCare for Moods is among the first comprehensive online depression care management programs supporting both patients and clinicians to be tested in specialty care. Compared with patients randomly assigned to usual specialty mental health care, patients randomly assigned to eCare improved significantly more on the primary outcome—weekly depression severity over two years. Furthermore, eCare participants were depressed less often, experienced better overall mental health, were more satisfied with their mental health care and coping skills, and felt more confident handling their depression. The eCare program’s favorable effects were sustained over two years, suggesting that eCare has lasting benefit. Because the proportion of eCare participants taking antidepressants did not increase over time, its effectiveness cannot be attributed to increased use of medication.

The treatment effects of eCare are comparable to those obtained in randomized clinical trials of care management (48) and Internet-based programs (15) that target depression. A meta-analysis of 37 collaborative care trials found the overall standardized effect size of the nine studies that reported 24-month follow-up data was .15 (CI=–.03 to .34) (48). The standardized effect size in our study was .60. It should be noted that programs in the meta-analysis were delivered in primary care, where the control groups probably received less intensive treatment than the usual-care participants in our study.

A meta-analysis of 12 randomized trials of Internet-based depression programs found that those that involved support from a therapist and those with wait-list comparison groups were the most successful (average standardized effect sizes of .61 and .56, respectively) (15). Interventions without the support of a therapist and those with usual-care comparison groups had small effect sizes (.25 and .23, respectively). Other studies show a similar pattern (2427,49,50).

This study had important strengths. It was a long-term, randomized trial with interviewers blind to treatment condition that was conducted in real-world specialty mental health settings. Extensive information on important outcomes was gathered over two years, including one year after the intervention ended. It demonstrated that an Internet-delivered program that gives depressed patients self-management tools and provides almost continuous access to the care delivery system and care management can produce better outcomes than usual care among patients being treated by mental health specialists. These findings offer hope to patients for whom depression is a lifelong burden. Given the cost-efficiencies of delivering care online, our findings could lead to greater access to care for people with mood disorders.

This study had limitations. First, the small sample limits the precision of our findings. Second, some eCare participants may have reported fewer symptoms than usual-care participants because they wanted to emphasize improvement, or they may have reported more symptoms because of heightened self-awareness. Third, the ability to generalize the results is limited by the inclusion of only English-speaking, Internet-using patients from two Kaiser Permanente clinics. Fourth, the p values reported are from multiple tests of secondary outcomes and should be interpreted cautiously.

Replicating this study with larger and more varied samples in other practice settings, especially primary care, would clarify the clinical effects and costs of the intervention and the generalizability of the findings. Similar programs could be beneficial for conditions such as schizophrenia, chemical dependency, diabetes, and asthma. Self-care, continuous care management, and clinical decision support—all integrated and delivered online—can improve the lives of patients with recurrent or chronic depression.

The eCare for Moods study was funded by Eli Lilly and Company and the Kaiser Permanente Garfield Memorial Fund. The Permanente Medical Group, Inc., of Northern California provided in-kind support. The Kaiser Foundation Health Plan provided usual care and medications to study participants according to health plan benefits. The study sponsors and funders had no role in the design of the study; in collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit it for publication. The authors thank the study participants. They gratefully acknowledge Paul Wilson, M.D., Ph.D., and Adele Adams, R.N., and the staff of the Psychiatry Department, Kaiser Permanente Redwood City Medical Center; and Mark McCormick, M.D., Anne Klevay, R.N., Sheri Ertzner-Groo, R.N., Lori Verne, R.N., and the staff of the Psychiatry Department, Kaiser Permanente Santa Clara Medical Center. Gwen Hebert, Jennifer Groebe, L.C.S.W, and Shella Chazen, M.S., provided outstanding operational support. The Depression Bipolar Support Alliance provided invaluable assistance in developing and testing the eCare program. The authors thank their technology development partner, the Dr. Global organization, especially Grant Goodman and Gregory Lynne. They are also grateful to Joseph Selby, M. D., for scientific insights and encouragement and to Donald Mordecai, M.D., Mason Turner, M.D., and Beth Streeter, M.P.H., for thoughtful comments on the manuscript. They thank the Permanente Medical Group leadership, including Patricia Conolly, M.D., Donald Dyson, M.D., Philip Madvig, M.D., James Chang, M.D., and Joseph J. Houska, Ph.D. They also acknowledge Robert Pearl, M.D., for his vision of the promise of online care.

Dr. Bauer has received royalties from Springer Publishing and New Harbinger Publishing for manuals related to collaborative care of mood disorders. The Permanente Medical Group, Inc., has submitted two patent applications related to online patient management systems and methods. Neither of the two authors (Ms. Hunkeler and Dr. Terdiman) who are named on the applications as inventors nor any other author stand to gain financially from the existence of these patent applications or patents should they be granted.

Badamgarav  E;  Weingarten  SR;  Henning  JM  et al:  Effectiveness of disease management programs in depression: a systematic review.  American Journal of Psychiatry 160:2080–2090, 2003
[CrossRef] | [PubMed]
 
Williams  JW  Jr;  Gerrity  M;  Holsinger  T  et al:  Systematic review of multifaceted interventions to improve depression care.  General Hospital Psychiatry 29:91–116, 2007
[CrossRef] | [PubMed]
 
Katon  W;  Robinson  P;  Von Korff  M  et al:  A multifaceted intervention to improve treatment of depression in primary care.  Archives of General Psychiatry 53:924–932, 1996
[CrossRef] | [PubMed]
 
Schulberg  HC;  Katon  W;  Simon  GE  et al:  Treating major depression in primary care practice: an update of the Agency for Health Care Policy and Research Practice Guidelines.  Archives of General Psychiatry 55:1121–1127, 1998
[CrossRef] | [PubMed]
 
Unützer  J;  Katon  W;  Callahan  CM  et al:  Improving Mood-Promoting Access to Collaborative Treatment: collaborative care management of late-life depression in the primary care setting: a randomized controlled trial.  JAMA 288:2836–2845, 2002
[CrossRef] | [PubMed]
 
Gilbody  S;  Whitty  P;  Grimshaw  J  et al:  Educational and organizational interventions to improve the management of depression in primary care: a systematic review.  JAMA 289:3145–3151, 2003
[CrossRef] | [PubMed]
 
Neumeyer-Gromen  A;  Lampert  T;  Stark  K  et al:  Disease management programs for depression: a systematic review and meta-analysis of randomized controlled trials.  Medical Care 42:1211–1221, 2004
[CrossRef] | [PubMed]
 
Hunkeler  EM;  Katon  W;  Tang  L  et al:  Long term outcomes from the IMPACT randomised trial for depressed elderly patients in primary care.  BMJ (Clinical Research Ed) 332:259–263, 2006
[CrossRef] | [PubMed]
 
Wells  K;  Sherbourne  C;  Schoenbaum  M  et al:  Five-year impact of quality improvement for depression: results of a group-level randomized controlled trial.  Archives of General Psychiatry 61:378–386, 2004
[CrossRef] | [PubMed]
 
Trivedi  MH;  Rush  AJ;  Crismon  ML  et al:  Clinical results for patients with major depressive disorder in the Texas Medication Algorithm Project.  Archives of General Psychiatry 61:669–680, 2004
[CrossRef] | [PubMed]
 
Fortney  JC;  Pyne  JM;  Edlund  MJ  et al:  A randomized trial of telemedicine-based collaborative care for depression.  Journal of General Internal Medicine 22:1086–1093, 2007
[CrossRef] | [PubMed]
 
Liu  CF;  Fortney  J;  Vivell  S  et al:  Time allocation and caseload capacity in telephone depression care management.  American Journal of Managed Care 13:652–660, 2007
[PubMed]
 
Ybarra  ML;  Eaton  WW:  Internet-based mental health interventions.  Mental Health Services Research 7:75–87, 2005
[CrossRef] | [PubMed]
 
Andersson  G;  Guijpers  P;  Carlbring  P  et al:  Effects of Internet-delivered cognitive behaviour therapy for anxiety and mood disorders.  Helix Review Series: Psychiatry 9:9–14, 2007
 
Andersson  G;  Cuijpers  P:  Internet-based and other computerized psychological treatments for adult depression: a meta-analysis.  Cognitive Behaviour Therapy 38:196–205, 2009
[CrossRef] | [PubMed]
 
Christensen  H;  Griffiths  KM;  Jorm  AF:  Delivering interventions for depression by using the internet: randomized controlled trial.  BMJ (Clinical Research Ed) 328:265, 2004
[CrossRef] | [PubMed]
 
Proudfoot  J;  Ryden  C;  Everitt  B  et al:  Clinical efficacy of computerised cognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial.  British Journal of Psychiatry 185:46–54, 2004
[CrossRef] | [PubMed]
 
Andersson  G;  Bergström  J;  Holländare  F  et al:  Internet-based self-help for depression: randomised controlled trial.  British Journal of Psychiatry 187:456–461, 2005
[CrossRef] | [PubMed]
 
Clarke  G;  Eubanks  D;  Reid  E  et al:  Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders.  Journal of Medical Internet Research 7:e16, 2005
[CrossRef] | [PubMed]
 
Spek  V;  Cuijpers  P;  Nyklícek  I  et al:  Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis.  Psychological Medicine 37:319–328, 2007
[CrossRef] | [PubMed]
 
Mackinnon  A;  Griffiths  KM;  Christensen  H:  Comparative randomised trial of online cognitive-behavioural therapy and an information website for depression: 12-month outcomes.  British Journal of Psychiatry 192:130–134, 2008
[CrossRef] | [PubMed]
 
van Straten  A;  Cuijpers  P;  Smits  N:  Effectiveness of a web-based self-help intervention for symptoms of depression, anxiety, and stress: randomized controlled trial.  Journal of Medical Internet Research 10:e7, 2008
[CrossRef] | [PubMed]
 
Warmerdam  L;  van Straten  A;  Twisk  J  et al:  Internet-based treatment for adults with depressive symptoms: randomized controlled trial.  Journal of Medical Internet Research 10:e44, 2008
[CrossRef] | [PubMed]
 
Meyer  B;  Berger  T;  Caspar  F  et al:  Effectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trial.  Journal of Medical Internet Research 11:e15, 2009
[CrossRef] | [PubMed]
 
Kessler  D;  Lewis  G;  Kaur  S  et al:  Therapist-delivered Internet psychotherapy for depression in primary care: a randomised controlled trial.  Lancet 374:628–634, 2009
[CrossRef] | [PubMed]
 
Perini  S;  Titov  N;  Andrews  G:  Clinician-assisted Internet-based treatment is effective for depression: randomized controlled trial.  Australian and New Zealand Journal of Psychiatry 43:571–578, 2009
[CrossRef] | [PubMed]
 
Vernmark  K;  Lenndin  J;  Bjärehed  J  et al:  Internet administered guided self-help versus individualized e-mail therapy: a randomized trial of two versions of CBT for major depression.  Behaviour Research and Therapy 48:368–376, 2010
[CrossRef] | [PubMed]
 
Learmonth  D;  Trosh  J;  Sadik  R:  The role of computer-aided psychotherapy within an NHS CBT specialist service.  Counselling and Psychotherapy Research 8:117–123, 2008
[CrossRef]
 
 Diagnostic and Statistical Manual of Mental Disorders , 4th ed.  Washington, DC,  American Psychiatric Association, 1994
 
First  MB;  Spitzer  RL;  Gibbon  M  et al:  Structured Clinical Interview for DSM-IV Axis I Disorders, Clinical Version .  Washington, DC,  American Psychiatric Press, 1997
 
Yellowlees  P:  Successful development of telemedicine systems—seven core principles.  Journal of Telemedicine and Telecare 3:215–222, 1997
[CrossRef] | [PubMed]
 
Johnston B: Developing a programme in e-mental health care; in Telepsychiatry and e-Mental Health. Edited by Wootton R, Yellowlees P, McLaren P. London, United Kingdom, Royal Society of Medicine Press, 2003
 
Lewinsohn PM, Muñoz RF, Youngren MA, et al: Control Your Depression (Revised and Updated). New York, Simon & Schuster, 1992
 
Bauer  MS;  McBride  L:  Structured Group Psychotherapy for Bipolar Disorder: The Life Goals Program , 2nd ed.  New York,  Springer, 2003
 
Clarke  G;  Reid  E;  Eubanks  D  et al:  Overcoming depression on the Internet (ODIN): a randomized controlled trial of an Internet depression skills intervention program.  Journal of Medical Internet Research 4:e14, 2002
[CrossRef] | [PubMed]
 
Muñoz RF, Miranda J: Group Therapy Manual for Cognitive Behavioral Treatment of Depression (MR-1198/4) Santa Monica, Calif, RAND, 2000
 
Unützer  J;  Katon  W;  Williams  JWJ  Jr  et al:  Improving primary care for depression in late life: the design of a multicenter randomized trial.  Medical Care 39:785–799, 2001
[CrossRef] | [PubMed]
 
Spitzer  RL;  Endicott  J;  Robins  E:  Research diagnostic criteria: rationale and reliability.  Archives of General Psychiatry 35:773–782, 1978
[CrossRef] | [PubMed]
 
Keller  MB;  Lavori  PW;  Friedman  B  et al:  The Longitudinal Interval Follow-up Evaluation: a comprehensive method for assessing outcome in prospective longitudinal studies.  Archives of General Psychiatry 44:540–548, 1987
[CrossRef] | [PubMed]
 
Rutter  CM;  Simon  GE:  A Bayesian method for estimating the accuracy of recalled depression.  Journal of the Royal Statistical Society 53:341–353, 2004
[CrossRef]
 
Sheehan  DV;  Harnett-Sheehan  K;  Raj  BA:  The measurement of disability.  International Clinical Psychopharmacology 11(suppl 3):89–95, 1996
[CrossRef] | [PubMed]
 
Ware  JE;  Kosinski  M;  Dewey  JE:  How to Score Version 2 of the SF-36 Health Survey .  Lincoln, RI,  QualityMetric Inc, 2000
 
Babor TF, Higgins-Biddle JC, Saunders JB: The Alcohol Use Disorders Identification Test: Guidelines for Use in Primary Care, 2nd ed. Geneva, World Health Organization, Department of Mental Health and Substance Dependence, 2001
 
Conigrave  KM;  Hall  WD;  Saunders  JB:  The AUDIT questionnaire: choosing a cut-off score. Alcohol Use Disorders Identification Test.  Addictions 90:1349–1356, 1995
[CrossRef]
 
Hatcher  L:  Step-by-Step Approach to Using the SAS System for Factor Analysis and Structural Equation Modeling .  Cary, NC,  SAS Institute Inc, 1994
 
Bentler  PM;  Bonett  DG:  Significance tests and goodness of fit in the analysis of covariance structures.  Psychological Bulletin 88:586–606, 1980
[CrossRef]
 
Cohen  J:  Statistical Power Analysis for the Behavioral Sciences .  New York,  Psychology Press, 1988
 
Gilbody  S;  Bower  P;  Fletcher  J  et al:  Collaborative care for depression: a cumulative meta-analysis and review of longer-term outcomes.  Archives of Internal Medicine 166:2314–2321, 2006
[CrossRef] | [PubMed]
 
de Graaf  LE;  Gerhards  SAH;  Arntz  A  et al:  Clinical effectiveness of online computerised cognitive-behavioural therapy without support for depression in primary care: randomised trial.  British Journal of Psychiatry 195:73–80, 2009
[CrossRef] | [PubMed]
 
Meglic  M;  Furlan  M;  Kuzmanic  M  et al:  Feasibility of an eHealth service to support collaborative depression care: results of a pilot study.  Journal of Medical Internet Research 12:e63, 2010
[CrossRef] | [PubMed]
 
References Container

Figure 1 Percentage of 51 eCare participants using the eCare Web site and each program component, by six-month interval

Figure 2 Mean depression severity over two years among 49 eCare and 51 usual-care participantsa

aPsychiatric status rating: 1, no symptoms; 2, mild depression; 3, moderate impairment; 4, marked impairment but below DSM-IV criteria for an episode; 5, meets criteria for an episode; and 6, meets criteria for a severe episode. For the difference between groups in depression severity trend, a repeated-measures model with a time-by-treatment interaction effect was fit. The trend over two years was more favorable for eCare participants by an average of –.74 (CI= –1.38 to –.09, p=.025).

Anchor for Jump
Table 1Baseline characteristics of participants in eCare and usual care
Table Footer Note

a N varies between 52 and 51 because of missing data.

Table Footer Note

b Pearson χ2 test, df=1, for dichotomous variables; t test for age and the remaining continuous variables

Table Footer Note

c Range 26 to 76 years

Table Footer Note

d Possible scores range from 1, free of depression, to 6, severe depressive episode.

Table Footer Note

e Scores on the 36-item Short-Form Health Survey are standardized to a mean±SD of 50±10; higher scores indicate better health, more vitality, better social functioning, and a greater ability to accomplish tasks unimpaired by mood (role-emotional).

Table Footer Note

f Possible scores range from 1, no disability, to 10, severe disability.

Table Footer Note

g Possible scores on the Alcohol Use Disorders Identification Test range from 0, abstinence, to 40, severe alcoholism.

Table Footer Note

h Range of lost work days, 0 to 130. A total of 28 eCare and 30 usual-care participants had been employed in the past 6 months.

Table Footer Note

i Possible scores on a six-item scale range from 0 to 24, with higher scores indicating more support.

Anchor for Jump
Table 2Depression severity and health-related secondary outcomes at baseline and follow-up interviews, by treatment group
Table Footer Note

a Reflects a time-by-treatment interaction in a repeated-measures model

Table Footer Note

b Possible scores range from 1, free of depression, to 6, severe depressive episode.

Table Footer Note

c Possible scores range from 0, free of depression, to 1, depression is present.

Table Footer Note

d Possible scores range from 0, no disability, to 30, severe disability.

Table Footer Note

e Scores on the 36-item Short-Form Health Survey are standardized to a mean±SD of 50±10; higher scores indicate better health, more vitality, better social functioning, and a greater ability to accomplish tasks unimpaired by mood (role-emotional).

Anchor for Jump
Table 3Satisfaction and self-efficacy outcomes at four follow-up interviews, by treatment groupa
Table Footer Note

a Sample sizes varied because of missing data.

Table Footer Note

b Reflects the eCare effect, on average, across the four follow-up interviews

Table Footer Note

c Possible scores range from 1, most unsatisfied, to 5, most satisfied.

Table Footer Note

d Possible scores range from 1, strongly disagree, to 5, strongly agree, that they learned new coping skills or gained confidence.

Table Footer Note

e Possible scores range from 1, no complaints, to 5, strong complaints.

+

References

Badamgarav  E;  Weingarten  SR;  Henning  JM  et al:  Effectiveness of disease management programs in depression: a systematic review.  American Journal of Psychiatry 160:2080–2090, 2003
[CrossRef] | [PubMed]
 
Williams  JW  Jr;  Gerrity  M;  Holsinger  T  et al:  Systematic review of multifaceted interventions to improve depression care.  General Hospital Psychiatry 29:91–116, 2007
[CrossRef] | [PubMed]
 
Katon  W;  Robinson  P;  Von Korff  M  et al:  A multifaceted intervention to improve treatment of depression in primary care.  Archives of General Psychiatry 53:924–932, 1996
[CrossRef] | [PubMed]
 
Schulberg  HC;  Katon  W;  Simon  GE  et al:  Treating major depression in primary care practice: an update of the Agency for Health Care Policy and Research Practice Guidelines.  Archives of General Psychiatry 55:1121–1127, 1998
[CrossRef] | [PubMed]
 
Unützer  J;  Katon  W;  Callahan  CM  et al:  Improving Mood-Promoting Access to Collaborative Treatment: collaborative care management of late-life depression in the primary care setting: a randomized controlled trial.  JAMA 288:2836–2845, 2002
[CrossRef] | [PubMed]
 
Gilbody  S;  Whitty  P;  Grimshaw  J  et al:  Educational and organizational interventions to improve the management of depression in primary care: a systematic review.  JAMA 289:3145–3151, 2003
[CrossRef] | [PubMed]
 
Neumeyer-Gromen  A;  Lampert  T;  Stark  K  et al:  Disease management programs for depression: a systematic review and meta-analysis of randomized controlled trials.  Medical Care 42:1211–1221, 2004
[CrossRef] | [PubMed]
 
Hunkeler  EM;  Katon  W;  Tang  L  et al:  Long term outcomes from the IMPACT randomised trial for depressed elderly patients in primary care.  BMJ (Clinical Research Ed) 332:259–263, 2006
[CrossRef] | [PubMed]
 
Wells  K;  Sherbourne  C;  Schoenbaum  M  et al:  Five-year impact of quality improvement for depression: results of a group-level randomized controlled trial.  Archives of General Psychiatry 61:378–386, 2004
[CrossRef] | [PubMed]
 
Trivedi  MH;  Rush  AJ;  Crismon  ML  et al:  Clinical results for patients with major depressive disorder in the Texas Medication Algorithm Project.  Archives of General Psychiatry 61:669–680, 2004
[CrossRef] | [PubMed]
 
Fortney  JC;  Pyne  JM;  Edlund  MJ  et al:  A randomized trial of telemedicine-based collaborative care for depression.  Journal of General Internal Medicine 22:1086–1093, 2007
[CrossRef] | [PubMed]
 
Liu  CF;  Fortney  J;  Vivell  S  et al:  Time allocation and caseload capacity in telephone depression care management.  American Journal of Managed Care 13:652–660, 2007
[PubMed]
 
Ybarra  ML;  Eaton  WW:  Internet-based mental health interventions.  Mental Health Services Research 7:75–87, 2005
[CrossRef] | [PubMed]
 
Andersson  G;  Guijpers  P;  Carlbring  P  et al:  Effects of Internet-delivered cognitive behaviour therapy for anxiety and mood disorders.  Helix Review Series: Psychiatry 9:9–14, 2007
 
Andersson  G;  Cuijpers  P:  Internet-based and other computerized psychological treatments for adult depression: a meta-analysis.  Cognitive Behaviour Therapy 38:196–205, 2009
[CrossRef] | [PubMed]
 
Christensen  H;  Griffiths  KM;  Jorm  AF:  Delivering interventions for depression by using the internet: randomized controlled trial.  BMJ (Clinical Research Ed) 328:265, 2004
[CrossRef] | [PubMed]
 
Proudfoot  J;  Ryden  C;  Everitt  B  et al:  Clinical efficacy of computerised cognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial.  British Journal of Psychiatry 185:46–54, 2004
[CrossRef] | [PubMed]
 
Andersson  G;  Bergström  J;  Holländare  F  et al:  Internet-based self-help for depression: randomised controlled trial.  British Journal of Psychiatry 187:456–461, 2005
[CrossRef] | [PubMed]
 
Clarke  G;  Eubanks  D;  Reid  E  et al:  Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders.  Journal of Medical Internet Research 7:e16, 2005
[CrossRef] | [PubMed]
 
Spek  V;  Cuijpers  P;  Nyklícek  I  et al:  Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis.  Psychological Medicine 37:319–328, 2007
[CrossRef] | [PubMed]
 
Mackinnon  A;  Griffiths  KM;  Christensen  H:  Comparative randomised trial of online cognitive-behavioural therapy and an information website for depression: 12-month outcomes.  British Journal of Psychiatry 192:130–134, 2008
[CrossRef] | [PubMed]
 
van Straten  A;  Cuijpers  P;  Smits  N:  Effectiveness of a web-based self-help intervention for symptoms of depression, anxiety, and stress: randomized controlled trial.  Journal of Medical Internet Research 10:e7, 2008
[CrossRef] | [PubMed]
 
Warmerdam  L;  van Straten  A;  Twisk  J  et al:  Internet-based treatment for adults with depressive symptoms: randomized controlled trial.  Journal of Medical Internet Research 10:e44, 2008
[CrossRef] | [PubMed]
 
Meyer  B;  Berger  T;  Caspar  F  et al:  Effectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trial.  Journal of Medical Internet Research 11:e15, 2009
[CrossRef] | [PubMed]
 
Kessler  D;  Lewis  G;  Kaur  S  et al:  Therapist-delivered Internet psychotherapy for depression in primary care: a randomised controlled trial.  Lancet 374:628–634, 2009
[CrossRef] | [PubMed]
 
Perini  S;  Titov  N;  Andrews  G:  Clinician-assisted Internet-based treatment is effective for depression: randomized controlled trial.  Australian and New Zealand Journal of Psychiatry 43:571–578, 2009
[CrossRef] | [PubMed]
 
Vernmark  K;  Lenndin  J;  Bjärehed  J  et al:  Internet administered guided self-help versus individualized e-mail therapy: a randomized trial of two versions of CBT for major depression.  Behaviour Research and Therapy 48:368–376, 2010
[CrossRef] | [PubMed]
 
Learmonth  D;  Trosh  J;  Sadik  R:  The role of computer-aided psychotherapy within an NHS CBT specialist service.  Counselling and Psychotherapy Research 8:117–123, 2008
[CrossRef]
 
 Diagnostic and Statistical Manual of Mental Disorders , 4th ed.  Washington, DC,  American Psychiatric Association, 1994
 
First  MB;  Spitzer  RL;  Gibbon  M  et al:  Structured Clinical Interview for DSM-IV Axis I Disorders, Clinical Version .  Washington, DC,  American Psychiatric Press, 1997
 
Yellowlees  P:  Successful development of telemedicine systems—seven core principles.  Journal of Telemedicine and Telecare 3:215–222, 1997
[CrossRef] | [PubMed]
 
Johnston B: Developing a programme in e-mental health care; in Telepsychiatry and e-Mental Health. Edited by Wootton R, Yellowlees P, McLaren P. London, United Kingdom, Royal Society of Medicine Press, 2003
 
Lewinsohn PM, Muñoz RF, Youngren MA, et al: Control Your Depression (Revised and Updated). New York, Simon & Schuster, 1992
 
Bauer  MS;  McBride  L:  Structured Group Psychotherapy for Bipolar Disorder: The Life Goals Program , 2nd ed.  New York,  Springer, 2003
 
Clarke  G;  Reid  E;  Eubanks  D  et al:  Overcoming depression on the Internet (ODIN): a randomized controlled trial of an Internet depression skills intervention program.  Journal of Medical Internet Research 4:e14, 2002
[CrossRef] | [PubMed]
 
Muñoz RF, Miranda J: Group Therapy Manual for Cognitive Behavioral Treatment of Depression (MR-1198/4) Santa Monica, Calif, RAND, 2000
 
Unützer  J;  Katon  W;  Williams  JWJ  Jr  et al:  Improving primary care for depression in late life: the design of a multicenter randomized trial.  Medical Care 39:785–799, 2001
[CrossRef] | [PubMed]
 
Spitzer  RL;  Endicott  J;  Robins  E:  Research diagnostic criteria: rationale and reliability.  Archives of General Psychiatry 35:773–782, 1978
[CrossRef] | [PubMed]
 
Keller  MB;  Lavori  PW;  Friedman  B  et al:  The Longitudinal Interval Follow-up Evaluation: a comprehensive method for assessing outcome in prospective longitudinal studies.  Archives of General Psychiatry 44:540–548, 1987
[CrossRef] | [PubMed]
 
Rutter  CM;  Simon  GE:  A Bayesian method for estimating the accuracy of recalled depression.  Journal of the Royal Statistical Society 53:341–353, 2004
[CrossRef]
 
Sheehan  DV;  Harnett-Sheehan  K;  Raj  BA:  The measurement of disability.  International Clinical Psychopharmacology 11(suppl 3):89–95, 1996
[CrossRef] | [PubMed]
 
Ware  JE;  Kosinski  M;  Dewey  JE:  How to Score Version 2 of the SF-36 Health Survey .  Lincoln, RI,  QualityMetric Inc, 2000
 
Babor TF, Higgins-Biddle JC, Saunders JB: The Alcohol Use Disorders Identification Test: Guidelines for Use in Primary Care, 2nd ed. Geneva, World Health Organization, Department of Mental Health and Substance Dependence, 2001
 
Conigrave  KM;  Hall  WD;  Saunders  JB:  The AUDIT questionnaire: choosing a cut-off score. Alcohol Use Disorders Identification Test.  Addictions 90:1349–1356, 1995
[CrossRef]
 
Hatcher  L:  Step-by-Step Approach to Using the SAS System for Factor Analysis and Structural Equation Modeling .  Cary, NC,  SAS Institute Inc, 1994
 
Bentler  PM;  Bonett  DG:  Significance tests and goodness of fit in the analysis of covariance structures.  Psychological Bulletin 88:586–606, 1980
[CrossRef]
 
Cohen  J:  Statistical Power Analysis for the Behavioral Sciences .  New York,  Psychology Press, 1988
 
Gilbody  S;  Bower  P;  Fletcher  J  et al:  Collaborative care for depression: a cumulative meta-analysis and review of longer-term outcomes.  Archives of Internal Medicine 166:2314–2321, 2006
[CrossRef] | [PubMed]
 
de Graaf  LE;  Gerhards  SAH;  Arntz  A  et al:  Clinical effectiveness of online computerised cognitive-behavioural therapy without support for depression in primary care: randomised trial.  British Journal of Psychiatry 195:73–80, 2009
[CrossRef] | [PubMed]
 
Meglic  M;  Furlan  M;  Kuzmanic  M  et al:  Feasibility of an eHealth service to support collaborative depression care: results of a pilot study.  Journal of Medical Internet Research 12:e63, 2010
[CrossRef] | [PubMed]
 
References Container
+
+

CME Activity

There is currently no quiz available for this resource. Please click here to go to the CME page to find another.
Submit a Comments
Please read the other comments before you post yours. Contributors must reveal any conflict of interest.
Comments are moderated and will appear on the site at the discertion of APA editorial staff.

* = Required Field
(if multiple authors, separate names by comma)
Example: John Doe



Web of Science® Times Cited: 2

Related Content
Articles
Books
Gabbard's Treatments of Psychiatric Disorders, 4th Edition > Chapter 1.  >
The American Psychiatric Publishing Textbook of Substance Abuse Treatment, 4th Edition > Chapter 17.  >
Dulcan's Textbook of Child and Adolescent Psychiatry > Chapter 54.  >
Dulcan's Textbook of Child and Adolescent Psychiatry > Chapter 63.  >
Dulcan's Textbook of Child and Adolescent Psychiatry > Chapter 63.  >
Topic Collections
Psychiatric News
APA Guidelines
PubMed Articles