Screening the Public for Depression Through the Internet
Abstract
OBJECTIVE: Despite increased public screening, many individuals with depression remain undetected or untreated. This study explored the performance of an Internet-based program in screening for depression. METHODS: The Centers for Epidemiological Studies Depression (CES-D) scale was adapted as an online screening test. The program advised persons whose score indicated a high probability of depression to seek treatment and asked them to complete a survey of attitudes and preferences that could be printed and taken to a health professional. Responses were collected anonymously for epidemiologic research. Demographic characteristics of participants were compared with those of the U.S. population and participants in previous community screenings. The costs of the program were calculated. RESULTS: The CES-D scale was completed 24,479 times during the eight-month study period. The respondents' median age category was 30 to 45 years, and almost 30 percent were male; 58 percent (N=14,185) screened positive for depression, and fewer than half of those had never been treated for depression. The proportion of younger individuals was larger than in previous public screenings, but was still lower than that in the U.S. population. Our sample contained a lower proportion of minorities than the U.S. population (16.6 percent versus 28.3 percent). Sunk costs totaled $9,000, and additional marginal costs to maintain the system totaled $3,750. CONCLUSIONS: The Internet provides a continuously available, inexpensive, easily maintained platform to anonymously screen a large number of individuals from a broad geographic area. However, older adults and minorities may visit screening sites less frequently than other populations.
Depression is a common disorder associated with high levels of disability, impaired functional status, and high use of medical services (1,2,3). Despite the increasing use of antidepressant medications, many persons with depression do not receive treatment (4,5,6,7). Although the majority of individuals with depression seek health care in the general medical sector, primary care physicians may fail to identify half of all patients who have depression (2,5,6,8,9). Undetected depression is a serious public health problem, because as many as 50 percent of undetected depressed patients still meet the criteria for major depression after 12 months (8,10).
To supplement the identification of depression in clinical practice, public programs such as Depression Awareness, Recognition, and Treatment (D/ART) and the National Depression Screening Day (NDSD) have been used to screen for depression and to increase public awareness of depression and how it can be treated (11,12,13). However, there is no evidence that these programs have reduced the number of individuals with untreated depression in the community.
Other methods of screening may encourage a greater number of individuals to seek treatment. For example, computers are increasingly being used in patient education and screening (14,15,16,17,18,19,20,21,22,23,24,25). The Center for Epidemiologic Study Depression (CES-D) scale has been adapted as a computerized screen for use in the D/ART program (24), and a computerized depression screening test has been adapted for use as an automated telephone program (26). Both of these programs were well received by the participants (24,25,26). Individuals who completed the D/ART computerized screening test felt it was nonthreatening, and many preferred the computer test to a written test (24).
Internet screening programs can reach a large number of individuals from a broad geographic area at low cost. Searching for health information is cited as one of the most popular reasons for surfing the Internet (27), and it is estimated that more than 17 million adults in the United States have engaged in this activity (28). The Internet has the potential to provide anonymous, immediate feedback about depression status. This feature might be attractive to individuals who are trying to understand whether their symptoms indicate that they have depression and whether they should seek treatment.
The purposes of our study were to determine whether an Internet-based depression screening test would be useful in detecting individuals with depression, to compare the test with other public screening efforts in the U.S. population, and to estimate the costs of such a project.
Methods
Program design and screening procedure
Our analysis was based on data from all individuals who completed an Internet depression screening test between March and October 1999. The test was adapted from the CES-D scale—a well-validated 20-question instrument that assesses symptom severity and has a long record of good performance (29,30,31,32,33,34,35,36,37,38). We designed feedback to explain the screening test results, and we designed a follow-up questionnaire for individuals whose test score indicated a high probability of depression.
The most common cutoff for a positive screen with the CES-D is a score greater than 16, with sensitivity at this level ranging from 86 to 100 percent and specificity ranging from 84 to 53 percent in different samples (29,35,37,38). Although maximizing sensitivity is critical for screening programs, several authors have recommended more stringent cutoffs because of high false-positive rates at the cutoff of 16 and the potential harm this could cause, especially in primary care or community settings (34,38). Sensitivities of up to 100 percent have been reported at cutoffs of 20, 22, and 27, with improvements in false- positive rates and specificity (30,33,34,38). Rather than choosing a single arbitrary cutoff, we created three levels of probability of depression: low probability (CES-D score below 16), moderate (CES-D score of 16 to 22), and high (CES-D score above 22).
The depression screening expert system, programmed in PERL 5 on a Unix OS WWW server, calculated the score of the HTML-based CES-D screen. Javascript was used for real-time data analysis to confirm that all 20 CES-D questions had been completed.
We placed the screening tests on Intelihealth, a large online health information portal. Intelihealth is a health media company owned by Aetna US Healthcare. Its popular consumer health information Web site includes a mental health zone. Any Internet user could access the screening test through links from search engines by typing in "health sites" or by going directly to www.intelihealth. com/depressiontest. An introductory page invited site visitors to "take the depression test."
Demographic data for participants, including age category, race, gender, and home zip code, were ascertained before they completed the screening test. To further characterize the participants, our data set was later linked to the 1990 Census Summary Tape (STF3) by zip code.
Participants who completed the CES-D screening test received feedback based on the probability of their having depression. Those who scored in the moderate range were instructed to seek further advice from a health professional or to return later and retake the test. Participants whose scores were above 22 were told that they had a high likelihood of clinical depression and were advised to seek treatment from a health professional. These individuals were then presented with a postscreening questionnaire designed to stimulate thought and provide a potential starting point for participation in treatment discussions. The questionnaire asked about the participants' past treatment, their attitudes concerning depression, and their treatment preferences. These participants were advised to print the summary of their score and the assessment and take them to a health professional for review.
Analysis
The associations between a high-probability screening score and demographic characteristics were calculated by determining the percentage of the high-probability population in each demographic category and calculating adjusted relative odds using logistic regression. However, because of the prevalence of positive screens, these odds should not be interpreted as estimates of relative risk, although they do provide useful information about the association between a high-probability screen and demographic characteristics after adjustment. From the postscreening questionnaire, we calculated the frequencies of preferences for depression care among participants who screened positive.
Because the screening test responses were anonymous, we incorporated a plan to assess their overall validity. To assess the potential number of repeat takers, we calculated the number of unique patterns of zip code, age, gender, and race. Ethnic groups often live in similar neighborhoods, so we assessed the relationship between the number of minority individuals in a given zip code according to census data and the reported race or ethnicity of the participant. Finally, we compared the relative frequency of response between males and females to the CES-D question about "crying spells," which was found to be higher among females in previous samples (32,33). If responses had been arbitrarily checked, the gender pattern might have been lost.
For our main analysis, the demographic characteristics of our participants were compared with those of 1992 NDSD participants (13), those of all Internet users in a cross-sectional survey conducted in 2000 (40), and with the U.S. population estimates for 1999 (39) using chi square tests of association.
The "sunk" costs of the program—those that were incurred at startup and would not be affected by future actions—were estimated as the cost of developing the program: initial HTML coding, client-side Javascript, and server-side programming of the expert system (41). During the eight months of the study, the system was monitored for any additional costs, including those for World Wide Web server usage and maintenance and any unforeseen marginal costs, such as the need to upgrade the software or hardware. Because the system was posted on the Intelihealth server and integrated into its content, the cost of the Web server was estimated from costs that would have been required to support the system on an independent server.
Results
Screening participation and demographics
The depression screening test was completed 24,479 times during the eight-month study period. The demographic characteristics of the respondents by CES-D score are presented in Table 1. The associations between demographic characteristics and a higher CES-D score were unchanged in a multivariate analysis that included age, gender, and ethnicity. Women and individuals 45 years of age and younger were significantly more likely than men and individuals over age 45 to meet the criteria for depression. African Americans, Hispanics, and Asian Americans were less likely than whites to have a positive depression score.
More than 85 percent of respondents had unique patterns of age, gender, race, and zip code, so we estimated that the number of repeat participants was low. The number of African Americans, Asians, or Hispanics in a census zip code area was strongly associated with the individual participant reporting African American, Asian, or Hispanic race, respectively, in univariate linear regressions (p<.001). The validity of the total number of anonymous responses was also supported by the finding that 11 percent (N=1,978) of women reported crying spells most or all of the time, compared with 3 percent (N=208) of men (χ2=435, df=1, p< .001), a pattern seen in previous studies (32,33).
Fifty-eight percent (N=14,185) of the participants were identified as having a high probability of depression, as indicated by a CES-D score above 22. Of this group, a few did not answer all of the questions in the follow-up questionnaire; however, more than 92 percent (N=13,055) answered the questions presented in Table 2. Fifty-eight percent (N= 7,941) of these respondents agreed that they needed treatment for depression at that time. Almost half of them reported that they had never been treated for depression, but fewer than half of those who had not been treated felt they needed treatment at that time. Interestingly, many of the respondents indicated a preference for counseling over medications.
Comparison of screening participants
Fifty-eight percent of the respondents in our screening test had a high probability of depression. Fifty-three percent of NDSD participants had moderate depressive symptoms, and 23 percent had severe symptoms, as measured by the Zung depression scale (13). Table 3 presents demographic data for participants in the Internet screening, NDSD participants, all U.S. Internet users, and the total U.S. population. Because of the large sample sizes, statistical comparisons of demographic proportions between these samples using chi square tests were uniformly significant at p levels below .01. We focus here on the patterns among these differences that are most relevant to the design and implementation of a public screening program.
Compared with the gender distribution of U.S. Internet users and the U.S. population, a larger proportion of females than males participated in both the NDSD and the Internet screening programs. This finding is consistent with results of previous public screenings for depression (13). A larger proportion of individuals 18 to 30 years of age participated in the Internet screening than in an earlier community-based screening (13), although the proportion was smaller than that of this age group in the U.S. population. The age and ethnicity of Internet screening participants were fairly similar to those of all Internet users; thus our screening sample had a lower proportion of minorities than the U.S. population (16.6 percent versus 28 percent).
We were able to link census data for 19,940 individuals. Internet screening participants reported zip codes from all 50 states, Guam, Puerto Rico, and the Virgin Islands. The mean of the median per capita income in each zip code area where these participants lived was $16,252 (95 percent confidence interval= 16,122 to 16,383), compared with the median U.S. per capita income of $14,420. Only 10 percent of the participants who screened positive lived in predominantly rural areas, compared with 24 percent of the U.S. population, according to 1990 U.S. census data.
Costs
The sunk cost of the Internet screening application development, including HTML coding, client-side Javascript, and server-side programming of the expert system, was $9,000. Additional costs included $1,500 for World Wide Web server use and 15 additional hours of maintenance and upgrade of the software at $150 an hour, or $2,250, during the eight-month study period. Considering these maintenance costs as a marginal cost per month of operation, the program cost $468 per month to maintain. Because the number of participants remained fairly constant and did not overburden the server, no additional hardware was required. No further marginal costs were observed.
Discussion
Our Internet-based screening test was completed more than 24,000 times during the eight months that data were collected. We found that the program identified not only a large number of individuals with depression, but also many who had never received treatment. Screening programs should attempt to reach vulnerable populations, and our program was more effective in identifying one demographic group—younger individuals with depression—than previous public screening programs (13). The National Depressive and Manic-Depressive Association has recognized the importance of testing new approaches to screening for depression (42). We feel that screening through the Internet may be a useful addition to other screening methods.
Individuals who have a high CES-D scale score do not necessarily need treatment, but because they may need a more thorough evaluation, we specifically encouraged them to seek the advice of a health professional. As in previous public screenings, a large proportion of our participants had never been treated (13). Fewer than half of the untreated participants who were advised to seek treatment agreed that they needed treatment at that time. A follow-up study of the NDSD participants found that 56 percent of individuals who were advised to seek treatment went for an appointment (12).
We would not expect our rate of treatment seeking to be as high as that seen in the NDSD, because advice from a computer screen is likely to seem less compelling than in-person advice. However, Internet screening can be maintained as a continuous service and thus reach a larger potential sample (12,13,26). The 1999 NDSD had approximately 8,000 individual screenings (43). Our screening test is still available online, and more than 60,000 individuals had been screened as of May 2000.
Internet programs often demand significant startup costs but require little additional maintenance. Our costs would have been higher if our program had required an independent Web site and we had had to account for the costs of the computers, access lines, and general support of the other components of the Internet site.
The main limitation of our program lies in the selection bias of the Internet, the so-called "digital divide." Research conducted in the electronic environment is subject to significant selection bias toward a younger, more affluent sample, which narrows the variation in participant characteristics (44). Although a major health communication objective of the federal initiative Healthy People 2010 is "to increase the proportion of individuals with Internet access at home" (45), a relatively small proportion of our sample consisted of minorities, older persons, and persons from rural areas.
The anonymous, self-report nature of our data limits the interpretation of findings, although our validation attempts support the validity of the responses. The association between demographic factors and CES-D scores cannot be strictly generalized because of the select nature of the screening population and the high prevalence of depression observed.
An unexpected finding was that participants from ethnic minority groups were less likely to have a positive depression score. Although it is possible that the screening questions or presentation have a hidden cultural bias, Roberts (46) found the same internal consistency and general factor analysis for the CES-D scale among whites, African Americans, and Hispanics. Our finding should be studied further in future public depression screening programs.
Conclusions
The practical significance of this study is that the Internet provides a platform for continuously screening the public for depression. Our program was most successful in identifying younger individuals with depression. We used the tailoring abilities of the Internet to link screening to treatment through an additional assessment of attitudes and preferences that individuals could print and take with them to a health provider. The recent Surgeon General's Report on Mental Health suggests that depression therapy be tailored to individual circumstances (47). Self-assessment of treatment preferences and attitudes can serve to enhance communication about treatment decisions between patient and physician.
Our study is the first we know of that evaluated the adaptation of a validated depression screening instrument for use on a large Internet health site. This project represented a significant level of cooperation between members of the academic and the private informatics arenas. The science panel on interactive health communication of the U.S. Department of Health and Human Services has emphasized cooperation between academic and private industry for high-quality evaluation of assessment programs (27). The cost of the program was significantly less because of this collaboration. In addition, the impact of the screening test was likely enhanced by its placement on an Internet site that already has heavy traffic.
We hope this research will continue to foster relationships between academia and industry in the realm of online health communication.
Dr. Houston, Dr. Cooper, Ms. Vu, and Dr. Ford are associated with the department of medicine at the Johns Hopkins University School of Medicine and School of Hygiene and Public Health in Baltimore, where Dr. Ford is also associated with the department of epidemiology. Dr. Ford and Dr. Cooper are also associated with the department of health policy and management. Dr. Kahn and Ms. Toser are associated with InteliHealth.Com, and Dr. Kahn is currently associated with WorldCare, Inc. Address correspondence to Dr. Ford, Associate Professor, 2024 East Monument Street, Suite 2-500, Baltimore, MD 21205-2223 (e-mail, [email protected]). Parts of this paper were presented at the national meeting of the Society of General Internal Medicine held April 29 through May 1, 1999, in San Francisco.
 |
Table 1. Scores on the Center for Epidemiologic Study Depression scale (CES-D) and the likelihood of depression by demographic characteristics of participants in an Internet depression screening test
 |
Table 2. Attitudes about and preferences for depression treatment among individuals whose scores on an Internet depression screening test indicated a high probability of depression1
1 Score above 22 on the Center for Epidemiologic Study Depression scale
 |
Table 3. Demographic characteristics of participants in an Internet depression screening test compared with participants in National Depression Screening Day (NDSD), US. Internet users, and the U.S. population1
1 All data are for 1999.
1. Diagnostic and Statistical Manual of Mental Disorders, 4th ed. Washington, DC, American Psychiatric Association, 1994Google Scholar
2. Simon GE, VonKorff M: Recognition, management, and outcomes of depression in primary care. Archives of Family Medicine 4:99-105, 1995Crossref, Medline, Google Scholar
3. Depression Guideline Panel: Depression in Primary Care: Volume 1. Detection and Diagnosis. Agency for Health Care Policy and Research 93-0550. Rockville, Md, AHCPR, 1993Google Scholar
4. Oquendo MA, Malone KM, Ellis SP, et al: Inadequacy of antidepressant treatment for patients with major depression who are at risk for suicidal behavior. American Journal of Psychiatry 156:190-194, 1999Abstract, Google Scholar
5. Marino S, Gallo JJ, Ford D, et al: Filters on the pathway to mental health care, I. incident mental disorders. Psychological Medicine 25:1135-1148, 1995Crossref, Medline, Google Scholar
6. Regier DA, Narrow WE, Rae DS, et al: The de facto US mental and addictive disorders service system. Archives of General Psychiatry 50:85-94, 1934Crossref, Google Scholar
7. Lin E, Parikh SV: Sociodemographic, clinical, and attitudinal characteristics of the untreated depressed in Ontario. Journal of Affective Disorders 53:153-162, 1999Crossref, Medline, Google Scholar
8. Rost K, Zhang M, Fortney J, et al: Persistently poor outcomes of undetected major depression in primary care. General Hospital Psychiatry 20:12-20, 1998Crossref, Medline, Google Scholar
9. Klinkman MS: Competing demands in psychosocial care: a model for the identification and treatment of depressive disorders in primary care. General Hospital Psychiatry 19:98-111, 1997Crossref, Medline, Google Scholar
10. Coryell W, Endicott J, Winokur G, et al: Characteristics and significance of untreated major depressive disorder. American Journal of Psychiatry 152:1124-1129, 1995Link, Google Scholar
11. Regier DA, Hirschfeld RMA, Goodwin FK, et al: The NIMH Depression Awareness, Recognition, and Treatment program: structure, aims, and scientific basis. American Journal of Psychiatry 145:1351-1357, 1988Link, Google Scholar
12. Greenfield SF, Reizes JM, Magruder KM, et al: Effectiveness of community-based screening for depression. American Journal of Psychiatry 154:1391-1397, 1997Link, Google Scholar
13. Magruder KM, Norquist GS, Feil MB, et al: Who comes to a voluntary depression screening program? American Journal of Psychiatry 152:1615-1622, 1995Google Scholar
14. Lancaster T, Snead LF: Self-help interventions for smoking cessation. The Cochrane Library vol 4, 1999. Available at www. cochrane.org. Accessed Feb 2000Google Scholar
15. Sciamanna CN, Ford DE, Flynn JA, et al: An evidence-based interactive computer program to assist physicians in counseling smokers to quit. MD Computing 16:54-60, 1999Medline, Google Scholar
16. Strecher VJ: Computer-tailored smoking cessation materials: a review and discussion. Patient Education and Counseling 36:107-117, 1999Crossref, Medline, Google Scholar
17. Ramelson HZ, Friedman RH, Ockene JK: An automated telephone-based smoking cessation education and counseling system. Patient Education and Counseling 36:131-144, 1999Crossref, Medline, Google Scholar
18. Bental DS, Cawsey A, Jones R: Patient information systems that tailor to the individual. Patient Education and Counseling 36:171-180, 1999Crossref, Medline, Google Scholar
19. Dijkstra A, DeVries H: The development of computer-generated tailored interventions. Patient Education and Counseling 36:193-203, 1999Crossref, Medline, Google Scholar
20. Brug J, Campbell M, van Assema P: The application and impact of computer-generated personalized nutrition education: a review of the literature. Patient Education and Counseling 36:145-156, 1999Crossref, Medline, Google Scholar
21. Campbell MK, Bernhardt JM, Waldmiller M, et al: Varying the message source in computer-tailored nutrition education. Patient Education and Counseling 36:157-169, 1999Crossref, Medline, Google Scholar
22. Bull FC, Kreuter MW, Scharff DP. Effects of tailored, personalized and general health messages on physical activity. Patient Education and Counseling 36:181-192, 1999Crossref, Medline, Google Scholar
23. Lubalin J: Consumer Health Informatics and Patient Decision-Making. Rockville, Md, Agency for Healthcare Policy and Research, 1997Google Scholar
24. Ogles BM, France CR, Lunnen KM, et al: Computerized depression screening and awareness. Community Mental Health Journal 34:27-38, 1998Crossref, Medline, Google Scholar
25. Baehring T, Schulze H, Bornstein SR, et al: Using the World Wide Web—a new approach to risk identification of diabetes mellitus. International Journal of Medical Informatics 46:31-39, 1997Crossref, Medline, Google Scholar
26. Baer L, Jacobs DG, Cukor P, et al: Automated telephone screening survey for depression. JAMA 273:1943-1944Google Scholar
27. Science Panel on Interactive Communication and Health: Wired for Health and Well-Being: The Emergence of Interactive Health Communication. Washington, DC, US Department of Health and Human Services, 1999Google Scholar
28. Miller TE, Reents S: The health care industry in transition. The online mandate to change. New York, Cyber Dialogue, 1998Google Scholar
29. Radloff LS. The CES-D scale: a self-report depression scale for research in the general population. Applied Psychological Measures 1:401, 1977Crossref, Google Scholar
30. Klinkman MS, Coyne JC, Gallo S, et al: Can case-finding instruments be used to improve physician detection of depression in primary care? Archives of Family Medicine 6:567-573, 1997Google Scholar
31. Schulberg HC, Saul M, McClelland M, et al: Assessing depression in primary medical and psychiatric practices. Archives of General Psychiatry 42:1164-1170, 1985Crossref, Medline, Google Scholar
32. Cole SR, Kawachi I, Maller SJ, et al: Test of item-response bias in the CES-D scale: experience from the New Haven EPESE study. Journal of Clinical Epidemiology 53:285-289, 2000Crossref, Medline, Google Scholar
33. Clark VA, Aneshensel CS, Frerichs RR, et al: Analysis of effects of sex and age in response to items on the CES-D scale. Psychiatry Research 5:171-181, 1981Crossref, Medline, Google Scholar
34. Zich JM, Attkisson CC, Greenfield TK: Screening for depression in primary care clinics: the CES-D and the BDI. International Journal of Psychiatry in Medicine 20:259-277, 1990Crossref, Medline, Google Scholar
35. Parikh RM, Eden DT, Price TR, et al: The sensitivity and specificity of the Center for Epidemiologic Studies Depression Scale in screening for post-stroke depression. International Journal of Psychiatry in Medicine 18:169-181, 1988Crossref, Medline, Google Scholar
36. Brody DS, Thompson TL, Larson DB, et al: Recognizing and managing depression in primary care. General Hospital Psychiatry 17:93-107, 1995Crossref, Medline, Google Scholar
37. Beekman AT, Deeg DJ, Van Limbeek J, et al: Criterion validity of the Center for Epidemiologic Studies Depression scale (CES-D): results from a community-based sample of older subjects. Psychological Medicine 27:231-235, 1997Crossref, Medline, Google Scholar
38. McDowell I, Newell C. Depression. In: Measuring Health: A Guide to Rating Scales and Questionnaires. 1st ed. New York, Oxford University Press, 1996Google Scholar
39. Resident Population Estimates of the United States. US Census Bureau, Population Division, Population Projections Branch, 1999. Available at www.census.govGoogle Scholar
40. Internet Users. Cyber Dialogue, Inc. 2000. Available at www.cyberdialogue.comGoogle Scholar
41. Finkler SA: Cost Accounting for Health Care Organizations. Gaithersburg, Md, Aspen, 1994Google Scholar
42. Hirschfeld RM, Keller MB, Panico S, et al: The National Depressive and Manic-Depressive Association consensus statement on the undertreatment of depression. JAMA 277:333-340, 1997Crossref, Medline, Google Scholar
43. National Depressive and Manic-Depressive Association. Why Screen for Depression, 2000. Available at www.ndmda.org/ screening.htm.Google Scholar
44. Houston JD, Fiore DC: Online medical surveys: using the internet as a research tool. MD Computing 15:116-120, 1999Google Scholar
45. Healthy People 2010. Washington, DC, Office of Disease Prevention and Health Promotion. Health Communication, 2000Google Scholar
46. Roberts RE. Reliability of the CES-D scale in different ethnic contexts. Psychiatry Research 2:125-34, 1980Crossref, Medline, Google Scholar
47. Mental Health: A Report of the Surgeon Genera, 2000. Available at www.surgeon general.gov/library/mentalhealth/home.htmlGoogle Scholar
