Depression is a common disorder associated with high levels of disability, impaired functional status, and high use of medical services (1,2,3). Despite the increasing use of antidepressant medications, many persons with depression do not receive treatment (4,5,6,7). Although the majority of individuals with depression seek health care in the general medical sector, primary care physicians may fail to identify half of all patients who have depression (2,5,6,8,9). Undetected depression is a serious public health problem, because as many as 50 percent of undetected depressed patients still meet the criteria for major depression after 12 months (8,10).
To supplement the identification of depression in clinical practice, public programs such as Depression Awareness, Recognition, and Treatment (D/ART) and the National Depression Screening Day (NDSD) have been used to screen for depression and to increase public awareness of depression and how it can be treated (11,12,13). However, there is no evidence that these programs have reduced the number of individuals with untreated depression in the community.
Other methods of screening may encourage a greater number of individuals to seek treatment. For example, computers are increasingly being used in patient education and screening (14,15,16,17,18,19,20,21,22,23,24,25). The Center for Epidemiologic Study Depression (CES-D) scale has been adapted as a computerized screen for use in the D/ART program (24), and a computerized depression screening test has been adapted for use as an automated telephone program (26). Both of these programs were well received by the participants (24,25,26). Individuals who completed the D/ART computerized screening test felt it was nonthreatening, and many preferred the computer test to a written test (24).
Internet screening programs can reach a large number of individuals from a broad geographic area at low cost. Searching for health information is cited as one of the most popular reasons for surfing the Internet (27), and it is estimated that more than 17 million adults in the United States have engaged in this activity (28). The Internet has the potential to provide anonymous, immediate feedback about depression status. This feature might be attractive to individuals who are trying to understand whether their symptoms indicate that they have depression and whether they should seek treatment.
The purposes of our study were to determine whether an Internet-based depression screening test would be useful in detecting individuals with depression, to compare the test with other public screening efforts in the U.S. population, and to estimate the costs of such a project.
Program design and screening procedure
Our analysis was based on data from all individuals who completed an Internet depression screening test between March and October 1999. The test was adapted from the CES-D scale—a well-validated 20-question instrument that assesses symptom severity and has a long record of good performance (29,30,31,32,33,34,35,36,37,38). We designed feedback to explain the screening test results, and we designed a follow-up questionnaire for individuals whose test score indicated a high probability of depression.
The most common cutoff for a positive screen with the CES-D is a score greater than 16, with sensitivity at this level ranging from 86 to 100 percent and specificity ranging from 84 to 53 percent in different samples (29,35,37,38). Although maximizing sensitivity is critical for screening programs, several authors have recommended more stringent cutoffs because of high false-positive rates at the cutoff of 16 and the potential harm this could cause, especially in primary care or community settings (34,38). Sensitivities of up to 100 percent have been reported at cutoffs of 20, 22, and 27, with improvements in false- positive rates and specificity (30,33,34,38). Rather than choosing a single arbitrary cutoff, we created three levels of probability of depression: low probability (CES-D score below 16), moderate (CES-D score of 16 to 22), and high (CES-D score above 22).
We placed the screening tests on Intelihealth, a large online health information portal. Intelihealth is a health media company owned by Aetna US Healthcare. Its popular consumer health information Web site includes a mental health zone. Any Internet user could access the screening test through links from search engines by typing in "health sites" or by going directly to www.intelihealth. com/depressiontest. An introductory page invited site visitors to "take the depression test."
Demographic data for participants, including age category, race, gender, and home zip code, were ascertained before they completed the screening test. To further characterize the participants, our data set was later linked to the 1990 Census Summary Tape (STF3) by zip code.
Participants who completed the CES-D screening test received feedback based on the probability of their having depression. Those who scored in the moderate range were instructed to seek further advice from a health professional or to return later and retake the test. Participants whose scores were above 22 were told that they had a high likelihood of clinical depression and were advised to seek treatment from a health professional. These individuals were then presented with a postscreening questionnaire designed to stimulate thought and provide a potential starting point for participation in treatment discussions. The questionnaire asked about the participants' past treatment, their attitudes concerning depression, and their treatment preferences. These participants were advised to print the summary of their score and the assessment and take them to a health professional for review.
The associations between a high-probability screening score and demographic characteristics were calculated by determining the percentage of the high-probability population in each demographic category and calculating adjusted relative odds using logistic regression. However, because of the prevalence of positive screens, these odds should not be interpreted as estimates of relative risk, although they do provide useful information about the association between a high-probability screen and demographic characteristics after adjustment. From the postscreening questionnaire, we calculated the frequencies of preferences for depression care among participants who screened positive.
Because the screening test responses were anonymous, we incorporated a plan to assess their overall validity. To assess the potential number of repeat takers, we calculated the number of unique patterns of zip code, age, gender, and race. Ethnic groups often live in similar neighborhoods, so we assessed the relationship between the number of minority individuals in a given zip code according to census data and the reported race or ethnicity of the participant. Finally, we compared the relative frequency of response between males and females to the CES-D question about "crying spells," which was found to be higher among females in previous samples (32,33). If responses had been arbitrarily checked, the gender pattern might have been lost.
For our main analysis, the demographic characteristics of our participants were compared with those of 1992 NDSD participants (13), those of all Internet users in a cross-sectional survey conducted in 2000 (40), and with the U.S. population estimates for 1999 (39) using chi square tests of association.
Screening participation and demographics
The depression screening test was completed 24,479 times during the eight-month study period. The demographic characteristics of the respondents by CES-D score are presented in t1. The associations between demographic characteristics and a higher CES-D score were unchanged in a multivariate analysis that included age, gender, and ethnicity. Women and individuals 45 years of age and younger were significantly more likely than men and individuals over age 45 to meet the criteria for depression. African Americans, Hispanics, and Asian Americans were less likely than whites to have a positive depression score.
More than 85 percent of respondents had unique patterns of age, gender, race, and zip code, so we estimated that the number of repeat participants was low. The number of African Americans, Asians, or Hispanics in a census zip code area was strongly associated with the individual participant reporting African American, Asian, or Hispanic race, respectively, in univariate linear regressions (p<.001). The validity of the total number of anonymous responses was also supported by the finding that 11 percent (N=1,978) of women reported crying spells most or all of the time, compared with 3 percent (N=208) of men (χ2=435, df=1, p< .001), a pattern seen in previous studies (32,33).
Fifty-eight percent (N=14,185) of the participants were identified as having a high probability of depression, as indicated by a CES-D score above 22. Of this group, a few did not answer all of the questions in the follow-up questionnaire; however, more than 92 percent (N=13,055) answered the questions presented in t2. Fifty-eight percent (N= 7,941) of these respondents agreed that they needed treatment for depression at that time. Almost half of them reported that they had never been treated for depression, but fewer than half of those who had not been treated felt they needed treatment at that time. Interestingly, many of the respondents indicated a preference for counseling over medications.
Comparison of screening participants
Fifty-eight percent of the respondents in our screening test had a high probability of depression. Fifty-three percent of NDSD participants had moderate depressive symptoms, and 23 percent had severe symptoms, as measured by the Zung depression scale (13). t3 presents demographic data for participants in the Internet screening, NDSD participants, all U.S. Internet users, and the total U.S. population. Because of the large sample sizes, statistical comparisons of demographic proportions between these samples using chi square tests were uniformly significant at p levels below .01. We focus here on the patterns among these differences that are most relevant to the design and implementation of a public screening program.
Compared with the gender distribution of U.S. Internet users and the U.S. population, a larger proportion of females than males participated in both the NDSD and the Internet screening programs. This finding is consistent with results of previous public screenings for depression (13). A larger proportion of individuals 18 to 30 years of age participated in the Internet screening than in an earlier community-based screening (13), although the proportion was smaller than that of this age group in the U.S. population. The age and ethnicity of Internet screening participants were fairly similar to those of all Internet users; thus our screening sample had a lower proportion of minorities than the U.S. population (16.6 percent versus 28 percent).
We were able to link census data for 19,940 individuals. Internet screening participants reported zip codes from all 50 states, Guam, Puerto Rico, and the Virgin Islands. The mean of the median per capita income in each zip code area where these participants lived was $16,252 (95 percent confidence interval= 16,122 to 16,383), compared with the median U.S. per capita income of $14,420. Only 10 percent of the participants who screened positive lived in predominantly rural areas, compared with 24 percent of the U.S. population, according to 1990 U.S. census data.
Our Internet-based screening test was completed more than 24,000 times during the eight months that data were collected. We found that the program identified not only a large number of individuals with depression, but also many who had never received treatment. Screening programs should attempt to reach vulnerable populations, and our program was more effective in identifying one demographic group—younger individuals with depression—than previous public screening programs (13). The National Depressive and Manic-Depressive Association has recognized the importance of testing new approaches to screening for depression (42). We feel that screening through the Internet may be a useful addition to other screening methods.
Individuals who have a high CES-D scale score do not necessarily need treatment, but because they may need a more thorough evaluation, we specifically encouraged them to seek the advice of a health professional. As in previous public screenings, a large proportion of our participants had never been treated (13). Fewer than half of the untreated participants who were advised to seek treatment agreed that they needed treatment at that time. A follow-up study of the NDSD participants found that 56 percent of individuals who were advised to seek treatment went for an appointment (12).
We would not expect our rate of treatment seeking to be as high as that seen in the NDSD, because advice from a computer screen is likely to seem less compelling than in-person advice. However, Internet screening can be maintained as a continuous service and thus reach a larger potential sample (12,13,26). The 1999 NDSD had approximately 8,000 individual screenings (43). Our screening test is still available online, and more than 60,000 individuals had been screened as of May 2000.
Internet programs often demand significant startup costs but require little additional maintenance. Our costs would have been higher if our program had required an independent Web site and we had had to account for the costs of the computers, access lines, and general support of the other components of the Internet site.
The main limitation of our program lies in the selection bias of the Internet, the so-called "digital divide." Research conducted in the electronic environment is subject to significant selection bias toward a younger, more affluent sample, which narrows the variation in participant characteristics (44). Although a major health communication objective of the federal initiative Healthy People 2010 is "to increase the proportion of individuals with Internet access at home" (45), a relatively small proportion of our sample consisted of minorities, older persons, and persons from rural areas.
The anonymous, self-report nature of our data limits the interpretation of findings, although our validation attempts support the validity of the responses. The association between demographic factors and CES-D scores cannot be strictly generalized because of the select nature of the screening population and the high prevalence of depression observed.
An unexpected finding was that participants from ethnic minority groups were less likely to have a positive depression score. Although it is possible that the screening questions or presentation have a hidden cultural bias, Roberts (46) found the same internal consistency and general factor analysis for the CES-D scale among whites, African Americans, and Hispanics. Our finding should be studied further in future public depression screening programs.
The practical significance of this study is that the Internet provides a platform for continuously screening the public for depression. Our program was most successful in identifying younger individuals with depression. We used the tailoring abilities of the Internet to link screening to treatment through an additional assessment of attitudes and preferences that individuals could print and take with them to a health provider. The recent Surgeon General's Report on Mental Health suggests that depression therapy be tailored to individual circumstances (47). Self-assessment of treatment preferences and attitudes can serve to enhance communication about treatment decisions between patient and physician.
Our study is the first we know of that evaluated the adaptation of a validated depression screening instrument for use on a large Internet health site. This project represented a significant level of cooperation between members of the academic and the private informatics arenas. The science panel on interactive health communication of the U.S. Department of Health and Human Services has emphasized cooperation between academic and private industry for high-quality evaluation of assessment programs (27). The cost of the program was significantly less because of this collaboration. In addition, the impact of the screening test was likely enhanced by its placement on an Internet site that already has heavy traffic.
We hope this research will continue to foster relationships between academia and industry in the realm of online health communication.
Dr. Houston, Dr. Cooper, Ms. Vu, and Dr. Ford are associated with the department of medicine at the Johns Hopkins University School of Medicine and School of Hygiene and Public Health in Baltimore, where Dr. Ford is also associated with the department of epidemiology. Dr. Ford and Dr. Cooper are also associated with the department of health policy and management. Dr. Kahn and Ms. Toser are associated with InteliHealth.Com, and Dr. Kahn is currently associated with WorldCare, Inc. Address correspondence to Dr. Ford, Associate Professor, 2024 East Monument Street, Suite 2-500, Baltimore, MD 21205-2223 (e-mail, firstname.lastname@example.org). Parts of this paper were presented at the national meeting of the Society of General Internal Medicine held April 29 through May 1, 1999, in San Francisco.
Scores on the Center for Epidemiologic Study Depression scale (CES-D) and the likelihood of depression by demographic characteristics of participants in an Internet depression screening test
Attitudes about and preferences for depression treatment among individuals whose scores on an Internet depression screening test indicated a high probability of depression1
1 Score above 22 on the Center for Epidemiologic Study Depression scale
Demographic characteristics of participants in an Internet depression screening test compared with participants in National Depression Screening Day (NDSD), US. Internet users, and the U.S. population1
1 All data are for 1999.