Challenging the Public Stigma of Mental Illness: A Meta-Analysis of Outcome Studies

Stigma has broadly harmful effects on the lives of people with mental illness. Public stigma (the prejudice and discrimination endorsed by the general population that affects a person) has been distinguished from self-stigma (the harm that occurs when the person internalizes the prejudice) with attempts to craft separate interventions for each type of stigma (1,2). Advocates from around the world have been trying to erase the public stigma of mental illness for more than 50 years (3). In the 1950s, Cummings and Cummings (4) used group discussions and films throughout a small Canadian town to affect mental illness stigma. Since 1996, the World Psychiatric Association has promoted its Open-the-Doors programs, which has produced a slew of evaluations (3,5). Australia’s beyondblue initiative to raise awareness and reduce stigma surrounding depression has been in place for more than a decade, whereas Canada and the United Kingdom have jumped into the antistigma fray only in the past couple of years.

Background

The United States first systematically pursued antistigma campaigns after the 1999 White House Conference on Mental Health. American efforts against stigma have soared given the energy and resources of professional groups (the American Psychiatric Association and American Psychological Association), advocacy groups (such as the National Alliance on Mental Illness [NAMI] and Mental Health America), pharmaceutical companies (Eli Lilly), and government bodies (the Substance Abuse and Mental Health Services Administration and the National Institute of Mental Health). Research has begun to examine the effect of various programs intended to erase public stigma. This article reports on a meta-analysis of findings reported in the available research literature.

Three approaches to change

Approaches to changing public stigma have been divided into three paradigms on the basis of a review of social- psychological research related to racial-ethnic and gender minority groups: education, contact, and protest (6). Educational approaches to stigma challenge inaccurate stereotypes about mental illnesses, replacing them with factual information (for example, contrary to the myth that people with mental illnesses are homicidal maniacs, the difference in the rate of homicides by people with serious psychiatric disorders versus the general public is very small). Educational strategies have included public service announcements, books, flyers, movies, videos, Web pages, podcasts, virtual reality, and other audiovisual aids (7,8). Some benefits of educational interventions include their low cost and broad reach.

A second strategy for reducing stigma is interpersonal contact with members of the stigmatized group. Individuals of the general population who meet and interact with people with mental illnesses are likely to lessen their levels of prejudice (9). Social-psychological research has identified factors that seem to moderate contact effects (10,11), including one-to-one contact so that people who engage with one another can learn of similar interests and potentially cultivate a friendship (12,13), contact that includes a common goal (14), and interactions with a person who moderately disconfirms prevailing stereotypes (15,16).

Social activism, or protest, is the third form of stigma change we examined. Protest strategies highlight the injustices of various forms of stigma and chastise offenders for their stereotypes and discrimination: “Shame on us all for perpetuating the ideas that people with mental illness are just ‘big kids’ unable to care for themselves.” There is anecdotal evidence suggesting that protest can reduce harmful media representations (17). However, some research implies protest campaigns that ask people to suppress prejudice can produce an unintended “rebound” in which prejudices about a group remain unchanged or actually become worse (18–20). In one set of studies, Macrae and colleagues (18) found that research participants directed to suppress stereotypes about skinheads showed greater stereotype activation and increased distance from members of that group.

Outcomes of public stigma change have been assessed in several ways, with self-reported social-cognitive measures dominating (21–23). These are sometimes understood in terms of a simple path model: attitudes, affect, and behavior (9). Attitudes reflect the stereotypes about mental illness and include ideas about blame, dangerousness, and incompetence. Affect concerns the emotional reaction to attitudes: dangerousness begets fear, and blame yields to anger. Discriminatory behavior results from stereotypic attitudes and affect; for example, the public is hesitant to employ (24–28) or to rent property (28–31) to people with mental illness. Behavior per se is a difficult construct to assess because most studies do not have resources to observe actual responses after stigmatizing attitudes and affect. This kind of assessment requires some form of independent observation in an overwhelming number of settings and times. As a result, many studies fail to assess change in discriminatory actions. More likely, self-report measures used as outcome indicators in public stigma change studies represent behavioral intentions—the self-reported likelihood that people somehow anticipate distancing themselves or otherwise avoiding a person with mental illness (22).

Past reviews

Two reviews have attempted to make sense of the body of research on public stigma change. Holzinger and colleagues (32) summarized 51 studies of interventions targeting public stigma conducted in real-world settings with members of the general public. They did not include investigations using experimental or other laboratory designs. Documents were located through review of PubMed, gray literature, and the Internet; psychological databases (PsycNET) and replication studies were excluded. They concluded that education and contact conditions seem to have positive effects on attitudes. Their findings on the impact on behavior were less clear. Outcomes in this review were coded solely as positive or negative, with no effect sizes, so inferential statistics were not used to more carefully make sense of education versus contact effects. Moreover, no careful coding schema was used to summarize treatment methods or outcome variables. The list of studies also did not include several prominent investigations in the field. Another older review focused solely on contact effects (33). The review considered 22 studies, dividing them into retrospective (reporting the effects of previous contact) and prospective (the effects of planned contact where an advocate interacts with a group). The authors concluded that both types had robust inverse effects on stigma. Like the review by Holzinger and colleagues, however, the Couture and Penn study (33) did not determine effect sizes or provide any careful codes of research design. Moreover, retrospective studies would not fall under the rubric of stigma change as examined here—that is, a planful effort to change stereotypes, prejudice, and discrimination of targeted groups.

Given this background, we pursued a meta-analysis of strategies to change public stigma, keeping several aims in mind. We sought to describe the degree to which the social-cognitive model of stigma (attitudes, affect, and behavior intention) has been tested. Consistent with research on the prejudice related to ethnicity and gender, we hypothesized that contact strategies would yield significantly better effects than education (6). We describe the quality of research designs used to examine these questions. Two-by-two (pre-post) randomized controlled trials offer the most rigorous test of contact versus education; hence, effect sizes from just those studies were separately examined. We examined the effects of milieu for establishing contact—via video or in person—and hypothesized that the grassroots nature of meeting in person would lead to better effects. We also tested how antistigma programs influence the stigmatizing attitudes and behavioral intentions of adolescents, an important target group.

Methods

Selection

We used the Centre for Reviews and Dissemination’s guidelines for systematic reviews in health care (34) to organize and conduct the meta-analysis. The guidelines include a comprehensive algorithm for locating relevant studies in the broadest sphere of published documents, coding the studies in a comprehensive and reliable manner, and determining valid effect sizes that serve as the foundation for analyses of important research goals. We adopted a snowball strategy for finding studies. We began with databases most likely to yield peer-reviewed or mentor-reviewed papers: PsycNET, PubMed, Scopus, and Dissertation Abstracts. Searches yielded primary sources, and we then reviewed the reference sections of those sources to identify additional candidates for consideration. Searches focused on articles from the inception of the database up to October 2010. We also made a special effort to identify articles in languages other than English. Several articles identified in the above searches were reviewed by colleagues bilingual in Chinese, German, French, Polish, and Spanish. In addition, we searched several Indian databases (Scientific Journal Publishing in India, HRH Global Resource Center Web page, connectjournals.com, Indian Academy of Sciences, Indian Institute of Science, Indian National Science Academy, and Indianjournals.com) and Chinese databases (the China National Knowledge Infrastructure and Chinese Electronic Periodical Services). Despite this effort, only two German language articles were found to meet inclusion criteria. Finally, we cross-walked our collection of articles with the recent review by Holzinger and colleagues (32).

We used three sets of search terms: stigma, mental illness, and change program. Mental illness was searched using the broad rubric plus more specific terms like “mental disorder” or individual diagnoses. Along with “change program,” seven additional search terms were used that represented specific programs (education or contact) or more general synonyms (stigma reduction or antistigma program). Using the search terms, we located more than 8,700 documents, and 72 of them ultimately provided useable data to address our goals. The review and winnowing process is summarized in Figure 1. Note that most articles yielded multiple effect sizes, depending on condition, trials, and outcome variables therein.

Coding

A code book was developed to summarize each study; codes represented key concepts about public stigma change described in our introduction. Specific codes included descriptors of research participants, research design, type of stigma change program, outcome measures, and moderating variables. Type of antistigma intervention was indicated by one of three codes: protest, contact, and education. Clear definitions of interventions were provided to ensure that studies were grouped appropriately (these definitions and the code book are available from the first author).

We coded for type of outcome variable in terms of attitude, affect, or behavioral intention. We entered number of research participants assigned to each group and target of the intervention. Some studies looked at developmental issues; therefore, we coded for whether studies included children and adolescents. Some studies looked at effects on power groups, including employers, health care professionals, and police officers. Demographic characteristics of targets were also entered, including gender, age, race-ethnicity, and education level. Two graduate student raters were trained on the coding schema and had to meet the reliability criterion on five articles (κ=.84) before working independently. After review of more than 30 articles, interrater reliability was checked again for drift but remained high (κ=.94).

Effect size analysis

The effect size used in this study was the standardized mean difference (35) comparing the means of persons who received the intervention versus a contrast condition, divided by the within-group standard deviation of scores. Effect sizes were scaled so that that a positive effect size would indicate less stigmatizing attitudes among those who received the intervention. Data were obtained from a variety of research designs, so effect sizes were computed with all available information from each design (36,37). Effect sizes were computed from means and standard deviations where available. In other cases, effect size was computed from available statistics (t or F) or p values with the use of formulas appropriate to each research design (38).

A few studies included more than one treatment condition. Wherever conditions represented different types of intervention, a separate effect size was computed for each intervention that met the inclusion criteria. Otherwise, effect sizes were averaged to produce a single effect size for the type of intervention. Many studies included multiple outcome measures. Variables representing distinct outcome constructs were maintained as separate effect sizes. When multiple measures of the same construct were provided, these were included as separate effect sizes for subsequent analyses. Comparisons between effect sizes were conducted with a weighted one-way analysis of variance (ANOVA) from the SPSS macro MetaF (39).

Results

We found 72 articles that were related to changing the public stigma of mental illness with data suitable to the meta-analyses described herein (7,40–109). Table 1 provides summary statistics of these studies, including mean frequencies of demographic characteristics determined by summing frequencies from each study that provided a specific demographic and dividing by total number of studies. We also determined mean and standard deviation of participants’ average ages in a similar manner. The 72 articles represented 79 independent studies with 38,364 research participants from 14 countries; by continent, they were from Europe (N=22,179), North America (N=14,307), South America (N=63), Asia (N=1,299), and Australia (N=516). There were none from Africa. The median number of research participants per study was 150.0 (mean±SD=540.3±1,279.3). Articles were published from January 1972 to October 2010. The literature search yielded 612 effect sizes. Of the research participants, 58.7% were female, and most were European or European American (>60%). The participants were mostly single (>70%) and students (≥60%) with a college education or higher (>60%).

An important goal of antistigma programs is to strategically target groups who, by virtue of age or role, have importance in terms of the broad stigma change picture (110). For example, employers who endorse stigma may be less likely to hire people with mental illness and therefore might be important targets. Table 1 provides the frequency with which the 79 studies targeted specific groups. Although seemingly an important group, children under age 12 were rarely the focus of stigma change (.8%); adolescents, on the other hand, were far more common targets, with almost 25% of studies we examined targeting stigma among teenagers. Professionals (4.7%) and students in professional programs (8.5%) were relatively more common targets of stigma change, as were criminal justice professionals (7.2%). This information on targeted groups was the only proxy coded for setting in which a specific antistigma program was conducted.

One way we sought to index the quality of the antistigma programs was whether there was evidence of some sort of manual or training guiding the program. About 40% of studies reported use of an antistigma program manual. About 14% acknowledged some sort of training of program facilitators. We also coded for studies that checked fidelity of the antistigma program and found none.

An important finding of meta-analyses is an audit of the type of outcome measures used to assess effects. The bottom of Table 1 provides frequencies with which constructs were examined in individual studies. They are divided into the three outcome areas: attitudes, affect, and behavioral intentions. Frequent measures of attitudes included dangerousness, competence, responsibility, and poor prognosis. Affect was much less often assessed and was represented by proxies of fear and anger. Behavioral intentions were most often represented by measures of avoidance. We coded psychometric values of assessments where available; summaries are provided in Table 1 by outcome type (see table footnotes). Test-retest reliabilities were not found for measures of affect; six alphas were obtained for this domain, with the mean being high (>.8). Thirty or more test-retest reliabilities or alphas were provided for measures of attitudes and behavioral intentions; all of their means were satisfactory, exceeding .725. The next section summarizes effect sizes by antistigma approach. Each analysis provides both an overall result (across all outcomes), as well as separate results for each type of outcome measure.

Effects of contact and education on public stigma change

The means and standard errors of effect sizes (d) are organized in Table 2 by antistigma approach; the table also includes the number of effect sizes gleaned from the literature per approach (K). Note that this number was often above the 72 articles and reports and 79 individual studies found from the review because most investigations used multiple dependent measures or comparisons to test for differences. [A forest plot summarizing the effect sizes from the overall analysis is provided online as a data supplement to this article. The graph depicts the effect size estimate from each study, along with the 95% confidence intervals (111).]

Two kinds of significant differences were examined in this meta-analysis. The first was whether the effect size differed significantly from zero effect. Mean d, representing the overall effect for protest, was .099, which did not differ significantly from zero effect, suggesting that protest led to no significant change in outcome. Note that Cohen (112) defined effect sizes as negligible (<.10), small (.10–.30), medium (.30–.50), and large (>.50). Only four effect sizes representing protest effects (K=4) emerged from the literature, and they were all obtained from only one study. Hence, for the remainder of the analyses, we considered only the pattern of effect sizes for education versus contact.

Table 2 shows overall effect sizes for contact and education each significantly differed from zero (p<.001) and were relatively small (between .10 and .30). This was based on 431 and 177 d values for education and contact, respectively. The second kind of significant difference examined in this study was between the distribution of effect sizes for antistigma groups. Results of a random-effects, one-way ANOVA showed mean effect sizes differed significantly between protest, education, and contact approaches to changing public stigma. However, an additional random-effects ANOVA failed to show that effect sizes between contact and education differed significantly. Mean d also differed significantly from zero for both approaches when change in attitudes was assessed and yielded medium effect sizes. Table 2 includes the weighted one-way ANOVA for difference in these d values (total K=373) and showed a significant difference (Q=6.17, p<.05) for attitude effects, with contact with persons with mental illness leading to greater improvement in changing public stigma. Education was shown to yield significant improvements in the affect and behavioral intention domains as well, with small effect sizes. Contact yielded significant improvements in behavioral intentions but not in affect. Mean d values did not differ significantly between education and contact across affect and behavioral intentions.

Variability of design quality across studies is a concern in meta-analyses, because effect sizes arise out of research studies that differ in experimental rigor. Meta-analysts have written much about indices of design quality and seem to agree that studies using randomized controlled trials (RCTs) require similar procedural manipulations to make them relatively similar and methodologically strong (113). Thirteen articles reported RCTs, yielding 187 effect sizes. These values are summarized in the bottom of Table 2. Once again, the mean overall d for education and contact was significantly different from zero effect (p<.01 and .001, respectively). Moreover, results of the weighted one-way ANOVA were significant, with the mean effect size for contact being significantly greater than for education (p<.05). Table 2 also includes effect sizes by attitude and behavioral intention. Education and contact yielded effect sizes significantly greater than zero for both constructs. Weighted one-way ANOVAs were significant for attitudes and behavioral intentions (p<.05), with mean effect size for contact larger than that for education in both instances.

Video versus in-person effects

Table 2 includes Qw as an index of homogeneity, or the degree to which variance across studies within a subgroup is larger than expected due to chance. Significant Qw values were found for half the distribution of effect sizes for contact approaches (top of Table 2), and for three out of four of the d distributions for the RCTs. These findings suggest that moderators may further explain some of the significance of the contact effect. Our review of the literature suggested one variable that might moderate effects: the means of contact (contact in person versus via video). Namely, was contact from someone with mental illness provided in person or on videotape? Table 3 summarizes mean d values for overall effect size as well as for attitudes and behavioral intentions across studies that used contact in person versus by video.

The mean d for overall effect size was significant for both types of contact. A weighted one-way ANOVA showed that compared with contact by video, in-person contact had a significantly greater effect size (p<.001). Moreover, effect sizes for the two types of contact were significantly greater than zero for attitudes and behavioral intention. One-way weighted ANOVAs showed in both cases that effect size after in-person contact was significantly greater than that yielded by videotaped contact.

Effects on adolescents

Some advocates believe that antistigma programs for children are an effective way to prevent stigma from emerging in adulthood; although very few studies targeted children under age 12, almost a quarter (N=19) targeted adolescents. Table 3 summarizes effect sizes across education and contact conditions for this age group. Findings were a bit different from those reported for overall samples. Mean effect sizes showed education and contact both led to significantly increased overall effects as well as to change in attitudes and behaviors. However, unlike the ANOVA for adults, the one-way weighted ANOVA examining mean effect sizes for attitudes showed education yielded a larger effect than contact. No significant differences were found in mean effect sizes for education and contact with overall effect size or behavioral intention.

Table 3 also shows results of in-person and video contact with adolescents; these findings mostly paralleled those of the total meta-analysis in the upper portion of Table 3. Effects by contact type led to significant changes in overall outcome. Moreover, the one-way weighted ANOVA showed that in-person contact yielded greater overall effects than contact by video. In-person contact yielded significant change in attitudes and behavioral intentions, whereas video contact showed significant improvements only in attitudes. A one-way ANOVA for difference in effect sizes for behavioral intentions yielded nonsignificant trends (p<.10), suggesting in-person contact had a bigger effect than video contact for adolescents.

Discussion

Although contact and education both seem to significantly improve attitudes and behavioral intentions toward people with mental illness, contact seems to yield significantly better change, at least among adults. This is especially evident in studies that used more rigorous research designs, such as RCTs. Mean effect sizes for contact when assessing overall effects as well as effects on attitudes and behavioral intentions were significantly greater than those found for education. Meeting people with serious mental illness seems to do more to challenge stigma than educationally contrasting myths versus facts of mental illness.

One additional finding—and the most important—emerged from the meta-analysis: face-to-face contact with the person, and not a story mediated by videotape, had the greatest effect. This was evident for overall impact as well as for changing attitudes and behavioral intentions. This finding juxtaposes two important agendas in stigma change: broad audience versus grassroots control. Videotaped contact has the potential for a broad audience: disseminating the video via a variety of online platforms and television networks exponentially increases exposure of the antistigma effort compared with face-to-face approaches. However, the social marketing campaigns that rely on videotaped contact diminish grassroots control of the effort. Social marketing campaigns are often managed by government and advertising consortia rather than by people with mental illness who are targeting key groups at the local level for meaningful stigma change. In-person contact leads to better effects, but both types of contact significantly diminish stigma. Those crafting an antistigma campaign need to balance relative effect sizes with population served and grassroots control issues.

Additional intriguing differences were found when we focused on contact and education effects on adolescents. Once again, both education and contact were shown to significantly affect stigma overall, as well as attitudes and behavioral intentions. However, contrary to the other findings across all studies, education yielded significantly greater effects on attitudes than contact did. Perhaps this difference emerged because adolescents’ beliefs about mental illness are not as firmly developed as adults’ and adolescents therefore are more likely to be responsive to education effects. Compared with adults, adolescents show more variance in response to stigma measures and hence have more room for change. What implications do these findings have for ongoing campaigns for children and adolescents? They may bolster the need for caution, as many have voiced, about using peers as contacts—that is, high school students telling their stories about mental illness and recovery. Advocates have been concerned about risks for persons with mental illness in this age group compared with risks for adults in similar circumstances. Still, it is unclear from the meta-analysis whether the relatively muted effects of contact compared with education occurred because the person providing contact to the adolescents was close to their age. Generally, similarity between contact and audience is needed for most credibility and greatest antistigma effects (110). Cohort and developmental differences need to be considered in future research on contact programs in schools. It is also important to note that in-person contact seemed to produce greater effects than video-based attempts.

We found that protest or social activism concerning public stigma was rarely examined in research. In part, investigators seem to be “voting with their feet,” suggesting that protest is not a good strategy to affect stigmatizing attitudes and behavioral intentions. In fact, the effect sizes found in the meta-analysis did not show that protest yielded significant changes in stigma. Still, protest probably should not be discarded entirely. Although it is not an effective avenue for changing attitudes, it may be useful in suppressing behaviors that promote stigma, especially in the media. Programs like NAMI’s StigmaBusters have targeted stigmatizing advertisements, news stories, and entertainment through strategic letter-writing campaigns. Anecdotally, these seem to have had some effects—for example, one campaign led the American Broadcasting Company to pull its television drama Wonderland in 2000 after two episodes. The show stoked stereotypic connections between mental illness and violence. Research is needed to determine whether anecdotes like these translate to meaningful impact on stigma in the media.

Meta-analyses are known for their problems, and this one had its share. The 22 outcome measures summarized in Table 2 were reduced to three effect sizes, thereby losing theoretical and methodological sensitivity. Contact and education are complex processes, yet each was squeezed into a single mold for our analyses. We were unable to report separate analyses on follow-up effects because of the significant differences in their definitions in the relatively few studies that reported them. Despite the call for targeted antistigma programs, such audiences were not strategically sought in the studies we found on stigma change. Another way to target stigma change is by diagnostic group: messages that frame the stigma of schizophrenia versus the stigma of major depression. Programs with this kind of distinction were largely not present in the literature. Outcome was largely limited to self-report. Given that discriminatory behaviors are perhaps the most egregious of stigma’s impacts, examination of change in behavior was regrettably missing.

Conclusions

One of the benefits of reviews like these, and the methodological audits on which they rest, is the directions signaled for future research. Both education and contact have some value in stigma change. Future research needs to identify moderators of these effects. They include adjustments within the intervention (for example, the means of contact) as well as to the target of the intervention. Degree of exposure to the program is an important parameter. Greater resources are needed to go beyond psychological self-reports to observe actual behavior change.

One area not discussed in this review is the impact of multiple stigmas and how antistigma programs should be crafted to address the interaction of multiple prejudices—for example, mental illness and demographic characteristics (such as race, gender, or sexual orientation) or other health conditions (such as HIV/AIDS or physical disabilities). This concern brings us back to the grassroots goal, namely that stigma is a local issue shaped by the experience of mental illness in a variety of social contexts. Hence, stigma change, and evaluation of this change, must be conducted at the local level. This calls for future research that is dominated by community-based participatory research and investigations marked by partnerships between those skilled in research methods and local advocates.