Evaluation of Antistigma Interventions With Sixth-Grade Students: A School-Based Field Experiment

Stigma critically influences the well-being and recovery of people with mental illnesses, affecting employment, income, social ties, quality of life, mastery, self-esteem, depressive symptoms, and access to medical and mental health services (1–15). In recognition of this problem, there has been a sustained effort to reduce stigma by educating the public about neurobiological bases of mental illnesses and available treatments, with the assumption that framing mental disorders as medically treatable “illnesses like any other” would reduce stigma (16–19). There is clear evidence that the public has adopted this understanding and that mental health treatment is increasingly viewed as beneficial (20–22) and sought by the public (23).

Nevertheless, these changes have not been accompanied by stigma reduction. Core aspects of stigma—emotional reactions, stereotypes, and social distance—remain unchanged or have worsened (20,21,24). What can explain this discrepancy? Research now shows that biological explanations tend to increase rather than decrease stigma (25–27). Thus it may be necessary to address stigma directly rather than by changing causal beliefs. Stigmatizing attitudes and behaviors may also be harder to change than causal beliefs because the former involve emotions and can have personal consequences. Addressing these attitudes before they are firmly set may be a promising approach to reducing stigma, with evidence suggesting that stigmatization occurs as early as childhood and adolescence (28–35). Therefore, the study reported here evaluated the effectiveness of a school-based intervention that directly focuses on stigma. Education- and contact-based interventions can reduce stigmatizing attitudes and behavioral intentions. Most studies focus on adults; fewer target adolescents (36–40). We located only six studies that targeted pre–high school youths (41–46). These studies, with samples ranging from 185 to 1,500 in more than 16 states, found significant reductions in stigma among youths between third and eighth grades. All employed a curriculum of some sort; one also included a contact intervention. Three studies included follow-ups of three to six months; three employed control conditions.

We implemented interventions designed to improve knowledge, attitudes, behavioral intentions, and behaviors about mental illness and help seeking among sixth-graders. To develop a stigma intervention that can be broadly disseminated, we evaluated a classroom-based curriculum, which was designed to appeal to teachers and students and easy for teachers to implement without specialized training. We also evaluated the effectiveness of a contact intervention and of an intervention in which classrooms were saturated with antistigma materials. Several aspects of social and psychological development led us to target sixth-graders rather than younger children. Preadolescents begin to understand that others have thoughts and feelings different from their own; preadolescents also include interpersonal and psychological features in their understanding of themselves and others and experience heightened social comparison (47).

In addition to augmenting a very small body of research, several strengths of the study allowed it to meaningfully extend what can be concluded from the existing literature. As in our study, previous studies have used a teacher-administered curriculum that does not entail extensive teacher training, suggesting that a relatively easily disseminated curriculum can reduce stigma. However, most of the previous studies relied on samples that involved teacher self-selection, allowing the possibility that effects will be found only when teachers favor an antistigma agenda. In our study, self-selection played a role at the school level but not the classroom level, which likely introduced less bias. Ours is the only study to include a fidelity measure, which allowed us to evaluate how faithfully the intervention was enacted in the classroom and whether fidelity was related to outcomes of the intervention. Finally, although our sample came from a single geographic locale, it provided excellent representation of the major racial-ethnic groups in the United States. All these features raise optimism that any reductions in stigma we found can generalize to the broad population.

Additional strengths introduced in this study include the evaluation of multiple interventions. Three of the previous studies included a control condition; however, in all but one case, the control condition was usual instruction. This leaves open the possibility that intervention per se—via novelty, special attention, and so forth—rather than intervention content produced attitude change. By testing three interventions, we were able to compare their effectiveness and attribute change to a particular intervention as opposed to intervention per se. Finally, by including vignettes describing specific disorders, we were able to assess changes in responses to these disorders as well as the more typical approach that focuses on the generic concept of “mental illness.”

Methods

Study Design

The delivery of the three interventions was independently varied in a fully crossed design, resulting in eight experimental cells, including a control group that received no interventions (Table 1). Sixteen middle schools (with separate zip codes) from an urban school district in Texas were ranked according to performance on the statewide standardized assessment of math, English, and science. Rankings based on percentage of families below the poverty line were nearly identical. We randomly assigned the top eight schools to one of eight cells; the bottom-ranked eight schools were then assigned to a cell in the reverse order so that, for example, the top- and bottom-ranked schools were paired. Each cell (two schools in each) was randomly assigned to a study condition. Before the study began, two schools dropped out for non–study-related reasons. The study was repeated during a second academic year with a new set of sixth-grade students in five of the original schools chosen because they had demographic characteristics similar to the lost schools. Thus a total of 19 classes from 14 schools were included in the study. There were no significant differences in sociodemographic characteristics between repeated and nonrepeated classrooms (results available on request).

Participants and Procedures

A total of 751 students (60% of those invited) agreed to participate. Of these, 721 (96%) completed the study. Loss to follow-up (N=30) did not differ by gender, school, or socioeconomic status, but it did differ by race-ethnicity, with African American students dropping out at a higher rate, mostly due to moving. Sociodemographic characteristics of the 721 participants and their families are presented in Table 2.

Pre- and posttest instruments were self-administered in physical education classes on laptop computers from February to May and September to December 2012. Each class received its assigned combination of interventions within one week of pretesting. Posttest instruments were administered within a week after the intervention. Informed consent of participants and their parents or guardians, following the Helsinki guidelines, was obtained after procedures were fully explained. All students in the classroom were exposed to the assigned intervention(s); only the students who provided consent completed the assessment instruments. The Columbia University Medical Center Institutional Review Board approved the study.

Interventions

Curriculum.

Eliminating the Stigma of Differences is a three-module, three-hour curriculum delivered by teachers over a three- to six-day period. PowerPoint slides provide a platform for classroom discussion. The teacher’s guide provides suggestions for questions to pose and information to convey to the class, as well as suggested in-class exercises and homework assignments. A demonstration video is also included. All materials were extensively pretested. Module 1 addresses the bases on which we judge others to be different; the definition, causes, and consequences of stigma, including for students themselves; ways to end stigma; a definition and description of mental illness; causes of mental illness; treatment for mental illness; barriers to help seeking; how stigma applies to mental illness; and sharing personal experiences with people who have mental illness. Modules 2 and 3 address attention-deficit hyperactivity disorder, anxiety disorders, depression, schizophrenia, and bipolar disorder and include descriptions of the disorders, discussion of causes and treatments, and content that stimulates empathy. Suicide is also discussed. The curriculum employs principles of active learning and the encouragement of empathy throughout.

Contact.

Two college students—a 27-year-old male with a history of bipolar I disorder and a 24-year-old female with a history of bipolar II disorder—each made a ten-minute in-class presentation (20 minutes total) describing onset and course of their symptoms, hospitalizations and treatments, their feelings about the illness, coping strategies, and impact of the illness on social relationships and functioning at school and work. Based on previous research (48), the talks were constructed to moderately disconfirm stereotypes of mental illness. The speakers practiced to ensure standardization of the presentations. Teachers moderated the presentations, which were followed by questions and answers.

Printed materials.

Teachers prominently displayed posters in the classroom for two weeks and provided students with bookmarks. The materials focused on individuals’ personal traits and abilities as opposed to language that labels a person as “mentally ill.” The curriculum and printed materials are accessible through the Web site of Mental Health Connection of Tarrant County (http://www.mentalhealthconnection.org/anti_stigma_materials.php).

Outcome Measures

Our primary goals were to reduce stigmatizing attitudes, beliefs, behaviors, and behavioral intentions and increase recognition of mental illnesses and favorable attitudes toward help seeking. We assembled a comprehensive assessment package utilizing existing measures with established psychometric properties for children and adolescents, measures extensively tested in adults that we adapted for adolescents, and new items developed for the study. We developed composite scales by using exploratory factor analysis. Internal consistency reliability of the scales was adequate to excellent for the overall sample and within gender, race-ethnicity, and socioeconomic-status groups. All measures were pilot-tested with a racially and ethnically diverse group of youths in the target age range. Measures were organized in terms of knowledge and attitudes, behavior and behavioral intentions, personal help-seeking attitudes, and vignette-based questions. Following a long tradition in stigma research (14,15,49), we created two vignettes. One vignette character (Julia) met DSM-IV criteria for bipolar disorder, and the other (David) met criteria for social anxiety disorder. Participants read the vignettes and responded to questions about Julia and David. Table 3 summarizes information about the outcome measures. [Tables in an online supplement to this article present wording of items, responses, vignettes, and pretest descriptive statistics for all items.]

Other Variables

Personal contact with mental illness was assessed by a method used previously (32). Scores indicated the most intimate level of contact reported, from 0 (“I have never observed a person with mental illness”) to 7 (“I have a severe mental illness”). Social desirability bias was examined with a reliable scale for research on children (50,51). Fidelity to the curriculum content, quality of delivery, and level of student engagement were assessed by two observers in each classroom (intraclass correlation=.93) with a 60-item measure (available on request) based on two existing tools with good psychometric properties (52,53).

Statistical Analysis

Given the experimental design, personal characteristics could not cause self-selection into intervention groups. Nevertheless, it was possible for groups to differ at baseline because of imperfect randomization or differential participation. Significant pretest differences were found between intervention and control groups on race-ethnicity, primary language spoken at home, parent or guardian’s education and income, and level of contact with mental illness. We included pretest values of the corresponding outcome measures in the main analyses to control for preintervention group differences. We also reran our main analyses including the personal characteristics that differed at baseline and social desirability bias to determine whether they were significantly related to the outcomes after control for pretest values; they were not. Therefore, final analyses controlled only for pretest values of the outcomes.

Using analysis of covariance, we tested the effects of each intervention on the aforementioned outcomes. Across all analyses, outcomes for the printed-materials-only group did not differ significantly from the no-intervention control group. Combining the printed-materials-only group with the curriculum or contact intervention did not change the outcomes for the curriculum or contact group alone. Therefore, we combined the printed-materials-only group with the control group. Those receiving materials plus contact were grouped with the contact-only group and so forth. Our analyses thus compared outcomes for four groupings: curriculum only, contact only, curriculum plus contact, and control. The level of significance was set at .05.

Results

Fidelity to Curriculum Intervention

The mean fidelity score was 187. Possible scores range from 60 to 240, with scores of 148 to 192 considered “good” and scores of 193 to 240 “high fidelity.” We found no evidence of a linear effect of fidelity on the outcome measures, which we attributed to generally good fidelity and lack of substantial variance on the measure.

Outcome Analyses

Aside from correlations between the overall measure of social distance and its two subscales, there were no extremely high correlations between outcome measures, indicating that measures represented distinct aspects of stigma [see table in online supplement]. The largest correlations were between knowledge and attitudes and social distance measures (–.56 to –.61) and between general and vignette-based social distance (.57 to .61). Other correlations ranged from nil to .40.

As shown in Table 4, intervention groups significantly differed from each other for ten of the 13 primary outcomes. In regard to the impact of specific interventions, we noted consistent patterns. The curriculum fairly consistently increased recognition of mental illness and positive orientations to help seeking, including stigma awareness and action, identification of the vignette characters as mentally ill rather than bad, endorsement of help seeking for the vignette characters and for oneself in a similar situation, and increased optimism about treatment effectiveness. The impact of the curriculum on social distance outcomes was weaker. Measures of avoidance and discomfort and social distance from the vignette characters and from “someone with a mental illness” did not differ significantly from those of the control group, although a subscale of the more acceptable forms of contact (be friends, be neighbors, or eat lunch together) showed significant improvement.

The impact of contact was more limited. In the ten instances in which the overall effect of intervention was significant, the contact-only group differed significantly from the control group in only three instances—the belief that both vignette characters have a mental illness and that they should seek help. Adjusted means for curriculum only and curriculum plus contact did not differ significantly from each other, suggesting that in general contact did not add to the effect of the curriculum.

We also assessed interactions of gender, family income, race-ethnicity, and primary language spoken at home with the intervention. Of the 52 interactions tested, only five were significant (p<.05), and these failed to show any consistent pattern across outcomes (one for gender, one for income, one for race-ethnicity, and two for language).

Discussion and Conclusions

Stigma surrounding mental illness has proved difficult to change in the population. One promising approach may be to attack stigma at a young age, before negative attitudes become entrenched. However, research on interventions with preadolescents is limited. We conducted a school-based field experiment with sixth-grade students that evaluated an antistigma curriculum, an in-vivo contact intervention, and a social marketing strategy that saturated classrooms with informative messages. We evaluated the impact of the interventions on knowledge and attitudes about mental illness, behavior and behavioral intentions related to mental illness, and help-seeking attitudes.

Unfortunately, the simplest intervention to apply—saturating classrooms with antistigma written materials—had no significant impact on any outcomes. The contact intervention was less effective than the curriculum. The impact of the contact intervention was limited to increasing the tendency to identify the vignette conditions as mental illnesses that should be treated. This is at odds with studies of adults, which generally find that contact interventions are more effective than educational ones in reducing stigma (36). However, it is consistent with Corrigan and colleagues’ (36) meta-analysis, which showed that the opposite is true with adolescents. We located only one previous study with preadolescents that used a contact intervention (46). Although that study found reduced stigma after exposure to a lecture combined with personal contact, the predicted effects of varying the degree of stereotype disconfirmation embodied in the contact intervention were not observed. Also, because contact was combined in every case with a lecture, it is not clear whether stigma reduction resulted from contact. Although enthusiasm for contact interventions is justifiably strong, further evaluation of their effectiveness with young audiences is needed.

The curriculum was the most effective of the three interventions. It delivered a consistently beneficial impact on a diverse set of outcomes, tapping recognition of mental illness, awareness and action related to stigma, personal inclinations to seek help, and more favorable orientations toward the vignette characters and their prospects for improvement. However, the curriculum group did not differ significantly from the control group on avoidance and discomfort, the belief that David (with social anxiety disorder) is a bad person, and most social distance outcomes. The first two null findings may have been influenced by methodological factors: reported instances of avoidance and discomfort, assessed over a two-week period, as well as the belief that David is a bad person were very low for all experimental groups pre- and posttest. Our curriculum was less successful at reducing social distance than two previous studies that used preadolescent samples (42,45). It is not clear what accounts for the discrepant findings.

Some limitations should be noted. With any experimental study, recruitment bias can be problematic. Sixty percent of students invited decided to participate in our study. Some nonparticipating students or their families may hold particularly negative views about mental illness, possibly limiting the generalizability of our findings. For non–study-related reasons, two of the originally selected schools dropped out, which led us to return to five schools the following school year. We found no significant differences in participant characteristics between repeated and nonrepeated classrooms. Nevertheless, it is possible that the two lost classrooms differed from the remaining classrooms in terms of participant characteristics or response to the interventions. The results are also limited in that they include no long-term follow-up and rely on hypothetical rather than actual help seeking.

Our study makes several advances over previous ones, allowing greater confidence in the validity and generalizability of findings. There was no self-selection of teachers into the study or particular intervention conditions. Our fidelity measure, inclusion of which was another advance in the literature, showed that fidelity was generally high without self-selection or teacher training, suggesting that neither voluntary involvement nor previous training is necessary for high-quality implementation. The assessment of multiple interventions allowed us to attribute effects to particular interventions rather than to intervention per se and led us to conclude that the curriculum intervention was superior to the contact and printed-materials interventions. Our sample had excellent representation of the major U.S. racial and ethnic groups; thus we were able to show that interventions had a similar impact among Latino, African American, and non-Hispanic white youths. These study features raise optimism that stigma reduction interventions can generalize to a broad application in the population and strengthen existing evidence that even brief interventions can reduce stigma and may be transferrable to real-world applications. Future steps should include a direct comparison of curricula employed in the different intervention studies, perhaps combining elements that appear to most effectively address various components of stigma, and working with educators to begin to establish stigma reduction as part of the regular school curriculum.