Development and Validation of a Scale Assessing Mental Health Clinicians’ Experiences of Associative Stigma

In his landmark description of the stigma process, Goffman (1) asserted that it is typical for persons who are “related through the social structure to a stigmatized individual” to “share some of the discredit of the stigmatized person to whom they are related.” Given the extent to which stigma pervades the social interactions of people diagnosed as having serious mental illnesses such as schizophrenia and bipolar disorder (2), it is plausible that individuals whose lives are intertwined with those who are diagnosed could also be affected by stigma. Considerable research has confirmed that forms of “associative” stigma are commonly experienced by the family members of people with serious mental illnesses, who report experiencing reduced “status” (3–5), demeaning social interactions with community members (6), and social avoidance by community members (7,8).

Mental health professionals, whose professional identity is intertwined with those whom they serve, might also be affected by associative stigma. Surveys of medical school faculty have documented that psychiatry has diminished status relative to other medical disciplines (9) and that a substantial proportion of psychiatrists perceive that the public has negative impressions of their discipline (10). A study of general community members found that they endorsed a number of stereotypes about mental health professionals (11). A study involving a variety of mental health professionals also documented that most endorsed hearing community members “making jokes” about their work (12) and found that experiences of associative stigma predicted greater emotional exhaustion (a component of burnout) and less job satisfaction. A limitation of this study was that the authors did not use a validated measure of associative stigma but instead examined whether participants responded affirmatively to one of four questions.

We previously conducted a qualitative study of the associative stigma experiences of mental health clinicians who work with people with serious mental illnesses (Vayshenker B, Deluca J, Bustle T, et al., unpublished data, 2017). Major themes included a reluctance to discuss one’s work with others outside the field, the portrayal of mental health professionals as unethical and experiencing their own psychological problems, and the view that work with people with serious mental illnesses could be done by anyone but that no one would want to do it if they had the choice. Based on our prior qualitative findings, and given the potential connection between associative stigma and burnout (12) and evidence that burnout predicts poorer quality of care (13), we believed that it would be valuable to develop a scale to assess associative stigma.

The purpose of this study was to develop a scale to assess associative stigma among clinicians who work with people with serious mental illnesses and to examine its factor structure and convergent and discriminant validity. We developed a scale and administered it to mental health professionals in the United States in fields that typically work with people with serious mental illnesses (social work, mental health counseling, and clinical psychology) to determine its psychometric properties.

Methods

Development of Initial Item Pool

We followed an established process to develop the scale (14). First, we developed an initial pool of 26 items on the basis of identified categories and themes from our previous qualitative study (Vayshenker B, Deluca J, Bustle T, et al., unpublished data, 2017). After development of the preliminary item pool, we presented a draft of the scale to seven persons with experience working with people with serious mental illnesses, who provided qualitative input and feedback on the clarity of items, identified possible inappropriate items, and suggested additional items. Revisions to the draft scale were made, and the scale was reduced to a total of 19 items for inclusion in the validation study.

Participants

A total of 473 individuals consented to participate in the online validation study, and 472 completed the survey. Participants were recruited through e-mail lists of U.S. professional associations whose members typically work with people with serious mental illnesses, including social workers, psychologists, and mental health counselors. Inclusion criteria consisted of working with adults with serious mental illnesses and fluency in English.

Procedure

Institutional review board approval was obtained from the City University of New York and the National Association of Social Workers (NASW). Data were collected during July and August 2016. We sent one e-mail invitation to 10,913 members of the NASW electronic mailing list who were listed as working in “inpatient and outpatient” mental health settings. Data indicated that 88% of the e-mails were delivered successfully, with an open rate of 25.4% and a link click rate of 4.4%. Thus, of the roughly 2,500 persons who opened the e-mail invitation, 480, or roughly 19%, clicked the survey link. Of these individuals, we estimate that 332, or roughly 13% of those who opened the e-mail, completed the survey.

Similarly, we sent another e-mail invitation to 3,349 members of the American Mental Health Counselors Association (AMHCA) electronic mailing list who were listed as performing clinical work; 87% of the e-mails were delivered successfully, with an open rate of 32.3% and a link click rate of 5%. Thus, of the roughly 1,070 persons who opened the e-mail invitation, approximately 167, or 16%, clicked the survey link. Of these individuals, 92, or roughly 9% of those who opened the e-mail, completed the survey.

We also submitted the survey invitation to the electronic mailing list of the American Psychological Association’s (APA) Society of Clinical Psychology, which has 1,443 subscribers. Information on the number of participants who received the e-mail and opened the survey link was not available for this recruitment platform; however, we estimated that 49, or 3% of potential e-mail recipients, were recruited through this source.

It was not possible to determine how many eligible individuals (that is, those who were currently working with adults with serious mental illnesses) elected to participate in the study. For NASW and AMHCA, surveys were distributed by a third-party marketing company that works with the professional agencies. For the APA, the survey was distributed through the e-mail list with approval from the administrator.

Prospective participants received a short message with an explanation of the study purpose and a link to the survey. Individuals who clicked on the study link were then presented with a consent document that participants needed to approve before proceeding. Participation was completely anonymous.

Measures

We selected other scales to evaluate convergent and discriminant validity properties with the scale being developed—the Clinician Associative Stigma Scale (CASS). Because no prior measure of clinicians’ associative stigma exists, we selected measures of constructs that previous research suggested might be related to associative stigma (12). We predicted that the CASS would show statistically significant relationships with measures of burnout and quality of care (convergent validity) and no statistically significant relationships with a measure of general self-efficacy or provider expectations of recovery (discriminant validity). Demographic and work-related information was also obtained from all participants.

CASS.

The original version of this scale included 19 items. Participants were instructed to think specifically about their work with adults with serious mental illnesses (including those with diagnoses such as schizophrenia and bipolar disorder) and to report the frequency of experiences on the following scale: 1, if the experience had never occurred; rarely, 2, if it had occurred only once or twice; sometimes, 3, if it had occurred repeatedly but irregularly; and often, 4, if it occurred regularly. Possible scores range from 19 to 76, with higher scores reflecting higher levels of associative stigma.

Oldenburg Burnout Inventory.

The Oldenburg Burnout Inventory (OBI) is a 16-item self-report measure designed to evaluate burnout in human services professions (15). Participants rate their agreement with each item on a scale from 1, strongly agree, to 4, strongly disagree. Possible scores range from 4 to 64, with higher scores indicating higher levels of burnout. The OBI measures two components of burnout: exhaustion resulting from physical, cognitive, and emotional job-related strain; and disengagement from work. It has been found to correlate highly with the Maslach Burnout Inventory, a widely known measure of burnout (16). In this study, we found good internal consistency estimates for the total scale (α=.87) and the exhaustion subscale (α=.87) and adequate levels for the disengagement subscale (α=.75).

Quality of Care Scale.

The Quality of Care Scale (QoCS) is a 17-item self-report measure that assesses behaviors and attitudes related to clinicians’ quality of care (13). It has three subscales, including client-centered care, general work conscientiousness, and low errors. Participants rate the frequency of each experience occurring within the past month from 1, never, to 5, very often. Possible scores range from 17 to 85, with higher scores reflecting better levels of self-reported quality of care. We found adequate levels of internal consistency in this study (α=.69); subscale alphas ranged from .63 to .78.

Provider Expectations for Recovery Scale.

The Provider Expectations for Recovery Scale (PERS), a ten-item self-report scale, examines clinicians’ attitudes and expectations of recovery among the clients whom they serve (17). Participants respond to statements on a scale ranging from 1, almost all of my consumers, to 5, none of my consumers. We recoded the items such that higher values reflected higher levels of recovery orientation. Possible scores range from 10 to 50. We found excellent internal consistency for this scale (α=.91).

General Self-Efficacy Scale.

The General Self-Efficacy Scale (GSES) is a ten-item self-report measure designed to evaluate perceived self-efficacy (18). The items are averaged to form a composite score, ranging from 10 to 40. In our study, good internal consistency estimates were observed (α=.86).

Analyses

We first examined descriptive information on responses to the 19 CASS items. We used expectation maximization in instances in which individual items were missing at random so as to minimize missing data (responses were excluded if more than half of items were missing).

Next, we conducted an exploratory factor analysis to examine whether responses to scale items fit with the associative stigma construct. The Kaiser-Meyer-Olkin measure of sampling adequacy was .84, above the recommended value of .5 (19), and Bartlett’s test of sphericity was significant (χ²=3,219.79, df=171, p<.01); thus factor analysis was deemed appropriate. After an examination of the scree plot, it was decided that a four-factor solution would be most appropriate. Principal-components analysis with a varimax rotation and Kaiser normalization was used to determine the underlying structure of the data. Using a cutoff factor loading of .4 (19), 18 items loading on four factors were retained (almost all items loaded above the .5 level with a single factor).

We next examined correlations between the CASS, OBI, QoCS, PERS, and GSES. Finally, we conducted exploratory analyses of the demographic correlates of CASS scores.

Results

Characteristics of participants are reported in Tables 1 and 2. Overall, participants were mainly European American and female. Over three-quarters of the sample reported having a master’s degree, and most respondents worked in outpatient settings. Social workers and mental health counselors each constituted approximately one-third of the sample. Participants had typically worked in the field for more than 15 years and at their current place of employment for nearly eight years, and they reported spending most of their time in direct service. Participants came from throughout the United States, with roughly a third from the Northeast, a quarter from the South, and the remainder from the West and Midwest. The modal work setting was “community mental health services” (over a third of participants); the second most common work setting was “private practice.” The remaining participants worked in inpatient, university-based, and other settings. The demographic characteristics of participants were similar to recent figures for members of mental health disciplines in the United States overall, which indicate that counselors, social workers, and psychologists are predominantly European American and female (20).

Table 3 summarizes data on responses to each CASS item. There was a range of endorsement of associative stigma experiences among participants, from roughly 10% endorsing the item “I have heard other people say that the work I do is useless” to over 85% endorsing the item “When people find out that I work with individuals with serious mental illness, they tell me they could never do that type of work.” The mean±SD overall score for the 19-item scale was 44.8±9.0 (range 22–77), indicating that, on average, participants scored above the midpoint of 38 for the scale and tended to endorse having associative stigma experiences “sometimes.”

Table 4 presents findings from the factor analysis. A four-factor solution was supported, and the factors were named “negative stereotypes about professional effectiveness” (six items, α=.77), “discomfort with disclosure” (four items, α=.84), “negative stereotypes about people with mental illness” (four items, α=.76), and “stereotypes about professionals’ mental health,” (four items, α=.68). These factors fit well with the themes previously discussed in our qualitative study (Vayshenker B, Deluca J, Bustle T, et al., unpublished data, 2017). One item (“I have heard people state or joke that mental health professionals must be “crazy”) was ultimately excluded because it failed to load with any of the factors above the .4 level. The resulting 18-item scale had a mean total score of 42.3±8.5 (range 21–72) and showed good internal consistency (α=.85). Scores indicated that, on average, participants scored above the midpoint of 36 for the scale and tended to endorse having associative stigma experiences at least “sometimes” (specifically, eight of the retained items were endorsed as being experienced at least “sometimes” by over 50% of the sample).

We next examined correlations between the revised CASS and the subscales of the OBI, QoCS, PERS, and GSES (Table 5). We used expectation maximization to replace missing values for all scale scores, but we were unable to use the expectation maximization approach for the QoCS, where several items were missing in a nonrandom manner because they were listed as “not applicable” by nearly a quarter of the sample. Therefore, we used mean imputation to minimize missing data for this scale. The CASS was moderately but significantly positively associated with the two subscales of the OBI, with a stronger association with the “emotional exhaustion” subscale. This indicates that, as hypothesized, participants who endorsed experiencing more associative stigma also tended to endorse feeling more emotionally exhausted about their work. The CASS was also weakly but significantly negatively associated with the general work conscientiousness and low-errors subscales of the QoCS but not with the client-centered care subscale. This indicates that, as hypothesized, participants who endorsed experiencing more associative stigma also tended to endorse less conscientiousness in their work and more errors; however, associative stigma experiences were not related to client-centered care. Consistent with hypotheses, no significant relationships were found between the CASS and both PERS and GSES.

We then conducted an exploration of the demographic and work correlates of CASS scores (data not shown in tables). These analyses indicated weak but significant relationships between CASS score and age (r=–.18, p<.01), female gender (r=.17, p<.01), and living in the South or West (versus the Northeast and Midwest) (r=.11, p<.01). However, no association was found between CASS score and percentage of time spent in direct care or private practice as work setting (versus all others).

Discussion and Conclusions

Overall, results from a validation study of the CASS with a national sample of service providers demonstrated evidence of internal consistency, a clear four-factor structure that aligned with prior qualitative research, and support for convergent and discriminant validity. Findings suggest that many experiences of associative stigma are relatively common (eight of the 18 items on the final scale were endorsed as occurring at least sometimes by most of the sample). Many participants reported that they avoid discussing their work with persons outside the mental health system and endorsed frequently encountering negative stereotypes about both mental health professionals and mental health service recipients. Professionals who were younger and female and who lived in the South and West were modestly but significantly more likely to endorse associative stigma experiences. These findings may reflect the perception of greater vulnerability among younger female professionals that aligns with the commonly held negative stereotype of dangerousness of people with serious mental illnesses. Regional differences in perceptions of associative stigma may reflect regional differences in the attitudes of community members, because national surveys have found that community members in the West and South generally endorse more negative attitudes toward people with mental illnesses (21).

Some limitations of this study should be noted. Although the characteristics of the sample were similar to those of clinicians who provide services to people with serious mental illnesses in the United States, only a small proportion of those who received the invitation e-mail responded to the study, and for this reason it was impossible to determine the representativeness of the study sample. It is plausible that those who responded had an interest in the topic because of prior experiences, which would result in a bias toward those with more associative stigma experiences. Future research should attempt to obtain a more representative sample of clinicians working in particular settings to determine whether the incidence of associative stigma reported by the participants in this study is typical. Another limitation was the exclusion of psychiatrists. We initially wished to include psychiatrists in the study, but we were unable to obtain access to an appropriate e-mail list for recruitment. Future research should examine the experiences of this important group, which prior research suggests might experience considerable associative stigma (10). In addition, we did not perform a confirmatory factor analysis of the scale. Future investigations should use confirmatory factor analysis to examine whether the factor structure observed in this study is retained. These studies could also evaluate whether the internal consistency observed in this study remains or decreases.

Findings from this study suggest that associative stigma is related to experiences of burnout among mental health professionals (weakly with disengagement and moderately with emotional exhaustion). These findings suggest that associative stigma might facilitate the erosion of mental health professionals’ compassion toward people with serious mental illnesses, which may diminish their effectiveness in working with this group. This connection is certainly plausible, but such an interpretation should be made with caution because it is also plausible that burnout leads clinicians to perceive more associative stigma. Future research should use prospective methods to clarify this relationship.

In conclusion, this study found evidence that the CASS is a measure that will be useful in studying the prevalence and correlates of associative stigma among mental health professionals.