Self-Esteem as an Outcome Measure in Studies of Vocational Rehabilitation for Adults With Severe Mental Illness

The concept of self-esteem is widely used in popular culture (1) and in social science research (2). Self-esteem has been defined in a variety of ways, all of which address the "evaluation of one's own worth, value, or importance" (1). Silber and Tippitt's definition (3), for example, states that self-esteem is "the feelings of satisfaction or dissatisfaction about the self which reflect the degree of congruence between a person's self-image and his own ideal self-image."

Self-esteem is assumed to be responsive to life events and treatment interventions (2). Life changes that bring a person closer to his or her ideal are thought to increase self-esteem. Such reasoning is the basis for using measures of self-esteem in vocational rehabilitation studies of people with severe mental illness. Work is highly valued in our general culture (4), and a large majority of people with persistent mental illnesses indicate that they desire competitive employment (5). Thus the hypothesis is that employment brings the self-image of a person with mental illness closer to his or her ideal self-image, resulting in higher self-esteem.

The widely held belief that the self-esteem of adults with severe mental disorders is closely linked to their employment status has not received strong and consistent research support, however. In a population of people with severe mental illness, Van Dongen (6) found that those who were competitively employed had higher self-esteem than those who were unemployed. In contrast, Arns and Linney (7) found that work status was not related to self-esteem. They did find that change in work status over six months was significantly related to self-esteem at the end of the time period, but they could not evaluate improvement in self-esteem as they had no rating of baseline self-esteem. Our group found an inconsistent relationship between work status and self-esteem (8) in the data we reexamine in more detail in this paper. We also reported that baseline self-esteem did not predict work later in the study (8). Two other recent investigations found no significant relationship between change in vocational status and change in self-esteem (9,10).

Why doesn't the research provide consistent, strong support for the impressions of clinicians and theorists? One possibility is that work does not have a powerful impact on self-esteem among adults with severe mental illness. Another is that the instrument being used to assess self-esteem is not effective at measuring changes that may take place. The most common measure of global self-esteem is the Rosenberg Self-Esteem Scale (11). Originally designed to measure overall feelings of self-worth in adolescents, the Rosenberg scale has been widely used in many populations (1,12). Little evidence exists, however, to support its sensitivity to change (1,12).

Although the Rosenberg Self-Esteem Scale is frequently used in studies of the treatment and rehabilitation of adults with severe mental illness (6,7,13,14,15), little has been written about its properties in this population. A few reports document its ease of use, face validity, good internal consistency, and test-retest reliability (6,7). One study found that the participants with a diagnosis of depression had significantly lower self-esteem than those with a diagnosis of anxiety disorders or psychotic illnesses (15), but otherwise the literature does not address the question of the validity of the Rosenberg Scale among adults with severe mental illness. The instrument's stability over time has not been examined.

The study reported here used data from an 18-month vocational rehabilitation study to examine self-esteem, as measured with the Rosenberg Self-Esteem Scale, among adults with severe mental illness. To address the overall question of whether global self-esteem should be included as an outcome variable in future vocational rehabilitation studies, the study examined the properties of the Rosenberg Self-Esteem Scale in this population. Four main questions were addressed. First, what are the instrument's psychometric properties? Second, how stable are its measurements over time? Third, how strongly are its measurements linked to desired rehabilitation outcomes? And fourth, how are its measurements correlated with other study variables, such as demographic, diagnostic, symptom, and functional status variables?

Methods

Participants

The study sample consisted of 143 persons with severe mental disorders living in two New Hampshire cities with populations of 119,000 and 166,000. The study participants were involved in a randomized controlled trial of two models of vocational rehabilitation during the period from January 1991 to January 1994. Persons were eligible for inclusion in the study if they were between 20 and 65 years old and not currently working, had an interest in competitive employment, had no serious medical condition that might affect long-term ability to participate in vocational programs, were willing to provide consent to participate in an evaluation of vocational programs, and attended four research induction sessions in which the purposes and requirements of the project were discussed (16). Diagnoses were confirmed by our research team using the Structured Clinical Interview for DSM-III-R (17).

A total of 48.3 percent of the study participants were men, and 95.1 percent were white; 9.8 percent were married, 50.3 percent had never married, and 39.9 percent were separated, divorced, or widowed. As for education, 74.2 percent of the sample had completed high school or had obtained a general equivalency diploma.

A total of 25.2 percent of the sample were between 20 and 30 years old, 36.4 percent were between 30 and 40 years old, and 38.4 percent were older than 40 years. The primary diagnoses of study participants were chronic psychotic illnesses for 46.9 percent and severe affective disorders for 42.7 percent; 10.4 percent of the participants had other disorders, primarily severe personality disorders.

Vocational programs

Data for the analyses reported here were collected in a randomized controlled trial of two models of vocational rehabilitation for persons with severe mental disorders: individual placement and support, an integrated supported employment approach, and group skills training, a model based on training in prevocational skills related to selecting, finding, and keeping a job. At baseline, participants were randomly assigned to one of the programs, in which they remained for one and a half years.

Both programs improved rates of competitive employment, with individual placement and support resulting in significantly higher rates of job attainment and more total hours of competitive employment than group skills training. During the study, about 60 percent of the participants obtained competitive employment, defined as a part-time or full-time job in an integrated work setting that paid competitive wages, and approximately 30 percent were working during any month (18). Because there was no reason to suspect that group membership would affect the relationship between self-esteem and other study variables, we combined the two groups for the present analyses.

Measures

Research assessments, which were conducted by a research interviewer at baseline, and six, 12, and 18 months later, included measures of symptom severity, life satisfaction, housing status, income, work status, and self-esteem.

Self-esteem was measured with the Rosenberg Self-Esteem Scale (11), which includes ten items. Each was rated on a 4-point scale, with total scores ranging from 10 (highest self-esteem) to 40 (lowest self-esteem). In other populations of adults with severe mental illness, internal consistencies of .82 (7) and .87 (6), as assessed by coefficient alpha, were reported for the instrument. In populations of college students, reports have indicated a one-week test-retest correlation of .82 (19) and a two-week test-retest correlation of .85 (3). In a population of adults with severe mental illness, a two-week test-retest correlation of .85 has been reported (6).

Psychiatric symptoms were assessed for the previous two-week period using the expanded version of the Brief Psychiatric Rating Scale (BPRS) (20), which was scored as five subscales: thought disorder, affect, activation, disorganization, and anergia. Test-retest reliability of the symptom subscales, evaluated by computing intraclass correlation coefficients for data on 41 participants (29 percent of the sample) over a one- to two-week period, were high, ranging from .59 for activation to .93 for thought disorder. Part of Lehman's Quality of Life Interview (QOLI) (21), a measure developed for and widely used in this population, was used to rate overall life satisfaction.

Employment, income, and housing variables were chosen for these analyses because of their potential impact on self-esteem. Current work status was reported as a dichotomous variable based on self-reported paid, competitive employment at the time of each follow-up interview. Total numbers of hours and weeks worked over the course of the study were also tracked using weekly reporting forms from vocational specialists. Total income was calculated for each time period by adding up self-reported income from work, gifts, and benefits. Housing status was reported as a dichotomous variable based on whether or not the participant reported living in independent housing at each assessment point.

Results

We first examined the psychometric properties of the Rosenberg Self-Esteem Scale and then assessed its stability over time. We then evaluated the associations between self-esteem and work. Last, we examined the relationship between self-esteem and other variables that were measured in the study, including demographic characteristics, diagnosis, life satisfaction, BPRS symptom subscale scores, income, job status, and housing status.

Psychometric properties

In this sample, coefficient alphas were high for the Rosenberg Self-Esteem Scale at each of the four time points: .87, .88, .90, and .87, indicating high internal reliability. Test-retest reliability, evaluated by computing intraclass correlation coefficients on data for 41 participants (29 percent) over a two-week period, was .87. The mean±SD scores on the self-esteem scale at baseline, six months, 12 months and, 18 months were 22.8±5.6, 21.6±5.7, 21.8±6, and 22.4±5.8, respectively. Scores ranged at each assessment point from 10 (the highest possible self-esteem score) to the high 30s (near the lowest possible self-esteem score).

Self-esteem over time

To evaluate the longer-term stability of self-esteem, correlations over the course of the study were explored. Self-esteem ratings, expressed as Pearson r, were highly correlated over adjacent assessment points: r=.71 for scores at baseline and six months, r=.72 for scores at six and 12 months, and r=.70 for scores at 12 months and 18 months. The correlation was lower over an 18-month interval: r=.61 for scores at baseline and 18 months.

As noted above, mean scores on the Rosenberg Self-Esteem Scale improved approximately 1 point on the 31-point scale during the first six months of the study, and then drifted back toward the baseline score at 12 and 18 months. Paired t tests indicated that the change from baseline to six months was significant (t=3.49, df=124, p<.01), as was the change from baseline to 12 months (t=2.24, df=128, p<.05), but not the change from baseline to 18 months.

Self-esteem and work

We used two approaches to measuring work in order to evaluate the relationship between self-esteem and work. The first approach examined work status—working versus not working—at the time of each follow-up interview. No participant was working at baseline. At each follow-up assessment point, participants who were currently competitively employed had lower scores on the Rosenberg Self-Esteem Scale—meaning better self-esteem—than unemployed participants, but the differences were not statistically significant.

As self-esteem may be more related to the cumulative experience of working than to current working status, the second approach measured the amount of work experience gained by the participants over the course of the study. A mixed-effects regression model revealed no significant relationship between the hours worked and self-esteem scores at the end of each six-month period.

So analyses could look at cumulative work experience a different way, participants were divided into three groups based on the number of weeks they worked in paid competitive jobs over the course of the study: no work (N=56), low amount of work (one to 19 weeks; N=40), and high amount of work (20 or more weeks; N=47). The break point between the low-work group and the high-work group was based on a natural break near the mean in the distribution of weeks worked. Paired t tests comparing self-esteem at baseline and 18 months showed no significant difference over time for the group that never worked and the group that worked a low amount. However, the self-esteem of the group that worked a high amount improved significantly (t=2.07, df= 40, p<.05). With Bonferroni correction for multiple tests, this finding would become nonsignificant.

Self-esteem, symptoms,and other variables

Next, the relationship between self-esteem and other study variables was examined by computing Pearson correlations between the measures. The other study variables included demographic characteristics, diagnosis, symptom subscales, life satisfaction, housing status, and income. These results are shown in Table 1. Although a variety of different measures were significantly correlated with self-esteem at different time periods, scores on the BPRS affect subscale—a variable made up of the BPRS items measuring depression, suicidality, anxiety, guilt, and hostility—and QOLI overall life satisfaction scores were most strongly and consistently correlated with self-esteem. Lower scores on the BPRS affective symptom subscale and higher satisfaction with life correlated with better self-esteem.

The BPRS affect subscale and life satisfaction scores were strongly correlated not only with the Rosenberg Self-Esteem Scale scores but with each other. For example, the Pearson correlation between BPRS affect and life satisfaction at the 18-month assessment point was −.62 (p<.001).

Partial correlations were calculated on the 18-month data to evaluate the relationships between the three variables more fully. When the analysis controlled for BPRS affect subscale scores, the correlation between the self-esteem scores and life satisfaction scores was −.47 (p<.001). Controlling for life satisfaction, the correlation between the self-esteem scores and BPRS affect was .34 (p<.001). Controlling for scores on the self-esteem scale, the correlation between BPRS affect and life satisfaction was −.35. Using the bivariate and partial correlations, we determined that all three variables shared about 25 percent of their variance.

Discussion

Global self-esteem of adults with severe mental illness can be practically measured using the Rosenberg Self-Esteem Scale. The scale demonstrates internal consistency and good test-retest reliability. In this 18-month study, scores on the Rosenberg Self-Esteem Scale changed little over time. Even the statistically significant finding that the sample's mean self-esteem improved from baseline to the six- and 12-month follow-ups cannot be considered clinically meaningful, as it represented a change of approximately one point in the midrange of a scale with a 30-point range. The stability of the scores over time supports an understanding of global self-esteem as a relatively stable personality trait, similar to happiness (22).

Self-esteem is measured in rehabilitation studies because clinicians and theorists have long believed that self-esteem is closely linked to positive rehabilitation outcomes for persons with severe mental illness (23). Our data are in accord with previous research showing, at most, a weak association between work and self-esteem. Further, a weak relationship was found longitudinally only for the group who worked a substantial length of time. Self-esteem also was not related to other objective measures of functional status, housing, and income.

Several factors may explain the lack of research support for the hypothesis that a strong direct relationship exists between self-esteem and desired rehabilitation outcomes. One explanation is that the trait-like nature of global self-esteem makes it insensitive to life changes. Another is that self-esteem is too distant from rehabilitation-induced functional improvement to register a change: even when rehabilitation leads to a strongly desired result in an important functional domain, such as work, so much else is going on in people's lives—such as interpersonal relationships, medication side effects, and symptoms—that the positive change may not be reflected in secondary measures, such as self-esteem, which are not specifically focused on the improved life domain.

Unlike functional status measures, variables that measure affective state are tightly linked to self-esteem. Despite approaching people's affect from substantially different angles, the BPRS affective symptom subscale, the QOLI life satisfaction measure, and the Rosenberg Self-Esteem Scale all overlap substantially. These findings are consistent with previous research showing that global self-evaluation is strongly influenced by mood. Self-esteem and measures of depression overlap significantly (24), and low self-esteem is a common characteristic of depression (15,25,26). Clearly, people's affective self-evaluation is strongly related to their general affective state. Because self-esteem is a more complex concept than affect, future researchers who obtain significant findings involving self-esteem should evaluate whether scales that directly measure affect can explain their findings more simply.

Conclusions

Global self-esteem, as measured by the Rosenberg Self-Esteem Scale, appears to be a relatively stable trait that reflects general life satisfaction and affective symptoms rather than objective functional status.

Acknowledgments

This work was supported by grants MH-00839 and MH-47650 from the National Institute of Mental Health, by grant SM5-1818 from the Center for Mental Health Services, and the New Hampshire Divisions of Mental Health and Vocational Rehabilitation.

Dr. Torrey is assistant professor and Dr. Mueser and Dr. Drake are professors in the department of psychiatry at Dartmouth Medical School in Hanover, New Hampshire. Dr. Mueser is also professor and Dr. McHugo is research associate professor in the department of community and family medicine at the medical school. Address correspondence to Dr. Torrey at the New Hampshire-Dartmouth Psychiatric Research Center, 2 Whipple Place, Lebanon, New Hampshire 03301 (e-mail, [email protected]).

Enlarge table