An RCT Evaluating the Effects of Skills Training and Medication Type on Work Outcomes Among Patients With Schizophrenia

Although supported employment helps people with schizophrenia and other psychiatric disorders obtain employment (1–3), data on job tenure among recipients of supported employment have been inconsistent. Average job tenures associated with supported employment have been reported to range from less than six months in a study with a one- to two-year follow-up (4) to over three years in a unique study with an extended five-year follow-up period (5).

One strategy to improve job tenure and to empower consumers to manage their own work challenges is to augment supported employment with more focused instruction on the generic work skills required to succeed across employment settings (6). The Workplace Fundamentals Module (WPFM) is a manualized, clinic-based behavioral group intervention designed to increase employment tenure by providing workers with the competencies needed to be successful in any job (7,8). Competencies include getting along with coworkers and managers and solving problems. The addition of WPFM to supported employment yielded mixed results for job tenure in prior research (7,9), but these investigations had significant methodological shortcomings.

This study evaluated the effects of adding WPFM to supported employment in a rigorous randomized trial for persons diagnosed as having schizophrenia or schizoaffective disorder. We hypothesized that participants who received individual placement and support (IPS) plus WPFM would have better employment outcomes—in particular, longer job tenure—compared with participants who received only IPS. In prior work, we found that differences in side effects of antipsychotic medications influenced the patients’ ability to benefit from nonpharmacological treatments (10). With this in mind, we also randomly assigned participants to risperidone or olanzapine and tested a secondary hypothesis that different medications would yield unique side-effect profiles that would have an impact on work outcomes. We anticipated that both medications might yield side effects that could have an impact on work outcomes; we did not predict any interactions between the skills training group and medication type.

Methods

Setting and Participants

We recruited 107 participants from two outpatient sites: a Dartmouth-affiliated community mental health center in Manchester, New Hampshire, and a conjoint U.S. Department of Veterans Affairs Greater Los Angeles Healthcare System/University of California, Los Angeles (UCLA), clinic. Participants were eligible for all services offered at the sites. Institutional review boards at each site approved and monitored the study; participants provided written informed consent. Data were collected from July 24, 2000, to March 31, 2007.

Eligibility criteria were ages 18 to 65, living in the community, currently unemployed but interested in competitive work, research diagnosis of schizophrenia or schizoaffective disorder verified by the Structured Clinical Interview (11), and having not been hospitalized for the past month. Exclusion criteria were organic brain disease, intellectual disability, substance dependence within the prior six months, or a chronic illness that would prevent employment.

Study Design

Phase 1: medication and job acquisition.

After obtaining psychiatric, social, and employment history, we randomly assigned participants to olanzapine or risperidone and to supported employment (IPS) alone (12) or to IPS plus WPFM. To test the hypothesis that IPS plus WPFM extends job tenure, the WPFM group began skills training upon being hired for their first job. All raters were blind to treatment condition. We assessed symptoms, health status, and medication side effects at baseline and at every three months thereafter. Symptoms were assessed with the Brief Psychiatric Rating Scale (BPRS) (13); health status by measuring body mass index (BMI), cholesterol, and glucose; and medication side effects by the Abnormal Involuntary Movement Scale (14), the Barnes Akathisia Scale (15), the Subjective Extrapyramidal Rating Scale (16), and the Modified Simpson-Angus Scale (17).

The risperidone dose range was 1–10 mg daily, with an initial target of 4 mg; the olanzapine dose range was 2.5–30.0 mg daily, with an initial target of 12.5 mg. We permitted concomitant use of antidepressants and mood stabilizers. Despite intensive efforts to maintain individuals on assigned medication unless clinical status or side effects made doing so infeasible, only 42 (39%) of the 107 participants remained on their assigned medication for one year after randomization (risperidone, N=20 of 55, 36%; olanzapine, N=22 of 52, 42%).

IPS followed the standard principles and procedures, as described by Becker and Drake (12). Employment specialists provided a full range of services and carried caseloads of approximately 15 to 18 participants.

Phase 2: work and social-functioning outcomes.

Participants were offered up to one year of IPS while they attempted to find a first job; 67 of the original 107 participants (63%) obtained competitive employment, permitting the study of the effect of WPFM on job tenure and other work outcomes among these individuals during phase 2. Participants who found a job were offered 24 additional months of IPS after beginning the job to permit time to test the job tenure hypothesis.

WPFM training was provided during twice-weekly, 90-minute small groups and used a format of seven sequential learning activities: skill introduction, videotaped demonstration, role-playing practice, resource problem solving, outcome problem solving, out-of-session practice, and problem-solving homework. These learning activities covered nine generic work skill areas, including benefits and costs of work, identifying specific job details, recognizing problematic job details, learning a seven-step method to solve problems, solving problems with symptoms and medication, coping with general medical issues and drug abuse, interacting with the supervisor to obtain feedback, understanding workplace culture and interactions on and off the job, and improving work motivation. The groups were structured, but there was ample opportunity to discuss the material, and participants were encouraged to provide personal details about how the material related to their work situation.

To accommodate various hiring dates, WPFM was offered on a revolving basis, so the teaching of the last module was followed immediately by the first module; in that way, participants could enter groups at the beginning of any skill area and complete the nine modules in approximately five months. Out-of-session assignments were used to support generalization of material. Make-up sessions were offered for absences. Also, if the WPFM facilitator was not able to cover all the material in a particular WPFM session, he or she alerted the IPS employment specialist about the missing material. In those instances, the IPS specialist covered the missed material in the next IPS session, which typically required extending the session somewhat.

Work outcomes, including job title, hours worked, and wages, were collected via weekly interviews. Research assistants rated social and role functioning at the time of job acquisition and every six months thereafter by using the Quality of Life Scale (QLS) (18), and they collected data on hospitalizations and number of WPFM classes attended.

IPS employment specialists held B.A. or M.A. degrees and either had extensive IPS experience or had participated in an intensive training program prior to working with study participants. To minimize contamination, employment specialists were randomly assigned to either IPS alone or IPS plus WPFM, and only the latter group received training on WPFM. WPFM facilitators had an M.A. or a Ph.D. degree in psychology and prior experience working with psychosis. WPFM employment specialists did not lead the WPFM groups but were encouraged to join clinic sessions and to use the WPFM materials in the IPS.

The Manchester site began with a strong IPS program, and the Los Angeles site received intensive on-site training before the trial began. Both teams met in person to learn WPFM together. Throughout the trial, the teams received consultation and problem-solving assistance by telephone during joint hour-long weekly sessions, which were conducted separately for staff of the IPS and IPS plus WPFM programs.

Assessor Training and Reliability on Symptom and Functioning Measures

Assessors, who were naïve to treatment condition, had graduate training in mental health, and participated in multiday, in-person training prior to the initiation of the study. BPRS interrater reliability was established by training all raters to the standards of the UCLA Diagnostic Assessment Laboratory (κ>.8) and was maintained through regular conference calls; reliability checks were conducted throughout the study via examination of videotaped study assessments. Consistency on the QLS was established through regular meetings of the assessors, both within and across sites, to discuss problematic items.

Fidelity to the Vocational Programs

In addition to participation, by condition, in weekly cross-site consultation phone calls, each site was evaluated yearly with the original 15-item IPS Fidelity Scale (19). Both sites scored above 65 (scale of 15 to 75 points) on all yearly reviews, indicating good implementation. For WPFM, consultants reviewed written records for each client weekly. Twenty-seven (10%) randomly selected WPFM session audiotapes were rated on the Therapist Fidelity Evaluation for Modules (20); 24 (89%) of the sessions had over 80% adherence on both of the subscales.

Data Analysis Plan

The primary hypotheses concerned the impact of the interventions on job tenure; secondary hypotheses addressed other work outcomes and the side-effect profiles of the two medications. Baseline comparisons of participants’ characteristics (chi-square tests for categorical variables and t tests for continuous measures) verified the success of randomization and checked for differences among participants. Because the majority of primary analyses examined the 39 participants in IPS only and the 28 participants in IPS plus WPFM who obtained jobs during phase 1, we also compared the demographic and clinical characteristics of these two groups at the point of obtaining a job. Finally, we used logistic regression to identify factors associated with differential dropout. Variables that showed significant differences or were associated with dropout were adjusted for in subsequent analyses.

We used Cox proportional-hazards models to identify variables that were related to both time to obtain a job and tenure of first job, and we used linear and logistic regression models adjusted for covariates to compare cumulative work outcomes between the intervention groups. For longitudinal measures, we used linear mixed-effects regression with group, time, and a group × time interaction to examine differential change in side effects between the medication arm during phase 1, when the largest numbers of participants were still using the assigned medications. We used parallel models to examine differential change over time in clinical variables between the two work programs during phase 2. Subject-level random intercepts and slopes were included in the mixed-effects models to account for correlations induced by multiple measurements per patient.

We fit preliminary models with group (medication or work program) × site interactions; in subsequent analyses we dropped the interaction effects if they were nonsignificant. We adjusted phase-2 models for medication assignment. Within the group that received IPS plus WPFM, we examined the effect of number of clinic sessions attended on the primary work outcomes.

Results

The demographic composition of the sample at study entry and at point of employment are shown in Table 1 and Table 2, respectively. Demographic variables did not differ by randomly assigned medication or by work program, but differences by site emerged for several variables. Of the 107 participants who entered the study, 28 were hospitalized during study participation (IPS only, N=15; IPS plus WPFM, N=13). We did not find a relationship between employment and timing of hospitalization—seven participants who were hospitalized during the study never got a job, eight had a first hospitalization before obtaining a job, six had a first hospitalization after obtaining a job, and seven were hospitalized after ending their first job. Participants in New Hampshire were more likely to be hospitalized than those in Los Angeles (χ²=4.91, df=1, p=.027).

Of the 67 participants who obtained a job, 39 were in IPS only and 28 were in IPS plus WPFM, a nonsignificant difference. [A CONSORT chart is available as an online supplement to this article.] Kaplan-Meier survival curves provided estimates of mean±SE and median days to employment for IPS only (243.03±23.84; median=224) and for IPS plus WPFM (244.64±22.23; median=212). The group difference, after adjustment for site, symptoms, and medication, was not significant. Notably, the estimated times to first job based on the survival analysis are substantially longer than the actual number of days it took to find a job during the study period (Table 3). We present the “raw means” in Table 3 to permit comparison with other studies of IPS; the disparity between the means estimated in the survival analysis and the raw means reported in Table 3 is due to the censoring of observations in the survival analysis. Typical jobs obtained included parking attendant, library assistant, and cleaner; most were entry level and part-time.

Medication did not have a differential impact on clinical or side-effect outcomes, except over the first 12 months of the study, BMI increased by an estimated annual rate of .966 units among olanzapine participants but decreased by an estimated annual rate of .949 units among those randomly assigned to risperidone, a statistically significant medication × time interaction (F=8.82, df=1 and 847, p=.003). The effect of site on BMI (higher in New Hampshire) was significant (F=22.14, df=1 and 847, p<.001). Medication had no apparent impact on any employment outcome.

Among participants who obtained jobs, the participants in IPS plus WPFM had significantly higher total BPRS scores compared with participants in IPS only (t=2.92, df=62, p=.005); as a consequence, we included BPRS total score at the point of employment as a covariate in analyses of phase-2 work outcomes. BPRS and QLS scores were stable subsequent to obtaining a job, with no longitudinal trends or group × time interactions. The IPS only and IPS plus WPFM groups did not differ in rates of attrition after adjustment was made for site, medication, and baseline BPRS total score. Twenty-nine (74%) participants in IPS only and 19 (68%) participants in IPS plus WPFM provided 24-month data. Among those who obtained employment, the groups did not differ on job tenure or any other work outcome, after adjustment was made for medication, site, and phase-2 baseline BPRS symptoms (Table 3).

The above analyses were intent to treat. However, five (22%) participants in IPS plus WPFM attended zero classes. An as-treated analysis that excluded those participants did not reveal any further WPFM benefits. The mean number of skills training classes attended was 18.93±13.51, with a median of 18.50 classes and a range of 0 to 42. A survival analysis conducted with the participants in IPS plus WPFM showed no significant relationship between number of classes attended and job tenure after adjustment was made for medication, site, and symptoms. A subsequent analysis found no benefit from class attendance even among individuals in IPS plus WPFM who held their initial job at least two months and thus presumably would have been most able to generalize skills from the clinic to the work environment.

Discussion

The primary finding was that clinic-based skills training showed no benefits compared with IPS alone on any work outcomes, despite careful attention to implementation and methodological issues. A secondary result was that type of second-generation antipsychotic medication had a differential effect on BMI but had no other associations with clinical or work outcomes among work-ready outpatients. The employment rate (63%) achieved here is comparable to the average (61%) of 11 IPS trials, which included individuals with a broader range of psychiatric diagnoses (21). Compared with the results of a review by Bond and colleagues (21), the mean annualized total of weeks worked over the two years of the trial was slightly less (47.31 weeks, or 23.65 weeks per year compared with 24.2 weeks per year reported by Bond and colleagues), our participants were less likely to work at least 20 hours per week when they did work (19% versus 43.6%), and they took longer to get a job (243.07 days [estimate from the Kaplan-Meier survival curve] compared with 137.6 days). These disparities may reflect the additional burdens associated with employment experienced by individuals with schizophrenia compared with persons with other psychiatric diagnoses (22).

The participants in IPS plus WPFM were offered structured classes that used a manualized format and were delivered by trained and well-supervised facilitators, to little apparent benefit. WPFM may not be tailored sufficiently to specific jobs, may not generalize to some work settings, or may be unnecessary in the context of IPS services. A parsimonious interpretation of the clinic training’s lack of effect is that job match, accommodations, supports, and other factors determine job tenure of people with serious psychiatric illnesses more than the worker’s job maintenance skills. In other words, successful job tenure results from a good person-environment fit and optimal supports. Consistent with this hypothesis, a recent naturalistic study of job tenure among adults with severe psychiatric illnesses receiving supported employment found that supervisor and coworker supports were the primary predictors of job retention (23). We lacked the data to explore this issue more fully here.

The practical challenges in delivering a clinic-based work support program at the same time that participants began a new job were noteworthy. Many participants struggled with the time and social demands of starting work and a new group simultaneously. The two other published studies that integrated skills training with supported employment dealt with the timing of the training differently. Wallace and Tauber (7) initiated the employment search and skills training simultaneously, meaning that some individuals participated in the modules although they did not yet have a job. Mueser and others (9) recruited people who had been successfully employed for up to two months prior to random assignment to WPFM or not, so some may have lost a job before benefiting from the training. The overall power of the IPS model may make it difficult to achieve further improvements based on skills training.

Several limitations of this study deserve note. First, only 63% of the participants obtained jobs. Although this number is in accord with other studies of IPS for people with schizophrenia (24), it reduced our power to find significant effects of clinic-based skills training. Statistical power was probably not a significant limiting factor in finding WPFM benefits, however, because our outcome data generally showed that the group that received IPS only did as well as or slightly better than the group that received IPS plus WPFM. Second, we randomly assigned participants to the work programs at study entry rather than at time of employment because, consistent with IPS principles, we wanted them to remain with a consistent employment specialist throughout the protocol. This early randomization may have inadvertently contributed to unbalanced groups and symptom nonequivalence at the point of employment. However, using covariate adjustment to control for symptom differences, we still did not find even a trend for a benefit from skills training.

Third, WPFM benefits were also likely constrained by limited opportunities to generalize skills to the work setting; a number of participants assigned to IPS plus WPFM either never came to the groups or continued to attend WPFM sessions even after they had lost their jobs, often very early in their group participation. Obviously, individuals who were not working during the time they participated in WPFM or did not attend the group had less opportunity to practice the skills at work. Thus we could not discern the optimal dose of WPFM (if there is one) from our results. Perhaps if resources have been available to provide transportation to the clinic, more WPFM participants might have attended groups. In addition, we lacked independent confirmation that individuals learned the skills taught in WPFM, although the employment specialists often noted that participants appeared to be using WPFM skills outside of the session. Finally, the requirement that participants agree to randomized assignment to a medication may have made this sample different from one found in a typical behavioral health clinic.

Conclusions

IPS is an effective intervention to help individuals with serious psychiatric illnesses obtain employment. Although concerns have been raised about job tenure among IPS recipients, our findings do not support the addition of clinic-based skills training to address the issue.