Modifiable Predictors of Supported Employment Outcomes Among People With Severe Mental Illness

Unemployment is common among individuals with severe mental illnesses and is associated with high economic costs, the largest being lost productivity (1–5). Evidence-based supported employment, also known as individual placement and support (IPS), is an evidence-based practice to assist people with severe mental illness in returning to work; multiple studies and meta-analyses have shown IPS to be more effective than conventional vocational rehabilitation in improving work outcomes such as job acquisition, job tenure, and wages (6–9). IPS principles include eligibility based on client choice (zero exclusion); attention to client preferences; competitive employment as the goal; rapid job search; integration of mental health treatment and supported employment; individualized, time-unlimited job support; systematic job development; and benefits counseling (10).

A growing body of research has addressed predictors of work outcomes in people with severe mental illness, both in general and among individuals receiving supported employment. Predictors of better work outcomes in general include higher education (11, 12), stronger or more recent work history (1, 13–16), and absence of psychosis (17). The relationship between age and work outcomes remains uncertain (6, 7, 10, 17, 18). Similarly, the link between ethnicity and work outcome remains inconclusive; some studies have found an association between Hispanic ethnicity and better work outcomes (15, 18), whereas others have not (19).

Ascertaining individual predictors of work outcomes in the context of IPS may help providers in addressing modifiable client factors for IPS service users. A landmark meta-analysis of four large IPS trials found that the effects of IPS on work outcomes were robust even when accounting for most demographic, clinical, and employment characteristics (7). Another study of 2,055 Social Security Disability Insurance beneficiaries showed that work history was the strongest predictor of supported employment outcomes (15). These and other studies have concluded that IPS should be offered to anyone with severe mental illness who wants to work (9, 20). However, the search for modifiable predictors of work outcomes continues to improve the typical job acquisition rate in IPS programs (61%) (10).

Many studies have suggested that neurocognitive and functional abilities may be strong predictors of work outcome (11, 21, 22), but relatively few IPS studies have included comprehensive neuropsychological and functional assessments. Better performance on measures of verbal learning (13, 18), working memory (23), executive functioning (11, 13), and processing speed (16) have been linked to better vocational outcomes. Additionally, greater baseline functional capacity (14) and better social skills (12) have been associated with better work outcomes.

Much of the prior research on the relationship between neuropsychological ability and work outcomes has been limited by small sample size, short follow-up periods, reliance on samples of individuals with the same diagnosis or the same type of disability benefits, limited neurocognitive test batteries, or the inclusion of clients not enrolled in supported employment. As such, this study aimed to fill some of the gaps in the published literature by determining the strongest neuropsychological and other modifiable predictors of work outcomes in a large sample of IPS service users with varying diagnoses. Participants received IPS for up to 2 years and were assessed by using a comprehensive neuropsychological battery and performance-based measures of functional and social skills, in addition to standard clinical measures.

Methods

Participants

The study was registered as a clinical trial (NCT00895258) and data were collected from June 2008 to February 2014. Study procedures were approved by the University of California, San Diego, Institutional Review Board, and all participants provided written informed consent. One hundred fifty-three unemployed outpatients with severe mental illness (schizophrenia or schizoaffective disorder, N=58; bipolar disorder, N=37; and major depressive disorder, N=58) enrolled in the trial. Inclusion criteria were age 18 and older; literate and fluent in English; DSM-IV diagnosis of schizophrenia, schizoaffective disorder, bipolar disorder, or major depressive disorder confirmed via Structured Clinical Interview for DSM-IV (24) or Mini International Neuropsychiatric Interview (25); and being unemployed for at least 30 days and being interested in working. All participants received IPS for up to 2 years, based on their preference, and were randomly assigned to also receive either compensatory cognitive training (N=77) or additional supported employment sessions (enhanced supported employment; N=76) for the first 12 weeks of the trial (see Twamley et al. [26] for further details). Each group had its own employment specialist. Fidelity to supported employment was rated as “fair” during the study period. Because work outcomes associated with compensatory cognitive training and enhanced supported employment did not differ (26), the groups were collapsed for all analyses. Table 1 provides the demographic and clinical characteristics of the sample.

Baseline neuropsychological, clinical, and functional assessments were used in the current analyses. Data from these participants have been used in prior publications (26–30); however, the analyses presented in this article have not been published previously.

Measures

The following assessments were administered at baseline, prior to randomization. All raters were trained to a high degree of interrater reliability (intraclass correlation coefficient ≥.90).

Neuropsychological Measures

Premorbid intellectual ability was estimated with the reading subtest of the Wide Range Achievement Test-III (31). The cognitive subtests of the Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery (32) assessed neuropsychological functioning in the domains of processing speed (Trail Making Test, Part A [TMT-A], Brief Assessment of Cognition in Schizophrenia Symbol-Coding [BACS-SC], and category fluency), sustained attention (Continuous Performance Test–Identical Pairs), working memory (Wechsler Memory Scale-III Spatial Span and University of Maryland Letter-Number Span [LNS]), verbal learning (Hopkins Verbal Learning Test–Revised [HVLT-R]), visual learning (Brief Visual Memory Test–Revised [BVMT-R]), and executive functioning (Neuropsychological Assessment Battery [NAB] Mazes). All t test scores were corrected for age and education. Additional tests of executive functioning measured set shifting (Trail Making Test, Part B [TMT-B] [33]), letter fluency using the letters F, A, and S (FAS) (33), and reasoning and set shifting (Wisconsin Card Sorting Test–64-card version [WCST-64] [34] total errors). Additionally, prospective memory ability was measured by using the Memory for Intentions Screening Test (35). We calculated a global deficit score (GDS) by transforming individual neuropsychological test scores (i.e., t test scores) to deficit scores ranging from 0, no impairment, to 5, severe impairment, which were then averaged across all tests (36).

Functional Skills and Symptom Severity

The University of California San Diego Performance-Based Skills Assessment–Brief (37) assesses performance-based functional capacity in the domains of financial management and communication. The Social Skills Performance Assessment (38) measures social skills relevant to neutral and adversarial situations. Positive and negative symptom severity and general psychopathology were measured with the Positive and Negative Syndrome Scale (39), and depressive symptom severity was measured by using the Hamilton Depression Rating Scale (40).

Work Outcomes

Work outcomes (competitive job attainment, total weeks of competitive employment, and wages earned) were gathered weekly during the 2-year study by the employment specialist (if engaged in weekly contact with the participant) or a research assistant; work participation and earnings were corroborated with paystubs. Competitive work was defined as employment paying at least minimum wage and not set aside for a person with a disability. Participants who dropped out of the study prior to obtaining a job were assumed not to have worked.

Statistical Analyses

One-way analysis of variance and chi-square tests were conducted to examine differences in characteristics between diagnostic groups. Prior to analyses, model assumptions were checked, including screening for outliers and evaluating for multicollinearity, with tolerance values of <.40 and variance inflation factor values of >2.5 suggestive of multicollinearity (41). Job attainment during the 2-year study period (0=no; 1=yes) was analyzed by using a logistic regression model. Examination of variable distributions for competitive weeks worked and wages earned showed positively skewed distributions, with excess zeroes. Thus these variables were log-transformed before being included in all analyses and modeled by using a zero-altered count regression approach, known as a hurdle model. Hurdle models are two-part models, in which all the zeroes are modeled with a probit regression, and nonzero counts are modeled by a truncated count regression (i.e., truncated because it does not include zero) (42). Our hurdle models reflected the two-stage process resulting in the observed distributions of competitive weeks worked and wages earned. That is, participants first had to attain a job (i.e., pass the “hurdle”) to report weeks of employment and wages earned. Thus the probit regression component of the hurdle model examined the predictors of job attainment, whereas the count regression in the hurdle models examined weeks worked and wages earned for those who attained a job (N=72). Analyses were conducted by using SPSS. version 24.0, except for analyses of hurdle models, which were conducted by using STATA/IC, version 15.0.

Bivariate Pearson and point-biserial correlations between individual tests of neuropsychological functioning (i.e., neuropsychological measures, including premorbid intellectual ability), GDS, psychiatric symptom severity, performance-based functional capacity and social skills, demographic variables, and work outcomes, were conducted. Bivariate-significant correlates of job attainment (p<0.05) were entered as predictors of job attainment in the logistic regression model as well as the probit component of the hurdle models. Similarly, for the subset of participants who attained a job, bivariate-significant variables were entered as predictors of weeks of competitive work and wages earned for the count regression models. (Although work history was not significantly associated with weeks worked, it was entered in the model because of computational requirements of hurdle model, i.e., algorithm limitations. Including work history did not affect model estimates. Our model excluding work history converged using an alternative linear maximum likelihood model [i.e., zero-inflated negative binomial model], and had similar results, thereby bolstering the robustness of the hurdle models.) There were significant demographic differences by diagnostic group in years of education and premorbid IQ estimate (p<0.05), which were controlled for in subsequent analyses.

Results

Demographic characteristics and outcomes for the 153 participants are summarized in Table 1. Table 2 summarizes characteristics of the 72 participants who attained work during the 2-year study. Bivariate correlations (N=153) determined significant associations between several participant characteristics and job attainment, including education, racial-ethnic minority status, work history, diagnosis, and psychiatric symptom severity (Table 3). Furthermore, better functional capacity, greater estimated intellectual functioning, and better performance on BACS-SC and LNS were associated with job attainment. As such, these variables were entered in the logistic regression model and the probit regression component of the hurdle models.

Forward entry likelihood ratio (LR) stepwise analysis found work history and negative symptom severity to be significant predictors of job attainment during the 2-year study period. Jointly, these variables improved model fit by 26% (χ²=32.92, N=151, df=2, p<0.001, Nagelkerke R²=0.26), with less severe negative symptoms (odds ratio [OR]=.910, Wald z=5.90, df=1, p=0.015, 95% confidence interval [CI]=0.843–0.982) and more recent work history (OR=.971, Wald z=13.25, df=1, p<0.001, 95% CI=0.956–0.987) associated with increased odds of obtaining employment.

For competitive weeks worked, variables significantly associated with job attainment were entered into the probit regression component of the hurdle model. Simultaneously, variables significantly associated with competitive weeks worked (i.e., positive symptom severity and GDS; Table 3) for the subset of participants who attained a job (N=72) were entered in the count regression component of the hurdle model, along with work history. (Note that the probit and count regression components of the hurdle model are separate, so including GDS in the count regression and individual test scores in the probit regression, respectively, did not introduce multicollinearity.) The hurdle model found more recent work history to be a significant predictor of job attainment, whereas lower levels of neuropsychological impairment (GDS) emerged as a significant predictor of greater competitive weeks worked (p<0.05).

Backward selection elimination of predictor variables with p>0.05 in the hurdle models was performed to determine the robustness of the findings and determine additional significant predictors that may not have been detected because of redundancies. This resulted in a more parsimonious final model, reported here, with additional significant predictors for job attainment. Specifically, consistent with the logistic regression model reported above, lower negative symptom severity emerged as an additional significant predictor of job attainment along with more recent work history (Table 4). GDS remained the only significant predictor of weeks of competitive work, with the overall hurdle model improving model fit by 14% (LR χ²=42.33, N=151, df=2, p<0.001, pseudo-R²=0.14). Postestimation analyses determined that higher GDS was related to fewer weeks worked; a participant with a GDS of 0 worked 33.67 weeks on average during the 2-year study duration, whereas a participant with a GDS of 2.5, indicating moderate impairment, worked an average of 9.23 weeks throughout the study duration, approximately four times less.

For wages earned, the results of the probit regression model were identical to the hurdle model reported above. For the count regression, variables significantly associated with wages earned (i.e., work history, positive symptom severity, and GDS; Table 3) for the subset of participants who obtained a job were entered into this component of the hurdle model. GDS emerged as a significant predictor of wages earned (p<0.05). Backward selection elimination identified work history as an additional significant predictor of wages earned, with the overall hurdle model improving model fit by 14% (LR χ²=48.89, N=151, df=2, p<0.001, pseudo-R²=0.14; Table 4). Further examination of these associations revealed that less neuropsychological impairment (GDS) was related to higher wages; on average, a participant with a GDS of 0 earned twice as much as a participant with a GDS of 1 ($7,942.28 vs. $3,387.44). Furthermore, more recent work history was related to higher wages; a participant who was unemployed for 1 month at study entry earned, on average, $9,331.54 over the 2-year study compared with $3,234.94 for participants who were unemployed for 1 year.

Additional exploratory analyses were conducted to examine associations between cognitive functioning, psychiatric symptom severity, and work history. Better working memory (LNS), executive functioning (NAB Mazes, WCST-64), and visual learning (BVMT-R) and less severe positive symptoms were associated with more recent work history (p<0.05 for all). There were no differences between the psychiatric diagnostic groups on work history.

Discussion

This study examined the strongest neuropsychological and other modifiable predictors of vocational outcomes in a sample of individuals with severe mental illness. Considering the transdiagnostic presentation of cognitive impairments in psychiatric disorders and poor associated work outcomes, understanding the differential relationship of neuropsychological abilities with vocational outcomes may reveal targets for cognitive training or remediation among people with severe mental illness.

Indeed, we found several neuropsychological abilities that were associated with work outcomes at the bivariate level, including processing speed (BACS-SC), working memory (LNS), premorbid IQ (Wide Range Achievement Test-III), and global neuropsychological performance (GDS). However, our hurdle models showed that only overall neuropsychological ability (GDS) remained a significant predictor of weeks worked and wages earned. Significant predictors of job attainment included more recent work history and less severe negative symptoms. Among those who obtained jobs, better global neuropsychological functioning (i.e., lower GDS) predicted greater weeks worked, above and beyond demographic and clinical characteristics and work history. After the analyses were controlled for demographic and clinical characteristics, better global neuropsychological functioning also predicted greater wages earned over the 2 years, along with more recent work history. These findings are consistent with previous studies demonstrating cognitive functioning as a significant predictor of work outcomes even after controlling for work history, itself a robust predictor of work outcomes (18). In the general population, cognitive ability is a strong predictor of work outcomes, and this association is partly mediated by the fact that better cognitive ability predicts better learning and job knowledge (43, 44). Our results suggest that, regardless of diagnosis, the cognitive impairments seen within severe mental illness uniquely predict vocational outcomes.

Consistent with previous findings (15), diagnosis did not emerge as a significant predictor of work outcomes. Our results highlight the importance of negative symptoms in predicting employment outcomes, a significance which may be explained through transdiagnostic models of negative symptom phenomenology ascribing a stronger role to these clinical symptoms compared with diagnosis (45). These findings underscore the importance of the independent examination of clinical phenotypes as discrete from diagnostic entities (46).

The findings may hold practical significance for mental health clinicians and employment specialists who work with clients with severe mental illness; targeting negative symptoms and providing cognitive training or remediation may improve work outcomes in supported employment service users. Although cognitive training programs are associated with improvements in varied cognitive domains (26, 47, 48) as well as negative symptoms (26, 49, 50), further research is needed to investigate the efficacy of such programs in improving work outcomes via cognition.

Given the significance of work history in predicting work outcomes and given that better cognitive performance was associated with more recent work, the inclusion of work history in our models may have served as a proxy for cognitive functioning. Thus, inclusion of individual cognitive tests as predictors may have had an insignificant effect on improving model fit when added along with work history. Despite these intercorrelations, our findings highlight the significance of overall cognitive performance for work success.

There were limitations to the current study that must be acknowledged. Given that our participants were community-dwelling, unemployed individuals with severe mental illness who received supported employment, the sample lacked a control group that did not receive supported employment and the results may be limited in generalizability to other samples. Also, the use of the Positive and Negative Syndrome Scale to assess negative symptoms precluded the examination of the role of primary versus secondary negative symptoms (e.g., negative symptoms secondary to depression) in predicting work outcomes. Future research should incorporate these distinctions in their investigations. The Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery includes limited assessment of attention, verbal learning, and visual learning (one test per domain); future studies should consider using additional measures of these constructs. Additionally, differential psychometric properties, such as sensitivity to deficits, may explain the lack of significance across all measures for domains assessed through multiple tests (51).

Conclusions

This study generated evidence for a transdiagnostic consideration of the role of neurocognitive deficits and negative symptom severity in predicting work outcomes in supported employment service users. The results suggest that improving negative symptom severity and cognitive performance may improve supported employment outcomes in people with severe mental illness