Competitive Employment Outcomes Among Veterans in VHA Therapeutic and Supported Employment Services Programs

Employment rates among Veterans Health Administration (VHA) patients with mental illness are lower than for VHA patients without mental illness, veterans who do not use VHA services, and nonveterans (1). Nationally representative data indicate that following the 2007–2009 Great Recession, people with mental illness had lower employment rates than people without mental illness (2).

VHA provides interventions to improve the employment status of veterans with mental illness through the Therapeutic and Supported Employment Services (TSES) program (3). Supported employment (SE) provides ongoing individualized employment support integrated with mental health treatment and emphasizes immediate community-based placement in competitive employment (4,5). Transitional work (TW) provides time-limited work placements brokered by VHA, located in a VHA medical center or in the community (3). Although the stated goal of TW is to assist veterans in achieving competitive employment (3), placements are not competitive because they are reserved for VHA patients. Incentive therapy or sheltered workshops (IT/SW) provide opportunities for veterans to engage in tasks (for example, assembly, light cleaning, and escorting patients) to improve basic work skills (for example, cooperation, accepting supervision, and punctuality) in preparation for employment (3). Competitive employment is not a stated goal of IT/SW (3). Ancillary vocational assistance, including prevocational assessment or training and job clubs or vocational support groups, is also available (3).

Evidence for the effectiveness of TSES varies. SE is the gold standard of employment services for people with severe mental illness (that is, schizophrenia and bipolar disorder). Meta-analyses of randomized controlled trials (RCTs) confirm that SE leads to competitive employment (6,7). Recently, an RCT indicated that SE was more effective than TW in improving competitive employment rates among veterans with PTSD (8). Another RCT found that TW did not increase rates of competitive employment for veterans with nonpsychotic psychiatric or substance use disorders compared with job placement services provided by state vocational rehabilitation agencies (9). IT/SW and vocational assistance effectiveness data are lacking.

To fill this gap, we compared the effectiveness of TSES to one another while adjusting for covariates that may affect the outcome of competitive employment. We examined whether receiving specific TSES was associated with competitive employment on program discharge during VHA fiscal years (FYs) 2006–2010. We hypothesized that veterans who received SE would have the highest likelihood of competitive employment. We hypothesized that a lower percentage of veterans would be competitively employed during and after the Great Recession than before it. Finally, we explored potential differences in percentages of competitive employment for different types of TSES over time.

Methods

Data Sources and Sample Selection

We used data from VHA’s Northeast Program Evaluation Center (10), which monitors employment outcomes for veterans enrolled in TSES. Data included clinician-recorded information on veterans’ admission and discharge forms. We used workload data to identify the main type of employment services received if it was not indicated on discharge forms.

We selected working-age (18–65 years) veterans with a complete discharge form during FYs 2006–2010 and for whom admissions forms were also available (N=46,271). We removed 1,578 records with missing data for the following variables: age, gender, marital status, military service era, and years of education. For veterans with more than one admission and discharge to TSES during the study period (N=6,494), we selected the most recent admission to TSES. The final sample included 38,199 unique veterans.

Dependent and Independent Variables

The dependent variable was whether or not a veteran was competitively employed (part-time or full-time) at the time of program discharge.

The main type of employment service each veteran received was the independent variable. Each veteran was categorized as having one main type of TSES: SE, TW in the community (TW-community), TW in the VHA (TW-VA), or IT/SW. We used the following criteria to categorize each veteran into one of the four TSES categories: the discharge form indicated that the veteran spent at least one day in the past 90 days working through a specific type of TSES, the discharge form indicated that the veteran received a particular TSES, or workload data indicated that the veteran had at least four visits to a specific type of TSES between the admission date and the discharge date. When the criteria indicated that a veteran received more than one main type of TSES (N=3,687 veterans, 9.7% of the sample), the veteran was characterized as having the type of TSES most integrated within the community and focused on competitive employment (11), making the categories mutually exclusive. The hierarchy was as follows: SE, TW-community, TW-VA, and IT/SW. Categorizing receipt of main TSES with this hierarchy allowed for the evaluation of whether a veteran was more likely to be employed if he or she received services focused on competitive employment (regardless of other types of TSES received). This reflects a conservative approach to assessing the relative effectiveness of TSES. Veterans who were not placed in any of the four TSES categories described above but who were included in the sample because they had TSES admission and discharge forms were placed into a fifth category indicating that they received no main type of TSES. Individuals in this category, like individuals receiving other primary types of TSES, may have received ancillary vocational assistance (described below).

Covariates

We sought to be inclusive when accounting for veteran characteristics and aspects of employment services veterans received that may have contributed to competitive employment. The covariates we included are described below.

We dichotomized ancillary vocational assistance reported on discharge forms (that is, prevocational assessment and training, job clubs or vocational support groups, and volunteer work) so that each veteran was categorized as having received or not received each service within the 90 days prior to discharge. Veterans could have received any ancillary services in addition to the main type of TSES they received. Veterans could have exclusively received ancillary services if they received none of the main types of TSES.

We categorized the length of time each veteran was enrolled in the VHA TSES program by using the months between the admission and discharge dates (less than six months, six to 12 months, and more than 12 months). Clinicians’ reports (yes or no) indicated whether the veteran “gave any indication that one factor for deciding to leave the program” was a concern that employment would jeopardize the financial support he or she received.

Demographic characteristics obtained from admission forms included age (18–44 and 45–65), gender, race (Caucasian, black, and other), marital status (single or widowed, married, and separated or divorced), level of education (less than high school diploma, high school diploma or GED, and more than high school diploma or GED), and period of military service (Vietnam, post-Vietnam, Persian Gulf, and OEF/OIF [Operation Enduring Freedom/Operation Iraqi Freedom]). Psychiatric diagnoses included whether a veteran had been diagnosed as having any of the following DSM-IV mental disorders: anxiety disorders (other than posttraumatic stress disorder [PTSD]), PTSD, depression, schizophrenia spectrum and psychotic disorders, bipolar disorders, adjustment disorders, personality disorders, alcohol abuse or dependence, and drug abuse or dependence. Clinicians’ reports of veterans’ medical conditions (head injury, cerebrovascular accident or stroke, spinal cord injury, arthritis, multiple sclerosis, liver disease, pulmonary disease or chronic obstructive pulmonary disease, hypertension, coronary heart disease, dementia, orthopedic problems, diabetes, and other medical problems) identified via chart review characterized general medical comorbidity. We summed the number of general medical conditions endorsed to obtain a global measure of comorbidity (12) and categorized them (0, 1–5, and >5).

We operationalized work history prior to TSES program admission by using clinicians’ responses to the question “What has been the veteran’s usual employment pattern during the past three years?” Responses were coded into categories: full- or part-time competitive employment, irregular employment (irregular part-time jobs), and not employed (including unemployed, retired, disabled, volunteer, and student). We assessed homelessness status by clinicians’ reports of whether the veteran lacked “a fixed, regular, and adequate night-time residence.” We assessed financial assistance by indicating whether the clinician recorded that the veteran received any Social Security Disability Insurance or Supplemental Security Income, income for a VA service-connected disability, other disability income, public support, or pension (yes or no).

We included discharge year as a covariate to account for the impact of the Great Recession on employment status.

Statistical Analyses

We computed descriptive statistics. Bivariate analyses assessed associations between covariates and type of employment service received with competitive employment status at discharge.

We used logistic regression to assess whether the main type of TSES received was associated with odds of being competitively employed, adjusting for all covariates described above. We used a “recycled predictions” analysis to assess the predicted probability (percentage) of veterans competitively employed as a function of the type of employment service received. This type of analysis calculates the mean predicted probability of a given outcome (competitive employment) by using coefficients from the regression model (13,14) and yields findings (percentages) that can be interpreted in a straightforward manner.

We assessed whether the odds of competitive employment differed on the basis of the interaction between main type of TSES received and discharge year by using logistic regression, controlling for covariates and main effects of discharge year and main type of TSES received. A recycled predictions analysis computed the predicted probability of competitive employment at discharge based on the interaction of main TSES received and discharge year while adjusting for covariates and main effects. We used SAS, version 9.3 for descriptive, bivariate, and logistic regression analyses and STATA 12 for recycled predictions analyses.

The VA Ann Arbor Healthcare System Institutional Review Board approved this study.

Results

Sample Description

As shown in Table 1, most of the 38,199 veterans were male (93.8%). Over two-thirds (76.1%) were ages 45–64, and 23.9% were ages 18–44. About half of the sample was Caucasian (52.5%), and 45.7% was black. Most veterans were separated or divorced (53.6%). For half the sample (50.4%), the highest level of educational attainment was a high school diploma or GED, whereas 44.8% had education beyond high school. The most common periods of military service were post-Vietnam (49.3%) and Vietnam (32.8%). An individual could have more than one psychiatric diagnosis, and the most common were alcohol abuse or dependence (62.8%), drug abuse or dependence (55.5%), and depressive disorders (38.3%). Most veterans (76.8%) had one to five general medical conditions. Nearly half of veterans (48.8%) had been competitively employed in the three years prior to program entry. Over half of the sample (54.2%) was homeless at program admission. Approximately half of veterans (49.0%) were receiving some form of financial assistance while half were not (51.0%).

On the basis of our operational definition of main type of TSES received, most veterans received TW-VA (57.5%), followed by SE (21.0%), IT/SW (9.0%), and TW-community (6.3%); 6.2% received no main TSES. A minority received ancillary prevocational assessment or training (12.3%), job club or vocational support groups (29.7%), and volunteer work (.8%).

Predictors of Competitive Employment

Findings from bivariate analyses (Table 1) of associations between predictor variables and competitive employment on discharge were generally consistent with the following covariate-adjusted analyses. Results of multivariable logistic regression analysis and associated recycled predictions analysis indicated that the main type of employment service received was significantly associated with competitive employment at discharge (Table 2 presents odds ratios, and Table 3 presents predicted probabilities). Receiving SE or TW-community was associated with greater odds of competitive employment at discharge, compared with receiving TW-VA. Recycled predictions analyses indicated that 34.9% and 34.8% of veterans receiving SE and TW-community obtained competitive employment, respectively, compared with 30.3% of veterans receiving TW-VA (p<.001 for all). Receiving IT/SW was associated with lower odds of competitive employment than receiving TW-VA (p<.001); 27.2% of veterans receiving IT/SW were competitively employed. There was no difference in odds of competitive employment between receiving no main type of employment service and TW-VA. The adjusted predicted probability of competitive employment for veterans receiving no main type of employment service was 30.3%.

Receipt of ancillary employment services was associated with odds of competitive employment. Specifically, prevocational assessment or training (p<.01) and job clubs or vocational support groups (p<.001) were associated with greater odds of competitive employment, whereas volunteering was associated with lower odds of employment (p<.001).

In addition, being enrolled in the TSES program for more time (six to 12 months or more than 12 months) was associated with higher odds of competitive employment on discharge compared with being enrolled for less than six months (p<.001 for both). Although a small percentage of the sample reported concerns that employment would jeopardize their benefits (2.5%), this group had lower odds of competitive employment than veterans who did not report this concern (p<.001). Additional covariates significantly associated with competitive employment are reported in Table 2.

Competitive Employment by Main Type of TSES Received Across Time

Results of the second multivariable analysis (full analysis not shown) indicated a significant interaction between discharge year and main type of TSES received (Wald χ²=99.39, df=16, p<.001). Figure 1 displays predicted probabilities of competitive employment derived from recycled predictions analysis as a function of the interaction between discharge year and main type of TSES received, with adjustment for covariates and main effects. During the years of the economic recession and the subsequent period when the national unemployment rate remained high (corresponding to VA FYs 2007−2010 in Figure 1), there were less pronounced differences in predicted probability of employment based on main type of TSES received compared with during FY 2006.

Discussion

We assessed relative effectiveness of VHA employment services in achieving competitive employment among veterans with mental illness. Findings indicated that veterans who received SE or TW-community were more likely than veterans who received TW-VA to obtain competitive employment. There was no difference in competitive employment outcomes between veterans receiving TW-VA and no main TSES. Veterans who received IT/SW were less likely than veterans who received TW-VA to obtain competitive employment. These results are consistent with decades of research demonstrating the efficacy of SE through RCTs (15) and extend the literature by demonstrating the relative success of SE services in a real-world national health care system.

Although our findings demonstrate the real-world effectiveness of SE and TW-community compared with TW-VA, the predictive probabilities of competitive employment at discharge after receipt of SE and TW-community services (approximately 35% for each) were lower than the percentages of individuals with severe mental illness in RCTs (52%−60%) who achieved competitive employment after receipt of SE (15). This discrepancy in findings for SE outcomes may reflect the “voltage drop” that occurs when interventions are translated from research to practice, and this may be attributed to heterogeneity in organizational and clinical contexts, providers, and veteran characteristics (16,17). Research is needed to assess what factors within VHA can be modified to increase competitive employment outcomes of SE. Because fidelity to SE models is known to affect whether individuals achieve competitive employment (18), future studies should assess fidelity to SE as a potential mediating factor of employment outcomes within VHA.

Because there is no uniform model of TW-community, it is unclear what features of this service drove the increased odds of competitive employment compared with TW-VA. Possibly, some aspects of TW-community services emulated SE features. Employers may enter a TW-community relationship with plans to hire the veteran at the end of the agreed-upon work period, and thus they may use TW-community as a trial work period. The known commonality between SE and TW-community is that both involve veterans working outside the VHA. Thus our findings indicate that employment services for veterans with mental health conditions in community settings were more effective than employment services in a medical center. This finding is notable because TW-VA was the most commonly received employment service (57.5%), whereas fewer veterans received community-based services of TW-community (6.3%) or SE (21.0%).

It is interesting that prevocational assessment or training and job clubs were associated with increased odds of employment. However, receiving no main type of TSES (that is, receiving only ancillary services) was associated with lower odds of employment compared with TW-VA. Thus ancillary services may be beneficial specifically when they are paired with a main type of TSES. Research is needed to evaluate the effectiveness of different combinations of main types of TSES and ancillary services. Future investigations should assess whether ancillary interventions not currently required by VHA policy (for example, cognitive-behavioral therapy tailored to work [19,20]) lead to improved competitive employment outcomes when supplementing SE and TW-community.

Findings indicate that during the Great Recession (2007–2009) and in the subsequent year, competitive employment was harder for veterans with mental illnesses to achieve, regardless of type of TSES received. This suggests that existing employment services designed to promote employment among veterans with mental illness cannot sufficiently mitigate national economic trends that negatively affect employment.

Associations between covariates and competitive employment were generally consistent with prior research regarding hypothesized or known challenges to obtaining competitive employment for people with mental illness (1,15,21–24). We found that most of the mental health conditions examined, poorer work history, less education, receipt of government financial assistance, concerns about losing financial assistance, and having more than five general medical comorbidities were associated with lower odds of competitive employment. We also found that veterans enrolled in the TSES program for more than six months had increased odds of obtaining competitive employment.

The study had some limitations. Program admission and discharge forms do not capture vocational services that veterans may have received outside VHA, including any vocational rehabilitation services provided by the Veterans Benefits Administration.

We did not assess geographic (for example, rurality), facility (for example, availability of services), clinician (for example, expectations and caseload size), or psychological (for example, self-efficacy, employment preferences, and preferences for types of services) factors— all of which are known or hypothesized to influence employment outcomes for individuals with mental illness (11,25–32). Future studies using socioecological approaches to capture factors across multiple levels of analysis would inform how such factors interact and could be modified to increase attainment of competitive employment through TSES.

Conclusions

Our findings regarding effectiveness of SE and TW-community compared with other TSES interventions have policy implications. The 2016 VHA TSES Transformational Plan prioritizes community-based competitive employment as the primary goal of services (33). Our findings indicate that the best way to achieve this outcome is through SE or TW-community. The plan calls for discontinuation of IT and SW services, a decision consistent with our finding that IT/SW services were associated with lower odds of competitive employment. The plan continues to promote SE and TW and a new intervention, community-based employment services, which uses principles of SE to provide community-based employment services for veterans with mild to moderate disabilities. Continuation of TW-VA as implemented during the years of this study was not well supported by our findings.

Although we found that both SE and TW-community yielded higher odds of competitive employment, decades of evidence for the effectiveness of SE coupled with uncertainty regarding which components of TW-community may contribute to competitive employment leads us to recommend an expansion of SE within VHA. We recommend further evaluations of conditions under which SE and TW-community are most effective and evaluations of critical ingredients of TW-community. Finally, we recommend evaluating the effectiveness of employment services not assessed in this study (community-based employment services and combinations of ancillary services with SE and TW-community).