Outcomes of Peer-Provided Individual Placement and Support Services in a Mental Health Peer-Run Vocational Program

The importance of employment in the recovery of people with mental illness is well established, as is the effectiveness of the supported employment model known as individual placement and support (IPS) (1). However, IPS availability is inadequate to meet the sizable need in the United States (2). One reason for IPS unavailability is a shortage of IPS providers. Workforce issues were one of the four most common barriers to implementing IPS in a recent U.S. national survey of state administrators (3); furthermore, research on IPS adoption in rural areas identified limited workforce availability as a severe challenge (4). Given the increasing use of peer specialists in the delivery system of mental health services (5, 6), and the fact that this workforce is among the fastest growing in behavioral health care (7), peer specialists may be valuable for delivering IPS services.

Peer providers were early proponents of physical wellness support services (8, 9), creating evidence-based models for health self-management (10–12), reduction of health risks such as smoking and obesity (13, 14), and health care navigation (15, 16). Given strong evidence that general health is a major determinant of employment status (17–19) and documented poor health among people with serious mental illness (20, 21), advantages may be gained by combining the well-established IPS model with health and wellness support. Previous research confirming the efficacy of peer-delivered health education and support for enhancing general health (10–12) suggests that applying this model in a supported employment context may assist people in seeking and maintaining competitive employment.

Peer-run organizations tend to prioritize employment as an important goal, unlike some traditional mental health programs that discourage work because of concerns about vocational stress and loss of disability benefits (22–24). Many principles of IPS overlap with the philosophy of peer support and self-help, including zero exclusion, consumer choice in job type and employment setting, rapid job search, and ongoing support with no time limits (25). This commonality suggests that peer-run programs may be well suited to deliver IPS services.

Findings from limited research suggest that peer support may enhance outcomes of supported employment services (26, 27). The authors of one randomized study reported a significantly higher 12-month employment rate for those receiving supported employment from both peers and nonpeers versus from nonpeers alone (28). Moreover, in a feasibility study of IPS adoption by a peer-run program, an IPS fidelity score of “fair” was achieved, and 33% of service recipients attained competitive employment over 2 years (29). Finally, results from a randomized trial comparing vocationally oriented peer support and generic peer support revealed no differences in employment but a significant increase in work readiness among recipients of vocationally oriented peer support (30).

The purpose of this study was to explore the potential of certified peer specialists as IPS service providers in a model that also included services to enhance work-related general health. We collaborated with a large peer-run program to introduce IPS combined with health promotion to its vocational programming. Our research questions included whether peer specialists could be trained to deliver IPS with a high level of fidelity, whether health promotion services combined with IPS would be beneficial, and whether employment outcomes of peer-delivered IPS plus health promotion were superior to those of usual supported employment services.

Methods

Study Setting

This study took place at Baltic Street Advocacy, Employment, and Housing, Inc. in New York City from July 2015 to July 2017. The agency provided advocacy and self-help services, support groups, residential assistance, and services bridging the transition from hospital to community. The agency’s intake process required referral from a treating clinician who completed a psychosocial assessment before admission confirming serious mental illness. The program had delivered vocational services for several years, loosely based on the choose-get-keep supported employment model (31, 32). Two vocational teams using identical staffing (a team leader plus two to three full-time-equivalent staff members), supervision format, and job descriptions were located at separate offices in different areas of the same city borough with similar geographic and job market features. Staff on each team were completely nonoverlapping.

Participants

Study participants included all supported employment recipients during the study period. Individuals enrolled in employment services by expressing interest in working or being referred from the larger program or external programs. Assignment to one or the other team was based on participants’ personal preference and convenience. Their deidentified vocational outcome data were used, regardless of participation in employment services. Another data source was information from IPS fidelity reviews of the intervention condition conducted at study start, midpoint, and end. The study was approved by the institutional review board of the University of Illinois Chicago.

Intervention and Comparison Conditions

Because random assignment of participants was impractical, we used a quasi-experimental design. The intervention consisted of IPS services combined with health promotion services delivered by employment specialists. IPS services involved completion of a vocational profile summarizing the person’s employment resources and strengths, work history, and desired type of job and working conditions. Employment specialists provided individualized, intensive job search support, including liaison with potential employers and coaching during the interview and hiring process. After employment was secured, ongoing support was provided to the worker and the employer (if desired) with no time limits. The physical wellness component of the intervention involved a structured set of activities during meetings with employment specialists. These activities included education about work-health connections regarding sleep and rest, physical activity, relaxation exercises and stress management, healthy eating, and medical care. Participants used the Physical Wellness for Work planning tool (https://www.center4healthandsdc.org/physical-wellness-for-work.html) to identify health habits and routines to support their specific work goals. For each goal, participants identified steps to take to establish health routines, find resources required, anticipate obstacles, seek assistance needed from employment specialists, and initiate accountability methods such as check-ins or reminder calls. During subsequent meetings, employment specialists inquired about progress toward establishing the desired health routines, helped remove any barriers, and modeled their own work-related wellness strategies. Specific activity codes were used to document in the person’s case file each step of this process (didactic instruction, planning tool completion, and identification of work-related habits and routines) and degree of progress made in changing health routines.

The comparison condition was usual vocational services delivered by peer staff on the second team, using the choose-get-keep approach (32). Services included supporting participants who engaged in preemployment activities to identify their preferred jobs and work settings. This step was followed by assistance during the hiring and onboarding process and then by support for sustaining employment through training and follow-along as well as help during job exit (if needed) and afterward (31).

The team that delivered IPS received training and supervision throughout the study from two external experts who were affiliated with the IPS Employment Center, the model’s national training and research entity. The IPS training expert (a private consultant) familiarized staff with IPS principles and practices, modeled and observed IPS service delivery activities, and provided ongoing supervision and support for employer outreach, job development, and support activities. Initial training involved assigned readings, completing the IPS Employment Center’s practitioner skills course, expert feedback at twice-monthly onsite visits, and weekly telephone supervision. Training was repeated whenever new staff were hired. Performance goals were established for employment specialists and included monthly number of new employer contacts and new jobs developed. Periodic IPS fidelity assessments identified areas for further training.

External health and wellness experts from peer-run Collaborative Support Programs of New Jersey also worked with IPS staff throughout the study period. They introduced tools and activities to help participants address general health issues that affect employment. This training included assigned readings, didactic instruction on links between physical wellness and successful job search and retention, wellness goal setting with specific wellness tools, modeling wellness tools for participants, and ongoing technical assistance via in-person visits and conference calls for mentoring and supervision. Additionally, the experts worked with managers to create logs and a notation system for documenting health promotion services and progress in the participant’s file.

Measures

The primary outcome was competitive employment defined in part 363 of the Rehabilitation Act of 1973 as amended (33) as a job in the competitive labor market belonging to the worker and not representing a set-aside for people with disabilities, paying minimum wage or above, and located in a socially integrated setting (34). Researchers verified that jobs met the definition of competitive employment. Monthly data on competitive employment status included hours worked, hourly wage, and job start and end dates. Participant background data included age, gender, race, ethnicity, education, number of months receiving employment services, and program year. Program measures for each team included caseload size, number of job starts, and job tenure.

Recognized as a critical component of the IPS model, the Supported Employment Fidelity Scale (35) was used to assess fidelity before the start of the study, midway through the study, and after study completion. Possible scores on this scale range from 25 to 125, with ≤73 indicating lack of supported employment; 74–99, fair fidelity; 100–114, good fidelity; and 115–125, exemplary fidelity. Following standard assessment procedures (36), an external expert and a trained partner conducted interviews with the agency chief executive officer, vocational program director, IPS team leader, IPS employment specialists, and a sample of IPS service recipients. Ten randomly selected client files also were reviewed, along with vocational outcome data. This information was then used to complete scale items divided into three sections assessing staffing, organization, and services.

Statistical Analysis

We assessed statistical significance of differences between teams with chi-square and t tests. We modeled the primary outcome of competitive employment by using a random-effects logistic regression model (RRM) with group assignment propensity score adjustment. The data’s longitudinal, repeated-measures structure, with varying number of measurements per participant, was appropriately handled by RRM (37). Given the quasi-experimental design, propensity score adjustment compensated for nonrandomized study group assignment (38, 39). We estimated propensity scores in a logistic regression model predicting team assignment (IPS vs. comparison team) by using six participant characteristics: age, gender, race, ethnicity, education, and number of months receiving vocational services. The RRM had an autoregressive covariance structure and included study condition (IPS vs. comparison condition) and calendar time (months 1–25). The model also controlled for monthly caseload of teams and number of months services were received. We modeled differences in the rate of change in competitive employment over time by using a condition × time interaction term. Analyses were conducted in IBM SPSS Statistics, version 25.0.

Results

IPS Fidelity

Programs are considered to be implementing IPS if they reach the threshold of fair fidelity (40), and studies of multisite IPS implementation consider model adoption to be successful when programs reach “fair to good” fidelity (41). At study baseline, the intervention team received a total fidelity score of 71, indicating that its services did not qualify as IPS supported employment. Feedback from consultants included advice to streamline the intake process, emphasize the employment specialists’ leadership role in the job search, and secure competitive jobs in a wider range of community settings. At study midpoint, a score of 99 was achieved, indicating “fair fidelity.” Consultant feedback included providing skill enhancement across different IPS service phases, creating an IPS procedural manual, and adopting monthly performance benchmarks for employment specialists. At the third and final fidelity assessment, a total score of 110 was achieved, indicating good fidelity. Consultants’ observations included kudos for adding a benefits counselor to the team, compliments for becoming a Ticket to Work provider and thus broadening sources of program revenue, suggestions for increasing participants’ employment rates, and acknowledgment of the team’s recent Recognition of Excellence for Wellness Award from the Substance Abuse and Mental Health Services Administration for its innovative blend of health, wellness, and IPS employment services.

Participant and Program Characteristics

Table 1 presents the characteristics of 348 individuals served by the two teams (IPS, N=184; comparison condition, N=164) over the 2-year period, along with team-level features, such as average caseload and number of clients served. The teams did not differ statistically significantly in any participant or team characteristics except for race, with the IPS team having a higher proportion of Black participants and a lower proportion of White participants than the comparison team.

Employment Outcomes

Employment outcomes are shown in Table 2. Overall, 43% of IPS participants achieved competitive employment compared with 21% of comparison participants (p<0.001). On average, 38% of IPS participants were in competitive employment each month compared with 18% of comparison participants (p<0.001). The IPS group had a significantly higher mean±SD number of job starts per month than the comparison group (3.4±2.7 vs. 0.9±0.9, p<0.001). Among those with jobs that ended during the study period (N=39), the IPS group had a longer mean job tenure than the comparison group: 134.1±133.6 days for the IPS group compared with 75.2±31.5 days for the comparison group (p=0.033). Finally, the groups did not differ in mean hourly wage ($13.25±$5.54 per hour) or mean hours worked per week in competitive employment (26.0±13.2).

Rates of competitive employment per month for each group are shown in Figure 1. In the first month of the observation period (July 2015), the IPS group’s rate was 14% (N=6 of 42) compared with 22% (N=10 of 46) for the comparison group. The IPS group’s rate increased across 25 months of observation to end at a high of 73% (N=51 of 70) in July 2017. Over the same period, the comparison group’s rate remained about the same, with declines in some months, ending at a low of 9% (N=6 of 69).

The results of the longitudinal RRM adjusted for propensity scores are shown in Table 3. Across the entire study period, participants in the IPS group were more than four times as likely to achieve competitive employment as participants in the comparison group. We noted a modest increase in the likelihood of competitive employment over time for both groups. Service month also was statistically significant, indicating that each additional month of employment services was associated with a substantial increase in the likelihood of obtaining competitive employment. Finally, caseload size was not associated with a change in likelihood of competitive employment.

Table 4 shows the rate of change in competitive employment over time, including the interaction of team assignment and time. At baseline, the IPS group had a significantly lower rate of competitive employment than the comparison group, and the rate of competitive employment in the comparison group declined over time. We observed a significant and positive interaction of IPS and time, indicating that, relative to the comparison group, the rate of competitive employment in the IPS group increased significantly over time. By multiplying the relative increase in the IPS group by the rate of competitive employment in the comparison group (1.32×0.91=1.20), we estimate that the odds of competitive employment in the IPS group increased by about 20% per month during the study period. Once again, participant service month was significantly associated with achieving competitive employment, whereas team caseload size was not.

Discussion

In this study, we evaluated the outcomes of peer-delivered IPS plus physical wellness support at a peer-run agency. Before the start of the study, both teams were providing supported employment services based on the choose-get-keep model. The team that adopted the IPS model demonstrated acceptable and increasing IPS fidelity after training began and during the 2-year observation period. Level of IPS fidelity has been shown to be related to superior vocational outcomes (42, 43), and the participants served by the IPS team also had increasingly positive competitive employment outcomes relative to recipients of existing employment services over the study period. Across the entire study period, 43% of the IPS participants obtained competitive employment compared with 21% of the comparison group. These relative rate differences are similar to those reported in a review of 28 studies of IPS (N=6,468), in which 55% of IPS participants achieved competitive employment compared with 25% of participants receiving other vocational services (1).

The two teams did not significantly differ in recipients’ hourly wages and number of hours worked per week; however, longer job tenure (among jobs that ended) was observed in the IPS compared with the comparison condition. Thus, the picture was mixed regarding job quality, at least as defined by tenure, earnings, and amount of work. Similarly, the teams did not differ in terms of caseload or number of clients served, making it unlikely that employment outcomes of the comparison team were influenced by greater staff burnout (44, 45).

Before the study started, the agency’s vocational model followed supported employment principles, including a focus on competitive employment, individual choice, and no time limits. Nevertheless, several challenges were encountered in moving to the IPS model. The first was a shift in the relationship between service providers and recipients. The traditional peer provider relationship emphasizes recipients taking the lead in making decisions and acting, whereas the IPS employment specialist role involves guiding the participant through a sequence of predefined service delivery steps. Peer IPS employment specialists needed additional training and support to feel comfortable taking the lead while also honoring peer support principles of mutuality, choice, and relationship building. The second issue was the absence of clinical treatment staff at the agency with whom to directly coordinate IPS services. To address this barrier, employment specialists sought and received clients’ permission to confer with their case managers, psychiatrists, and therapists via telephone and e-mail to discuss medication regimens, concurrent therapies, and other psychosocial employment supports. Another issue was the use of benchmarks to evaluate the job performance of employment specialists, which differed from the expectations held for staff in the generic supported employment model.

At the same time, the agency used features of the peer support recovery model that facilitated the transition to IPS service delivery. One was the existing emphasis on physical wellness in the larger program. Participants already appreciated the importance of health and wellness to recovery, making them receptive to the vocational health promotion component. Similarly, staff were comfortable providing health education and connecting it to achievement of employment goals. Another advantage of the peer support context was rapid engagement of people in IPS services, given that staff quickly established rapport with potential IPS recipients and were trusted given their association with the larger agency. Another feature that facilitated adoption of IPS service delivery was that employment specialists could model for their clients two important principles: that people in recovery can successfully hold competitive jobs and build lasting careers and that building intentional health habits and routines can contribute to vocational success.

One study limitation was our nonrandomized design, which precluded us from making causal claims about the intervention, although use of group assignment propensity scores is an accepted means of compensating for group differences in quasi-experimental designs (38, 39). A second limitation was the use of a single peer-run agency; in future studies, researchers should address potential variation in IPS adoption across different types of peer-run programs. A third limitation was sole reliance on administrative data; in future studies, researchers should include information about previous work history, Social Security Administration disability status, diagnosis and symptoms, disability severity, as well as staff members’ and participants’ satisfaction with peer-delivered IPS and their perspectives on the IPS model. A fourth limitation was our inability to account for the steep decline in the comparison group’s employment rate in the study’s final months. A fifth limitation was that training and supervision were provided over an extended period, and the costs and intensity of this level of support may exceed the available resources of many peer-run programs, limiting generalizability. A final limitation was our inability to separate out the impact of the two intervention components; thus, we cannot say with certainty whether IPS services alone or in combination with vocationally focused health promotion were active ingredients that influenced outcomes. A multiarm randomized study testing each intervention component separately is needed to gauge the value of health promotion as an adjunct to supported employment.

Conclusions

Peer specialists are increasingly employed in U.S. behavioral health service delivery systems (46, 47), especially now that peer support is a Medicaid-reimbursable service when included in state plans (48). The potential for peer specialists to help address behavioral health workforce shortages has led many states to invest in peer training and certification (49). At the same time, a recent survey of U.S. state vocational rehabilitation and behavioral health organizations (50) revealed an extremely low IPS penetration rate, even among states offering IPS services. Because one-quarter of all U.S. mental health facilities offer peer services (48), the organizational capacity exists for peer IPS service delivery. Ours is the most rigorous study to date of peer-delivered IPS, suggesting that it achieves better employment outcomes compared with peer-delivered generic supported employment. The fact that peers can provide IPS with high fidelity also has implications for the hiring of peer employment specialists in non–peer-run programs. We hope that in future studies, researchers will test the efficacy of peer-delivered IPS in different kinds of peer- and non–peer-operated organizations by using rigorous, multisite research designs. Finally, to adequately understand training needs, researchers should investigate the kinds and amounts of education and support required by peer-run programs seeking to adopt the IPS model.