A Case Report of the Conversion of Sheltered Employment to Evidence-Based Supported Employment in Canada

Bolstered by consistent findings from nine randomized controlled trials, supported employment, as defined by a set of evidence-based principles, is now recognized as an evidence-based practice for persons with severe mental illness (1,2). As articulated by Becker and Drake (3), the individual placement and support model of supported employment incorporates six commonly accepted principles: competitive employment is the goal, the job search occurs soon after program entry, eligibility is based on consumer choice, job choice follows consumer preference, support is provided over time and is based on consumer need, and vocational and mental health services are integrated.

Among the many programs that identify themselves as providing supported employment, the principle that they most often lack from the individual placement and support model is the integration of vocational and mental health services (4), even though close coordination between the employment staff and the mental health treatment team can yield tangible benefits in terms of program effectiveness (5).

A major issue in implementing evidence-based practices concerns identifying strategies that will be palatable to provider agencies. In some cases agencies create entirely new programs in response to perceived needs; in many such cases the new program does not replace or compete with an existing program. Resistance to change may be minimal in such circumstances, because introducing the new services may not require abandoning ideological viewpoints or relinquishing habitual patterns of providing services. In other cases the new, evidence-based practice replaces existing program services. The conversion of an existing program challenges cherished beliefs and is often difficult to implement, unless there is dissatisfaction with existing services (6,7).

In the vocational domain, early proponents of supported employment believed that it would logically replace the facility-based programs that had dominated the vocational field for many decades. For individuals with developmental disabilities, there have been some dramatic examples of converting sheltered workshops to supported employment (8). However, the rehabilitation field has found that supported employment has generally been added as a new service to comprehensive rehabilitation centers instead of replacing sheltered workshop and day programs (9,10).

In the mental health field some noteworthy demonstration studies have shown the successful conversion of day treatment programs to supported employment (11,12,13,14,15). Although a sheltered workshop for individuals with psychiatric disabilities in the United Kingdom made a successful conversion (personal communication, Rinaldi M, 2004), to our knowledge, there have been no published reports documenting the conversion of a sheltered workshop to an evidence-based supported employment program for this target population.

Case study

The Canadian Mental Health Association (CMHA) is a voluntary association that exists to promote the mental health of Canadians. The Vancouver/Burnaby branch of CMHA has been providing vocational services since the late 1980s, although it does not offer mental health treatment, which is under the jurisdiction and financing of the provincially funded health authorities. The case study presented here examines the transformation of the employment services of the organization over a 15-year period as it made successive steps toward improvement.

Phases of the evolution

Phase 1: facility-based sheltered employment. Starting in the late 1980s, CMHA provided vocational services to clients with severe mental illness in Vancouver and Burnaby (a suburb of Vancouver). Three vocational counselors, one business representative, and one coordinator staffed the program. The service offered packing and light assembly work. Clients worked one or two days each week and earned $100 each month. Although clients thought of themselves as working and contributing to the business community, there was little movement from the workshop into competitive employment (less than 5 percent annually).

Phase 2: addition of prevocational component. In the early 1990s CMHA expanded the services to include employment preparation classes devoted to self-paced life skills training and computer training. No new staff were dedicated to helping clients obtain competitive employment. Although significant changes were observed in client behaviors as a result of these classes, no increase was seen in the number of clients who made the transition into competitive employment.

Phase 3: brokered supported employment services. In 1995, after discussions with contract managers from the health authority, CMHA decided to convert its focus to supported employment. The business representative role was shifted to that of job marketer, with vocational counselors providing ongoing support to clients once they obtained jobs. The program coordinator continued to manage the team and assess new admissions. The employment preparation classes continued, and most clients attended them for six weeks.

Around the same time, CMHA started an independent employment program based in Vancouver to help clients who were more "job ready" secure competitive employment. The differences between the independent employment program and the supported employment program became less clear as they became more established. By 1997 they were merged into one program.

Although these reforms were moving in the right direction, the merged program did not achieve the desired level of success. Even though more clients were entering competitive employment, the annual rate of clients who achieved competitive employment remained below 25 percent. Moreover, administrators noted structural problems in the program. One barrier was that the role of case managers in the vocational process was largely limited to referring clients to the program and providing baseline information about the client's background and treatment history. They rarely met face to face with vocational counselors, and they did not collaborate with the counselors on plans to support the client. A second issue was that the two job marketers could not keep pace with the backlog of clients who were ready for employment. Clients became discouraged with the delay in finding placements. Consequently, the program had a high dropout rate.

Phase 4: individual placement and support model. In 2000, CMHA and contract managers from CMHA's funding agencies reviewed the supported employment program. Influential in this discussion was the emerging literature on the individual placement and support model of supported employment. The review group recommended converting the brokered supported employment program (in which vocational services were centralized at the CMHA offices) to an integrated supported employment program (in which employment counselors were deployed to the various mental health treatment teams). In collaboration with the mental health provider organizations in Vancouver and Burnaby, CMHA committed to a pilot study involving complete conversion to the individual placement and support model. Specifically, CMHA would continue to manage the program but would implement the changes in partnership with specific community mental health teams, following a jointly developed operational plan.

Shift to evidence-based supported employment services

The shift to individual placement and support required significant planning and organizational change. Several key milestones were involved. First, staff roles were redefined so that the newly titled vocational rehabilitation counselors undertook responsibility for the full range of services to their caseload—including admission through assessment, marketing and placement, and ongoing support—following the generalist model advocated by the individual placement and support model. This reorganization required a change in position descriptions and complex negotiations with union and employer representatives to reclassify the jobs to a new collective agreement and to ensure the transfer of existing staff. To fund an increase in wages, CMHA agreed to reduce the staffing from seven full-time employees to five full-time employees and one half-time employee (this was achieved without layoffs).

A second key milestone was the training and orientation of both vocational and mental heath staff, which were provided by an outside consultant, a senior coordinator of an established individual placement and support program in Manchester, New Hampshire. This consultation included a three-day training event, followed by a meeting with the participating mental health treatment teams and a community seminar for other vocational service providers in the Vancouver and Burnaby area. The consultation was perceived as extremely helpful in legitimizing the individual placement and support model. The consultant gave concrete examples of what previously had been only a theoretical model for local staff, partners, and funders. One significant shift in the operational plan that resulted from the consultation was the decision to create a half-time team supervisor position to provide vocational team supervision, ensure communication with the treatment teams, and to coordinate the evaluation. (The supervisor was employed full-time, with 50 percent of the time devoted to direct service for supported employment clients.)

Third, operational plan development revolved around the formation of two advisory committees for the two regions (Vancouver and Burnaby) that were involved in the project. The two committees approved almost identical operational plans, with the exception of admission procedures. The Vancouver committee decided that clients would be referred by case managers and admitted as openings became available, whereas the Burnaby committee decided that clients who had been on a waiting list would be served on a first-come-first-served basis.

Between November 2000 and March 2001 the four existing CMHA vocational counselors were transferred to mental health treatment teams. In subsequent months, an additional one full-time counselor and one half-time counselor were hired: the full-time counselor worked at a fifth team, and the half-time counselor increased the capacity at one of the original four teams. From the inception of the program, vocational counselors continuously educated case managers to help them understand the program and referral process. Although the vocational counselors worked out of the treatment team offices, the entire group of vocational counselors continued to meet weekly for group supervision and peer support, consistent with the individual placement and support model. Staff reported that they perceived this continued connection as a specialized vocational team as an important factor in their effectiveness. Team meetings helped staff clarify and maintain high fidelity to individual placement and support principles, encouraged peer support, and provided a forum for case reviews and joint problem solving. This weekly meeting also allowed the vocational counselors to make practical arrangements for staff coverage in the case of absences.

Project evaluation

Procedure

Project staff conducted a program evaluation over a 27-month project phase (January 2001 to March 2003) for five project sites. During this time, they systematically examined client background characteristics, competitive employment outcome rates, and client and stakeholder satisfaction. Satisfaction was measured by using an 11-item checklist—which included four items that were answered as yes, no, or partially and seven items that were measured with a 5-point Likert scale (1, excellent; 2, very good; 3, good; 4, fair, and 5, poor)—and by using an additional item regarding wait time. The stakeholders included occupational therapists, case managers, and team directors from the treatment teams and contract managers from the health authorities.

In addition, fidelity of implementation was assessed by using the 15-item Individual Placement and Support Fidelity Scale (16). Each item is rated on a 5-point, behaviorally anchored scale, for a maximum possible score of 75. An item score of 5 indicates that the program has fully implemented the individual placement and support standard, a score of 4 indicates moderate implementation, and a score of 1 indicates the program has not met the standard at all. The first author conducted the fidelity assessment, using operational policies, employment data, and other information provided by the supported employment team. A total score of 66 or higher is considered good fidelity to individual placement and support.

Communities served

The communities served by the five teams differed in certain ways. The socioeconomic and educational levels of clients served by the West Side Vancouver team were relatively high. Although this environment fostered expectations of higher wages for work, motivation to work was sometimes undermined by financial support to clients from families.

The Northeast Vancouver team served a lower-income neighborhood of the city that had a high degree of transience. The vocational counselor had a number of clients whom she continued to follow even though they moved to distant neighborhoods outside the team's appointed area. This community also has a substantial Asian population. The individual placement and support program accommodated a large percentage of clients whose first language was not English by enlisting the help of mental health staff who spoke the client's native language.

The South Vancouver team served the area in the most southerly part of Vancouver. This community was culturally diverse with sizeable Indo-Canadian and Asian populations. These multicultural communities often have a strong work ethic. If this ethic is combined with a lack of understanding of mental illness, families can have difficulty understanding why the person is not already working. This team had a multicultural worker who provided a bridge to clients who did not speak English.

The Central Burnaby team served an urban area adjacent to Vancouver that had a mix of economic and multicultural communities.

Unlike the other sites, Maple Ridge is a small town that is surrounded by a semirural area and is located to the east of Vancouver and Burnaby. The vocational counselor of the Maple Ridge team reported that the challenge of identifying potential jobs was significant in a smaller economic market.

Sample

Over the project period, 249 clients were admitted to the individual placement and support program—129 during the first 15 months and 120 over the final 12 months. The 195 clients who were enrolled for six or more months in the individual placement and support program during the project constituted the primary focus of the outcome analysis. Forty-three clients were seen by the Westside Vancouver team, 54 by the Northeast Vancouver team, 26 by the South Vancouver team, 41 by the Central Burnaby team, and 31 by the Maple Ridge team. Among the 195 clients, 113 (58 percent) were women, 111 (57 percent) were 40 years or younger, and educational attainment varied from less than high school (48 clients, or 25 percent) to college graduates (62 clients, or 32 percent). Ninety-nine (51 percent) had a diagnosis of a schizophrenia spectrum disorder, 76 (39 percent) had never worked or had not worked in the past five years, and 67 (34 percent) did not speak English as a first language.

Project findings

Program fidelity. The individual placement and support fidelity score was 68, which suggested good implementation of the individual placement and support model. The single item that was scored below 4 was community-based services, which suggested that the amount of time that employment specialists spent outside of the office was suboptimal.

Competitive employment outcomes. Among the 195 clients in the sample, 27 (14 percent) were employed at the time of program entry. Among these 27 clients, 12 (44 percent) were working at the end of the project period, nine (33 percent) were working at the time they left the program, and the remaining six (22 percent) were still actively searching for a new job at the end of the project period.

Of the 168 clients in the sample who were unemployed at the time of program entry, 84 (50 percent) were competitively employed at some time during the study period. An additional 29 (17 percent) were still actively searching for a new job at the end of the project period, while 56 (33 percent) were terminated without obtaining competitive employment. The competitive employment rates were similar for the five treatment teams, ranging from 46 to 55 percent. At the end of the project period, 62 clients (37 percent) were competitively employed (ranging from 32 to 46 percent for the five teams). A total of 107 jobs were held by the 84 clients who entered the program unemployed and who were employed at any point during the project period. Of these 107 jobs, 79 (74 percent) were 15 hours or more a week and 33 (31 percent) were full-time jobs; 65 (61 percent) earned a monthly wage of at least $800 (Canadian). All the positions paid competitive wages.

Among the 84 clients who entered the program unemployed and who found work at some point during the project period, 62 (74 percent) did so within six months after program admission. Staff individualized their job search approaches: 19 clients (23 percent) received mostly hands-off consultative support, 35 (42 percent) received intensive coaching, and 30 (36 percent) received both intensive coaching and direct marketing to employers on the client's behalf.

Client satisfaction. In the summer of 2003, satisfaction surveys were sent to 251 clients involved in the pilot project, of which 53 (21 percent) were returned. For the item "How would you rate the overall service you received from this program?" 20 (40 percent) answered excellent, 14 (28 percent) answered very good, 11 (22 percent) answered good, three (6 percent) answered fair, and two (4 percent) answered poor. (Three responses were missing.) For the item "Were your employment goals met?" 21 (40 percent) indicated yes, 20 (38 percent) indicated partially, and 11 (21 percent) responded no. (One response was missing.) Thirty-three of 46 respondents (72 percent) indicated that they waited two weeks or less to first meet with their vocational counselor.

Stakeholder satisfaction. An independent facilitator was hired to gather stakeholder input during a workshop held in April 2003. Attending were 15 stakeholders—including case managers, occupational therapists, and program managers from the five mental health treatment teams—and contract managers from the health authority. Twelve stakeholders rated their overall satisfaction with individual placement and support programs as follows: one person (8 percent) was extremely satisfied, five (42 percent) were very satisfied, four (33 percent) were satisfied, one (8 percent) was somewhat satisfied, and one (8 percent) was not satisfied. The workshop focused on a range of specific issues related to employment specialists' integration onto treatment teams, the referral process, staff selection, documentation, and organizational structure. In each of these areas, specific recommendations were developed for incorporation into future quality improvements.

Discussion and conclusions

This article describes one agency's successful efforts in converting to evidence-based supported employment. Of interest was the finding of relatively similar outcomes in five dissimilar neighborhoods, suggesting that community factors were not an insurmountable barrier to implementation.

Several factors help to explain the process of change. First, the agency was committed to quality improvement, using both process and outcome data to guide its practice. Regarding process measures, during the facility-based period of development, one chronic problem was staff turnover. During the brokered supported employment phase, staff continued to be demoralized because, while they felt proud of their individual role performance, they were disappointed by their lack of tangible results. In addition, collaboration between the employment program and the community treatment teams was inconsistent. Each phase of organizational change also was prompted by the agency's dissatisfaction with its competitive employment rates. Rapp (17) has long advocated for the power of outcome supervision and planning as a means to improving services, and this case study supports this view. Second, the agency was inspired and guided by information on the individual placement and support model that appeared in a special issue of Psychiatric Rehabilitation Journal (18). At the time, because staff were unhappy with their outcomes, they were receptive to change. They decided to reconfigure their services to align with evidence-based principles. Third, strong leadership, encouragement, and financial support from local health authorities made organizational change viable. Fourth, the agency sought and received expert consultation from an experienced individual placement and support team leader, who provided the vocational staff with the confidence that the individual placement and support program could be implemented and who met with mental health treatment team staff to build consensus. Fifth, the agency devoted substantial time to consensus-building and training before launching the new project.

As with any case study, it is a matter of speculation how replicable this process might be in different circumstances. The unique characteristics of the current organization might well foster questions about the generalizability to other settings, particularly to programs in the United States. What this case study suggests is that conversion from sheltered employment to evidence-based supported employment is possible when there is felt need for change, key leaders are motivated, and a systematic change process is followed. •(c)

Acknowledgments

This study was supported by Vancouver Coastal Health and Fraser Health Authority, with additional assistance from the United Way of the Lower Mainland.

Mr. Oldman and Ms. Thomson are affiliated with the Canadian Mental Health Association, Vancouver/Burnaby Branch, 175 West Broadway, Vancouver, British Columbia, Canada V5Y 1P5 (e-mail, [email protected]). Ms. Calsaferri is with Vancouver Coastal Health Authority. Ms. Luke is with Fraser Health Authority in Vancouver. Dr. Bond is with the department of psychology at Indiana University-Purdue University Indianapolis.