The Use of Standardized Discharge in IPS Supported Employment Programs

Individual placement and support (IPS) has been established as the most effective form of vocational rehabilitation for job seekers with serious mental illness (1–3). IPS research has focused on assessment of vocational outcomes in randomized controlled trials (RCTs). In these trials, about 45% of recipients remain unemployed throughout follow-up. However, these RCTs use intent-to-treat designs and do not explicitly allow for recipient discharge. That is not the case in routine practice settings, where IPS programs routinely discharge recipients to create new capacity. Recipients who disengage from services, are unsuccessful in gaining employment, or gain employment and no longer require supports are typically discharged, either permanently or temporarily, when the client, the treatment team, or both determine that there is no benefit to be gained from continued participation (4).

This natural attrition of IPS recipients stands in contrast with standardized discharge policies based on individual employment outcomes. Standardized discharge policies may improve cost-effectiveness compared with natural attrition, but they may also ignore information considered important by clients and teams for evaluating individual outcomes.

The only study of standardized IPS discharge illustrates the tension between maximizing program outcomes through a more efficient use of limited resources and delivering services of the highest quality to every enrolled recipient. The study tested whether IPS using standardized discharge conditions based on consecutive periods of both unemployment and employment (IPS-Lite) was more cost-effective than standard IPS, which has no standardized discharge policies (5). Under the IPS-Lite discharge conditions, recipients who remained unemployed for nine months after enrollment or who achieved four consecutive months of employment were discharged. The nine-month time limit was based on previous research showing that most clients in IPS who gain employment do so in the first nine months (6). Assuming that replacement enrollees would have attained the same rate of employment as the program’s initial enrollees, the investigators estimated that recipients of IPS-Lite would have found 17% more first jobs than recipients of standard IPS. At that rate, automatic discharges would have improved overall program cost-effectiveness, as long as new recipients enrolled in each program by filling the capacity created by attrition.

Recipients of IPS-Lite were less likely to find a job than recipients of standard IPS (odds ratio=.82), although the difference was not significant. The nonsignificance of this effect may have been due to the small sample size. The IPS-Lite study was not designed to test a hypothesis of noninferiority, and, with only 123 participants, it lacked the power to test a finding of no difference (7,8). The possibility of superior vocational outcomes among those not discharged from IPS suggests that employment-based discharge conditions may unfairly penalize those who require more time to benefit from IPS.

In this study, we examined the related hypothesis that the potential benefits of IPS, as measured by time to first job, are confined to the first nine months of treatment. If the benefits of IPS are not confined to the first nine months, automatic discharge of recipients who had not found employment after nine months would place these recipients at a disadvantage compared with IPS recipients not subjected to the nine-month rule.

Methods

Using Mental Health Treatment Study (MHTS) data (9), we compared the quarterly probabilities of obtaining a first job for the IPS and control groups and determined the periods of initial unemployment during which IPS recipients continued to benefit from enrollment. The MHTS was a two-year RCT comparing the effectiveness of an intervention consisting of IPS, systematic medication management, and other behavioral services with treatment as usual (control) at 23 sites in the United States. Study participants included adult beneficiaries of Social Security Disability Insurance who had not worked during the 30 days prior to enrollment, had a primary diagnosis of schizophrenia or mood disorder, and lived within the catchment areas served by the study sites. The final analysis included 2,055 participants. Participants reported on their recent employment history every three months (9). For this time-to-event analysis, we defined an event as the first quarter during which a participant reported at least one day of competitive employment. All procedures contributing to this work complied with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

Our analysis of the time to acquisition of a first job among recipients of the IPS intervention yielded both the survival function (overall proportion of the original sample obtaining a first job by a given quarter) and the quarterly probability of obtaining a first job (among those yet to find a first job). We used the Kaplan-Meier survival function in this analysis because this nonparametric method imposes no assumptions on the treatment-specific survival functions and the relationships between them. Naïve bootstrapped resampling of the intervention and control outcome data provided the estimates and confidence intervals. For each treatment condition, we conducted 1,000 bootstrapped resamplings of the same size as the observed data sets, computing, for each resampling, the Kaplan-Meier survival functions and corresponding quarterly probability of finding a first job among those who had yet to find a first job. We then randomly paired each of the 1,000 IPS and control resamplings and computed the quarterly difference in probability of finding a first job for each pair. To provide confidence intervals for estimates of probability by quarter of finding a first job and to test for the significance of the differences between those probabilities, we determined the 95% confidence intervals (CIs) for the treatment-specific probabilities and the differences in probability for each quarter based on each statistic’s observed distribution across 1,000 resampled data sets.

Results

Figure 1 shows treatment-specific probabilities by quarter of finding a first job among those yet to find first jobs. Among recipients of the intervention, the median quarterly probability of finding a first job decreased from .15 (CI=.13–.18) in the first quarter to .04 (CI=.02–.06) in the last quarter. In the control group, the quarterly probability of finding a first job decreased from .10 (CI=.08–.12) to .02 (CI=.01–.03). Throughout the first 18 months of follow-up, IPS recipients were significantly more likely to find a first job than the control group. During this period, IPS recipients (N=1,004) found 493 first jobs, compared with 325 in the control group (N=1,051). Between the ninth and 18th months of follow-up, and despite a smaller sample size, IPS recipients found 62 more first jobs than did the control group. [A quarterly survival function and a table describing the quarterly probability of finding a first job in the original data set are available in an online supplement.]

Discussion

Our analysis of the large-sample MHTS data showed that IPS recipients, compared with participants in the control group, were more likely to find a first job over 18 months. This finding suggests that making continued enrollment in IPS solely contingent on finding a first job within the first nine months would compromise the long-term employment outcomes of those discharged.

Natural attrition, the de facto means of discharge at most IPS programs, is based on individualized decisions made by recipients and their treatment teams. As such, it draws on the full range of available information, including recipient engagement, to determine continued enrollment status. Given the difficulty of studying the deliberative processes underlying natural discharge practices or the program-level retention patterns resulting from these practices, coupled with the need to make the most efficient use of the limited resources available to IPS programs, we must consider the utility of standardizing discharge policies based on objective measures of recipient activity prior to any widespread adoption. Basing recipient discharge on standardized conditions can guarantee a stable program-level enrollment and discharge dynamic, enable optimization in favor of a given outcome measure, and ensure that more people will receive IPS services. However, this approach prioritizes program efficiency over the outcomes of enrolled individuals and has no provision for other recipient characteristics to affect discharge decisions.

For example, a rule limiting participation in IPS to nine months disregards engagement level, ensuring both that disengaged recipients may remain enrolled for nine months and that engaged recipients who have yet to find a first job are discharged after nine months. In the MHTS, more than half of the nonworkers either had not engaged in IPS or disengaged during the study (10). Despite the inclusion of the disengaged recipients in the primary analysis, those enrolled in IPS continued to benefit relative to control participants for at least 18 months. Thus, a more nuanced approach to discharge, in which those who remain interested in working are retained in IPS, while others leave naturally, may be appropriate ethically and clinically.

Other real-world studies show that IPS specialists typically help clients to discontinue services or transfer support to case managers in much less time than 18 months (4). The extent to which natural attrition affects the cost-effectiveness of IPS compared with standardized discharge policies remains an open question. Also unknown is how well natural attrition, compared with standardized discharge policies, predicts the employment potential of IPS recipients.

Limitations of our study included the characteristics of the MHTS sample, the imposition of experimental conditions, and potential confounding of the effect of IPS by supporting services such as systematic medication management. Nevertheless, the MHTS included a large group of participants with significant geographic, ethnic, and racial diversity (9), and there is no evidence that the supportive services offered alongside IPS enhanced employment outcomes. We therefore conclude that our findings support continued reliance on clinical decision making as ethically defensible and, in the absence of contradictory data, the most efficient means of evaluating the continued enrollment of IPS recipients.

Conclusions

IPS recipients who do not find employment within nine months of enrollment may continue to benefit for several additional months. In real-world settings, IPS teams create new capacity naturally by allowing those who lose interest or who are not benefiting to drop out, at least temporarily, or, if they are working successfully, to discontinue IPS services and transfer support to case management. Although IPS programs must discharge some enrollees in order to provide services to additional recipients and improve cost-effectiveness, a Procrustean bed may foster disability rather than recovery. Further research into the potential gains in cost-effectiveness associated with a range of discharge practices, whether informal or standardized, will help IPS practitioners understand and minimize the impact of those practices on those who may not benefit as quickly from IPS.