Staff Time in the Community: An Enduring Critical Component of Individual Placement and Support in the Digital Age

The individual placement and support (IPS) approach to supported employment is considered the gold standard evidence-based practice (EBP) for supported employment (1–6). IPS is based on a well-defined set of principles and practices, and most IPS services are provided in the community. This emphasis on employment staff time in the community—meeting with IPS recipients and employers—is viewed as a key feature (4) and is highlighted in the IPS fidelity scale (https://ipsworks.org/wp-content/uploads/2017/08/IPS-Fidelity-Scale-Eng1.pdf).

The world was a different place when IPS was introduced more than three decades ago. Yet despite changes in the economy, employment opportunities, hiring practices, and the evolution of the Internet, IPS has continued to be the gold standard. That said, a reasonable question to ask is whether digital platforms (e.g., employers’ websites and online application processes) have diminished the importance of IPS employment specialists’ time spent in the community as a contributor to successful employment outcomes.

The Center for Practice Innovations is an intermediary organization (7) funded by the New York State Office of Mental Health to assist behavioral health care organizations across New York State to implement EBPs, including IPS, for adults diagnosed as having serious mental illness. Even before the COVID-19 pandemic, we noticed that sites were providing some employment-related services remotely and that some sites seemed to achieve good employment outcomes even when fewer IPS services than recommended were provided via the office site. We wanted to explore whether these observations were supported by data.

The focus of this study was to better understand the relationship between IPS employment specialists’ time spent in the community and employment outcomes in the current digital age, which features increasing use of technology and online hiring practices. Using data collected in 2018 and 2019, before the COVID-19 pandemic, we looked specifically at the time specialists spent providing IPS services in the community and its relationship to employment outcomes.

Methods

Community psychiatric rehabilitation programs were encouraged but not required to participate in this initiative. Each state facility was required to involve at least one outpatient clinic. In total, data from 78 sites in 2018 and 84 sites in 2019 were analyzed. (More detailed characteristics of 2018 and 2019 sites can be found in an online supplement to this report.)

Throughout 2018 and 2019, participating programs and clinics provided monthly IPS performance indicator data via the Qualtrics online survey platform. We chose a monthly reporting period, rather than the 3-month reporting period that is more typically reported in the literature, to closely monitor implementation efforts and quickly identify sites that would benefit from additional technical assistance. The monthly submissions provided information about several elements of IPS implementation, including IPS employment specialists’ time spent in the community and the number of employer contacts.

We specifically examined IPS employment outcomes and staff time spent in the community. For each year of up to 12 months of performance indicator data, the employment rate was calculated by dividing the number of people who were competitively employed (at least 1 day in any given month) by the caseload. We aggregated the mean monthly employment rate for each site and for each year. We also created a binary variable (1=yes, 0=no) to indicate whether a site met an employment rate of at least 41% for at least half of the reported data within a year. The 41% benchmark represents the 50th percentile found in a large national learning collaborative on supported employment for people diagnosed as having serious mental illnesses (8), albeit for a 3-month reporting period rather than the 1-month reporting period in this study.

The monthly indicator data for time spent by employment staff in the community was surveyed in four categories: ≤20%, 21%–40%, 41%–64%, and ≥65%. We collapsed the highest two categories into one (≥41%) because very few programs reported ≥65% time spent in the community. Specifically, about three programs per month in 2018 (mean±SD=2.9±1.9, median=3.0) and about two programs per month in 2019 (mean±SD=2.2±1.2, median=2.0) reported ≥65% time spent in the community. Across a year of reported monthly data, we created a categorical variable with four levels of time in the community. The first three levels, low (≤20%), medium (21%–40%), and high (≥41%), indicated that the monthly indicators fell into those categories in at least half of the months reported. For example, if a site reported 8 of 12 months of time spent in the community at ≥41%, then the site would fall within the high group for the year. A fourth category, “mixed,” indicated that the site did not fall within any of the low, medium, or high categories for at least half of the months reported, indicating that the times reported by these sites were highly variable throughout the period examined.

Within each year, we applied a Pearson’s chi-square test on a 2×3 table of employment outcomes (i.e., met the benchmark for an employment rate of ≥41% at least 50% of the time, yes or no) and most staff time spent in community (low, medium, or high). We excluded programs (N=4 in 2018, N=2 in 2019) in the mixed category of staff time spent in the community from these analyses. We also applied a linear mixed-model analysis to examine the effect of the amount of time staff spent in the community (low, medium, or high) on mean monthly employment rate, with time (year of participation) as a repeated effect. The New York State Psychiatric Institute Institutional Review Board determined that this evaluation did not meet the definition of human participants research.

Results

We found a significant relationship (p=0.009 in 2018 and p=0.001 in 2019) between most of IPS staff time spent in the community (low vs. medium and high) and better employment outcomes (see Table 1). For 2018, 74% of sites in the medium and 71% of sites in the high categories had met the benchmark for a good employment rate (≥41%) for more than half of the time during the year, compared with 39% of sites where most of staff time spent in the community was in the low category. For 2019, the trend was similar, with 76% of sites in the medium community time category and 85% of sites in the high community time category meeting the benchmark for a good employment rate more than half the time during the year, compared with 41% of sites in the low category. In a mixed-model analysis with repeated measures across both years examining mean employment rate and most of staff time spent in the community, sites in the low community time category had statistically significantly worse employment outcomes than sites in the medium (p=0.009) and high (p=0.004) community time categories.

Discussion

We observed a significant and positive relationship between the time IPS employment specialist staff spent in the community and better employment outcomes for IPS recipients, a finding that reinforces the value of community-based over office-based services observed across EBPs (9, 10). At the same time, we note that very few IPS sites reported that staff spent ≥65% of their time in the community, which represents the highest rating on the fidelity scale. In fact, sites reporting as little as 21%–40% of staff time in the community achieved good employment outcomes, as shown in Table 1. If this finding can be replicated, it would suggest that although time in the community is essential, positive employment outcomes may be achieved in settings where staff must limit time in the community (i.e., 1–2 days per week). It also suggests a need to look more closely at this fidelity item. If programs are achieving good employment outcomes with less time in the community, it is possible that the standard of ≥65% of time in the community is too high.

One limitation of the study was that all participating sites were in one state and received training and implementation support from the same intermediary organization. Therefore, we are not certain whether these results would generalize to other settings and training entities. In addition, we relied on self-report for all data, and we were unable to validate the accuracy of these data. However, previous studies have reported that the self-assessments of fidelity of these programs did not differ from assessments of objective raters (11) and were positively related to employment outcome (12). Furthermore, no benchmarks for monthly employment rates are currently available in the literature. We applied the ≥41% benchmark rate, typically applied over a 3-month period but in our study applied over a 1-month period. Future studies including both monthly and quarterly measures would be helpful to establish appropriate monthly benchmarks.

The findings from this study point to several gaps in the literature and additional suggestions for future research. For example, about 40% of IPS sites in 2018 and 2019 that reported that staff spent low levels of time in the community also reported employment outcomes of ≥41% for at least half of the reported data within a year. This finding may be due to several reasons—local employment rates and the nature of the community, among others—and it would be informative to study these programs in the future to gain a better understanding of what characterizes these sites and what adaptations they may have made that seemingly contributed to their success. It is also possible that higher fidelity on other items, as well as other innovations not captured with the current fidelity scale, compensated for the lower amount of time staff spent in the community. Findings obtained prepandemic, specifically regarding the employment market, may or may not be relevant going forward postpandemic. As the postpandemic world unfolds, we must learn about continuing changes in IPS service delivery and the employment market as well as adaptations that may prove helpful to training of IPS employment specialists.

Conclusions

The amount of time IPS staff spent in the community with employers and IPS recipients was positively associated with better employment outcomes, highlighting the continued importance of employment specialists’ spending time in the community to achieve optimal outcomes for IPS recipients.