Implementing Illness Management and Recovery Within Assertive Community Treatment: A Pilot Trial of Feasibility and Effectiveness

Despite advances in pharmacological treatment, many individuals with serious mental illness experience significant functional impairments, severe psychiatric symptoms, and frequent rehospitalizations. Effective psychosocial treatments can improve functioning; however, only a small percentage of people receive those treatments (1–3).

Assertive community treatment (ACT) teams serve individuals with the most severe symptoms, who are often difficult to engage in services (4). Controlled studies show that ACT improves outcomes, including rehospitalization, housing, and treatment retention, but that it is less effective in improving psychiatric symptoms, social functioning, and other functional outcomes (5–15). Furthermore, ACT has been criticized for not being recovery oriented (16,17).

Illness management and recovery (IMR), a curriculum-based program designed to help individuals pursue personal recovery goals (18), seems well suited for enhancing outcomes in areas where ACT is less effective. A 2014 review of experimental studies reported that clients who received IMR in community-based clinics (19,20) and in supported housing (21) showed significant positive effects on illness self-management, clinician-rated symptom severity, and psychosocial functioning compared with clients in treatment as usual or a waitlist control group, respectively. A more recent randomized controlled trial (RCT) comparing IMR to an active control group showed no significant differences, but participation rates in both treatments were low (22).

Several characteristics of ACT suggest that it may be a promising platform for IMR. These characteristics include a flexible nature, which allows IMR to be delivered in both individual and group modalities; a community-based approach, which provides more opportunities for practicing IMR skills in natural settings; and a focus on working with natural supports, which creates opportunities to martial extra support to help individuals achieve recovery goals.

Although IMR has been implemented and evaluated within ACT, several implementation and methodological issues prevent strong conclusions about the effectiveness of combining them. Two quasiexperimental studies of IMR embedded in ACT (hereafter referred to as “ACT+IMR”) found significant reductions in hospitalizations (23,24); one study also showed significant reductions in substance use (23), and the other study also showed significant reductions in emergency room visits (24). However, in both studies IMR was implemented only by specialists (one peer, up to two clinicians) rather than by training the whole team, and both studies lacked well-developed guidelines for implementing IMR. These and similar studies (25,26) suggest that IMR can be successfully integrated into ACT services, but only if careful implementation and integration of both are made.

Our team undertook a series of research and development activities for implementing ACT+IMR, including developing a manual for implementing IMR within ACT teams (unpublished report, Gingerich S, Miller J, Monroe-DeVita M, et al., 2013), conducting a small-scale, open pilot test of this manualized approach, and conducting a qualitative process evaluation to identify barriers to, facilitators of, and advantages of implementing IMR within ACT. In this article, we report on a pilot evaluation of ACT+IMR in a small-scale, cluster randomized clinical trial that aimed to provide data about the feasibility of implementing the program and preliminary data on its effectiveness (27).

Methods

A pilot cluster randomized controlled trial was conducted in which ACT teams were randomly assigned to provide ACT+IMR or standard ACT (ACT only). The impact of ACT+IMR versus ACT only on IMR outcomes was based on assessments conducted with subsets of randomly selected clients from each team at baseline and six-month and one-year follow-ups. The study was approved by the institutional review boards affiliated with the two principal investigators (University of Washington, Community Alternatives and Places for People).

Study Teams and Randomization

Eight ACT teams in two states, four teams per state, were recruited. Selection criteria included no prior IMR training and good fidelity to ACT during state-sponsored fidelity assessments in 2012 (score of ≥3.5 out of 5.0 on the Tool for Measurement of Assertive Community Treatment [28,29]). The mean±SD fidelity score for teams assigned to ACT+IMR was 4.11±.26, compared with 4.02±.35 for teams assigned to ACT only.

Four teams served 80 to 100 clients each; four served 45 to 50 clients each. Randomization to ACT+IMR or ACT only was stratified by state and team size, resulting in one large team and one small team assigned to each condition in each state. ACT+IMR teams were compensated financially for service reimbursement that was lost because of staff training time.

Participants

Twelve to 15 clients from each ACT team were randomly selected for recruitment, based on the following criteria: chart diagnosis of schizophrenia, schizoaffective disorder, or bipolar disorder; ACT admission at least 60 days prior to the study; and projected length of stay in ACT for at least 12 months. Researchers met potential participants to explain the study, inquire about participation interest, and obtain written informed consent from those interested. No clients refused, although one was replaced because of a program transfer.

The study enrolled 101 participants, 53 in ACT+IMR and 48 in ACT only, with a mean age of 43.9±11.6; this sample size would require effect sizes equal to Cohen’s d of ≥.56 to achieve power of .80, before the analyses controlled for baseline covariates and random effects of treatment site.

Treatment Conditions

ACT is a multidisciplinary, team-based approach to providing treatment, rehabilitation, and support to people with serious mental illness, who experience significant functional impairments and high service utilization. Most services are provided in the client’s home or community, and services are available 24/7. IMR follows a manualized, 11-module curriculum to help individuals pursue personal recovery goals and to teach them information, strategies, and skills via group or individual format to manage their psychiatric illness.

ACT+IMR was developed and manualized for this study (unpublished report, Gingerich S, Miller J, Monroe-DeVita M, et al., 2013). This model involves providing IMR training to all ACT team members in the ACT+IMR condition; ACT+IMR specialists provide individual and group-based IMR, and all staff provide community follow-up assistance (for example, role-playing) to assist individuals with practicing IMR skills and pursuing recovery goals. ACT+IMR teams communicate regularly (for example, during daily meetings) regarding participants’ IMR goals, progress, and follow-up interventions. The ACT team leader, also trained as an ACT+IMR specialist, provides regular IMR supervision.

The ACT+IMR teams received training that included written and video materials on IMR, the ACT+IMR treatment manual, a two-day training provided by IMR and ACT experts, a one-day booster training conducted six to eight months after start-up, and consultation by an IMR expert twice a month for the first six months and monthly for the second six months of implementation.

ACT-only teams provided usual ACT services, receiving no IMR training during the study period.

Outcome Measures

Masters-level interviewers were trained to administer standardized outcome measures. Interviewers conducted face-to-face interviews and were not blinded to treatment condition. Participants were paid $15 at baseline, $20 at six months, and $25 at 12 months.

Illness self-management.

The clinician and client versions of the IMR Scale evaluate illness self-management across 15 items, each rated on a 5-point behaviorally anchored scale, with higher scores indicating better illness management. Overall scores are sums of the 15 items (each ranging from 1 to 5, with a possible total score of 75) (18). Clinician ratings were completed by the ACT team member other than a primary IMR provider who had the most knowledge about the research participant; primary IMR providers were excluded from completing clinician ratings to minimize rater bias. Client ratings were completed by each participant. The IMR scales have strong psychometric properties (21,30–32).

Mental health symptoms.

The expanded Brief Psychiatric Rating Scale (BPRS) (33) is a semistructured interview with 24 items, each rated on 7-point Likert-type scale. Higher scores indicate greater symptom severity. The measure is reliable (34) and sensitive to change following IMR (21).

Psychosocial functioning.

Research interviewers rated participants’ functioning by using the Daily Living Activities Scale (DLA-20), the Global Assessment of Functioning (GAF), and the Quality of Life Scale–Abbreviated (QLS-A) (35–37). The DLA-20 is a functional assessment consisting of 20 items, each measured on a 7-point Likert-type scale. The DLA-20 has adequate internal consistency and interrater reliability (38). The GAF is a widely used measure of psychological, social, and occupational functioning with good reliability and validity (39,40). Scores range from 0 to 100; higher scores indicate better functioning. The seven-item QLS-A (35–37) has predictive validity similar to the longer version (36). Items are rated from 0 (virtually absent/low) to 6 (adequate/high), with questions focusing on social functioning, motivation, and positive emotions.

Recovery.

The Recovery Assessment Scale (RAS) includes 41 items, each rated on a 5-point Likert-type scale. The RAS has good psychometric properties and is sensitive to change after IMR (20,30); the measure’s total score was used as a primary outcome.

Community integration.

Participants’ involvement in community activities was rated by using the Community Integration Measure (CIM). The CIM consists of 10 items, each rated on 5-point Likert-type scale, with higher scores indicating poorer community integration. The CIM has shown good internal consistency (41).

Emergency mental health services.

We examined emergency room and hospital admissions that were associated with mental health reasons. Research staff collected these data from ACT staff for the 12-month study period.

Statistical Analyses

We compared the two conditions on baseline demographic, clinical, and outcome measures, using t tests for continuous measures and chi-square tests for categorical measures. We used repeated-measures analysis of covariance (RM-ANCOVA) in a mixed-effects regression context to test for differences between groups at six and 12 months. Baseline scores on each outcome were entered as covariates; ACT team (that is, site) was specified as a random effect to control for heterogeneity between teams. (Whether to control for site in a small-scale, clustered RCT is a matter of debate; thus, we analyzed the data both ways. Results were similar, except the analysis that did not control for site showed one additional significant finding: improved QLS-A scores for the ACT+IMR condition.) Analyses tested both main effects for condition and condition × time interactions for differential change between conditions from six to 12 months. The between-groups effect size and 95% confidence interval were calculated as Cohen’s d, based on the adjusted means at 12-month follow-up.

Secondary RM-ANCOVA analyses were conducted to test for differences between groups at six and 12 months on the subscale scores of the BPRS, RAS, and CIM. Given the low utilization of emergency rooms and psychiatric hospitalizations during the study period, these data were dichotomized as no admission versus any admission. We used Fisher’s exact test to evaluate differences between conditions for these two binary outcomes, because the event rate was low. IMR session completion and module completion were used to evaluate the degree of exposure to IMR and to test whether exposure was associated with baseline and 12-month follow-up variables. Session completion was categorized as low (<10 sessions), medium (10–24), or high (≥25), and module completion was categorized as low (<5 modules) or high (≥5). Participants in the ACT+IMR condition were grouped by session completion and module completion categories, and the differences between these subgroups were then evaluated by ANOVA (for baseline values of demographic and outcome variables) or by ANCOVA (12-month outcomes), which controlled for baseline values of the same outcome variable. Secondary analyses used p<.01 to determine statistical significance.

Results

Table 1 shows the characteristics of the overall study group and comparisons between the treatment groups on baseline demographic, clinical, and outcome measures. There were significant differences (p<.05) between participants in the two conditions in ethnicity, living situation, primary psychiatric diagnosis, and client IMR Scale score. Compared with ACT-only participants, ACT+IMR participants were more likely to be housed and to have a mood disorder, were less likely to be Latino, and had lower mean client IMR Scale scores. The two groups were similar on the other measures.

Table 2 presents an overview of each of the 11 IMR modules. ACT+IMR participants completed 21.3±13.3 IMR sessions (range 0–42) and 4.5±3.4 IMR modules. We found significant differences in ACT+IMR teams in terms of sessions completed and modules completed. Participants on team 3 had significantly lower exposure to IMR compared with participants on other teams (Table 3).

Table 4 displays the statistical results at follow-up for the eight primary outcome measures. A significant group difference favoring participants in the ACT+IMR condition was found for the clinician IMR Scale, with a medium effect size (d=.51). There were no significant group differences on the other seven primary outcomes; however, a medium effect size favoring participants in ACT+IMR was observed on the QLS-A (d=.64). Effect sizes for the other continuous measures were small.

Differences in binary outcomes between participants in the two conditions were also small: eight (15%) ACT+IMR participants had at least one ER visit versus one (2%) ACT-only participant (Fisher’s exact test, p=.03), and 11 (21%) ACT+IMR participants had at least one psychiatric hospitalization versus four (8%) ACT-only participants (Fisher’s exact test, p=.10). RM-ANCOVA analyses of the five BPRS subscales, the five RAS subscales, and the three CIM subscales found no significant differences between treatment conditions.

Baseline variables were evaluated as predictors of IMR session and module completion to identify which participants were more likely to receive greater IMR exposure (Table 5). Participants who completed high school/GED, did not have a co-occurring axis II disorder, or had higher baseline QLS-A scores were more likely to complete 25 or more sessions (high exposure) compared with fewer than 10 sessions (low exposure). Participants with higher baseline DLA-20 scores were more likely to complete 10–24 sessions (medium exposure) compared with fewer than 10 sessions (low exposure). Participants who completed high school/GED, did not have a co-occurring substance use or axis II disorder, or had higher baseline QLS-A scores were more likely to complete five or more IMR modules compared with fewer than five.

Results of analyses of associations of IMR session and module completion with 12-month outcomes, adjusted for baseline measures of each outcome, are also presented in Table 5. Completing 25 or more IMR sessions was associated with more improvement at 12 months on the client IMR Scale and the DLA-20. Completing 10 to 24 IMR sessions was associated with more improvement on the DLA-20. Completing five or more IMR modules was associated with more improvement at 12 months on the client and clinician IMR scales, the DLA-20, and the CIM. Although nonsignificant, there was a trend toward lower BPRS scores and higher GAF and QLS-A scores among participants who completed more IMR sessions and modules.

Discussion

These pilot results provide support for the feasibility of implementing IMR within ACT teams. Many participants were able to achieve moderate to high levels of IMR exposure within 12 months, although the variability in exposure to IMR across participants and teams suggests that implementation was not without challenges. These challenges should be addressed in future studies.

This study found superior outcomes for ACT+IMR participants on only one of eight primary outcomes, clinician-rated illness self-management; however, the effect sizes for other variables and evidence for a potential dose-response relationship with four outcome measures provide some support for the potential effectiveness of ACT+IMR. Consistent with prior IMR research, we found significant improvement on clinician-rated illness self-management, with a medium effect size (42). It should be noted, however, that there was possible bias in the clinician ratings because the rater, a clinician, was not blind to the intervention. Inconsistent with prior research, this study did not find a main effect of treatment on client-rated illness self-management, psychiatric symptoms, or psychosocial functioning as measured by the QLS-A (42); however, a medium effect size was found for the QLS-A, and a small effect size was found for client-rated illness self-management, which is consistent with prior research (22). A small effect size for community integration was observed; although this variable has not been measured in other studies, it could be interpreted as an extension of functioning. The lack of significant treatment effect on other measures of functioning and other distal outcomes is consistent with the literature on IMR (42).

There are several potential reasons for the lack of significant treatment effects. First, this small-scale trial had relatively low power because of the use of an active treatment comparison condition and the relatively small sample size. Measures with low frequencies of endorsement—notably psychiatric hospitalizations and ER use—likely suffered from especially low power. In addition, the variable rates of exposure to IMR within the ACT+IMR condition, especially low exposure within one team experiencing high staff turnover, likely contributed to the lack of significant effects. Consistent with at least one IMR study, future ACT+IMR work may benefit from a longer period of evaluation (20), given that participants may have experienced delayed benefits that occurred beyond the study period, and from enhanced implementation strategies to address the need for follow-up IMR training due to staff turnover. Furthermore, it should be noted that most participants continued to receive IMR at the end of the study, suggesting that a longer interval is needed both to evaluate the effects and to effectively deliver IMR to this challenging population.

Our standardized measures may not have been sensitive to the benefits of the ACT+IMR intervention, given that subjective reports from participants and clinicians indicated that participants were making progress toward achieving personal recovery goals across several behavioral domains. Future research should explore idiographic improvements through qualitative research methods, examination of individual goals (43), or statistical methods that accommodate individual changes across a range of outcomes (44). Finally, it is possible that the lack of significant differences reflects the fact that ACT staff were in the relatively early stages of learning IMR and that greater practice using the model would yield stronger treatment effects.

Future work should also explore alternative ways of increasing the effectiveness and cost-effectiveness of recovery-oriented interventions within ACT. A qualitative process evaluation conducted across three small pilot studies, including this study, indicated that implementing ACT+IMR can be a time-intensive and complex process replete with barriers (for example, symptom severity, staff workload, and communication problems); however many of these barriers can be overcome by specific consultation and implementation strategies, such as tailoring IMR consultation to ACT specifically, focusing on client engagement, providing peer support, being flexible with the order and number of IMR modules, and improving team communication and service integration. Future larger-scale research efforts on ACT+IMR should build on this knowledge of barriers and implementation strategies, which may then lead to better, more cost-effective outcomes; a hybrid effectiveness-implementation design would provide the mechanism for such study (45). Future work may also target ways to increase IMR exposure to clients who are likely to receive fewer sessions, including participants with less education, substance use or axis II disorders, and lower baseline psychosocial functioning.

A different research direction concerns the duration of IMR. Completing the 11-module IMR curriculum requires about one year of weekly sessions. A more “targeted” approach, delivered individually, that focuses only on IMR topics related to specific goals of each client, could potentially reach more clients in a cost-effective manner. Similarly, other personalized approaches to illness management, especially those that incorporate technology in the delivery of interventions (46–50), also have the potential to be cost-effective strategies for helping people manage their illness and pursue recovery goals.

Conclusions

This study provides support for the feasibility of implementing IMR within ACT teams. Although many results were not statistically significant, this study provides initial evidence of a potential dose-response relationship and some medium (but nonsignificant) effect sizes favoring ACT+IMR. Further, larger-scale efforts using a hybrid effectiveness-implementation design would help to directly test more rigorous consultation and implementation strategies to maximize IMR exposure and the effectiveness of IMR for improving recovery and functioning outcomes for people served by ACT teams.