A Randomized Controlled Trial of the Effectiveness of the Illness Management and Recovery Program

Over the past decade there has been a growing effort to identify and provide evidence-based practices for persons with severe mental illness ( 1 ). An evidence-based practice is an intervention for which there is a sufficient body of research demonstrating its effectiveness for improving valued outcomes ( 2 , 3 ).

Compelling evidence exists for the effectiveness of several psychosocial interventions for persons with severe mental illness. However, these evidence-based practices have been found to be rarely available to the consumers who could benefit from them ( 4 ). To facilitate the implementation of these practices, the Substance Abuse and Mental Health Services Administration (SAMHSA) and the Robert Wood Johnson Foundation funded the National Evidence-Based Practices Implementation Project, which was developed and spearheaded by the Dartmouth Psychiatric Research Center. As part of this project, resource kits were developed for five psychosocial evidenced-based practices. These kits can be downloaded from the SAMHSA Web site (www.mentalhealth.samhsa.gov).

This article presents findings from a multisite study carried out in Israel that used a randomized controlled trial design to evaluate the effects of one evidenced-based practice, the Illness Management and Recovery program. This program is a standardized curriculum-based approach designed to provide persons with severe mental illness information and skills necessary for managing their illnesses effectively and working toward achieving personal recovery goals ( 5 ). Since the program became available in 2003, it has been adopted and implemented in many states in the United States and in several countries ( 6 ).

Although research findings have been accumulating that demonstrate the effectiveness of the specific program components, such as developing a relapse prevention plan and teaching strategies for coping with persistent symptoms ( 7 ), efforts to evaluate the effectiveness of the overall program have just begun ( 6 , 8 ). Noncontrolled pilot studies of this intervention have provided limited support for its feasibility ( 6 ) and effectiveness ( 8 ). Although the results of these studies are promising, randomized controlled trials are still needed. The results of such studies will be less vulnerable to threats to their external and internal validity ( 9 , 10 , 11 ).

This article describes the first randomized controlled trial of the Illness Management and Recovery program. The study tested the hypothesis that clients who participate in the program will show greater improvement in components of illness self-management and recovery, social support, and coping efficacy than clients who receive usual services.

Methods

Setting and participants

The study was conducted at 13 psychiatric community rehabilitation centers in several cities in Israel. Approval for the study was obtained from an institutional ethics review committee. After receiving a detailed explanation of the study, all research participants provided written informed consent.

The 210 study participants were identified as having severe mental illness and were receiving treatment and rehabilitation services at one of the 13 centers. Psychiatric diagnoses were obtained from clients' medical files. Inclusion criteria were fluency in Hebrew and sufficient competence to provide informed consent.

Although the educational curriculum of the Illness Management and Recovery program pertains specifically to schizophrenia, major depression, and bipolar disorder, the program's primary focus on self-management, personal goals, social support, appropriate use of medication, relapse prevention, and coping with symptoms is applicable to a broader range of disorders.

Instruments

Illness Management and Recovery Scale. Parallel client and clinician versions of the Illness Management and Recovery Scale have been developed as outcome measures for the Illness Management and Recovery program ( 12 ). Items, as formulated by practitioners and consumers, refer to perceived outcome in the content areas of the intervention: personal goals, knowledge of mental illness, involvement with significant others, functioning, symptoms, stress, coping, relapse prevention, hospitalization, medication, and use of drugs and alcohol. The possible score on each scale item ranges from 1 to 5, with higher scores indicating better illness management and recovery. Previous research has provided evidence for the scale's internal consistency, test-retest reliability, and validity ( 13 ).

For the purpose of this study, the questionnaires were first translated into Hebrew and then back-translated into English. The back-translated English version was compared with the original English version ( 14 ). Discrepancies revealed by this procedure were then resolved. Preintervention measures that were collected from participants in this study were used to assess the psychometric properties of these versions of the scale ( 15 ). On the basis of preintervention data, Cronbach's alpha coefficients for the client and clinician versions of the Illness Management and Recovery Scale were .55 and .73, respectively.

Factor analysis of both scales produced three similar client and clinician factors: knowledge and goals, coping with illness, and a factor that combined effective medication use and reduced alcohol and drug abuse. These factors explained 40% of the total item variance for the client scale and 47% of the total item variance for the clinician scale. These factors were found to have moderate reliabilities as estimated by Cronbach's alpha coefficients: for the client questionnaire, .73 for coping, .50 for knowledge and goals, and .74 for effective use of medication; for the clinician questionnaire, .83 for coping, .67 for knowledge and goals, and .47 for effective use of medication.

In each of the 13 rehabilitation centers, each study participant's case manager (social workers, psychologists, or occupational therapists) completed the clinician version of the Illness Management and Recovery Scale for his or her clients. These case managers met with the participating clients at least once a week.

Coping Efficacy Scale. The Coping Efficacy Scale presents a broad list of 14 different symptoms—four negative symptoms, two positive symptoms, two cognitive symptoms, three affective symptoms, two substance abuse problems, and one sleep problem ( 16 ). For each symptom respondents indicate whether they have experienced it, the strategies they use to cope with it, and the extent to which they perceive these strategies to be efficacious. An efficacy index was calculated for each participant by averaging the efficacy ratings across the symptoms and coping strategies. Possible scores range from 1 to 5, with higher scores indicating more effective coping. The Illness Management and Recovery intervention may have changed the status of participants in the experimental group. Therefore, a test-retest reliability of .44 (p<.001) as estimated by a Pearson product-moment correlation was calculated for the 64 research participants who were randomly assigned to the control group and who responded to the Coping Efficacy Scale at the end of the intervention (one year after the intervention was initiated).

Multidimensional Scale of Perceived Social Support. The Multidimensional Scale of Perceived Social Support is a measure of the extent to which an individual has access to three kinds of social support: family, friends, and significant others ( 17 ). Possible scores range from 1 to 7, with higher scores indicating higher perceived social support. A Hebrew translation of the scale was used ( 18 ). Internal consistencies (Cronbach's alpha) for the overall score and for the three measures of social support were .91, .80, .88, and .90, respectively.

Illness Management and Recovery Fidelity Scale. The Illness Management and Recovery Fidelity Scale is a 13-item scale that measures the extent to which the implementation of the intervention is consistent with the principles of the Illness Management and Recovery program ( 9 ). Strict adherence to a model may engender therapeutic rigidity, which is considered by some to be a disadvantage of such structured interventions as Illness Management and Recovery ( 19 ). Clinicians in this study were encouraged to use their clinical skills flexibly. Thus, although the program is standardized, it is also individualized as each person works toward his or her personally defined goals. Fidelity assessment was based on site visits by two assessors who used specific observational criteria to independently rate each item on the scale. Possible scores for each item range from 1 to 5, with higher scores indicating higher fidelity. After independently rating each item, the assessors met to decide on consensus ratings.

Intervention

Clinicians employed by the 13 psychiatric rehabilitation centers who received training in the Illness Management and Recovery program randomly assigned by lottery the clients who agreed to participate in the research to either a control group that received treatment as usual or a group that received the Illness Management and Recovery intervention. At certain sites, the number of potential participants was relatively small. At these sites, the randomization process was modified so that each participant had the same higher probability to be selected for the intervention. This ensured a sufficiently large intervention group even at these sites.

The treatment as usual that was offered at the 13 agencies varied. It generally included various forms and degrees of social, leisure, support, and work activities. Participation in the Illness Management and Recovery program was in addition to receipt of existing services. Educational handouts that are a central part of the Illness Management and Recovery program were translated into Hebrew and adapted for use in Israel. The sequence in which the handouts were covered in the intervention group was the same across the 13 sites. Intervention sessions were led by two clinicians, one of whom had received training. Sessions lasted for approximately an hour and were held weekly for about eight months.

Procedure

Thirteen clinicians who led the Illness Management and Recovery groups were selected from a group of 18 clinicians who had expressed interest in receiving training and agreed to take part in this research. These clinicians participated in weekly training sessions at Bar-Ilan University. Twelve sessions were held, each lasting for four hours. Upon completion of the training, the clinicians, along with an additional clinician who acted as a co-leader, began implementing the program at their respective centers. The co-leaders did not participate in the training. For the first eight months of this implementation, the clinicians attended monthly two-hour group supervision sessions.

Before the initiation of the intervention and upon its completion, participants completed all the outcome measures and their case managers independently completed the Illness Management and Recovery Scale. In one center the intervention was not completed because of technical problems. This center was not included in the analysis of results. Midway through the implementation—four to five months after its initiation—sites were visited by two independent fidelity assessors. Means of fidelity scores ranged from 2.66 to 4.77. In nine of the 12 centers in which the implementation was completed, the fidelity score was above 3.70. Interrater reliabilities (Pearson correlations) for the fidelity scores ranged from .65 for goal setting to 1.00 for number of participants. Posttreatment data were collected immediately after the intervention for both groups. The duration of the intervention varied from eight to 11 months. All data were collected from March 2005 to March 2006.

Results

Table 1 summarizes the pertinent clinical and sociodemographic characteristics of the study sample. Most of the clients had completed high school and were employed at their rehabilitation centers in noncompetitive jobs, such as printing, assembly work, and book binding. No statistically significant differences were found between the control and intervention groups on any of the sociodemographic variables.

Enlarge table

During the intervention, eight (4%) of the participants were hospitalized, two (1%) refused to answer the postintervention questionnaires, 22 (10%) left the center where the intervention took place, and 15 (7%) did not complete the intervention. In addition, 13 (6%) did not fully complete the questionnaires. These participants were classified as dropouts and were not included in the statistical analysis. No statistically significant difference in hospitalization during the intervention was uncovered between the groups.

Repeated-measures analyses of variance were conducted to examine the impact of the Illness Management and Recovery program, with group (intervention versus control) as the independent variable and assessment point (preimplementation versus postimplementation) as time. For these analyses a group-by-time interaction is a test of whether clients in the Illness Management and Recovery program improved more than clients in the control group. An analysis was performed for each of the outcome measures, including the three factors of the client and clinician Illness Management and Recovery Scale: knowledge and goals, coping with illness, and effective medication use plus reduced alcohol and drug abuse ( 15 ). Means and standard deviations for all of the pre- and postimplementation outcome measures for both groups are shown in Table 2 .

Enlarge table

Statistically significant interactions between group and time were found for the total Illness Management and Recovery Scale completed by clinicians (F=4.18, df=1 and 146, p<.05) and for both the clinician and the client knowledge and goals factors of the scale (F=15.30, df=1 and 146, p<.001 and F=7.07, df=1 and 148, p<.01, respectively). There was also a trend toward significance between group and time for the total Illness Management and Recovery Scale completed by clients (F=3.64, df=1 and 148, p<.06).

The means and standard deviations in Table 2 indicate the source of these interactions. Data in the table indicate that from the perspectives of both the client and the clinician, Illness Management and Recovery Scale total scores improved significantly more for the participants in the program than for those who received treatment as usual. In particular, from the perspectives of both clinician and client, clients appeared to have increased their knowledge of their illness and to have improved their personal goal identification and attainment as a consequence of the intervention.

In addition, statistically significant main effects emerged from these analyses. The means and standard deviations associated with these effects are presented in Table 2 . For the clinician Illness Management and Recovery Scale total scores, both the group and time main effects were statistically significant (F=7.60, df=1 and 146, p<.01 and F=41.16, df=1 and 146, p<.001, respectively). The data in Table 2 indicate that from the clinicians' perspective, participants in the intervention received higher total scores on the Illness Management and Recovery Scale before initiation of the intervention and upon its completion. Also, on the basis of the clinicians' scoring all participants, regardless of whether they were in the intervention group, received higher total scores at the completion of the intervention. The main effect for group noted above was also found for clinicians' knowledge and goal factor scores (F=14.10, df=1 and 146, p<.001) and for time of assessment (F=17.50, df=1 and 146, p<.001). For the coping factor score on the scale completed by clinicians, a statistically significant main effect for time of assessment (F=19.80, df=1 and 146, p<.001) indicated that scores for both the intervention group and the control group were higher after the intervention.

From the clients' perspective, statistically significant main effects emerged for the Illness Management and Recovery Scale total score (group, F=7.20, df=1 and 148, p< .01), the knowledge of illness and personal goal factor scores (group, F=11.08, df=1 and 148, p<.001; time of assessment, F=9.75, df=1 and 148, p<.01), the coping factor score (time of assessment, F=7.38, df=1 and 146, p<.01), and the coping efficacy score (time of assessment, F=4.78, df=1 and 128, p<.05). For the knowledge and goal factor score, these main effects were similar to those found for clinicians, whereas for the total score on the Illness Management and Recovery Scale, only the group main effect was statistically significant and only the time of assessment main effect was statistically significant for the coping factor score. For the Illness Management and Recovery Scale total score, higher scores were found for participants in the intervention group both before and after the intervention. For the coping factor score and for the coping efficacy score, statistically significant higher scores were found for participants in both the control and the intervention groups at the completion of the intervention compared with before the intervention.

Adding the 12 sites to the original analyses as a covariate did not change the results. However, when site was added to group and time in the analyses as an independent variable, two statistically significant three-way interactions between group, time, and site emerged—one for social support (F=1.97, df=10 and 124, p<.05) and one for the clinician knowledge and personal goals factor (F=1.99, df=10 and 128, p<.05). Mean scores for both groups on the clinician knowledge and goals factor suggest that the intervention groups showed more improvement on this factor at some sites than at others; at certain sites control group participants even showed a postintervention decline on this measure. However, mean scores on the social support measure showed a pattern of increases and decreases both before and after across the different sites and for both groups.

Separate analyses for the nine sites that had high fidelity (above 3.70) showed two interactions between group and time in addition to the interactions that were found for the whole sample. These interactions were for clients' total score on the Illness Management and Recovery Scale (F=6.87, df=1 and 111, p<.05) and for the coping factor on the Illness Management and Recovery Scale completed by clinicians (F=4.00, df=1 and 111, p<.05). Clients' total scores on the Illness Management and Recovery Scale and clinicians' scores on the scale's coping factor were relatively higher for the intervention group than for the control group only at the termination of the intervention. This difference appears to be the source of these interactions.

Discussion

This study found that persons who participated in the Illness Management and Recovery program showed significant improvement in their knowledge of illness and in identification and attainment of personal goals. Clinicians' ratings of clients also indicated significant improvement in overall illness management and recovery outcome as a result of participation in the intervention.

Psychoeducational interventions have been found to help persons gain basic knowledge of their illness ( 20 ). However, psychoeducation alone does not appear to help clients manage their illness better or engage in the recovery process as a consequence of this newly gained knowledge ( 7 , 20 ). Our findings seem to support the premise that the Illness Management and Recovery program not only provides information but also promotes recovery, at least, in terms of personal goal identification and attainment.

Additional objectives of the Illness Management and Recovery program are to help clients improve their social relationships and to cope more effectively with their symptoms ( 5 ). This study did not find differences between clients who participated in the program and those who received usual care on the coping factor score and coping efficacy, although both groups improved significantly over time. It is possible that services offered to both of the groups, such as vocational rehabilitation and case management, contributed to the improved coping found for both groups.

At sites with high fidelity ratings additional statistically significant interactions were found. These interactions indicated that for these sites, clients' total score on the Illness Management and Recovery Scale and clinicians' score on the coping factor indicated significantly greater improvement for program participants than for those in the control group, which emphasizes the importance of fidelity. Consistent with the only published study on the effectiveness of the Illness Management and Recovery program ( 6 ), no statistically significant results emerged for measures of social support. One possible explanation is that initiating and establishing social relationships and developing and applying coping strategies are more complex and demanding than are internalizing information about illness and identifying and implementing personal goals. The former activities may include integrated sets of affect, cognition, and behavior and may therefore require more comprehensive and intensive practice. An alternative explanation is that since outcome was assessed immediately after termination of the intervention, participants did not have sufficient time to put into practice the more complex activities involved in establishing social relationships and developing effective coping strategies.

Several limitations of this study should be noted. First, because it involved an intervention carried out in the field with a relatively large sample, practical considerations precluded the use of objective measures of the extent to which participants had attained the intervention goals. Second, although the measures that were used have been used in recent pilot studies to examine the effectiveness of the Illness Management and Recovery program ( 6 ), they are relatively new and require further validation. Therefore, outcome was assessed by a number of self-report measures. Furthermore, some of these measures might be considered secondary outcome measures. For example, items on the Illness Management and Recovery Scale were derived from a questionnaire that was constructed as a short, user-friendly subjective measure of the extent to which participants in the intervention attain the goals of its nine modules. Other measures might be considered primary because they were developed independently of the intervention and therefore may be relatively more standardized. These measures, which focus more specifically on such intervention goals as coping efficacy and social support, might be related to relatively long-term outcome. Thus a major weakness of the study is that only the secondary measures showed evidence of the intervention's effectiveness. Future studies that include more objective measures of the intervention's primary and secondary goals should be designed to examine both the intervention's effectiveness and the relation between the secondary and primary goals. Such measures could also assess relapse and rehospitalization as outcomes.

A third limitation is that the case managers completed the ratings and thus were not blind to the random assignment of the participants to the intervention and control groups. The case managers were naturally invested in improving the status of their clients. Therefore, they may have been biased in favor of the intervention group. Fourth, although a multisite study provides grounds for the generalizability of its findings, it also can be a source of variance and ambiguity as to the meaning of these findings. Finally, the statistical analyses included only individuals who completed treatment. Thus the posttreatment means may be artificially elevated. In future research, an effort should be made to contact participants who drop out.

Conclusions

This randomized controlled study showed that the Illness Management and Recovery program is effective in increasing clients' knowledge of their illness and helping them make progress toward personal goals. Adhering to the program protocol also increased clients' perceived ability to cope with their illness. However, the program, in its present form does not seem to lead to gains in general coping efficacy and social support.

Acknowledgments and disclosures

The authors thank Israel's Ministry of Health and the National Council for the Rehabilitation of Persons With a Psychiatric Disability in the Community.

The authors report no competing interests.