Factors Associated With a Positive Response to Cognitive Remediation in a Community Psychiatric Sample

Cognitive impairment is a persistent symptom of severe psychiatric disorders that is significantly related to overall treatment outcome (1,2,3). As behavior-based cognitive treatments become more available (4), it is important to know who is most likely to benefit from this service and whether there is a threshold of treatment intensity that is necessary for an effect (5). This information can guide efficient, cost-effective implementation of cognitive services at the community level.

The aim of this study was to examine factors that may be associated with a positive response to cognitive remediation. Because the goal of cognitive remediation is not merely to improve behaviors seen during remediation training but rather to improve behaviors outside the treatment setting (6), we chose to measure remediation success by looking at improvement in attention as applied to common clerical duties as well as at behaviors relevant to successful employment.

Furthermore, because our aim was to understand determinants of a positive response to treatment rendered in a community setting, we gathered data at a large community-based cognitive remediation program rather than in an experimental laboratory. Implementing a cognitive remediation program in a community clinic presents unique challenges due to a lack of motivation for treatment among participants and agency restrictions on treatment hours. The neuropsychological educational approach to remediation addresses those challenges by encouraging active engagement and intrinsic motivation to facilitate the transfer of learned cognitive skills to everyday situations (7,8).

The study addressed two questions. First, would participants in a community mental health clinic demonstrate improvements in the functional use of cognitive skills after receiving cognitive remediation? Second, what factors are associated with positive outcomes?

Methods

The study was part of a comprehensive yearly program evaluation (2003 to 2004) that detailed treatment outcomes and identified variables for enhancing positive outcomes. All participants completed 26 hours of cognitive remediation while working at their individual pace on computer-assisted cognitive exercises. They worked in small clinician-run groups of five to ten participants, which were scheduled for two one-hour sessions per week.

The participants were 48 adult psychiatric outpatients who had been referred for cognitive remediation from various treatment programs in a large community mental health agency in New York City. Consent to cognitive remediation was provided as part of an inclusive agency intake in which patients signed an agreement to voluntarily participate in any or all treatment programs available in the clinics as referred by their primary clinician. As is the standard intake policy at the agency, all participants who were enrolled in the cognitive remediation program were made aware that their progress would be frequently monitored and recorded and that all identifying information would be encoded to ensure confidentiality. In addition, all participants were given an initial description of the program and were provided with the opportunity to withdraw from cognitive remediation at any time without penalty. The agency's administration granted approval to conduct the study as part of a yearly program evaluation.

Participants' demographic characteristics were consistent with the demographic profile of the clinic overall. The sample included 30 men (62 percent) and 18 women (38 percent), with a mean±SD age of 34.91±5.51 years and an average of 10.94±2.22 years of education. The ethnic distribution was 54 percent African American (26 clients), 27 percent Hispanic (13 clients), and 19 percent Caucasian (nine clients). Twenty-six clients (54 percent) received from their treating psychiatrist a diagnosis of a schizophrenia spectrum disorder; ten clients (21 percent), a diagnosis of bipolar disorder type I; nine (19 percent), severe major depression with psychotic features; two (4 percent), major depression without psychotic features; and one (2 percent), psychosis not otherwise specified. All participants were free of active substance abuse.

The Minnesota Clerical Test, Second Edition (MCT) was used as a functional measure of attention. The MCT is a 200-item paper-and-pencil test that measures sustained attention and clerical accuracy (9). The test is commonly used by employment agencies to select applicants for various clerical jobs and has excellent test-retest reliability, with noted practice effects that disappear after 30 days (9).

The Work Behavior Inventory (WBI) was used as a functional measure of work behavior. The WBI is a 34-item supervisor-rated scale that measures work-related behaviors that are essential for successful employment (10). Items are used to generate a score for each subscale: social skills, cooperativeness, work quality, work habits, and self-presentation. The WBI has good to excellent interrater reliability and is widely used to assess work function among persons with psychiatric illness (10). Although the WBI was designed with a working sample in mind, the authors used the instrument in this study with a nonworking sample to assess functional behavior changes that are highly relevant to remediation goals.

All participants were assessed before and after treatment with the MCT and the WBI by independent clinicians from various clinics who were trained in test administration and the treatment model before the remediation groups were conducted. In addition, the clinicians were supervised weekly by the authors to ensure proper test administration and faithful implementation of the remediation model.

Frequency of attendance was used as a behavioral measure of treatment intensity. All participants were encouraged to attend cognitive remediation twice a week. Because cognitive remediation was being offered as part of the clients' standard care within the clinic, clients were not financially compensated for their participation in the study. Thus participants completed the program of their own accord, with treatment intensity (high versus low) measured as participant-determined frequency of attendance. If a client completed 26 sessions within 120 days, he or she was considered to have received high-intensity cognitive remediation. Clients who took longer to complete the required number of hours were recorded as having received low-intensity treatment.

Data were assessed for normality, and nonparametric measures of Spearman's rho and Wilcoxon tests were substituted for skewed distributions in comparisons of groups. In the case of data that were normally distributed, comparisons of means between groups were calculated by using analysis of variance (ANOVA). Effect size calculations and ranges were based on Cohen's d.

Results

The results showed significant posttreatment improvement on the MCT and the WBI; improvement on the MCT was associated with treatment intensity. Treatment effects on the MCT were moderate (effect size=.55), and treatment effects on the WBI subscales ranged from small (for self-presentation and social skills) to moderate (for work quality, effect size=.47) to large (for work habits, effect size=.59, and for cooperation, effect size=.64). Participants who were deemed to have received high-intensity treatment (27 clients, or 56 percent) completed the required number of sessions in far fewer days (98.37±18.93 days) than those who received low-intensity treatment (21 clients, or 44 percent; 161.56±33.03 days). Notably, there was a significant negative correlation between the number of days it took participants to finish 26 sessions and pre-post change on the MCT, a measure of attention on a clerical task (Spearman's rho=-.638, p<.01). Despite the fact that there were no significant differences in baseline attentional ability, work-related behaviors, or diagnostic distribution between groups, participants who received high-intensity treatment showed considerably more improvement on the MCT than participants who received low-intensity treatment (F=6.820, df=1, p<.013). Participants who received low-intensity treatment showed a negligible treatment effect on the MCT (-.01), compared with a large treatment effect for the high-intensity group (.90).

The ANOVAs showed no relationship between treatment intensity and behavior change on any of the WBI subscales and no posttreatment differences on the MCT and the WBI due to gender, diagnosis, or ethnicity. No significant correlations were found between change in cognition or work behavior and participants' age or educational level.

Discussion and conclusions

Participants from a community sample showed significant improvement in cognition and work-related behavior after receiving cognitive remediation based on the neuropsychological educational approach to remediation. Higher treatment intensity, as measured by attendance, was associated with greater cognitive improvement. Importantly, there was a threshold of treatment intensity below which there was no treatment effect on the attention measure. Specifically, participants who came into treatment intermittently and subsequently took more than four months to complete the sessions did not obtain any cognitive benefits. On the other hand, a high level of treatment intensity was associated with a very large treatment effect (effect size=.90). Interestingly, improvement in work-related behavior was not related to the number of days it took participants to finish the program, which suggests that changes in behavior were not a function of treatment intensity. Learned work-related behaviors from cognitive remediation seem to be maintained regardless of irregularity in attendance. Only cognition was subject to a certain degree of treatment intensity.

The findings of this study have several clinically relevant implications. First and foremost, the neuropsychological educational approach to remediation is an effective modality of cognitive remediation that can be successfully implemented for use in community-based programs. Treatment evaluation measures in this study indicated that community participants benefited from treatment in terms of enhancing attentional ability and vocational readiness. The model presents remediation training in an attractive format so that clients are naturally drawn to the enjoyable activities without the need for a host of extrinsic rewards. Such a model of treatment is especially relevant to community agencies, because paying clients for their enrollment in cognitive remediation is not financially feasible.

Second, the benefits of this model of cognitive remediation translate to practical gains for the participants. The clinical skills developed can be readily applied to meaningful vocational goals, a point well taken by clients and staff and by itself a motivating factor for program participation.

The third implication bears out the need to consider how participants' motivation may be associated with treatment intensity and thus affect cognitive remediation. Given that treatment intensity was established by participant-determined attendance at sessions that did not provide financial rewards, one can surmise that frequency of attendance was also a behavioral barometer of motivation for treatment. Indeed, attendance is recognized in the motivation research as a measure of intrinsic motivation for engaging in a treatment (10). We propose that sufficient motivation and treatment intensity are intertwined to create an environment that significantly enhances the remediation process. Clients who are more motivated may have received a higher intensity of treatment because they made themselves available for treatment on a more regular basis. This study provides increasing evidence that participants with persistent mental illness can make practical improvements if sufficiently motivated to engage in cognitive training and, in so doing, attend sessions on a regular basis to maximize treatment outcome.

This study was limited by its design. A larger, randomized, controlled study with a diagnostically homogeneous sample that systematically tests possible confounding or mediating variables—for example, symptom severity and nonspecific stimulation—would clarify any causal relationship between motivation, treatment intensity, and outcome.

Dr. Choi is affiliated with the department of psychiatry of Yale University School of Medicine in New Haven, Connecticut, and with the rehabilitation research and development service of the Department of Veterans Affairs in West Haven. Send correspondence to him at VA Connecticut Healthcare System, Psychology Service (116-B), 950 Campbell Avenue, West Haven, Connecticut 06516 (e-mail, [email protected]). Dr. Medalia is affiliated with the department of psychiatry and behavioral sciences and the department of neurology at Montefiore Medical Center and Albert Einstein College of Medicine in the Bronx, New York.