A Randomized Controlled Trial of Cognitive Remediation Among Inpatients With Persistent Mental Illness

Cognitive impairment is a core feature of patients with severe and persistent mental illness and an important determinant of course of illness. Thus treatments that improve cognition are of significant interest. Several randomized controlled trials of cognitive remediation have been conducted, mainly among patients with schizophrenia ( 1 ). Despite significant differences across these studies in patient populations (inpatients versus outpatients), methods used for cognitive practice (computerized versus paper-and-pencil drill, practice with and without strategy coaching, and group versus individual practice), hours and duration of cognitive programming, and cognitive domains targeted, the overall results of cognitive remediation indicate modest but consistent and significant improvements on a broad range of cognitive functions.

Several randomized controlled trials of cognitive remediation have been conducted with long-stay psychiatric inpatients ( 2 , 3 , 4 , 5 , 6 , 7 ). However, cognitive remediation methods in these studies differed significantly. For example, the number of hours of cognitive remediation ranged from seven to 72 hours of computerized or paper-and-pencil drill, with duration of treatment ranging from five to 24 weeks. Results of these studies, similar to the larger group of cognitive remediation studies, indicate cognitive benefits associated with the delivery of a cognitive remediation program. Two of these studies evaluated effects of cognitive remediation on broader psychosocial functioning, particularly on social functioning ( 5 , 6 ). However, no studies have reported the longer-term, posttraining impact of cognitive remediation on psychosocial functioning in inpatient populations. Thus the long-term impact of cognitive remediation on functional outcome among inpatients is not clear.

In addition, there have been no inpatient studies evaluating the effects of cognitive remediation on work as a primary functional outcome. Understanding the effects of inpatient cognitive remediation on work is of interest for several reasons. First, cognitive dysfunction has been found to be related to employment status and performance at work ( 8 , 9 , 10 , 11 ). Furthermore, impaired social functioning has been found to be related to work functioning ( 10 ). Second, impaired cognitive functioning is predictive of (and related to) less benefit from vocational rehabilitation. Limited evidence suggests that cognitive remediation may improve response to vocational rehabilitation of outpatients ( 12 , 13 ). Finally, work functioning as a long-term goal has an intrinsic and potentially self-affirming effect for patients participating in cognitive remediation.

This study was the first to evaluate the effects of cognitive remediation on work outcomes among psychiatric inpatients. Our primary aim was to examine the feasibility and effectiveness of a computerized cognitive remediation program designed for use by psychiatric inpatients. We hypothesized that cognitive remediation would improve cognitive performance on a standardized neuropsychological battery. We also hypothesized that cognitive remediation would improve work activity in the hospital vocational rehabilitation program over a one-year follow-up. Finally, we explored whether cognitive remediation would improve psychiatric symptoms.

Methods

The study was a randomized controlled 12-week trial evaluating a computerized cognitive remediation program for intermediate- to long-stay psychiatric inpatients at Manhattan Psychiatric Center in New York. All of the research procedures were approved by the hospital's institutional review board. All participants signed informed consent statements before participating in the study.

Participants

Eligibility criteria for participation in the study were a DSM-IV ( 14 chart diagnosis of schizophrenia, schizoaffective disorder, or bipolar disorder; absence of psychiatric history of mental retardation, brain injury, or neurological disorder; stable use of medication for at least three months without plans for changing medication; and proficiency in English. Patients were recruited in the inpatient services of a state psychiatric center through referrals by inpatient staff and remained on inpatient status for extended periods because of either partial treatment response or the lack of appropriate placements in the community. A total of 85 patients were enrolled in the study over a period of 2.5 years from November 2003 to May 2005. The sample was predominantly male (89%) with a mean±SD age of 43.5±10.3 years and an education of 10.6±3.1 years.

Measures

Neuropsychological functioning was assessed at baseline and 12 weeks after cognitive remediation treatment. Symptoms were assessed at baseline, midpoint (six weeks), and endpoint (12 weeks) and at six- and 12-month follow-ups. In addition, work activity in the hospital was recorded weekly throughout the 12-month follow-up.

Cognitive functioning

A broad range of cognitive functions were assessed at baseline and after the computerized cognitive remediation period, including attention, psychomotor speed, verbal working memory, verbal learning and memory, and executive functions. The tests used in the cognitive battery are explained next.

Premorbid academic achievement. The Wide Range Achievement Test (WRAT 3) reading subtest was used to measure premorbid academic achievement ( 15 ). This instrument measures word-recognition reading performance. Performance on this test has been shown to be relatively preserved in schizophrenia, thereby providing an index of premorbid educational attainment ( 16 ). The WRAT 3 results in a total score for words read correctly, converted to the grade-equivalent score. This measure was administered only at baseline.

Verbal working memory. This was measured with the digit span subtest of the Wechsler Adult Intelligence Scale-Revised (WAIS-R) ( 17 ). Respondents are given a number string and asked to repeat it in the same order of presentation (digit span forward) or backward (digit span backward). The measure of interest was number correct for each condition.

Psychomotor speed. This was measured with Trail Making Test Part A ( 18 ). Trail Making Part A is a timed measure that requires participants to connect numbers in order. The measure used was time, in seconds, to complete the task.

Information processing speed. This was assessed with the digit symbol substitution test of the WAIS-R ( 17 ). For the test, respondents are asked to copy unique symbols below individual numbers (1 through 9) for 120 seconds. Number of symbols accurately copied is totaled and is the dependent variable.

Verbal learning and memory. This was assessed with the Rey Verbal Learning and Memory Test (RAVLT) ( 19 ). The RAVLT involves the repeated presentation of a word list that consists of common items. The measures of interest were acquisition, determined by the total words recalled during the five acquisition trials (RAVLT 1–5), and retention, determined by the total words recalled in the long-delay free-recall test, which occurs 20 minutes after the last acquisition trial.

Executive functioning. This was assessed with Trail Making Test Part B and the Wisconsin Card Sorting Test (WCST) ( 20 ). Trail Making Part B is similar to Part A but is a more challenging task because it requires participants to connect consecutively numbered and lettered circles by alternating between the two sequences ( 18 ). The measure used was time, in seconds, to complete Trail Making Part B. The WCST is a commonly used test of executive functioning that measures cognitive flexibility and problem-solving skills. Participants are asked to match a series of cards to a set of four target stimuli, which are also cards. Participants are provided with feedback on an item-by-item basis after they sort each of the item cards. After they determine one of the correct dimensions, referred to as categories, ten correct responses are required before the correct category is shifted to the next one. Continued matching to a category that is no longer correct is considered a perseverative error. The variables of interest were the number of categories achieved (WCST categories) and percentage of perseverative errors (WCST PP).

Overall cognitive functioning. A composite measure of overall cognitive functioning (not including premorbid intelligence measured on the WRAT 3) was computed by standardizing each of the cognitive measures (computing z scores) and summing those z scores separately for the baseline and follow-up assessments.

Symptoms

Psychiatric symptoms were assessed with the Positive and Negative Syndrome Scale (PANSS) covering the prior week of functioning ( 21 ). Symptoms were assessed at baseline and at six-week, 12-week, six-month, and 12-month follow-ups. Interviews were conducted by trained psychiatrists and psychologists. Interrater reliability among the four PANSS raters was .90 (Cronbach's alpha). Outcomes on the PANSS were examined with the five-factor solution described by Lindenmayer and colleagues ( 22 ), which includes the following subscales: positive, negative, activation, depression, and cognitive.

Work measures

All study participants were offered employment within the hospital on completion of both cognitive remediation and control condition. Patients were selected for jobs after an interview. Interviewers were blind to the condition to which the patient had been assigned during the treatment phase. Work measures were collected by the vocational rehabilitation director (not blind to the conditions) during the 12-month follow-up and included cumulative hours, weeks worked, and wages earned.

Treatment programs

All study participants had access to the broad range of treatments available to inpatients at Manhattan Psychiatric Center while involved in the trial. These treatments included pharmacological treatment and management, and psychosocial group interventions administered in a required 20-hours-per-week "treatment mall" program. This program included aggression management; mental illness and chemical abuse interventions; social skills training and preparation for community living; and a comprehensive educational program that teaches patients about their medication, mental illness, and healthy lifestyles. Both groups attended the same mix of mall group programs, both in terms of number of groups and type of groups.

Work program

The Patient Work Program at the psychiatric center offers various jobs, including ward activity, custodial, and food service assistant positions that are paid within a range of $2.70 to $6.00 per hour. Wages are earned on the basis of the skill level required to perform job duties, with each elevated salary level requiring increased skills or knowledge. The program is modeled after community-based employment; the patient must interview for the job and, if the job is obtained, sign a work contract, wear job identification cards, and adequately perform job requirements. All jobs are supervised, and support is offered by the manager of the work area.

Cognitive remediation program

Patients were engaged in approximately 24 hours of computer-based cognitive exercises (COGPACK version 6.0) ( 23 ) that provided equivalent practice time in areas of attention and concentration, psychomotor speed, learning and memory, and executive functions from a standardized curriculum. Each session was 45 minutes long and included exercises for practicing each of these areas of cognition. Patients received performance scores reflecting accuracy and speed after completion of each exercise, which were recorded and referenced in order to reinforce progress. Computer exercises were designed to be enjoyable and reinforcing to complete, with difficulty gradually increasing over time and often depicting an everyday event. For example, a scene of a city street, depicting moving vehicles, street signs, people, and stores, is presented for several seconds, and participants are then asked to recall details of the scene.

Participants received two hours of computer practice and a one-hour discussion group per week for a total of 12 weeks. This frequency and duration of training with this program has been shown to result in cognitive improvement on independent tests ( 4 , 12 ). The weekly group discussions focused on the importance of cognitive skills, on performing activities of daily living and work, and on the development of compensatory strategies for managing persistent cognitive problems. Cognitive remediation was delivered in cohorts of six to eight patients and supervised by three hospital staff members, including two psychologists and a psychology or occupational therapy intern.

Computerized control condition

The computerized control condition controlled for staff time and computer exposure, which consisted of three weekly, one-hour computer sessions. Similar to the group assigned to cognitive remediation, the groups comprised six to eight patients and were supervised by two or three hospital staff. Patients were assigned to a computer station and given instruction on using the mouse. Patients were then able to play computer games (for example, Frogger, Microsoft Pinball, and Microsoft Solitaire) or learn typing skills (using Mavis Beacon Typing, version 5).

Study referrals were obtained by making presentations to hospital treatment teams about the nature and purpose of the study. Clinicians on the treatment teams referred potentially eligible patients to the study coordinator, who met with the patient, confirmed eligibility, explained the purposes of the study, and obtained informed consent. Baseline assessments were then conducted. After completion of baseline assessments, patients were randomly assigned to either the cognitive remediation group or the computer control group, stratified by reading level with the WRAT 3 reading scores to ensure that the two groups had similar reading levels. Randomization was conducted with a computer-generated randomization sequence (www.randomization.com) conducted by the study coordinator.

Statistical analysis

First, in order to determine whether there were any baseline differences between patients who were assigned to cognitive remediation and the control group, we performed a series of chi square analyses (for categorical variables) and t tests (for continuous variables) on the demographic characteristics and the clinical and cognitive measures.

Second, the rate of exposure to cognitive remediation was determined by computing the number of clients who completed a minimum of six cognitive training sessions.

Third, intent-to-treat analyses were conducted to compare the cognitive remediation group and the control group on changes in cognitive functioning from baseline to the three-month assessment. Analyses of covariance (ANCOVAs) for this comparison were conducted; cognitive performance measures at the three-month assessment were the dependent measure, treatment group was the independent variable, and cognitive performance at baseline was the covariate.

Fourth, employment outcomes for patients who were not working at the beginning of the study were compared between the clients in the cognitive remediation and computer control groups. The relative proportion of patients who worked was compared between the two groups by performing chi square analyses. Cumulative hours, weeks worked, and wages earned over the 12-month follow-up period were compared between the cognitive remediation and control groups by t tests.

Fifth, differences over time in symptoms between the two groups from baseline to the six-week, three-month, six-month, and 12-month follow-up assessments were evaluated with mixed-effects linear modeling. The five PANSS subscale scores across all the assessments formed a profile of dependent variables, and treatment group served as the independent variable. For these analyses, a time effect reflected changes across both groups over time, and the group-by-time interaction reflected changes over time between the two treatment groups.

Mixed-effects models have advantages over traditional repeated-measures analyses of variance because they can accommodate missing data, thereby permitting the inclusion of all subjects with data in the analyses ( 24 ). These analyses were performed with the PROC MIXED procedure of Statistical Analysis Software ( 25 ). The other analyses were conducted with the Statistical Package for Social Sciences, release 13.0 ( 26 ). For all analyses p<.05 was used as the level of significance.

Results

Eighty-five patients participated in the study. Forty-five were randomized to cognitive remediation and 40 to the control condition. Seventy-two patients completed the entire trial of 24 hours (41 patients in cognitive remediation and 31 in the control condition). The mean dropout time for patients who did not complete the entire study was at 10.53±5.23 hours (median=12 hours). Reasons for termination included patient refusal (four patients in the control condition and one in cognitive remediation), discharge to the community (four patients in the control condition and one in cognitive remediation), and psychotic decompensation (one patient in the control condition and two in cognitive remediation). The groups did not differ significantly in the type of antipsychotic medications that they received (41 participants, or 91%, in the cognitive remediation group received second-generation antipsychotics, compared with 37 participants, or 92%, in the control group).

Chi square analyses and t tests indicated no statistically significant differences between the two groups at baseline on any demographic, clinical, or cognitive variables ( Table 1 ). All patients assigned to the cognitive remediation group completed a minimum of six sessions (in other words, they were treatment exposed) and completed 23.0±3.4 sessions.

Enlarge table

The ANCOVAs evaluating changes in cognitive functioning between the two groups are summarized in Table 2 . Significant group effects were found for Trail Making Part A and RAVLT (total and trials 1–5), with a trend-level significant effect for the cognitive composite score. All effects indicated greater improvement for patients in the cognitive remediation group than in the control group.

Enlarge table

Employment outcomes

Among the 85 study participants, 68 (80%) were not employed in the hospital at the beginning of the study. Forty-eight participants (56%) were employed after the study. Among the 37 unemployed patients at baseline in the cognitive remediation group, 51% obtained a job during the 12-month follow-up, compared with 35% of 31 initially unemployed patients in the control group, which was not a statistically significant difference. Patients who received cognitive remediation worked significantly more weeks than patients in the control group and showed a trend to work more hours and earn more wages ( Table 3 ).

Enlarge table

Symptoms

Mixed-effects linear analyses indicated significant improvements over time on the PANSS positive subscale (F=4.95, df=1 and 267, p=.027) and the activation subscale (F=8.51, df=1 and 267, p=.003). There was a trend for improvement in depressive symptoms (F=7.10, df=1 and 267, p<.08) but no significant changes on the negative or cognitive subscales. The group-by-time interaction was not significant for any of these analyses, suggesting similar changes over time for the two treatment groups. Thus cognitive remediation did not appear to affect the rate of improvement of psychiatric symptoms over the study period.

Discussion

A majority of patients randomly assigned to the cognitive remediation program were successfully engaged in and completed the program (41 of 45 patients, or 91%). Furthermore, cognitive remediation improved cognitive performance compared with the control group. These findings support the feasibility of implementing cognitive remediation in psychiatric inpatient settings and suggest that such a program may be beneficial for patients with significant functional and symptomatic impairments.

The specific domains of cognitive performance that were improved among patients who received cognitive remediation included attention and psychomotor speed (Trail Making A) and learning (RAVLT, trials 1–5), with a statistically significant trend (p=.05) for overall cognitive performance, whereas effects were not found for executive functioning (WCST categories, WCST PP, and Trail Making B) or memory (RAVLT delayed recall). This pattern of results is similar to those found in a previous randomized controlled trial of cognitive remediation using the same software (COGPACK) for psychiatric outpatients assessed with a similar battery of cognitive measures ( 12 ). In that study patients assigned to cognitive remediation demonstrated significantly greater improvements in overall cognitive performance, learning and memory (California Learning and Memory Test, total words recalled, trials 1–5, and delayed free recall), and psychomotor speed (Trails B) than patients who received usual services. Thus effects of cognitive remediation on cognitive performance with the aid of COGPACK were similar between the two inpatient and outpatient samples.

In addition to greater improvements in cognitive functioning, patients assigned to cognitive remediation worked significantly more weeks than the patients in the computerized control group, and there were trends for the patients participating in cognitive remediation to work more hours and to earn more wages. These findings suggest that participation in cognitive remediation may have contributed to better work outcomes, as has been reported in three studies of outpatients involved in vocational rehabilitation ( 12 , 13 , 27 , 28 ).

It is unclear how cognitive remediation may have improved work performance. All patients in the study had equal access to jobs, and job interviewers were blind to the treatment group to which participants had been assigned. It is possible that improved cognitive functioning contributed to better work outcomes because cognitive functioning is a well-established correlate of work ( 29 ). It is also possible that some other aspect of the cognitive remediation program contributed to better work outcomes, such as learning compensatory strategies in the weekly group discussion for coping with cognitive impairment. Nonspecific effects of participation in the cognitive remediation program, such as improved motivation or sense of self-confidence, are another potential contributor to improved work outcomes ( 30 , 31 , 32 ). A greater understanding of the relationship between improved cognitive functioning and better psychosocial functioning in areas such as work is needed to identify appropriate cognitive targets for treatment.

A strength of this study is that the control group had the same number of hours of computer exposure and staff attention as the treatment group. Contrary to our results that indicated no cognitive benefit of a computer control intervention that consisted of playing computer games, the results of a recent study by Kurtz and colleagues ( 33 ) demonstrated comparable cognitive benefits between a computer control intervention consisting of training in general computer literacy and use of Microsoft Office and a cognitive remediation treatment program. In their study, however, working memory showed greater improvements in the treatment group. A recent meta-analysis of controlled studies of cognitive remediation in schizophrenia found no differences in effect sizes in cognitive performance between studies with active versus passive (treatment as usual) controls ( 34 ). In light of the findings of this meta-analysis, the different effects of the computer control groups in our study and that of Kurtz's team suggest that control interventions that systematically teach skills rather than simply engage the patient in some active manner may improve cognitive functioning. More work is needed to evaluate the effects of different types of control groups on cognitive performance in schizophrenia and to identify the common and unique features of active control programs and cognitive remediation programs, based on theories of how each program works.

Several limitations of our study should be noted. The relationship of the observed cognitive improvements associated with cognitive remediation and work performance could not be fully explored because we did not measure cognitive performance at the six- and 12-month follow-ups. Although we did not assess the quality of work, our finding that the cognitive remediation group earned significantly more money suggests a higher level of job performance in this group, because the jobs for which patients were eligible were paid within a range of $2.70 to $6.00 per hour, depending on the level of skills or knowledge required to perform the job. We also did not include measures of self-esteem and motivation, which have been shown to be improved by cognitive remediation in other studies ( 30 , 32 ). Cognitive remediation programs involve extensive positive feedback for effort and improved performance in which each patient works to improve his or her level of cognitive functioning. This in turn may reinforce motivation and improve a sense of self-mastery, which in itself could contribute to better cognitive performance.

We did not find any clear effect of cognitive remediation on change in symptoms as measured by the PANSS. In contrast, preliminary data have shown that cognitive remediation may improve some of the clinical symptoms of schizophrenia and performance of adaptive living skills ( 2 , 3 , 13 ). The effects of cognitive remediation may have been reduced by the relatively severe psychopathology of our inpatient sample.

Conclusions

This study is the first, to our knowledge, to evaluate the impact of cognitive remediation on work functioning in an inpatient sample. Improvements in cognitive functioning were evident after the cognitive remediation period, and better work outcomes were evident over the one-year follow-up period. These findings are encouraging considering the degree of cognitive impairment in this inpatient sample. Future research is needed to evaluate the impact of cognitive remediation on inpatients after discharge from the hospital.

Acknowledgments and disclosures

The authors report no competing interests.