Rehab Rounds: Cognitive Modifiability as a Measure of Readiness for Rehabilitation

Based on a growing body of research linking neurocognitive variables with social and work functioning (4), one might define capacity for rehabilitation operationally by using neurocognitive constructs. A pragmatic reason for determining readiness for rehabilitation based on cognitive measures is to facilitate intervention as early as possible in the treatment process. If cognitive functioning is a sensitive indicator of a person's responsiveness to social and vocational rehabilitation, services can be delivered to those most likely to benefit in an expeditious manner. Moreover, as the technology of cognitive remediation improves, we envision that an individual's readiness for rehabilitation will be accelerated and geared to his or her specific level of cognitive functioning.

A "dynamic" approach to assessing rehabilitation capacity using cognitive measures has been developed by Karl Wiedl and his colleagues in Germany. They have proposed the malleability of cognition or an individual's learning capacity as a proxy for rehabilitation readiness. In this column, Dr. Wiedl explains how he used the Wisconsin Card Sorting Test to categorize individuals in terms of learning capacity and correlated these findings with performance on a skills training exercise. In our UCLA Center for Research on Treatment and Rehabilitation of Psychosis, we have used "errorless learning" procedures to normalize the performance of patients with schizophrenia, thereby documenting the dynamic nature of neurocognitive functioning among patients with this disorder. The next generation of studies will examine the degree to which improvements in neurocognitive functioning will open the gates to better social and vocational functioning in the community.

Most studies of neurocognition among patients with schizophrenia have cross-sectionally evaluated stable cognitive abilities such as attention, concentration, memory, and executive functioning and shown how persons with schizophrenia perform significantly more poorly than normal control subjects (4). In an alternative approach, termed dynamic assessment (5), salient cognitive functions are assessed with respect to their modifiability. Dynamic assessment may provide clinically helpful diagnostic information because individual variability with respect to performance can be studied in response to interventions such as instructions and reinforcement. Assessment instruments used in the dynamic assessment battery integrate specific behavioral interventions into the testing procedures, in effect turning these instruments into learning or trainability tests. Theoretically, differences in individual performance on these tests would reflect differences in learning ability, cognitive modifiability, or even rehabilitation potential (5).

One test that has attracted considerable interest in cognitive remediation research is the Wisconsin Card Sorting Test (WCST) (6), a test of concept formation that is thought to be related to frontal lobe functioning. Although individuals with schizophrenia as a group perform worse than normal control subjects on the WCST, results indicate that there are significant differences between individuals with schizophrenia in their capacity to learn the operations required for this test (7), with some conservatively diagnosed individuals with schizophrenia performing well within the normal range.

Differences in cognitive modifiability among individuals with schizophrenia may be assessed and used to predict rehabilitation potential. We have conducted several pilot studies with two aims in mind: developing an algorithm for classifying subjects according to cognitive modifiability or learning capacity, which is assessed using a dynamic, learning version of the WCST, and estimating the validity of this classification with respect to clinical and experimental criteria.

Method of test administration

In the original version of the WCST, subjects were required to match 128 cards to one of four target cards. Matching rules are color, shape, or number of symbols. Subjects infer these rules from feedback about whether the match was right given by the tester after every match. After ten consecutive correct matches, the tester changes the rule without preannouncement.

In our studies, the WCST was given in three stages—as a pretest, as a test plus training, and as a posttest. All three stages were administered within a single session with each block comprising 64 cards. Standard WCST procedures were used for the pretest and the posttest (6). The test plus training block was administered with instructions and reinforcement according to the trial-by-trial intervention procedures described by Green and his colleagues (8).

After finishing the first block, subjects were told that they would now receive help in performing the task. Before starting the second block, the three sorting rules—color, shape, and number—were explained. After every card sort, the subjects also were told why their choice was right or wrong—for example, "That was wrong. We don't sort for color now, but for shape or number." Correct choices and changes of category were announced—for example, "Now that you have performed ten consecutive correct sorts, the rule will change. You will now no longer sort for color but for shape or number." Between each block of trials, five-minute breaks were provided. Altogether, administration of the WCST took between 30 and 45 minutes. Although a variety of test scores can be computed, number of correct responses was selected for analysis because of its normal distribution in this sample.

Classification of subjects by learner status

The algorithm we developed was derived by measuring change using residuals of linear regression (9). It was based on the internal consistency of the test and the standard error of prediction. For every pretest score, a test score on a hypothetical parallel test was predicted and compared with the real posttest score. With the help of the z-distribution, derivations of the real posttest score from the predicted parallel test score were designated as indicating change or no change (p=.05). Because a coefficient of internal consistency cannot be computed for the WCST, retest-reliability was used. A test-retest reliability of .77 for correct responses was found, indicating a relatively high level of reliability. Following this algorithm, a cutoff of 15 points, or correct responses, with a standard deviation of about 1.5 was set up for the classification of significant changes of performance, with floor and ceiling effects controlled.

Given a ceiling of 64 cards per block and the cutoff defined as above, improvement cannot be expected above a pretest score of 43 correct responses. Accordingly, subjects were classified as high-scorers if they scored 43 or higher in the pretest and posttest blocks, as learners if they had improved by at least 15 points between pretest and posttest blocks, and as nonretainers of learning if they did not meet criteria for either of the other two classifications. A category was also provided for subjects whose performance declined by at least 15 points; however, no subject met this criterion.

Testing the classification system

To test the classification system proposed above, studies were conducted with 56 participants in rehabilitation programs who met DSM-IV criteria for schizophrenia and were symptomatically stable on antipsychotic medication regimens (10,11,12). There was no correlation between use of typical versus atypical antipsychotic medication and learner status. Overall, participants had their lowest scores during the pretest WCST (mean±SD= 36.1±12.7), a significant increase after specific training (mean±SD=60± 3.5), and a significant decline at the posttest (mean±SD=49.3±12.1). Classification according to learner status yielded 13 nonretainers, 22 learners and 21 high-scorers. The scores of the different groups are illustrated in Figure 1. Correlational analyses and intergroup comparisons indicated that this classification system had good external validity because group assignment was related to proficiency in classification training (10), verbal learning (11), and the placement of the patients in rehabilitation settings (12).

In a clinical validation study that included 29 of the subjects tested in the previous study (six nonretainers, 12 learners, and 11 high-scorers), the classification was analyzed for its predictive validity in rehabilitation training. Three nonretainers, seven learners, and eight high-scorers were taking conventional antipsychotics, and three nonretainers, five learners, and three high-scorers were taking atypical antipsychotics. Cross-classification of learner status and type of medication was nonsignificant.

The 29 participants were trained in medication self-management techniques and problem-solving skills. The training consisted of eight 45-minute sessions conducted in small groups. The material was culled from the medication management module designed by the UCLA Clinical Research Center for Schizophrenia and Psychiatric Rehabilitation (13) and translated and adapted into German (14). For evaluation, subjects were given an interview structured according to the steps of a problem-solving task before and after the training. The task was to describe how to proceed in solving problems such as realizing, after arriving at work, that one has forgotten to bring medications from home. The subject was required to state the problem, list at least three possible solutions, identify at least one positive and one negative consequence for each solution, compare the advantages and disadvantages of the possible solutions, and select one solution and describe how it would be carried out. The specific problem situation was not the subject of training during the rehabilitation program.

Two independent raters determined whether the subjects' reports met the criteria of the five problem-solving steps assessed in the interview. Interrater reliability was excellent. Next, the scores obtained in the pre- and post-problem-solving interviews were compared for each subject. Subjects were rated as to their degree of improvement between the first and second sessions. Possible ratings included gain, no gain, and consistently high performance. No subjects had a decline in performance.

We compared subjects grouped by the ratings of their degree of improvement across the steps of problem-solving skills using the model of binomial distribution. The analysis showed that WCST nonretainers had a significantly higher proportion of no-gain ratings (86 percent) compared with the WCST learners and high-scorers (35 percent and 36 percent, respectively) (χ²=8.99, df=3, p<.03). Compared with the nonretainers, the WCST learners and high-scorers were more likely to be rated as showing gain and consistently high performance (65 percent and 64 percent, respectively, compared with 14 percent for the nonretainers) (χ²= 24.7, df=3, p≤.01 for comparison with the learners and χ²=20.4, df=3, p≤.01 for comparison with the high-scorers).

Conclusions

Our results showed that individuals with schizophrenia could be classified according to modifiability in concept formation using a dynamic assessment design that utilized the WCST in pretest, training, and posttest phases. Members of one subgroup—the high-scorers—had a high level of cognitive modifiability, suggesting that they may benefit from skills training approaches without the need for special cognitive remediation. The large number of poor performers could be differentiated into the subgroups of learners, who improved durably with the help of a specific cognitive intervention, and nonretainers, who showed only transient improvement.

Classification of learner status was related to progress in a skills training group. Whereas WCST nonretainers were clearly unable to gain adequate knowledge of problem-solving techniques, a substantial proportion of the WCST learners and high-scorers were able to demonstrate that they had learned the problem-solving steps. These results suggest that the distinguishing feature between the learners and the high-scorers is not in the level of performance after skills training, but the former group's ability to benefit from verbally mediated instruction. Classification according to cognitive modifiability, using WCST learner status or an analogous measure, may thus be a useful tool for rehabilitation planning for patients with schizophrenia, much in the way that evaluation of cognitive modifiability has been utilized in assessment and treatment of individuals with developmental disabilities and learning disorders (15).

Afterword by the column editors: The rehabilitation readiness test proposed by Dr. Wiedl is simple and can thus be used in ordinary clinical practice to help the practitioner predict the chance of success in rehabilitation training. However, it is important to add that patients classified as nonretainers should not be excluded from training. Rather, interventions should be conceived that fit the special needs of those who score poorly on neurocognitive tests such as the WCST. Over the years, many creative methods have been shown to improve the quality of life of individuals with treatment-refractory schizophrenia (16,17,18). Therefore, cognitive modifiability as an aspect of rehabilitation potential should not be defined as a trait, but rather as a way of helping patients with differing cognitive impairments to meet their needs for rehabilitation.

One question that will have to be answered by empirical data is whether indirect, cognitive, or proxy measures of learning capacity—such as patients' responses to training on the WCST—are a superior means of determining rehabilitation readiness, compared with more direct, behavioral methods. One example of a direct measure of capacity to learn social and independent living skills comes from the work of Charles Wallace, Ph.D., at the UCLA Center for Research on Treatment and Rehabilitation of Psychosis. Wallace and his colleagues have found that a micro-module test of ability to learn an assortment of skills is an excellent predictor of how well the individual will learn skills in a psychosocial rehabilitation program. Micromodule tests have been devised for predicting rehabilitation responsiveness to several of the programs included in the UCLA social and independent living skills series.

Dr. Wiedl is professor of clinical psychology at the University of Osnabrück, Knollstrasse 15, D-49069 Osnabrück, Germany (e-mail, [email protected]). Alex Kopelowicz, M.D., and Robert Paul Liberman, M.D., are editors of this column.