Dr. Watzke is affiliated with the Department of Psychiatry, Psychotherapy, and Psychosomatics, Martin Luther University of Halle-Wittenberg, Julius-Kuehn-Str. 7, Halle-Saale 06097, Germany (e-mail: stefan.watzke@medizin.uni-halle.de). Dr. Kuss is also with the university, at the Institute of Medical Epidemiology, Biostatistics, and Informatics. Dr. Brieger is with the Department of Psychiatry, Bezirkskrankenhaus Kempten, Kempten, Germany. Dr. Schoettke and Dr. Wiedl are with the Department of Psychology, University of Osnabrueck, Osnabrueck, Germany.

Vocational rehabilitation for patients with severe mental illness aims for reintegration into employment and invigoration of the individual's functional (1) and work-related skills (2,3). In schizophrenia, symptoms (4), premorbid functioning (5), intelligence (6), illness onset, and duration of the illness (7) have been found to be related to rehabilitation success. However, the predictive values of these variables remain unsatisfactory.

In contrast, cognitive impairments have immense consequences on adaptive skills (8,9) and are highly relevant for everyday functioning (10), as well as for occupational outcome in schizophrenia (11,12). Neurocognitive deficits are widely recognized as central features (13,14) and represent stable characteristics of the disease (15) mostly independent from symptom severity (16).

Still, most studies examining the relation between cognition and social functioning have evaluated static cognitive abilities (17). In contrast, with reference to research based on dynamic assessment (18,19), Green and colleagues (11) addressed learning potential as a mediator between basic cognition and functional outcome and suggested "a fundamental shift in assessment: from what the individual currently knows to what the individual is capable of learning." Dynamic assessment (18,20) is a diagnostic approach that incorporates specific behavioral interventions into cognitive testing procedures and thus turns these procedures into learning or training tests (21). Changes in individual performance on dynamic tests reflect the individual's cognitive modifiability or learning potential.

Methodologically, dynamic assessment applies a typological approach (described in the Methods section) that results in dividing the sample into a group of nonlearners (without improvement of test scores within the dynamic test procedure), learners (significant improvement of test scores), and high scorers (superior test performance throughout the test procedure). External validity of this a priori classification has been shown in several studies where learner groups differed in their performance on tests of verbal memory (22,23), sustained attention (24), and auditory divided attention (25). Learning potential also was related to the performance in manualized skill training groups (26), the outcome of cognitive rehabilitation training (27), training of social problem solving (19), and cognitive and social rehabilitation training (22). High scorers and learners always showed better performance and achieved higher gains than nonlearners.

In contrast, Woonings and colleagues (28) reported in their study that learning potential was not associated with social functioning before or after participation in a rehabilitation program. However, closer inspection of their results indicates that patients with low cognitive modifiability (nonlearners) showed lower levels of social functioning than patients with higher cognitive modifiability (learners) both at baseline and after program termination. Sergi and colleagues (29) found that learning potential predicted the patients' work performance immediately after training as well as at a three-month follow-up.

These past studies mostly used cross-sectional designs with criteria assessed concurrently with dynamic testing. Prospective studies have not been reported yet. Knowledge about longitudinal relations between dynamic testing and rehabilitation outcome is needed to clarify the degree to which learning potential contributes to a better understanding of future social and vocational functioning (19).

The aim of this study was to determine the relationship between individual learning potential and the success of vocational rehabilitation for patients with schizophrenia or schizoaffective disorders. The following questions were to be answered. Is it possible to classify persons according to different patterns of learning potential in a comprehensive vocational rehabilitation program for patients who are chronically mentally ill? Do persons with different learning potential as measured by dynamic assessment differ in their rehabilitation outcome, as measured by work capability during rehabilitation, general level of functioning, and level of vocational integration after program termination?

Research participants

The study took place within the framework of the research project "Vocational Rehabilitation and Integration of Persons With Mental Illness" (30), which was approved by the local research ethics committees. Study participants were recruited from a comprehensive vocational rehabilitation program for persons with chronic mental illness in Halle-Saale, Germany. The program aims to improve work skills and future vocational prospects for people with mental and social impairments through support and individual training under simulated (sheltered) work conditions (30).

The program took place in the Rehabilitation Psychisch Kranker (Institution of Rehabilitation for the Psychiatrically Ill and Disabled) in the city of Halle. The institution contains classrooms for teaching and enclosed workshops for woodwork or metal crafts. Patients initially stay in the program for nine months and can complete three to six more months if recommended by the supervisors. The patients' mean±SD length of stay in the rehabilitation program was 332±96 days.

In the first six to eight weeks (diagnostic phase), the rehabilitation team evaluated the patients to determine their current mental health status. The seven-hour (8 a.m. to 3 p.m.) daily program comprises several courses of general vocational training as well as classes in specialized subjects such as mathematics, basic computer skills, administration or bookkeeping, and metal crafts or woodwork (training and stabilization phase). Depending on individual needs, courses are offered for training in social communication and social competence, concentration and memory, as well as management of daily activity. In the course of the program, demands are increased qualitatively and quantitatively in a stepwise manner but supplemented by stress relief and relaxation periods to support and stimulate the patients. Medical and psychotherapeutic treatments as well as social support accompany the rehabilitation program.

Work training in natural settings takes place within the last three months of the program (hands-on phase), after which patients return to the rehabilitation institution for final evaluation. Eighty-two patients were admitted to the program from November 2002 to January 2004. Program participants were eligible for the study if they fulfilled the DSM-IV diagnosis of schizophrenia or schizoaffective disorder, were symptomatically stable on neuroleptic medication, had German as their native language, were between the ages of 18 and 65 years, had no co-occurring diagnosis of substance abuse or dependence according to DSM-IV, and had premorbid IQ not lower than 70 points.

Thirty-nine rehabilitation participants had mental disorders other than schizophrenia or schizoaffective disorders or mismatched other inclusion criteria. Two patients refused to participate. Thus the study sample consisted of 41 persons (16 women), all of whom completed the program and the study.

Measures

The Wisconsin Card Sorting Test (WCST). The WCST (31) is a widely used measure of concept formation and cognitive flexibility. Under standard administration, respondents are required to match 128 stimulus cards containing different geometric shapes (varying in color and number) to one of four target cards. After each response, test takers receive feedback that the response was right or wrong. Respondents are not informed of the rule for correct matching or that the rule changes after they have matched ten consecutive cards.

In this study the WCST was conducted in a pretest-posttest sequence within one session, with each block comprising 64 cards. Pretest (trial A) and posttest (trial C) were identical to the standard procedure. Between trial A and trial C, a training block (trial B) was administered in accordance with the approach of dynamic testing (21) and with the trial-by-trial interventions described by Green and colleagues (32). The sorting rules were explained to the research participants, and they were told whether and why their response was right or wrong (for example, "This was right; we're actually sorting for color" or "This was wrong; we aren't sorting for color now, but for form or number"). In addition, participants were informed of category change (for example, "With ten consecutive correct sorts now completed, the rule will change. You will no longer sort for color but for form or number").

For the analyses of intraindividual change of performance, the number of correct responses was selected on the basis of its advantageous distributional characteristics. Change was assessed with a model developed by Schoettke and colleagues (33), which has a psychometric basis, controls for ceiling and floor effects, and is appropriate for its ease of clinical application. It provides distinct categories of individual learning potential (learner status), is based on the analysis of the individual case, and was successfully validated in various studies (19,22,23,24,25,26,27). On the basis of the standard error of prediction, a confidence interval derived from the model that specified the range wherein deviations from a hypothetical parallel test score (on subsequent test administrations) could be considered as indicators of real change or not. The algorithm results in the following classification rule: for significant change of intraindividual performance, a cutoff of 15 points was defined, which represents approximately 1.5 standard deviations in the original study.

With a maximum of 64 cards per trial, measurable improvement cannot be expected above an initial score (trial A) of 43 correct responses. Participants were classified as high scorers when they scored 43 or higher in trials A and C. Persons who improved by at least 15 points from trial A to trial C were classified as learners, whereas those who did not improve their performance by a minimum of 15 points and did not achieve a score of 43 in trials A and C were classified as nonlearners. Study participants who scored 43 or better on trial A and less than 43 on trial C also were classified as nonlearners. Finally, a category for participants whose performance declined by at least 15 points was provided.

Control variables. For the assessment of sociodemographic data and characteristics of the illness (age at first admission, duration of the illness, and type and dosage of medication), a standardized sociobiographic interview was administered at enrollment to the program (time 1). The severity of psychiatric symptoms was assessed at each assessment point (times 1 to 4) with the Positive and Negative Syndrome Scale (PANSS) for schizophrenia (34,35). The PANSS is a 30-item rating scale with three rationally derived subscales (positive symptoms, negative symptoms, and general symptoms).

Verbal intelligence as a measure of premorbid performance was tested at study intake (time 1) with subtests 1 and 2 of the Performance Assessment System (Leistungspruefsystem, or LPS) (36). This valid and reliable instrument (retest reliability of .95) assesses the main factors of intelligence outlined by Thurstone (37). Both subtests were administered with standardized instruction and were conducted with time limitations. The patients were required to identify misspellings in a succession of words of increasing difficulty.

Outcome measures. For the assessment of work capabilities at program intake (time 1), during rehabilitation (time 2), and at program termination (time 3), we used the Osnabruecker Arbeitsfaehigkeitenprofil (O-AFP, or Osnabrueck Ability to Work Profile) (38). The O-AFP represents a reliable translation and adaptation of the Work Personality Profile (39) for use in Germany. This profile is especially suitable for programs that most closely simulate the general work environment. It consists of 30 items scored on a 4-point scale, ranging from 1, "problem area that will definitely limit the chances for employment," to 4, "a definite strength, a plus for employment." Assessment was provided by the patient's immediate supervisor and based on the general labor market and its demands as standard. A rating of 3 to 4 means that the rehabilitation participant is able to work in the general labor market; a 2 indicates likely employment in the sheltered work environment.

Assessment of the rehabilitation participant was performed on three subscales of the O-AFP, each with ten items: learning ability, social and interactional competence, and social adaptation and motivation. Internal consistency ratings for these subscales were, respectively, α=.954, α=.909, and α=.899.

General functional outcome was measured by the Level of Functioning Scale (40). With this four-item rating scale, outcome is measured quantitatively according to four dimensions, each rated on a 5-point scale: work (from 0, none, to 4, daily work activities including studies, housework, sheltered employment, and day center activities within the past year [time 1] or since the last assessment [times 2 to 4]), symptomatology (from 0, severe, to 4, no symptoms within the past month), social relations (from 0, no social relations, to 4, meeting friends at least once a week), and hospitalization (from 0, more than 75% of the time, to 4, never within the past year [time 1] or since the last assessment [times 2 to 4]).

In accordance with Reker (7), the level of vocational integration was assessed qualitatively with an ordinal scale containing five levels: level 1, competitive employment or regular apprenticeship; level 2, work or apprenticeship under protected conditions; level 3, vocational rehabilitation for persons with mental illness; level 4, unemployment (no current employment but being available for the job market); and level 5, permanent disability payments (no current employment and not being available for the job market). This scale takes into account that, under the present economic conditions in Germany, vocational integration is often reached only partially through integration in protected work environments or continuing vocational rehabilitation for patients with severe mental illness. The level of vocational integration was assessed on the basis of the patients' level of vocational activity at the time of the follow-up assessment.

Procedures

We applied a repeated-measures design with four assessment points: initial assessment at time 1 took place at the patient's admission into the rehabilitation program; time 2 assessment was conducted after 26 weeks of attendance in the program. Time 3 assessment took place at individual program termination, which was after 332.0±96.1 days' stay in the program. Time 4 assessment was conducted three months after individual program termination.

Time 1 assessment started after study participants had given their written consent. Sociodemographic data and characteristics of the illness were obtained, and diagnostic assessment was conducted with the German version of the Structured Clinical Interview for DSM-IV (41). Within the first session, psychopathology, as determined with the PANSS, was examined, and level of functioning was initially rated. During the second week of the rehabilitation program, we tested intelligence (with the LPS performance assessment) and assessed executive functioning and learning potential (with the WCST). Patients were classified into the WCST learner groups according to the classification algorithm (33).

The rating of work capability (O-AFP) was provided by the patient's immediate supervisor in a 15-minute consultation of all rehabilitation staff members working with the patient. The time 1 O-AFP evaluation took place only about four weeks after the patient's entrance into the rehabilitation program and therefore represents no true baseline. This delay occurred because some exposure to the rehabilitation environment is necessary before attaining a score on this instrument.

At times 2 and 3 psychopathology (PANSS), level of functioning and work capabilities (O-AFP) were assessed. At time 4 assessment included rating of psychopathology (PANSS), level of functioning, and the level of vocational integration or reintegration.

Analyses

After completion of the WCST at time 1, study participants were classified according to their individual learner status with the algorithm (33) described above. Correlations of covariates and outcome at baseline were assessed by the Spearman correlation coefficient.

In a first evaluation step, we calculated a doubly repeated measurement model (42) to account for the two levels of correlation in our data set (first, with the repeated measurement-in-time scale, and second, by several outcome measures at each time point) simultaneously. Next, group comparisons of the single-outcome measures were evaluated with repeated-measures models, including a group factor (WCST learner status), a time factor (measures at different time points), a group × time interaction, and the PANSS measurements as covariates.

For the intraindividual correlation in the time course, we initially hypothesized four different patterns: independent, compound symmetry, first-order autoregression, and completely unstructured. For each outcome measurement model, we chose the intraindividual correlation pattern that resulted in the best model fit according to the Bayes information criterion. Group effects on the different time points were tested by post hoc tests in the respective models. As a sensitivity analysis, we performed nonparametric analyses for the four outcome measures, following the ideas of Brunner and colleagues (43). Because the results were essentially unchanged, we report only the results of the parametric analyses. Finally, differences on vocational integration between groups were assessed by nonparametric methods (Kruskal-Wallis and Wilcoxon tests). Statistics were computed with SPSS 12.0 and SAS 9.1.3.

Demographic and illness-related characteristics of the participants are provided in Table 1.

Group formation and group differences in control variables

Learning potential classification yielded 18 high scorers (44%), 16 learners (39%), and seven nonlearners (17%). No participants with a declining performance of a minimum 15 points were identified. Groups did not differ in gender, age, LPS intelligence subtests 1 and 2, education level, age at first psychiatric hospitalization, duration of illness, or duration of participation in the rehabilitation program. We found no significant group difference in the distribution of the type of schizophrenic illness, neuroleptic medication, daily dose of chlorpromazine equivalents, and PANSS psychiatric symptoms at time 1.

However, baseline negative symptoms were correlated with the Level of Functioning score (r=-.437, p=.004) and the O-AFP social adaptation and motivation subscale (r=-.384, p=.013) at time 1. PANSS general symptoms were found to be correlated with level of functioning (r=-.477, p=.002). Thus PANSS symptoms were entered as time-varying covariates in the analyses described next.

Group differences in outcome measures

The doubly repeated measurement model resulted in a significant overall group effect for the WCST learner status (F=4.62, df=8 and 111, p<.001), a significant effect of time (F=6.25, df=9 and 342, p<.001), and a significant group × time interaction (F=1.99, df=18 and 342, p=.01). The model results for the specific outcome measures are reported next.

Work capability.Table 2 presents the O-AFP data for the learner groups at the first three assessment points. With PANSS subscales entered as time-varying covariates, a hierarchical linear model for the learning ability O-AFP subscale yielded a significant effect for the group factor (p=.004) but not for the repeated-measures factor or for the interaction. Tests of effect segments yielded a significant group effect for all assessment points (time 1, p=.006; time 2, p=.004; time 3, p=.039).

At time 1 learners did not differ from high scorers and from nonlearners, whereas high scorers differed from nonlearners (p=.002). At time 2 learners differed from high scorers (p=.024) but not from nonlearners. Again, high scorers and nonlearners differed significantly (p=.002). At time 3 learners did not differ from high scorers, but learners differed from nonlearners (p=.017), and nonlearners and high scorers still showed a significant difference (p=.011). For the O-AFP subscale of social and interactional competence, no significant group effect or effect of repeated testing was found, but there was a significant group × time interaction (p=.038). However, post hoc tests showed no significant difference between any combinations of factor levels. On the social adaptation and motivation subscale of the O-AFP, neither a significant group effect nor an effect of repeated measurement and interaction was found.

Level of functioning. General level of functioning according to the Level of Functioning score for the three learner groups and all four assessment points is presented in Figure 1. A hierarchical linear model with PANSS subscales entered as time-varying covariates yielded a significant effect for the repeated-measures factor (p<.001) but not for the group factor and the group × time interaction. Lack of statistically significant differences over the long term did not preclude different levels of outcome in the learning potential groups after program termination. To test for this, we conducted tests of effect segments that yielded a significant group effect for time 4 (p=.002). Single comparisons for time 4 showed that learners did not differ significantly from high scorers but differed significantly from nonlearners (p=.001). High scorers significantly differed from nonlearners (p=.001).

Level of vocational integration. At time 4 groups significantly differed in their level of vocational integration (Kruskal-Wallis χ²=8.29, df=2, p=.016): six of the high scorers (33%) and six of the learners (33%) but none of the nonlearners were competitively employed. Another six of the high scorers (33%), one of the learners (6%), and none of the nonlearners were integrated in work programs under supported conditions. No high scorer, one of the learners (6%), and one of the nonlearners (14%) took part in another rehabilitation program for persons with mental illness. Six of the high scorers (33%), eight of the learners (50%), and four of the nonlearners (57%) were unemployed. Two of the nonlearners (29%) but none of the high scorers and learners received permanent disability payments from social pension funds.

Post hoc single comparisons (Bonferroni adjustment for three comparisons from α=.05 to α=.017) revealed no difference between learners and high scorers, but there was a trend toward a difference between learners and nonlearners (Mann-Whitney U=24.50, p=.022) and a significant difference between high scorers and nonlearners (Mann-Whitney U=18.00, p=.004).

Cognitive dysfunctions limit the success of vocational reintegration of patients with schizophrenia (44). Dynamic testing enabled us to assign individuals with initially low cognitive performance to a group with sufficient learning potential (learners) and a group with stubborn (32) impairments (nonlearners), whereas persons with constant superior performance were identified as high scorers. All three groups were found in unevenly distributed but considerable proportions in our sample, demonstrating that the paradigm of dynamic testing and learning potential classification is not limited to clinical samples but can also be applied in rehabilitation samples.

Our study is one of the first to examine the relation between learning potential classification and repeated measures of functional outcome and vocational integration under statistical control of psychopathology. High scorers were generally superior to nonlearners regarding work-related learning ability, level of functioning, and level of vocational reintegration at follow-up. This finding is in line with a large body of research on the relation between functional outcome and cognitive performance in schizophrenia showing that unimpaired cognitive performance is related to better functioning (17).

However, the results regarding the group of learners as opposed to nonlearners contribute to a better understanding of the role of learning potential as a mediator of functional outcome. Although initially the cognitive performance of learners was impaired similarly to the nonlearners' performance, learners showed an advantage in work-related learning during the course of rehabilitation and a significant superiority in comparison with the nonlearners at the end of the rehabilitation program. We found that learners had a higher level of functioning than nonlearners at a three-month follow-up (about 15 months after the initial testing). Also, whereas 66% of high scorers and 44% of learners were integrated into competitive employment or supported work programs, none of the nonlearners found paid work at follow-up.

Using only conventional measures of WCST would not have enabled us to differentiate between nonlearners and learners and would have missed the implications of this differentiation. These results indicate that the ability to benefit from training on instructions within the testing situation generalized to patients' profiting from the rehabilitation program.

Our results are consistent with other studies supporting the usefulness of classifications of learning potential in predicting rehabilitation success. High scorers, learners, and nonlearners differed in their performance in manualized skill training groups (26), in cognitive rehabilitation training (27), problem-solving training (19), and cognitive and social rehabilitation training (22).

Green and colleagues (10) emphasized that cognitive performance is especially related to the change of functional outcome and therefore is important in predicting the patient's gain from psychiatric and rehabilitation services. Assessment of learning potential broadens this view by indicating which patients are likely to benefit from programs and which patients need further or different support. And in contrast to the studies reported above, our contribution extends these results with regard to long-term prediction.

Our findings should also be discussed in regard to their implications for the further development of vocational rehabilitation programs. Further research should test whether comprehensive rehabilitation programs or programs of supported employment are an appropriate strategy for all patients or whether nonlearners are likely to benefit more from other approaches. Those patients may require more sweeping and sustained support, as provided by errorless learning (45,46). Accordingly, Sergi and colleagues (47) reported that patients with low cognitive learning potential had better rehabilitation success in errorless learning than in standard rehabilitation.

The following restrictions of our study have to be considered. Inclusion of nearly all patients with schizophrenia or schizoaffective disorders in the admission period was realized; nevertheless the resulting small sample essentially limits the generalizability of the results. There might also be a preselection bias because the patients were chosen for the program by social welfare institutions on the basis of criteria beyond the scope of this study. Nonetheless, the proportion of nonlearners (17%) in this study corroborates with general clinical experience and previous research showing proportions from 16% (28) to 23% (19). Small samples primarily restrict the generalizability of the results because of lower statistical power. We presume that the trends shown by the study results would be statistically significant with a larger sample.

Data arose from only one specific rehabilitation approach and therefore represent only a limited part of the rehabilitation spectrum for patients with severe mental illness. Accordingly, the results may be valid for future cohorts of comparable programs and patients with similar demographic characteristics. Generalization should be made only with appropriate caution. These limitations necessitate a replication of our study with a larger sample (an estimated sample of 120 patients would be needed to obtain the appropriate group size of 20 nonlearners) and with patients from different rehabilitation programs.

The algorithm for the identification of distinct learner groups (33) has been successfully applied in many studies. Nevertheless, it should be scrutinized in regard to flexible application in samples with different levels of cognitive performance and the algorithm's tendency to promote ceiling effects.

Our study supported the assumption that learning potential is an informative predictor for rehabilitation outcome and may add information beyond static cognitive measures. Results should be validated with larger samples of patients in different rehabilitation programs. Learning potential should be considered in the further development of vocational rehabilitation programs.

This study was supported within the framework of a larger research project by grant 01-GD-0116 from the Forschungsverbund Rehabilitationswissenschaften (Research Network for Rehabilitation Sciences), Sachsen-Anhalt/Mecklenburg-Vorpommern.

The authors report no competing interests.