Development and Validation of a Measure of Patients' Knowledge About Schizophrenia

Attempts to measure patients' knowledge about schizophrenia are currently proliferating in response to the growing prevalence of patient education programs and the need to evaluate patient outcomes. Available instruments in the English language vary greatly in their item content, mode of administration, and scoring method, but they appear to have in common a lack of demonstrable psychometric properties.

Differences among measures in item content reflect a focus either on medication management (1) or on diverse aspects of the illness, including but not limited to medication management (2,3,4,5,6). Instruments tend to vary in the mode of administration, including patient self-reports (2,3,4,5) and clinician ratings (6). The scoring methods have included varied multiple-choice formats (2,3,4,5) and scoring by clinicians of open-ended questions (6).

Additional tools for assessment of patients' knowledge about schizophrenia are available in French (7) and German (8,9), with one of them offering substantial documentation of its psychometric properties (8).

This paper describes the development and validation of an English-language measure of patients' knowledge about schizophrenia. Its psychometric properties were investigated in four studies, including one cross-validation study. The studies provided information about the questionnaire's construct validity and its reliability, as manifested by precision, internal consistency, and stability indicators.

Methods

The Knowledge About Schizophrenia Questionnaire (KASQ) was derived from a pool of 78 items, including 13 items from a measure devised originally for relatives of patients with schizophrenia (4). Items were aimed at measuring patients' knowledge about the diagnosis of schizophrenia and its prevalence, etiology, course, and prognosis, as well as medication and side effects, nondrug treatments, stress factors, and legal issues. A total of 25 items were judged by four educators to best reflect the content areas covered in a psychoeducation program for patients diagnosed as having schizophrenia (10). A copy of the KASQ is available from the author.

Participants were 136 adult inpatients at a Midwestern state psychiatric facility, diagnosed using DSM-IV criteria as having schizophrenia (128 patients, or 94 percent) or a schizoaffective disorder (eight patients, or 6 percent). Eighty-two patients (60 percent) were male, and 121 (89 percent) were single. A total of 124 patients (91 percent) had a high school education or less. The mean±SD age of the patients was 35±11.4 years (range, 18 to 58 years). All participants were undergoing a medication regimen with antipsychotic medications.

The patients participated in four studies conducted between February 1990 and April 1995. Patients were referred by their treatment teams to the group education program, which offers daily one-hour sessions over a three-week period and which is led by various mental health professionals. Study 1 involved administration of the KASQ before and after participation in the 15-session educational intervention. Study 2 proceeded with cross-validation, by repeating study 1 with a different sample of 53 patients who participated in the education program about a year later. Study 3 involved test and retest, at a three-week interval, of ten patients who did not participate in previous studies and did not attend the education program.

Studies 1 and 2 included 53 patients each, and study 3 involved ten patients. Study 4, a controlled study, involved 20 patients, all diagnosed as having schizophrenia, whose mean± SD age was 36±10.8 years. In study 4 the mean±SD number of previous hospitalizations for the 20 patients was 5.7±2.9, and the mean±SD number of years of education was 12.5± 2.9. All 20 patients were single, and 13 patients (65 percent) were male. The mean±SD current Global Assessment of Functioning score for the patients in study 4 was 33±8.3.

Study 4 involved random assignment of 20 patients to the education group or a waiting-list control group. Each group completed the test twice—the former group completed it before and after the intervention, and the latter group waited three weeks before taking it a second time. The groups did not differ significantly in gender, age, or marital status. Individual differences in guessing tendency were kept under control by instructing patients to guess on every item that they were unable to answer. Attempts were made to balance the response formats by varying the frequency of the correct alternative and the number of alternatives per item.

Results

Reliability

Estimates of the KASQ's internal consistency and precision were calculated for study 1 and again for its cross-validation with a different sample in study 2. A high coefficient alpha (KR-20) was found in both studies—.89 and .85, respectively. Internal consistency was further measured using item-to-total correlation to estimate the contribution of each item to the reliability of the test. The average item-to-total correlation coefficient was above the appropriate value of .40. It was .51 in study 1, and .46 in study 2.

The item difficulty level, as indicated by the percentage of correct responses to the item, pointed to a balanced mix of difficulty levels, with an average difficulty level of 75 percent for both studies combined. Table 1 shows the difficulty level and item-to-total correlation for each item, averaged across studies 1 and 2.

Special attention was given to three items in which the correct multiple-choice response is "all of the above", because we were concerned about the prudence of using this level of complexity with our compromised psychiatric population. Results were surprising, indicating that the difficulty level of these items was only intermediate (a mean difficulty level of 68 percent, and a range of 64 percent to 74 percent). More important, the item-to-total correlations for these items were high (mean of .64; range of .62 to .75). The correlations indicated that they were among the items on the test with the best psychometric properties.

Another aspect of reliability, that of precision and stability of scores, was investigated in study 3, using test-retest at a three-week interval of patients who did not participate in the education program. The analysis yielded a high and significant correlation of .83 (p<.005).

Validity

To demonstrate that the KASQ measures what it intends to measure and that it is sensitive to change resulting from deliberate interventions, data from studies 1, 2, and 4 were examined. Results from study 1, in which subjects completed the KASQ before and after participating in the education program, indicated that participants significantly improved their score by an average of 20 percent (t=5.3, df=51, p<.001).

These findings were replicated in study 2 with a different sample one year later; study 2 patients showed a 21 percent average improvement in their score (t=6.5, df=51, p<.001). The controlled study, study 4, provided further support for the test's validity and sensitivity. Results showed a lack of significant group differences on the KASQ before the educational intervention; however, scores were significantly different for participants in the educational program when they completed the KASQ a second time (an average 18 percent improvement; t=2.5, df=9, p<.001). The control group's mean KASQ scores did not change significantly.

Discussion

This study found that the Knowledge About Schizophrenia Questionnaire has sound psychometric properties. Its reliability, as measured by precision, internal consistency, and stability indicators, is high, and its sensitivity to change indicates good construct validity. These properties, which were replicated in a cross-validation study, along with its ease of administration and scoring, support its use in outcome studies in which comprehensive patient education about schizophrenia is provided.

However, because this measure was initially constructed to evaluate a particular education program (10), it may not be readily applicable to other education programs. Thus the utility of the KASQ may be restricted to this and similar education programs that go beyond education about medication management and include information about the diagnosis of schizophrenia, its prevalence, etiology, course, prognosis, nondrug treatments, stress factors, and legal issues. This type of comprehensive patient education is becoming more prevalent partly due to growing emphasis on psychosocial rehabilitation and patient empowerment and partly due to the need to meet standards of care for patient education mandated by the Joint Commission on Accreditation of Healthcare Organizations.

The KASQ's psychometric properties appear comparable to those of the ERWIPA, a German instrument for evaluating patients' knowledge of their illness (8). However, no underlying factor structure for the KASQ was demonstrated, nor does it have alternate forms. (A parallel or alternate form of an instrument tends to have similar psychometric properties, which allows repeated measures without concern for the effect of memory on retesting.) The KASQ was found to be more reliable than the ERWIPA, and patients' KASQ scores were shown to be stable over time. In addition, the KASQ has varied response alternatives (in addition to true, false, and do not understand), and a cross-validation study of the KASQ confirmed its reliability.

Acknowledgments

The author thanks Ann Claveaux, M.A., Sandra Rochford, R.N., Donna Cisco, C.C.S.W., and Eric Martin, Ph.D., for assistance with the patient education program and the outcome projects.

Dr. Ascher-Svanum is chief psychologist in the general psychiatry service of Larue Carter Hospital, 2601 Cold Spring Road, Indianapolis, Indiana 46222-2202 e-mail, [email protected]. She is also assistant professor of clinical psychology in the psychiatry department at Indiana University School of Medicine. An earlier version of this paper was presented at the annual meeting of the American Psychological Society held June 29-July 2, 1996, in San Francisco.

Enlarge table