A Self-Administered Instrument to Measure Psychiatric Discomfort of Persons With Serious Mental Illness

A self-report instrument is needed to quantitatively measure the intrapsychic discomfort of persons with serious mental illness—one that places little burden on the respondent and is sensitive to changes over time. Clinicians now recognize that self-reports about inner states can be valid and reliable (1,2).

Relapses occur frequently in persons with serious mental illness, and a considerable body of knowledge exists about prodromal symptoms of impending illness (3,4). Early intervention with medication and crisis counseling at the start of a prodromal period can return most patients to their former level of symptoms and role functioning and prevent costly hospitalizations (5). In addition to providing information to the clinician about the client's subjective state, a quantitative instrument also is needed to document clinical outcomes.

Data from descriptive studies suggest that individuals have consistent prodromal signs from relapse to relapse. However, individuals vary greatly in the symptoms experienced (6,7). Assessment of psychiatric discomfort may predict an acute onset of illness and allow for early intervention in an outpatient setting. This paper describes the development and evaluation of an instrument to assess psychiatric discomfort.

Methods

Instrument development

The Psychiatric Discomfort Scale (PDS), the instrument developed in this study, is historically related to the Hopkins Symptom Checklist (8), which was subsequently revised and renamed the Derogatis Symptom Checklist 90-R (9). The first version of the PDS consisted of 60 items representing the clinical presentation of symptoms of depression, psychosis, and mania, plus subjective states associated with akathisia, interpersonal sensitivity, and anger-hostility. The PDS asks subjects to answer the question "How much am I bothered by ______" by checking a 5-point scale; 1 indicates not at all; 2, a little bit; 3, moderately; 4, quite a bit; and 5, extremely.

A pilot study was conducted with 26 psychiatric inpatients at a Veterans Affairs medical center in 1993. Patients were receptive to the PDS and able to complete it with little or no difficulty, although one patient refused to answer any of the questions. Using item-remainder coefficients, 20 weak items were found in the instrument. The decision was made to modify the scale, deleting the weak items, and to conduct a study to gather further evidence on the reliability and validity of the PDS.

Evaluation design

Phase 1

Phase 1 of the study was conducted on an inpatient psychiatric unit of a VA medical center during a three-month period beginning in October 1993. We gathered evidence on the instrument's validity and reliability by asking four questions: Do the PDS summary scores for a psychiatric population differ significantly from the summary scores for a nonpsychiatric population? Is there a relationship between PDS scores and the scores from known instruments? What is the internal reliability (Cronbach's alpha) for the PDS? Is the PDS responsive to changes in a patient's clinical status? The first two questions address construct validity, the third evaluates internal consistency, and the fourth assesses the sensitivity of the instrument to change.

Patients were asked to complete the PDS on admission to the hospital, on day 3, and at discharge. Veterans receiving outpatient medical treatment were used as a nonpsychiatric comparison group.

Two instruments commonly used for evaluating patients' perceptions of symptoms in drug trials are the Bunney-Hamburg Scale and the Modified Nurses' Observation Scale for Inpatient Evaluation (NOSIE). Three days after admission, the primary care nurse completed the Bunney-Hamburg Scale and the milieu nurse completed the NOSIE scale for each patient. Responsiveness of the PDS to changes in the patient's condition was evaluated by comparing summary scores of discomfort that were recorded at admission, on day 3, and at discharge. The items to be retained in the instrument were assessed for clarity, comprehensiveness, and redundancy by an expert in instrument development.

Phase 2

The second phase of the study field tested the modified instrument in two centers treating outpatients diagnosed with psychosis. This phase took place between January 1996 and January 1998. Possible scores on this version of the PDS ranged from 35 to 175, with higher scores indicating higher levels of discomfort.

In this part of the study, we sought to answer two questions. What is the factor structure of the PDS when administered to a population with a clinically recognized condition of psychosis? (The factor structure is the degree of relatedness of the items on the questionnaire.) Does the modified PDS have internal consistency as measured by the Cronbach's alpha?

Results

Phase 1

A total of 55 psychiatric inpatients participated in phase 1, and the comparison group consisted of 71 medical outpatients. The mean±SD PDS score of the inpatient sample differed significantly from that of the outpatient group (112.62± 48.24 versus 63.29± 23.91; F=4.07, df=54,47, p<.001). The self-ratings on the PDS were not significantly correlated with staff ratings on the Bunney-Hamburg scale or the NOSIE scale. However, we also found no relationship between the scores on the Bunney-Hamburg scale and the NOSIE scale.

The mean total PDS score on admission, on day 3, and at discharge from the hospital decreased monotonically, and the three scores were significantly different as measured by an analysis of variance (F=3.23, df=2,52, p=.047). The items on the scale were found to be highly intercorrelated (Cronbach's alpha=.98). An outside evaluation of the instrument for clarity and redundancy of items by an expert in instrument development resulted in separation of two of the items into four items and deletion of one item. Eight weak items were deleted. This process resulted in a scale containing 35 items to be field tested in a more homogeneous population.

Phase 2

The revised instrument was completed by 292 outpatients in treatment for psychosis in two large metropolitan areas. Both groups were participating in clinical studies of atypical antipsychotic medications and had been diagnosed as having schizophrenia or schizoaffective disorder.

The internal structure of the instrument was explored using the principal factor analysis program in SAS (version 6.11). Using the criterion of eigenvalues greater than 1, three factors that had eigenvalues of 13.812, 2.339, and 1.931 were retained. The three-factor result was supported by the scree test. Promax rotation was chosen as it was anticipated that the factors were correlated. All the items had loadings greater than .40.

The factor pattern and structure matrices were not similar, which indicates that the factors may not be related. The first factor in the rotated factor structure can be described as disordered thinking, the second factor suggests irritability, and the third factor is akathisia. Items loading on each factor were evaluated for bloated specifics (10). The decision was made to delete 12 items, resulting in a 23-item instrument. The revised PDS was found to be internally consistent (Cronbach's alpha=.93). Information on specific item loadings and interfactor correlations is available on request.

Discussion and conclusions

The major goal of this project was to develop a self-report instrument that can provide quantitative information on the subjective experience of discomfort from psychiatric symptoms. The initial evaluation of the Psychiatric Discomfort Scale on an inpatient psychiatric unit found evidence for construct validity, in that discomfort scores decreased significantly between admission and day 3 of hospitalization and between day 3 and discharge. Patients reported little burden in using the instrument, and it was sensitive to changes in clinical states over time.

A field trial in a homogeneous population resulted in a 23-item instrument with good internal consistency. Further research is needed to validate the instrument against external criteria and to reproduce these results with differing psychiatric populations. A copy of the instrument can be obtained from the author.

Acknowledgment

This study was supported by a grant from the Rabinowitz Research Fund.

Dr. Betemps is assistant professor in the College of Nursing of the University of Cincinnati, P.O. Box 210038, Cincinnati, Ohio 45221-0038 (e-mail, [email protected]).