Reliability and Validity of the Interview and Self-Report Versions of the BASIS-32

The BASIS-32 (1) is an increasingly popular outcome measure. Consistent with the current zeitgeist, it is a brief scale designed to assess both psychiatric symptoms and functional abilities. Three studies, all using inpatient samples, have reported on the psychometric properties of the BASIS-32 (1,2,3).

In the first study, exploratory factor analysis of the interview version was used to create five subscales: relation to self and others, daily living and role functioning, depression-anxiety, impulsive-addictive behaviors, and psychosis (1). Only one study directly compared an interview and a self-report version of the BASIS-32 (3); however, no data were reported on the psychometric properties of the two versions. The purpose of this study, therefore, was to assess the impact of changing the method of administration—interview versus self-report—on the reliability and validity of the BASIS-32.

Methods

A convenience sample of 120 adults was recruited from three psychosocial rehabilitation programs in July and August 1996. Information on demographic characteristics and psychiatric diagnosis was provided by the Missouri Department of Mental Health. For psychiatric diagnosis, the diagnostic formulation of the psychosocial rehabilitation program's psychiatrist was used; only information for the primary diagnosis was available.

A total of 78 participants (65 percent) were Caucasian, and 29 (24 percent) were African American. Sixty-five participants (54 percent) were men. The mean±SD age of the sample was 44±11 years. Eighty-two participants (68 percent) had psychotic disorders; 12 (10 percent) had bipolar disorder; 15 (13 percent) had major depression, recurrent; and five (4 percent) had other diagnoses. For six participants (5 percent), the diagnosis was unknown. Bivariate comparisons revealed no differences between participants assigned to the self-report and interview conditions in gender, race, age, diagnostic distribution, and the presence of substance abuse.

Psychiatric symptoms were assessed for the study with a 43-item derivative of the Hopkins Symptom Checklist (MSCL-43) (4) and the 18-item Brief Psychiatric Rating Scale (BPRS) (5,6). The MSCL-43 has three scales: subjective distress (alpha=.94), anxiety-depression (alpha=.94), and thought disorder-suspiciousness (alpha=.89). Three BPRS scales were used: anxiety-depression, hostility-suspicion, and thought disorder. On both scales, higher scores indicate greater symptomatology.

Case managers completed a two-item substance abuse severity scale—one item each for alcohol and for drug abuse severity. Items were rated using anchors borrowed from the Addiction Severity Index (7). Social functioning was assessed by case managers' ratings of the adequacy of clients' natural support (such as family and friends) and professional support, with higher scores indicating better support. Case managers also rated the amount of conflict in clients' networks, with higher scores indicating more conflict. Finally, case managers indicated the number of leisure activities in which clients were involved and whether clients were involved in vocational activities (0=no, 1=yes).

Participants were randomly assigned to either a self-report or an interview condition. Participants in the self-report condition completed the BASIS-32 and the MSCL-43 and then completed the BASIS-32 again one week later. Interview participants were administered the BASIS-32 and the BPRS and then completed the MSCL-43. Interviews were conducted by social workers at the rehabilitation programs. Two hours of training in the administration of the BASIS-32 and the BPRS were provided. Interviewers were trained to query and assign a rating for each item of the BASIS-32 and BPRS. A related study using identical procedures achieved good interrater reliability (8). Significance levels were set at .05 for all statistical tests, resulting in power just under .80 to detect a medium effect size for the analyses of variance and just under .80 to detect a large effect size for the comparisons of correlations (9).

Results

Reliability

Internal consistency coefficients for the self-report version of the BASIS-32 were significantly higher than coefficients for the interview version on five of six comparisons: full scale (.95 versus .90; z=1.90, p<.05), relation to self and others (.89 versus .72; z=2.71, p<.01), depression-anxiety (.85 versus .68; z=2.25, p<.05), daily living and role functioning (.85 versus .77; z= 1.65, p<.05), and psychosis (.73 versus .24; z=3.60, p<.01). Reliability for the impulsive-addictive behaviors subscale was inadequate for both the self-report condition (.65) and the interview condition (.60). Each self-report subscale had a one-week test-retest correlation above .72.

Concurrent validity

Validity correlations are presented in Table 1. Correlations between the BASIS-32 full-scale score and the subjective distress score of the MSCL-43 were significantly higher for the self-report version (z=3.77, p<.01). Further, validity correlations were significantly higher for the self-report version of the depression-anxiety subscale (z=1.90, p<.05) and the psychosis subscale (z=2.52, p<.01).

The BASIS-32 leisure item correlated significantly with the case manager's leisure rating in the self-report condition only (.33, p<.01). Clients who reported more difficulty with leisure activities were rated as involved in more leisure activities. The case manager's rating of alcohol abuse correlated significantly with the item of the BASIS-32 on alcohol abuse in both the interview condition (.37, p<.01) and the self-report condition (.39, p<.01). The case manager's rating of drug abuse correlated with the BASIS-32 drug abuse item in the interview condition only (.46, p<.01).

Discriminant validity

Multivariate analyses of variance (MANOVAs) revealed no differences in subscale scores by diagnostic group, although the results approached significance for interview participants (F=2.24, df=5,51, p<.07). Further, the MANOVAs revealed no differences in BASIS-32 scores by involvement in vocational activities.

A one-way (condition) MANOVA did not reach significance (p<.07), although a trend toward higher scores for self-report participants on all scales was noted. Comparisons by condition of scores on the subjective distress subscale of the MSCL-43 and the other MSCL-43 subscales and ratings of leisure activities, substance abuse severity, and social support also were not significant.

Discussion and conclusions

The main objective of this study was to examine the impact of changing the administration procedures—self-report versus interview—on the psychometric properties of the BASIS-32. Our results suggest that the interview and self-report versions may not have equivalent reliability and validity. In addition, we found the BASIS-32 functional domains to have poorer validity than did previous studies.

Consistent with previous studies, internal consistency coefficients were acceptable for most subscales; however, reliability was uniformly higher for the self-report version. The internal consistency of the impulsive-addictive behaviors subscale was less than desirable. In addition, consistent with other studies (1,2), the interview version of the psychosis subscale had very poor internal consistency. The fact that the psychosis subscale contains only four items undoubtedly affects its reliability (10). However, the low internal consistency found here also raises a concern that the interview version of the psychosis subscale is essentially a group of unrelated items.

In general, the symptom subscales for both the interview and the self-report versions of the BASIS-32 correlated well with other symptom scales, although correlations were higher for the self-report version. Unlike previous studies, we were unable to demonstrate concurrent or discriminant validity for the self-report version of the functional domains. For the interview version, however, clinicians' ratings of conflict in participants' social networks were associated with scores on the relation to self-others subscale of the BASIS-32 and leisure activities.

The results of validity analyses were only slightly better when specific items of the BASIS-32 were examined. Clients who reported greater difficulty with leisure activities were rated by case managers as involved in more leisure activities; no such pattern existed for the interview version. Ratings for severity of alcohol abuse correlated relatively well from all three perspectives—client, interviewer, and case manager. However, no such agreement was found for ratings of drug use; case manager ratings correlated with interviewer ratings but not with self-reports. In fact, all clients who reported difficulty with drug use were rated by case managers as having no drug problem, and all clients who were rated by case managers as having a drug problem reported no difficulty with drug use. Consistent with another study (3), we found a trend for participants who completed the self-report version to report somewhat greater distress on the BASIS-32 subscales than participants who were administered the interview version.

The most significant weakness of this study is that we did not employ a multitrait-multimethod matrix design. Such a design would have added confidence to our differential judgments about concurrent and discriminant validity. Another limitation of the study was that some of the measures of client functioning did not perform as anticipated. For example, we had originally planned to assess the type of housing in which clients resided, but our housing measure had very little variance and was dropped. In addition, concerns about burdening clinical staff with long questionnaires resulted in the selection of brief assessments of client functioning. Future studies should use more detailed assessments of client functioning to assess the validity of the BASIS-32.

Nevertheless, this study raised important questions about the impact that varying the method of administration had on the psychometric properties of the BASIS-32. In addition, our study found the BASIS-32 functional domains to be less valid than other studies found them to be. Future research should continue to examine these critical issues, particularly in light of the increasing popularity of the BASIS-32.

Acknowledgments

The authors thank the Missouri Department of Mental Health for funding this project. They also thank Robert Calsyn, Ph.D., and Debbie Zand, Ph.D., for reviewing earlier versions of this paper and Michael Trusty, Ph.D., for statistical consultation.

Dr. Klinkenberg is research assistant professor and Dr. Cho is associate professor at the Missouri Institute of Mental Health of the University of Missouri School of Medicine, 5400 Arsenal Street, St. Louis, Missouri 63139-1361. Mr. Vieweg is deputy director of the office of information systems for the Missouri Department of Mental Health.

Enlarge table