OBJECTIVE: The Health Plan Employer Data and Information Set (HEDIS) is the most widely used "report card" system comparing health care plans across different dimensions of performance. HEDIS uses only one measure of the quality of behavioral health care-the rate of follow-up after hospitalization for major affective disorder. This study used data from a national Veterans Affairs database to evaluate the generalizability of the HEDIS behavioral health quality measure. METHODS: Using administrative data from a nationwide sample of 114 VA hospitals, the HEDIS (version 2.5) quality measure was compared with several related performance measures including readmission rates and outpatient follow- up rates for other psychiatric disorders and for substance use disorders. The magnitude and statistical significance of Pearson's r value for correlation between measures was calculated. RESULTS: The HEDIS measure was moderately correlated with 30-day follow-up after hospitalization for other psychiatric disorders and with other performance measures of outpatient care. However, it was poorly correlated with follow-up for substance use disorders, inpatient measures including readmission rates, and several other measures of quality. CONCLUSIONS: Caution is needed in drawing conclusions about the quality of behavioral health plans based on the single measure used in HEDIS, version 2.5. Inclusion of other performance measures may be warranted.