A Brief Scale for Assessing Patients' Satisfaction With Care in Outpatient Psychiatric Services

The assessment of patients' satisfaction with medical services has been a rapidly developing area of research for almost 20 years (1,2,3,4,5). However, the assessment of patients' satisfaction with psychiatric clinics has only recently started to come into focus (6,7,8,9). Several specific measures have been developed for evaluating patients' satisfaction in a variety of psychiatric settings, including inpatient (10,11,12) and outpatient (13,14,15) clinics. Furthermore, satisfaction has been shown to be associated with improved clinical outcomes and administrative measures of quality at the individual level (12,16,17), although it is not clear whether patients' satisfaction is a cause or an effect of changes in symptoms.

In either case, psychiatric clinics that have the goal of maintaining or expanding a practice, particularly in a managed care environment, should measure patients' satisfaction to identify opportunities for improvement and to remain competitive. In addition, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) and other accrediting organizations require information on patients' satisfaction as a performance indicator.

Many patient satisfaction measures described in the literature do not use the item response formats that have been shown to produce more sensitive measures of satisfaction. As a result, many surveys yield highly skewed response distributions with low variability (18). In a study of methods for measuring patients' satisfaction with medical services, Ware and Hays (18) compared two response formats: a 5-point scale offering the response options of excellent, very good, good, fair, and poor (an E5 scale) and a 6-point scale offering the response options of extremely satisfied, very satisfied, somewhat satisfied, neither satisfied nor dissatisfied, somewhat dissatisfied, and very dissatisfied (an S6 scale). Their study was conducted in two types of settings: fee-for-service clinics in rural and urban areas of the Midwest and the West and primary and specialty care clinics in a health maintenance organization in the East.

The E5 format yielded less skewed results with greater variability in both settings. Furthermore, the E5 format better predicted whether patients intended to return to the same doctor for future visits, recommended the doctor to a friend, and complied with their medical regimen. On the basis of their findings, Ware and Hays concluded that the E5 format is preferable to the S6 format in measures of patients' satisfaction with medical services.

The study reported here was conducted to evaluate the reliability and preliminary validity of the Charleston Psychiatric Outpatient Satisfaction Scale, a 15-item measure of patients' satisfaction. This scale was designed to be brief enough to be practical and user-friendly but thorough enough to cover key clinical, administrative, and environmental factors that are important to psychiatric outpatients. It incorporates the E5 response format, which is intended to minimize positive response bias and optimize variability and predictive validity.

Methods

The respondents were 282 male and female adult outpatients treated at 19 clinics of the Institute of Psychiatry, a public-academic psychiatric institution affiliated with the Medical University of South Carolina in Charleston. The clinics include general psychiatric clinics that treat a broad range of disorders as well as specialty clinics, such as treatment programs for substance abuse and eating disorders. Patients in these clinics represent a broad range of socioeconomic statuses. To protect the respondents' anonymity and to encourage honest ratings, data on demographic characteristics such as age and gender were not collected and the surveys were not coded.

The data were collected over a one-week period in 1995. All patients who received services at any outpatient psychiatric clinic affiliated with the Institute of Psychiatry during that week were asked by the receptionist to complete the survey. To standardize survey administration, the receptionists were trained with the following script: "Ms./Mr. ____, the staff here are very interested in your opinions about the services you have received here, and we would like to ask you to complete this brief, anonymous survey. Your honest answers, both positive and negative, will help us improve our services. When you have finished, please drop it off in the hanging file located _____. Thanks so much for your feedback." The survey could be completed in two to five minutes. Anecdotal comments from receptionists suggested that the survey was well received by patients, and the receptionists reported no instances of patients' being unable to complete the survey because of disability or literacy problems.

The 15-item Charleston Psychiatric Outpatient Satisfaction Scale, shown in Table 1, uses a 5-point Likert-type scale with an E5 response format—5, excellent; 4, very good; 3, good; 2, fair; and 1, poor; plus a "does not apply" option—for all but the last item in the scale. The last item, which assesses behavioral intent to recommend the clinic, uses a 4-point response format: 4, yes, definitely; 3, yes, probably; 2, no, probably not; and 1, no, definitely not.

The 15 items were developed and selected in several ways. Some were selected from a large pool of items from an inpatient satisfaction scale previously pilot-tested on the inpatient units at the Institute of Psychiatry. Items that were the strongest predictors of overall ratings of quality of care and intent to recommend the facility were selected for the outpatient scale. Other items were developed to assess administrative and environmental factors unique to the outpatient setting. All of the items were developed on the basis of input from psychiatric clinicians and a comprehensive review of studies that used surveys and focus groups to identify service features that are important to psychiatric patients. These features include staff attitudes and therapeutic components as well as amenities such as parking and the physical environment of the clinic.

Two items measuring overall satisfaction with care (item 8) and likelihood of referring family and friends to the clinic (item 15) were included as anchor items. Anchor items are used to test preliminary validity and to identify items that are the best predictors of general satisfaction (18).

The psychometric properties of the Charleston Psychiatric Outpatient Satisfaction Scale were evaluated through a series of analyses that were planned a priori. Internal reliability, the consistency or preciseness of a measurement tool, was evaluated with Cronbach's alpha. Convergent validity, the extent to which the scale is correlated with similar constructs, was evaluated by using Pearson's product-moment correlation to test the relationship between overall scale scores and scores on the anchor items.

Results

The internal reliability of the Charleston Psychiatric Outpatient Satisfaction Scale, as measured by Cronbach's alpha, was high (alpha=.87). Preliminary convergent validity of the instrument was supported by the finding that scores for all items were significantly correlated with the scores for the two anchor items (r=.27 to .86 for anchor item 8, satisfaction with overall quality of care, and r=.19 to .63 for anchor item 15, likelihood of recommending the clinic to others) (Table 1). In addition, the total scale scores—that is, the sum of the scores for all items except the anchor items—were significantly correlated with the scores for both anchor items.

The best predictors of overall ratings of satisfaction with care were satisfaction with the helpfulness of the services (item 7, r=.86) and satisfaction with respect shown for patients' opinions about treatment (item 5, r=.84). The best predictors of recommendation of the program to others were satisfaction with the matching of the treatment plan to patients' individual needs (item 6, r=.63) and satisfaction with the respect shown for patients' opinions about treatment (item 5, r=.61). The mean scores for all items ranged from 3.6 (item 13, parking) to 4.5 (item 1, helpfulness of secretary, and item 8, overall quality of care), indicating that overall satisfaction with services in this sample was high—that is, very good on the 5-point response scale.

Discussion

Measurement of patients' satisfaction in psychiatric clinics is important because patients' satisfaction has been correlated with improved clinical outcomes and administrative measures of high-quality care—for example, fewer readmissions. In addition, measurement of patients' satisfaction allows organizations to identify areas of service delivery that can be improved. Ongoing improvement of service delivery and clinical outcomes is essential if a psychiatric clinic is to become and remain competitive in the current mental health care environment. Furthermore, JCAHO and other accrediting organizations require measurement of patients' satisfaction as a performance indicator. Thus implementation of a reliable and valid measure of patients' satisfaction can serve several important purposes relevant to the overarching mission of psychiatric clinics.

The patient satisfaction survey was designed specifically for outpatient psychiatric clinics. This 15-item measure represents an advance in the ability to measure patients' satisfaction in two ways. First, its items were developed on the basis of input from outpatient psychiatric clinicians and a comprehensive review of previous studies that have identified areas important to psychiatric patients. Second, it uses a response format—the E5 format—that minimizes positive response bias and optimizes variability and predictive validity. Because use of this response format is supported by the empirical literature, this scale is likely to be a more sensitive measure of psychiatric patients' satisfaction than scales that use a response format with options ranging from satisfied to dissatisfied.

Conclusions

The results of this study provide preliminary support for the reliability and validity of the Charleston Psychiatric Outpatient Satisfaction Scale, a 15-item measure of patients' satisfaction designed for use with adult psychiatric outpatients. However, this scale should be tested further to confirm its advantages and ensure its generalizability to psychiatric outpatients in other settings.

Dr. Pellegrin is affiliated with Discovery Alliance International, Inc., in Mobile, Alabama. Dr. Stuart, Dr. Frueh, and Dr. Ballenger are affiliated with the department of psychiatry and behavioral sciences at the Medical University of South Carolina in Charleston. Dr. Stuart is also with the College of Nursing at the university. Dr. Frueh is also with the Veterans Affairs Medical Center in Charleston. Ms. Maree is with Hamilton Medical Center in Dalton, Georgia. Send correspondence to Dr. Frueh at the VA Medical Center, 109 Bee Street, Charleston, South Carolina 29401 (e-mail, fruehbc@ musc.edu).

Enlarge table