Assessment of clinical outcomes and satisfaction with care is rapidly becoming a routine part of clinical practice (1). Assessment serves quality improvement purposes and fulfills requirements of licensing, accrediting, and managed behavioral health care organizations (2). To engage staff and to minimize administrative burden, it is desirable to use outcome assessment tools that are clinically meaningful and useful (3).

Current literature supports the involvement of patients at all levels of the mental health service delivery system (4,5). The Rehabilitation Act Amendments of 1992 require patients' involvement in treatment, and a recent national survey found that 34 states in the U.S. (68 percent) have paid positions for consumers in either central or field offices of their mental health agencies (6,7). Positive effects of patients' involvement have been described in rehabilitation, psychotherapy outcome, and supportive housing for persons with psychiatric disabilities (8,9,10).

Patients' involvement may enhance the therapeutic alliance between the physician and patient, an important factor in clinical outcome (11,12,13). The therapeutic alliance is fundamental to the theory and practice of psychotherapeutic intervention. In addition to its effect on outcome, the therapeutic alliance is associated with adherence to treatment recommendations, reduction in violent behavior, satisfaction with care, and retention in treatment (14,15,16,17,18,19,20,21). The benefits of a good therapeutic alliance have been demonstrated in inpatient and outpatient settings and among diverse groups of patients across the diagnostic spectrum (22,23,24).

This study was undertaken to explore the utility of using a standardized self-report rating scale (25) to more fully involve patients in their care. Recognizing and acknowledging patients' perspectives on their symptoms and problems may provide a systematic way to include patients in the evaluation and treatment planning process, thereby enhancing the therapeutic alliance. A more effective alliance may, in turn, result in improved experiences of the care received as well as enhanced clinical outcomes.

An assessment conducted in 1995 of patients' perceptions on discharge from inpatient care at McLean Hospital indicated that 25 percent of respondents thought that their involvement in their treatment was fair or poor. (Seventy-five percent reported it was good or excellent.) This study evaluated efforts to improve this particular aspect of care by implementing an intervention that engaged patients directly in the treatment planning process. The hypothesis of the study was that patients who received the intervention would report more involvement in decisions about their treatment than those who did not receive the intervention.

Study design

The study used a quasiexperimental design with three groups of inpatients. One group received the intervention, and the two comparison groups did not. The intervention was implemented by third-year psychiatric residents who rotated through the program between April and September 1997. Patients in one comparison group were treated by either of two attending psychiatrists; patients in the other comparison group were treated by third-year psychiatric residents who rotated through the program between October 1997 and April 1998. The same two attending psychiatrists were assigned patients throughout the study period. Attending psychiatrists were board certified, and each had more than ten years of postresidency experience.

The patients were sequentially assigned to one of the three groups based on admission date. The research team was not involved in the clinical care of the patients in the sample, and neither the psychiatric residents nor the attending psychiatrists were otherwise involved in the research project.

Sample

The sample consisted of 109 consecutively admitted inpatients. The majority were male (69 patients, or 63 percent). The mean±SD age was 37±10. Most patients were diagnosed as having schizophrenia or schizoaffective disorder (61 patients, or 56 percent) or bipolar manic or bipolar mixed disorder (34 patients, or 31 percent). The majority (75 patients, or 69 percent) were insured by Medicare (for disability) or Medicaid.

Length of hospitalization ranged from three to 118 days, with a mean±SD stay of 19±17 days and a median stay of 16 days. Forty-three patients (39 percent) had been hospitalized within one year of the index hospitalization. At admission the mean±SD Global Assessment of Functioning (GAF) score or axis V rating (26) was 32±8.76, indicating some impairment in reality testing or communication or major impairment in several areas.

No significant differences were found among the three groups on any of these variables or on self-reported difficulties with symptoms or problems at admission. The patients treated by residents were significantly more likely than those treated by the attending physicians to have a secondary axis I diagnosis of a comorbid substance use disorder (χ²=7.95, df=2, p<.02).

Data sources

Behavior and Symptom Identification Scale (BASIS-32).

BASIS-32 is a 32-item self-report symptom- and problem-rating scale designed to assess treatment outcome across five domains: relation to self and others, depression and anxiety, daily living skills, impulsive and addictive behavior, and psychosis. Patients rate the degree of difficulty they have been having in each area during the past week on a 5-point scale from no difficulty to extreme difficulty. The instrument's reliability, validity, and sensitivity to change have been demonstrated in similar populations, and this information has been reported elsewhere (25).

Perceptions of Care (PoC) survey.

The PoC survey was developed by the authors after a review of existing consumer satisfaction surveys for medical and psychiatric care (27,28,29,30,31,32,33. Items that are unique to inpatient psychiatric care were added. The survey includes 16 structured questions listed in t1. The survey, which is available from the authors, is part of a performance measurement system for behavioral health care that is approved by the Joint Commission on Accreditation of Healthcare Organizations (34).

Medical records.

Data on age, sex, payer, previous hospitalizations, DSM-IV diagnoses, GAF rating at admission, and length of stay were obtained from medical records.

Procedures

All patients completed the BASIS-32 on admission as part of the clinical evaluation. Written consent was not obtained because both the BASIS-32 and the PoC survey had been approved by the institutional review board as routine components of the hospital's continuous quality improvement program. Patients in all three groups also completed the BASIS-32 and PoC survey within 24 hours before discharge. Patients were told that these instruments were used in the hospital's quality improvement program to evaluate the care they received.

Intervention group.

Within one business day of admission, a psychiatric resident met individually with the 23 patients assigned to the intervention group. At the meeting, the resident reviewed the patient's responses to items on the BASIS-32 responses and subscale scores, conveyed to the patient an appreciation of his or her self-reported difficulties, and explored further the symptoms and problems of concern to the patient. Each area was discussed with the patient, and residents reported at rounds the BASIS-32 domains or items that the patient had identified as problem areas on which to focus treatment. Thus, for patients in the intervention group, a systematic mechanism was established for the clinical team to target patients' self-reported problems as a focus of treatment. The intervention was modeled for the residents by one of the authors (LIS), and written training materials were provided.

Comparison groups.

Patients in the two comparison groups—47 patients treated by attending staff psychiatrists and 39 treated by psychiatric residents—received treatment as usual. At the patient's initial meeting with the attending physician or resident, diagnostic assessment and formulation were undertaken. However, the physician or resident did not review patients' BASIS-32 responses or make any systematic effort to use the information for treatment planning.

Data analysis

Discharge scores on the PoC survey and the BASIS-32 were compared for the three groups to assess differences as a result of the intervention. The hypothesis that patients who received the intervention would report more involvement in decisions about their treatment than those who did not was tested by comparing mean ratings on the PoC item about the degree of involvement in treatment decisions. One-way analysis of variance was used to assess the overall difference among means for the three groups, followed by multiple range tests to assess specific group differences. This procedure was also followed for the remaining PoC items that were rated on a continuum.

To assess whether group differences could be explained by differences in comorbid substance abuse, we performed an analysis of covariance using the substance abuse diagnosis as a covariate. Chi square analyses were performed to assess group differences on the categorical PoC items. The effect of the intervention on clinical outcome was assessed using multivariate repeated-measures analysis of variance with the three groups and the time point (admission and discharge) as independent variables and the five BASIS-32 subscale and overall mean scores as dependent variables. All data analyses used SPSS version 9.0.

Results presented in t1 show that the overall F ratio was statistically significant, indicating a difference among the three groups in perceived involvement in treatment decisions. The multiple range test indicated that patients in the intervention group perceived their care more favorably than patients in the comparison group treated by residents. The group difference remained statistically significant when the analysis controlled for comorbid substance abuse.

In addition, statistically significant differences among the three groups were found on two other PoC items: "How often did the staff treat you with respect and dignity?" and "Would you recommend this facility to others?" On each of these items, patients in the intervention group perceived their care more favorably than those in the comparison group treated by attending psychiatrists. As for recommending the facility to others, patients who were not in the intervention group but who were treated by residents were more likely to say they would recommend the facility to others than were patients treated by attending psychiatrists. These results should be interpreted cautiously due to the number of statistical tests performed.

Treatment outcome

Results indicated no statistically significant effect of the intervention on BASIS-32 scores at discharge. Thus the intervention did not have an effect on short-term treatment outcome as measured by this instrument. However, a statistically significant effect was noted for time point, indicating that improvement from admission to discharge was statistically significant for each of the BASIS-32 subscales and for the overall mean score.

The results of this study indicate that patients' perceived involvement in treatment decisions was significantly greater for those who met individually with a psychiatric resident to review their self-reported symptoms and problems. Thus a brief intervention designed to engage patients more fully in the treatment planning process followed by presentation of the patient's perspective to the treatment team was successful in improving perceptions of how involved patients felt in treatment decisions.

These results are consistent with previous work by a primary care physician who reported how a general health status survey (SF-36) was useful in improving care provided by a single practitioner (35). To our knowledge, our study is the first report of the use of a standardized, self-report, behavioral health outcomes instrument to increase consumers' involvement in their care.

The effect of the intervention on patients' perceptions of inclusion in the treatment process may have been mediated by the therapeutic alliance. A staff-patient interaction in which a staff member listens to the patient and engages the patient in a collaborative process is consistent with efforts to enhance the therapeutic alliance. Because we did not directly assess the strength of the therapeutic alliance, we cannot be sure that the impact of the intervention was mediated by alliance, although this would be a worthwhile avenue for future studies.

Patients who were treated by residents, whether or not they received the intervention, also reported that they were more likely to recommend the facility to others than patients in a comparison group treated by attending psychiatrists. These results suggest possible differential effects of treatment by residents and attending physicians, which would be fruitful to explore in further research.

The lack of an effect on treatment outcome may be viewed as inconsistent with earlier research showing a positive impact of consumer involvement and therapeutic alliance on outcome. However, most of the earlier research dealt with longer-term outcomes in outpatient settings. Postdischarge follow-up assessment would be needed to determine whether the intervention implemented during hospitalization affects long-term outcome.

A limitation of the study is the relatively small sample size. A larger sample size with a resulting increase in statistical power may yield additional significant effects. It is also possible that unmeasured differences in patient characteristics could have accounted for the results reported here. Because of these limitations, the study findings must be cautiously interpreted.

This study reported on a process in which an outcome assessment tool was used to increase patients' involvement in treatment. When a tool developed for outcome assessment is applied to other purposes, its usefulness is increased beyond meeting licensing, accreditation, and payer requirements. Future research efforts will be directed at exploring additional ways of involving patients in their care and assessing the effects of such involvement.

Dr. Eisen is assistant director and Dr. Dickey is director of the department of mental health services research at McLean Hospital, 115 Mill Street, Belmont, Massachusetts 02478 (e-mail, seisen@mclean.harvard.edu). Dr. Sederer is medical director of McLean Hospital. The authors are also associate professors in the department of psychiatry at Harvard Medical School in Boston.