Best Practices: Using Case Vignettes to Train Clinicians and Utilization Reviewers to Make Level-of-Care Decisions

Medical necessity has emerged as the de facto standard for decisions about payment for behavioral health services, despite criticisms leveled from theoretical (1,2) and practical (3) perspectives. Moreover, it has been difficult to define best practices within the current framework of medical necessity, largely because of the many sources of variation in decisions about appropriate levels of care.

In practice, when clinicians and clinical case managers—that is, utilization reviewers—communicate information about a patient, usually by telephone, they use a narrative case presentation. Discussion is focused on assessment of necessity criteria, such as symptom severity, dangerousness, social support, and resource availability, that would support a higher level of care, such as inpatient hospitalization. Unfortunately, the dialogue may break down without resolution of differences, and with considerable residual ill will between parties (4).

In this column we report on our use of case vignettes as a training device to help clinicians and clinical case managers make consistent decisions about appropriate levels of care and to develop best practices.

Development and use of case vignettes

Partly because of their research and teaching missions, academic centers have been slow to react to changes in health care financing and have a reputation of inefficiency and overuse of intensive levels of care (5). In 1995 Wake Forest University established a health maintenance organization with about 50,000 enrollees. In the first year of operation, the university contracted with an outside, for-profit managed behavioral health organization to provide behavioral health utilization review and case management. Eventually the department was able to establish its own internal managed behavioral health care organization (6).

Initially, however, the department struggled to meet the demands of managed care, working with the outside organization. To promote greater uniformity in decision making, we designed four case vignettes and used them in joint training with our clinicians and the clinical case managers from the outside organization.

Case vignettes have been used previously to compare decision-making strategies of different groups (7). Because use of vignettes limits variation in how people perceive a case by providing all persons with the same information, vignettes offer training advantages over real-life patients. Case vignettes are ideal when the primary objective is to identify conflicts in judgment (8).

Each vignette developed by the department is a typical narrative case presentation and includes details about the patient's history and mental status. Each patient has a different diagnosis—delirium, comorbid depression and substance abuse, chronic depression, and schizophrenia. In two vignettes, the patient presents as an outpatient. In another, the setting is an emergency room. In the fourth case, the patient has been referred for a consultation to an inpatient general medical setting. The vignettes do not convey any expectations about case disposition or information about insurance status.

Thirty-one persons participated in the training—seven attending physicians, 16 house officers (psychiatric residents), and eight clinical case managers. After reading each vignette, respondents were prompted to choose the most appropriate treatment setting—inpatient care, partial hospitalization, or outpatient care. They provided up to five of their own reasons for each decision. In a translational process paralleling the interaction between clinician and reviewer, the reasons were examined to determine whether they referred to either of two common medical-necessity criteria: the patient's level of dangerousness and the patient's support system. These reasons were tallied separately and compared by group using Fisher's exact tests.

Results are summarized in Table 1. All the respondents recommended that the patient with delirium receive inpatient care. Similarly, for the patient with comorbid depression and substance abuse, 94 percent recommended inpatient care, and only 6 percent recommended partial hospitalization.

For the patient with schizophrenia who was experiencing negative symptoms, 52 percent of respondents recommended outpatient services, 36 percent recommended partial hospitalization, and 13 percent recommended inpatient care. For the patient with chronic depression, 39 percent recommended outpatient services, 26 percent recommended partial hospitalization, and 36 percent recommended inpatient care.

No significant differences were found between attending physicians and clinical case managers on any of the four level-of-care decisions. Only house officers selected inpatient treatment for the patient with schizophrenia. Compared with attending physicians and case managers, house officers selected higher levels of care for this patient; however, the difference was not significant.

Compared with attending physicians and case managers, house officers were significantly less likely to take into account the patient's support system as a factor in decision making in two cases—the patient with schizophrenia (Fisher's exact test, p=.09) and the patient with chronic depression (p=.09). Attending physicians were less likely than house officers and case managers to take into account the patient's level of dangerousness in their decision about the patient with chronic depression (Fisher's exact test, p=.06).

Discussion

Our study failed to demonstrate significant differences between groups of clinicians and utilization reviewers in level-of-care decisions for any of four common psychiatric presentations. Similarly, we identified very few differences between groups in their use of particular criteria as a rationale for their decisions.

Both the clinicians and the utilization reviewers in our setting have expressed surprise at these findings, because they run counter to the expectation that level-of-care decisions and decision rules used by each group would be quite different. In a study of implementation of a managed care plan during the course of which use of inpatient services markedly declined, it was shown that over time clinical case managers rated fewer patients as severely disturbed while ratings by clinicians remained unchanged (8).

Use of the vignettes has effectively demystified the process of utilization review for clinicians in our department. Two vignettes engendered strong agreement by all respondents. The others revealed more variation in decisions about the most appropriate level of care, both across all respondents and within respondent groups. This finding raises the question of how we can increase the level of agreement for more equivocal cases. Level-of-care decisions must be reliable—that is, care managers must make similar decisions in similar cases across time. Without some degree of reliability, a meaningful best practice is unlikely to emerge.

First, we must develop meaningful and reliable criteria. Some progress has been made. In one study, when clinicians were presented with a broad and unstructured list of variables, they were unable to achieve an acceptable level of agreement about indicators for hospitalization (9). On the other hand, expert panels using modified Delphi techniques have achieved high levels of agreement in decisions about levels of care for both hypothetical and actual cases, and in the process they have identified and developed anchored ratings for a number of key variables (10,11).

Second, level-of-care decision criteria must not remain the sole province of health services researchers and clinical case managers. Instead, they should be widely disseminated to the network of providers. At our facility, the outside managed behavioral health care organization shared its criteria with clinicians from the outset. When the department took over care management, this practice was continued. Also, medical-necessity criteria have been incorporated into the admission forms, admitting orders, and computerized treatment planning documentation of the inpatient and partial hospital unit (12).

Experience and training would seem to be likely sources of variation in decision making in clinical and managed care settings. Although our study was limited by its small sample size, the results suggest that house officers may differ from more experienced psychiatrists and clinical case managers in their decisions and approach. By ensuring that the house officers encounter the decision criteria in the course of their daily work and by providing them with training material on managed care principles, we hope to create a working model of best practices against which they can compare their decisions. Senior residents may also participate in an elective rotation in managed care during which they can review cases and make interpretations of medical necessity.

Criterion-based admission policies and procedures clearly narrow the range of variables used in level-of-care decisions. However, we will continue to encounter equivocal cases. One approach to improving the reliability of decisions would be to conduct field tests to systematically identify sources of variation in decision making. Once we know the sources, we may more clearly define what constitutes best practice. In a study using videotaped interviews conducted in an emergency room, agreement between raters was low for recommended disposition, psychopathology, impulse control problems, ability to care for self, and danger to self (13). A somewhat higher level of agreement was reached for psychosis and substance abuse.

Clinical case managers and medical directors continue to oversee care management in our system through traditional review processes. Every three months appeals are presented for discussion and comment before a quality improvement committee composed of a rotating group of network clinicians. Level-of-care criteria are reviewed and amended annually. This body recently voted to begin using the Criteria for Short-Term Treatment of Acute Psychiatric Illness, jointly published by the American Academy of Child and Adolescent Psychiatry and the American Psychiatric Association (15).

As practice guidelines become more detailed, and more reflective of best practices, we anticipate an eventual eclipse of more generic level-of-care criteria. For example, the use of the Clinical Institute Withdrawal Assessment protocol (16) in our facility has supplanted the need for concurrent review of necessity and intensity of service for alcohol detoxification because such a review is part of the protocol. The measure of our success will be how well we work collectively to meet the needs of patients as we develop our mental maps, whether they are vignettes, criteria, practice guidelines, or protocols.

Dr. Rosenquist is assistant professor and Dr. Kramer is associate professor in the department of psychiatry and behavioral medicine at Wake Forest University School of Medicine in Winston-Salem, North Carolina. Ms. Briggs is chief executive officer of Carolina Behavioral Health Alliance in Winston-Salem. Dr. Colenda is professor in the department of psychiatry at Michigan State University in East Lansing. Dr. Lancaster is regional medical director of Value Options in Raleigh, North Carolina. Send correspondence to Dr. Rosenquist at the Department of Psychiatry and Behavioral Medicine, Medical Center Boulevard, Winston-Salem, North Carolina 27157-1087 (e-mail, [email protected]). William M. Glazer, M.D., is editor of this column.

Enlarge table