Computerized Monitoring of Valproate and Physician Responsiveness to Laboratory Studies as a Quality Indicator

The Joint Commission on Accreditation of Healthcare Organizations has mandated development of computerized quality performance measures. Some state psychiatric hospital systems have reported successful use of computerized monitoring to improve performance using a number of quality indicators including medication management practices (1,2). One community-based computerized quality improvement system demonstrated lower hospitalization rates in programs adhering to American Psychiatric Association practice guidelines for pharmacotherapy of depression (3,4).

Computerization of hospital pharmacy systems is widespread (5). Adverse events in drug therapy may be more effectively monitored with computerized surveillance than with traditional paper-based methods (6). Computerized physician performance feedback (7) and laboratory alert systems (8) have been described in psychiatric settings, and computer-based guideline implementation systems have been used in various medical settings (9,10).

This report illustrates the use of computerized pharmacy, laboratory, and hospital utilization data in therapeutic drug monitoring to study physicians' responses to laboratory data and their relationship to clinical course for patients treated in a large state psychiatric hospital system. Focusing on laboratory monitoring of pharmacotherapy with valproate, this report also assesses physician practices following didactic feedback on the monitoring data.

Methods

The Illinois statewide clinical information system serves all state-operated psychiatric hospitals and developmental facilities. The ten psychiatric hospitals serve 10,000 patients annually and employ 175 staff physicians. In fiscal year 1997, prescriptions totaled 1.2 million, and laboratory studies exceeded one million. Computerized prescription, laboratory, and hospitalization records were imported monthly into relational databases (Paradox 7 on a Windows NT system).

During fiscal year 1997, therapeutic medication monitoring guidelines were developed. Literature-based recommendations were subject to ongoing review by all state hospital medical directors, with input from their medical staffs. Recommendations about specifically indicated physiological and pharmacological laboratory studies and time frames for obtaining them were thereby fine-tuned by practicing staff physicians and tailored to individual hospitals.

Two sequentially developed computerized monitoring modules were used to recognize out-of-range laboratory values and to search for physician responses—for example, medication adjustment or repeat laboratory study—consistent with clinical expectations. The second application employed "danger multipliers" to vary expected response times according to presumed level of clinical urgency. A subtherapeutic valproate level, for example, required a response delay of seven days to register as inappropriate, but a toxic drug level or low prothrombin time activated a mathematical function searching for progressively shorter response times corresponding to the value's degree of deviation from the therapeutic or safe range. Comprehensive physician response data, including percentage of computer-ascertained appropriate responses, were summarized in tabular and graphical form monthly for all medical directors. Physicians could also provide information overriding computer-based determinations.

During the nine-month period from February 1, 1997, to October 31, 1997, physician responses flagged by computerized monitoring as inappropriate were selected for one hospital, and all corresponding patient charts were reviewed by one of the authors (MQ). Chart review included patient demographic characteristics, diagnosis, medical and psychiatric history, previous medication responses, and prior out-of-range laboratory values; baseline and current laboratory data, timing of out-of-range values in relation to prescription changes, and nature and timing of the physician's response; and a review of current medication, including compliance and possible drug interactions. A judgment was then made about whether the computer-identified inappropriate response was clinically significant—that is, whether it affected the patient's clinical course, including symptom severity and duration.

Computerized monitoring data were drawn from four metropolitan hospitals for the same period, focusing specifically on valproate monitoring. Both the appropriateness of physician responses to out-of-range laboratory values and the relationship between physician response and patient length of stay were studied.

First, baseline data were collected over a six-month period, during which the first author delivered didactic feedback on study data to the medical staffs at each of the four hospitals in a lecture program targeting 120 physicians. Second, computerized monitoring data were collected from each of the hospitals after the lecture presentations. The postlecture monitoring period ranged from eight to 12 months in the four hospitals, averaging ten months.

Chart review findings were described by frequency distributions. Physician response, patient length of stay, and diagnostic and demographic variables were examined by univariate analysis (Pearson correlation, t test, and one-way analysis of variance). Variables that were significant in the univariate analysis were then included in a multiple regression analysis with physician response as the independent variable of interest and length of stay following the out-of-range laboratory test as the dependent variable.

Results

The single-hospital study yielded 1,083 out-of-range laboratory values. Physician responses were identified as inappropriate in 73 instances. In the chart review 20 of the 73 computer-flagged responses were found to be clinically significant, and 19 of the 20 involved serum medication levels. All 19 serum medication levels were low rather than high, and the majority, 63 percent, involved valproate. Of the 73 responses flagged as inappropriate, 28 involved valproate, and 12 of these, or 43 percent, were clinically significant.

For all four hospitals, the computer program identified 475 low valproate levels in 343 patients. Demographic and diagnostic data were available for 275 of these patients. Two-thirds were male. The majority had a diagnosis of schizophrenia or schizoaffective disorder (51 percent); a large proportion had mood disorders (33 percent), and the remainder had other diagnoses. The patients' mean±SD age was 43±15 years.

Of the 475 low valproate levels, physician response was identified as appropriate in 226 of the cases, or 48 percent. Patient demographic variables did not influence physician responses significantly. Each patient's length of stay from the detection of a low valproate level until discharge was calculated, and these values were compared for differences between the group whose physicians responded appropriately after laboratory values were determined and the group whose physicians did not. Among patients whose physicians responded appropriately, the mean±SD length of stay was 57±65 days, whereas for patients whose physicians did not respond appropriately it was 96±77 days (t=5.08, df=328, p<.001).

Patients with schizophrenia or schizoaffective disorder were hospitalized for longer periods (81±67 days) than patients with mood disorders (56±58 days) or other psychiatric disorders (60±66 days; F=4.83, df=2, 273, p<.001). Gender differences were marginally significant, with women having shorter stays than men (60±60 days versus 75±67 days; t=1.82, df=210, p<.07). No relationship was found between length of stay and either age or hospital.

In a multiple regression analysis controlling for diagnosis and gender, only physician response predicted length of stay to a significant extent (F=7.11, beta=.231, df=4,217, p< .001). For physicians who received at least three out-of-range laboratory reports (N=45), the higher the physician's overall percentage of appropriate responses to low valproate levels, the shorter the average length of hospital stay for that physician's patients (r=-.46, df=44, p<.001).

After the lecture program presenting study findings to the medical staffs at each of the four hospitals, the proportion of appropriate responses to out-of-range serum valproate levels increased from 57.8 percent to 66.9 percent (from 537 of 929 to 354 of 529; χ²=9.29, df=3, p<.05).

Discussion and conclusions

This study illustrated the usefulness of computerized monitoring for assessing physician adherence to therapeutic drug monitoring guidelines. However, only a minority (27 percent) of physician responses flagged by computerized monitoring as inappropriate were found to be clinically significant on detailed chart review. Clinical review helped focus the monitoring process on meaningful variables. For this study, that meant examining valproate levels in the larger sample. In general, clinical review provides a basis for selecting certain laboratory studies for continued monitoring while eliminating others. It also serves as a model for use of computer-generated data for performance review.

Clinically significant deviation from guidelines predominantly involved physician responses to subtherapeutic drug levels rather than toxic levels or significant physiological aberrations. For low valproate levels, physician responses flagged by computerized monitoring as inappropriate were associated with longer hospitalizations for patients. Although hospital course is complex, delayed progress may be related partly to physicians' lack of responsiveness to laboratory studies. Didactic feedback to physicians was associated with improved performance inasmuch as physicians' behavior showed evidence of change in the direction of guideline adherence after exposure to the data.

Acknowledgment

The authors thank Steven Banks, Ph.D., for his review of an earlier version of the paper and for his statistical consultation.

Dr. Luchins, Dr. Klass, Dr. Hanrahan, Dr. Qayyum, Mr. Malan, and Dr. Fichtner are affiliated with the Office of Mental Health of the Illinois Department of Human Services. Dr. Luchins is also associate professor, Dr. Hanrahan is research associate professor, and Dr. Raskin-Davis is clinical associate professor in the department of psychiatry at the University of Chicago. Dr. Fichtner is associate professor and Dr. Qayyum is clinical assistant professor in the department of psychiatry and behavioral sciences at Finch University of Health Sciences- Chicago Medical School. Address correspondence to Dr. Luchins, University of Chicago, Department of Psychiatry, MC3077, 5841 South Maryland Avenue, Chicago, Illinois 60637 (e-mail, [email protected]).