Use of Chart and Record Reviews to Detect Medication Errors in a State Psychiatric Hospital

Despite the medical profession's best efforts to provide safe and effective care, research has increasingly identified an alarming incidence of unintended harm to hospitalized patients. In the landmark 1991 Harvard Medical Practice Study, it was estimated that as many as 1.3 million injuries occur nationwide to patients receiving hospital care, with adverse drug events accounting for 19 percent of all injuries (1,2). Subsequent research by Bates and colleagues (3,4,5) also found that medication errors are a significant cause of potential harm in adult general care settings. Their findings indicated that 28 percent to 56 percent of all adverse drug events that occur in such settings might have been prevented.

The Institute of Medicine's 1999 report (6) suggested that more deaths occur annually from medication errors than from industrial accidents. Medication errors are also costly from a financial perspective. The national estimate of costs to hospitals is $2 billion per year, excluding malpractice costs and costs of injuries (7).

However, most available data come from general care units. Few published studies have examined medication errors in psychiatric hospitals (8,9,10), and we could not find any that provide estimates of medication error rates or a systematic examination of error detection methodologies. We therefore undertook a study at a psychiatric institution and focused on improving the detection and reporting of medication errors, with an ultimate goal of decreasing their frequency.

We have previously reported on our use of personal digital assistants (PDAs) in an attempt to decrease medication errors by increasing physician access to up-to-date patient medication profiles and to a pharmacology database (11). Errors in discharge medication lists have been reduced at our facility by the use of PDAs to directly print discharge medication lists using a printer with an infrared port, thereby nearly eliminating transcription errors at the time of discharge (12). However, at the time of the study reported here, all medication orders were written by hand in the usual fashion.

Previous studies have found that the usual method of self-reporting errors may substantially underestimate actual error rates. In a study of medication errors at Brigham and Women's Hospital, Bates and associates (13) compared the number of errors detected by independent review with the number of self-reported errors. The independent review process detected far more errors than self-reporting, by a ratio of 1,000 to 1. In a study by Classen and colleagues (14), computerized monitoring detected more adverse drug events than self-reporting by a ratio of 81 to 1. We therefore hypothesized that the current method of self-reporting medication errors at the Augusta Mental Health Institute, a state psychiatric hospital in Maine, was underreporting the medication error rate. To test this hypothesis, we assigned a review team to retrospectively examine patient records, and we compared the team's results with the results of self-reporting.

Methods

Definition

In this study we used the definition of a medication error established by the National Coordinating Council for Medication Error Reporting and Prevention (15): "A medication error is any preventable event that may cause or lead to inappropriate medication use or patient harm while the medication is in the control of the health care professional, patient, or consumer. Such events may be related to professional practice, health care products, procedures, and systems, including prescribing; order communication; product labeling, packaging and nomenclature; compounding; dispensing; distribution; administration; education; monitoring; and use."

Study site

The Augusta Mental Health Institute is a fully accredited 103-bed state psychiatric hospital in central Maine. Patients are referred from emergency departments and transferred from other psychiatric hospitals, and the census is usually near or at the maximum. Civil and forensic patients are admitted.

Criteria

The medication error criteria that we used during chart review, which are listed in Table 1, are an amalgam of recommendations drawn from the Institute for Safe Medical Practices, the U.S. Pharmacopoeia medication errors reporting program, and contracted consultants. We also added new criteria when errors were detected that did not fit existing criteria. The review process included examination of each medication order on the order sheet, its transcription onto the medication administration record, documentation of its administration on the medication administration record, and documentation of its dispensing in the pharmacy and the medication room on the unit.

Medication errors were categorized by type—prescription, transcription, administration, and dispensing errors—and each error was rated on its risk of harm, defined as the likelihood of causing an adverse drug reaction, of diminishing the effectiveness of the intended pharmacotherapy, or of being potentially life threatening. The risk of harm was rated from 1 to 3—low, moderate, or high. Examples of low-risk errors include not administering a dose of citalopram every night at bedtime and not specifying on an as-needed order of ibuprofen the criterion for administration—for example, severe pain. An example of a moderate-risk error is resuming or initiating treatment with divalproex before ordering blood tests to determine, among other things, the divalproex level for a patient who is currently taking the drug, the status of liver enzymes, and whether the patient is pregnant. Another moderate-risk example is a "now" order for diphenhydramine that is given but not documented on the medication administration record. (A "now" order is for a medication given to a patient on only one occasion, and the patient must receive the medication within 60 minutes. This type of order is different from a STAT order, which is also for a medication given on one occasion but which must be administered within 15 minutes.) Examples of high-risk errors include ordering a "now" dose for insulin that is neither transcribed nor given and not administering four doses of risperidone during a 14-day period without an explanation.

Data collection

A computer program was used to randomly select 31 of the 95 patients discharged between June 1, 2001, and October 15, 2001. The director of pharmacy and a nurse practitioner (a full-time medical staff member) functioned as the review team and conducted a chart review of each patient's entire hospitalization. The total number of patient-days reviewed was 1,448. Patient-days were calculated as the patient census per day divided by the number of days.

The team reviewed each component of prescription, transcription, and administration for all medications prescribed. In rating all of the 2,180 errors detected, they disagreed in their ratings of eight putative errors. The medical director reviewed the eight potential errors independently by using the same criteria, and his rating was accepted as final. The assistant director of nursing and the director of pharmacy reviewed and agreed on all dispensing errors.

Because dispensing of medication can be reviewed only concurrently, not retrospectively, the review team could not determine dispensing errors. Instead, pharmacy staff concurrently self-reported all dispensing errors that they detected in the pharmacy, and nursing staff reported all dispensing errors they detected in the unit medication rooms for all inpatients hospitalized between November 1, 2001, and December 31, 2001.

Fifty dispensing errors were detected in 78,571 dispensing events for the 5,105 patient-days in that time period. However, to be consistent with the number of patient-days for which the prescription, transcription, and administration error rate was calculated, the rate of dispensing errors was calculated on the basis of 1,448 patient-days, yielding a rate of 14 errors per 22,000 dispensing events.

The review team that conducted the chart reviews also rated all errors for risk of harm. On two occasions during the first month of the study, the medical director made a random selection of 15 charts in which errors had been detected by the review team. He agreed that all the prescription, transcription, and administration errors found by the team were indeed errors, but he disagreed with two ratings of risk of harm. The review team ultimately agreed with the medical director's assessment that the risk of harm was moderate rather than low. The review team also rated all dispensing errors for risk of harm and agreed in all their ratings.

During the study, the hospital's usual practice of self-reporting errors was continued. Through this process, all clinical staff members are encouraged to report medication errors by filling out an incident report. These data are collected and reviewed by a psychiatric registered nurse on the clinical risk management team and reported monthly to nursing and medical staff and the administrative executive committee.

Analysis

The total number of errors detected was calculated, as were the error rates within rating categories and the rate for each category as a percentage of the total errors. The rates of errors detected by the use of self-reports and by the review team were compared by using the chi square statistic.

Results

The sample of 31 patients whose charts were reviewed for prescription, transcription, and administration errors and the 70-patient sample used to determine dispensing errors were similar. The patients' respective mean ages were 38±15.4 years and 39±12.6 years. Eleven of the 31 patients (36 percent) and 32 of the 70 patients (46 percent) were women. Both groups had long hospital stays—means of 38±47.5 days and 39±45.9 days, respectively.

For both samples, a total of 2,194 medication errors were detected, whereas nine errors were self-reported for the same patient groups, yielding a ratio of 244 to 1 (χ²=2,167, df=1, p<.001, using the McNemar chi square statistic).

A total of 22,000 dispensing events over 1,448 patient-days were reviewed, and 14 errors were detected.

The distribution of errors was as follows: prescribing errors, 11 percent; transcription errors, 23 percent; administration errors, 66 percent; and dispensing errors, 1 percent. Nineteen percent of the total errors were rated as having a low risk of harm, 23 percent were rated as having a moderate risk, and 58 percent were rated as having a high risk. Fifteen percent of all medication orders included a prescribing error, 31 percent of all orders included a transcription error, and 7 percent of all orders included an administration error. Less than 1 percent of the dispensing events included an error (Table 2). Error rates per category were also calculated per 1,000 patient-days, a measure commonly used in comparisons of error rates.

Discussion

The error rates found by the review team are consistent with those reported for general care units (13,14), but the distribution of errors across categories differs (3). At our hospital, the teams detected far more errors than were detected through self-reporting—a ratio of 244 to 1.

We could not find published reports of error rates for other psychiatric hospitals that were based on use of a review team. As noted above, our overall error rate is comparable to that reported for a large, multidisciplinary teaching hospital (13,14). Compared with the teaching hospital, our hospital has a smaller permanent medical staff and fewer beds and treats only psychiatric patients. Thus a lower overall rate might have been expected for our hospital. On the other hand, ours is a psychiatric tertiary care center—our patients all have severe and persistent mental illness, many have comorbid medical and neurological disorders, and most are taking multiple medications.

One limitation of our study is that the design did not include independent review of the data collected by the two-member review team and a measurement of interrater reliability. Another limitation is the inconsistency in the measurement methods—using concurrent review for detection of dispensing errors and retrospective review for detection of prescription, transcription, and administration errors. The proportions of medication errors in each category in our study—prescription, transcription, administration, and dispensing—differ from those reported for a general care setting by Bates and colleagues (3). In our study the rate of prescribing errors was significantly lower, and the rate of administration errors was much higher. The lower rate may be attributable to the factors noted above—the general hospital setting includes multiple medical disciplines, a larger medical staff, and a broader array of prescribed and administered medications. Another possible explanation of the lower rate of prescription errors may be the medical staff's use of PDAs, which has increased access to patient-specific clinical and pharmacological data. Lack of access to such data accounts for the two most common causes of errors in prescribing (3).

The data from our study show that detection of medication errors by an independent review team yields a much higher rate than self-reporting. The increase is consistent with findings from other hospital settings. In a study at Brigham and Women's Hospital, Bates and associates (13) identified 530 medication errors in 10,070 orders and projected that there were nearly 300,000 medication errors a year at that 700-bed hospital, whereas only a few hundred errors were self-reported, for a ratio of about 1,000 to 1. The ratio in our study was 244 to 1. In another study Classen and colleagues (14) found 731 adverse drug events when they used computerized monitoring, compared with only nine self-reported errors, for a ratio of 81 to 1.

In our institution, self-reporting typically yields less than 100 total medication errors a year, which is actually higher than the number reported in many health care facilities (16). In contrast, the team approach—which included retrospective review of charts, medication orders, medication administration records, and documented outcomes—plus concurrent review of dispensing events found 2,194 errors per 1,448 patient-days, yielding an annualized rate of 44,000 errors. The evidence supports our hypothesis that independent error detection and reporting identify error rates substantially higher than those detected by self-reporting.

The difference in rates found between the team and the self-reporting methods may be greater in our hospital than in others. Our use of medical technicians rather than licensed nurses for medication transcription and administration—a practice approved by licensing agencies but one that has since been changed—may lead to a higher error rate than in similar institutions. In addition, like many other state psychiatric hospitals, we are still recovering from the consequences of considerable downsizing, which has had implications for staff recruitment, retention, and morale, all of which can have an impact on staff performance.

Our results, as well as those of Bates and colleagues (13), Classen and associates (14), and others (17,18,19,20,21), raise questions about the adequacy of self-reporting as an error detection method. Yet many hospitals continue to use self-reporting by physicians, nurses, and pharmacists as the primary means of detection of medication errors. Impediments to self-reporting include fear of punishment for oneself or a colleague, insufficient emphasis on the value of detecting medication errors, and competing demands for the attention of clinical staff because of lower staff-patient ratios and a higher proportion of acutely ill patients.

Fifty-eight percent of errors in our study were rated as having a high risk of harm, which is of concern. All the errors listed in this category have been reviewed by nursing and medical staff, and processes to rectify them are being implemented, including a transition from unlicensed medical technicians to licensed nurses for all transcribing of orders and administration of medication. In addition, we are further motivated to move to computerized physician order entry and the use of informatics, which have been demonstrated to prevent errors (22).

Our results strongly support the benefits of using a review team to detect medication errors. The benefits include not only higher detection rates but also richer descriptive data for each error and therefore an improved ability to determine the clinical significance of errors. Corrective measures are also better guided by more comprehensive descriptive data as well as by more detection and reporting. However, the continuous use of a review team may not be necessary. An alternative might be periodic error sampling by a team, which would be less costly—excluding the potential costs of undetected errors—and could serve as a comparison for the accuracy of the self-reporting method.

Concerns about potential punitive actions were evident among our nursing staff, consistent with published surveys (23). We employed several strategies to address these concerns and to diminish staff resistance and anxiety. These strategies included education of the hospital's leadership about the value and importance of better detection of medication errors, use of a multidisciplinary team, continuous close collaboration with nursing leadership and reassurance that there would be no punishment for individual staff, and implementation by medical staff of related performance improvement activities, such as our previous use of PDAs to reduce the rate of medication errors.

Various stakeholders are contemplating ways of directing the medical profession toward improving patient safety. Legislators in several states have introduced bills designed to increase patient safety, including, in some states, mandated reporting of medication errors and financial penalties for failing to report (24). Others, such as the Leapfrog Group, are proposing nonpunitive measures, such as financial incentives to hospitals that adopt measures to improve safety (25).

We believe that it is preferable for the medical profession to establish its own clinical measures for decreasing medication error rates than to have less effective or clinically relevant remedies imposed from outside. We encourage our psychiatric colleagues in particular to initiate further studies of patient safety during hospitalization, establish acceptable thresholds for medication errors, determine whether greater hospital expenditures are needed to implement computerized physician order entry, and improve data collection and information management systems. They should also become more aware of current policy and legislative actions advocating mandatory disclosure of medication errors, take proactive measures to improve regulatory standards to further assist detection and reporting of medication errors, and improve psychiatry's response to reasonable patient demands for information on hospital performance and safety.

Dr. Grasso and Ms. Jordan are affiliated with the Institute for Self-Directed Care, 95 India Street, Portland, Maine 04101 (e-mail, [email protected]). Mr. Genest is with McKesson Medication Management. Dr. Bates is with Brigham and Women's Hospital and Harvard Medical School in Boston.

Enlarge table

Enlarge table