Use of Administrative Data to Identify Off-Label Use of Second-Generation Antipsychotics in a Medicaid Population

Second-generation, or atypical, antipsychotics, have largely supplanted conventional antipsychotics, such as haloperidol, for the treatment of schizophrenia. Spurred by increasing use both for indications approved by the U.S. Food and Drug Administration (FDA), such as bipolar disorder, autism, and treatment-resistant depression, as well as for many off-label conditions, utilization of second-generation antipsychotics has grown enormously over the past two decades (1). Data from Medicaid and the U.S. Department of Veterans Affairs suggest that close to two-thirds of second-generation antipsychotic prescriptions are used for off-label conditions, such as attention-deficit hyperactivity disorder, anxiety, insomnia, and dementia-associated symptoms (2–4).

Prescribing off the FDA label is not problematic in and of itself, but the evidence supporting the practice typically is not as strong as evidence for approved indications. In fact, off-label use may be an important contributor to adverse events (5). A recent systematic review by the Agency for Healthcare Research and Quality of off-label uses of second-generation antipsychotics found that use of the medications for many of the conditions was not well supported by evidence (6). Prevalent off-label use of second-generation antipsychotics is also concerning because these drugs are associated with numerous serious adverse effects, such as metabolic side effects, diabetes, and stroke and death in elderly patients. Several second-generation antipsychotics have gained additional FDA indications for once off-label conditions, for example, adjunctive treatment of depression and irritability associated with autistic disorder, but the preponderance of evidence supporting use of second-generation antipsychotics relates to treatment of schizophrenia and bipolar disorder (7).

It is estimated that $4 to $5 billion is spent annually to treat a variety of conditions with second-generation antipsychotics despite a lack of strong evidence to support their use (8). State Medicaid programs shoulder a significant portion of mental health care in the United States and, as a result, finance much of this off-label use. As state budgets constrict and Medicaid populations expand, many states are exploring innovative policies to stretch scarce resources. Several studies have demonstrated that patients with severe mental illness, such as schizophrenia or bipolar disorder, are adversely affected by policies that restrict medication access (9–11). Developing ways to exclude these vulnerable populations from commonly used pharmacy management policies, such as cost-sharing, may be advantageous.

Programs and policies aimed at improving off-label second-generation antipsychotic use are predicated on the ability of payers to correctly determine conditions present when these drugs are prescribed. This determination could be facilitated by identifying International Classification of Diseases, ninth revision (ICD-9), coding from health administrative claims data. Although the literature quantifying the magnitude of off-label second-generation antipsychotic use derived from claims data is generally consistent with estimates from other sources, it is unclear how accurate claims data are for determining off-label prescribing of second-generation antipsychotics.

Work by Lurie and others (12) suggested that Medicaid claims data are reasonably accurate for determining the presence of schizophrenia but less than ideal for ruling out its absence. In that study, a definitive diagnosis of schizophrenia could not be ruled out for nearly half of patients who lacked a Medicaid claim for schizophrenia. Less is known about the accuracy of claims data for identifying or ruling out bipolar disorder. The objective of this study was to determine the ability of Medicaid claims to identify off-label use of second-generation antipsychotics by estimating concordance with diagnoses in the medical record.

Methods

We used a cross-sectional diagnostic study approach to determine the sensitivity, specificity, positive predictive values (PPV), and negative predictive values (NPV) of state Medicaid data in determining the presence of off-label prescribing among patients using a second-generation antipsychotic. Electronic health records from a regional health care system were used as our comparative data source. By the end of 2009, three of the five most prescribed second-generation antipsychotics (olanzapine, quetiapine, and aripiprazole) were FDA-approved for the treatment of schizophrenia and bipolar disorder as well as for adjunctive treatment of major depressive disorder (treatment-resistant depression). Neither risperidone nor ziprasidone were FDA-approved for adjunctive treatment of depression. Although risperidone was also approved for treatment of irritability in children with autism during this period, we did not consider this diagnosis in our analysis, given that it was not a major prescribing indication at the time (1).

Population and data sources

Medicaid claims data from the state of Oregon were used to identify the study sample. The study sample comprised patients with at least one prescription fill for risperidone, olanzapine, quetiapine, ziprasidone, or aripiprazole during 2009. Patients were also required to be enrolled in the Medicaid program for at least 75% of the year.

The comparative data source consisted of medical record data acquired from the Oregon Health & Science University (OHSU). The university is the only academic medical center in the state of Oregon and comprises a large tertiary care hospital with associated primary and specialty care clinics on its main campus in Portland. Additionally, OHSU has several satellite family practice clinics around Portland and southwest Washington. OHSU uses EpicCare as its core electronic health record software in both ambulatory and inpatient settings.

Data from EpicCare are also collected in a research data warehouse (RDW) maintained by the Oregon Clinical and Translational Research Institute, OHSU's Clinical & Translational Science Award Center. Since 2006, the RDW has accumulated data for over 20 million patient encounters by over 2.5 million unique patients. The RDW contains a diversity of rich clinical, demographic, and family and social history data, including problem lists, medications, laboratory results, and narratives of health care encounters that document provider notes. We attempted to obtain the electronic medical records of patients in the study sample by matching Social Security numbers from the claims data with those in the Epic RDW in 2009. Only individuals with two distinct outpatient encounters at an OHSU health care facility were included in the study sample to ensure sufficient capture of clinical data in the RDW.

Diagnostic algorithm and analysis

After establishing the study sample, Medicaid claims were analyzed to determine how many users of second-generation antipsychotics did not have an FDA-indicated diagnosis. Because only three of five second-generation antipsychotics were approved for use for treatment-resistant depression, we considered two separate analyses. First, we evaluated the utility of the claims database to determine off-label use of second-generation antipsychotics with long-standing FDA approval for treatment of schizophrenia and bipolar disorder. Second, we evaluated the utility of the claims database to determine off-label use in a subset of patients using second-generation antipsychotics with an indication for treatment-resistant depression (olanzapine, quetiapine, and aripiprazole). The following ICD-9 codes were used to define these diagnoses in the claims data set: schizophrenia (295xx), bipolar disorder (2960x, 2961x, and 2964x−2968x), and depression (2962x, 2963x, 2980x, 3004x, 3091x, and 311xx). Only patients who had filled a prescription for a newer antidepressant (selective serotonin reuptake inhibitor, serotonin-norepinephrine reuptake inhibitor, and bupropion) within 30 days of filling a prescription for a second-generation antipsychotic were considered to have a diagnosis of treatment-resistant depression.

Because of the prominent role Medicaid plays in the care of children, we also evaluated the concordance of claims data and the medical record in a secondary analysis of those younger than 19 years of age and of those older than 18 years of age.

The OHSU electronic health record problem list is a modifiable list of current medical conditions that is maintained by health care providers. Conditions are structured by the ICD-9 hierarchy and coding system. We analyzed the problem list as well as the encounter diagnoses to determine whether the same ICD-9 codes appeared in the clinical data set during the same time period. Because providers are able to enter a more detailed free-text narrative progress note for each encounter, we also conducted a manual chart review on a random subsample of 25% of patients. The medical record of each patient in the subset was reviewed manually for mention of the three conditions of interest to ensure that diagnostic information was not omitted. For each diagnosis we calculated the kappa statistic to determine the level of agreement between the chart review and the automated electronic health record data. A kappa statistic greater than .80 is generally considered to indicate a high level of agreement (13).

For the primary study sample, we determined the diagnostic accuracy of not having a claim for schizophrenia, bipolar disorder, or neither disorder. Because our principal interest was determining the accuracy of the claims data for assessing the lack of certain diagnoses in the medical record, typical measures of diagnostic accuracy were defined slightly differently. We defined sensitivity (true positive) as the probability of not having a Medicaid claim for a diagnosis given a lack of evidence for it in the medical record. Specificity (true negative) was defined as the probability of having a Medicaid claim for a diagnosis given evidence for it in the medical record. Similarly, PPV and NPV were calculated to assess the ability of the claims data to predict the absence and presence, respectively, of FDA-approved diagnoses in the medical record. Because both PPV and NPV are dependent on the observed prevalence in the study sample, we used Bayes’ theorem to compute PPV and NPV for a range of off-label prescribing prevalence estimates (14) as shown below:

The same analytic approach was also applied to the subgroup of users of second-generation antipsychotics approved for use with treatment-resistant depression and to the age subgroups. Statistical analyses were conducted by using Stata, version 12. This study was approved by the institutional review board at OHSU.

Results

In calendar year 2009, a total of 17,384 unique Medicaid beneficiaries filled one or more prescriptions for a second-generation antipsychotic. From this group, the 788 patients who met the inclusion criteria and had two or more distinct encounters documented in the OHSU medical record were included in the study sample.

Table 1 shows the demographic characteristics of the study sample. In general, the characteristics of the sample were consistent with a Medicaid-enrolled population. A majority of patients were female. The average age was 34 years, and 31% were younger than 19. Quetiapine was the most frequently used second-generation antipsychotic (38%), followed by risperidone, aripiprazole, olanzapine, and ziprasidone. Sixteen percent of patients filled prescriptions for multiple second-generation antipsychotics over the course of the year. Only 17% of patients had documentation of either schizophrenia or bipolar disorder in their medical records.

The manual chart review of the 25% random sample (N=200) found high agreement between the diagnoses in the electronic health record and the provider narrative for patients with schizophrenia or bipolar disorder. Of 53 patients with schizophrenia, 11 were identified though manual review of the provider narrative (kappa=.85). Of 72 patients with bipolar disorder, 16 were identified through review of the provider narrative (kappa=.82). However, 45 of 84 patients with depression were identified by the provider narrative (kappa=.50). In this sample, therefore, additional diagnostic data provided through manual review had a small impact on the sensitivity and PPV of claims data for identifying patients without schizophrenia or bipolar disorder. However, manual review had more pronounced effects on sensitivity and PPV of claims data for identifying patients without a diagnosis of depression. [The results of the manual review are available online as a data supplement to this article.]

Table 2 shows the concordance of claims data and medical records for indicating the absence of schizophrenia, bipolar disorder, or both. Electronic health records contained no documentation of schizophrenia for 727 (92%) patients and no documentation of bipolar disorder for 707 (90%) patients. Taken together, 651 (83%) individuals had neither diagnosis. The kappa statistic ranged from .38 to .49, indicating fair to moderate agreement. Table 3 provides the same information for the subgroup of patients (N=559) who used the three second-generation antipsychotics approved for adjunctive treatment of depression. The kappa statistic for schizophrenia and bipolar disorder was similar to that for the full study sample. Agreement between the data sources for indicating absence of depression was lower among persons using an antidepressant (kappa=.29).

The sensitivity and specificity of the claims database for identifying patients in the medical record without a diagnosis of schizophrenia was 91% and 77%, respectively (Table 4). For patients without bipolar disorder, claims data provided a sensitivity of 80% and a specificity of 85%. The sensitivity and specificity of claims for determining absence of either diagnosis was 72% and 85%, respectively. The prevalence of neither diagnosis was 83%, resulting in a predictive ability (PPV) of 96%. Although the sample size was limited, PPVs were marginally higher among younger (<19) versus older (>18) persons. [Analyses of concordance of claims data and medical records by age are available online in the data supplement.]

Analysis of the subsample of patients who were prescribed a second-generation antipsychotic approved for treatment-resistant depression and an antidepressant indicated that the medical claims were less able to predict a lack of depression diagnosis than a lack of schizophrenia or bipolar disorder diagnoses, although the PPV was still relatively high (91%). The claims data were least able to predict off-label indications (87%) for the three second-generation antipsychotics with indications for treatment-resistant depression when all approved conditions were considered.

Figure 1 depicts the PPVs of the Medicaid claims data to determine estimates of off-label prescribing of second-generation antipsychotics. Because PPV is dependent on the underlying prevalence of off-label prescribing, we computed PPV for a range of estimates of prevalence of off-label prescribing. The solid line reflects the accuracy of the claims for identifying patients without long-standing FDA-indicated diagnoses (schizophrenia and bipolar disorder). The dashed line reflects the accuracy of the claims for identifying patients without any FDA-indicated diagnosis in the subgroup prescribed a second-generation antipsychotic with indications for treatment-resistant depression. The range of plausible off-label prescribing rates, indicated by the shaded area, reflects varying estimates of the prevalence of off-label prescribing, from 60% in the most recent external estimate (1) to 83% in the study sample. Given this range of plausible values, the sensitivity and specificity estimates derived in this study suggest claims data can predict the absence of common FDA-indicated diagnoses in the medical records of patients prescribed second-generation antipsychotics with 81% to 96% accuracy.

Discussion

This study found that among Medicaid patients using second-generation antipsychotics, the sensitivity and specificity of claims data are reasonably good for identifying patients without FDA-indicated diagnoses relative to the medical record. Given the high prevalence of not having conditions identified by the FDA as indications for second-generation antipsychotic treatment, the data also had very high positive predictive ability for identifying individuals without schizophrenia (98%) and without bipolar disorder (98%). The ability of claims data to predict off-label prescribing of second-generation antipsychotics also indicated for adjunctive treatment of resistant depression was lower (87%).

When PPVs were calculated with other observed estimates of off-label prescribing, the results ranged from 81% to 96% (Figure 1). In other words, program administrators and policy makers concerned about off-label prescribing can be fairly confident that claims data can identify a large majority of patients lacking documentation in the medical record of an FDA-indicated diagnosis. Moreover, claims data were accurate 96% of the time in predicting the absence of diagnoses of schizophrenia and bipolar disorder among those using second-generation antipsychotics.

Although not perfect, our manual review of the medical record suggested that Epic RDW accurately identified schizophrenia (kappa=.85) and bipolar disorder (kappa=.82). Estimates of sensitivity and PPV were not altered appreciably when additional information from the progress note was included. However, in more than half of all cases, depression was mentioned in the progress note but was not included in the problem list or identified as an encounter diagnosis. The underdocumentation of depression is a contributing factor that likely explains the lower sensitivity and specificity of the claims data for detecting off-label use of second-generation antipsychotics with indications for treatment-resistant depression. The reasons for this discrepancy are unclear, although it could represent a difference in severity of illness between those with depression and those with schizophrenia or bipolar disorder (15). [The results of the manual review are available online as a data supplement to this article.]

Currently, Medicaid and Medicare are the primary payers for a majority of second-generation antipsychotic use nationally. The management of second-generation antipsychotics, however, is challenging because of the complicated and vulnerable nature of patients with mental illness and the heterogeneity of the drugs themselves (16). Attempts to control utilization through automated drug policies, such as prior authorization, medication limits, and cost-sharing, have been shown to produce deleterious effects in certain populations of persons with mental illness (9,10,17,18). Accordingly, the ability to craft nuanced drug policy that minimizes medication disruptions for those with severe mental illness is attractive. For example, there is considerable interest at the state and federal levels in improving the efficiency and safety of prescribing second-generation antipsychotics for residents of long-term care facilities who are enrolled in Medicaid or in both Medicare and Medicaid (19). The ability to use claims data sources to quickly and accurately identify potentially questionable prescribing practices for education, medication therapy management, and other forms of outreach or intervention is appealing.

Using State Drug Utilization Data provided by the Centers for Medicare and Medicaid Services, we estimate that Medicaid programs in 45 reporting states paid over $3.6 billion for antipsychotics in 2009. Assuming conservatively that half of all second-generation antipsychotic use is off label, interventions aimed at reducing off-label use could yield savings of up to $1.8 billion. Additionally, curbing questionable off-label use would reduce unnecessary exposure to a class of medications known to have serious adverse effects (20,21). However, because most studies of policies related to second-generation antipsychotics to date have included only individuals with schizophrenia or bipolar disorder, little is known about how policies that limit or discourage utilization of second-generation antipsychotics affect individuals with other disorders.

Several limitations should be acknowledged. Although our findings may be informative for state Medicaid programs because of the prominent role they play in financing care for individuals who use antipsychotics, issues of generalizability remain. First and foremost, these data were generated from one relatively small state’s Medicaid program and may not reflect patterns of use of second-generation antipsychotics in other states or from claims data from other payers such as Medicare. Furthermore, our sample contained very few individuals who were over the age of 65 and likely living in a long-term care facility, a population known for high and problematic use of second-generation antipsychotics (22–24). Because our inclusion criteria required Medicaid enrollment for most of the year, extrapolation to patients who are newly enrolled may be limited. Another limitation concerned our comparison data set. We compared claims-based diagnoses to those identified in the medical record by querying the problem list and encounter diagnoses provided in the OHSU RDW. We relied on the documentation of the provider and did not attempt to establish the true diagnostic presence of the conditions we evaluated. As a result, the RDW data source does not represent a true gold standard for disease presence. Because the electronic health record includes both definitive disease and other related symptomatic manifestations, it may be an exaggerated record of possible diagnoses.

Conclusions

Medicaid health care claims data sources were reasonably accurate at determining the absence of FDA-indicated conditions in the medical records of individuals using second-generation antipsychotics. Individuals without a claim for schizophrenia or bipolar disorder were very unlikely to have documentation of the condition in the medical record. With this in mind, it may be possible to develop or refine policies to manage utilization and costs yet minimize administrative access barriers for those with severe mental illness. Future research is needed to determine the extent to which commonly used drug policies affect the spectrum of conditions treated with second-generation antipsychotics.