The American Psychiatric Association (APA) has updated its Privacy Policy and Terms of Use, including with new information specifically addressed to individuals in the European Economic Area. As described in the Privacy Policy and Terms of Use, this website utilizes cookies, including for the purpose of offering an optimal online experience and services tailored to your preferences.

Please read the entire Privacy Policy and Terms of Use. By closing this message, browsing this website, continuing the navigation, or otherwise continuing to use the APA's websites, you confirm that you understand and accept the terms of the Privacy Policy and Terms of Use, including the utilization of cookies.

×

Abstract

Objective:

This study examined whether outpatients with a psychotic disorder who are at risk of hospitalization can be identified by using data from electronic medical records (EMRs).

Methods:

Data from EMRs of outpatients enrolled in two clinics for treatment of psychotic disorders were abstracted. Monthly data were collected for 75 patients over two years. The study examined the association of medication nonadherence, substance use, participation in psychiatric rehabilitation, and long-acting injectable antipsychotic use in any given month with the risk of hospitalization in the subsequent month by using generalized estimating equations.

Results:

The only variable found to increase the relative risk of future hospitalization was recorded medication nonadherence (adjusted relative risk=7.19, p<.001).

Conclusions:

Results suggest that recording medication nonadherence in EMRs is feasible and that these data may be used to identify patients at high risk of future hospitalization, who may require more intensive intervention.

Emergency hospitalizations are among the major life stressors experienced by patients with schizophrenia and related psychotic disorders, as well as a major contributor to the cost of illness in this patient group. Inpatient care for patients with schizophrenia cost an estimated $2.8 billion in 2002 (1). More recent estimates of daily community hospital costs in the United States for this patient group range from $5,707 to $8,509 (2). Per patient, Medicaid pays significantly more for treatment of schizophrenia than for any other chronic condition (3), with hospitalizations accounting for a large portion of the cost. Psychotic disorders are the second most common diagnosis of Medicaid patients ages 18 to 64 who are rehospitalized within 30 days of an index hospitalization, and rehospitalizations of patients with psychotic disorders cost $302 million in 2011 (4). Hospitalization is frequently used as an outcome measure in studies of populations with schizophrenia (5), because it is both easily understood by clinicians and undesired by patients.

One of the aims of outpatient management of patients with psychotic disorders is to prevent hospitalization. A number of interventions have been developed that aim to reduce the rate of unwanted hospitalization, including long-acting injectable (LAI) antipsychotics (6) and a variety of psychiatric rehabilitation programs (7). Critical risk factors for hospitalization include substance use (8) and medication nonadherence (9).

Many of the protective and risk factors for hospitalization are routinely recorded in electronic medical records (EMRs), which are increasingly used in health care settings since the introduction of the Affordable Care Act. Implementation of EMRs requires “meaningful use,” including improvement in patient care, according to standards set by the U.S. Department of Health and Human Services (10). EMRs also provide an opportunity for methodical assessment of outcomes, both for clinical and research purposes. Previous efforts mining EMR data to analyze outcomes have typically been limited to laboratory test results and biometric data (11). There is insufficient evidence supporting examination of data from clinicians’ EMR notes for research purposes or demonstrating that this methodology yields results consistent with previous findings regarding patient outcomes.

This retrospective study was conducted in outpatient clinics for patients with psychotic disorders. We utilized EMRs as both a proof of principle that EMR data can answer clinical research questions and to determine whether inpatient admission was associated with four treatment variables extracted from EMRs: LAI antipsychotic use, participation in a psychiatric rehabilitation program, substance use, and medication nonadherence.

Methods

Data were collected from records created between May 1, 2013, and April 30, 2015. Patients included in the study were enrolled in the clinics as of April 30, 2015; were age 13 or older; and had a diagnosis of a primary psychotic disorder. Participants provided informed consent in a procedure approved by the Johns Hopkins University School of Medicine Institutional Review Board. Patients were excluded if they had a diagnosis of intellectual disability and were therefore unable to provide informed consent.

The study included patients from two clinics that focus on patients with psychosis at a major university hospital with a centralized EMR. All patients in these clinics had been assigned a psychiatrist and a master’s-level therapist. Diagnoses were made by the patients’ psychiatrist through a formal clinical interview using DSM-IV-TR criteria. Therapists provided demographic information (sex, race-ethnicity, and age). Records were reviewed by one of the authors (CA) to assess all other variables. Hospital discharge summaries and therapists’ intake notes were used to determine duration of illness and number of hospitalizations prior to the observation period. Treatment in the clinic preceded EMR implementation (May 1, 2013) for 23 patients (31%) in the final sample of 75. In these cases, paper charts were reviewed to obtain hospitalization history. Monthly variables (LAI antipsychotic use, psychiatric rehabilitation program attendance, substance use, and medication nonadherence) were assessed exclusively with data from EMRs.

Hospitalizations of clinic patients were recorded beginning in May 2013 for patients enrolled in the clinic prior to EMR implementation and 30 days after enrollment for patients initiating treatment after this date. The 30-day lapse accounted for patients who were enrolled during an acute phase of illness and thus rapidly hospitalized without significant treatment in one of the clinics. Data on medication nonadherence and substance use were collected for each patient from his or her therapist’s first appointment note of each month. Therapists’ notes include fields for medication adherence and substance use, which are assessed at every visit. For the purpose of data collection, when the response in these fields was not “yes” or “no” (for example, “questionable”), “nonadherent” was assumed for medication adherence, and “yes” was assumed for substance use.

The free text of the appointment note was also reviewed for further mention of medication nonadherence and substance use. For patients who had an appointment only with a physician in a given month, the physician’s note was reviewed in lieu of the therapist’s note. For patients using LAI antipsychotics who had neither a therapist’s note nor a physician’s note of medication adherence, notes from injection appointments were reviewed to determine whether patients were adherent to medication injections. Medical records were also searched for the presence of notes from psychiatric rehabilitation program staff to indicate program attendance. Data were not collected for months in which patients did not attend any appointments, including months during which they were hospitalized continuously.

We analyzed whether medication nonadherence, substance use, participation in psychiatric rehabilitation, and LAI medication use in any given month were associated with the risk of hospitalization in the subsequent month by using generalized estimating equations (GEEs) with logarithmic link and binomial distribution of residuals—that is, relative risk regression. GEEs correct for the repeated measures of multiple months per individual. In addition to the unadjusted models, multivariable models adjusted for the number of prior hospitalizations and time since onset of illness.

Results

Ninety-eight patients were enrolled in the clinic as of April 30, 2015, and were thus potentially eligible for participation in the study. Of these, 75 (77%) consented to participate, ten (10%) refused, two (2%) had not been enrolled in the clinic for over 30 days, two (2%) had diagnoses of intellectual disability, two (2%) were judged by their clinicians to be too symptomatic to consent, three (3%) did not have a primary psychotic disorder, and four (4%) were not asked to consent for other reasons at their clinicians’ discretion.

The ages of the 75 participants ranged from 13.6 to 59.0 (mean±SD=29.1±11.9), and 80% (N=60) were male. Most patients were African American (N=44, 59%), followed by non-Hispanic white (N=22, 29%), Asian (N=3, 4%), Hawaiian/Pacific Islander (N=1, 1%), and multiracial (N=2, 3%); three (4%) did not identify as one of these categories. Diagnoses included schizophrenia (N=42, 56%), schizoaffective disorder (N=17, 23%), psychotic disorder not otherwise specified (N=7, 9%), affective disorder with psychotic features (N=6, 8%), and other psychotic disorders (N=3, 4%).

The 75 patients contributed 1,118 months of observation, with a mean length of observation of 14±8 months. In collecting data on medication adherence and substance use, a physician’s note was used for five of the 1,118 months; therapists’ notes were used for all other months. Of 19 instances of LAI nonadherence, 15 (79%) were due to missed injections—that is, injections that were received seven or more days after they were due. In the other four instances (21%), injections were received six or fewer days after they were due and were thus considered late. During the observation period, 19 of the 75 patients (25%) were hospitalized (33 hospitalizations). Hospitalization was not significantly associated with psychiatric rehabilitation program attendance, substance use, or LAI antipsychotic use in the preceding month. However, medication nonadherence was associated with increased risk of hospitalization in the following month (adjusted relative risk=7.19) (Table 1).

TABLE 1. Association of four treatment variables and subsequent hospitalization among 75 patients with psychotic disorders

UnadjustedMultivariable adjusted
VariableRRa95% CIpARRb95% CIp
Medication nonadherent (reference: adherent)c6.963.18–15.10<.0017.193.31–15.62<.001
PRP attendance (reference: nonattendance)d.74.27–2.03.571.18.46–3.05.73
Substance use (reference: no use)e.78.18–3.31.74.71.16–3.20.65
LAI antipsychotic use (reference: no use)f.38.07–2.19.28.23.05–1.01.05

aRelative risk; correcting for multiple monthly measures per person

bAdjusted relative risk; adjusted for number of prior hospitalizations and time since onset of illness and correcting for multiple monthly measures per person

cNonadherent, 148 person-months; adherent, 761 person-months

dPRP, psychiatric rehabilitation program. Attendance, 296 person-months; nonattendance, 822 person-months

eUse, 99 person-months; no use, 1,019 person-months

fLAI, long-acting injectable. Use, 156 person-months; no use, 962 person-months

TABLE 1. Association of four treatment variables and subsequent hospitalization among 75 patients with psychotic disorders

Enlarge table

Discussion and Conclusions

Our data confirm previous findings that medication nonadherence is a major factor in the risk of hospitalization for patients with psychotic disorders. Assessing medication nonadherence, however, remains a difficult task. Our results show that assessment and recording of medication adherence by using EMRs is feasible and clinically useful for detecting changes in risk of future hospitalization, despite evidence that physicians tend to overestimate adherence (12).

A strength of our study—with important implications for application of our findings—was that the vast majority of adherence data were gathered by therapists and that this information was collected at every therapy appointment, which prevented missing data. Patients may be more willing to reveal medication nonadherence to nonphysician providers (13), and in many settings, therapists see patients more frequently than do psychiatrists or other prescribers. Although we make no claim that we completely captured all instances of nonadherence, an additional strength of our study is that assessment of adherence was within the scope of routine practice. In addition, we used a standard EMR system that can be employed in any type of clinic, although our clinics are affiliated with a large medical institution. Furthermore, even though nonadherence was strongly associated with hospitalization, it is possible that other, more proximal factors resulted in both nonadherence and subsequent hospitalization. Nevertheless, EMR recording of medication nonadherence may alert the provider to an elevated risk of hospitalization. In addition, because patients were not randomly assigned to participation in a psychiatric rehabilitation program or to LAI medication use, this study could not examine the utility of these measures in preventing hospitalization.

The widespread implementation of EMR systems provides an opportunity to improve clinical practice and facilitate research. Our results demonstrate that EMR data may be used for research designed to answer questions of direct clinical relevance to a specific patient population or in a single clinic of modest size. Our results also suggest the utility of EMRs in achieving the goal of personalized psychiatric treatment, in this case by automated tracking of medication nonadherence to identify patients at risk of hospitalization. Patients thus identified may be appropriate targets for more intensive outpatient care, with specific attention to changes that may improve adherence.

Currently, assessment of medication adherence by using EMR data primarily focuses on the relationship between electronic prescriptions and subsequent filling of prescriptions—the medication possession ratio (14). This methodology does not capture missed doses, a shortcoming that may be particularly problematic in studies of patients with psychotic disorders, compared with studies of patients with other chronic conditions. Collection of EMR medication adherence data from provider notes could supplement and enhance pharmacy data, improve outcomes, and reduce costs significantly in this patient population.

Except for Dr. Vaidya, the authors are with the Department of Psychiatry and Behavioral Sciences, Johns Hopkins University School of Medicine, Baltimore. Dr. Vaidya is with the Department of General Internal Medicine, Johns Hopkins University School of Medicine, Baltimore. Send correspondence to Dr. Frederick Nucifora, Jr. (e-mail: ).

Dr. Vaidya acknowledges institutional support from the Biostatistics, Epidemiology and Data Management Core, Johns Hopkins Center for Child and Community Health Research.

Dr. Vaidya reports serving as a consultant for Consumable Science, Inc. Dr. Margolis reports receipt of support from Auspex, Neurocrine, and Teva Pharmaceuticals. Dr. Sawa reports receipt of support from the National Institute of Mental Health (MH-094268) Silvio O. Conte Center (MH-092443 and MH-105660). The other authors report no financial relationships with commercial interests.

The authors thank Constantine Lyketsos, M.D., M.H.S., and the Department of Psychiatry and Behavioral Sciences at Johns Hopkins University School of Medicine for their support of this report.

References

1 Wu EQ, Birnbaum HG, Shi L, et al.: The economic burden of schizophrenia in the United States in 2002. Journal of Clinical Psychiatry 66:1122–1129, 2005Crossref, MedlineGoogle Scholar

2 Stensland M, Watson PR, Grazier KL: An examination of costs, charges, and payments for inpatient psychiatric treatment in community hospitals. Psychiatric Services 63:666–671, 2012LinkGoogle Scholar

3 Garis RI, Farmer KC: Examining costs of chronic conditions in a Medicaid population. Managed Care 11:43–50, 2002Google Scholar

4 Hines AL, Barrett ML, Jiang HJ, et al: Conditions With the Largest Number of Adult Hospital Readmissions by Payer, 2011, Statistical Brief 172. Rockville, MD, Agency for Healthcare Research and Quality, 2014Google Scholar

5 Burns T: Hospitalisation as an outcome measure in schizophrenia. British Journal of Psychiatry Supplement 50:s37–s41, 2007Crossref, MedlineGoogle Scholar

6 Kirson NY, Weiden PJ, Yermakov S, et al.: Efficacy and effectiveness of depot versus oral antipsychotics in schizophrenia: synthesizing results across different research designs. Journal of Clinical Psychiatry 74:568–575, 2013Crossref, MedlineGoogle Scholar

7 VanMeerten NJ, Harris JI, Nienow TM, et al.: Inpatient utilization before and after implementation of psychosocial rehabilitation programs: analysis of cost reductions. Psychological Services 10:420–427, 2013Crossref, MedlineGoogle Scholar

8 Cantor-Graae E, Nordström LG, McNeil TF: Substance abuse in schizophrenia: a review of the literature and a study of correlates in Sweden. Schizophrenia Research 48:69–82, 2001Crossref, MedlineGoogle Scholar

9 Sun SX, Liu GG, Christensen DB, et al.: Review and analysis of hospitalization costs associated with antipsychotic nonadherence in the treatment of schizophrenia in the United States. Current Medical Research and Opinion 23:2305–2312, 2007Crossref, MedlineGoogle Scholar

10 Blumenthal D, Tavenner M: The “meaningful use” regulation for electronic health records. New England Journal of Medicine 363:501–504, 2010Crossref, MedlineGoogle Scholar

11 Lin J, Jiao T, Biskupiak JE, et al.: Application of electronic medical record data for health outcomes research: a review of recent literature. Expert Review of Pharmacoeconomics and Outcomes Research 13:191–200, 2013Crossref, MedlineGoogle Scholar

12 Stephenson JJ, Tunceli O, Gu T, et al.: Adherence to oral second-generation antipsychotic medications in patients with schizophrenia and bipolar disorder: physicians’ perceptions of adherence vs pharmacy claims. International Journal of Clinical Practice 66:565–573, 2012Crossref, MedlineGoogle Scholar

13 Haddad PM, Brain C, Scott J: Nonadherence with antipsychotic medication in schizophrenia: challenges and management strategies. Patient Related Outcome Measures 5:43–62, 2014Crossref, MedlineGoogle Scholar

14 Carroll NM, Ellis JL, Luckett CF, et al.: Improving the validity of determining medication adherence from electronic health record medications orders. Journal of the American Medical Informatics Association 18:717–720, 2011Crossref, MedlineGoogle Scholar