Use of Administrative Data to Identify Potential Service Gaps for Individuals With Serious Mental Illness

Public mental health leaders have called for a greater emphasis on evidence-based policy and data-driven decision making in policy and program development (1,2). For example, Frazier and Casper (3) demonstrated that clinical events identified from Medicaid claims data predicted future psychiatric hospital readmissions and could be successfully used to guide early intervention strategies. As information technology advances, administrative databases offer the potential to inform policy initiatives and provide real-time clinical decision support.

In 2008, New York City and State government leaders convened a panel of experts to examine incidents of violence involving individuals with serious mental illness. The panel concluded that fragmented public mental health services and limited accountability among providers contributed to gaps in services for individuals with serious mental illness living in the city (4,5). The panel recommended the creation of care monitoring teams in New York City to use Medicaid claims and other state administrative data to examine patterns of service use by high-need individuals at risk of poor outcomes. The panel's recommendations were accepted in June 2008, and the city's public mental health leadership subsequently designed the Care Monitoring Initiative (6). In it, trained clinicians review cases triggering flags of potential clinical concerns with providers who were identified on the basis of Medicaid claims data as having served the individual in the prior 12 months. The reviewers determine whether individuals are receiving adequate and appropriate services, and if not, they work with providers to develop plans for outreach and reengagement and where possible, continue contact with the provider until the individual has reengaged in appropriate services.

The project described in this study was conducted in the spring of 2009, several months before the first care monitoring team started in Brooklyn. The project assessed whether the service use algorithms accurately identified clinically needy individuals who were disengaged from appropriate services.

Methods

Individuals eligible for the Care Monitoring Initiative include residents of New York City who have ever been ordered by a court to receive assisted outpatient treatment (AOT) under New York State's outpatient commitment statute, even if the order had expired (current and expired AOT cohorts), received assertive community treatment (ACT) or case management in the prior 12 months (subdivided into intensive case management [ICM] and blended case management [BCM] cohorts), received mental health services in a forensic psychiatric setting in New York State and were presently living in the community (forensic cohort), or have had two or more psychiatric emergency room or inpatient hospitalizations in the prior 12 months (multi-acute user cohort). The cohorts are ranked by degree of service need, and individuals eligible for more than one cohort are assigned to the group with higher needs. Preliminary data analyses indicated that these groups in total include approximately 20,000 city residents at any point in time.

The study focused on individuals with Brooklyn addresses who belonged to one or more of the high-need cohorts as of September 30, 2008 (N=6,408). Medicaid service claims were analyzed to identify individuals in these cohorts who may have been experiencing gaps in needed services (approximately 85% of individuals in the above cohorts had Medicaid eligibility in the prior year). Patterns of nonuse of services use that triggered notification flags were failure to fill a psychotropic medication prescription in the prior 60 days; not receiving community-based mental health services in the prior 120 days; or having had two or more psychiatric inpatient admissions or emergency visits in the prior 120 days.

The look-back periods (60 and 120 days) for the notification flags were determined based upon review of claims lag data. For example, almost all Medicaid pharmacy claims in New York State are paid at the time that the prescription is filled, yielding very timely pharmacy data. Therefore, when a Medicaid-eligible individual has no pharmacy claims over a two-month period, it is likely that the individual is no longer taking medication. For clinical services (community-based and inpatient or emergency room treatment) the claims lag was significantly greater—on average, 60 days were required to ensure that at least 75% of providers had submitted claims for services. To avoid a high number of false-positive notifications—delays in claims submission as opposed to service gaps—the look-back period was set at 120 days for claims for clinical services.

Of the 6,408 individuals in the overall population, 4,826 met one or more of the notification flags as of October 1, 2008, or November 1, 2008. A random sample of 500 individuals was generated from this group, including at least 50 from each of the high-need cohorts.

A review panel of seven master's level mental health clinicians who had program oversight responsibilities for the New York City or New York State mental health authorities completed reviews. Orientation to the project included a review of the panel report (5) and discussion of case vignettes illustrating levels of engagement in services for high-need individuals. Over a two-week period, the reviewers contacted providers who had been identified based on Medicaid claims as having served the individuals in the prior 12 months. A script created to guide telephone discussions contained specific questions eliciting information about individuals' prior use of services, degree of current engagement in care, and the provider's most recent risk assessment. The goal was to complete reviews for at least 20 individuals from each high-need cohort except the current AOT cohort and to complete reviews for all 58 individuals with current AOT orders. The study was considered a program evaluation activity that did not involve human subjects interviews and did not require institutional review board review.

On the basis of discussions with mental health administrators in New York City and New York State and the work of the review panel described above, the project leadership developed three categories to summarize case reviews. Level 1 cases included those in which the pattern of service use or nonuse was determined not to represent a clinical concern. An example involved individuals receiving medication from clinics' sample supplies without using their Medicaid prescription cards. These individuals triggered the psychotropic medication notification flag but were, in fact, receiving medication.

Level 2 cases involved individuals who were not adequately engaged in care but whose providers had put in place an adequate reengagement plan. An example included an individual receiving ACT services who had had a relapse of psychotic symptoms and who had been hospitalized twice in the prior four months. Because the ACT team was aware of the hospitalizations, had identified the stressors associated with the relapse, and was working with the inpatient treatment team to modify the individual's care plan in an effort to address the stressors and treat the relapse, the case was categorized as level 2.

Finally, level 3 cases involved individuals who were not adequately engaged in care and whose previous providers had not reported having put in place adequate reengagement plans. Examples included individuals who had refused treatment and were lost to care in spite of outreach efforts as well as individuals whose providers had made referrals to inappropriate levels of care that resulted in disengagement and a subsequent lack of follow-up.

Results

Clinicians initiated reviews with providers for 230 of the 500 individuals identified. Reviews were incomplete for 42 (18%) individuals because the clinicians were unable to obtain sufficient follow-up information from providers during the two-week review period. Examples included situations in which providers cited confidentiality concerns or individuals were discharged from care several months previously and no information was available to determine whether they were engaged in services at the time of review. Staff completed reviews for the remaining 188 individuals (82%); each review required an average of three phone contacts (range one to 14) and a time commitment of an average of 15 minutes (range one to 116).

Figure 1 summarizes the number of individuals in the three category levels from each high-need population For the total population, 66 reviews (35%) were categorized as level 1, or of no clinical concern. Eighty-four cases (45%) were labeled as level 2, an indicator that service gaps had been acknowledged and adequately addressed. Thirty-eight cases (20%) were coded as level 3, an indicator of continued need for outreach and reengagement efforts to address a recent service gap.

The percentages of individuals assigned to the various levels of care disruption varied by population. Individuals receiving the most intensive services (AOT and ACT) had lower rates of service gaps. Individuals in the multiacute user cohort (two or more psychiatric inpatient or emergency visits in the prior 12 months) were the most likely to be experiencing service gaps, given that nearly 90% of the gaps were classified as either level 2 or 3. The extent to which individuals had a current community-based provider who was familiar with their care and needs also varied by cohort; rates were highest for the AOT current cohort (90%), followed by ACT (61%), case management (59%), forensic (40%), AOT expired (33%), and multiacute user (15%).

Since this study ended, a care monitoring team was established in Brooklyn. Data from the first six months of care monitoring in Brooklyn yielded somewhat different proportions than those found by the study. The care monitoring team identified 7,242 individuals who triggered a notification; 4,801 reviews were initiated, and 2,545 (53%) were completed within one month (compared with the study rate of 82% for reviews completed in a month). Of the completed reviews, 30% were coded level 1; 18%, level 2; and 52%, level 3, These rates compared with rates of 35%, 45%; and 20%, respectively, for individuals in levels 1–3 reported by the study).

Discussion

In this study, trained clinicians contacted providers and completed timely reviews of high-need individuals with serious mental illness identified via secondary data analyses as possibly experiencing a gap in services. The 188 individuals reviewed were from a random sample of 500 of the 4,826 individuals who met defined notification triggers, and unknown biases related to the relatively small number of reviews and their distribution across the various client cohorts may have influenced the proportion of cases deemed to present higher or lower levels of clinical concern.

Nonetheless, the findings indicate that the service use algorithms were an efficient means of identifying individuals who were in need of treatment but had not become adequately engaged in appropriate services.

We anticipated that the large high-need cohorts defined by the study would yield high numbers of level 1 assignments, given the expectation that claims would lag and the knowledge that Medicaid eligibility and utilization vary significantly over individuals and across time. Such instances were the minority, however, for all populations except for the AOT cohort. These individuals, who were under court orders to participate in community-based care, received extensive monitoring and were expected to have minimal service gaps. Of note, however, is that a substantial number (15%–35%) of individuals in the ACT and case management cohorts also were disengaged from appropriate services. These high-need individuals had been accepted into intensive services in the prior 12 months and had failed to remain engaged in care. Efforts to understand their disengagement from services and what was needed to reengage them in appropriate services should identify opportunities to strengthen the public mental health safety net.

The study illustrated some of the challenges of using secondary data. The 60- to 120-day look-back period for clinical services is necessary, because of claims lag, to limit the numbers of false positive cases. Because of the lengthy look-back period, however, for some individuals a lapse in care was not identified for at least two months, increasing the chances that providers would not be able to contact them after they had been lost to care for so long. Also, the notification definitions relied on Medicaid claims, and individuals who were not eligible for Medicaid were not identified. These are unfortunate limitations, but given the large numbers of high-need individuals with Medicaid who triggered notifications, the initiative offers many opportunities for outreach and engagement of individuals identified as in need of continuing services.

As noted above, after implementation of the Brooklyn care monitoring team by a contracted vendor, more cases were categorized as having level 3 service gaps than were found in this study. There are several potential reasons for this disparity. For example, the vendor staff consisted of newly hired clinicians who were still becoming familiar with the New York City provider system of care and, therefore, were still establishing relationships with providers and educating them about the program. Providers may have been more likely to respond to staff of New York City or New York State who conducted the study because they had already established relationships with them through work than to vendor staff who they didn't know. Finally, the vendor contract required more than identification of gaps in services. Vendor care monitors documented risk factors and other clinical data, followed up cases of individuals disengaged from care, and helped providers develop outreach and reengagement plans. These follow-up requirements and associated activities could have influenced judgments about specific cases. Further experience with the initiative, including case audits, will clarify these issues.

Conclusions

This study confirmed that applying service use algorithms to Medicaid claims data could successfully identify individuals with serious mental illness who are in need of outreach and engagement.

Figures and Tables