The Impact of the Massachusetts Behavioral Health Child Screening Policy on Service Utilization

Child behavioral health (BH) conditions frequently are seen in pediatric primary care. In fact, primary care providers are children’s default source of mental health care (1). As a consequence, BH screening is recommended by national organizations, and it is increasingly covered by public and private insurers to increase identification of BH issues and enhance families’ engagement in needed services (2,3). However, evidence of the impact of screening on subsequent BH care is lacking. In particular, there are no studies comparing states with policies requiring screening to states without similar mandates. A legal settlement in Massachusetts, mandating screening of children covered by the state’s Medicaid program, provided an opportunity to study the policy impact of screening mandates on rates of subsequent use of BH services at the population level.

The Massachusetts mandate resulted from a class action lawsuit, Rosie D. versus Patrick (4). In 2006, a U.S. District Court found Massachusetts out of compliance with federal Medicaid law (5). The resulting remedial plan required Massachusetts to provide screening and follow-up to Medicaid-eligible children in what has become one of the largest child mental health system changes in the nation (6). As the first phase of the plan, primary care providers were required, starting in January 1, 2008, to screen for BH at well-child visits for all children and youths through age 21. It was not until July 2009 that additional mental health services were implemented as part of the plan, thus providing an 18-month period in which to study the impact of mandated BH screening as a stand-alone intervention (7).

Prior studies of child BH screening have found that screening leads to an increase in outpatient BH visits, although outcomes have been mixed and vary across health systems and diagnoses (8,9). Several studies of the Massachusetts screening mandate in particular found that claims for BH screening and evaluations rose after BH screening began (10,11) and that two factors—identification of a new BH problem at a screening visit and being in foster care at the time of screening—independently predicted subsequent receipt of a BH visit (12). However, these studies did not control for the potential impact of regional or national trends in the use of children’s BH services (increasing nationwide) (13,14) or for ongoing efforts to improve children’s access to BH services (9,15,16). The studies also did not compare trends in Massachusetts with trends in any other states. In this study, we examined the impact of the screening mandated under Rosie D by using an interrupted time-series design—a strong quasi-experimental approach for assessing changes in service utilization—along with longitudinal data from another state for comparison. Our goal was to determine whether a policy of mandated BH screening increased outpatient, emergency, and inpatient BH care utilization rates for Medicaid children in Massachusetts at a population level.

Methods

This study used Medicaid Analytic Extract (MAX) claims data from Massachusetts and California to compare utilization of BH services (screening, outpatient, emergency department [ED], inpatient, and medication) after initiation of a BH screening mandate for children. California was chosen as the comparison state because it has a large diverse and stable Medicaid population and had no competing intervention. We initially considered New York, a more proximate state, for comparison, but during the analysis we discovered that it had also initiated a BH policy and that it was similar to the Massachusetts policy. An interrupted time-series with control series design was utilized.

BH Screening Policy in Massachusetts Under Rosie D

The Massachusetts policy requires primary care providers to conduct BH screening at well-child visits. Providers must select screens from a menu of validated tools (17) and are compensated if they submit claims using a specific billing code. The state monitors screening rates for Medicaid enrollees and provides feedback to pediatric practices.

Data

Data were obtained for all Medicaid-eligible children for the period January 1, 2006, to December 31, 2009, in two states, Massachusetts and California. The data included demographic and enrollment data (personal summary file), hospital inpatient claims (inpatient file), outpatient claims (other file), and pharmacy claims (Rx file). We used the encrypted Medicaid Statistical Information System identification number to identify individuals. The Massachusetts BH screening policy went into effect January 1, 2008, providing 24 months of data prior to the policy implementation and 18 months of postpolicy data prior to the implementation of additional BH program components in fall 2009.

This study was approved by the institutional review board at Cambridge Health Alliance and Group Health Cooperative.

Population

The population of interest in this study was children of “screenable” school age. We limited our sample to children older than 4.5 years and younger than 18 years at the time of screening because most providers used the same BH screen (18)—the Pediatric Symptom Checklist and the Youth Pediatric Symptom Checklist (19)—which covers this age range. We included all BH claims for youths up to age 20 in order to capture utilization that occurred after a screening encounter at 17.99 years of age (in other words, to capture BH utilization in the “run-out” period).

Inclusion Criteria

For inclusion, all individuals required at least ten months of Medicaid eligibility in each calendar year to ensure that most utilization was considered. However, the cohort of youths included was “rolling” in the sense that individuals could enter and exit the study cohort in a given year.

Population Standardization

Because we used a rolling cohort of youths, change in the composition of the patient population was a threat to the validity of the interrupted time-series design (20). For example, changes in the distribution of sex or age over study years could confound any change in BH utilization rates. We examined the demographic and enrollment characteristics of the population in each year and found differences in managed care enrollment, eligibility status (including foster care and poverty), race, and Hispanic ethnicity. Thus we standardized the population of youths on these characteristics to the distribution in January 2008, the date that the screening policy was implemented. It is also important to note that there were no sudden changes in level or slope of demographic characteristics that could confound observed effects. Because the analysis of trends was within populations, there was no need to standardize populations between states.

Outcome Variables

Definitions of utilization outcomes have been used in prior research (12,21) and were met if a minimum of one visit per utilization type (categories 1–5) per day occurred. Outcomes included BH screening, defined by any outpatient visit with a 96110 Current Procedural Terminology code (developmental testing); BH-related outpatient visits, defined as psychiatric services (such as diagnostic interviews, psychopharmacology management, and psychotherapy), health behavioral assessment and intervention services, visits to other mental health professionals, the Massachusetts Behavioral Health Partnership codes used to track services not otherwise identified with existing codes (crisis intervention, family counseling, and case management), and well-child visits or ambulatory visits that had an associated BH-related ICD-9 code (290–319); BH-related ED visits, defined by the place-of-service code on records in the “other” MAX data file, and mental health–related ED visits that were identified by an associated BH-related ICD-9 code; BH-related hospitalizations, defined as inpatient stays with a BH-related ICD-9 code; and psychotropic medication use, defined by National Drug Codes (NDCs) included on the Mental Health Research Network list.

The medication categories included attention-deficit disorder–other (nonstimulant medications), antidepressants, antianxiety-other (nonbenzodiazepines), anticonvulsants, antipsychotic–first generation, antipsychotic–second generation, benzodiazepines, “combo” (all combination psychotropic medications), hypnotic-other (such as zolpidem), lithium, and stimulants. A full list of study medications and NDCs is available on request. Drugs with possible dual use were excluded, such as antidepressants used primarily for migraines and enuresis among children (imipramine and amitryptiline), antidepressants used for sleep (doxepin and trazodone) when no other psychiatric medication was being used and there was no BH ICD-9 code, and anticonvulsants unless accompanied by any BH ICD-9 code. For example, if a patient had a diagnosis of bipolar disorder on any prior visit and also used an anticonvulsant, that patient was included for use of psychopharmacology.

Calculation of Utilization Rates

We calculated monthly population utilization rates and adjusted for managed care enrollment, eligibility status (such as foster care and poverty), race, and Hispanic ethnicity as described above. The numerator for rates was the presence of a claim for service in the calendar month for one of the diagnostic or procedure codes listed above. The denominator for rates was all youths eligible to receive services (in other words, they were currently insured by Medicaid) and of screenable age in the same calendar month. That is, we did not remove youths from the denominator in subsequent study months when they received BH screening in prior study months.

We further adjusted the population utilization rates for seasonality using the Census Bureau algorithm (proc X-11 in SAS) (22,23).

Interrupted Time-Series Analysis

We fit segmented regression models (24–26) for each of the BH utilization rates in the 24 months prior to mandatory screening and 18 months postmandate. The segmented regression models included terms for the change in intercept (immediate-level change), secular trend (overall slope), and change in trend (increase versus decrease in slope postimplementation). We then constructed difference-in-differences models comparing Massachusetts with California to determine whether there were significant changes in outcomes related to the policy.

Results

The demographic characteristics of the Massachusetts and California populations are shown in Table 1. The two states were fairly similar with regard to age and gender distributions but differed substantially by race and eligibility criteria. However, these differences did not affect the almost identical baseline trends in outcome measures.

After reviewing the time-series analyses, we observed that the overall monthly rate of screening stabilized at nine months postmandate. Preliminary data from MassHealth also indicated that achieving a 50% BH screening rate at well-child visits took nine months. Thus we censored the first eight months of observations in the time-series analyses in order to evaluate the impact of the policy when it achieved at least “half strength.” Such censoring has been applied in several previous studies where the analysis excluded the implementation or “phase-in” period (27–30).

Figure 1 shows the rate of BH screening among all eligible youths (including those without ambulatory visits). Use of the screening code rose to about 13 per 1,000 youths per month during the first nine months after the screening mandate was implemented. The rate of BH screening remained relatively stable over the next ten months, averaging about 11 per 1,000 youths per month. During the same time frame, there was no concomitant rise in screening in the California data.

Figure 2 shows the monthly rate of any BH-related outpatient service utilization. The adjusted rate was about 35 per 1,000 youths per month in the two years prior to the mandate, with a slightly increasing secular trend (.8 per 1,000, p=.03). After the phase-in period for screening, rates of outpatient use increased dramatically in the fall of 2008, to about 50 per 1,000 youths per month. More specifically, it began to rise during the phase-in period and then remained stable thereafter. The trend in utilization increased at about 4.5 per 1,000 youths per month (p<.001). No increase was seen in California. Estimates for policy (immediate change), time (secular trend), and time after the mandate (postmandate trend) resulting from the regression models are shown in Table 2.

Figure 3 shows the rate of BH-related ED visits over the study period. BH-related ED use averaged about 17 per 1,000 youths per month in the premandate period. Once the mandate took effect, rates of ED use increased at a rate of 4.8 per thousand per month (p<.001). However, the difference-in-differences analysis revealed that a similar increase occurred in California during the same period.

BH-related hospitalizations averaged about .3 per 1,000 youths per month in the premandate period. Beginning in the fall of 2008, inpatient stays with a BH diagnosis began to increase by about .1 per 1,000 youths per month, with an expected rate of about .4 per 1,000 by June 2009. The difference-in-differences model did not show any statistically significant change in inpatient stays in Massachusetts compared with California. The monthly rate of any psychotropic medication use was about 126 per 1,000 in the premandate period, with a decreasing secular trend of 2.2 per 1,000 per month (p=.02). After the implementation period, the trend reversed, with the rate increasing at about 9.7 per 1,000 per month compared with the California control (p=.01). Again, the difference-in-differences interrupted time-series analysis did not show statistically significant change for psychotropic medication use.

Discussion

The Massachusetts policy mandate to screen all children and youths for BH problems is the first of its kind in the nation (21). In examining the policy’s impact on service utilization rates compared with those of California, we found that screening (use of the 96110 code) significantly increased in Massachusetts and was associated with a concomitant increase in rates of BH-related outpatient services. However, when assessed in comparison with trends in California, we found that BH hospitalizations, BH-related ED utilization, and psychotropic medication use in Massachusetts showed no evidence of being influenced by the new policy.

It is important to put our findings into context. During the study period, BH-related ED visits for children rose (31,32). Evidence also suggests that psychotropic medication use and outpatient mental health treatment for children increased between the periods 1996–1998 and 2010–2012 (33). Inpatient hospitalizations related to BH conditions for children also rose between 2006 and 2011 (32) Other changes occurred in the delivery of child and adolescent mental health services, including the black-box warning on the use of antidepressants (30), rising rates of specific disorders (specifically, bipolar disorder) (34), and heightened interest in identifying and treating BH issues in pediatric offices (35). However, none were concurrent with the changes we noted postpolicy in Massachusetts.

The increase in screening rates after the mandate has been documented elsewhere (10,11), but the investigators for the previous study did not control for external factors, nor did they compare trends in other states. It is possible that we missed screening conducted without the recommended code; however, a prepolicy Massachusetts Medicaid chart review found that formal screening tools were used in only 4% of well-child visits (36). In the comparison state, California, screening may have occurred as part of Early and Periodic Screening, Diagnosis and Treatment visits without specific reimbursement for identified screening codes, but trends did not change over time. Our analysis provides evidence that mandates—coupled with reimbursement—can rapidly increase BH screening rates in pediatric practice and increase the utilization rates of outpatient BH services.

Several questions remain unanswered. First, does the increased BH utilization translate to better or more appropriate care? Further investigation is needed to ascertain treatment quality and outcomes. Second, does early entry into treatment ultimately mitigate the need for more expensive care, such as ED visits and hospitalizations? Although we did not see a significant change in these outcomes, our time frame for evaluation was limited because Massachusetts added programs after 18 months, which would confound any longer-term analysis of outcomes related strictly to the screening policy. Further work is needed to determine the relationship between screening, treatment, and inpatient utilization.

Several other findings merit discussion. For example, despite an increase in outpatient care rates, we saw no concomitant increase in psychotropic medication utilization rates. This finding is consistent with prior studies of screening programs (21,37). It is important to note that BH-related outpatient services include visits delivered by BH professionals as well as primary care clinicians. It is possible that clinicians initially used first-line–recommended “talk therapy” options (2,38–41) rather than psychopharmacologic interventions or that visits were likely to be for BH issues encountered in primary care (adjustment reactions, depression, for example) rather than more severe mental illness. Prior research indicates that newly identified children were more likely to have internalizing than externalizing symptoms (8). This finding may also reflect recent concern regarding overmedication of children and Food and Drug Administration black-box warnings among pediatricians (30). Further research would be needed to examine specific diagnoses and whether psychotropic medication would have been indicated.

In addition, although both states noted an increase in BH-related ED visit rates, we did not see a significant difference between the states. This is contrary to a recent study of a single system in Massachusetts where a combination of screening and colocated primary and BH care led to an increase in BH-related ED visit rates (42). However, that study did not use a comparison state and focused on the impact of screening and colocated care, which may have contributed to increased ED use for reasons cited in the article.

Our study had a number of limitations. First, we lacked information on BH need and clinical outcomes at the population level. The use of an interrupted time-series approach assumed that no cointerventions occurred simultaneously with the policy of interest. However, our discussions with Massachusetts program staff and investigation of relevant documents did not identify any other policy changes in the same period. In addition, it is important to note that this study did not examine the impact of the policy on individual children identified as having BH issues or on children who received screening. It was intended to examine the population-level impact of the screening policy on utilization rates. Our interrupted time-series design explicitly controlled for secular trends in two states and was thus a strong quasi-experimental design for examining the impact of policy on rates of service utilization (43).

Conclusions

The goal of BH screening in primary care is to identify previously unidentified children with BH issues and provide earlier treatment options. As in other screening programs, the hope is to ultimately deter longer-term negative outcomes. Although we could not measure the long-term outcomes (recovery rates) of the Massachusetts policy, the policy increased screening at preventive care visits. The increase in screening, in turn, was associated with an increase in use of outpatient BH care in the Medicaid-insured population of youths in Massachusetts.