The Current State of Behavioral Health Quality Measures: Where Are the Gaps?

Valid and reliable clinical quality measures are integral to implementing and evaluating ongoing health care reforms (1). Reporting of quality measures and benchmarking performance are components of incentive programs for the use of electronic health records and for reforms in delivery systems, such as health homes, advanced primary care, and accountable care organizations (2). Furthermore, quality measures are critical for monitoring changes in the delivery of care in response to Medicaid eligibility expansions and the implementation of health care exchanges, both of which will likely increase access to care for individuals with behavioral health conditions (3). Given that many of today’s delivery system and financing reforms focus on populations at high risk of mental and substance use disorders, there is particular need for quality measures that assess behavioral health care.

To guide the conceptualization of measures for behavioral health care, the Substance Abuse and Mental Health Services Administration (SAMHSA) developed the National Behavioral Health Quality Framework (NBHQF) (4). The framework, modeled from the National Quality Strategy, prioritizes prevention, treatment, and recovery goals at a variety of levels that range from health systems to providers. In addition, it lays the groundwork for developing and applying measures that may be used to monitor and improve the quality of care at the provider, health plan, or population level for individuals with behavioral health conditions. However, it is unclear whether measures are available to address the wide range of needs of individuals with behavioral health conditions as described in the NBHQF. Furthermore, because measures are increasingly used in national and state public reporting programs and as the basis for financial incentives, there is a need for information on the extent to which measures have demonstrated reliability and validity.

Currently, the National Quality Forum (NQF) endorses health care quality measures through its consensus development process by using a multistakeholder panel that independently reviews a given measure to determine its importance, scientific acceptability (reliability and validity), feasibility (data availability and reporting burden), and usability for quality improvement. In addition, NQF promotes alignment of measures and reduction of reporting burden via the creation of a portfolio of fully harmonized quality measures.

As part of a larger project to develop behavioral health quality measures, we conducted a review of existing quality measures applicable to behavioral health. The review sought to determine to what extent existing measures address a range of behavioral health conditions and how these measures align with NBHQF priority areas. In addition, we determined the number of quality measures that are NQF endorsed and used in public reporting programs.

Methods

The review identified measures related to behavioral health care in the three most comprehensive databases of measures: the National Quality Measures Clearinghouse (www.qualitymeasures.ahrq.gov), NQF (www.qualityforum.org), and the online inventory maintained by the Center for Quality Assessment in Mental Health (www.cqaimh.org). The search included measures in these inventories up to March 2015. For each data source, key terms and phrases were used to identify measures (available from the authors on request). Search terms were restricted to behavioral health conditions that included only mental disorders and substance use disorders, substance abuse, or substance dependence. The review did not include measures related to dementia or measures for general medical conditions (some of which could be relevant for individuals with comorbid conditions).

To identify any additional measures not captured by the above sources or any measures under development by federal agencies, we interviewed representatives from SAMHSA, the Agency for Healthcare Research and Quality (AHRQ), the National Institute of Mental Health, and the Office of the Assistant Secretary for Planning and Evaluation (ASPE) of the U.S. Department of Health and Human Services. In addition, we reviewed measures developed by the Veterans Health Administration (5) and the National Association of State Mental Health Program Directors (6).

Measures were categorized by steward, description, numerator, denominator, exclusions, NQF identification number (if any), data source, level of specification (for example, health plan, hospital, or provider), type of measure (structure, process, or outcome), behavioral health condition, and age range of the relevant population. Measures were also assigned to an NBHQF priority area (that is, effective; patient-, family-, or community-centered; coordinated; healthy living; safe; and affordable-accessible) and domains and subdomains created within the framework to provide greater specificity. The additional domains and subdomains were based on a categorization scheme developed for the International Initiative for Mental Health Leadership project, which conducted a review of international initiatives in mental health quality measurement (7).

In addition, the review identified whether measures are used in the following selected federal and state reporting programs: adult Medicaid core set (8) and child Medicaid core set (9) (reporting by state Medicaid and Children’s Health Insurance Programs), Medicaid health home core set (10) (reporting by health home providers), star ratings for Medicare Advantage and prescription drug plans (11) (reporting by plans for incentive payments), Physician Quality Reporting System (12) (reporting by providers for incentive payments), Medicare and Medicaid Electronic Health Record (EHR) Incentive Programs or meaningful use (13) (incentive payments to organizations that use EHRs to improve patient care), and Centers for Medicare and Medicaid Services (CMS) Inpatient Psychiatric Facility Quality Reporting Program (14) (reporting by inpatient psychiatric facilities).

Results

The review identified 510 measures that address all of the NBHQF priority areas (Table 1). The largest number of available measures are for effective treatment (N=147) and healthy living (N=129). Some topics have many measures, which often address the same concept or variations on a theme. For example, we identified 25 measures that address follow-up after hospital discharge. They focus on various populations or subpopulations (for example, follow-up after hospitalization for mental illness versus follow-up after hospitalization for schizophrenia) or assess the same type of event but specify different time frames (for example, readmission to facilities within 14 versus 30 days).

Only 10% of the 510 behavioral health measures (N=53) are endorsed by NQF. The largest numbers of endorsed measures are in the “screening and assessment” subdomain of the healthy living NBHQF priority area. Of the 98 measures identified, 13 are endorsed by NQF, which includes several measures that consider screening for general medical conditions (for example, diabetes screening for people with schizophrenia or body mass screening for those with serious mental illness). Although some areas have a number of measures, few are endorsed. For example, although we found 60 measures addressing safety, only two are NQF endorsed. Similarly, in the affordable-accessible priority area, we found 64 total measures and one NQF-endorsed measure.

Most measures found were process based (72%), and nearly one-third (32%) of the measures broadly defined mental health or substance use populations in the denominator rather than a single condition or diagnostic group (Table 2). In addition, most measures are specified for providers or ambulatory care (49%) and use administrative claims data (89%). Among the measures focused on a single disorder, the largest number (22%) focus on depression.

Selected quality reporting programs use a total of 28 (5%) unique measures. As summarized in Table 3, some reporting programs include relatively few behavioral health measures. The Medicare and Medicaid EHR Incentive Program for Eligible Professionals (also known as “meaningful use” and currently in stage 2) includes 11 measures. We found no behavioral health measures in the Medicare and Medicaid EHR Incentive Program for Eligible Hospitals and Critical Access Hospitals. The child Medicaid core set includes four behavioral health measures. As a result of efforts to align reporting requirements, a number of measures are used in more than one program, with antidepressant medication management (NQF 0105) and follow-up after hospitalization for mental illness (NQF 0576) used in four and five programs, respectively.

Eleven of the 28 total behavioral health measures used in public reporting programs focus on depression care. Depression remission at 12 months (NQF 0710) is the only NQF-endorsed behavioral health outcome measure used in any of the national reporting programs. The other 27 measures focus on screening-assessment, medication management, coordination, restraint, and seclusion.

Discussion

Even though many behavioral health quality measures exist, many measures address similar content areas, and relatively few measures have received national endorsement or are used in major quality reporting programs. Although several NBHQF priority areas have few measures, having more measures in a particular domain is not necessarily better. Some domains have many measures, but it is unclear which of these measures have the strongest potential to improve the quality of care. This underscores the importance of obtaining stronger consensus on well-validated measures that are most useful for improving quality in particular areas where quality issues persist. Nonetheless, opportunities for measure development exist for combined pharmacotherapy and psychosocial treatments, family or caregiver involvement in care, medication errors, and injuries, because no NQF-endorsed measures were found in these topic areas.

This review identified several other limitations associated with existing measures. First, most measures rely solely or in part on administrative or claims data. Even though measures based on claims data are less burdensome to implement than measures that require data collection from medical records, they may not provide the clinical detail sufficient to guide quality improvement. In addition, measures that rely solely on claims data may not lend themselves to comparisons across provider organizations, health plans, or states because of the use of different billing codes for similar behavioral health services (15). Second, few NQF-endorsed measures rely on data collected from EHRs. Even though such measures could provide rich clinical detail, the lack of adoption of EHRs among behavioral health providers limits their use (16). To realize the potential of EHR measures in behavioral health will require the continued development of data systems and infrastructure to support reporting on such measures and using them for quality improvement. Furthermore, data sharing between primary care and behavioral health providers and managed care organizations will be needed for robust measurement and streamlined data collection and to accurately measure the delivery of care.

Most measures focus on processes of care rather than on structures or outcomes, probably because structure and outcome measures are particularly difficult to specify and may lack evidence. At the same time, even though process measures may help improve clinical processes, it is often difficult to link processes to outcomes. Therefore, future measurement development and implementation efforts may wish to focus directly on outcomes, although challenges such as appropriate risk adjustment may impede such efforts. Structural measures may also have value to help guide the field in the implementation of evidence-based practices, but future work is needed to understand what structural aspects of care are associated with the implementation and outcomes of evidence-based care (17).

Several ongoing measure development projects are likely to produce new measures relevant to the NBHQF. Under the Child Health Insurance Program Reauthorization Act, AHRQ and CMS have designated seven Centers of Excellence in Pediatric Quality Measurement. Two of the centers are developing behavioral health measures, including measures related to adolescent depression and attention-deficit hyperactivity disorder. In addition, several behavioral health topics are proposed for future assignment to the centers. Other federal projects include behavioral health measure development through a partnership between the ASPE and SAMHSA. This effort has developed measures for states and health plans that focus on screening, follow-up, and monitoring of chronic general medical conditions among people with serious mental illness and alcohol and other drug dependence. Screening measures specifically for the CMS Inpatient Psychiatric Facility Quality Reporting Program are also being developed by CMS and ASPE. There is at least one federal grant from the National Institute on Alcohol Abuse and Alcoholism that focuses on quality measurement. ASPE, in collaboration with the National Institute of Mental Health, is developing quality measures for posttraumatic stress disorder.

This review has a few important limitations. The scope of the review was largely limited to three databases. Even though the databases are national in scope and widely used, they may not include measures applied in local efforts or research (for example, Medicaid or state behavioral health agencies). In addition, the field is dynamic and rapidly changing, such that the data represent a point-in-time perspective. To our knowledge, this review is the most current aggregation of behavioral health measures in the literature.

Conclusions

Given that health care reforms will likely influence the organization and financing of behavioral health care, strong quality measures will be a pivotal component of efforts to monitor the delivery of care and identify opportunities for quality improvement. Despite the existence of a wide array of behavioral health measures, few have received national endorsement or been adopted by reporting programs. Future measure development and implementation efforts should focus on identifying the strongest measures within each domain of the NBHQF and filling gaps where existing measures are insufficient.