A Comparison of Assertive Community Treatment Fidelity Measures and Patient-Centered Medical Home Standards

Assertive community treatment (ACT) teams provide intensive support to persons with serious and persistent mental illness who demonstrate difficulty engaging in care (1). ACT teams combine the services of a psychiatrist, a psychiatric nurse, and a variety of professionals who treat mental illness and substance use disorders for delivery in community-based settings. They address a range of consumers’ biopsychosocial needs while seeking to reduce fragmentation of care, maximize continuity, and increase community integration. ACT teams are cost-effective (2) and improve behavioral health outcomes, including reduced psychiatric hospitalizations, increased housing stability, greater retention in treatment, improved consumer and family satisfaction (3), and reduced homelessness (4). ACT teams can effectively implement new evidence-based practices, such as integrated treatment of co-occurring disorders (5,6), illness management and recovery (7,8), and supported employment (9,10).

As is true of fidelity tools for other evidence-based practices, ACT fidelity measures were developed to counteract the reduced effectiveness associated with deviation from the original program model (11,12), as well as to better ensure replication and prevent program drift (13). National standards for ACT teams (14) were established to assist widespread implementation (15,16). Recently, a newer fidelity measure incorporating a more comprehensive set of standards has been piloted in ten states and three countries, with the primary purpose of guiding ongoing quality improvement (13,17,18).

Clients enrolled in ACT tend to engage in unhealthy behaviors, such as smoking, poor diet, and lack of physical activity, that lead to diabetes, dyslipidemias, heart disease, and other common chronic general medical problems. These illnesses can have a severe impact on functioning, quality of life, and health (19–22) and may be negatively associated with the symptom burden of psychiatric disorders (23). National program standards have charged ACT with helping clients obtain general medical care (14). Consequently, many teams have incorporated elements of general medical care management, such as enhanced referral to primary care clinics (24).

At the same time, the patient-centered medical home (PCMH) has evolved into an important service delivery model in primary care. According to this model, the delivery of primary care is team based, evidence driven, accessible, continuous, comprehensive, and patient centered (25). The National Committee for Quality Assurance (NCQA) developed fidelity criteria for accreditation of PCMHs that involve a rubric for scoring the breadth of services offered (26). Some practices that recognize NCQA PCMH standards are partaking in novel reimbursement strategies with the Centers for Medicare and Medicaid (27,28).

There are large-scale pilot efforts to integrate primary care into mental health settings but little attention toward expanding the general medical role of ACT teams. Given that both ACT and the medical home have accountability criteria, a comparison of these standards may offer insight into the similarities and differences between the two methods of service delivery. The purpose of this analysis was to systematically compare NCQA standards defining a PCMH to ACT fidelity standards. The goal was to identify the extent to which the inherent infrastructure of high-fidelity ACT teams overlaps with PCMH criteria.

Methods

Medical home accreditation

In 2011, the NCQA revised PCMH standards and scoring guidelines that are used to determine whether programs qualify as medical homes. They are divided into six standards subdivided into 28 major elements comprising 152 individual factors (26). Each element is assigned from 2 to 6 points (weighted), for a maximum score of 100. Practices scoring between 35 and 59 are recognized as a level 1 medical home; between 60 and 84, level 2, and 85 or higher, level 3. Additionally, each standard has one “must-pass” element. A practice must satisfy 50% of the factors within that element to pass.

ACT fidelity measures and program standards

There are two ACT team fidelity measures and one set of program standards. The Dartmouth Assertive Community Treatment Scale (DACTS) is the original national standard measure of fidelity of ACT (18). ACT teams are scored on 28 items by using a 5-point behaviorally anchored scale (12,29). Similarly, the Tool for Measurement of Assertive Community Treatment (TMACT) is a newer fidelity measure comprising 47 items, each also scored on a similar 5-point scale (13). The 2003 National Program Standards for ACT Teams (14) were developed on the basis of the ACT start-up manual (30) and provide the foundation for the evidence-based practices tool kit for ACT developed by the Substance Abuse and Mental Health Services Administration (15,31). The standards include examples of standard program operation. These three indices are the measures employed most commonly to assess ACT teams’ adherence to the ACT model (18).

Comparison of PCMH standards and ACT measures

Two authors (JMC and ERV) independently applied the NCQA medical home standards to the three ACT measures to determine whether the ACT measures met requirements for each NCQA factor. Explicit satisfaction of a NCQA factor was defined a priori as a direct mention in the fidelity measure or program standards. Implicit satisfaction was defined as meeting the spirit of the factor. An example of a factor that meets implicit but not explicit criteria is the NCQA requirement that practices document the provision of general medical advice in the medical record. The TMACT does not require teams to record such data in charts as part of their daily routine, but teams undergoing a TMACT fidelity review must document the occurrence of necessary clinical contacts, thereby meeting the NCQA criterion implicitly. For purposes of analysis, the authors agreed that the process of fidelity review could be considered to implicitly satisfy NCQA standards regarding quality improvement (32). The authors were blinded to the point value and must-pass status of each NCQA element throughout the review process. Items unrelated to ACT teams, including items related to pediatric practice only (N=3), depression screening (N=1), and phenotypic syndrome surveillance (N=1) were discarded, leaving a total of 147 NCQA factors for comparison.

The two authors reconciled differences in determination of factor satisfaction through discussion and mutual agreement. After further comparison, they scored the TMACT again by using the complete protocol rather than the summary scale alone, given that TMACT fidelity audits are usually performed utilizing the entire protocol: the summary scale is only part of the fidelity review process. The authors differed on a maximum of only 26 NCQA factors for any ACT measure. Of factors with disagreement, 64% were up-coded after discussion to indicate the NCQA factor criteria had been satisfied. The two authors had no relationship to the development of ACT or NCQA measures. One author (MMDV) contributed additional expertise regarding proper interpretation of the TMACT once scoring was completed.

Results

Explicit and implicit NCQA factor overlap

The number of NCQA factors with explicit satisfaction by the ACT measures was 23 (16%) for the DACTS, 51 (35%) for the TMACT, and 61 (41%) for the program standards. NCQA factors with implicit satisfaction numbered 47 (32%) for the DACTS, 75 (51%) for the TMACT, and 76 (52%) for the program standards (Table 1). There was a qualitative trend for the NCQA factors to overlap more consistently with the program standards and the TMACT than with the DACTS.

At least one factor in each NCQA standard was satisfied implicitly by all ACT measures. NCQA standards 1 (enhance access and continuity), 3 (plan and manage care), and 4 (provide self-care support and community resources) consistently demonstrated stronger overlap across ACT measures compared with standards 2 (identify and manage patient populations), 5 (track and coordinate care), and 6 (measure and improve performance) (Table 1). NCQA standard 6, pertaining to quality improvement, overlapped more with the TMACT and the program standards than with the DACTS. Because ACT teams hold daily meetings to discuss all patients, all ACT measures had significant overlap for elements regarding the identification of high-risk clients. ACT measures shared much in common with NCQA requirements pertaining to client care management, the provision of referrals to community resources, and coordination with families and the management of care transitions.

ACT measures met none or only one of the factors in elements requiring the use of an electronic medical record (EMR) or other electronic components—such as electronic prescribing, patient identification or management, or other communication—automatically disqualifying 50 NCQA factors. A majority of NCQA standard-2 factors (21 of 34) and one in three standard-5 factors (eight of 25) require the use of an EMR. These standards were among the three with the smallest overlap with NCQA factors, in part because of the significant reliance on EMR. Although there was considerable overlap between ACT measures and the comprehensive health assessment element in standard 2, some factors were not satisfied. No ACT measure mentions advanced care planning—for example, a living will and medical advance directives—although the TMACT supports the implementation of wellness management strategies, such as Wellness Recovery Action Plans and psychiatric advance directives. Because wellness management strategies are inherently nonmedical, they were not counted explicitly or implicitly. No ACT measure explicitly details the processes necessary for testing (laboratory or imaging) or referral coordination and follow-up.

NCQA scoring

Total scores for explicit satisfaction of NCQA factors ranged from 16.75 (DACTS) to 49 (program standards), and scores for implicit satisfaction ranged from 36.25 (DACTS) to 61 (program standards) (Figure 1). Only the DACTS, when explicit scoring criteria were applied, lacked the requisite 35 points to achieve level 1 medical home status. By using a comprehensive EMR for data management, tracking of medical tests, and follow-up and monitoring of chronic disease outcomes, ACT teams could obtain an extra 23 possible points. Adding these points to baseline scores would move all ACT measures into the scoring range for a level 1 medical home and move the TMACT into the level 3 scoring range.

Must-pass elements

High-fidelity ACT teams that adhere to the program standards and the implicit TMACT criteria met the essential must-pass elements of standards 1, 3, 4, and 6 (Table 1). Must-pass elements of standards 2 and 5 had little overlap with ACT measures. All three measures failed to explicitly satisfy the must-pass element in standard 5, although teams adhering to the intent of full TMACT fidelity would likely pass with a score of 100%. This element pertains to the tracking and coordination of referrals.

Discussion

Findings

The analysis demonstrated that high-fidelity ACT teams are equipped with the infrastructure necessary to support the requirements of a medical home. We found a marked overlap between NCQA medical home criteria and the ACT program standards, the TMACT fidelity criteria, and, to a lesser degree, the DACTS. Were it not for scores for two “must-pass” elements related to referral coordination and identification and management of three prevalent chronic disease states, high-fidelity ACT teams would qualify as level 1 or 2 medical homes.

Standards 1, 3, and 4 had the most overlap with ACT fidelity measures and program standards. These standards include factors such as “the practice provides same-day appointments,” “the practice conducts previsit preparations,” and “the practice counsels at least 50% of patients to adopt healthy behaviors.” The domains of enhanced access and continuity, care management, and supported community living are original tenets of ACT (33).

The least amount of overlap was found for NCQA standards 2, 5, and 6, which specify how a practice coordinates care and provides quality improvement for chronic general medical illnesses. ACT teams are uniformly required to coordinate care with emergency departments and admitting facilities and, concordantly, scored highly on standard 5. However, ACT measures lack specific criteria for coordinating consultation for the results of laboratory tests or imaging. As an example, the must-pass element of standard 5 requires that practices assume responsibility for care coordination by establishing communication with other providers, tracking the status of referrals, and documenting referral results in the medical record. The TMACT is the only measure that details this process. Additionally, standard 6 relies heavily on quality improvement processes and self-evaluation. Only the TMACT (in full-fidelity review) and the program standards detailed the processes necessary to demonstrate satisfaction of NCQA quality improvement criterion.

The only overlap of NCQA criteria for standard 2 (identify and manage patient populations) and the ACT measures was for the comprehensive health assessment. The TMACT and the program standards require ACT teams to perform a comprehensive health assessment and charge ACT nurses with the identification and management of comorbid general medical illnesses or appropriate referral of ACT clients with these illnesses. The companion manual for ACT program start-up that supports the program standards includes a worksheet for health assessment that must be performed within 72 hours of enrollment (30). The DACTS does not mention general medical health assessment and, therefore, could satisfy only a fraction of the factors in this element.

ACT measures lacked overlap in the other three elements of NCQA standard 2, which describe systematic management of clinical data. The must-pass element requires practices to proactively identify common chronic diseases and systematically engage in the management of at least three. One of those chronic conditions must be a psychiatric or substance use disorder. Because no ACT measure requires teams to systematically review their population, identify highly prevalent disease states, and manage persons at a population level, none were able to satisfy this must-pass element.

NCQA criteria are intended for primary care practices, with the assumption that the care delivered is medical. As a result, the bulk of NCQA factors refer primarily to the processes of care. These factors are similar to items in the fidelity instruments in their processes of care. ACT teams following the program standards or undergoing TMACT fidelity review could have the necessary infrastructure to serve as a medical home if they were properly equipped to provide direct general medical care.

Although ACT teams may not be equipped to provide clinical advice on urgent general medical conditions around the clock, they are frequently the first point of medical contact for many clients and often are instrumental in triaging general medical concerns and providing transportation to hospitals or clinics. Many ACT providers already support a broad provision of medical services by their teams. Some ACT teams have had success providing general medical care by using peer support and nursing (24). Others have argued that ACT may be underutilized as a means for providing general medical care (21,34). No studies have systematically examined the role of ACT teams in the general medical care of their clients. It remains to be seen whether ACT teams are willing to adapt their services more formally and whether the care they deliver could be translated into improved health outcomes for clients. The exact prevalence of chronic diseases among ACT clients is unknown. However, the prevalence of chronic general medical conditions among patients with serious and persistent mental illness is high, and because all ACT clients have serious mental illnesses, it can be inferred that many ACT clients have comorbid vascular disease risk factors such as hyperlipidemia and hypertension that easily satisfy the NCQA requirements for chronic illnesses (19,35,36).

In a nationwide analysis using the 2008 NCQA PCMH criteria, Hollingsworth and others (37) concluded that 46% of primary care practices would not qualify for PCMH recognition. In results similar to those of our analysis, most deficiencies among solo primary care practices were related to electronic prescribing, tracking of test results, and quality improvement. Primary care systems and ACT teams face similar challenges to PCMH accreditation.

Limitations

There were several limitations to this analysis. We attempted to limit coding bias among the authors who applied the standards by blinding them to the NCQA scoring algorithm and must-pass elements. This analysis did not take into account the practices of ACT teams in vivo. As mentioned previously, many ACT-like programs may lack full implementation of the ACT model. However, a 2010 survey of state mental health program directors demonstrated that 38 of 42 states with ACT programs require monitoring of services with a fidelity instrument (18). Generally, high fidelity is tied to better outcomes across many evidence-based practices, including ACT (11). This analysis is most applicable to programs that adhere closely to the program standards and fidelity measures. However, it may underestimate the degree to which ACT teams that already engage in general medical care could reach medical home accreditation (21,24,38).

Conclusions

High-fidelity ACT teams interested in becoming medical homes should first ensure they have developed systems for elaborate referral follow-up, tracking of laboratory and radiologic tests, and communication. They could then work to proactively identify one or two common general medical problems to monitor and manage. ACT teams could invest in collaborations with local primary care providers or systematic in-service training for their staff to provide the medical care oversight required of a PCMH. Completing these requirements would satisfy all NCQA must-pass elements, and the team would then likely have enough points to qualify as a level 1 PCMH.

Several examples of primary care and mental health collaboration have proven efficacy and affordability and could provide models of mental health and primary care intervention (39,40). Fidelity scales and the program standards did not mention the use of an EMR, a common requirement in NCQA standards. The inclusion of comprehensive EMRs would have no effect on meeting the must-pass criteria, but would simplify many activities required by the NCQA. The addition of absent NCQA must-pass criteria to the TMACT could provide a comprehensive fidelity tool to ACT teams interested in additional medical home accreditation and needs further study.

ACT teams may be interested in adopting medical home principles for several reasons. NCQA criteria more rigorously detail quality improvement exercises and systematic population management than current ACT fidelity measures or the program standards. Adoption of these more rigorous standards may improve traditional ACT outcomes. Adding direct primary care services to an ACT team may reduce the barriers to general medical care commonly encountered by patients with persistent mental illness and overcome gaps that have emerged as states, such as Missouri and Rhode Island, undertake the redesign of an integrated “health home” system (41,42). Finally, as ACT teams struggle to justify expenditures in an era of reduced funding streams, accreditation as a medical home may offer additional sources of support that could foster programmatic development and reduce emergency health care utilization.