Adoption of a Blended Training Curriculum for ACT in New York State

Although assertive community treatment (ACT) is a widely used team-based evidence-based practice (EBP), scant evidence informs how to best train ACT teams. Nevertheless, training ranks among the most crucial components of sustainable EBPs (1). Expert consensus and the experience of states in implementing ACT teams provide some general concepts for training. For example, effective training can provide program leadership with a clear concept of the model’s goals and principles (2). Others emphasize that ACT teams should be trained in areas such as professional values, attitudes, and beliefs to strengthen team members’ ability to interact with consumers in various stages of illness and their families (3). Experiential learning and supervision are also viewed as vital concepts in training ACT teams (4). In addition, training teams together can help enhance practitioners’ understanding of the model and encourage mutual problem solving (5).

Specific examples of training programs for ACT teams are also limited. The ACT Training Institute at the University of Illinois at Chicago trained state ACT teams by using traditional didactics, mentoring by experienced ACT team members, and observation of veteran teams (6). Measures of changes in knowledge demonstrated significant increases in posttest scores, but the overall average posttest scores remained low at 60% of questions answered correctly (6). The Washington Institute for Mental Health Research and Training in Washington State trained ACT teams by using several days of in-person didactics on core ACT principles, direct observation of another state’s ACT teams, one booster training session, and ongoing phone and on-site support and technical assistance. Contracts required teams to meet specific fidelity scale items; preliminary analyses suggested that these requirements enhanced fidelity (7). The authors described the provision of ongoing training and technical assistance as playing a key role in development of high-fidelity ACT teams (7).

In this column, we describe the training provided by the ACT Institute in New York which utilizes a “blended learning” approach, with both face-to-face and distance learning strategies. Because the New York State Office of Mental Health (NYSOMH) requires training for licensing, we also report on tracking methods and training uptake across the state and discuss the role of the state in driving adoption of training activities. Finally, we discuss how New York State and the ACT Institute use data to guide training efforts and improve outcomes.

ACT in New York State

The NYSOMH first implemented ACT in the early 1990s. The Division of Community Care coordinated training in ACT, which included a four-day, in-person training session for all team members. In these earlier years, teams operated with few guidelines and lacked uniform standards. In 2000, outside consultants and experts in ACT implementation completed a fidelity review that created a foundation allowing NYSOMH to develop standards of care and establish ACT as a licensed model. It also enabled the model to be approved in the New York State Medicaid State Plan, with funding from a combination of Medicaid dollars (50% federal share and 50% state and local) and state aid for people who were not Medicaid eligible. The need for training to enhance fidelity became evident, leading to the establishment of the ACT Institute in 2003. Initially housed in a community-based organization, the ACT Institute moved to the Center for Practice Innovations (CPI) at the New York State Psychiatric Institute (NYSPI) in 2007. By then, the number of ACT teams had climbed to 79. Scaling up a complex intervention presented a challenge to develop and provide ongoing training that could reach teams across a large and geographically diverse state to meet the needs of new and established teams with significant rates of staff turnover.

ACT curriculum and training.

In-house clinical experts and implementation specialists at CPI provide ACT Institute training. Across all modalities, the training curriculum incorporates the overarching concepts of recovery-oriented values and practice, emphasizing a strengths-based perspective. The ACT Institute training curriculum employs blended learning, consisting of two one-day, face-to-face classroom training sessions conducted regularly in the state’s five regions on ACT core principles and person-centered treatment planning, plus seven 45-minute online learning modules, which include tests of knowledge that must be passed at 80% or greater in order for the learner to receive credit for the training. Clinicians complete the training twice, once within the first six months of ACT employment and again after five years. Combining experienced and new clinicians from all disciplines promotes interprofessional collaboration, which is critical to a successful team. The online training is structured to facilitate knowledge acquisition about the theory of the ACT model and key practice theories by using the “flipped classroom” model of teaching—a form of blended learning in which trainees learn new content by watching online teaching modules individually and then by processing the information with the trainer during face-to-face training (8). When a trainee acquires the knowledge disseminated online, the face-to-face training can offer personalized and targeted guidance, interaction, and skill building through problem-based learning and enhance critical thinking while also fostering a learning community.

The ACT Institute also provides a variety of additional educational and skill-building opportunities. Monthly podcasts, during which CPI staff conduct interviews with experts and discuss critical ACT content areas, are designed to reinforce theoretical knowledge of ACT and practice application. In addition, interactive one-hour webinars impart didactic material in key practice areas; the webinars are recorded and can be viewed by teams at any time. Finally, on-site training and technical assistance are provided for the whole team when a practice area has been identified by the state or the team leader as not meeting fidelity requirements.

Learning management system.

To track training and implementation, CPI developed an online learning management system (LMS) that gives users access to training modules, event registration, and resource libraries. The LMS enables CPI to track participants’ module completion and attendance at training and allows participants to evaluate training sessions for all CPI initiatives. By means of the LMS, CPI also collects several levels of evaluation from learners: satisfaction with and usefulness of training, knowledge mastery, self-efficacy, and practice change as a result of the training. In addition, the LMS allows program managers to assign training and obtain real-time reports of staff members’ status regarding required training.

Training Uptake

New York State currently has 81 ACT teams. Most teams are centered in the New York City area, but almost half of the counties in the state have ACT teams. Teams in rural areas serve 48 consumers per team, whereas urban teams serve 68 consumers per team. ACT teams employ more than 700 staff members. Daily enrollment is around 5,200 consumers, with teams serving more than 6,000 unique individuals annually.

To be considered fully trained, team members must complete in-person training and all seven online training modules. By December 2015, 74% (565 of 767) of ACT team members were fully trained, up from 60% in December 2014. Individual module completion rates are also high, ranging from 85% to 95%. [More information about training uptake is available in an online supplement to this column.] Despite the high proportion of individual staff members with completed training, fewer teams had completed training (100% of staff trained) because of high staff turnover. Roughly 20 to 30 teams each month experience staff turnover. To provide incentives to teams to fully train staff, in 2015 CPI began to award Certificates of Excellence signed by the NYSOMH commissioner; CPI director; and ACT Institute director, training director, and implementation specialist. These teams are acknowledged at the monthly ACT team leader meetings and in monthly ACT newsletters.

CPI monitors requests for on-site consultations, which are performed by CPI staff and implementation specialists. In 2015, CPI performed 40 on-site consultations. Most of the consultations focus on training in specific skills, such as using person-centered language, creating high-quality treatment plans, and fostering team accountability through community visits. In addition, team leaders requested specific consultations to support their development of a continuous quality improvement (CQI) project. Feedback from team leaders indicated that they found the consultations to be useful in connecting ACT teams to valuable resources and helping them set attainable and measurable goals, prioritize tasks, and improve communication among staff and between staff and consumers.

The state’s role in enhancing adoption of training activities.

CPI has utilized a variety of interventions to enhance adoption of training activities, and interventions by NSYOMH have had a strong impact. In addition to providing direct funding for the development of a training curriculum for ACT teams at the ACT Institute, NYSOMH has instituted practice mandates (that is, required training for licensing) and state-run audits of training uptake. In September 2012, NYSOMH sent a letter to agencies and teams reinforcing the mandated nature of the training. After this intervention, CPI observed a sharp increase in training uptake, which then continued to rise at a steady rate [see online supplement]. Interventions of this nature by state agencies have been shown to increase an organization’s predisposition to adopt new practices, particularly in the early stages of adoption (9). CPI continues to encourage teams to complete training by sending monthly e-mail reminders and reports to teams.

Use of data to drive training uptake and enhance training outcomes.

Moving forward, the ACT Institute plans to enhance its use of data to further drive training uptake and evaluate training outcomes. The addition of pre- and posttraining knowledge tests for the in-person training sessions will provide data on knowledge acquisition by team members and help guide training efforts in areas of lower acquisition. In spring 2015, ACT teams were introduced to the CQI process to provide a framework for quality improvement, with a well-outlined, stage-based strategy for monitoring and improving team operation and practice through a data-driven decision-making process that requires active participation of all team members. The ACT Institute will continue to provide training for teams in the use of CQI to strengthen the quality of services provided to consumers.

To maximize the data-oriented tools available to assist teams in identifying practice areas in need of improvement and to help CPI in developing targeted training, the ACT Institute adapted the Tool for Measurement of Assertive Community Treatment, a validated fidelity scale (10), into an easy-to-use, online self-assessment survey to be completed by the team. Teams that complete an online self-assessment of fidelity receive a detailed feedback report from the ACT Institute, which highlights areas of high fidelity and addresses any areas for improvement. In order to better target consultations, the ACT Institute is using an outcome matrix (ACT Profile) made available from the NYSOMH that provides an individual team score on key indicators, such as housing, hospitalizations, emergency department visits, substance use, treatment adherence, and length of stay. The ACT Profile provides data to teams quarterly so that they can benchmark their scores against those in their region and statewide.

Conclusions

The ACT Institute has created a blended learning curriculum that has provided training for more than 700 ACT team members across a large and geographically diverse state. By offering a variety of incentives and requirements for training, in collaboration with the NYSOMH, training completion rates for ACT team members have risen steadily, and in 2015 nearly three-quarters of all staff were fully trained. The ACT Institute has harnessed the use of data to drive training uptake and plans to continue to enhance this effort to improve training outcomes. State mental health agencies, in collaboration with academic institutions, can build innovative and effective training programs for ACT teams in order to provide high-fidelity ACT to mental health consumers in an ever-changing health care landscape.