Assertive Community Treatment: Facilitators and Barriers to Implementation in Routine Mental Health Settings

Almost a decade ago, the landmark Schizophrenia Patient Outcomes Research Team study ( 1 ) documented a startling gap between recommended services for individuals with severe mental illness and actual practice. Since then, researchers and policy makers have increasingly advocated for the nationwide implementation of evidence-based practices ( 2 , 3 , 4 , 5 ), which are well-defined interventions that have been shown to produce better outcomes for clients than found with alternative treatments or no treatment at all ( 2 ). However, achieving the goal of widespread adoption of evidence-based practices has been hampered by the dearth of knowledge on how new practices can be effectively introduced into routine mental health settings. To address this gap in knowledge, a national project was undertaken to identify barriers and facilitators to the faithful implementation of a range of evidence-based practices. In this study, we report barriers and facilitators for assertive community treatment, a widely studied evidence-based practice for adults with severe mental illness ( 6 ).

The assertive community treatment model is specifically designed for persons with severe mental illness who have a recent history of psychiatric hospitalizations, criminal justice involvement, homelessness, or substance abuse. The model is based on a team approach, a low staff-to-client ratio, and the delivery of a comprehensive package of services to clients in the community. A substantial body of research now supports assertive community treatment's effectiveness in reducing psychiatric hospitalizations and improving housing stability ( 7 , 8 ). However, these positive outcomes are at least partly dependent on the degree to which a given assertive community treatment team faithfully implements prescribed elements of the model and thus achieves high fidelity ( 7 , 9 , 10 , 11 , 12 ). Studies have shown that high-fidelity assertive community treatment teams achieve better outcomes for their clients than lower-fidelity teams ( 7 , 11 , 12 , 13 ).

Given the necessity for more widespread availability of evidence-based practices and the critical importance of ensuring high-fidelity implementation, the National Implementing Evidence-Based Practices Project ( 14 ) was launched in 1999 to study the process of implementing high-fidelity evidence-based practices, including assertive community treatment. The first phase of the project focused on creating resource kits to support the implementation of five psychosocial evidence-based practices (assertive community treatment, integrated mental health care and substance abuse treatment for persons with co-occurring disorders, supported employment, illness management and recovery, and family psychoeducation) ( 15 , 16 ). The second phase of the project, begun in 2002, sought to identify barriers and facilitators to the implementation of these evidence-based practices.

Because of the complexity of studying an implementation intervention, the project used a mixed-methods design, which combined qualitative techniques with more traditional quantitative approaches. Implementation monitors closely observed the implementation processes over the course of 24 months, administered fidelity scales, interviewed key informants at six-month intervals, and collected a range of other data from team members, team leaders, and middle and upper administrators. Using standard qualitative techniques to distill these data and to cross-check conclusions for validity, we identified key facilitators and barriers for assertive community treatment implementation in 13 programs across two states.

Methods

Team sample

Assertive community treatment teams were the unit of analysis in this study. Thirteen agencies from two states were selected by state mental health authorities on the basis of each agency's interest in and readiness for assertive community treatment implementation. Teams were situated in rural (one team), suburban (three teams), and urban (nine teams) environments; ten were operated by community mental health centers and three by hospital providers. Institutional review board approval was obtained within each state and by the coordinating center for the overall project.

Procedures

Intervention. The intervention has been described in detail by McHugo and colleagues ( 17 ). In brief, implementation took place over a two-year period, from 2002–2003 to 2004–2005. Each program was assigned a consultant-trainer. In the first year, teams received intensive two-day training, monthly on-site visits, and periodic communication by e-mail and phone from the consultant-trainer. The consultant-trainer made less frequent visits and contacts in the second year, and the consultation was gradually phased out between months 18 and 24.

Data collection. An implementation monitor assigned to each site was responsible for data collection. Standardized instructions guided all aspects of data collection and ensured rigor and comparability across sites. Site visits took place monthly and included observations of all aspects of team functioning, including the morning meeting and community visits with clients. Implementation monitors wrote field notes for each site visit and conducted structured interviews with the team leader and the consultant-trainer at six-month intervals and with center directors and senior administrators at baseline and 24 months. Interviews were tape-recorded and transcribed verbatim.

Implementation fidelity

Fidelity was assessed at six-month intervals (baseline and at six, 12, 18, and 24 months) with the 28-item Dartmouth Assertive Community Treatment Scale (DACTS) ( 18 ), which has been widely used in research, program evaluation, and policy making and has shown good psychometric properties, including predictive and discriminant validity ( 19 ). The implementation monitor and the consultant-trainer conducted and scored these assessments together. Each item is rated on a 5-point behavior scale anchored from 1, indicating no implementation, to 5, indicating full implementation. We used the average score for all items as our index of implementation success.

Qualitative analysis plan

Through review of the relevant literature and consultation with experts, we first identified 26 dimensions constituting five broad domains hypothesized to be important to the implementation process. The five domains were prioritization, leadership, workforce, workflow, and reinforcement. Data were coded with Atlas.ti, a qualitative software package ( 20 ). Dimensional displays were created to summarize barriers and facilitators associated with each code, including impact ratings of each dimension (from 2, high facilitator, to -2, high barrier). Consistent with recommendations by Miles and Huberman ( 21 ), this method distills the large amounts of data generated by a multisite qualitative study into manageable chunks. In addition, implementation monitors wrote detailed and comprehensive site reports, using a standard format.

To further distill the essential barriers and facilitators across the 13 sites, the first and second authors independently analyzed each site report. This form of multiple coding is considered a robust check and balance on rigor in qualitative research ( 22 ). In these analyses the authors moved back and forth in levels of specificity and abstraction, extrapolating from specific examples to broader themes and then from broad themes to specific examples. This approach allowed us to best capture the pooled data but also ensured that these broader themes accurately reflected concrete examples in the data.

Through this iterative process, we sought to identify the three most prominent barriers and facilitators for each site. We defined prominence as the themes that were labeled with a 2 or a -2 in the dimension display and that co-occurred in the synopsis; that were described in strong, forceful language by the implementation monitor; and that best accounted for the pooled data at that site. The first two authors then compared findings to adjudicate any differences. A high degree of agreement was observed, with almost identical determinations of barriers and facilitators for a number of sites. We then examined the top barriers and facilitators across sites and identified larger themes (hereby referred to as dimensions). We identified dimensions that were most prevalent across sites and appeared to have the most impact within sites.

Results

Overall, the implementation project supported the achievement of moderately high levels of assertive community treatment fidelity, with scores as follows (mean±SD): baseline, 2.91±1.05; six months, 4.02±.34; 12 months, 4.26±.17; 18 months, 4.22±.22; and 24 months, 4.18±.28. Comparisons of overall fidelity, when aggregated from six to 24 months, between state 1 (4.14±.16) and state 2 (4.25±.15) yielded small and nonsignificant differences. In qualitative analyses, five dimensions emerged as principal facilitators, barriers, or both ( Table 1 ). These dimensions had a complex influence. Most dimensions were both a barrier and a facilitator, often at the same site (for example, leadership showed mixed effects at almost all sites). Table 2 summarizes prominent barriers and facilitators at six lower-fidelity sites and seven higher-fidelity sites.

Enlarge table

Enlarge table

State-level facilitators and barriers

State mental health authority. In both states, the mental health authority had a licensed program category for assertive community treatment and provided comprehensive supports for initial and sustained implementation. These included financial supports, such as Medicaid reimbursement and funding for start-up costs, and various technical supports, such as extensive training regimens, ongoing technical assistance and consultation, and fidelity monitoring. Given the complexity of the assertive community treatment model and its departure from traditional modes of care, these system-level supports appeared to be crucial to initial and sustained implementation, although it should be noted that we lacked a control group of teams with no system-level support. Nevertheless, we could compare the specific nature of system-level supports across the two states. This allowed us to draw some tentative conclusions about the impact of different policies on implementation, as well as broader conclusions based on findings from both states.

Financing. Both states developed Medicaid billing mechanisms and provided start-up funding. The Medicaid billing structure was particularly important because it made the model far more economically viable and attractive to providers. However, the method for determining reimbursement differed considerably between states. In state 1 the case rate reimbursement was calculated on a daily basis, depending on whether a physician or licensed psychologist was present at the daily team meeting, whereas in state 2, reimbursement was calculated on a monthly basis, depending on the number of visits to a given client (full reimbursement for six or more visits per month; partial reimbursement for fewer than six but at least two visits). State 2's billing structure created a disincentive for staff to serve a client beyond this threshold of six visits per month (which compares with the DACTS standard of 12–16 visits per client per month). Indeed, mean±SD aggregate fidelity scores for service frequency were lower and less variable in state 2 (2.70±.37) than in state 1 (3.31±.98).

Licensing processes. A basic component of licensing was fidelity monitoring, which appeared to support fidelity achievement and to protect against "program drift," as indicated by stable and moderately high levels of fidelity throughout the implementation. However, the specific nature of licensing processes varied between states, and these differences appeared to have implications for implementation. State 1 incorporated fidelity elements into its certification standards, but licensing staff used a checklist approach, and standards for those items were less stringent than the fidelity items on which they were based. More detailed assessment and feedback on adherence to fidelity were largely the responsibility of a state-funded technical assistance center that was independent of the state. In contrast, licensing staff in state 2 used a lengthy, detailed, and comprehensive approach to fidelity monitoring, which included the administration of the entire fidelity scale. The close attention paid to fidelity and program functioning appeared to ensure that teams did not significantly diverge from the model. For example, although no differences were observed in overall fidelity between states, some key fidelity elements had higher fidelity scores in state 2 than in state 1, including direct services provided by the team leader (4.50±.73 versus 3.47±.57), level of psychiatrist staffing (4.95±.11 versus 3.97±.18), providing comprehensive services (4.45±.51 versus 4.06±.18), and providing 24-hour crisis services (4.75±.43 versus 4.19±.56).

Despite some potential benefits of close monitoring, the regulatory intensity and complexity of state 2 also appeared to have negative effects on implementation. State and local mental health authorities placed extensive and sometimes duplicative requirements on teams, including outcome reporting, paperwork for clients with outpatient commitment orders, coordination with central referral committees, and program audits (by both state and local authorities in some cases). To address these varied requirements, a number of sites in state 2 reserved staff time (one afternoon a week) specifically for documentation. Such bureaucratic requirements stretched the time resources of the team, were often perceived as onerous, and in some cases heightened tensions between the provider and state and local authorities.

Technical assistance and consultation. Technical assistance and consultation emerged as a facilitator. Two factors appeared important: the personal qualities and skillfulness of the consultant-trainer and the way technical assistance was organized at a system level. Consultant-trainers were most effective when they advocated on behalf of the team and worked in a collaborative and nonauthoritarian spirit with team staff and middle and upper managers. Technical assistance was particularly useful in helping teams to master the complex procedural components of assertive community treatment, such as conducting the morning meeting. However, for teams that had a poor relationship with the consultant-trainer, the consultation was ineffective and appeared to have negative effects. Particularly problematic was the pairing of an experienced team leader with a relatively inexperienced consultant-trainer.

In both states a variety of supports were made available to teams, including extensive training by experts in the field and a consultant-trainer assigned to each team. In state 2, technical assistance was embedded in licensing processes and was intended to support fidelity achievement as well as compliance with state program requirements. In contrast, in state 1 a technical assistance center had greater independence from the state mental health authority and coached rather than policed around licensing standards. Perhaps for this reason, sites viewed technical assistance favorably in state 1.

Organization-level facilitators and barriers

At the level of the organization and the program, the presence or absence of effective leadership was decisive in both higher- and lower-fidelity implementations.

Middle and upper management. Effective middle and upper managers were actively involved in implementing and sustaining assertive community treatment. They showed a clear understanding of the assertive community treatment model, communicated that understanding to agency staff, allocated sufficient resources to the team (enough staff, office space, and cars), and monitored the team's fiscal viability. Managers supervised the team leader and empowered him or her to make independent decisions. Less attentive middle and upper managers posed significant problems for implementation, including a programmatic failure to understand the treatment model, an excessive focus on the bottom line or productivity expectations, poor selection and management of team leaders, and a general reluctance to dedicate resources to essential program functions. On teams where these barriers were present, team members were pressured to increase service intensity and productivity, new hires were not always suitable for the position, internal staff transfers were more common, and the team leader was sometimes burdened with duties outside of the team. On one team, important programming decisions appeared to be made solely on the basis of fiscal concerns and at times undermined fidelity.

Team leadership. Effective team leaders had a thorough understanding of the model, imposed accountability on staff, and promoted morale. They were active in service delivery, empowered team members to make independent decisions, and inspired a general sense of mission. On teams with less effective middle and upper managers, a strong team leader was often able to promote reasonably effective team functioning by advocating for the team and serving as a buffer between the team and agency management. Less effective team leaders tended not to empower staff, organize staff activities effectively, address personnel problems, or manage workload equitably, which had substantial effects on implementation, including lower-intensity services, organizational disarray, and lower morale among staff.

Staffing. Staffing was a critical factor in implementation and was reflective of management's leadership. On better-functioning teams, staff problems were less common and were quickly addressed when they occurred. Practitioners were usually more skilled and dedicated and more likely to embrace a community-based philosophy of treatment. In contrast, staff members on teams with lower performance were less skillful, were less supportive of one another, and tended to have more negative attitudes toward the implementation process. Within-team conflict and staff turnover were much more common on lower-fidelity teams. One low-fidelity team had a succession of team leaders and other staff during the 24-month implementation period. Staffing strengths and problems could often be linked to hiring procedures. For example, one higher-fidelity team vetted prospective employees very carefully. Another higher-fidelity team used a year-end surplus to give a bonus to staff for being on call after hours and instituted regular compensation for work done after standard hours. Actions such as these had clear positive effects on morale. In contrast, one lower-fidelity team made poor hiring decisions, haggled over pay, and required staff to pay high premiums for health benefits. A stalwart member of this team quit after being denied a small raise. Nevertheless, the broader picture suggested that higher-fidelity teams were generally composed of more competent staff, and this overall competence, as opposed to specific actions or skill sets, appeared to be most responsible for implementation success.

Change culture. A number of sites appeared reluctant to adapt the existing mode of services to the assertive community treatment model. This lack of a change culture primarily reflected leadership priorities and was most apparent in state 1. Leadership on lower-fidelity teams often seemed willing to make only incremental changes, shoehorning the assertive community treatment team into other services by, for example, transferring staff from other programs or burdening staff with other responsibilities within the agency. A number of sites referred clients to their own clubhouses or day treatment programs, a practice contrary to the treatment model. Another site was outspoken in its opposition to the supported employment and integrated treatment models for persons with co-occurring disorders. One team leader expressed considerable skepticism about fidelity itself.

Discussion

As part of a national project to better understand how to introduce evidence-based practices into routine mental health settings, this study sought to identify critical barriers and strategies associated with starting up and sustaining assertive community treatment teams. Obstacles to implementing an assertive community treatment team have historically included the cost of starting up a team, the absence of appropriate billing structures, and the complexity of the model ( 23 ). The state mental health authority played a central role in addressing these obstacles, through Medicaid reimbursement frameworks, start-up funding, licensing standards, and technical assistance. At the level of the organization, effective leadership was essential to implementation and was largely reflected in allocation of sufficient resources, promotion of a change culture, and sound personnel practices. These findings are consistent with a growing body of literature that has underscored the central importance of multilevel implementation efforts and "hospitable leadership" ( 24 ).

We found several differences in barriers and facilitators across higher- and lower-fidelity teams, although none were substantial ( Table 2 ). For example, when agency change culture was a barrier to implementation, it was more salient for lower-fidelity teams (N=4; 67%) than higher-fidelity teams (N=1; 14%). Similarly, when staffing was a barrier, it was more salient for lower-fidelity teams (N=3; 50%) than higher-fidelity teams (N=1; 14%). Despite these differences, our method may not have captured other, more subtle differences in the roles of barriers and facilitators. For example, our findings might suggest that leadership, which was examined from multiple perspectives over the course of 24 months, had equal prominence as a barrier and facilitator across higher- and lower-fidelity sites. But in our qualitative impressions of the pooled data, significant leadership problems were far more evident at lower-fidelity sites than at higher-fidelity sites. Given that our method did not capture this pattern directly, we acknowledge this as a limitation of our study.

Because this study was carried out in two states, we were able to compare the impact of different policies for assertive community treatment licensure across the states. The precise nature of fiscal, licensing, and reimbursement policies appeared to have implications for implementation. For example, a threshold billing structure, which was used in state 2, may have had the unintended consequence of promoting less variable visit schedules and may have lowered overall service intensity. These findings suggest that fiscal and billing policies need to be carefully aligned with implementation goals to avoid creating reverse incentives.

Although fidelity monitoring and technical assistance supported fidelity achievement across states, the specific approach to monitoring and technical assistance also appeared to bear on implementation. On balance, the findings suggest a complex picture in which closer and more intensive oversight and regulation in licensing and fidelity monitoring may yield some implementation benefits but also costs in terms of team morale and time resources. In contrast, a somewhat more laissez-faire approach to licensing may pose risks of more pronounced departures from the model in some areas but also places substantially less stress on staff and organizations. States may want to balance each of these concerns in their design of licensing initiatives. We also note that state 2's program standards were more stringent than those of state 1 because they directly reflected the fidelity instrument, and this difference in itself may account for the higher levels of implementation of some program elements. Given that some negative effects of extensive regulation were apparent, this suggests that states and localities should carefully weigh the requirements they place on assertive community treatment teams (such as data reporting and documentation), because assertive community treatment is already an exceptionally demanding model of care for staff and administrators. Findings also indicated that technical assistance that is provided outside of licensing processes was perceived as more helpful, suggesting that independent technical assistance centers can be an important support for implementation.

At the level of the organization and the program itself, the most critical factor was leadership. At different sites a team leader, middle manager, or even a team member had a significant positive impact on implementation, often ameliorating the negative effects of other barriers. On the other hand, the most prominent barrier to successful implementation was ineffective leadership, at the level of the team or the provider organization. Perhaps the clearest example of poor leadership was the failure to provide sufficient resources to the team. Another example was a poor hiring process and the failure to quickly address personnel problems. These barriers were most apparent on lower-fidelity teams, which often saw high turnover in the team leader, nurse, and other positions. In addition, the findings showed that staff added to the team, whether transferred from elsewhere in the organization or hired as new employees, must be competent clinicians who feel comfortable with and committed to a community-based model of treatment. In the absence of effective team leadership or committed staff, the demanding work environment of an assertive community treatment team easily translates into high levels of interpersonal conflict, as was apparent on several lower-fidelity teams.

Finally, several aspects of the agency culture stymied the development of high-fidelity assertive community treatment teams, particularly when leadership was reluctant to modify existing modes of service delivery. This barrier was primarily present in state 1, where it was more typical for participating agencies to modify preexisting treatment teams as opposed to creating new assertive community treatment teams from the ground up. External funding sources may want to scrutinize organizational culture when deciding how to allocate scarce funds and closely consider the advantages and disadvantages of developing new teams versus reforming existing teams, where there may be more entrenched attitudes and habits.

A number of limitations to the study must be considered. First, because we lacked a comparison group of assertive community treatment teams with no system-level support, our conclusions about the role of the state mental health authority merits further study and replication. Second, in our cross-state comparisons, an important caveat is that the two states likely also differed on variables over which we had no control, including aspects of the service context and provider-specific characteristics. Therefore, we emphasize that comparisons across states must be considered as merely suggestive. Third, because our method was primarily qualitative, the sample of teams was necessarily small and thus may not be representative of assertive community treatment teams more generally. However, because we collected longitudinal data over a 24-month period under naturalistic conditions, this study had an unusually high level of ecological validity. Data of this kind provide a window into implementation processes that are simply not accessible by other means. Thus, although we caution that our findings are not based on a representative sample of assertive community treatment teams, we suggest that they nonetheless hold clear implications for how to start up and sustain an assertive community treatment implementation across various service contexts. Furthermore, because the findings concern basic program and organizational functions, they are germane to implementation processes more generally.

Conclusions

Effective implementation of assertive community treatment requires multilevel coordinated efforts on the part of state mental health authorities, senior program administrators, and program staff. Broad policy frameworks, which should include dedicated billing mechanisms, technical assistance centers, and program monitoring, can promote high-fidelity implementation. However, bureaucratic requirements related to program monitoring should be carefully calibrated to manage burden to programs. At the organization level, successful implementation requires committed leadership, allocation of sufficient resources, and careful hiring procedures.

Acknowledgments and disclosures

This research was supported by a grant from the Robert Wood Johnson Foundation, by contract 280-02-8070 from the Center for Mental Health Services at the Substance Abuse and Mental Health Services Administration, and by a gift from the West Family Foundation.

The authors report no competing interests.