The American Psychiatric Association (APA) has updated its Privacy Policy and Terms of Use, including with new information specifically addressed to individuals in the European Economic Area. As described in the Privacy Policy and Terms of Use, this website utilizes cookies, including for the purpose of offering an optimal online experience and services tailored to your preferences.

Please read the entire Privacy Policy and Terms of Use. By closing this message, browsing this website, continuing the navigation, or otherwise continuing to use the APA's websites, you confirm that you understand and accept the terms of the Privacy Policy and Terms of Use, including the utilization of cookies.

×
ArticlesFull Access

Implementing Person-Centered Care Planning: A Randomized Controlled Trial

Published Online:https://doi.org/10.1176/appi.ps.202000361

Abstract

Objective:

Person-centered care is a key quality indicator and central to promoting integrated and recovery-oriented services. Person-centered care planning (PCCP) is a manualized intervention promoting the collaborative cocreation of a recovery-oriented care service plan on the basis of an individual’s most valued life goals. This cluster randomized controlled trial tested the effect of PCCP training on person-centered care delivery in community mental health clinics.

Methods:

Fourteen clinic sites were randomly assigned to receive either PCCP training (N=7; experimental condition) or service planning as usual (N=7; control condition). Data were collected from online surveys, and service plans were completed by 60 provider teams. The Person-Centered Care Planning Assessment Measure was administered via chart review at baseline, 12 months, and 18 months, and surveys were used to measure supervision, implementation leadership, and program type. The main effect was examined with linear mixed-effects regression models, with observations over time.

Results:

Analyses controlling for service user and program characteristics revealed that at 12 months, the group assigned to PCCP training showed significant improvements in delivering person-centered care compared with the control group (b=1.10, SE=0.50, p=0.03). At 18 months, this effect was even more pronounced (b=1.47, SE=0.50, p=0.01), representing a medium-to-large effect size of d=0.71 (95% confidence interval=0.23–1.20).

Conclusions:

These findings indicate that training providers in PCCP increases provider competency in delivering person-centered care. Using an objective measure of person-centered care, the authors show that a comprehensive training strategy can target both the philosophical shift and the technical skills needed to promote client recovery.

HIGHLIGHTS

  • Providers in community mental health clinics trained in person-centered care planning (PCCP) were found to be more competent in the delivery of person-centered care than providers who were not trained in PCCP, and this positive effect increased over time.

  • The findings of this study indicate that by using a train-the-trainer implementation strategy that targets clinical supervisors, effective clinic-wide training sessions in person-centered care can be implemented across programs.

  • The authors used an objective measure of person-centered care by rating service plans that can be used for future research on person-centered care and quality improvement efforts within clinics.

Person-centered care is one of the key quality indicators in health care (1). Defined as “providing care that is respectful of and responsive to individual patient preferences, needs and values” (2), person-centered care has been an organizing principle for mental health systems as they move toward more integrated health care delivery (3). System transformation efforts aimed at creating recovery-oriented services have also embraced a person-centered approach (4). In response, extensive state and local training initiatives for community mental health clinics have emerged, but many providers still struggle to translate the broader principles of person-centered care into routine practice (57).

One intervention that translates the principles of person-centered care into a specified manualized practice is person-centered care planning (PCCP) (8, 9). PCCP provides a framework for the collaborative cocreation of a recovery-oriented care plan that is driven by an individual’s most valued life goals. Providers learn how to elicit and empathize with clients’ subjective experiences as a whole person and how to help their clients identify and articulate their interests, preferences, and personal recovery goals in the service plan. Symptoms and impairments are reframed as barriers to goal attainment, and providers identify short-term, realistic, and measurable objectives. Barriers are explicitly connected to longer term aspirations, and the action network is expanded to include natural supporters as well as professional providers (8).

By using service planning, a process that is common across community mental health programs, organizations can disseminate PCCP clinic wide. Overall, person-centered planning interventions have been found to increase people’s ability to self-manage chronic conditions and improve mental health outcomes (10). Among persons with developmental disabilities, person-centered planning has led to community integration and higher quality of life (11). Within mental health settings, PCCP has led to more client involvement in the care process (12), and in a randomized controlled trial conducted across multiple states, PCCP increased clients’ service engagement and adherence to medication (13).

Despite its promise, some efforts to implement PCCP have not achieved optimal results. The Consolidated Framework for Implementation Research has identified key organizational variables in the inner setting that are potential barriers to the successful adoption of evidence-based practices (14). In the case of PCCP, lack of supervision, high caseloads, shortage of time, staff turnover, and inadequate training have all been identified as barriers to the effective implementation of PCCP (6). Provider buy-in has been hampered by concerns that clients with severe mental illness do not have the capacity to set life goals, that PCCP increases agency liability, and that person-centered plans do not fulfill reimbursement requirements (5). Successful PCCP implementation requires both rigorous training strategies to address provider concerns and leaders who champion PCCP within their agencies to motivate providers, provide needed resources, and align agency procedures (15).

The objective of this cluster randomized controlled trial was to test the effect of PCCP implementation on the delivery of person-centered care in community mental health clinics across two U.S. states. The study used an objective measure of person-centered care to examine whether training in PCCP increased the delivery of person-centered care at the provider level, controlling for program and service user characteristics.

Methods

Sample

The study was a cluster randomized controlled trial of PCCP set in 14 community mental health clinics in two northeastern states (16). Randomization occurred at the clinic level to avoid within-clinic contamination. Eight sites were located in one state and six in the other state. These clinic sites served approximately 8,000 service clients and provided a range of behavioral health services, including outpatient therapy, case management, residential programs, and community support programs. Site eligibility criteria included serving persons diagnosed as having a serious mental illness and having had no previous PCCP training.

The provider sample consisted of 60 provider teams that each had the same supervisor throughout the study (out of a possible 81 teams trained in PCCP). The average number of teams per site was 4.2 (range=2–9). The teams included one supervisor and two direct care staff nominated by the supervisor for their leadership capacity, which was defined as being a role model and having supervisor potential. The study (ClinicalTrials.gov ID NCT02299492) was approved by the New York University Institutional Review Board and conducted from 2013 through 2018.

Intervention

The 14 clinic sites were randomly assigned to either PCCP training (N=7; experimental condition) or service planning as usual (N=7; control condition). Using SAS, version 9.3, we generated randomization blocked by state, ensuring three experimental and three control sites in one state and four experimental and four control sites in the other state. All provider teams in the experimental condition (N=34) received training in PCCP, and the provider teams in the control condition (N=26) delivered service planning as usual. Usual service planning was based on problem-focused care and shared with PCCP the technical elements of setting measurable objectives and identifying billable services. The PCCP implementation strategy consisted of a 2-day dynamic in-person training with distribution of educational materials (manual, training slides, and PCCP tip sheets), followed by two 1-hour technical assistance (TA) calls per month with PCCP trainers over a 12-month period.

The PCCP training used a train-the-trainers model (17) with supervisors as the primary target to prepare them to train their direct care teams in PCCP. One monthly TA call involved supervisors only in order to discuss their progress in training their teams, and the second monthly call was for each team to take turns presenting a service plan for trainer and peer feedback. Three trainers who were PCCP experts with graduate credentials in mental health divided up the sites and conducted all the training at their sites. Trainers completed all components of the training, documenting their adherence to the standardized in-person training, the discussion for the supervisor calls, and their service plan feedback for the case-based calls. Overall, 88% of supervisors attended the training, 73% presented a case plan, 59% attended at least 10 of 12 supervisory TA calls, and 41% attended at least 10 of 12 case-based TA calls. Lower rates of attendance at the TA calls were due to competing work demands and because supervisors attended only the case-based TA calls for their teams.

Data Collection

Data were collected from a baseline online survey and chart reviews. Supervisors and direct care staff in the experimental and control conditions completed an online survey administered via e-mail with an embedded URL linking participants to the consent form and survey. The survey took approximately 40 minutes to complete, and each participant received a $20 gift card. Response rates were 100% for supervisors and 83% for direct care staff.

Chart reviews of service plans were conducted for the experimental and control conditions by researchers not blinded to the intervention. Medical records staff who had no connection to the study were instructed to randomly select service plans from a list of participating supervisors. For each site, 20 plans were selected at each of three time points: 1 month before intervention (baseline), 1 month before 12 months of the intervention, and 1 month before 18 months of the intervention. Because of low service user enrollment at one site, only 18 service plans (six from each time point) were selected. In total, 798 charts were randomly sampled from 60 teams.

Measures

The data source for person-centered care and service user characteristics was the chart review; the data source for supervision, implementation leadership, and program type was the baseline survey.

Person-centered care.

The PCCP Assessment Measure (PCCP-AM) (18) was used to measure person-centered care, which was conceptualized as a service outcome and a quality indicator (19). The service plan sets the blueprint for care by articulating the goals and steps to attain those goals and determines the client’s service experience. The measure was originally developed by experts who designed the intervention and refined by the research study team, which edited items for clarity. The 13-item measure operationalizes the delivery of person-centered care according to PCCP principles, which include orienting care to the client’s individual life goals, being strengths based, including natural supports, and emphasizing community settings over professional settings (8). These principles are assessed across key service plan domains: assessment, narrative summary, objectives, goals, and interventions. The items are rated on a scale ranging from 1 to 4, indicating the level of person-centered care: 1, “needs improvement”; 2, “approaches standard”; 3, “meets standard”; and 4, “exceeds standard.” For ease of interpretation in identifying the number of items in the service plan that met the standard of person-centered care, we dichotomized the item ratings: 0, “did not meet standard,” and 1, “met or exceeded standard.” The total summed score range was 0–13. PCCP-AM scores were averaged across service plans per team and per time point. A subsample of service plans (N=62) was coded by two raters with an interrater reliability of 81% at the dichotomized level. The PCCP-AM had Cronbach’s α=0.7 across all service plans (N=798).

Service user characteristics.

Service user age, race, and primary diagnosis were extracted from the chart review. For analysis, these variables were aggregated to the team level to represent service user characteristics based on the proportion or average number of service users served by the team.

Supervision.

Supervision was calculated as the number of hours per week in supervision focused on clinical content (e.g., case conceptualization), as reported by direct care staff. This score was aggregated to create a team-level mean score representing the average number of hours per week of clinically focused supervision.

Implementation leadership and program type.

Direct care staff rated their supervisor’s implementation leadership, which was the extent to which they were proactive, knowledgeable, and supportive and persevered with the PCCP implementation. The 12-item Implementation Leadership Scale had Cronbach’s α=0.90 (15). A mean score was calculated across the 12 items among direct care staff sampled from each team and aggregated to create a team mean score. Participants indicated which program they worked on from a list of community mental health programs on the online survey.

Data Analysis

The main effect of the experimental condition (i.e., PCCP) on person-centered care was assessed with three-level linear mixed-effects regression models that incorporated PCCP-AM observations (i.e., time) at level 1, teams at level 2, and clinic site at level 3. Linear mixed-effects models are ideal for longitudinal data that are clustered within clinic sites because they enable the use of all available measurements, generate unbiased parameter estimates when data are randomly missing, and adjust standard errors for clustering at both team and site levels (20). At level 1, PCCP-AM observations collected at baseline, 12 months, and 18 months were modeled via a piecewise linear growth model with coefficients representing initial status, change from baseline to 12 months, and change from baseline to 18 months. Program characteristics were included in the model at level 2. The experimental condition was included at level 3 because it was randomized at the site level.

Preliminary analyses confirmed significant clustering of PCCP-AM scores at the site (intraclass correlation coefficient [ICC]=0.27, p=0.001) and team (ICC=0.43, p=0.001) levels. Models were estimated with HLM software, version 7.03 (21). The models estimated the effect of experimental condition on PCCP-AM initial status (i.e., baseline equivalence), change in PCCP-AM scores from baseline to 12 months, and change in PCCP-AM scores from baseline to 18 months after controlling for service user and program characteristics. Effect sizes were calculated as Cohen’s d, which represents the standardized mean difference in change from baseline to the endpoint between experimental and control groups; Cohen defined effects as small (d=0.2), medium (d=0.5), and large (d=0.8) (22).

Results

The demographic characteristics of participating supervisors were consistent with those of the workforce in the two participating states (Table 1). The sample of participating programs represented a continuum of treatment modalities, reported acceptable amounts of supervisory support, and served populations with a high percentage of persons who identified as White and approximately half of whom had been diagnosed as having psychosis (Table 2). At baseline, the raw mean PCCP-AM scores for the control and experimental groups were 6.10 and 6.55, respectively. At 12 months, they were 6.28 and 7.12 and at 18 months, 5.79 and 6.73, respectively.

TABLE 1. Characteristics of all community mental health provider team supervisors and by control and person-centered care planning (PCCP) interventions

Providers
All (N=60)Control (N=26)PCCP (N=34)
CharacteristicN%N%N%
Race-ethnicitya
 White366016622059
 Black1627727926
 American Indian000
 Asian12013
 Native Hawaiian12140
 Other471439
 Hispanic23026
 Do not know231413
Gender
 Male1525727824
 Female457519732676
Education level
 Doctorate12013
 Master’s degree416821812059
 Bachelor’s degree16275191132
 Associate’s degree or lower23026
Discipline
 Social work345716621853
 Psychology1423623824
 Human services915415515
 Other357039
Age (M±SD)43±1246±1344±11
Years in mental health16±915±1016±8
Years in the agency9±79±69±7
Years in their role4±44±45±4

aSum exceeds total, because the subcategories were not exclusive.

TABLE 1. Characteristics of all community mental health provider team supervisors and by control and person-centered care planning (PCCP) interventions

Enlarge table

TABLE 2. Characteristics of all community mental health provider teams and by control and person-centered care planning (PCCP) interventions

Provider teams
All (N=60)Control (N=26)PCCP (N=34)
CharacteristicN%N%N%
Program type
 Outpatient therapy813415412
 Residential and housing1423281235
 Case management915519412
 Assertive community treatment91562339
 Community support71241539
 Young adult712312412
 Other61028412
Program characteristics (M±SD)
 Supervisiona2.2±1.91.8±1.22.4±1.9
 Leadershipb3.9±.93.9±1.03.8±.9
Service user characteristics (M±SD)c
 Primary diagnosis: psychosis .46±.32.44±.29.48±.34
 Age in years45±1643±1446±17
 Race: White.68±.26.66±.31.69±.23

aRepresents supervisor’s time in hours that focused on clinical content.

bPossible scores on the Implementation Leadership Scale range from 0 to 4, with higher scores indicating greater implementation leadership.

cRace and diagnosis variables are average proportions.

TABLE 2. Characteristics of all community mental health provider teams and by control and person-centered care planning (PCCP) interventions

Enlarge table

The linear mixed-effects regression model tested the effect of experimental condition on change in person-centered care (Table 3). At baseline, no difference was detected between the experimental and control conditions in person-centered care after controlling for covariates. However, baseline person-centered care was scored significantly lower in outpatient clinics than in other service settings (b=−2.41, SE=0.68, p=0.001) and in teams serving older clients (b=−0.05, SE=0.02, p=0.01).

TABLE 3. Effects of training in person-centered care planning on person-centered care at three time points

Baseline12 months18 months
CharacteristicCoefficientSEpCoefficientSEpCoefficientSEp
Parameter
 Intercept6.20.45.00.10.37.79−.66.36.07
 Experimental condition−.09.63.891.10.50.0301.47.50.01
Program characteristics
 Supervision−.58.13.66.24.17.17.25.13.06
 Leadership−.57.28.05.08.28.77.39.27.16
 Outpatient therapy–2.41.68.001.08.68.12.70.68.31
Service user characteristicsa
 Age in years−.05.02.01.02.02.38.02.02.19
 Race: White1.58.99.121.58.94.10–2.12.93.03
 Primary diagnosis: psychosis.11.89.90–1.71.88.06–1.87.87.04

aService user characteristic variables are team averages. Race and diagnosis variables are proportions.

TABLE 3. Effects of training in person-centered care planning on person-centered care at three time points

Enlarge table

The primary experimental effects were estimated by coefficients indicating the difference in the change in person-centered care between experimental and control groups from baseline to 12 months and from baseline to 18 months. At 12 months, and after we controlled for service user characteristics and program characteristics, the experimental group improved significantly more than the control group on person-centered care (b=1.10, SE=0.50, p=0.03). This represented a medium effect size of d=0.53 (95% confidence interval [CI]=0.05–1.02). At 18 months, the effect of the experimental condition on person-centered care was even more pronounced (b=1.47, SE=0.50, p=0.01), having a medium-to-large size of d=0.71 (95% CI=0.23–1.20).

No significant variance was identified in the associations between other model variables and PCCP at 12 months. However, at 18 months, teams with a higher proportion of service users who identified as White (b=−2.12, SE=0.93, p=0.03) and teams with a higher proportion of service users with a primary diagnosis of psychosis (b=−1.87, SE=0.87, p=0.04) delivered care that was significantly less person centered.

Discussion

The results of this study indicate that training supervisors in PCCP effectively increases the delivery of person-centered care, a key indicator of service quality, in community mental health clinics. At the conclusion of the 12-month intervention period, the PCCP training had a medium effect size on person-centered care, and this effect size further increased 6 months later. This finding shows that PCCP-trained providers were more likely to orient care to meet patients’ personal life goals than to manage symptoms. Their interventions were also more likely to use clients’ individual strengths and natural supports and to be based in the community, essential elements of the mental health recovery approach.

Teams with a higher proportion of service plans with White service users or service users with psychosis had lower rates of person-centered care. More research is needed on our finding related to service user race, particularly because our observation is in contrast to previous research finding that people of color experience lower levels of person-centered care in the health system (23). That service users with psychosis had a lower rate of person-centered care indicates that providers persist in their belief that some service users are “too sick” to determine their own care (5).

The results of this study indicate that providers could implement PCCP with rigorous training and support targeted to supervisors. The training also had a positive and sustained impact on the delivery of person-centered care across program types despite baseline differences in service planning. This observation suggests that clinic-wide PCCP training of supervisors can be applied across different treatment modalities. The supervisors were positioned to translate their new knowledge of PCCP within their teams, indicating the important role of supervisors in the adoption of evidence-based practice (24). Therefore, leveraging supervisors’ available time with their supervisees can be a mechanism by which to successfully implement and sustain person-centered care delivery (25).

Although not everybody received all PCCP training components, the implementation strategy promoted sustained practice change among the providers in this study. The train-the-trainers design enabled translating a nuanced practice that is both philosophical and technical in nature into routine practice. A recent review of person-centered care dimensions suggested that person-centered care elevates the consumer-provider relationship to “a higher ethical approach,” stating that person-centered care requires a paradigm shift at the policy level and in individual practice. Such shifts demand prolonged engagement in training rather than “one-shot” training sessions that cannot sustain practice change (26).

Finally, we note that the present study included an objective measure of service quality. Providers often endorse that they deliver person-centered care even when objective indicators suggest otherwise (5, 7). This confirms the lack of reliability of self-report measures in mental health, particularly for complex practices that are socially desirable (27). Having a measure that can be applied directly to service plans can provide an important quality improvement tool available to both researchers and agencies. Person-centered care is increasingly being included in accreditation and reimbursement requirements, creating a need for tools that can operationalize and accurately assess the extent of person-centered care in daily practice (28).

This study had several limitations. The chart review was not blinded, so it may have been subject to researcher bias. The PCCP-AM did not capture all aspects of person-centered care, most notably the quality of clinical interactions, which is an important aspect of the service user experience. Although the measure had acceptable reliability, more research is needed on PCCP-AM’s psychometric properties. Direct care characteristics of all those who completed the service plans were not available, which precluded evaluation of individual staff-level determinants of person-centered care plans.

Conclusions

PCCP as an intervention was found to change clinical practice in a way that reflects both the philosophical shift and the technical skills necessary to deliver person-centered care in routine mental health settings. The practice orients care to a person’s individual life goals and shapes a collaborative effort by both provider and service user to attain those goals. The findings of this study indicate that comprehensive training efforts can lead to observable and sustained practice change across different mental health programs. More research is needed to examine the impact of delivering person-centered care on engagement and clinical outcomes.

Silver School of Social Work, New York University, New York City (Stanhope); School of Social Work, University of Minnesota, Minneapolis (Choy-Brown); School of Social Work, Boise State University, Boise, Idaho (Williams); School of Social Policy and Practice, University of Pennsylvania, Philadelphia (Marcus).
Send correspondence to Dr. Stanhope ().

This study was presented at the Society for Social Work and Research Annual Conference, January 17, 2020, Washington, D.C.

This research was funded by the National Institute of Mental Health (award R01 MH-099012).

Dr. Marcus has received consulting fees from Allergan and Sage Therapeutics. The other authors report no financial relationships with commercial interests.

References

1 Berwick DM : What “patient-centered” should mean: confessions of an extremist . Health Aff (Millwood) 2009 ; 28 : w555 – w565 Crossref, MedlineGoogle Scholar

2 Institute of Medicine : Crossing the Quality Chasm: A New Health System for the 21st Century: The Brief Report . Washington, DC , National Academies Press , 2001 Google Scholar

3 Alexander L , Druss B : Behavioral Health Homes for People With Mental Health and Substance Abuse Conditions: The Core Clinical Features . Washington, DC , Center for Integrated Health Solutions , 2012 Google Scholar

4 National Consensus Statement on Mental Health Recovery . Rockville, MD , Substance Abuse and Mental Health Services Administration , 2004 Google Scholar

5 Tondora J , Miller R , Davidson L. The top ten concerns about person-centered care planning in mental health systems . Int J Personalized Med . 2012 ; 2 : 410 - 420 Google Scholar

6 Lodge AC , Kaufman L , Stevens Manser S : Barriers to implementing person-centered recovery planning in public mental health organizations in Texas: results from nine focus groups . Adm Policy Ment Health 2017 ; 44 : 413 – 429 Crossref, MedlineGoogle Scholar

7 Petros R , Solomon PL : Social workers’ propensity to endorse recovery-oriented service provision: a randomised factorial design . Br J Soc Work 2019 ; 50 : 42 – 61 Google Scholar

8 Tondora J , Miller R , Slade M , et al. : Partnering for Recovery in Mental Health: A Practical Guide to Person-Centered Planning . New York , Wiley , 2014 CrossrefGoogle Scholar

9 Adams N , Grieder D : Treatment Planning for Person-Centered Care: The Road to Mental Health and Addiction Recovery: Mapping the Journey for Individuals, Families and Providers , 2nd ed . New York , Academic Press , 2014 Google Scholar

10 Coulter A , Entwistle VA , Eccles A , et al. : Personalised care planning for adults with chronic or long-term health conditions . Cochrane Database Syst Rev 2015 ; 3 : CD010523 Google Scholar

11 Claes C , Van Hove G , Vandevelde S , et al. : Person-centered planning: analysis of research and effectiveness . Intellect Dev Disabil 2010 ; 48 : 432 – 453 Crossref, MedlineGoogle Scholar

12 Tondora J , O’Connell M , Miller R , et al. : A clinical trial of peer-based culturally responsive person-centered care for psychosis for African Americans and Latinos . Clin Trials 2010 ; 7 : 368 – 379 Crossref, MedlineGoogle Scholar

13 Stanhope V , Ingoglia C , Schmelter B , et al. : Impact of person-centered planning and collaborative documentation on treatment adherence . Psychiatr Serv 2013 ; 64 : 76 – 79 LinkGoogle Scholar

14 Damschroder LJ , Aron DC , Keith RE , et al. : Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science . Implement Sci 2009 ; 4 : 50 Crossref, MedlineGoogle Scholar

15 Aarons GA , Ehrhart MG , Farahnak LR : The Implementation Leadership Scale (ILS): development of a brief measure of unit level implementation leadership . Implement Sci 2014 ; 9 : 45 Crossref, MedlineGoogle Scholar

16 Stanhope V , Tondora J , Davidson L , et al. : Person-centered care planning and service engagement: a study protocol for a randomized controlled trial . Trials 2015 ; 16 : 180 Crossref, MedlineGoogle Scholar

17 Pearce J , Mann MK , Jones C , et al. : The most effective way of delivering a train-the-trainers program: a systematic review . J Contin Educ Health Prof 2012 ; 32 : 215 – 226 Crossref, MedlineGoogle Scholar

18 Stanhope V , Matthews EB : Delivering person-centered care with an electronic health record . BMC Med Inform Decis Mak 2019 ; 19 : 168 Crossref, MedlineGoogle Scholar

19 Proctor E , Silmere H , Raghavan R , et al. : Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda . Adm Policy Ment Health 2011 ; 38 : 65 – 76 Crossref, MedlineGoogle Scholar

20 Verbeke G , Molenberghs G : Linear Mixed Models for Longitudinal Data . New York, Springer , 2001 Google Scholar

21 HLM 7 for Windows. Lincolnwood, IL , Scientific Software International , 2011 Google Scholar

22 Cohen J : Statistical Power Analysis for the Behavioral Sciences . New York , Academic Press , 2013 CrossrefGoogle Scholar

23 Meyer OL , Zane N : The influence of race and ethnicity in clients’ experiences of mental health treatment . J Community Psychol 2013 ; 41 : 884 – 901 Crossref, MedlineGoogle Scholar

24 Dorsey S , Pullmann MD , Kerns SEU , et al. : The juggling act of supervision in community mental health: implications for supporting evidence-based treatment . Adm Policy Ment Health 2017 ; 44 : 838 – 852 Crossref, MedlineGoogle Scholar

25 Choy-Brown M , Stanhope V : The availability of supervision in routine mental health care . Clin Soc Work J 2018 ; 46 : 271 – 280 CrossrefGoogle Scholar

26 Stanhope V , Ross A , Choy-Brown M , et al. : A mixed methods study of organizational readiness for change and leadership during a training initiative within community mental health clinics . Adm Policy Ment Health 2019 ; 46 : 678 – 687 Crossref, MedlineGoogle Scholar

27 Walfish S , McAlister B , O’Donnell P , et al. : An investigation of self-assessment bias in mental health providers . Psychol Rep 2012 ; 110 : 639 – 644 Crossref, MedlineGoogle Scholar

28 Behavioral Health Standards Manual. Tucson, AZ , Commission on Accreditation of Rehabilitation Facilities , 2018 Google Scholar