Strengths Model Case Management Fidelity Scores and Client Outcomes

Strengths model case management (SMCM) was developed in the mid-1980s as a response to traditional deficit-oriented approaches. SMCM is both a philosophy of practice and a set of tools and methods designed to enhance recovery by taking a goal-oriented approach to service delivery (1). Practitioners use a strengths assessment to help individuals identify important and meaningful goals as well as skills, talents, and environmental strengths that will help in achieving their goals. The personal recovery plan is used to help people take small, specific, and measurable steps toward a goal until it is achieved. Problems, barriers, and challenges are not ignored in SMCM; rather, they are addressed in the context of the goals the individual wants to achieve. Identified strengths are used to either overcome identified barriers or increase options for alternative routes to goal attainment.

Nine studies have tested the effectiveness of using SMCM to assist people with psychiatric disabilities. Four of the studies used experimental or quasi-experimental designs (2–5) and five used nonexperimental methods (6–10). These studies have reported positive outcomes with SMCM to help people with psychiatric disabilities in the areas of hospitalization, housing, employment, symptoms, leisure time, social support, and family burden.

Full implementation of SMCM consists of structural components (low caseload sizes, low supervisor-to-case manager ratio, weekly group supervision using a structured format for case presentations, and so on) and practice components (such as use of the strengths assessment and personal recovery plan tools, use of naturally occurring resources to achieve goals, and in-person service delivery). It also requires specific supervisory behaviors to teach and reinforce practice skills of frontline staff (that is, weekly review of SMCM tools, specific feedback on the use of tools in practice, and field mentoring).

The Substance Abuse and Mental Health Services Administration (11) identified a strengths-based approach as one of the ten important elements for recovery-based services. It is important that the term “strengths-based approach” is anchored in concrete practices that can be evaluated in relation to client outcomes to prevent the term from being applied loosely to a large number of practices that have little impact on recovery. The SMCM contains specific practices and methods that are designed to assist the practitioner in working from a strengths-based orientation. Although positive outcomes have been demonstrated with SMCM, there is no empirical evidence demonstrating an association between the level of model adherence (fidelity) to SMCM and the use of SMCM to help people with psychiatric disabilities and improve client outcomes. This study aimed to examine the relationship between the fidelity of SMCM and the client outcomes of psychiatric hospitalization, competitive employment, postsecondary education, and independent living. High fidelity to SMCM was expected to improve the team's performance in client outcomes.

Methods

The fidelity scale for SMCM was developed to ensure adherence to key components of the model for implementation purposes (1). The scale consists of 11 items in three core areas: structure, supervision, and service and clinical issues. Items are rated on a 1–5 scale with 5 signifying higher fidelity. For example, “What percentage of consumer contact time occurs in the community?” is rated as 1, less than 40%; 2, 50%–64%; 3, 65%–74%; 4, 75%–84%; and 5, 85% or more. Fidelity reviews were conducted at baseline and then every six months during the first two years of implementation. Annual reviews occurred thereafter. A baseline fidelity review was conducted after an agency had committed to implementing SMCM but before any formal training and technical assistance had started. Total fidelity scores can range between 11 and 55, with a score of 45 and higher considered to indicate high fidelity to the model. Data sources used to rate fidelity included observation of team meetings, chart review, review of SMCM tools used in practice, and interviews with program staff and people receiving services.

Secondary data analysis was conducted. Data were collected over an 18-month period between May 2004 and October 2010 during regularly scheduled fidelity reviews for each of the 14 case management teams representing ten agencies that served an average of 953 clients who met criteria for serious mental illness. There were 1,195 clients at baseline, 965 at six months, 849 at 12 months, and 802 at 18 months. The intervention time varied among the teams.

Fidelity scores were added to a database once the review was conducted. Each review was conducted by at least two consultant-trainers who had extensive experience in SMCM implementation at the University of Kansas School of Social Welfare. Interrater reliability (intraclass correlation) between the two raters of the fidelity scale was .97, representing a high level of agreement.

In addition to rating fidelity, all case managers on the team completed reports regarding the current status of each client on their caseload in the areas of psychiatric hospitalization, competitive employment, postsecondary education, and independent living. These outcomes were provided for the ratio of successful events (such as being competitively employed) over total number of clients in a team. Demographic characteristics of case managers and their clients were not available in this data set. In order to assess the relationship between the fidelity scores and client outcomes, time-varying covariate linear growth modeling was applied via SAS version 9.2 with PROC MIXED. The study was approved by the institutional review board of the University of Kansas.

Results

Marginal means and standard deviations for fidelity scores over an 18-month period were 27.4±9.7 at baseline, 39.2±6.9 at six months, 44.7±4.8 at 12 months, and 47.0±4.4 at 18 months. Competitive employment rates at baseline and respective follow-ups were 15.5%±6.5%, 17.6%±5.6%, 18.1%±3.4%, and 20.2%±5.1%. Psychiatric hospitalization rates (percentage hospitalized) were 10.8%±5.7%, 9.9%±4.8%, 5.4%±3.2%, and 4.2%±3.1% at baseline, six, 12, and 18 months, respectively. Postsecondary education rates (percentage attending school) were 3.6%±3.0%, 4.5%±4.0%, 5.4%±4.0%, and 5.7%±3.1%, respectively. Percentage of persons living independently were 89.1%±5.0%, 90.3%±6.4%, 95.0%±3.7%, and 95.7%±3.2%, respectively.

The fidelity score (β=6.43, p<.001) and outcomes, including psychiatric hospitalization (β=–2.48, p<.001), postsecondary education (β=.65, p<.05), and independent living (β=2.42, p<.001) significantly improved over time, and the change in the rate of competitive employment was marginally significant (β=1.43, p=.06). However, once a time-varying covariate (fidelity score) was introduced, the time effect disappeared for employment, hospitalization, and education, and the time-varying covariate effect of the fidelity score was significant (p<.05) for all but independent living (p=.85). That is, the improvement of outcomes was explained by the improvement of the fidelity score (Table 1).

Concurrent correlations between the fidelity score and outcomes at each time period showed expected directions (that is, the correlation coefficients ranged from .2 to .4 for employment, from −.3 to −.6 for hospitalization, and from .4 to .9 for education) except for independent living (that is, the correlation coefficients were .2 for baseline, −.3 for six months, and 0 for both 12 and 18 months).

Discussion

To our knowledge, this is the first published study to examine the impact of SMCM fidelity on case management teams' performance in terms of clients' outcomes, including competitive employment, psychiatric hospitalization, postsecondary education, and independent living rates. Our preliminary findings showed that client outcomes improved over time, but improvements depended on the fidelity scores, except for independent living. The nonsignificant association for independent living may be explained by the fact that the state average in Kansas for independent living rates was already high at 90%. This may have contributed to the minimal effect of fidelity on this particular outcome. However, the effort made for improving SMCM fidelity seemed to enhance the clients' outcomes over time. Most teams were able to achieve high fidelity within 12 months, with some doing so within six months, making SMCM a feasible practice for agencies to implement.

The study had some limitations. First, secondary data with a convenience sample were used. A lack of demographic information for clients as well as case managers limits the generalizability of the study results. Second, individual client outcome was aggregated at the team level as a proxy of the team level achievement of SMCM. That is, client outcomes were calculated as rates restricted to the cohort active on the caseload at each time point. This would restrict the direct linkage of the improvement during 18 months for the observed outcomes. Third, a number of clients declined over time. Information about the reasons for dropout is not available in this data set. Under naturally occurring circumstances, case managers and clients might come and go from the teams as a result of turnover and disengagement. Future experimental designs will be needed to ensure internal validity. Finally, the overall improvements of the outcomes were modest. For a sizable portion of the 18 months, teams had yet to achieve high fidelity, meaning that clients did not receive 18 months of high-fidelity SMCM. It may take a longer period of time for clients to receive the full benefits of SMCM services. Continued effort to examine the effect on client outcomes with more time points will be needed.

Despite the limitations, our findings add to the existing literature on SMCM. The strengths perspective, along with the recovery movement, has gained popularity over recent decades. Empirical evidence linking fidelity and client outcomes has been needed. Future studies are needed that focus on intraindividual growth with SMCM and on attributes of potential clients who receive greater benefit from SMCM, such as demographic characteristics, psychiatric symptoms, and interactions with other mental health services use. How specific items on the fidelity scale contribute to outcomes will also need to be examined.

Conclusions

This is the first study to investigate the association between SMCM fidelity and client outcomes. The results suggest that SMCM had a significant positive effect on the client outcomes of competitive employment, psychiatric hospitalization, and postsecondary education rates over an 18-month period. The fact that all 14 teams reached high fidelity suggests that SMCM is an implementable practice. Given the dismal performance for these outcomes by many mental health systems, further research that avoids the limitations extant in this study is both warranted and necessary.

Figures and Tables