Shared decision making has been defined as a process by which clinician and client engage in a two-way communication of information, clarify values and influences regarding a treatment decision, and agree on a plan of action (1). Shared decision making has been advocated by influential reports (2,3) and is congruent with broader philosophical and ethical perspectives on the treatment of people diagnosed as having severe mental illness (4–6). Furthermore, a participatory model of decision making is a central element of several evidence-based psychosocial mental health treatments (6–8) and a core component of the Chronic Care Model (9). Two European randomized controlled trials and one recent meta-analysis showed that when client preferences for mental health care are queried and honored, outcomes are better along several dimensions (10–12). Finally, for many mental health care decisions, no one course of action has emerged as the optimal clinical path to recovery from severe mental illness.

At the same time, several decades of research have indicated that mental health clients and their case managers have divergent opinions regarding prioritization of goals, services, and outcomes of care (13–25). Why this discordance exists is uncertain. Case managers may presume that their clients want the same services that they would choose themselves (16). Clients of mental health services may have difficulty expressing differences of opinion with their case managers and may not tell their case managers when they disagree with a course of treatment (26).

Regardless of the reasons for this discordance, information technology that facilitates communication within the dyad may offer clients more involvement in self-management of their mental illness. Considering a mental health workforce that turns over at a rate of approximately 50% per year (27–29) and is made up of 40% of mental health workers without advanced clinical training (30), electronic decision support may be a practical way to facilitate shared decision making and communication of the client perspectives. Electronic decision support can provide a structural support to both client and clinician in the care planning process (4).

This study examined whether use of an electronic decision support system (EDSS) to create a shared-decision-making plan will lead to greater client and case manager satisfaction with the care planning process than the usual agency procedures for creating a care plan. Also investigated was whether clients are more engaged in the process of care planning, as measured by recall of the goals included in the care plan three days after the planning session.

Design

This study involved a cluster randomized controlled trial of a new EDSS designed to support greater client involvement in goal setting and to assist both clients and case managers in engaging in shared decision making. [A depiction of the study design is available as an online supplement at ps.psychiatryonline.org.] Case managers from three clinics were randomly assigned to the intervention group or treatment as usual. Clients were assigned to the same group to which their case manager was assigned. Because of the cluster randomization, all case managers participated multiple times, but all clients participated only one time. Because one clinic was larger than the others, one clinic recruited approximately one-half the participants and the other two recruited approximately one-quarter each. Two of the clinics provided traditional case management and community support. One clinic provided slightly more intensive services, because it previously provided assertive community treatment and continued to serve the same population. The participating agency the clinics belong to espouses a mission to help mental health clients maintain autonomy over their lives and achieve recovery-oriented goals (Wilkniss S, personal communication, 2009). Data were collected between September 2008 and January 2009.

For the intervention, a three-step EDSS process inverts the usual care planning procedures. First, clients indicate their top priorities and ideas for services at a touchscreen-enabled computer kiosk. Second, the information is electronically sent to the clients' case managers, who then complete a similar process. Finally, the two perspectives are merged electronically and presented graphically in a shared decision-making session with the dyad. [An appendix showing an example of EDSS screens is available as an online supplement at ps.psychiatryonline.org.]

The EDSS was intentionally designed so there would be minimal training burden on both clients and case managers. Clients in the study had not been exposed to the EDSS before. Case managers were given a brief manual and a one-hour didactic and practice session, in which they were able to try out the technology and ask questions. These sessions were conducted by the first author once for each site.

Clients who were assigned to receive usual care planning met with their case managers at the time that their six-month care plans were due. Case managers and clients were told to complete their care plans in the way that they normally would. Care plan discussions were audio recorded, but no research staff member was present. The usual care plan included use of a behavioral health electronic medical record designed to aid with billing and to theoretically help case managers create recovery-oriented care plans. There appeared to be significant heterogeneity in how care plans were completed, as is the case in real-world case management.

Clients who were assigned to the intervention group met with their case managers a minimum of three months before their regularly scheduled six-month care plans were due, so they could create care plans with the EDSS without worry that finishing the six-month care plan would bias the EDSS results. Clients in the intervention group eventually completed their six-month care plan in the usual way for billing purposes; however, they were able to use the intervention care plan as a guide.

Measures

Two questionnaires were developed for this study, one on the case manager satisfaction with the care planning process and the other on client satisfaction with the care planning process. The case manager version contains six statements related to the dimensions of case manager-client communication, time, flow, credibility as a clinical tool, and organization of information. The statements were rated on a 5-point Likert scale, ranging from 1, strongly disagree, to 5, strongly agree; higher summary scores indicate more satisfaction. The Cronbach's alpha for the entire scale was .74.

The case manager satisfaction measure asks case managers to rate their satisfaction with each care planning encounter, instead of their overall satisfaction with the care planning process. Pilot testing indicated that case managers strongly felt that the EDSS would work well for some clients but not others.

The client version contains seven statements rated by the client on a 5-point Likert scale, ranging from 1, strongly disagree, to 5, strongly agree; higher summary scores indicate more satisfaction. Based on the literature regarding what makes clients satisfied with a mental health encounter, the items were designed to assess case manager-client communication, involvement in decision making, being informed about decisions that were made, and having a clear management plan (31). The questionnaire also asks which types of information were communicated for clients who state that the EDSS helped them communicate important information about themselves to their counselor. The Cronbach's alpha for the entire scale was .62.

Clients were asked about their knowledge of the care plan two to four days after the care planning session. The researcher first asked the client to note which goal areas appeared in the care plan. The client was then provided with a list of goal areas not already indicated by the client and asked whether they were in the care plan. For each area correctly identified in either manner, the client was asked to relate what his or her individualized goals or objectives or shared decisions (as applicable) were within each quality-of-life area. All measures except the client knowledge of care plan were completed immediately after participation in the study.

Case managers were randomly assigned to intervention or control conditions within each of three clinics that were part of the same agency. Three clinics were necessary to achieve minimal sample size because clients could participate only if they were scheduled for a regularly occurring six-month care plan. Because case managers were randomly assigned to groups within sites instead of by site, no imbalances due to site characteristics should be present. Twenty case managers at the three clinics participated in the study with an average of four clients each. Clients were clustered by case manager, so all clients assigned to a particular case manager participated in the same condition. Clients participated only one time. Thus 40 dyads were exposed to the prototype EDSS at the time of their biannual case management planning review, and 40 dyads were in the control group and completed their usual care planning sessions. Researchers collected demographic, clinical, and workforce data before the planning sessions.

Both client and case manager questionnaires on satisfaction with the planning process were administered with a self-report computerized format. For the client questionnaire, a research assistant was present to provide assistance as needed. The knowledge of care plan questionnaire was administered by a research assistant over the telephone or in person at the mental health center if the client did not have access to a telephone. Each client was asked to recall which broad areas the care plan or shared-decision-making plan focused on. The proportion of the care plan goals that were recalled by clients was calculated by summing the number of care plan goals appearing in the care plan for the denominator, and summing the number of correctly recalled goals for the numerator. The maximum denominator was 11 goals.

The Dartmouth Committee for the Protection of Human Subjects and the agency's internal institutional review board approved the study. The study was deemed to be of minimal risk to participants. Full informed consent was obtained for research participants by the study research assistant.

Data analysis

Descriptive statistics were used to characterize the case manager and client samples. The differences between the intervention and the control groups' summary scores and individual items for the client and case manager questionnaires on satisfaction with the care planning process were examined via linear regression. Linear regression was also used to control for case manager age in the model that used the questionnaire on case manager satisfaction with the care planning process and to control for years working with the case manager in the model that used the questionnaire on client satisfaction with the care planning process.

Intraclass correlation coefficients (ICCs) were computed within case managers for all questionnaires and to adjust for clustering as appropriate (32,33). The ICCs indicated that case manager responses regarding individual sessions were highly correlated within individual counselors and required clustering of standard errors (ICC=.65) but that client responses did not (knowledge of care planning, ICC=.03; client satisfaction with the care planning process, ICC=.10) (34).

Because of a nonnormal distribution in the variable for client knowledge of the care plan, a nonparametric two-sample Wilcoxon rank-sum (Mann-Whitney U) test was used to assess the hypothesis that clients creating their care plans using the EDSS would have significantly greater knowledge of the goals in their care plans than clients in the usual care group.

Table 1 shows the demographic and workforce characteristics of the case managers (N=19). (Twenty case managers participated; however, demographic data were missing for one of the case managers in the control group.) The workforce and demographic characteristics were diverse. Although case managers were randomly assigned within site, case managers in the control group were significantly younger than case managers in the intervention group (31 years versus 47 years; p<.001). This led to an imbalance in both years of total mental health experience and years worked at the agency.

Table 2 shows the demographic and clinical characteristics of the clients who participated (N=80). Clients in the intervention and control groups were balanced on all variables except for length of time working with the case manager, which is related to the randomization failure on case manager age. The total sample was diagnostically and demographically heterogeneous.

Case managers rated how satisfied they were with the care planning process for each client immediately after the care planning session. Case managers responded for all 40 intervention sessions and 38 control group sessions. The mean summary score for the case manager satisfaction with the care planning process questionnaire was 4.0±.5 for the intervention group and 3.3±.5 for the control group. The multiple linear regression, controlling for counselor age in years, showed that intervention status significantly and positively predicted a higher summary score (intercept=3.29, β=.62, adjusted p=.01). Age of the case manager was not significant in the model. The model explained approximately 30% of the variance in the summary score (p<.001).

Items were also examined with raw means and frequencies, and then each item was analyzed with multiple linear regression that controlled for case manager age, with the standard errors clustered by case manager (Table 3). For all questions except the two that were intended to access information regarding “credibility as a clinical tool,” the mean scores for the intervention group were significantly higher (or lower, if an item was reverse-coded) than the mean scores for the control group.

For the questionnaire on client satisfaction with the care planning process, the intervention and control groups were virtually identical on mean summary score (3.88±.54 in the intervention group versus 3.78±.56 in the control group). In the linear model only client experience with the case manager revealed a clinically insignificant (but statistically significant) negative effect—the longer clients were with their case mangers (in months), the more dissatisfied they were (β=–.003, adjusted p<.001). The fit of the model was not significant and explained little of the variance in score.

There were no statistically significant differences between the intervention and control groups on any individual items on the questionnaire on the client satisfaction with the care planning process. A clear ceiling effect (or floor effect, in the case of the reverse-coded items) was observed for four out of the seven items. Still, a substantial proportion of the clients in both the intervention and control groups said that they wished they had more of an opportunity to discuss something on their minds with their counselors before creating their plans and were not sure what they would be working on with their counselors in the coming months (Table 4).

Sixty-nine (86%) of the 80 clients were successfully contacted two to four days after participation in the care planning sessions. Thirty-six clients (90%) in the control group and 33 clients (83%) in the intervention group were contacted. Clients in the intervention group had significantly greater knowledge of their care plan goals than the clients in the control group (mean proportion of plan goals recalled=75%±28% versus 57%±32%; z=−2.367, p=.02). There were no significant differences in participants' incorrect recall of goals between the intervention and control groups (mean proportion of plan goals that were incorrectly recalled=17%±16% in the intervention group versus 20%±16% in the control group).

An EDSS designed to facilitate shared decision making on goal setting improved case managers' awareness of clients' concerns and satisfaction with the process and increased clients' awareness of the goals and services in their plans, compared with the treatment-as-usual group. Client satisfaction with the process did not improve.

Our findings are congruent with the literature on job satisfaction among mental health workers. Perceived organizational, material, and training supports correlate with job satisfaction among case managers (29). In both the usual care and the intervention group, the process of care planning was facilitated by electronic supports. The usual care group used an electronic medical record designed to support documentation and billing of services. In the EDSS group, the care planning process was guided by empirical and practical considerations regarding what is needed for case managers to access needed information and to collaborate with clients to help them make decisions about individual goals.

Our finding that the clients who used the EDSS were no more satisfied with the care planning process than those in the usual care condition could have occurred for several reasons. Measuring satisfaction among receivers of health care in general (35), and mental health care specifically (36), has been controversial for some time. At least one study found that when mental health care users rate quantitative items positively on satisfaction measures, they may be considering a range of factors, including culpability for problems, before they assign a satisfaction rating (37). These raters appear to make allowances for what the user considers extenuating circumstances, such as service system limitations. Mixed-methods studies have found that quantitative measurement of satisfaction also often masks dissatisfaction with services (36–38). It is possible that measuring satisfaction with the care planning process is attempting to measure a process too far removed from receipt of actual services to be highly relevant to clients.

The finding that clients in the intervention group had greater recall of their care plans than clients in the control group could be due to more involvement in the care planning decision making process, to receipt of information in a more accessible format, or to both. The 2005 Institute of Medicine Quality Chasm series on mental health noted that clients are not routinely informed of decisions affecting their care (3). If being informed regarding one's own care decisions can be thought of as a prerequisite for a participatory model of decision making (39), the EDSS at least enhances clients' awareness of their plans for care.

Limitations

This study has some limitations. First, case managers in the intervention group could have rated the EDSS more highly not because of its actual utility but because of its novelty (40). Second, there are many ways to define case manager and client satisfaction (35,36). The satisfaction measures were based on the literature. It is possible that the general literature on satisfaction with mental health services does not apply to satisfaction with the case management care planning process.

Future research

Research on how to maximize the potential benefits of information technology in the public sector mental health arena is in its infancy. Longitudinal studies that test systems that were designed to follow up on shared decisions over time and that test clinically meaningful outcomes are required in order to truly judge the usefulness of EDSSs in supporting shared decision making and improving care for people diagnosed as having severe mental illnesses.

This study demonstrated that clients can build their own care plans and negotiate and revise them with their case managers by using an electronic support. The EDSS brought to light preferences held by clients that were not previously known by case managers. Case managers and clients then appeared to be able to work together to take action on these items. Clients were also more aware of the care decisions that were being made either directly by them or on their behalf.

This study was funded by the West Family Foundation and the Segal Family Foundation.

The authors report no competing interests.