Applying a Model of Stakeholder Engagement to a Pragmatic Trial for People With Mental Disorders

Stakeholder partnerships represent a critical element in patient-centered trials. Involving stakeholders throughout all phases of research helps ensure study feasibility and appropriateness of the program being tested and helps balance fidelity of the program with the need to adapt to differences across complex clinical sites. Stakeholders, or partners, can include patients, caregivers and family members, providers, purchasers, payers, advocacy organizations, and policy makers (1,2). The goals of engagement are to develop meaningful and effective partnerships, increase the relevance of research to patients, facilitate the use of research results in health care settings, and improve patient health (3).

Although stakeholder engagement is valuable throughout the development, implementation, and dissemination of a research project, there are few examples of partnerships in pragmatic trials aimed at improving mental health care. Here we describe a stakeholder engagement strategy for a multisite comparative-effectiveness study assessing the Engaging Patients in Care program. Engaging Patients in Care is a care navigator program that aims to increase linkage between emergency departments (EDs) and outpatient treatment at community mental health centers (CMHCs) for patients with mental disorders. The study compares care navigation provided by mental health professionals and peer support specialists in facilitating linkage from the ED to outpatient treatment and supporting ongoing engagement in care. The care navigators in the Engaging Patients in Care program use shared decision making, motivational interviewing, and goal setting to guide patients in identifying and addressing barriers to engaging in outpatient care. In this column, we describe partnerships throughout the study phases, with a focus on who was involved, how stakeholders were engaged, and lessons learned.

Stakeholder Engagement

Overview.

Our stakeholder engagement approach, which is based on the Patient-Centered Outcomes Research Institute’s model for research on patient-centered outcomes (3,4), incorporates the following principles: stakeholders are involved in every step of the study, from design to dissemination and determination of sustainability; we seek to actively engage a variety of stakeholders; with our partners, we aim to foster an environment of open communication, colearning, and mutual respect; and stakeholders are full partners in all key study decisions.

This study was developed as a partnership between Emory University, the University of South Carolina, and the South Carolina Department of Mental Health. Stakeholders include patients who use mental health emergency care; mental health providers; public-sector health care payers and purchasers, including the South Carolina Department of Mental Health and the South Carolina Department of Health and Human Services; and mental health workforce and advocacy organizations. Partnerships with these groups are essential for mental health programs, given the continued fragmentation between the mental health, substance use, and medical health care systems and the challenges in providing services that may fall outside of the typically reimbursed services (5). Below, we describe how we have engaged these stakeholder groups. [A table showing additional information on engagement strategies, roles, and results of the various partnerships is available as an online supplement to this column.]

Establishing and maintaining partnerships.

The study is guided by a national advisory board and state implementation team. These groups ensure that the research findings are relevant to a variety of audiences and plan for program sustainability (3). Existing collaborators at state governmental and advocacy agencies helped us identify and approach members for these two groups, including a patient and peer support specialist from a local CMHC. We have not encountered difficulties in finding patients and providers willing to serve on the groups and have experienced minimal turnover.

The advisory board, which meets annually, oversees key study decisions and will spearhead planning for program sustainability and national dissemination of results. The advisory board is chaired by the director of a consumer-run mental health agency and includes four additional members from national-level managed behavioral health care, patient advocacy, and mental health workforce development organizations. Following advisory board recommendations, mental health recovery was added to the study proposal as an outcome measure. The board also provided input that was incorporated into the care navigator program

The implementation team oversees the planning and execution of the study, ensuring that findings apply to patients and the public mental health system. The implementation team also facilitates colearning between the partners and the research staff; stakeholders gain insight into the research process through involvement in study planning and decision making, and research staff learn how to best engage patients and incorporate patient-centeredness in the clinical sites. The implementation team consists of five members who represent patients and providers from local CMHCs in South Carolina, a patient advocate from the National Alliance on Mental Illness South Carolina, and representatives from the South Carolina Department of Mental Health and the South Carolina Department of Health and Human Services. The implementation team has contributed to refining recruitment and enrollment protocols, as well as developing the intervention manual and materials. It has also provided the study team with a greater understanding of the state mental health care system, such as which billing codes apply for care navigation strategies. [A table showing examples of changes resulting from stakeholder engagement is available as an online supplement to this column.]

Site visits are another strategy to initiate partnerships with stakeholders involved in participant recruitment or program implementation. The research team made site visits to build rapport with partners, such as mental health liaisons, ED nurses, telepsychiatrists, and CMHC clinic staff; learn about site-specific contextual factors; and finalize training with care navigators. CMHC clinic directors helped identify professionals and peers at their clinics who could serve as care navigators. These visits also resulted in adjustments to the care navigators’ training, patient recruitment protocols, and the warm-handoff phone call, which is the first interaction between the care navigator and patient in the ED. [A table showing examples of changes to the care navigator program based on stakeholder engagement and input is available as an online supplement to this column.] Speaking with site representatives led to the development of site-specific back-up plans, such as training extra staff to step in when the normal care navigator is sick or on vacation. Mental health liaisons proved to be valuable partners because they are Department of Mental Health employees who split their time between the ED and CMHC, thus providing a linkage between the two systems. However, not every CMHC has a mental health liaison, and staff turnover with liaisons and peer support specialists has been a challenge because it necessitates forming new relationships.

Gathering and using stakeholders’ perspectives.

We used qualitative methods to gather patients’ and clinicians’ perspectives and to inform the program development, implementation, and revisions. Qualitative methods allow us to gain in-depth information about patients’ and clinicians’ experiences with care linkage and engagement, as well as feedback on the care navigator program.

During the development of the care navigator program, we conducted patient focus groups and key informant interviews to ensure the feasibility, patient-centeredness, and applicability of the program. Two focus groups at two CMHCs were conducted with 12 patients who had previous experience in an ED. Key suggestions from these focus groups included that care navigators should treat each patient as an individual and not assume that all experiences are the same. Patients recommended that care navigators convey compassion and genuine caring. As one patient explained, “You need to listen, you need to pay attention to us, you need to be aware where you're coming from and not judge us…. We need you to listen and feel, because sometimes it's hard for us to explain these things.”

Interviews with seven key informants provided insights into the current practices in the ED or CMHC in which they worked and any linkages between the two. Key informants discussed the importance of validating patients’ concerns, as well as being nonjudgmental and supportive. As one key informant noted, “I would think to try to find some way to say [to patients], ‘We’re here for you.’” They also suggested that when care navigators talk to patients in the ED, they should describe what the patients can expect when they arrive at the CMHC for their first appointment after discharge. One key informant said, “And the other thing that might be helpful for new [patients] is actually having brochures about the mental health center and what we can do. Because a lot of times they’re like, ‘I was just told to show up here, I don’t know what I’m doing.’” That doesn’t mean they weren’t told, but when you’re in a crisis, you don’t hear and process well.”

The recommendations from patients and key informants were incorporated into the care navigator program, training, and materials. We revised wording in the program manual to clearly convey empathy and support, added examples of barriers to receiving mental health care to the program manual, and developed materials to give to the patients in the ED.

Qualitative data collection will continue annually with patients and care navigators for the length of the study. Patient interviews will examine experiences with care navigation and assess acceptability of the care navigator program. Patients discuss the barriers they face in engaging in mental health care, whether and how the care navigator program has been helpful in overcoming these barriers, and recommended improvements for the program. Interviewing care navigators provides an in-depth understanding of their experiences in delivering the program. Care navigators describe encounters with patients, how well the program fits with their clinical duties, successes and challenges with program delivery, and feedback on how to improve the program. Each year, findings from the interviews will inform program implementation, with adjustments being made as needed.

Lessons Learned

Based on our experiences so far, three main lessons regarding stakeholder engagement are the benefits of early stakeholder involvement; gathering insight into the particular characteristics of each site, while keeping overall program fidelity in mind; and early initiation of discussions with stakeholders about program sustainability.

First, early involvement from a diverse range of stakeholders allows for guidance on key aspects of the study, including the choice of outcome measures, program design, and logistical planning for recruitment and program implementation. The nature of our study, which focuses on transitions of care, necessitated partnerships across a range of settings, including CMHCs, EDs, and the relevant state-level departments. Patient input was invaluable to understanding patients’ experiences in the ED and in being linked to the CMHC. Likewise, providers at the CMHCs provided insight into how the care navigator program would best fit into their daily work. Our other stakeholders helped us better understand how this program would work in the state health care system. Important adaptations to the care navigator program that were based on our stakeholders’ input included training on billing codes, preparing the patient for the first visit to the CMHC, and ensuring that discussions of barriers to care and goal setting were meaningful and easy to follow. Having multiple ways in which to engage stakeholders and elicit feedback has been central to our goals of ensuring the patient-centeredness and feasibility of the care navigator program.

Second, stakeholder perspectives have been important in the process of balancing the need to implement the program with fidelity, while allowing for flexibility to meet the diverse needs of local clinical sites and patients. Procedures differed across ED and CMHC sites in multiple ways, including staffing, patient populations, connections between the ED and CMHC, and appointment scheduling methods. Therefore, we needed to develop the care navigator program so it could be adapted to different sites and personalized for patients, while still maintaining the same core structure. We designed the encounters between care navigators and patients to be implemented in a similar manner across sites, with other aspects, such as the logistics of making appointments, being based on the specifics of the site. Qualitative data collection and documentation of the differences between sites will help determine which settings and for which patients the care navigation program works best.

Finally, early engagement of stakeholders in discussions on issues within the health care system, such as identifying reimbursement methods for care navigation services, sets the course for longer-term planning for program dissemination and sustainability. The implementation team has provided valuable insight into how care navigators can currently bill for services. The team will lead planning for dissemination and sustainability of the care navigator program within the state. The advisory board will be central in determining how to roll out the program nationally.

Conclusions

Pragmatic trials examining the effectiveness of programs to improve mental health care benefit from the involvement and insights of diverse stakeholders. Establishing a stakeholder engagement plan facilitates early, intentional, and meaningful stakeholder involvement. Stakeholders can provide valuable input on the design and implementation of interventions and study designs, particularly around the heterogeneous factors at the patient, provider, and system levels that can affect treatment linkage and engagement. Continued involvement can contribute to program refinement, dissemination of results, and sustainability planning.