The Recovery-Oriented Care Collaborative: A Practice-Based Research Network to Improve Care for People With Serious Mental Illnesses

Dissemination and implementation of evidence-based practices are critical to improving community health. Unfortunately, an average gap of 17 years between research findings and their adoption into community-based practice persists (1). One major reason is the disconnection between research and everyday practice (2). One solution to this problem is the establishment of practice-based research networks (PBRNs), which are collaborations between academic researchers and groups of health care providers. The goals of PBRNs are to identify questions that center on clinicians’ experience and to actively include clinicians in developing a research study, collecting and analyzing the data, and disseminating and implementing research findings, with the expectation of ongoing collaborative projects (3). PBRNs have become one engine for translational research in the U.S. health care system. The Agency for Healthcare Research and Quality lists 152 active PBRNs on its Web site (pbrn.ahrq.gov/pbrn-registry). By linking questions related to community-based practice with rigorous research methods, the findings of PBRNs are more easily adopted into routine practice. Although resources provide guidance for primary care–based PBRNs (4), few articles are available on mental health–based PBRNs (3,4–6), and less than 5% of PBRNs have included mental health practitioners.

The Recovery-Oriented Care Collaborative (ROCC), a PBRN focused on improving services for people with serious mental illnesses, was established in 2012 by four community-based mental health agencies. Four phases of development followed: building an infrastructure, developing a research study, executing the study, and consolidating the PBRN. Here we report key replicable components of the ROCC’s development and activities and provide general recommendations based, in part, on structured interviews completed with PBRN members.

Phases in Building the ROCC

First Phase: Building an Infrastructure

The goals for the first phase included providing initial leadership, selecting and recruiting PBRN members, establishing the leadership structure and commitment, and securing financial support.

Recruiting PBRN members.

The idea for the ROCC came from a respected agency leader in the mental health community. He contacted the leadership of three other agencies and an academic partner. The status of this individual in the mental health community was crucial to the initial momentum. Selection of initial members was critical to the ROCC’s success. Before the PBRN was formed, each agency had successfully completed a pilot innovation project funded by the Los Angeles County Department of Mental Health. The ROCC agency leaders knew and trusted one another because of this prior work. The academic partner had a history of conducting research with the agencies, and this also fostered trust. An interest in innovative services, a shared belief in the importance of using evidence for agency decision making and in allowing staff the time for participation, and the fact that the agencies served similar client populations were also important criteria.

To solidify relationships among PBRN members (3), memorandums of understanding were created in July 2012 with defined roles and expectations for the partners. Each mental health agency committed four or five practitioners (for example, peer providers, case managers, and psychiatrists) to participate. The agencies offered practitioners incentives to participate by providing exemptions for some billing requirements to attend meetings, by leveraging the trust that CEOs had in their agency leaders, and by inviting members who were drawn to practice innovations.

Leadership structure.

The next step was forming and convening the steering committee, composed of one executive staff member from each agency and the research team. The research team consisted of the academic principal investigator, three academic and clinical postdoctoral fellows, and a research associate from the Southern California Clinical and Translational Science Institute (CTSI). This committee vetted ideas, provided direction, and ensured that network activities were generating value. A committee member from one agency was chosen to lead the PBRN. Once these roles were affirmed, the ROCC considered the strategic development of PBRN efforts, communication mechanisms, time and effort commitments, and network roles, as recommended by other established networks (3,5,6).

Connecting to PBRN mentors.

Because there are few PBRNs in mental health, finding a mentor with experience in a primary care PBRN was invaluable for orienting members to PBRN strategies. We registered with AHRQ and attended PBRN conferences to learn more about PBRN operations before our first project.

Initial financial support.

Securing funding for an initial project and for ongoing maintenance is critical (3,5). The ROCC’s initial project was funded through the CTSI. General medical PBRNs can connect with other PBRNs for assistance, but this option is limited in mental health because there are fewer networks. The ROCC funds covered costs related to meetings and administrative support.

Second Phase: Development of Research Question and Method

The selection, design, and execution of a research project can be a significant challenge for PBRNs (3), but these steps are essential for consolidating the network. The ROCC used the development of the research study to acquaint members with each other's practices and to generate input from all participants on the initial focus of the PBRN.

Identifying topics to study.

The ROCC used the reflective practitioner method to generate, refine, and select a research topic. This process occurred during three full-day meetings over three months (fall 2012 and winter 2013), which could have been an obstacle for busy mental health agencies. However, participants reported satisfaction with this process because it provided an opportunity for staff and peers to interact. Several participants credited the relationship building and enthusiasm during this process for sustaining the PBRN during the waits for institutional review board approval and between meetings. Those in leadership positions noted early reservations about the time involved, but afterward they appreciated what had been accomplished.

The selected research question was “What is the impact of integrated health care services on emergency room and primary care usage, physical health, mental health, lifestyle, and overall satisfaction with services?” PBRN members noted that this topic was interesting to multiple agency stakeholders, and they felt that they were participating in “cutting edge” research relevant to many community-based mental health agencies.

Identifying a study method.

A card study method was chosen because it is a strategy for rapid data collection that is feasible for clinicians. It is administered by clinicians with a single data sheet (a card) that requires focus and brevity. Once the instrument was developed, the academic team offered feedback on its structure and content and led a small pilot study to assess its feasibility and clarity.

Third Phase: Study Execution

For this PBRN study, staff members from each agency approached consumers with serious mental illnesses to participate, and 237 consumers from the four agencies completed the card survey during September 2013 (7).

Despite the positive results of the initial PBRN activities, the ROCC had significant challenges to address. The time required to complete community-engaged research can be challenging for researchers and community members (3). Developing the research question and refining the card study took several months. Frequent delays, such as waiting for regulatory approvals, revising the study instrument, and conducting a pilot test are familiar to academic researchers; however, this can be frustrating to community providers. Conversely, the time required to involve agency members in the development of a research question can be frustrating to researchers.

Using agency providers to administer card surveys had benefits and challenges. Providers had to complete training in responsible research conduct, taking time away from practice hours. On the other hand, the response rate was exceptionally high, probably because the staff were well known to the consumer participants, and staff were highly motivated to facilitate study participation.

Fourth Phase: Consolidation and Internal Reflection

Dissemination of findings.

One major goal of PBRNs is to generate knowledge that is immediately relevant to the clinical and administrative functions of usual-care practice; however, in a national survey of PBRNs, only about 50% reported being actively involved in a research project and over 25% of PBRNs had not yet completed a research project (2). Overall, the results of the ROCC study indicated that a variety of innovative strategies delivered by mental health providers can have a positive impact on the health of individuals with serious mental illness (7). Because the data were easy to input and manage, there was quick turnaround to the ROCC members. Each agency used the results for internal purposes, such as building morale among clients and staff and reporting to boards and external funders, and for public relations.

Results of the card study (reported in a letter to the editor in this issue [7]) were reported to the entire ROCC. The ensuing discussion included various perspectives on interpretation and dissemination of findings. This discussion framed the results, built consensus, and created enthusiasm for the ROCC. Several mechanisms were used to build awareness and ensure sustainability, including a press release to local media, announcements in agency and national newsletters, and presentations at local and national practitioner conferences. Academic partners assisted with opportunities for publishing and presenting in academic settings.

Internal reflection and adjusting expectations.

Feedback from PBRN members can provide important insights. After the initial card study, all ROCC members met to reflect on the process and outcomes of their efforts. Several key issues emerged. The common experience with integrated care projects allowed the ROCC to launch a card study that was responsive to the needs of the leadership, practitioners, and clients of the agencies. Actively engaging direct service providers and clients in the selection of the research question resulted in enthusiastic participation during the entire process, including collection of all surveys in four weeks. Agency leaders and providers also saw this study as relevant to the survival of their organization because it showed that mental health agencies can be the locus for integrating health and mental health services.

A notable challenge was research question selection. Even though practitioners were informed in advance that, the selected question would not resonate with everyone, a small number reported frustration that their topic was not chosen. The PBRN discussed this challenge and agreed to find a topic for the next study that satisfied the concerns of these staff members. However, this challenge may have been the result of the reflective practitioner process because it was not noted by other PBRNs.

The remaining participant concerns involved communication and leadership. During the ROCC’s development, the primary leader moved on, as did his replacement. These disruptions coincided with decreased communication within the PBRN. The need for a consistent, inspiring leader to be a champion for the PBRN became apparent. However, the enormous interest among practitioners in gathering evidence for their practices and the strength of relationships built during the reflective process sustained participants’ enthusiasm as new leadership emerged.

Expansion and sustainability of the ROCC.

Findings from the initial study were used for marketing the PBRN to potential members as a way to improve sustainability. Plans for securing financial support are ongoing. As they mature, PBRNs often obtain funding for projects to help fund studies—and also, we hope, to fund the infrastructure to maintain the network continuously.

Conclusions

The ROCC is one of the few PBRNs focused on mental health issues and, to our knowledge, the only one focused on services to persons with serious mental illness. Given the expansion of mental health services under the Affordable Care Act, it is critical to foster the development of PBRNs in mental health. We have found that a PBRN can be a critical mechanism for bridging the gap between research and practice and for advancing the national translational research agenda.