The American Psychiatric Association (APA) has updated its Privacy Policy and Terms of Use, including with new information specifically addressed to individuals in the European Economic Area. As described in the Privacy Policy and Terms of Use, this website utilizes cookies, including for the purpose of offering an optimal online experience and services tailored to your preferences.

Please read the entire Privacy Policy and Terms of Use. By closing this message, browsing this website, continuing the navigation, or otherwise continuing to use the APA's websites, you confirm that you understand and accept the terms of the Privacy Policy and Terms of Use, including the utilization of cookies.

×

Abstract

Objective:

Implementation facilitation is an effective strategy that increases uptake of behavioral health interventions. Facilitation is grounded in partnerships with leadership and clinical stakeholders. Researchers have documented some negative consequences of facilitation—time, financial, and opportunity costs. Clinical leaders often agree to facilitation with the promise of increased implementation of an intervention. This study examined whether unintended positive consequences of facilitation might offset known costs.

Methods:

This study was part of a stepped-wedge, hybrid type 2, pragmatic trial that used implementation facilitation to implement primary care mental health integration (PCMHI) via telehealth technology in six satellite Veterans Health Administration (VHA) clinics. Two facilitators provided facilitation for at least 6 months. This study included a focused analysis of an emerging phenomenon captured through weekly debriefing interviews with facilitators: unintended positive consequences of implementation facilitation, termed “lagniappes” here. A rapid content analysis was conducted to identify and categorize these consequences.

Results:

The authors documented unintended positive consequences of the facilitation across the six VHA sites and categorized them into three clinically relevant domains: supporting PCMHI outreach at other clinics not in the original catchment area (e.g., providing tools to other sites), strengthening patient access (e.g., resolving unnecessary patient travel), and improving or modifying work processes (e.g., clarifying suicide assessment protocols). The positive consequences benefited sites and strengthened ongoing partnerships.

Conclusions:

Documenting unintended positive consequences of implementation facilitation may increase leadership engagement. Facilitators should consider leveraging unintended positive consequences as advantages for the site that may add efficiency to facility processes and workflows.

HIGHLIGHTS

  • Implementation facilitation, a strategy that increases uptake of evidence-based interventions, was used across six Veterans Health Administration clinics to support adoption of a telehealth intervention.

  • Facilitators who conducted debriefing interviews noted unintended positive consequences that could be categorized into three domains: supporting primary care mental health integration outreach at other clinics, strengthening patient access, and improving or modifying work processes.

  • Facilitators should consider leveraging unintended positive consequences (“lagniappes”) as advantages for implementation sites that may add efficiency to facility processes and workflows.

Implementation strategies seek to enhance uptake of interventions into clinical settings (e.g., audit and feedback and changing funding streams) (1, 2). Implementation facilitation, referred to henceforth as facilitation, is a strategy in which trained individuals (called facilitators) build relationships with local stakeholders and help sites identify and address implementation barriers in order to increase and sustain the uptake of an intervention (3). Facilitation can address barriers at multiple levels in health care systems and is particularly useful when implementing complex clinical programs. Facilitation can effectively increase the reach of an intervention to patients for whom it was intended and enhance adoption of an intervention by providers (4). Compared with simpler implementation techniques, facilitation improved provider adoption of a program to reengage patients with serious mental illness in services (5). The experience level and interpersonal relationships of facilitators with clinical stakeholders are important for successful implementation, because facilitation can rely on relationships to identify and reduce barriers (6, 7).

There is ample evidence for intended positive consequences of facilitation, including provider adoption of clinical interventions and increased reach of evidence-based practices to patients (4, 8, 9). Costs of facilitation, such as staff time and effort, financial investment, and opportunity costs, have also been documented (1012). These costs may make clinical leaders less likely to support facilitation for implementation, quality improvement, or systems redesign.

However, no reports exist of unintended positive consequences of facilitation, which may enhance a site’s likelihood of participation in implementation projects, because such unexpected benefits contribute to the health care system beyond the scope of the specified clinical intervention. Indeed, scholars have called for more documentation of unintended general consequences of implementation efforts (13). It is possible that unintended positive consequences of facilitation may offset its opportunity, time, and financial costs.

We employed facilitation as the primary strategy in a pragmatic hybrid type 2 effectiveness-implementation trial (14) of primary care mental health integration (PCMHI) delivered via telehealth (tele-PCMHI) (15). The setting was in the southern United States, within the Veterans Health Administration (VHA). During weekly project team meetings, two facilitators (E.N.W. and K.A.O.) provided updates on activities to the implementation team. Although most activities targeted implementation of tele-PCMHI, in some instances, facilitators reported uncovering and intervening in issues beyond the scope of the implementation effort. At first, we labeled these “unintended positive consequences” of the facilitation strategy, until one team member noted they were akin to “lagniappes” in parts of the southern United States.

Unintended Positive Consequences: Lagniappe

Lagniappe (pronounced lan-yap) is a Cajun term from New Orleans (16), rooted in American Spanish linguistic culture (17). A lagniappe in New Orleans now refers to the 13th roll sometimes given in a bread baker’s dozen (18). Generally, lagniappes are anything given as a bonus or gratuity, as “a little something extra.” Thus, we coined the unintended positive consequences in this study as lagniappes, which seemed particularly apropos given the southern context of the clinics in which facilitation was taking place.

We defined a lagniappe resulting from facilitation as an unintended positive consequence initiated directly or indirectly by facilitators in the course of using implementation strategies to increase uptake of a specific intervention (3). These unintended positive consequences represented an emerging phenomenon and provided potential or actual benefit in a way that was unforeseen and beyond the scope of facilitation goals to implement only a specific clinical intervention. A defining feature of lagniappes was their unintended effect, such that they addressed an issue outside the target of the facilitation effort.

The Intervention: Tele-PCMHI

We implemented a highly complex program to integrate mental health into primary care in rural clinics using telehealth technology. Within VHA, this intervention is referred to as PCMHI (19). Its overarching goals are to increase access to high-quality behavioral health care in the VHA and create an efficient stepped model of care, from primary care to PCMHI, if appropriate before (or in place of) referral to specialty mental health care (20). PCMHI involves blended collaborative care and care management models and an interdisciplinary team of mental health providers, including psychotherapists, prescribers, and care managers (21). This team provides several services, including same-day access for brief assessment or follow-up treatment, including brief psychotherapy, medication management, care management via telephone, or electronic consultation with primary care providers. In our telehealth model, PCMHI providers located at a large VHA medical center provided care via telehealth modalities to satellite VHA clinics.

If facilitation is successful, tele-PCMHI will allow implementation of integrated mental health services in rural and smaller clinics, where hiring specialists as staff is not feasible. The intended positive consequences of facilitation would be implementation of PCMHI core components, greater reach of mental health services to more patients, primary care teams’ referral of at least one patient to tele-PCMHI during the study period, and improved mental health among patients receiving tele-PCMHI (4, 20).

Methods

Study Design

The tele-PCMHI implementation trial is occurring in six sites over 24 months. The study was approved by the Central Arkansas Veterans Healthcare System Institutional Review Board from April 2016 through October 2020 and at the time of this writing is ongoing (15). The trial followed a stepped-wedge design in which study sites sequentially began implementation in three steps or phases—implementation planning, active implementation, and step-down—at approximately 6-month intervals (22). Facilitation occurred from January 2018 through April 2020. Facilitators completed a 2-day training and followed the Implementation Facilitation Training Manual developed by the VHA Behavioral Health Quality Enhancement Research Initiative (3). Facilitation involved many activities, such as 1-day site visits, biweekly implementation planning calls and sustainability planning with sites (e.g., clinic champions), and sharing updates with and incorporating feedback from primary care, telehealth, and mental health leadership. The study reported here was a focused analysis of lagniappes captured through part of a broader process evaluation to document facilitation activities. Data collection for the lagniappes project is complete (23).

We used an implementation science framework to inform facilitation focus and to document domains that lagniappes affected. The integrated Promoting Action on Research Implementation in Health Services (i-PARIHS) framework proposes that successful implementation of interventions results from facilitation of a given innovation (of an intervention, i.e., tele-PCMHI) with the recipients of the intervention (e.g., providers and patients) in a particular context (e.g., setting in which it will be implemented). In the i-PARIHS framework, facilitation is proposed as the essential ingredient promoting implementation success (6).

Sites

Sites included three rural and three urban clinics (community-based outpatient clinics), classified by the Department of Veterans Affairs Site Tracking system for 2019. These clinics primarily provided primary and mental health care in the southern United States.

Data Collection

As part of broader process evaluation of facilitation activities, one of the authors (K.L.D.), who is a medical anthropologist and implementation scientist, documented facilitation activities in detail through weekly implementation team meetings and debriefing interviews with facilitators (E.N.W. and K.A.O.). Facilitators also shared notes and reflections with the medical anthropologist via e-mail to provide supplemental information. During implementation team meetings in which facilitators discussed updates on activities, future planning, and issues, the medical anthropologist took detailed field notes of all conversations, including verbatim quotes. After each meeting, she also conducted unstructured debriefing interviews with both facilitators, creating a near-verbatim transcript typed in a document in real time. In these debriefings, facilitators shared detailed updates and reflections on facilitation activities at each site. The implementation team meetings and debriefing interviews lasted approximately 1 hour each, for a total 2 hours of documentation per week.

Data Analysis

We used a rapid content analysis technique to extract lagniappes as an emerging theme. Rapid analysis is a qualitative technique used in health services research that maintains rigor and produces quick turnaround of results, which is desirable when the results need to be fed into an ongoing implementation effort (24, 25). We maintained data reliability and validity in many ways. First, one author (K.L.D.) reviewed all documentation and extracted potential lagniappes into a single summary document and circulated it to both facilitators for confirmation (akin to member checking) (26). To enhance reliability, the authors on the project then met to discuss potential lagniappes and achieve consensus on inclusion of confirmed lagniappes in the final sample.

We then organized the lagniappes in two ways. First, we grouped lagniappes similar in type together to better describe “what” they achieved for clinical impact. For example, there were three instances at different sites in which facilitators “ensured accurate workload capture for PCMHI providers not directly providing care for tele-PCMHI.” Therefore, these three were grouped together in considering clinical impact. Two authors (E.N.W. and K.L.D.) independently reviewed lagniappes and met to discuss clinical-impact types of lagniappes for grouping. Next, we grouped the clinical-impact types into three clinically relevant domains: supporting PCMHI outreach at other clinics (outside the six study sites), strengthening patient access to other services, and improving or modifying work processes for providers outside PCMHI. First, two authors (E.N.W. and R.E.H.) independently developed preliminary domains, because these authors are clinical providers and thus understand the clinical relevance of each type of lagniappe. Then, two authors (E.N.W. and K.L.D.) refined the preliminary domains. We agreed that one lagniappe type affected two domains.

Second, we categorized each individual lagniappe by “where” in the i-PARIHS domains it had an effect (innovation, recipients, or context) and created a matrix organized by these domains. This matrix allowed us to understand where in the implementation landscape the lagniappes had a most direct effect. To enhance the reliability of categorization, two authors (E.N.W. and K.L.D.) independently categorized lagniappes, reached consensus, and then sent the list to a third author (K.A.O.) for review. To enhance the validity of the categorization, it was also reviewed independently by three authors involved in clinical operations: a frontline provider (R.E.H.) and two mental health clinical managers (S.S.M. and B.C.W.).

Results

We identified six lagniappe types during facilitation of the tele-PCMHI implementation, grouped into three clinically relevant domains: supporting PCMHI outreach at other clinics (outside the six study sites), strengthening patient access to other services, and improving or modifying work processes for providers outside PCMHI. Again, these were unintended positive consequences of facilitation not related to implementing tele-PCMHI at the six study sites. Table 1 presents the matrix of lagniappes and their impact.

TABLE 1. Matrix of lagniappes and their impact documented in a facilitated implementation of primary care mental health integration via telehealth (tele-PCMHI), categorized by i-PARIHS and clinically relevant domain

Domain where lagniappe was most impactfulaDomain describing clinical relevance of lagniappe
LagniappeInnovationRecipients (patients, staff, or providers)Local clinicParent hospital facilityVA or non-VASupporting PCMHI outreachStrengthening patient accessImproving or modifying work processes
Provided tele-PCMHI implementation tools for other clinics
Connected emergency department and PCMHI leads
Resolved mental health assessment delays and unnecessary travel for patients
Improved scheduling to maintain access and increase staff morale
Ensured accurate workload capture for PCMHI providers not directly providing care for tele-PCMHI
Clarified suicide protocol

aDomains are from the integrated Promoting Action on Research Implementation in Health Services (i-PARIHS) framework. PCMHI, primary care mental health integration; VA, Veterans Affairs.

TABLE 1. Matrix of lagniappes and their impact documented in a facilitated implementation of primary care mental health integration via telehealth (tele-PCMHI), categorized by i-PARIHS and clinically relevant domain

Enlarge table

Supporting PCMHI Outreach at Other Clinics

Provided tele-PCMHI implementation tools for other clinics.

Facilitators provided tools and information and completed tasks that benefited an impending larger rollout of tele-PCMHI for another facility. The facilitators resolved some issues that the other facility would have encountered in its larger rollout—specifically, by developing implementation checklists for the facility’s use and resolving and sharing information on technical changes affecting clinical documentation of care provided through telehealth technology.

Connected emergency department and PCMHI leads.

Facilitators were notified by medical center leadership that the emergency department wanted to refer patients to onsite PCMHI services (not telehealth) but that personnel were not aware of how to reach or use such services. Facilitators connected the leads from the emergency department and PCMHI, which encouraged interdepartmental collaboration and referral. This was especially important given new facility performance metrics that tracked how many patients from the emergency department were seen by PCMHI providers.

Strengthening Patient Access

Resolved mental health assessment delays and unnecessary travel for patients.

Patients at a rural clinic were driving more than 2 hours one way to the medical center to have mental health evaluations required for accessing mental health care at their rural clinic. Facilitators identified this barrier to accessing care and consulted with chief-of-staff leadership and a mental health clinical manager. As a result, the leadership added 2 days of psychiatric care via telehealth to the rural clinic, including mental health evaluations, which enabled evaluation of patients at their rural clinic without prior evaluation at the distant medical center.

Improved scheduling to maintain access and increase staff morale.

Facilitators enabled more backup scheduling capabilities for clinical staff members who were responsible for scheduling and documenting patient appointments. Previously, one staff member went on vacation only when the providers she assisted were also on vacation. To support her, facilitators advocated that scheduling capabilities be shared with more staff members to provide backup in her absence. Facilitators used this process at other sites to ensure adequate coverage for scheduling and documenting patient appointments in the event of staff illness or vacation.

Improving or Modifying Work Processes

Of note, we classified the lagniappe “improved scheduling to maintain access and increase staff morale” in two domains: strengthening patient access and improving or modifying work processes.

Ensured accurate workload capture for PCMHI providers not directly providing care for tele-PCMHI.

Workload capture in this environment is the use of a standardized metric to quantify a provider’s contribution to patient care within a capitated system; thus, workload capture is important for provider productivity monitoring. Facilitators helped resolve several issues related to missing or inaccurate workload capture for PCMHI providers not involved in delivering tele-PCMHI, including the following: a long-standing scheduling problem caused an onsite PCMHI provider’s workload to be missed, some pharmacists were not receiving workload credit for responding to electronic consultations, and PCMHI was mistakenly getting credit for a mental health therapist’s workload credit for providing specialty mental health services, resulting in this provider’s workload not being recorded accurately. In these instances, facilitators helped successfully advocate for, provide guidance on, or prepare documents for changing workload capture to make it more accurate.

Clarified suicide prevention protocol.

At one site, facilitators identified a perceived lack of clarity on mental health emergency plans for non–mental health staff caring for patients who presented with suicide risk. In fact, one staff member described significant grief and guilt about a patient who had completed suicide 3 years before the facilitation site visit. Facilitators discussed this with medical center leadership, which resulted in a site visit from a mental health clinical manager who ensured that all staff at the clinic could discuss their concerns about the deceased patient, understood the policy, and felt supported when patients presented with suicide risk. Then the clinical manager began working with other rural clinics on this issue—and by the time of site visits to clinics conducted during the second step of the trial, the manager had clarified proper suicide risk protocols with all clinics.

Also, Table 1 indicates in which i-PARIHS domain the impact of lagniappes was felt. Only once did a lagniappe affect the broadest level of the health care system. Lagniappes most often benefited recipients, such as patients and providers, of tele-PCMHI at other clinics or PCMHI (not via telehealth); local clinics, in a way unrelated to tele-PCMHI; and the parent hospital facility to which clinics belonged.

Discussion

This study reveals unintended positive consequences of implementation facilitation that influenced helpful changes or shifts at various levels of the health care system. Facilitation that is used outside research projects can also produce lagniappes. We know that this is important within our own health care system (the VHA), where operational leadership has sometimes used facilitation as a strategy for implementation efforts (27). Therefore, documenting lagniappes applies also to implementation efforts that are not part of research.

All facilitators can begin to identify, document, and categorize the impacts of lagniappes to identify unexpected gains for their clinical, community, or operational partners. Researchers may wish to apply i-PARIHS or other implementation determinant frameworks to further categorize or analyze lagniappes, as we did in our methods. This may assist implementation researchers in assessing whether our results are replicable and generalizable. The lagniappe concept can also be used by training facilitators so that these positive consequences can be recognized and documented as they arise. If more data on lagniappes are assembled, predictable domains that are affected might emerge, such as access to care, process improvement, or employee morale. Lagniappes might have predictable impact on different levels of the health care system.

Lagniappes most frequently affected recipients (patients, staff, or providers) and clinical context (local clinics or the parent hospital facility). An interesting finding was a lack of a confirmed effect of lagniappes on the outer context of the health care system, at large or beyond; this may have been because facilitators worked mostly at local clinics and their parent facility.

It might be useful to begin tracking unintended positive consequences during facilitation as secondary implementation outcomes and also to categorize them by who primarily receives their benefits or where their impact occurs and whether an implementation determinant framework is used or not. We expected facilitation to enhance implementation outcomes for PCMHI at the study sites, but we did not anticipate improvement of outcomes at other sites (not included in the study) or other interventions (i.e., other than PCMHI). Documenting lagniappes during facilitation would allow evaluation of their impact on recruitment of implementation sites and on implementation outcomes. If indeed the relationship between facilitators and stakeholders is the mechanism driving implementation change, perhaps lagniappes have a positive impact on those key relationships. Thus, lagniappes may boost implementation outcomes via improved facilitator-stakeholder relationships. This is an empirical question for further study. Future efforts might involve circling back to stakeholders, describing documented lagniappes, and asking for their feedback on how the lagniappes affected stakeholder views of the value of facilitation.

Our study was limited by the small sample of lagniappes that we documented. Although we tracked them over a sizable period—18 months—only six lagniappe types were documented. Therefore, our results are preliminary, and more work is needed to expand their validity. In addition, we cannot conclude that lagniappes may have generalizable positive impact on recruitment of sites or implementation outcomes (e.g., adoption and reach); these unresolved questions would be excellent areas to study next.

Conclusions

Lagniappes are unintended positive consequences of implementation facilitation and may improve uptake of behavioral health programs. The lagniappes documented in this study were not simply bonuses for sites. Our clinical partners (including S.S.M. and B.C.W.) suggested that these unintended positive consequences also galvanized the collaborative relationships between facilitators and clinical staff or operational leadership. Lagniappes truly demonstrate that facilitation can add value for members of the implementation teams. An important point that one clinical leader emphasized to facilitators is not only to identify opportunities for lagniappes but also to introduce facilitation carefully. When facilitators can approach clinical staff or leadership with grace and a collaborative spirit that is characteristic of facilitation, opportunities to produce lagniappes and solutions become more likely. Approaching clinical staff or leadership without the emphasis on collaboration may lead to defensiveness and, perhaps, to the end of the collaboration.

Center for Mental Healthcare and Outcomes Research, Central Arkansas Veterans Healthcare System, North Little Rock (Woodward, Drummond, Oliver, Bartnik, Owen); Department of Psychiatry, University of Arkansas for Medical Sciences College of Medicine, Little Rock (Woodward, Drummond, Owen, Kirchner); Mental Health Service, Central Arkansas Veterans Healthcare System, North Little Rock (Meit, Wright); Mental Health Service, Jesse Brown VA Medical Center, Chicago (Hicks); Program for Team-Based Behavioral Health, Quality Enhancement Research Initiative, Department of Veterans Affairs Little Rock Regional Office, North Little Rock, Arkansas (Kirchner).
Send correspondence to Ms. Bartnik ().

The findings were previously presented at the Annual Conference on the Science of Dissemination and Implementation in Health, Washington, D.C., December 3–5, 2018, and at the International Conference on Practice Facilitation, Tampa, Florida, December 10–11, 2018.

The research study was funded through grant CRE 12-130 from the Collaborative Research to Enhance and Advance Transformation and Excellence Program, Veterans Health Administration, and by grant QUE 15-289 from the Veterans Affairs’ Quality Enhancement Research Initiative Program for Team-Based Behavioral Health for training and mentoring the implementation facilitators on this project. Drs. Meit and Wright were supported by the Mental Health Service at Central Arkansas Veterans Healthcare System.

Dr. Woodward reports serving as an implementation science consultant for ViiV Healthcare. The other authors report no financial relationships with commercial interests.

References

1 Powell BJ, Waltz TJ, Chinman MJ, et al.: A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci 2015; 10:21Crossref, MedlineGoogle Scholar

2 Baker R, Camosso-Stefinovic J, Gillies C, et al.: Tailored interventions to address determinants of practice. Cochrane Database Syst Rev 2015; 17:CD005470Google Scholar

3 Ritchie MJ, Dollar KM, Miller CJ, et al.: Implementation Facilitation Training Manual: Using Implementation Facilitation to Improve Care in the Veterans Health Administration, version 2. Washington, DC, Veterans Health Administration, Quality Enhancement Research Initiative for Team-Based Behavioral Health, 2017. https://www.queri.research.va.gov/tools/implementation/Facilitation-Manual.pdfGoogle Scholar

4 Kirchner JE, Ritchie MJ, Pitcock JA, et al.: Outcomes of a partnered facilitation strategy to implement primary care-mental health. J Gen Intern Med 2014; 29(suppl 4):904–912Crossref, MedlineGoogle Scholar

5 Kilbourne AM, Almirall D, Goodrich DE, et al.: Enhancing outreach for persons with serious mental illness: 12-month results from a cluster randomized trial of an adaptive implementation strategy. Implement Sci 2014; 9:163Crossref, MedlineGoogle Scholar

6 Harvey G, Kitson A: Implementing Evidence-Based Practice in Healthcare: A Facilitation Guide. New York, Routledge, 2015CrossrefGoogle Scholar

7 van der Zijpp TJ, Niessen T, Eldh AC, et al.: A bridge over turbulent waters: illustrating the interaction between managerial leaders and facilitators when implementing research evidence. Worldviews Evid Based Nurs 2016; 13:25–31Crossref, MedlineGoogle Scholar

8 Seers K, Rycroft-Malone J, Cox K, et al.: Facilitating Implementation of Research Evidence (FIRE): an international cluster randomised controlled trial to evaluate two models of facilitation informed by the Promoting Action on Research Implementation in Health Services (PARIHS) framework. Implement Sci 2018; 13:137Crossref, MedlineGoogle Scholar

9 Nagykaldi Z, Mold JW, Robinson A, et al.: Practice facilitators and practice-based research networks. J Am Board Fam Med 2006; 19:506–510Crossref, MedlineGoogle Scholar

10 Ritchie MJ, Liu CF, Townsend JC, et al.: Time and cost of “extreme” implementation facilitation to address challenging clinical contexts. Presented at the 4th Biennial Society for Implementation Research Collaboration, Seattle, Sept 8–9, 2017Google Scholar

11 Liu C-F, Rubenstein LV, Kirchner JE, et al.: Organizational cost of quality improvement for depression care. Health Serv Res 2009; 44:225–244Crossref, MedlineGoogle Scholar

12 Sharp ND, Pineros SL, Hsu C, et al.: A qualitative study to identify barriers and facilitators to implementation of pilot interventions in the Veterans Health Administration (VHA) Northwest Network. Worldviews Evid Based Nurs 2004; 1:129–139Crossref, MedlineGoogle Scholar

13 Rycroft-Malone J, Seers K, Crichton N, et al.: A pragmatic cluster randomised trial evaluating three implementation interventions. Implement Sci 2012; 7:80Crossref, MedlineGoogle Scholar

14 Curran GM, Bauer M, Mittman B, et al.: Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care 2012; 50:217–226Crossref, MedlineGoogle Scholar

15 Owen RR, Woodward EN, Drummond K, et al.: Using implementation facilitation to implement primary care mental health integration via clinical video telehealth in rural clinics: protocol for a hybrid type 2 cluster randomized stepped-wedge design. Implement Sci 2019; 14:33Crossref, MedlineGoogle Scholar

16 Merriam-Webster Dictionary: Lagniappe. Springfield, MA, Merriam-Webster, Inc. https://www.merriam-webster.com/dictionary/lagniappe. Accessed Feb 1, 2019Google Scholar

17 Bryson B: Made in America: An Informal History of the English Language in the United States. New York, Avon, 2001Google Scholar

18 Troy E: What is a Lagniappe? Culinary Lore, March 10, 2015. https://culinarylore.com/food-history:what-is-a-lagniappe. Accessed March 1, 2019Google Scholar

19 Post EP, Van Stone WW: Veterans Health Administration primary care-mental health integration initiative. N C Med J 2008; 69:49–52MedlineGoogle Scholar

20 Dundon M, Dollar K, Schohn M, et al.: Primary Care-Mental Health Integration Co-Located, Collaborative Care: An Operations Manual. Washington, DC, US Department of Veterans Affairs, Center for Integrated Healthcare, 2011. http://www.mentalhealth.va.gov/coe/cih-visn2/Documents/Clinical/Operations_Policies_Procedures/MH-IPC_CCC_Operations_Manual_Version_2_1.pdfGoogle Scholar

21 Uniform Mental Health Services in VA Medical Centers and Clinics. Washington, DC, US Department of Veterans Affairs, Veterans Health Administration, 2015. https://www.va.gov/vhapublications/ViewPublication.asp?pub_ID=1762. Accessed April 21, 2020Google Scholar

22 Brown CA, Lilford RJ: The stepped wedge trial design: a systematic review. BMC Med Res Methodol 2006; 6:54Crossref, MedlineGoogle Scholar

23 Stetler CB, Legro MW, Rycroft-Malone J, et al.: Role of “external facilitation” in implementation of research findings: a qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implement Sci 2006; 1:23Crossref, MedlineGoogle Scholar

24 Beebe J: Rapid Assessment Process: An Introduction. Walnut Creek, CA, AltaMira Press, 2001Google Scholar

25 Hamilton A: Qualitative methods in rapid turn-around health services research. Presented at the US Department of Veteran Affairs HSRD Cyberseminar, Dec 11 2013Google Scholar

26 Birt L, Scott S, Cavers D, et al.: Member checking: a tool to enhance trustworthiness or merely a nod to validation? Qual Health Res 2016; 26:1802–1811Crossref, MedlineGoogle Scholar

27 Ritchie MJ, Dollar KM, Kearney LK, et al.: Research and services partnerships: responding to needs of clinical operations partners: transferring implementation facilitation knowledge and skills. Psychiatr Serv 2014; 65:141–143LinkGoogle Scholar