Evaluating the Implementation of Home-Based Videoconferencing for Providing Mental Health Services

Insufficient access to care is one of the most pressing problems in meeting the mental health needs of the U.S. population (1). The Veterans Health Administration (VHA) has addressed the problem, in part, with telehealth that uses videoconferencing to allow a provider at one clinic to provide services to a patient at another clinic that is closer to his or her home (2,3). More recently, the VHA has implemented video-to-home (V2H) telehealth, which allows patients to connect with a provider from their own homes by using a computer Webcam. This technology holds promise for significantly improving treatment access because it completely eliminates geographical and travel barriers.

The VHA implementation of V2H has occurred during a time when evidence supporting the delivery of mental health care through home-based telehealth is expanding rapidly. Earlier evidence indicates that the use of telehealth technology to link a patient in one clinic to a mental health provider in a separate clinic is feasible and acceptable and provides clinical benefit (4–8). More recent studies have examined home-based technologies, such as V2H, and show similarly promising outcomes (9–11).

This new modality will radically transform the delivery of mental health services, which have long emphasized face-to-face visits, and some reports indicate that clinicians are cautious about adopting this new technology (3,12). Research is therefore needed on issues surrounding the implementation of home-based telehealth. Such research can inform ongoing VHA implementation efforts, as well as those of other health care systems. Responding to this need, we conducted a formative evaluation of V2H implementation for mental health services (13). The first aim focused on identifying barriers and facilitators encountered by staff with various levels of experience with V2H technology. The second aim focused on describing barriers and facilitators in implementing V2H services encountered by three facilities with varying numbers of V2H visits and growth in visits.

Methods

Data collection used a concurrent, embedded mixed-methods design (14), purposively sampling three facilities based on facility-level quantitative data. We conducted semistructured interviews with staff at a facility with a high number of V2H visits in fiscal year (FY) 2015 (facility 1), a facility with a large increase in V2H visits from FY 2014 to FY 2015 (facility 2), and a facility that was low on both of these indicators (facility 3). These facilities served urban, suburban, and some rural (<3% of patients) areas in the Northeast. Data were analyzed by using the Consolidated Framework for Implementation Research (CFIR) (15) with previously published methods (16). The CFIR is a consolidation of various implementation science theories and is organized into five major domains (intervention characteristics, outer setting, inner setting, individuals, and implementation process), with each domain comprising several constructs.

Participants

Participants were VHA mental health providers from a wide array of programs who were familiar with or had been assigned to or encouraged by local leadership to deliver V2H services as well as facility staff who assisted with implementing V2H telehealth. Providers who had not yet provided V2H services were not excluded, allowing a greater range of V2H experience to be assessed. A snowball sampling strategy was used, beginning with candidates suggested by regional-level staff, followed by candidates suggested by local telehealth staff, and finally by candidates suggested by the participants themselves. Invitations were e-mailed and included key informed-consent elements (study purpose, procedures, risks, benefits, and rights). All interviews were preceded by verbal consent and permission for audio recording. This study protocol was given exempt status by the local institutional review board. Figure 1 summarizes the timeline of V2H implementation within this VHA region as well as the study timeline. Study interviews were conducted between September 2014 and April 2015.

Procedures

The CFIR contains 39 constructs across five domains that may be associated with implementation outcomes. To focus on the constructs most relevant to V2H implementation, a questionnaire listing all 39 constructs was completed by implementation staff (N=6) prior to initiating the study. Nineteen constructs spanning all five CFIR domains were rated as most relevant to V2H implementation, and they were included in the semistructured interview. Interviews ranged from 30 to 60 minutes. [The interview guide is available as an online supplement to this article.]

Qualitative Data Coding

Interview recordings were transcribed and imported into a qualitative analysis software program (Atlas.ti version 6.2). The analytic team consisted of the lead investigator (AI) and two individuals (AK and LSH), who identified the CFIR constructs (coding) and evaluated the degree to which the construct affected V2H implementation (rating). The lead investigator had previous experience in qualitative research and trained the other members of the analytic team. All team members had previous experience in qualitative research. The team members independently read each transcript and deductively coded CFIR constructs. A coding guide was developed with definitions for each CFIR construct to guide subsequent coding. When definitions were refined, previously coded transcripts were reviewed again to ensure that consistency remained. After all CFIR coding was completed, a valence rating method was used to quantitatively capture the degree to which CFIR constructs affected V2H implementation by three groups of individuals with various levels of V2H experience and each of the three facilities. Valence ratings indicated whether the implementation impact was strongly negative (−2), negative (−1), neutral or mixed (0), positive (+1), strongly positive (+2), or not ratable because of insufficient information (M). This method was previously published with more extensive description (16). [A more detailed summary of the valence rating method and of aggregate valence ratings for constructs that were rated differently by experience level and facility is available as an online supplement to this article.]

All codes and valence ratings were assigned by consensus among at least two analysts. Initial transcripts were coded and rated by all team members to establish a shared understanding of the constructs. Once that had been established, two analysts continued with the transcripts independently. Agreement was reviewed and consensus was used in instances of disagreement, with a third rater contributing to the consensus process if necessary. In addition, external collaborators (CR and LD) with expertise in this study’s particular methodology and CFIR constructs were occasionally consulted.

Results

A total of 56 individuals were invited to participate in the study, and 33 agreed to participate. Table 1 summarizes the characteristics of the 33 participants. Some CFIR constructs were further subdivided into subconstructs that were specific to V2H telehealth. For example, we used equipment (for example, Webcams), technical support, dedicated time, and implementation staff as subcodes for the CFIR construct for available resources. Also, functionality problems (technical problems) emerged inductively from our analysis.

All CFIR constructs identified as relevant to V2H implementation, from the aim 1 and aim 2 analyses, are reviewed in the sections that follow, with construct names appearing in parentheses. They are also displayed in Table 2, along with strategies for addressing the issues associated with these constructs. Many of these recommendations were derived from expert consensus compilations of implementation strategies (17,18).

Aim 1: CFIR Constructs and Level of V2H Experience

Each participant’s level of V2H experience was identified based on his or her described experience with V2H telehealth (Table 1). Thirteen participants had no direct experience with providing V2H telehealth or its implementation but were familiar with the technology. Seven participants had limited experience with providing V2H telehealth, ranging from attempting to provide V2H services to having provided generally less than ten sessions, usually to one or two patients. Finally, 13 participants reported being involved with V2H implementation or making regular use of V2H telehealth with three or more patients, generally for more than ten sessions.

No direct V2H experience.

This group was distinguished by the greatest concerns about the fit between a videoconferencing modality and mental health practice (compatibility). Concerns were expressed that V2H services would be less conducive to developing a therapeutic alliance or may constrain a clinician’s ability to perceive nonverbal cues, such as trembling. Concerns were also expressed regarding the suitability of V2H telehealth for patients who are at high risk of suicide or who have psychotic symptoms.

Moreover, because mental health treatment often helps patients to counter avoidance patterns (for example, avoiding activity by staying at home worsens depression), other concerns pertained to using V2H technology in instances in which leaving the home more often is clinically indicated. Although these cautions were described by participants in each of the experience groups, the concern was most consistently described among those with no direct V2H experience. These participants also expressed the greatest need for knowledge about V2H setup and operation (self-efficacy), expressing a desire for step-by-step guidance should they decide to adopt the technology. “If you want people to use ‘telemental’ health, you need to have a very hands-on procedure in place so that someone can come, call, set it up for you, take you through it,” observed a provider with no V2H experience (participant 19, facility 2). Participants with at least limited V2H experience also described having these concerns before they used the technology.

Limited V2H experience.

These participants were notable for initial adoption of V2H services and exposure to logistical barriers, such as the time and number of steps to set up V2H technology (complexity). The process of V2H setup includes training, obtaining the equipment, software installations, and scheduling templates as well as assisting patients with setup at home. In the most problematic instances, participants noted that information for each step was often obtained from different sources (access to knowledge and information), with time lags in between and with relatively limited guidance. As one provider with limited V2H experience recalled,

Also, although providers with no direct V2H experience had not been exposed to functionality problems (technical problems), such difficulties begin to emerge among those with at least limited V2H experience. These included instances of difficulty connecting or of poor sound. Other challenges included finding patients who were interested in telehealth (instead of face-to-face contact) and who had the needed technology in their home (patient needs and resources [patient perspectives and barriers subconstruct]). According to a provider with limited experience,Altogether, the set of findings for participants with limited V2H experience indicate that once engaged to adopt V2H telehealth, providers will subsequently encounter a number of logistical barriers. Two quotes by a participant with limited V2H experience illustrate how logistical barriers can negatively affect enthusiasm for V2H services. “I love [V2H] for a lot of reasons,” he said. “I feel like . . .we’ve entered a different era of how we deliver services and . . . treatment is more patient centered.” Yet the same participant reported concerns about functionality problems: “I think there’s sort of a sense that beyond the first couple of meetings, . . . the effort and the concern about it working may not make it worth it at this point” (participant 28, facility 1).

Most V2H experience.

Because these participants had accumulated the most V2H experience, they reported more instances of patient satisfaction and being able to provide care to patients who otherwise had difficulty accessing care (patient needs and resources [patient perspectives subconstruct]). However, experienced participants reported the greatest frustration with functionality problems and perceived the availability of local technical support as being insufficient (available resources [technical support subconstruct]). Some of these participants continued to express concerns about the compatibility of delivering mental health services via V2H; however, they did not express these concerns as consistently compared with the group with no V2H experience (compatibility). Finally, participants who were experienced with V2H services were unique in having the greatest amount of protected time for initial setup and related V2H support activities (available resources [dedicated time subconstruct]). As one experienced V2H provider explained,

Aim 2: Factors Among Different Facilities

Facility 1 produced the largest number of V2H visits during FY 2015 (N=740), along with moderate growth in visits compared with FY 2014 (N=583, an increase of 27%). Facility 2 produced fewer V2H visits during FY 2015 (N=263) compared with facility 1, but showed high growth in visits compared with FY 2014 (N=172, an increase of 53%). Finally, facility 3 had the lowest number of visits in FY 2015 (N=171) and had the lowest growth in V2H visits compared with FY 2014 (N=166, an increase of 3%).

Facility 1.

Facility 1 was notable for having a team of providers who were detailed solely to provide mental health services by using telehealth technology and had protected time to do so; thus the dedicated time subconstruct of the available resources construct was rated as having a strong positive impact on V2H implementation at this facility. However, this team of providers was operating at capacity and other providers at the facility were less likely to use V2H, which limited growth. Facility 1 was also notable for engaging patients by promoting telehealth for mental health treatment at multiple community events and setting up a mental health referral process that prompted referring providers to consider appropriateness for telehealth (engaging patients). These and other efforts resulted in a systemwide process for identifying and engaging appropriate patients for V2H services. According to one provider, “We’re the access center of all mental health. . . . We do the screening, find out what’s going on and if [telehealth] is what they want” (participant 34, facility 1).

Facility 2.

This facility was notable for the presence of an active local champion (available resources [champion subconstruct]). In fact, some providers with no V2H experience reported that they were considering initial V2H adoption because of the efforts of the local champion. According to one provider with no V2H experience,

Several participants at facility 2 described a process to engage providers to adopt V2H (engaging providers). E-mails eliciting interest were sent to providers, and those who were interested were invited to a subsequent V2H in-service training. The training was followed by discussions that focused on areas of need and methods for making V2H available. Moreover, ongoing logistical support was provided by the local champion, in terms of explaining the process, demonstrations of the equipment, compiling resources, and follow-up assistance (available resources [implementation staff support subconstruct]). The availability of this support may have mitigated some of the barriers, such as setup complexity and technology problems, given that these barriers were reported to have the least negative impact at this facility.

Facility 3.

Participants at this facility reported the greatest difficulty with V2H setup (complexity) and the greatest frustration with functionality (functionality problems). These findings coincided with a perception of fewer supports for V2H telehealth in the areas of training, setup assistance, and technical support (available resources [technical support subconstruct]). Finally, participants from facility 3 acknowledged the potential benefits of V2H services (patient needs and resources [need subconstruct]). However, this facility served an urban, densely populated area, and participants reflected on the relative lack of geographical dispersion in their urban service area and a lower perceived need for V2H telehealth. This coincided with a concern that the implementation impetus was external to the facility (intervention source) and was driven by performance measures (external policy and incentives). Thus participants expressed a misalignment between local need for V2H services and the impetus for implementation. According to one provider with limited V2H experience,

Discussion

This formative evaluation identified several issues involved with gaining provider buy-in for adopting V2H telehealth, which is consistent with reports of provider misgivings about the changes that V2H services bring to mental health practice (3,12). Among the concerns reported by providers was reconciling home-based care with treatment goals that encourage leaving the home, goals that are common in treating depression or anxiety (19–21). Our study revealed various approaches to the issue of avoidance, even among providers who were experienced with V2H. Although some providers view avoidance as a V2H contraindication, others see telehealth as an opportunity to establish a critical link to treatment that can work with patients to reduce avoidance (such as participating in activities outside the home). This latter view is consistent with recent studies showing that V2H telehealth is effective for treating disorders where reduced avoidance is a key clinical goal (9–11).

A second issue involving provider buy-in pertained to the perceived local need for implementation. Specifically, responses at the site with the fewest V2H visits and the lowest growth in visits reflected that V2H services were less needed, given the site’s highly urban environment. Staff at this site also perceived that the impetus for implementation of V2H technology was external to the facility and was driven by performance measures. As a result, alignment between providers and leaders, considered a key factor for successful implementation, was lacking (22,23).

In addition to underscoring the importance of gaining provider buy-in, the results pointed strongly to the need for strong logistical support to providers and patients adopting V2H use. A key finding is that by itself, increased provider willingness to adopt the modality was insufficient to improve adoption. Instead, our results point to the need for guidance with setup, training, and ongoing operation of the technology. This is consistent with a previous description of efforts by the VHA to implement telehealth for mental health services, which noted the need for close collaboration with information technology resources (3). The lack of these supports may lead to missed opportunities and failure by providers to progress to sustained use.

In addition to examining the impact of CFIR constructs across sites, we examined constructs that were associated with various levels of V2H experience. This provided a glimpse of how site-level implementation helps providers progress from nonuse to sustained use. This process was particularly illustrated by constructs that differentiated groups with various levels of experience with V2H services and sites with varying numbers of V2H visits. For example, our analyses revealed that individuals with protected time for V2H services were most likely to have regular V2H experience and that the facility providing protected time for V2H services produced the greatest number of V2H visits. Complexity of V2H setup and functionality problems serve as additional examples of constructs that were associated with both provider experience and site-level use of V2H telehealth. Our study showed that barriers related to complexity and functionality are encountered when providers first progress to initial V2H use. Participants from the facility with the fewest V2H visits and the lowest growth in visits perceived the fewest available resources from implementation staff and the greatest impact from issues related to the complexity of setup and functional problems.

The key findings of this study indicate that implementation efforts for V2H services should concentrate on provider engagement and buy-in as well as logistical support. A climate of provider engagement that is driven by local leadership and a local champion helps achieve the necessary visibility for providers to consider V2H use and tailor implementation to local needs. These recommendations align with those of experts in implementation science, particularly those involving identifying and preparing a local champion, conducting educational meetings, and holding local consensus discussions (17,18). The second set of recommendations focuses on helping providers navigate the process of V2H setup and use. For provider setup, this study points to the value of a local point of contact who can provide ongoing, individualized facilitation. Consistent with expert recommendations, providers uniformly expressed a preference for local logistical support. Quality monitoring systems can track how well logistical support is provided by examining various indicators, such as the amount of time between a provider’s request for V2H setup and first use and occurrence of technological difficulties.

This study had limitations that are worth noting. First, compared with similar research (16), this study evaluated a relatively small number of sites, which constrains the degree of inference that can be made about facility-level factors. Thus current results should be interpreted as descriptive and as hypothesis generating. Second, the study did not directly evaluate patients’ perspectives. Third, because our study focused on staff perspectives, it did not measure the degree to which certain issues actually occurred (for example, technical problems). Fourth, our study was conducted in a region that includes some rural areas but serves mostly urban and suburban catchment areas. Thus our findings, especially those pertaining to low perceived need and challenges in finding interested patients, may not generalize to facilities with more rural catchment areas. Finally, our recruitment process focused on staff who were at least familiar with V2H services (even those with no V2H experience). As a result, a large percentage of facility staff was not included in our sampling strategy.

Conclusions

This study outlines provider and organizational considerations regarding the implementation of V2H services for mental health care. Key issues highlighted by the study include the need for provider engagement and buy-in efforts, especially efforts that address clinicians’ concerns about the compatibility between V2H technology and clinical practice. Also, several CFIR constructs emerged that point to the importance of logistical support, especially for providers who are newly adopting the technology. Other key issues involve implementing V2H resources in areas of greatest patient need and creating protected time for providers to adopt the technology.