Telehealth Acceptability and Feasibility Among People Served in a Community Behavioral Health System During the COVID-19 Pandemic

Since social distancing became a part of the defense against the COVID-19 pandemic, many health care providers have closed their doors and deferred procedures to protect patients from COVID-19 infection (1–3), significantly altering how care is delivered. However, continued management of chronic diseases that increase COVID-19 risk, such as diabetes mellitus and hypertension, remains a crucial part of response to the pandemic. Because of a combination of comorbid conditions and adverse social determinants of health (e.g., housing instability, economic hardship, and social isolation), people with serious mental illness may be at particular risk for increased COVID-19–associated morbidity and mortality and for the adverse social and psychological effects of the pandemic (4). In many states, mental health providers responded to the COVID-19 emergency with rapid and widespread transition to telehealth, supported by relief from many regulatory and legislative barriers established before COVID-19 (5).

Many people with severe mental illness require complex multidisciplinary care management with frequent engagement to help them recover, maintain a degree of independence, and avoid hospitalization (6, 7). For individuals served by teams of mental health professionals, such as in assertive community treatment (ACT) or supported housing, these teams may be the greatest potential source of COVID-19 transmission risk. Transition of these treatment models to telehealth became essential during the pandemic but faced unique barriers.

A growing body of evidence supports the notion that with regard to outcomes and patient satisfaction, telepsychiatry is comparable to face-to-face psychiatry in clinical settings (8–10). A subset of evidence has examined telepsychiatry for the management of serious mental illness, finding that most clients appreciate these services (6, 11–14). Social isolation is a common problem among people with serious mental illness, sometimes compounded by paranoid ideation or negative symptoms (15–25). It is possible that being able to connect with mental health professionals without leaving the home or receiving visitors may offer some relief.

Available studies most frequently involve telepsychiatry offered through videoconferencing or smartphone technology augmenting primarily face-to-face treatment. However, low computer literacy and limited access to Internet technology are prevalent among individuals with serious mental illness and represent barriers to telehealth access. Older individuals may experience even greater issues with telehealth access because of difficulties with hearing, speech, vision, and cognitive impairment (21). When New York State expanded regulations to allow its licensed programs to bill Medicaid for telehealth visits in March 2020 in response to the COVID-19 pandemic, telephone-based interventions were permitted to address technology barriers. Little is known about client preferences and experiences regarding telephone-based services, particularly among people with serious mental illness.

In this study, we examined the acceptability and feasibility of telehealth service provision shortly after its introduction in a community behavioral health agency. In particular, we examined client subjective experience, preferences, and satisfaction with services across behavioral health program types, and we compared their responses with those from a 2018 sample. We also examined utilization patterns for these behavioral health services and for emergency and hospital services.

Methods

Population

We invited individuals receiving services from May 8 to 18, 2020, in programs that had introduced telehealth services during the COVID-19 pandemic at a New York City community behavioral health agency. The total census of these programs was 6,546, which included 2,668 individuals in treatment programs (mental health clinics, ACT, Intensive Mobile Treatment [IMT] program, and Personal Recovery Oriented Services [PROS] day program), 2,540 in outreach programs (including care coordination and several small program types), and 1,796 in housing programs (including Treatment Apartment Program [TAP] and supported housing). Some individuals were enrolled in more than one program. For the 6,546 individuals served by these programs, most had a diagnosis of serious mental illness, with 39% (N=2,553) diagnosed as having a psychotic disorder and 41% (N=2,684) diagnosed as having mood disorders. Most participants identified as Black (60%, N=3,929), with 22% (N=1,440) identifying as Hispanic, 14% (N=916) identifying as White, 3% (N=196) identifying as multiracial, and 1% (N=65) identifying as Asian. Participants had a mean±SD age of 45±17 years (range 5–93). For 233 minors among the individuals served, their guardians were asked to take the survey on their behalf. This study was performed as part of routine agency performance improvement. Institutional review was not required.

Interview Procedures

The survey was administered by clinicians and case managers during scheduled or routine meetings with participants via the telephone, by videoconferencing call, or in person during the study period. Clinicians and case managers were provided with a script to introduce the survey, which explicitly stated that the survey was voluntary. The survey took approximately 5 minutes to complete, and the clinicians and case managers recorded the survey responses in SurveyMonkey.

Measures

The agency developed a brief survey to measure participants’ experience with telehealth services. The first three survey items measured perceived level of connection and support experienced before the pandemic compared with the present. The next four items were extracted from the agency’s annual consumer satisfaction survey to allow for historical data comparison. The next item asked whether the individual would like to continue receiving services remotely after the pandemic ended. If the response was yes, the individual was asked whether they would prefer all telehealth or a mixture of telehealth and in-person service. The next item asked what type of device was used to engage in telehealth (e.g., phone or laptop). The clinician or case manager was then asked to rate their perception of the individual’s level of distress related to the COVID-19 pandemic.

Monthly aggregate service utilization data for Medicaid-reimbursable programs, including clinics, ACT, PROS, and TAP, were extracted from electronic health records to examine changes in client access to and engagement in telehealth and face-to-face services during the period immediately before and after the beginning of the pandemic (February 1–April 30, 2020). Self-reported emergency and inpatient utilization data were obtained from the electronic health record’s Healthy Living Questionnaire, administered quarterly or semiannually in all participating programs other than IMT and care coordination (for the periods January–April 2019 and January–April 2020), to enable comparison.

Data Analysis

All collected data were coded and recorded into a computer data file and analyzed with IBM SPSS Statistics, version 26. Data were aggregated by program type (treatment, outreach, and housing), and ordinal logistic regressions were used to test for differences among program types on responses to the 5-point, Likert-rated survey questions (1, a lot less than before; 2, a little less than before; 3, the same as before; 4, a little more than before; 5, a lot more than before). Ordinal logistic regressions were used to identify differences between historical responses to the client satisfaction survey items in 2018 and responses during the study period in 2020. Ordinal logistic regressions were also used to determine whether provider-rated client distress due to the pandemic was correlated with experience of services and satisfaction. Results were considered statistically significant at p<0.01 to account for multiple testing. Service use frequency data were summarized in graphs.

Results

Survey Participation

In total, 1,482 individuals responded, including 363 (25%) in treatment programs, 225 (15%) in outreach programs, and 894 (60%) in housing programs and representing 38% (N=3,933) of individuals scheduled for visits during the study period. (Estimated survey participation rates by program type can be found in an online supplement to this article, with a higher participation rate in housing programs than in other programs.)

Comparison of Responses by Program Type

Survey responses are detailed in Table 1 and separated by program type. For the overall sample, most participants responded that their ability to connect to staff (85%, N=1,260), get support when needed (90%, N=1,334), and get an appointment when wanted (82%, N=1,215) was the same as or more than before the pandemic.

We conducted ordinal regressions to determine whether program type affected responses to the survey questions (for all odds ratios [ORs], see the online supplement). Individuals in the housing and outreach groups were less likely than those in the treatment group to report, “I can get an appointment when I want” (housing: OR=0.64, p<0.001; outreach: OR=0.36, p<0.001). Clients in the housing group were more likely than clients in the treatment group to report that staff were sensitive to traumatic experiences (OR=2.11, p<0.001), and staff in that group were more likely to note that clients were in distress (OR=1.77, p<0.001). Participants in the outreach group were less likely than those in the treatment group to report, “The quality of my life is improving” (OR=0.58, p<0.001).

Comparison of Client Satisfaction Survey Items by Year

Responses to questions from the annual client satisfaction survey for the 2018 versus the 2020 samples are detailed in Table 2. We used ordinal logistic regressions to determine possible relationships between year and survey responses (for all ORs, see online supplement). Clients were more likely to report that their quality of life was improving in 2020 than in 2018 (OR=1.28, p<0.001).

Comparison of Survey Responses by Staff-Reported Client Distress

Staff reported that clients were distressed (very mildly to severely) by the COVID-19 pandemic in 60% (N=889 of 1,482) of the survey responses. Survey responses for clients reported to be distressed versus not distressed are shown in Table 3. Ordinal logistic regressions were used to determine relationships between distress and survey responses (for all ORs, see online supplement). Clients who reported not being distressed were more likely than those who reported being distressed to agree that they feel connected to staff (OR=1.38, p=0.001), that they are able to receive support when needed (OR=1.45, p<0.001), that they are able to get an appointment when wanted (OR=1.49, p<0.001), and that their quality of life is improving compared with before the pandemic (OR=1.86, p<0.001).

Continuation of Telehealth and Use of Technology

Of all respondents, 1,182 (80%) indicated that they would be interested in continuing to receive services remotely after the COVID-19 pandemic ended, with 984 (83%) of those interested in remote services preferring a mix of remote and in-person services, and 198 (17%) preferring all services to be completed remotely. No significant differences were found by program type. The devices used for telehealth included smartphones (53%, N=785), basic cellphones (29%, N=430), landline phones (16%, N=237), and computers (2%, N=30), and almost all services were provided via audio calls and not videoconferencing.

Service Utilization

The number of face-to-face and telehealth services provided from January to April 2020 was obtained for a subset of Medicaid-reimbursable programs, showing an increase in overall service provision for ACT but a large decrease for PROS; clinics and TAP remained roughly stable (Figure 1). The frequency of self-reported emergency and inpatient service uses is shown in Figure 2. We observed a slight decrease in use for mental health reasons and an increase in use for general medical health reasons between 2019 and 2020.

Discussion

This investigation of clients’ perceptions regarding telehealth is notable for its focus on people with serious mental illness across multiple program types, including office- and community-based care and some longitudinal data comparisons. The rapid transition to telehealth in March 2020 in response to the COVID-19 pandemic bypassed the usual processes by which behavioral health agencies seek client buy-in before implementing new services. Consistent with previous studies of telepsychiatry (6, 11–14), the high level of perceived support and satisfaction with telehealth services, largely regardless of program type, is promising, with no worsening of satisfaction noted during the pandemic in 2020 and compared with historical (i.e., 2018) responses.

Mental health care settings that have made large-scale use of telehealth are increasingly confident in developing and sharing adaptations to virtual care delivery (23), which can be used to accommodate the significant number of clients who requested to continue exclusively with telehealth services after the pandemic. However, for some of these clients, this preference may not be clinically viable. For example, clients requiring direct inspection of living conditions, visualization of mental status, and private locations for telehealth visits or accommodations for hearing or speech disabilities or cognitive impairment, cannot be adequately served without face-to-face visits at least some of the time.

Telehealth may augment face-to-face visits, and determining the “dosing” of telehealth visits will require more experience and research (6). Examination of the impact of videoconferencing versus telephone visits, and of the replacement of office versus field visits, will be important. Notably, replacing office visits with videoconferencing visits may offer a glimpse of a client’s home life that enhances a provider’s understanding of the client’s situation, whereas when replacing a field visit with a phone call, it is likely that this information will be lost. There may also be differences when a client interacts with one provider via telehealth versus when they are served by individual members of a team of providers.

We are not aware of other investigations into the impact of client distress on the perception of telehealth. The negative impact of staff-rated distress on client-rated perceived support suggests the need to increase support for clients perceived to be distressed. However, the relative stability of emergency and hospital use compared with the same period in 2019 is reassuring. Outcomes of other rapid transitions to telehealth also indicate the capacity for safe and equitable care delivery (24).

The stable or increased numbers of total visits for clinic, ACT, and TAP programs support the continued telehealth provision in these programs if the reimbursement for telehealth continues to be viable. It also suggests a benefit to some clients in increasing their access to care beyond prepandemic levels. Other venues have reported increases in outpatient attendance consistent with our results (23). Although individual telehealth care models showed promise, group models such as PROS and day programs do not easily lend themselves to a rapid transition to telehealth, as evidenced by decreased visit numbers. The culture change to provision of groups via phone or video call was very challenging, with availability of technology and the high degree of necessary coordination being the main barriers. After a group telehealth model is better established at this agency and others, comparison of telehealth satisfaction in individual versus group settings will be useful.

A subset of the population lacked access to any communication device. To increase telehealth service participation, the agency provided 97 individuals with mobile phones and arranged use of communal phones and devices. As all programs begin to accrue the necessary technology, it seems likely that both the frequency and the quality of telehealth will increase. However, although many clients were noted to have smartphones, anecdotal reports suggest that data plans supporting video call capability are difficult to sustain financially for most of the people in this population. This assertion needs to be examined in a follow-up survey, with further exploration of technology needs and opportunities. In the meantime, the feasibility and acceptability data from this study suggest that regulators and funders should continue to reimburse telephone-based services to support access to care among people with severe mental illness.

Although the sample size was large, the short survey period reduced the participation rate as a proportion of the total census, and a possible selection bias existed toward individuals who had made themselves available for telehealth services and who were willing to cooperate with a survey administered in a telehealth session. A notable omission was that we did not record whether the survey was administered in person or via a telehealth session and whether it was administered via videoconferencing or a telephone call. Demographic data were also not recorded for survey respondents, limiting the ability to generalize the results to the total population. However, demographic data available for the total population do support generalizability to other behavioral health populations.

Another source of bias was that survey responses were recorded by staff interviewing their own clients; this procedure likely affected the correlation found between staff-rated client distress and the client’s satisfaction and quality-of-life responses. The challenges in obtaining data from a clientele who frequently lack Internet access and who have literacy issues continue to affect community research quality. In the case of the data used in the present study, it was important to obtain these data quickly to inform the agency’s practices and to advocate for regulatory changes to meaningfully improve community-based telehealth. This survey was administered only 2 months after telehealth began to be offered; repeated survey administration will be necessary to understand client preferences, outcomes, and risks in the longer term.

Conclusions

The findings of this study suggest that people who are served in a variety of behavioral health treatment programs find telehealth, including telephone-based services, to be an acceptable and even preferred mode of service delivery. In addition, service use numbers indicate its feasibility in individual care settings but possibly not in group programs. Continued investigation into the optimal dosing of face-to-face versus telephone and videoconferencing call services (in office-based, community-based, and group-based settings), as well as viable billing models for these services, will be needed to inform practice during and after the COVID-19 pandemic.