Procedural and Methodological Issues in Telepsychiatry Research and Program Development

Medical and mental health services often are inadequate in remote geographical areas with few specialty providers. Communications technology has been used to bring a variety of services to such areas (1,2,3), including clinical mental health care. In this review we use the term telepsychiatry broadly to refer to any form of mental health service delivered via remote videoconferencing technology.

Telepsychiatry offers hope for an affordable means of solving long-standing workforce problems in providing clinical, consultative, and educational services to populations in remote regions or to other isolated groups (4,5,6,7). The literature on telepsychiatry falls into three general categories: anecdotal accounts of novel clinical demonstrations, descriptions of program projects, and empirical studies of feasibility and effectiveness for general or specific populations. We review these reports, and after addressing their limitations, we discuss procedural and methodological issues that will shape future research.

Methods

In order to conduct a comprehensive review of the telepsychiatry literature, we conducted a search of the MEDLINE database, from 1970 to February 2000, using the keywords telepsychiatry, telemedicine, and videoconferencing. Studies were selected for review if they included the use of videoconferencing technology for the provision of any form of mental health care services. Some reports of program descriptions were eventually excluded if they were judged to overlap substantially with other reports of the same or similar types of regional programs.

Results

Novel clinical demonstrations

Many published reports on telepsychiatry are anecdotal accounts or case reports of novel applications. This kind of report is appropriate given how recently the technologies involved have been developed.

Case reports describe use of telepsychiatry for conducting interviews in community mental health clinics with patients who have schizophrenia (8,9,10,11,12); multiple-session evaluations for a range of disorders (13); ten sessions of psychological treatment of a female-male transsexual (14); 16 sessions of cognitive-behavioral treatment of a child with a disruptive behavior problem (15); a session of family therapy (16); treatment of a depressed Hispanic patient (17); clinical supervision and trauma debriefing of a remote mental health treatment team (18); psychiatric sessions using low-cost videophones (19); and a group treatment session for veterans with posttraumatic stress disorder (20). These reports provide good early evidence of the wide range of potential applications for telepsychiatry.

Program descriptions

Several larger-scale telepsychiatry programs have also been described in the literature. The largest of these programs, and the most thoroughly described, is an integral part of South Australia's Rural and Remote Mental Health Service (21,22,23,24,25,26). Since 1994 the program has used telecommunications for delivery of a wide range of clinical psychiatric services to community patients dispersed over nearly one million square kilometers. More than 2,000 clinical consultations—emergency services, inpatient liaison, postdischarge follow-up, and the like—were conducted in the first four years of the program's operation. More than 80 percent of the system's usage is for direct clinical purposes; clinical supervision is also provided.

Other general telepsychiatry programs have been described, including Oregon's RodeoNet and a program at the University of Kansas Medical Center (27,28,29); a program at the University College Hospital in Galway, Ireland, as a link to the island of Inishmore (13); the South Carolina Department of Mental Health's Deaf Services Program (30); programs in rural Appalachia (31), the Highlands of Scotland (32), and the U.S. Federal Bureau of Prisons (33); and programs at the University of Nebraska Medical Center (34), the University of Kentucky (35), Texas Tech University (7), the University of Oulu, Finland (36), and Cairns Base Hospital in Queensland, Australia (37). These programs have provided a wide range of services, including general adult and child psychiatric consultations and treatment of incarcerated inmates. Finally, brief surveys of active telepsychiatry programs have been published (4,38).

Prospective empirical evaluations

Reliability of clinical assessments. Several studies have furnished evidence of the reliability of psychiatric evaluations conducted by telepsychiatry. Baigent and colleagues (39) examined the interrater reliability of two psychiatrists who conducted semistructured interviews, including the Brief Psychiatric Rating Scale (BPRS), with 63 randomly assigned subjects in an observer-interviewer split configuration in remote and same-room settings. Although several differences between the telepsychiatry and same-room settings emerged— "degree of concern" for the patient and the frequency of some mental status findings were lower in the telepsychiatry settings—generally the diagnoses made were equally reliable in both settings.

Baer and associates (40) examined rating scales for obsessive-compulsive, depressive, and anxiety symptoms administered to 16 patients in a same-room setting and ten remote patients and found that reliability was excellent in both settings. Elford and coworkers (41) found an interrater agreement of 96 percent among same-room and remote diagnostic interviews conducted with 23 children ranging in age from four to 16 years.

Ruskin and associates (42) examined the interrater reliability of psychiatric diagnoses made by telepsychiatry and in same-room settings of 30 psychiatric inpatients using the Structured Clinical Interview for DSM-III-R. Fifteen patients had two same-room interviews, and 15 patients had one same-room and one remote interview by telecommunication. Interrater agreement was calculated for diagnoses of major depression, bipolar disorder, panic disorder, and alcohol dependence. The resulting reliability coefficients were nearly identical for the two groups.

Zarate and associates (43) used the BPRS, the Scale for the Assessment of Positive Symptoms (SAPS), and the Scale for the Assessment of Negative Symptoms (SANS) with 45 patients with schizophrenia assigned to one of three interview conditions: same-room setting, telepsychiatry at high bandwidth (384 kilobits per second [kbps]), and telepsychiatry at low bandwidth (128 kbps). Results showed that the global severity of schizophrenia and the overall severity of positive symptoms were reliably evaluated by both telepsychiatry conditions, although negative symptoms were less reliably assessed in the low-bandwidth condition. In addition, telepsychiatry was well accepted by patients, although they generally preferred the high-bandwidth transmission. Similar findings were obtained in another study with a small sample (N=7) (44).

Finally, remote cognitive examinations have been conducted with teleconferencing technology (45). Montani and associates (46,47) evaluated samples of ten and 15 elderly patients for cognitive deficits using the Mini Mental Status Examination (MMSE) and the clock face test. Each patient participated in two evaluations, conducted a week apart, one by telepsychiatry and the other in a same-room setting. Results showed that scores on both tests were lower in the telepsychiatry condition; the difference was small but statistically significant. The authors attributed the lower scores to patients' difficulties hearing and maintaining attention in telepsychiatry evaluations and concluded that telepsychiatry can be used reliably in the examination of elderly patients (46).

Ball and colleagues (48) demonstrated with a sample of 12 patients that the MMSE could be reliably conducted via remote technology. A study by Craig and associates (49) conducted with 23 patients produced evidence that even neurological examinations can be conducted reliably via remote technology.

Satisfaction and acceptance. Studies have demonstrated generally high rates of patients' and clinicians' satisfaction with telepsychiatry. Satisfaction has been high among incarcerated patients (28,32), hospitalized elderly patients (47), rural outpatients (21,36,50), clinician referrers (21), caregiver support groups (51), psychiatric inpatients (5,52), patients with schizophrenia (43), and rural children and their parents (30,31,41). Telepsychiatry interviewing has been well accepted even among patients with paranoid schizophrenia who had ideas of reference, with no exacerbation of delusional symptoms (52). In Australia, 90 percent of persons referred for mental health services reported that they would use telepsychiatry if it were offered (53), and two-thirds of rural respondents in the United States were willing to use it (54).

Clinical outcome. Only two clinical outcome studies have been completed to date. Brown and associates (51) examined the use of telepsychiatry compared with a traditional caregivers support group. In a quasi-experimental design comparing the two groups, similar outcome results were found on the Profile of Mood States and other relevant measures.

Zaylor (55) retrospectively compared Global Assessment of Functioning (GAF) scores for 49 patients with depression or schizoaffective disorder who were treated by telepsychiatry or same-room sessions for at least six months. No differences were found in the percentage change in GAF scores from initial visit to six-month visit between the two groups, suggesting the clinical efficacy of telepsychiatry. Furthermore, the patients treated by telepsychiatry had greater attendance rates and required dramatically shorter sessions compared with those in the same-room group.

Cost-effectiveness. Only one cost study of a telepsychiatry application has been reported (37). In a comparison of telepsychiatry and conventional mental health services in rural Queensland, Australia, the cost savings produced by telepsychiatry were estimated to be $181,716 in the first year, primarily from reduced travel by patients and clinicians; annual savings in subsequent years rose to $209,126. However, cost estimates failed to incorporate costs for maintenance and upgrading of equipment.

Limitations of the literature

Key limitations of the current literature are the paucity of reports of any kind on the use of telepsychiatry and the lack of rigorous empirical study of telepsychiatry applications. Most reports published to date are case reports or program descriptions. In empirical studies the focus has been primarily on the reliability of telepsychiatry assessments and general user satisfaction rather than on broader biopsychosocial research. Critically lacking are reliable baseline data before implementation of telepsychiatry programs, evaluation of clinical process and outcome, randomized assignment and control groups, and efforts to determine the efficacy of telepsychiatry for specific patient populations.

Furthermore, most reports of telepsychiatry programs indicate that little effort has been made to monitor program changes and adaptations as newer technology has been introduced, making systematic comparisons within and across programs problematic. Perhaps most significant is the lack of cost analyses. Given that the primary reason for implementing telepsychiatry applications is to save money in the course of expanding access to care, the degree to which telepsychiatry fulfills its promise of providing an affordable means of solving workforce problems remains largely unknown.

These limitations may be ascribed to the novelty of telepsychiatry applications; thus far the process of implementation has taken precedence over evaluation of outcomes. Below we offer recommendations about the procedural and methodological issues that should shape future empirical research.

Discussion

Our review of the literature suggests several conceptual issues in telepsychiatry research that would benefit from further development and several avenues of research that might be particularly useful.

Research methodology

Future telepsychiatry research must incorporate state-of-the-art research designs and methodologies to meet the empirically rigorous standards of clinical research in mental health care (56). For example, studies are needed to examine the effectiveness of telepsychiatry on a range of outcome variables, psychiatric disorders, and levels of illness severity, using randomized, controlled trials.

Demonstration of effectiveness

Future research must include outcome data that fall into three broad categories: process variables, clinical outcome variables, and economic outcome variables. Process variables include ease of implementation and checks on providers' adherence to standards of use; patient, referrer, and clinician satisfaction; session attendance; compliance with prescribed psychiatric medication regimens; system usage and referral rates; and improved access to services for previously underserved regions or populations.

Clinical outcome variables include general and disorder-specific measures of illness severity and functional impairment related to the effectiveness or efficacy of various telepsychiatry interventions.

Economic outcome variables include overall system costs and cost-effectiveness, cost-benefit, and cost-utility analyses to justify use of telepsychiatry over alternatives. Research should be conducted in collaboration with medical economists to address the fundamental issue of whether telepsychiatry actually does provide an affordable means of solving current workforce problems and to understand how its cost-efficiency can be maximized.

At least one study that shows how a range of important outcome variables can be examined in one project is under way. It is a randomized, controlled trial for the treatment of depression among 144 veterans treated within the Department of Veterans Affairs system (57). The study will evaluate clinical outcomes, patient compliance, patient satisfaction, and cost-effectiveness.

Telepsychiatry applications

Telepsychiatry has many possible applications, mostly when clinicians are distant from patients or from other clinicians, such as in sparsely populated rural areas or in military contexts, or when security is a concern, such as in some prison or inpatient consultations. Examples include diagnostic assessment; cognitive and mental status assessment; individual psychotherapy and pharmacotherapy; structured and unstructured group therapy; consultation with family members; consultation with clinical experts; supported employment interviews; discharge planning, such as networking with community resources to facilitate continuity of care; patient and clinician education; and clinical supervision. Knowing where and when to employ telepsychiatry appropriately will require studies of differential effectiveness across applications, clinical contexts, and psychiatric disorders.

Consideration of patient characteristics

Those who implement and study telepsychiatry applications should take patient needs and characteristics into account. Some individuals from certain patient groups might not feel as comfortable using telepsychiatry programs as others because of situational concerns, such as confidentiality in the case of litigants, or because of the nature of their illness—for example, patients who suffer from paranoia. Researchers in one study reported having difficulty recruiting subjects because of concerns about confidentiality (44). Hence flexibility and sensitivity are likely to be critical in the successful implementation of telepsychiatry programs.

Program design and implementation

Staff training and support. As with any form of organizational change, it is important that staff understand the need for making changes and have the opportunity to participate in program development and implementation. The unique aspects of telepsychiatry also make it crucial that staff receive appropriate training, consultation, and technical support in use of new technologies.

Program sustainability. Practical issues related to the long-term sustainability of a developing telepsychiatry program include consideration of how the program will fare once research or grant funding expires, whether third-party billing for telepsychiatry services is allowable, and how future advances in technology may be incorporated.

Program flexibility and sensitivity. Patients must perceive that the system has been set up to serve their interests. Thus they, too, should have an understanding of why telepsychiatry is needed—for example, greater convenience for them as well as delivery of a wider variety of services. Furthermore, as with staff, their input should be sought during the development and implementation of programs. Finally, whenever possible, patients should have the option to choose same-room consultations over telepsychiatry. For many patients, it may help to start treatment with more personal encounters and then move to remote encounters once some level of trust and rapport has been established.

Privacy and confidentiality assurances. Some patients will have justifiable concerns about the privacy or confidentiality of their consultations. Full explanations of the telepsychiatry equipment and procedures may ease such concerns for many patients, but given the erosion of privacy that new technologies often create, this issue must be addressed carefully on a case-by-case basis (6,58).

Ethical and legal considerations. Several instances in which it is unclear whether telepsychiatry can be used ethically and legally have been identified (4,6). They are crisis intervention, involuntary commitment, and obtaining informed consent. Moreover, issues related to licensure, malpractice insurance coverage, and billing may generate confusion if the clinician-provider's practice and the patient's domicile are not in the same state. Different states, disciplines, and insurance companies may have different rules and standards governing such issues—all of which may change rapidly over the next few years. Program designers must pay special attention to handling these psychiatric situations while developing administrative procedures.

Use of future technology

Clinicians, program developers, and policy makers must stay abreast of technological innovations and market conditions that support teleconferencing. Rapid advancements in digital equipment, computer software and hardware, and network reliability, speed, and security will continue to reduce the up-front costs of telepsychiatry. For example, in-home camera units that can be connected to a television and a telephone line now cost as little as $200, making it feasible in many cases to bring telepsychiatry services into private homes.

Changes in state and federal telecommunication regulations are increasing access to telemedicine services in rural areas by bringing line charges down to the same level as those in urban areas. Although the studies reviewed above used a range of transmission bandwidths, today most programs use higher bandwidths, which provide better video quality and are now relatively inexpensive. In addition, domestic and global market forces, including telecommunications deregulation worldwide, will contribute to decreased costs and increased access.

All of these factors may combine to improve the quality, access, and ease of use for telepsychiatry applications. However, they also impart a strong element of uncertainty to the future. It is difficult to predict whether a limited number of dominant formats will be available for telepsychiatry, or whether multiple technologies will be used in combination. Consider, for example, the related phenomena of online treatment and support groups and medical consultation Web sites such as WebMD. These two technologies may effectively merge, or they may be used collaboratively. Although it is unlikely that old-fashioned same-room mental health services will disappear, convenience and cost-effectiveness may make telepsychiatry services market competitors of same-room mental health services.

Conclusions

Over the past few years telepsychiatry applications have become more widely used as a means of providing mental health services to persons in remote or restricted areas. Empirical data suggest that psychiatric interviews conducted by telepsychiatry are generally reliable and that patients and clinicians generally report high levels of satisfaction with telepsychiatry. Early research has been encouraging, but it has been silent on whether, when, how, and for whom telepsychiatry is indicated. Hence the literature is underdeveloped and is characterized by a lack of scientific rigor.

As the field matures and technology improves, research on telepsychiatry is focusing less on what is technically possible and more on what is clinically effective. Future research should include reliable baseline data before implementation of programs, evaluation of clinical outcomes, randomized experimental design, cost analyses, and determination of the effectiveness or efficacy of telepsychiatry for specific patient populations.

Acknowledgments

This work was supported in part by National Institute of Mental Health grant K08-MH-0166001 to Dr. Frueh and grant R24-MH-53558 to Dr. Santos.

The authors are affiliated with the department of psychiatry and behavioral sciences of the Medical University of South Carolina and the Veterans Affairs Medical Center (VAMC) in Charleston, South Carolina. Send correspondence to Dr. Frueh, Mental Health Service (116), VAMC, 109 Bee Street, Charleston, South Carolina 29401 (e-mail, [email protected]).