Ecological Momentary Assessment and Intervention in the Treatment of Psychotic Disorders: A Systematic Review

Developments in the use of digital technologies in treating mental disorders have been flagged as key elements in the future of mental health care (1–5). Online and smartphone-based modalities of delivery offer opportunities for simple, accessible interventions, facilitating dissemination; they also offer a way to empower service users to take an active role in mental health management as a key element of their recovery (6). There has been dramatic growth in mobile phone ownership in both clinical and nonclinical populations (1,7,8), and smartphones are an especially common means of Internet access among people with severe mental illness (9). Preliminary findings of studies investigating the use of digital interventions in psychotic disorders have been promising. For example, Alvarez-Jimenez and colleagues (10) identified 12 trials of online, social media, and mobile interventions for psychosis and noted that usability and satisfactions ratings among users were highly positive, suggesting that patients are willing and motivated to engage with these types of interventions. Such findings highlight the importance of examining new therapeutic approaches that make best use of digital platforms, particularly smartphones (3).

A key clinically relevant methodology that can utilize mobile technology is ecological momentary assessment (EMA) and ecological momentary intervention (EMI) (11,12). Also known as experience sampling methodology, EMA methods involve collecting data on repeated occasions, in real time and in the context of daily life (11). When phenomena are measured as they arise and evolve over time, they can be measured more accurately, and a depiction of the dynamic relationship between state variables can be derived. EMA can capitalize on several digital technologies that offer the unique capability of automated delivery and recording of data. These include use of smartphones for prompting and recording, as well as passive capture of, ambulatory physiological data (for example, actigraphy, heart rate, and skin conductance) and context-sensitive data (13,14).

Among persons with psychotic disorders, EMA has most commonly been used as a research tool to examine mechanisms underlying symptoms (15). The reliability and validity of this method have been established in this population (16–18), and there is growing interest in using EMA methods for clinical purposes. These include assessing symptoms more accurately, identifying and monitoring signs of relapse, and monitoring treatment effects (19).

EMI is a derivative of EMA that extends the methodology of repeated within-environment prompting into the domain of clinical intervention. EMIs were first defined by Heron and Smyth (12) as interventions that provide treatment to people during their everyday lives—in real time and in natural settings. EMIs use technology similar to that of EMA, including smartphone apps and SMS text messages that can deliver statements or instructions to promote positive behaviors and coping when needed in daily life. Heron and Smyth conducted a systematic review that identified 27 EMI studies, using palmtop computers or mobile phones to deliver regular, momentary intervention for health targets, such as smoking cessation, weight loss, anxiety, and eating disorders. The identified EMIs had a variety of characteristics, highlighting broad potential for these interventions, with evidence supporting their acceptability, feasibility, and efficacy in improving illness-related outcomes.

EMA and EMI have potential applications to psychosis management, capitalizing on a number of key capabilities of mobile technologies to provide a means of accurate assessment; they can also be used to remind people of intervention strategies in the moment they are needed (5,20,21). Such assessments and reminders may be especially beneficial in psychotic disorders, because difficulties with memory and executive functioning can limit accurate recall of past events and motivational difficulties can impede generalization of intervention strategies outside the consulting room (22–25). There is also evidence that psychotic symptoms improve through regular self-monitoring (26), a potential use of EMA shown to be effective in other disorders (27,28). Remote monitoring of information may also help to identify signs of relapse that indicate the need for escalated intervention (29,30). EMI has shown promise in promoting uptake of intervention strategies in daily life, with improved outcomes in anxiety and mood disorders (31–34). In psychotic disorders, prompts could be used to promote coping with persisting positive symptoms, to compensate for negative symptoms in managing daily functioning, or to facilitate learning skills. Overall, these applications seem potentially valuable in empowering people in more effective self-management.

Reviews of the use of EMI in psychiatry (35) and of the use of mobile devices in severe mental illness (21,36–38) have highlighted the growing potential of this technology for treating psychosis. However, to our knowledge, the therapeutic application of EMA and EMI methods to psychosis has yet to be synthesized in a systematic review. Therefore, a systematic review was conducted of research investigating the feasibility, acceptability, and clinical outcomes of EMA and EMI for the support or delivery of treatment for psychotic disorders.

Methods

We followed the PRISMA protocol for conducting systematic reviews (39). The search was conducted on July 7, 2016.

Search Protocol

Relevant studies were retrieved by searching PubMed, PsycINFO, PsycARTICLES, and the Cochrane Central Register of Controlled Trials with combinations of the following search terms within keywords, title, or abstract: mobile phone; text messaging; SMS; cell phone; ecological momentary assessment; ecological momentary intervention; momentary assessment; real time; daily life; sampling method* or experience sampling; and schizo*, psychosis, or psychotic. Relevant MeSH terms (for example, mobile device and mobile application) were used for PubMed and Cochrane databases, and index terms were used for PsycINFO and PsycARTICLES. [Further details about the database search syntax are available in online supplement 1 to this article.] Identified studies were imported into the referencing database, and duplicates were systematically removed. The abstract of each study was screened by the first author for basic criteria, and full texts were then independently evaluated by two authors (IHB and MHL) against the inclusion and exclusion criteria. Reference lists of studies meeting inclusion criteria and of prior relevant reviews were manually searched for additional articles. Authors of included studies were contacted for unpublished work to reduce risk of bias.

Study Selection

Inclusion criteria were as follows: the study was published in peer-reviewed journal between 1980 and July 7, 2016; participants were diagnosed as having a psychotic disorder; and the study involved the examination of an EMI, which we defined as any technology-based device or application that can enhance care of patients with a psychotic disorder through the delivery of regular, momentary intervention in the context of daily life outside face-to-face therapy. We also included studies that utilized EMA methods (that is, the regular and momentary assessment of clinically significant processes in daily life) to assist in clinical management or assessment of patients with a psychotic disorder. Only studies that assessed the outcomes of feasibility, acceptability, and clinical measures according to the aims of the intervention were included. We included studies without control groups because of the early stage of this literature. Exclusion criteria were as follows: the sole focus of the EMI was for medication reminders, because these have been reviewed elsewhere (40,41); the study was a neuroimaging or biomedical investigation; and the study was a case study, review, thesis, or book chapter.

Data Extraction

Two authors (IHB and MHL) agreed on the data extraction protocol, including efforts to extract both significant and nonsignificant findings to reduce risk of bias. One author (IHB) then conducted the extraction. Extracted data included the following: basic study information (for example, author, year, and location), intervention aim, sample characteristics (for example, number of participants, primary diagnosis, age, and gender), study design, characteristics of the intervention (features, design, and platform), feasibility (dropout and engagement with and adherence to intervention), acceptability (feedback from participants), and clinical outcomes depending on the target of the intervention (for example, symptoms, relapse rates, and quality of life). A narrative synthesis of the findings was used in this study because of the broad nature of the studies identified, which precluded quantitative comparisons.

Study Quality Assessment

Study quality was evaluated using the Downs and Black checklist (42), a measure suitable for both controlled and uncontrolled trials.

Results

Of 1,623 studies identified, 78 underwent detailed evaluation and nine met inclusion criteria for the review. [A PRISMA flow diagram is included in the online supplement 2.] Three of the nine articles were sourced through manual searching of reference lists (43–45). No unpublished work was declared by any of the authors.

The characteristics and findings of the nine included studies are shown in Table 1. Of these nine studies, five met the definition of EMI (44,46–49), using prompts to promote self-management and psychosocial functioning. The remaining four studies investigated one intervention that met the broad definition of EMA (43,45,50,51), targeting illness and symptom monitoring. Six studies had controlled designs (43,45,47,49–51).

Across the included studies, a total of 459 participants with a psychotic disorder were included. Participants’ mean±SD age was 36.9±8.15, and ages ranged from 27.0 to 48.7. Most studies included samples of patients diagnosed as having schizophrenia or schizoaffective disorder (44–48). Two studies included additional patients with acute polymorphic psychotic disorder (15% of the sample) (50,51), and another study included patients with nonspecific psychotic spectrum disorders (8% of the sample) (49).

Illness and Symptom Monitoring

Four studies involved remote monitoring of the prodromal warning signs of relapse among patients with a diagnosis of a psychotic disorder to prevent hospitalization by using an intervention called Information Technology–Aided Relapse Prevention Program in Schizophrenia (ITAREPS) (43,45,50,51). Patients and caregivers completed the Early Warning Signs Questionnaire (52) weekly, which was uploaded to an online system, or the results were reported to managing nurses and were monitored by the treatment team. The response to patients who were assessed to be at risk of relapse was heightened monitoring and increased medication. The inclusion of this intervention required a broad definition of EMA, given that the assessments were conducted weekly and do not assess momentary phenomena. However, these studies were included in this review to demonstrate the potential application of remote monitoring of illness by using EMA for the purpose of clinical management.

Feasibility and acceptability.

Adherence to the monitoring protocol by patients and caregivers was relatively low in the two initial ITAREPS trials (50,51). In the randomized controlled trial (RCT), patient and caregiver adherence increased; however, only 39% of alerts were responded to by the treatment team in accord with the protocol (45). The low rate was mainly attributed to conflicts between protocol-indicated medication increases and psychiatrists’ clinical judgement (65% of cases). To account for this, Komatsu and colleagues (43) engaged nurses to conduct assessments by telephone and to monitor medication responses, which increased adherence to the response protocol.

Clinical outcomes.

The initial ITAREPS trials showed reductions in hospitalization events and total hospital days for the treatment group (sustained at follow-up), compared with the control groups, particularly for participants and treatment teams that adhered to the monitoring and responding protocol (50,51). Because the subsequent RCT suffered from a low rate of adoption and adherence by the treatment team, it was difficult to assess efficacy (45). Reductions in hospitalization events and total hospital days were noted for the group in which the treatment team adhered to the protocol, compared with the nonadherent group. In the later trial by Komatsu and colleagues (43), in which better clinician adherence was achieved, greater reductions in hospitalization and severity of relapse were observed; however, no follow-up data were reported. A cost-benefit analysis conducted by Španiel and colleagues (45) showed that the cost of treatment was lower in the group of patients with protocol-adhering treatment teams.

Self-Management and Psychosocial Functioning

Intervention characteristics.

Five of the nine studies targeted self-management and psychosocial and daily functioning for patients with a psychotic disorder. The app-based FOCUS (46) and SMS-delivered Mobile Assessment and Intervention for Schizophrenia (MATS) (48) involved EMIs targeting illness self-management through the delivery of repeated prompts to users throughout the day. These prompts provided medication reminders and momentary intervention items based on therapeutic models for challenging cognitions, coping, and promotion of healthy behaviors. Both FOCUS and MATS involved initial interviews with participants regarding treatment targets and subsequent personalization of intervention components. The FOCUS app also includes resources for participants to access on demand (62% of use), as well as prompts (38% of use) (46).

The remaining three studies focused on compensating for cognitive difficulties through promoting goal achievement and daily living activities, such as attending appointments, managing medications, and inhibiting undesirable behavior. The Skills Training and Empowerment Program trial was a pilot study aimed at enhancing the efficacy of a face-to-face intervention for patients with schizophrenia while shortening the intervention’s duration through regular phone calls (47). The app-based Mobus trial (44) and the SMS-based EMI designed by Pijnenborg and colleagues (49) involved programming of personalized daily living activities and goals to remind patients to complete these in daily life. In the Mobus trial, caregivers were engaged to work with the patient to identify goals and keep track of their progress by using the system (44).

Feasibility and acceptability.

The mean dropout rate across the EMI studies was 15%, with a range from 0% (47) to 36% (44). Reasons for dropout included loss of device, severity of illness, loss of interest, and intervention nonengagement. Mean response rates to the prompting components of the EMIs across the studies was 80% (rates were not reported in Pijnenborg and colleagues [49]). Ben-Zeev and colleagues (46) found no association between response rates and cognitive functioning or symptom severity. However, they found that the percentage of days on which the app was used was lower among participants who had a greater reduction in depressive symptoms over the course of the intervention (r=−.36). In the MATS trial, the participants who stopped responding to the intervention had more severe negative symptoms, more impaired daily living skills, and lower premorbid IQ (48). Participants with more severe paranoia in the Mobus trial were unable to engage with the intervention because of suspiciousness of the monitoring nature of the device (44). No other studies reported reasons for response rates. Technical issues were reported in three of the five trials (44,46,48), and specific issues related to confidentially and risk of carrying cell phones in public settings were mentioned by participants. Overall acceptability ratings for the interventions were moderate to high. In the MATS trial, participants reported increasing helpfulness of the app over time; however, some technical issues were reported that were related to the usability of the phone response and navigation functions (48).

Clinical outcomes.

The three trials targeting cognitive impairments or daily living skills found improvements in the percentage of goals achieved over the intervention (49), improvements in daily living skills (44,47), and evidence for improved cognitive capacity (44,49). The only study that conducted a follow-up (two weeks postintervention) did not find evidence for sustained increase in goals achieved in the treatment group (49). Four of the five EMI studies also included outcome measures related to change in psychotic symptoms. The Mobus (44) and MATS (48) studies did not detect pre-post changes in symptoms; however, the latter found improvements over time in medication adherence and attitudes, increased socialization, and positive effects on hallucinations (decreased occurrence and severity and controllability). The FOCUS study found significant reductions in depression and in total and positive symptoms but not in negative symptoms (46). Although Pijnenborg and colleagues (49) did not report symptom changes as overall outcomes, they noted that the group of participants classified as “responders” to the intervention (>20% increase in goal achievement over the intervention) showed pre-post reductions in negative symptoms but not in positive symptoms.

Study Quality Analysis

Overall, the reporting of key information across all studies was good, with clear descriptions of intervention components, participant characteristics, and methods and sound justification for the statistical analysis used. The primary methodological limitation was related to internal validity and power. Only two of the studies were RCTs (both ITAREPS studies [43,45]), and neither contained matched samples. The other two ITAREPS studies used historical data of a matched period prior to the intervention as a control (50,51). Of the EMI studies, only two included a control group; in one of the studies the control group was not randomly assigned (49), and the other relied on a matched sample from a prior intervention with inadequate reporting of quantitative data (this study was a preliminary reporting of pilot data [47]). The remaining three EMI studies employed a pre-post design (44,46,48). Only one study reported power calculations to determine sample sizes (45). These methodological limitations reflect the fact that this research area is in its infancy—most of the included studies were at the feasibility or pilot stage.

Discussion

The findings of this systematic review provide preliminary support for the clinical utility of EMA and EMI in the treatment of people with a diagnosis of a psychotic disorder. A small number of studies have demonstrated the use of EMI for illness self-management through momentary reminders or instructions to promote medication adherence, management of symptoms and psychosocial impairments, daily living skills, and goal achievement. Across these studies, dropout rates were low and compliance was moderate to high; overall acceptability and usability ratings were positive. Four of the five EMI studies conducted analysis to evaluate changes in clinical outcomes, although these should be interpreted cautiously because most trials were uncontrolled. Two of these four studies found improvements in overall and positive symptoms (46) and in negative symptoms (49). The remaining two EMI studies did not find significant pre-post changes to symptoms (44,48), although Granholm and colleagues (48) found improvements in medication adherence, auditory hallucinations, and socialization over the course of the intervention. Three of the five EMI studies found improvements in cognitive impairments, goal achievement (44,49), and daily living skills (44,47). The only intervention identified that made use of EMA-based methods (43,45,50,51) found that remote monitoring of illness states minimized relapse and hospitalization. However, this intervention highlighted important acceptability issues that were related to automatic protocols for responding to clinically significant changes among patients.

The varied features of the interventions included in this review demonstrate the broad application of EMA and EMI in the treatment of psychotic disorders. EMI can be used for autonomous illness self-management by providing statements or instructions on demand or by prompts via SMS or native smartphone apps (those stored locally on the device) or through phone calls aimed at enhancing standard face-to-face treatments. The ability to provide support on demand and in daily life is an important capability because barriers to treatment among persons with psychotic disorders could be overcome by providing support in daily life (20–25). Further studies are needed to build on these findings and extend the use of EMA and EMI to other domains. For example, EMA data are a rich source of contextually sensitive information that could be analyzed to make a variety of clinically relevant predictions. This potential was demonstrated by Burns and colleagues (32), who developed an algorithm capable of predicting a variety of psychological states and contexts (for example, mood, motivational states, activity, and social context) on the basis of sensor data (for example, location, light, and recent calls) collected via ambulatory devices. Such algorithms could be used to analyze EMA data and derive individualized prompts by using EMI or for individualized case formulation and management. Applications of EMA that warrant further investigation include the therapeutic benefit of illness self-monitoring, which has been shown to be helpful in treating depressive symptoms (27,28). Providing patients with tailored feedback regarding patterns in their psychotic experiences and the variables that influence them may improve awareness and identify possible intervention targets.

Another use of EMA and EMI has been referred to as an “ecological interventionist–causal model approach” (53). This approach involves testing interventions by linking their specific effects to the putative mechanisms and the targeted outcome (for example, psychotic symptoms). This innovative approach offers a highly reliable method of investigating the causal relationship between intervention and putative psychological mechanisms, which may ultimately lead to enhanced and more sustained effectiveness.

A number of identified studies relevant to this review were excluded because they did not present clinical outcome data. Many of these feasibility and acceptability studies demonstrated additional applications of EMA and EMI, and they provided indicators of whether these technologies are useful and acceptable to patients. Examples include the use of passive EMA sensing to detect early relapse indicators (13,14), assessment in inpatient settings using EMA (54), SMS-based EMA and EMI for monitoring and intervening with clinical changes remotely via clinicians (55), EMI for setting and prompting goal achievement to enhance motivation (56), and use of the FOCUS system for monitoring signs of relapse posthospitalization (57,58) and for presenting short videos of intervention strategies (59). These studies highlight the broad range of technologies that utilize EMA and EMI methods, including smartphones, video recordings, and sensing devices.

It is notable that some of the biggest effects found in the studies reviewed here were with the ITAREPS intervention (43,45,50,51), even though this intervention was characterized by poor rates of adherence among both consumers and practitioners. The primary issue raised by the ITAREPS intervention was the treatment team’s ability to maintain its autonomy in determining the best methods of responding to signs of relapse. It is important that future research investigate clinicians’ attitudes toward new technology-based interventions and explore how interventions may best be integrated in routine practice. The use of EMA for monitoring and detecting signs of relapse clearly has clinical utility; however, future research may benefit from approaching this from a more empowering perspective by allowing users to self-monitor and direct their own needs for intervention. An example of this is the EMPOWER trial (ISRCTN99559262), which is an ongoing study examining the use of a smartphone app in self-monitoring for early warning signs of relapse. This approach emphasizes user control over responses; users are able to notify their case coordinator and nominated caregiver if they choose, and the app suggests strategies for self-management.

The future of mobile technology in mental health care is growing increasingly prominent in the literature. A number of articles have described the various ways in which these technologies can be used in treating psychotic disorders (20) and mental disorders in general (1–5,19,21). The literature highlights the various challenges faced by this new area of research, including issues related to costs, access, user attitudes and interest, and the technology. The studies reviewed here also highlighted confidentiality, the risk of loss of or damage to the devices in public settings, technological limitations, and usability as issues related to feasibility and acceptability. Evidence was also found that response to and engagement with the interventions may be related to certain illness factors, such as the severity of symptoms and cognitive impairments (44,46,49). Patients in the MATS intervention reported difficulty using the phone in some conditions, including navigating and responding to SMS (48). The authors suggested that future improvements to smartphone technology may improve usability, making them more acceptable to patients. Research comparing traditional, SMS-based assessment to more modern and sophisticated smartphone technology has found that people with serious mental illness find both acceptable and feasible; however, they prefer native smartphone apps, which may be simpler to use (60). With technological advancements evolving at a rapid rate, interventions can become more user friendly, which may enhance outcomes. To aid this advancement, future research can investigate barriers and potential solutions, particularly focusing on feedback from people with lived experience and health care providers.

This systematic review had some limitations. Because of the lack of controlled studies, a meta-analysis was not conducted, and thus the results related to clinical outcomes are preliminary estimates. The definition of EMI used was relatively broad, and EMA is not yet well defined in treatment contexts; therefore, it is possible that not all relevant studies were captured. Although the ITAREPS studies (43,45,50,51) were included on the basis of our broad definition of EMA, this intervention demonstrated the potential utility of remote monitoring of illness, allowing momentary detection of relapse risk and treatment response. Future research that provides a more standardized definition of these types of methods would reduce the ambiguity of the inclusion criteria. Furthermore, variation across EMA and EMI methods (for example, native apps versus telephones) also made comparison challenging. As this literature grows, future research may compare the various delivery methods in order to isolate active components.

Conclusions

Overall, studies to date suggest good acceptability and feasibility of EMA and EMI in the treatment of psychotic disorders, and clinical outcome data from uncontrolled trials are promising. Controlled trials are needed to assess the potentially wide-ranging application of such technologies among individuals with psychotic disorders.