A Virtual Hope Box: Randomized Controlled Trial of a Smartphone App for Emotional Regulation and Coping With Distress

In 2014, up to one-fifth of all active-component soldiers on some U.S.-based installations had a diagnosed behavioral health disorder (1). These figures reflect a notable prevalence of psychological conditions, including posttraumatic stress, depression, anxiety, mood, and adjustment disorders, for active-duty military personnel and military veterans (2), which in turn have been related in some cases to suicidal behavior (3–5).

Cognitive-behavioral therapies (CBT) (6,7) have been successfully applied to the management of emotional dysregulation and distress, including suicidal thoughts and related behaviors of civilians (8–10) and active-duty military service members (11). To augment CBT, clinicians sometimes implement a “hope box” (12,13) for patients to use when experiencing periods of acute or significant distress, emotional disequilibrium, or suicidal ideation. A hope box is a physical container (such as a shoebox) containing items that serve as reminders of positive life experiences, people who care, coping or distracting resources, or reasons for living (14).

The adoption of personal mobile technology for use in psychological health care is widespread (15–17). Substantial proportions of the military and veteran communities use their smartphones to access health resources (18,19). Our Virtual Hope Box (VHB) uses smartphone capabilities to enhance and personalize the hope box experience on a highly accessible medium. We designed the VHB to help a user restore emotional equilibrium during instances of distress and to increase overall skills in managing negative thoughts and feelings (20,21). [Additional information about the VHB is available in an online supplement to this article.]

We conducted a randomized controlled trial of the VHB in a sample of patients at risk of self-harm who were being treated in Department of Veterans Affairs (VA) behavioral health clinics. Our objectives were to assess the primary impact of VHB on stress coping skills over 12 weeks, the secondary impact of VHB on suicidal ideation and reasons for living, the use of VHB for addressing emotional disequilibrium away from the clinic, and the patient experience of VHB through objective usage patterns and self-reported usability and perceived benefits. Our primary hypothesis was that participants in the VHB group would report increased coping self-efficacy and reasons for living and lower suicidal ideation compared with participants in the control group at all measurement points postrandomization.

Methods

All study procedures were approved by the VA Portland Health Care System (VAPORHCS) and the Army Human Research Protection Office. Written informed consent was obtained from all participants.

Setting and Sample

Potential participants were identified and referred to the study by 63 behavioral health clinicians from 13 treatment programs within the VAPORHCS. The research team also screened patients for potential eligibility who had been referred to the dialectical behavioral treatment (DBT) program; patients recently hospitalized for suicidal behaviors, including suicidal ideation; and patients on a high-risk-of-suicide registry maintained by the local VA suicide prevention team.

Inclusion criteria.

Study eligibility required participants to be U.S. service veterans in active treatment (two mental health appointments in the six months prior to study recruitment), currently expressing suicidal ideation or had expressed suicidal ideation within the three months before recruitment, and confirmed by their clinicians as suitable for VHB study participation as part of treatment. All participants were also required to own and regularly use or carry their own iPhones or Android phones.

Exclusion criteria.

Patients were deemed ineligible if they had moderate or severe dementia or significant cognitive disturbance documented in the medical record, were considered terminally ill according to documentation in records, were unable to provide consent without the assistance of a legally authorized representative or guardian, or were currently admitted to an inpatient unit.

Design

We used a parallel-group randomized design with 1:1 subject allocation to two treatment arms. In the control condition (enhanced treatment as usual, N=60), participants received treatment as usual supplemented with printed information about coping with suicidal thoughts and other prevention resources (20). In the intervention condition (N=58), patients received treatment as usual supplemented with VHB. In both conditions, patients received appropriate care for elevated risk of suicide throughout their study participation. Participants were followed for 12 weeks with measurements obtained at baseline and at three, six, and 12 weeks, in person or by phone as necessary. [A participant flow chart (CONSORT diagram) is included in the online supplement.]

Procedures

Recruitment and randomization.

Potential participants were approached between March 2014 and April 2015, initially by letter of invitation and subsequently by phone, to establish interest and eligibility. This was followed by an in-person meeting with a study coordinator to describe the study in more detail, obtain informed consent, and enroll the patient in the study. Participants received $20 at baseline and $20 at 12 weeks to compensate them for time and travel. Following enrollment and baseline survey administration, the clinical coordinator (CC) opened sealed envelopes indicating the arm to which the participant was assigned. Random assignments were generated by using a permuted-block randomization of size 10.

Intervention.

Patients assigned to the VHB condition met with the CC for instructions on downloading and using the app on their personal smartphones and subsequently met with their clinician to guide individual tailoring of VHB content and use of VHB for stress management and emotional regulation. Patients then used VHB as needed while away from the clinic. Participants assigned to the control condition similarly met with the CC following enrollment for orientation to the control group’s printed materials (20). As with the VHB condition, participants in the control group were guided by the CC and subsequently their clinician to consult their printed materials for stress management and emotional regulation as needed. [More information is available in the online supplement.]

Outcome Measures

Our primary outcomes and most of the secondary outcomes were based on self-reported scale measures. [More information is available in the online supplement.]

General demographic information.

We collected data on age, sex, race-ethnicity, education, marital status, and military service variables from patient reports at baseline. We extracted psychiatric and general medical diagnosis information in the six months prior to study enrollment from VA administrative data sets.

Primary outcome measures (baseline and at three, six, and 12 weeks).

For our primary measure of VHB effectiveness in supporting stress coping, we used two subscales from Chesney and colleagues’ (21) Coping Self-Efficacy Scale (CSE): stop unpleasant emotions and thoughts and enlist support from friends and family. The CSE has shown reliability and validity in depressed samples and can be used to assess change in coping ability over time (21).

For our longitudinal measure of suicidal ideation, we used the first five items of the Beck Scale for Suicidal Ideation (BSS) (22). The five-item version of the BSS has been used as a tool for screening for the presence or absence of suicidal ideation (23–26) and, more recently, as a brief measure of change in suicidal thoughts and ideation over time (27). The BSS has high internal reliability and concurrent validity (22,28).

To identify changes in a patient’s perceived reasons for living, we used nine of 12 items from the Brief Reasons for Living Inventory (BRFL) (29). This inventory possesses good psychometric properties and is consistent with Linehan and colleagues’ (30) 48-item measure.

Secondary measures.

Secondary outcome measures—the Interpersonal Needs Questionnaire (INQ) (31), Cohen and colleagues’ (32) Perceived Stress Scale (PSS), and the Columbia-Suicide Severity Rating Scale (C-SSRS) (33)—were collected at baseline and 12 weeks. Remaining measures gathered at 12 weeks included self-reported use of the respective interventions and self-rated helpfulness, benefit, ease of use, likelihood of continued use, and likelihood of recommending to others. We conducted structured interviews with participating clinicians at the trial midpoint. Detailed usage logs of the VHB were recorded on participants’ smartphones and downloaded at each assessment.

Sample Size

The sample size for this study was set at 120 participants a priori as a feasible recruitment goal given the study period and eligibility criteria. On the basis of this sample size, we estimated that the minimum detectable standardized difference between randomized groups—with a two-tailed α of .05 and β of .20—would be .56 with an intraclass correlation coefficient of 0. With an intraclass correlation coefficient of .5 and four measurement occasions, we would be able to detect a standardized difference of at least .40 (34).

Data Analysis

We compared the treatment groups on demographic factors by using standard bivariate comparisons. To account for modest imbalance given the moderate sample size, we used quintiles of an estimated propensity score (35) for treatment assignment using baseline characteristics.

We used generalized estimating equations (GEE) with an identity link and a Gaussian error distribution to test the primary hypothesis of differences between the treatment groups after randomization. For all outcomes measured more than once, the models were fitted by using a categorical specification of time. This allowed us not to constrain the shape of change over time to a functional form and to directly test for differences at a specified measurement occasion. An interaction term between measurement occasion and treatment assignment served as the primary test of a difference in means of the treatment groups at the specified time points. All participants were included in intent-to-treat models if they had data at one or more time points. We also examined a per-protocol analysis restricted to participants who completed all four measurement occasions.

We used effect coding (VHB=1; control group=−1) to include treatment assignment in the model. We report the unstandardized regression coefficients and the associated 95% confidence intervals for all parameter estimates associated with change over time in the total study population. We also report the unstandardized and standardized differences between the treatment groups at each time point and the associated 95% confidence intervals. Standardization involved the division of the unstandardized difference by the baseline standard deviation of the outcome variable (36).

The variables associated with the C-SSRS and treatment satisfaction were collected at 12 weeks postrandomization. We used a negative binomial regression model to compare the count outcomes of the C-SSRS between the treatment groups. We used ordinal logistic regression models to compare the responses to the treatment satisfaction items between the study groups. We used Stata 13 (37) to estimate the GEE models.

Results

Sample

Of 326 eligible patients approached between March 2014 and April 2015, a total of 118 enrolled in the study, completed baseline measures, were randomly assigned to the two experimental groups, and began using the VHB (intervention) or enhanced materials (control) as supplements to treatment as usual. Table 1 shows the demographic characteristics of the sample. No statistically significant differences in demographic or clinical characteristics were identified between the groups at baseline. Forty-six participants in the VHB group (79%) and 52 participants in the control group (87%) provided data at all four measurement occasions.

Outcomes

Detailed descriptive statistics on the primary and secondary measures are included in Table 2.

Primary outcomes.

The results of the GEE models for the primary outcomes are shown in Table 3. Average sum scores for the two subscales of the CSE increased (improved) over time in the total study population. The “stop unpleasant emotions and thoughts” subscale of the CSE measure demonstrated the strongest differences over time and between treatment groups. Participants in the VHB group had higher average scores on this subscale than the those in the control group at all time points postrandomization. These differences were statistically significant at both three and 12 weeks [also see online supplement]. The largest difference between the treatment groups was at 12 weeks. In contrast, the CSE “enlist support from friends and family” subscale showed a small increase in the VHB group only at 12 weeks postrandomization; however, there were no statistically significant differences on this measure between the treatment groups at any time point.

Average sum scores on the BSS decreased over time in the total study population, with a statistically significant decrease relative to baseline observed at six weeks postrandomization (Table 3). No statistically significant differences between the two treatment groups were seen at any time point. Similarly, a small reduction in the BRFL sum score was noted at 12 weeks postrandomization in the total study population, and there were no statistically significant differences between the treatment groups at any time point. Restriction of the analysis to participants with complete data and the estimation of a selection model under an assumption that data were not missing at random did not appreciably alter the conclusions.

Secondary outcomes.

There were no statistically significant differences between the treatment groups on the secondary outcome measures.

Users of the VHB reported higher frequency of use of study materials, compared with participants in the control group (Table 4). The self-reported frequency of use was not associated with change over time on any of the primary outcome variables. More recorded days of VHB use were associated with lower BRFL scores at 12 weeks postrandomization.

Perceived benefit.

At 12 weeks postrandomization, participants in the VHB group had greater odds than those in the control group of identifying the study intervention as helpful and were more likely to indicate an intention to use the intervention again or to recommend it to someone else (Table 4). The most frequently cited reasons for using the VHB by participants in the VHB group (N=49 respondents) were for distress (N=34, 69% of respondents); when emotions were overwhelming (N=28, 57%); when they felt like hurting themselves (N=15, 31%); and for relaxation, distraction, or inspiration (N=25, 51%).

Clinician feedback.

Data from structured interviews suggested that clinicians appreciated the VHB's capacity to serve as an additional therapeutic tool and valued the fact that the VHB served to validate patients' existing coping skills and gave them an outlet to practice these skills. More detailed findings from these data are beyond the scope of this article and will be reported elsewhere.

Discussion

We designed VHB to serve as a suite of tools to aid coping with emotional dysregulation, stress, and distress in everyday life. In the clinical arena in particular, we intend the VHB to be used by patients as a supplement or accessory to ongoing treatment while away from the clinic. VHB is a way for patients to practice the tailored coping techniques and strategies that are commonly taught as part of treatment—efficiently, conveniently, privately, flexibly, and easily.

Our primary objective was to assess the impact over time of the VHB on coping self-efficacy. Results confirmed our hypothesis that the VHB did indeed significantly improve coping self-efficacy to stop unpleasant thoughts and emotions during treatment compared with a control condition. The VHB was not associated with increases in coping efficacy to enlist support from friends and family.

An additional objective was to evaluate the impact of the VHB on clinical outcomes. In this study, the sample comprised patients considered to be at risk of suicidal behaviors, and the treatment outcomes focused on elements of suicidality: presence and intensity of suicidal ideation (BSS and C-SSRS); importance of reasons for living (BRFL); feelings of thwarted belongingness (INQ); and how unpredictable, uncontrollable, and overloaded individuals found their lives (PSS). We found no statistically significant advantage of treatment augmented by the VHB over treatment without the VHB for any of these outcomes. We did, however, find modest but statistically significant improvements over the course of treatment for both VHB and enhanced treatment as usual in coping, suicidal ideation, and perceived stress. Incorporating the VHB into treatment did not diminish therapeutic effects, nor did clinicians or patients report any issues with the safety of the app or other detriments in clinical use.

A key rationale for having a hope box on a smartphone is to make an existing, successfully employed, therapeutic tool (conventional hope box) more accessible, convenient, discreet, and flexible to the user. The ideal result is support at any time or place that it is needed. We know that personal cellphone use is extremely high among active and recently retired military personnel (18,19). We also know that service members are extremely mobile and thus the need for psychological support might frequently emerge in the absence of health care providers. Usage data from this study confirmed that the VHB was used regularly and frequently, was reported as easy to use, was found helpful and beneficial in dealing with stress and emotional difficulties, was likely to be used in the future, and would be recommended to others. Moreover, user self-reports indicated that the VHB was used as intended—for relaxation and distraction or inspiration when feeling distressed, when emotions were overwhelming, or when they felt like hurting themselves. Clinician feedback was similarly positive.

A limitation of this study was the sample size. This had two effects on our study. It provided limited power to detect small differences and allowed for covariate imbalance, because the assumption of balance through randomization is asymptotically based. Although we did not detect statistically significant differences in baseline characteristics between the study groups, there were several large differences relative to the sample variance. Use of the propensity score adjustment helped to improve the direct comparability of the study groups.

A second limitation is related to suicide severity among the study population. All participants were identified in the medical system as having current or recent suicidal ideation per clinician assessment. However, the study population, on average, did not score very high on the BSS at baseline (a mean score of 3.43; possible range of 0–10). Although we were able to detect a small reduction in suicidal ideation over time in the total study population, a floor effect may have contributed to an inability to discern any differences between the treatment groups on this outcome.

A third limitation was that the VHB was designed to help individuals restore equilibrium when under distress. Therefore, measures that are administered at specific time points may not capture changes in state associated with these temporary events.

Conclusions

Military behavioral health patients experiencing distress, emotional dysregulation, or suicidal ideation are often separated from direct clinical support when they need it. Although impacts on some outcomes in this study were limited, users of the VHB nonetheless noted multiple benefits. The VHB smartphone app offers a highly portable, accessible and discreet suite of tools for effectively increasing coping self-efficacy. VHB has been—and can further be—easily disseminated across a large population of users. Consequently, VHB may have broad, positive utility in behavioral health care.