Outcome of a Randomized Study of a Mental Health Peer Education and Support Group in the VA

Recovery-oriented peer support services are an important part of contemporary consumer-driven mental health services. These services have been implemented across a broad range of programs in public and private facilities (1). The Veterans Health Administration (VHA) has been particularly committed to the transformation of mental health services in order to focus on recovery and promote self-determination, hope, empowerment, choice, and community integration.

Peer support is an important component of a recovery-focused mental health system in which veterans with mental illness assist other veterans with symptom management, goal-setting, skills acquisition, and other aspects of community integration. A prominent example is the Vet-to-Vet program (founded by MA), in which veterans with mental illness lead groups structured around published recovery materials. Vet-to-Vet is a consumer-provider partnership model in which peer-led groups are complementary to and embedded within the mental health care system (2).

There is increasing, albeit mixed, evidence supporting the effectiveness of peer support. A quasi-experimental study of Vet-to-Vet showed that participants had significantly better outcomes on empowerment, confidence, functioning, and alcohol use compared with veterans enrolled in the same program before implementation of Vet-to-Vet (3). Other studies, including two recent randomized clinical trials (RCTs) have also found positive effects of peer support, including greater understanding, practical knowledge, empowerment, coping skills, social network, social support, functioning, and quality of life (4–6). A recent meta-analysis found greater reductions in depressive symptoms among persons who received peer support versus usual care (7). However, some studies have shown no difference between peer-provided versus professionally provided intervention (8), and two recent RCTs reported increased psychopathology and deterioration in social integration and self-efficacy with peer support (9,10). [Additional references are available online as a data supplement to this article.]

Using a randomized research design, our aims in this study were first, to compare the effect of a peer-led recovery group (Vet-to-Vet), a clinician-led recovery group, and usual care on traditional and recovery-oriented mental health outcomes and second, to examine the effect of group attendance on outcomes. We hypothesized that participants randomly assigned to Vet-to-Vet would report superior outcomes on recovery-oriented and traditional outcome measures compared with participants randomly assigned to either a clinician-led recovery group or usual treatment. In addition, we hypothesized that higher group attendance would be associated with better outcomes.

Methods

Veterans were randomly assigned to a weekly, peer-led recovery group (Vet-to-Vet), a clinician-led recovery group, or “usual treatment.” The first two intervention groups also received usual treatment. The study was conducted at two Veterans Affairs (VA) hospitals, and institutional review board approval was obtained from both sites.

Veterans were eligible for the study if they were at least 18 years old, spoke English, had at least one psychiatric diagnosis, had received mental health services at the participating site in the preceding 12 months, and were not currently attending Vet-to-Vet groups. Altogether 298 veterans enrolled in the study. Of these, 240 (81%) completed the three-month follow-up and were included in the analyses. Respondents who completed the follow-up compared with those lost to follow-up showed that follow-up respondents were slightly older on average (53.4 versus 50.6 years of age; t=1.97, df=297, p=.05), more likely to be employed (15% versus 3%; χ²=5.88, df=1, p=.015), and more likely to have a psychotic disorder diagnosis (27% versus 12%; χ²=5.52, df=1, p<.02). Respondents were less likely than nonrespondents to have a diagnosed depressive disorder (79% versus 91%; χ²=4.53, df=1, p<.05), an alcohol use disorder (61% versus 77%; χ²=5.31, df=1, p<.05), or a substance use disorder (48% versus 67%; χ²=6.54, df=1, p<.02).

Veterans were informed about the study at treatment program meetings. Those who expressed interest were contacted by a research interviewer, who evaluated eligibility and further explained the study. Those who agreed to participate provided written informed consent. Baseline measures were completed through a structured interview, after which the interviewer opened an envelope indicating the random assignment. If assigned to a group, participants were informed where and when the group was scheduled to meet. Follow-up interviews included the same measures obtained at enrollment, three months later.

Both peer- and clinician-led groups met for 45 minutes weekly. Vet-to-Vet groups used written recovery materials such as the Spaniol Recovery Workbook available from the Boston University Center for Psychiatric Rehabilitation. Clinician-led groups were not required to use such materials; however, most clinician group leaders did so.

Peer-led groups were led by two peer facilitators who were paid per group as VA contractors. Clinician-led groups were led by a master’s-level clinician. Both peer and clinician group leaders received training in the Vet-to-Vet model from the study team. Peer facilitators received ongoing supervision from a clinician and a peer supervisor. Clinician group leaders were supervised by their respective program directors. All groups retained the same leaders throughout the three-month study period. The major differences between peer- and clinician-led groups were in the professional training of group leaders and in their personal experience as veterans with a mental illness. Peer facilitators shared their personal experiences as veterans with mental illness.

Because peer-delivered services are designed to address outcomes that differ from traditional services, recovery measures were included in the protocol as were traditional measures of symptoms and functioning. All measures used had previously demonstrated reliability and validity with veterans. Recovery outcomes included patient activation, hope, personal empowerment, social support, and mental health recovery and were obtained with the following instruments. The Patient Activation Measure is a 13-item assessment of patient knowledge of illness management and recovery and skill and confidence in self-management (11). It has high reliability (Cronbach’s α=.87) and validity. The Snyder Hope Scale includes 12 items gauging ability to generate and meet goals. Internal consistency reliability ranges from .77 to .84, and good validity has been demonstrated (12). The Rogers Empowerment Scale comprises 28 items encompassing self-efficacy, self-esteem, perceived power, community activism, righteous anger, and optimism. It has good reliability (α=.86) and validity (13). The Medical Outcomes Study Social Support Survey includes 19 items measuring emotional and informational support, tangible support, affectionate support, and positive social interaction, with good reliability (α=.91) and validity (14). Finally, the Recovery Assessment Scale includes 41 items gauging goal orientation, self-efficacy, self-esteem, and optimism (15). The scale has good internal consistency (α=.93) and test-retest reliability (r=.88) (15).

Traditional mental health outcomes included the Veterans Short Form (VR-12), which assesses physical and mental health status (physical component score and mental component score, respectively). The VR-12 instruments are among the most widely used functional status measures in the world and have high reliability and validity. The revised 24-item Behavior and Symptom Identification Scale (BASIS-24) assesses depression and functioning, difficulty in interpersonal relationships, self-harm, emotional lability, psychotic symptoms, substance abuse, and overall mental health. Reliability of subscales ranges from .77 to .91, with good validity.

Lifetime psychiatric diagnoses were assessed with the Structured Clinical Interview for Diagnosis. Attendance was taken at each group, and total number of groups attended was tallied for each participant for up to 12 weeks, which was the specified study period.

Data analyses were performed with SAS 9.2 and SPSS 18.0. Missing ratings on the outcome measures were imputed with multivariate iterative regression, a method that generates results comparable to full multivariate imputation.

Outcome measures were scored with each measure’s published algorithms. To compare group outcomes over time, an intention-to-treat analysis was performed with mixed-model regression. Time (enrollment and three-month follow-up), group, and the time × group interaction were population-level fixed effects; time was also a participant-level random effect to adjust for baseline response. A significant time × group interaction would indicate an effect of the intervention. To examine the “dose effect” (number of groups attended), we conducted an “as treated” analysis for the peer- and clinician-led groups using a multivariate regression model. Number of groups attended and baseline (time 1) status on each measure were used as predictors to examine the effect of attendance on follow-up outcomes.

Results

Most participants were male (92%, N=220), white (66%, N=159), and 36–60 years old (72%, N=173). African Americans represented 23% (N=54) of the sample and Latinos 10% (N=24).

Hypothesis 1, better outcomes for the Vet-to-Vet intervention, was not supported. Results indicated no statistically significant group × time interactions on any of the outcome measures, suggesting no differences in mental health outcomes among the three groups. However, there was a significant main effect of time, which indicated statistically significant improvement over the three-month study period on the BASIS-24 depression and functioning subscale (F=4.35, df=1 and 237, p<.05), psychotic symptoms subscale (F=4.85, df=1 and 237, p<.05), and summary score (F=7.02, df=1 and 237, p<.01), as well as on the mental component score of the VR-12 (F=16.71, df=1 and 237, p<.001). [Mean ratings on each outcome measure for each of the three study arms and time points are provided online as a data supplement to this article.]

To assess the “dose effect” (hypothesis 2), we examined number of groups attended as a correlate of outcomes for the pooled group of participants assigned to attend either type of group. Results indicated a statistically significant effect of attendance on several traditional mental health outcomes (BASIS-24 summary score, depression and functioning, psychotic symptoms, and substance abuse) but no significant effect on any of the recovery outcomes (Table 1).

Discussion

These results are consistent with earlier studies suggesting no difference between a peer-provided versus a professionally provided intervention added to usual treatment, and they are more favorable than studies reporting worse outcomes for peer-run services. One RCT of a supported socialization program found that clients who met with peer supporters had worsening levels of self-rated and clinician-rated psychopathology (9). In contrast, clients improved when they met with a community support volunteer who had no previous psychiatric diagnosis. A recent study comparing outcomes of individuals randomly assigned to community mental health services alone versus in combination with consumer-operated services found deterioration in social integration and self-efficacy among those who received combined services (10).

On the other hand, our results did not indicate that peer services were more effective than traditional services, as some RCTs, a quasi-experimental study of Vet-to-Vet (3), and a recent meta-analysis of the efficacy of peer support for depression have found (7). As noted previously, peer support services are varied and multifaceted. These variations may account for differences in results obtained from different studies. The peer-led services in this study were offered within a comprehensive health care system in a provider-consumer partnership model, not as a totally consumer-run service. Furthermore, all participants in our study continued to receive usual treatment, many in intensive day treatment programs. Thus, the 45-minute weekly recovery group that was part of the study was a small component of their treatment program. In addition, their attendance at the study-related groups was limited. Across peer- and clinician-led groups, participants attended an average of 3.8 groups during the 12-week study period, and 26% did not attend any groups. Thus participants’ exposure to the intervention was fairly weak, although our as-treated analysis suggests that clients with better attendance benefited more than those with worse attendance.

This study had both strengths and limitations. Among the strengths, this was the first randomized study of the Vet-to-Vet intervention in the VA. Both traditional and recovery-focused outcome measures were used, all validated and used in previous research. All participants were followed up, regardless of how many groups they attended, achieving a follow-up rate over 80%.

This study also had limitations. It was conducted at only two sites, and participants were primarily middle-aged male veterans. The sample reflects the population of veterans in the programs and sites from which we recruited but does not represent the increasing number of younger and female veterans, many of whom are not in these programs, which are largely geared toward veterans with longstanding psychiatric problems, substance abuse problems, or both. Another limitation is that outcomes were obtained only three months after baseline. Earlier research showing benefits of Vet-to-Vet assessed nine-month follow-up outcomes (3). Differential change may require a longer period to emerge. Although our study used a three-month intervention period, participants were permitted to continue attending groups beyond that point, and some chose to do so, although we did not assess their outcomes at later time points. We also conducted qualitative interviews with a subsample of study participants and group leaders (to be reported in a future paper), which may yield additional insight into the impact of the intervention.

Conclusions

This study adds to the evidence base (8), indicating that there was no short-term incremental benefit (or harm) from peer services compared with usual care. Given the inconsistent results of research in this area, it seems likely that peer services may be beneficial to some individuals in some contexts and settings. Consequently, in further research it will be important to identify participant, program, and setting characteristics to determine who might be most likely to benefit from peer support services.