Integrating Peer-Provided Services: A Quasi-experimental Study of Recovery Orientation, Confidence, and Empowerment

Peer-run services, broadly defined as mental health services provided by individuals who are themselves recipients of mental health services, have received increasing emphasis in the development of mental health service systems in recent years ( 1 ). Despite this trend, the research on the benefits of such services is both limited and often poorly controlled. The published literature largely consists of descriptive studies, often with small samples and cross-sectional designs or with longitudinal designs without comparison groups ( 2 , 3 ). One reason for the lack of rigor is that traditional randomized clinical trial methodologies, widely considered the gold standard in determining effectiveness, can be difficult to implement in studies of peer-run services, which are consumer driven and voluntary. The use of random assignment limits the availability of an intervention, and for peer services built from a philosophy of inclusion, randomization may fundamentally alter the peer service under investigation.

A further complication in the literature is the great variability of services provided by peers and the programmatic structures within which they operate. Additionally, definitions of terms used to describe peer-provided services have not been standardized; terms include consumer-operated services, peer support, mutual support, and self-help.

Despite the variability in models and terminology, there are some commonalities among peer support models ( 4 , 5 ). Most peer-provided mental health services believe that consumers can benefit from interacting with people who have themselves experienced similar difficulties, learned to cope with them, and found reasons for hope for the future. Peer services are founded on core values, such as empowerment, taking responsibility for one's own recovery, the need to have opportunities for meaningful life choices, and the valuation of lives of people with disabilities as equal to those of people without disabilities ( 6 ).

In 2002 the Errera Community Care Center of the Department of Veterans Affairs (VA) Connecticut Healthcare System, in partnership with Moe Armstrong, a combat veteran with schizophrenia, developed and implemented the Vet-to-Vet program, a partnership model of peer education and support that has been described previously ( 7 ). The study presented here is a quasi-experimental cohort study of 296 veterans that compared outcomes of those treated before implementation of the Vet-to-Vet program with those treated after implementation. In the spirit of participatory action research ( 8 ), the study was designed in close collaboration with both Mr. Armstrong and staff of the Errera Community Care Center, and all aspects of the study were negotiated and agreed upon before study initiation.

After comparing the cohorts on diverse baseline measures, we report on changes over time on four recovery-oriented primary outcome measures: recovery orientation, self-confidence, general empowerment, and empowerment with respect to mental illness. We hypothesized that after the analyses adjusted for baseline differences, persons in the Vet-to-Vet cohort would have more positive outcomes on primary outcome measures, compared with those in the control cohort. We also conducted exploratory analyses to examine changes in a series of secondary measures that address symptom severity and community functioning.

Methods

Vet-to-Vet program

The Vet-to-Vet program is as a peer-professional partnership model of peer education services. Although meetings are entirely peer led, Vet-to-Vet programs are affiliated and co-located with VA mental health services, and VA mental health staff serve as consultants to peer facilitators, provide initial training and weekly supervision, and help to obtain additional resources. Meetings occur five days per week on VA premises, follow a "read and discuss" format using an established curriculum of recovery-oriented publications, and have a voluntary attendance policy. Details about the program and its development have been published previously ( 7 , 9 ).

Participants and procedures

Participants became eligible for enrollment in the study at least one week after admission to the Community Reintegration Program at the Errera Community Care Center, a VA program for veterans with severe mental illnesses, many of whom are also homeless. The one-week delay was adopted in order to avoid recruiting individuals who would not return for treatment after screening.

The Community Reintegration Program is adjunctive to psychiatric treatment, and referrals come from veterans' primary mental health clinician. There are two "tracks": one addressing acute psychiatric crises and the other focusing on longer-term rehabilitation goals for veterans with severe mental illness. Each veteran's treatment is individualized, but it includes access to an array of therapeutic and psychoeducational groups and individual clinical attention from a primary coordinator. There are no time limits to participation.

All individuals admitted to the program were informed about the study by their clinician, and information on the study was also presented at programwide community meetings. Interested individuals were referred to an independent study coordinator, who verified eligibility, completed the informed consent process, and conducted the baseline interview.

There were two study cohorts. Recruitment for cohort 1, the control group, began in February 2002, before the implementation of the Vet-to-Vet program, and continued for approximately six months, as was agreed upon by Mr. Armstrong and the Community Reintegration Program staff. At this point, 78 participants had been enrolled. The Vet-to-Vet program was initiated at this time. Cohort 2 recruitment began in January 2003, approximately three months after recruitment of the first cohort was completed. Recruitment continued for three years at which time 218 veterans had entered cohort 2. Cohort 2 is, thus, considered the treatment-exposed group. Neither interest in attending the Vet-to-Vet program nor actual attendance was required for study participation.

Follow-up interviews were conducted at one month, three months, and nine months after the baseline assessment by an experienced independent rater trained in the administration of all study instruments. The one-month interview was brief and served primarily to continue to engage participants and to document attendance in the Vet-to-Vet program (cohort 2 only), whereas the other two follow-up interviews included all assessments administered at baseline. Participants were interviewed regardless of attendance in the Vet-to-Vet program or any other VA service.

The protocol was approved by both institutional review boards affiliated with VA Connecticut Healthcare System.

Background characteristics

Background variables were based on self-report data obtained during interviews conducted at the time of study entry and included age, gender, education level, race and ethnicity, marital status, the number of days worked for pay in the 30 days before the baseline interview, and total monthly income.

Primary outcome measures

>Recovery attitudes. Recovery orientation was measured with nine items from the Recovery Attitudes Questionnaire (RAQ) ( 10 , 11 ). All seven items from the short version were included (RAQ-7), and two additional items from the long version were used that address issues not included in the other primary outcome measures. Item responses range from 1 to 5 and are recoded such that higher scores indicate stronger agreement with recovery attitudes. Scale scores were created by averaging all items, and thus the total score ranges from 1 to 5, with higher scores representative of greater recovery attitudes. Coefficient alpha was .57 at baseline and .60 at both three and nine months.

Confidence. The Mental Health Confidence Scale is a measure of general confidence and self-efficacy ( 12 ). Item responses range from 1 to 6. The total score is the average of all items, and thus it ranges from 1 to 6, with higher scores indicating greater levels of confidence. Alphas were .92, .94, and .95 at baseline, three, and nine months, respectively.

General empowerment. The 28-item Making Decisions Scale was used as a measure of general empowerment ( 13 ). Responses range from 1 to 4, with items recoded such that higher scores indicate stronger agreement with statements reflecting empowerment. The total score is the average of all items, and thus it ranges from 1 to 4, with higher scores indicating greater levels of empowerment. Coefficient alpha was .75 at baseline, .81 at three months, and .79 at nine months.

Empowerment over illness. Symptom-related items were selected from the Recovery Assessment Scale ( 14 ) to specifically measure mastery and empowerment over symptoms of mental illness. Items are scored from 1 to 4 and are all recoded such that higher scores indicate greater empowerment, and items are averaged to create a total score that ranges from 1 to 4. Coefficient alpha was .76 at baseline, .80 at three months, and .81 at nine months.

Secondary measures

Functional status. The Activities of Daily Living Scale is a 12-item scale measuring areas of daily functioning, such as bathing, shopping for necessary items, and doing laundry ( 15 ). Respondents endorse the frequency with which they perform each activity during a typical week on a 5-point scale, ranging from 1, almost never, to 5, nearly always. The total score is the average of all items and thus ranges from 1 to 5, with higher scores indicating higher functional ability. Internal consistency was .84 at baseline, .85 at three months, and .88 at nine months.

Overall functional status was also assessed with the Global Assessment of Functioning (GAF) ( 16 ), a standardized assessment of several functional dimensions consisting of a single item ranging from 1 to 99, with higher scores indicating greater functioning. This item is based on a rater assessment—that is, not on self-report data.

Substance use. Alcohol and drug use was measured with two self-report questions from the Addiction Severity Index (ASI) ( 17 ): specifically, the amount of money spent on drugs or alcohol in the prior 30 days and the number of days of the past 30 days in which the participant reported using drugs or alcohol. The total score ranges from 0 to 30, with higher scores representing higher levels of alcohol and drug use.

Symptom severity. Severity of general psychiatric symptoms was measured with the Brief Psychiatric Rating Scale (BPRS), a 19-item scale commonly used in research with heterogeneous groups of people with serious mental illness ( 18 ). The BPRS is an interviewer-rated scale (that is, not self-report) that measures general psychiatric symptoms (for example, depression and anxiety) and symptoms more directly associated with a severe mental illness (for example, hallucinations, self-neglect, and unusual thought content). Each item is rated on a scale from 0, not present, to 6, severe, and the total score is based on the average of individual items. Thus the total score ranges from 0 to 6, with higher scores indicating greater severity. Internal consistency was .69 at baseline and .77 at both three and nine months.

Severity of posttraumatic stress disorder (PTSD) symptoms was measured with the PTSD Check List-Stressor Specific Version (PCL-S), a 17-item scale with items ranging from 1 to 5, with higher scores indicating greater severity. The total score is the sum of all items, and it thus ranges from 17 to 85, with higher scores representing greater severity of specific PTSD symptoms and general indicators of subjective distress. A variety of cutoff points have been suggested as indicating a diagnosis of PTSD; two commonly cited are 44 and 50 ( 19 ). Internal consistency was .93 at baseline and .95 at both the three- and nine-month follow-up points.

Trauma history. History of traumatic events was measured at baseline only with a checklist adapted from the Traumatic Life Events questionnaire ( 20 ). Each item assesses whether a type of traumatic event has happened in the participant's lifetime. A score of 1 is given if the event happened once, and 2 points are given for any event type that occurred more than once. The total score is the sum of the number of types of traumatic events that have been endorsed and can range from 0 to 26, with higher scores indicating a more severe trauma history. Internal consistency was .72.

General life satisfaction. General life satisfaction was measured with a single item from the Lehman Quality of Life scale ( 21 ). Possible scores ranged from 1, terrible, to 7, delighted, with higher scores indicating greater satisfaction.

Participation in the Vet-to-Vet program. Participants were asked to report the number of Vet-to-Vet sessions they had attended since their prior research interview, using the following response choices: none, 1–2, 3–10, 10–20, and more than 20.

Data analysis

Three analytic groups were constructed. Cohort 1 consisted of participants enrolled before the initiation of the Vet-to-Vet program, cohort 2 included all participants enrolled after the Vet-to-Vet program began, and cohort 2-V consisted of those who enrolled after the Vet-to-Vet program began and who reported having participated in more than ten Vet-to-Vet groups since the last research interview on at least one follow-up interview. Comparisons between cohort 1 and cohort 2 thus constitute the intention-to-treat analysis, and comparisons between cohort 1 and cohort 2-V represent an as-treated analysis.

Frequencies and means of study variables were generated. Bivariate comparisons between groups using t tests and chi square tests addressed both baseline study variables and study completion rates for both the intention-to-treat and as-treated study groups.

To compare differences between groups on longitudinal outcomes, two sets of mixed-effects models were conducted by using SAS PROC MIXED ( 22 ). The first compared cohort 1 with cohort 2, and the second compared cohort 1 with cohort 2-V. Because treatment assignment was not randomized, all baseline measures were included as covariates in each longitudinal analysis to adjust for potential confounding. Group-by-time interactions were also examined.

All four recovery measures were used as primary outcome measures with an alpha of p<.05. Secondary outcome measures were examined for descriptive purposes, also with an alpha of p<.05, and included measures of functioning, drug and alcohol measures, and both symptom measures.

Results

Baseline characteristics and follow-up rates

The total sample consisted of 296 participants: 78 in cohort 1 and 218 in cohort 2. Of those in cohort 2, a total of 102 reported at one of three time points (one, three, or nine months) that they had participated in ten or more Vet-to-Vet groups during the period since the last research interview, and thus they make up cohort 2-V. Table 1 contains descriptive information about participants. Overall, the sample was predominantly male (95%) and white (66%) and was either currently married or had been married in the past (72%). About one-third had been homeless during the 30 days before the baseline assessment, and the average number of days worked in paid employment was close to zero. The severity of overall psychiatric symptoms on the BPRS was mild (mean±SD=1.3±.5), but mean±SD scores on the PCL-S (46±17) indicate significant PTSD symptomatology. Only 19% of participants were receiving VA service-connected disability benefits for either medical or psychiatric reasons.

Enlarge table

There were several significant differences between groups on baseline measures ( Table 1 ). Compared with cohort 1, cohort 2 had significantly higher scores in functioning (the GAF and Activities of Daily Living Scale) and on general empowerment (the Making Decisions Scale) and reported less money spent on alcohol and fewer days using drugs, but cohort 2 was also more likely to be homeless. There were no statistically significant differences between these two groups on baseline values of recovery attitudes, empowerment over illness, or confidence, nor were there differences on gender, marital status, race, VA service-connected disability benefits, age, education, days worked in the past 30 days, total income in the past 30 days, general life satisfaction, days used alcohol in the past 30 days, money spent on drugs in the past 30 days, and on both symptom severity scores.

Compared with cohort 1, cohort 2-V also had significantly higher scores on the GAF (functioning) and the Making Decisions Scale (general empowerment) and spent less money on alcohol during the prior 30 days. There were no significant differences between these two groups on the other primary outcome measures or on gender, marital status, race, homeless status, VA service-connected disability, age, education, days worked in the past 30 days, total income in the past 30 days, general life satisfaction, days used alcohol in the past 30 days, money spent on drugs in the past 30 days, days used drugs in the past 30 days, and on both symptom severity scores.

The groups also differed on follow-up rates. Compared with cohort 1, cohort 2 had a significantly higher three-month follow-up rate, and cohort 2-V had significantly higher follow-up rates at both three and nine months. To identify baseline predictors of successful follow-up, correlations between the number of follow-up interviews completed (zero, one, or two) and all baseline variables showed that within cohort 1, greater age (r=.37, p=.001) and lower severity of trauma history (r=-.27, p=.02) were both significant predictors of successful completion of future follow-up interviews. There were no significant correlations between entry characteristics and completion of follow-up assessments within either cohort 2 or cohort 2-V.

Primary outcomes

Of the four recovery-oriented primary outcome measures, only general empowerment had significant group differences on both the as-treated model (p=.03) and the intention-to-treat model (p=.02), with an effect size between small and moderate in magnitude (d=.33 and d=.31, respectively) favoring cohorts 2 and 2-V ( Table 2 ). Although the mixed-model analyses used all three time points, in Table 2 we present only the mean of the scores from the three- and nine-month follow-up time points combined for ease of presentation. The intention-to-treat comparison on general empowerment is graphically depicted in Figure 1 , which shows that the two groups were similar at baseline and at three months. At nine months, empowerment scores diverged, with an increase in cohort 2 and a decline in cohort 1. There were no significant interactions between cohort and time.

Enlarge table

In the as-treated models, there was, in addition to general empowerment, significantly greater confidence in cohort 2-V (p<.01), with a similar effect size (d=.30). No cohort-by-time interactions were significant. As seen in Figure 2 , confidence scores increased for both groups from baseline to three months. At nine months they sharply declined in cohort 1, while the scores for the cohort 2-V remained steady.

Secondary outcomes

The largest group effect across all outcomes was for the GAF, with a moderate to large effect for both the intention-to-treat (d=.69) and as-treated (d=.71) analyses. The other measure of functioning, the Activities of Daily Living Scale, also had significant group differences in both analyses, but the effect size was small in magnitude, at d=.13 for the intention-to-treat analysis and d=.08 for the as-treated analysis. Group differences for both measures of alcohol use were also significant in both models as well, but they also had small effects. No other significant differences were observed.

Discussion

In both the intention-to-treat analyses as well as the as-treated analyses, veterans enrolled after the Vet-to-Vet program was implemented (cohort 2 and 2-V) showed superior outcomes on several measures, including a measure of general empowerment and the GAF. As expected, the relationships were slightly stronger in the as-treated comparisons, which were limited to veterans in the second cohort who actually attended a substantial number of Vet-to-Vet meetings (cohort 2-V). The as-treated analysis revealed significantly higher scores for participants in cohort 2-V than for those in the group before the Vet-to-Vet program was implemented (cohort 1) on the measure of confidence, the measure of general empowerment, and the GAF. For those variables in which significant differences between cohorts emerged, outcomes followed a pattern in which the average score for the intervention-exposed cohorts (cohorts 2 and 2-V) either increased or remained steady, while outcomes for veterans in the control cohort decreased below baseline levels ( Figures 1 and 2 ), suggesting deterioration over time.

Existing studies suggest that consumers can be effective providers of mental health services in "professional" settings ( 23 , 24 , 25 , 26 ), and the employment of peers as service providers is an increasingly common practice ( 27 ). A feasibility study of the Peer-to-Peer Psychoeducation Program found that individuals with schizophrenia participating in the program showed an increase in knowledge of illness and trust in psychiatric treatment ( 28 ). However, to our knowledge, the study presented here is the first study of an education-based mutual-support group in the literature, and thus there are few studies with which it can be compared.

Studies of other types of group-based models of consumer-provided services and self-help provide some context for our findings. For example, a multisite randomized trial of peer-provided services compared outcomes of individuals randomly assigned to consumer-operated service programs with those of individuals randomly assigned to standard mental health services. Those in the experimental group reported higher levels of empowerment than those in the group with standard services ( 5 ). Consistent with these findings, one of the significant benefits of the Vet-to-Vet program was in general empowerment, although as in the study by Rogers and colleagues ( 5 ), these effects were generally small to moderate in magnitude. Dumont and Jones ( 29 ) also reported improvements in empowerment for a group randomly assigned to a peer-designed and peer-run crisis hostel, compared with those randomly assigned to treatment as usual. Finally, a cross-sectional comparison of individuals receiving services from a community mental health center found that those who chose to receive consumer-operated services had decreased rates of admission to inpatient treatment and shorter inpatient stays, compared with those who did not use these services ( 30 ).

One of the strengths of this study was in the diversity of measures. Although we used traditional measures of symptom severity and functioning, our primary outcome measures addressed domains we thought would be most likely to be influenced by peer-support services, such as recovery orientation, empowerment, and confidence. Although those in the Vet-to-Vet cohort did report significantly higher levels of general empowerment and confidence, the strongest effect size was for the GAF. Because this was the only global interviewer-rated measure in the assessment battery, it may also have been most influenced by bias, because the interviewer was not blind to the cohort that veterans were in or whether they participated in the Vet-to-Vet program, and in fact, participation in the Vet-to-Vet program could have been considered, in itself, to represent a higher level of functioning.

The most important limitation of the study is that in the absence of random assignment, we cannot be sure that the increase in empowerment and confidence did not reflect general trends in attitudes of VA consumers or in VA service delivery, as well as the possibility that differences on other unmeasured variables existed between groups. Randomized clinical trials are the standard for tests of efficacy because they reduce the risk of such biases, but they also require an extensive infrastructure and, to some extent, a restriction on choice of services, albeit within the context of informed consent. Because the Vet-to-Vet program, like other peer-provided services, is run by consumers, we believed that the imposition of a randomized controlled design might have fundamentally altered the experience of the program from a voluntary gathering of peers to an "assigned" treatment program or that it might completely discourage some from participating at all. We thus made every effort to ensure that the Vet-to-Vet program was implemented as intended—as a program run by and for veterans with professional assistance only as requested by participants.

Conclusions

These data suggest that participation in peer support can be associated with enhanced personal well-being, as measured by both recovery-oriented and more traditional clinical measures.

Acknowledgments and disclosures

This article is based on work supported by the New England Mental Illness Research, Education, and Clinical Center (MIRECC), Bruce Rounsaville, M.D., director; and the Office of Academic Affiliations, Department of Veterans Affairs Special MIRECC Fellowship Program in Advanced Psychiatry and Psychology. The authors thank Jim Silas, B.A., for research coordination and Mary Sperrazza, O.T.R./L., Laurie Harkness, Ph.D., Moe Armstrong, M.B.A., M.A., and the Vet-to-Vet facilitators for their support and assistance.

The authors report no competing interests.