Adoption by VA Residential Programs of Two Evidence-Based Psychotherapies for PTSD: Effect on Patient Outcomes

Residential treatment for posttraumatic stress disorder (PTSD) was the first specialized treatment program for PTSD in the U.S. Department of Veterans Affairs (VA), starting in the 1980s (1). When these programs were developed, there was a general expectation that the combination of communal experience with fellow veterans and intensive therapies would lead to greater efficacy over individual psychotherapy and medication management models (2). However, over time, informal empirical studies (3, 4) and more rigorous empirical investigations (5–8) revealed poor to modest outcomes.

Beginning in 2006, the VA initiated national training programs for mental health providers to enhance veterans’ access to two evidence-based psychotherapies (EBPs) for PTSD (9)—cognitive processing therapy (10) and prolonged exposure (11). Both treatments are considered first-line, trauma-focused EBPs for PTSD (12, 13). VA mandated that all sites of care must offer access to one or both EBPs (prolonged exposure and cognitive processing therapy) in-person, via telehealth, or by referral to non-VA care.

The national training initiatives in prolonged exposure and cognitive processing therapy offered an opportunity to improve services for veterans with severe, chronic PTSD who were in residential care. Two studies in residential programs found that patients who received cognitive processing therapy had improvements pre- to posttreatment (14, 15). In a sample of 101 veterans, Chard et al. (15) found that after receiving cognitive processing therapy, nearly 60% of those who had served in Iraq and Afghanistan and 40% of those who had served in Vietnam no longer met PTSD diagnostic criteria. In another study by Chard et al. (14), 42 veterans with comorbid PTSD and traumatic brain injury completed a residential program that was augmented with a modification of cognitive processing therapy. A large effect size was noted from pre- to posttreatment for PTSD and depression. However, neither investigation had a comparison group. In a study that had a comparison group, veterans who were treated after programwide implementation of cognitive processing therapy were compared with a historical control group of those treated in the program prior to implementation (16). Veterans who received cognitive processing therapy had greater symptom improvement, compared with those who did not. The effect sizes for depression and psychological distress were small and moderate, respectively, and the effects for PTSD were not initially significant but then improved over time.

We conducted a 5-year mixed-methods longitudinal investigation of the use of prolonged exposure and cognitive processing therapy in VA residential programs (17, 18). EBP adoption data have been reported from the first 2 years of the study (17, 18). Although many providers received VA training, use of prolonged exposure in these programs ranged from none to only a few patients. In contrast, adoption of cognitive processing therapy ranged from no use, use of only one aspect (e.g., specific worksheets), and strict adherence to the manual with all patients.

In this study, in collaboration with the VA’s Northeast Program Evaluation Center (NEPEC), which performs national program evaluation of mental health services, including VA’s PTSD clinical programs, the effects on patient outcomes of adopting prolonged exposure and cognitive processing therapy were examined at the program level in a naturalistic sample of veterans receiving VA residential PTSD treatment across the United States. Given that the evidence supporting use of prolonged exposure and cognitive processing therapy is based on outpatients, the generalizability to residential settings is unknown. In a study of veterans who received cognitive processing therapy, outpatients reported lower PTSD symptoms at both pre- and posttreatment, compared with residential patients (19). However, on the basis of evidence showing that prolonged exposure and cognitive processing therapy are effective treatments (13), as well as the research showing positive outcomes of cognitive processing therapy in individual programs (14–16), we predicted that patients in programs that used cognitive processing therapy and prolonged exposure more regularly would have better patient outcomes.

Methods

This study was exempted for review by the Yale Human Research Protection Program and approved by the VA Connecticut Healthcare System’s Institutional Review Board.

Procedure

Data presented here are part of a larger study on the implementation of two EBPs for PTSD in VA residential treatment programs (17, 18). All programs providing outcome data to NEPEC were invited to participate. Of the 48 VA residential PTSD programs, 39 that reported outcome data to NEPEC were surveyed as part of this study. No significant differences were found between programs that participated and those that did not with regard to number of beds or full-time employees and average length of stay. During our investigation (2008–2015), NEPEC changed two key aspects of its outcome monitoring, using different measures and adding a follow-up point collection immediately after discharge. Therefore, only data from the final year of collection (2015) were used in the analyses.

Participants

Programs and providers.

Of the 214 providers approached in the larger investigation, 171 (80%) participated in this wave of data collection (fiscal year [FY] 2015). Twenty-five providers (12%) in the original recruitment sample had retired or left their position, and 18 (8%) did not respond to recruitment attempts. Of the providers who participated, most were white (N=141, 82%). The mean±SD age of the 171 providers was 44.57±10.88. The primary professions represented included psychologists (N=76, 47%), social workers (N=61, 36%), psychiatrists (N=13, 8%), nurses (N=5, 3%), and others (N=7, 4%).

Veterans.

Out of 4,153 veterans who received residential PTSD care at one of the 39 participating residential rehabilitation treatment programs (RRTPs) in FY 2015, a total of 3,029 (73%) had NEPEC evaluation data and completed the civilian version of the PTSD Checklist (PCL) (20) at program entry. Of the 3,029 RRTP participants who completed the PCL at program entry, 2,447 completed the PCL at program discharge, and 784 completed it at 4 months postdischarge. The number of participants who had two or more observations (one of them had to be at program entry) was 2,834. For participants with more than one episode of care during the observation period, we examined the first episode of care for which the participant completed at least two PCLs.

The mean PCL baseline score reflecting PTSD symptom severity did not differ between veterans who completed the PCL at discharge and at 4-month follow-up (58.47±13.05) and those who completed the PCL only at discharge (58.54±11.96). (Possible scores on the PCL range from 0 to 80, with higher scores indicating symptom severity and a score of 33 or above suggesting diagnostic probability.) No significant differences between these groups were found on any other clinical measure. Minor demographic differences were noted between the veterans who participated in this study and those excluded for missing data. Participants were slightly younger (45.35±13.40 versus 46.99±13.51, p<.01). They were also more likely to have served in the wars in Iraq or Afghanistan (N=1,162, 55%, versus N=551, 49%, p<.01), to be male (N=2,362, 89%, versus N=978, 87%, p=.02), and to be classified as “other” race (N=325, 11%, versus N=85, 8%, p<.01) and less likely to be classified as black (N=696, 23%, versus N=304, 27%, p<.01). Therefore, the study sample was generally representative of the entire population served in the VA RRTPs.

Measures

NEPEC collects measures for all VA PTSD RRTPs at intake, discharge, and 4 months postdischarge. In this study, clinical data, in addition to PTSD symptom severity, included length of stay, alcohol and drug use, and distress.

The PCL, a 17-item self-report measure based on DSM-IV criteria, was used to measure PTSD symptoms. Veterans rated the extent to which they were bothered by each symptom during the past month on a 5-point scale ranging from 1, not at all, to 5, extremely. The Kessler Psychological Distress Scale (21) is an abbreviated version of the K10, a widely used measure of distress. It assesses, over the past 30 days, how often the person felt nervous, hopeless, restless or fidgety, so depressed that nothing can cheer him or her up, that everything was an effort, and feeling worthless; it uses a 5-point Likert scale. The Brief Addiction Monitor (22) is a 17-item multidimensional scale designed to assess frequency of substance (alcohol and drug) use. It includes both symptom and functional outcomes. We used the three items that sum the total number of days of substances used in the past 30 days (alcohol, illegal drugs, and prescribed medication) that are included in NEPEC’s outcome monitoring, coded as 0 (0 days), 1 (1–3 days), 2 (4–8 days), 3 (9–15 days), and 4 (16–30 days). Participants reported their satisfaction at discharge with attaining their recovery goals and with the care they received. Response options ranged from 1, not at all satisfied, to 5, completely satisfied.

To assign the EBP adoption codes, we used the following procedure. Providers completed a Web-based survey and a semistructured telephone interview on their use of prolonged exposure and cognitive processing therapy. Detailed descriptions of the survey and interview guide are provided elsewhere (23). Using the adoption of prolonged exposure and cognitive processing therapy codes that we established in earlier waves of data collection (18, 24), we coded the level of adoption of cognitive processing therapy and prolonged exposure in these programs for FY 2015. Six levels of adoption were coded for both prolonged exposure and cognitive processing therapy: the EBP was not adopted; some elements of the EBP were offered; the EBP was offered on a selective individual basis; different treatment “tracks” were developed, and those in a particular track received the EBP; the full EBP protocol was given to all patients; and the EBP was de-adopted.

Assigning the codes involved an interview and independent process by two licensed clinical psychologists (JMC and VS) with training in EBPs. These two authors (JMC and VS) independently reviewed surveys and interview transcripts to determine what treatments were being offered and providers’ reports of the percentages of patients receiving prolonged exposure and of those receiving cognitive processing therapy; established tentative codes (separately); discussed coding with one another and reached consensus when discrepancies existed; and verified coding with survey data. For the purposes of these analyses, we then collapsed the six categories for both prolonged exposure and cognitive processing therapy into three categories that combined prolonged exposure and cognitive processing therapy adoption: little to no adoption of prolonged exposure or cognitive processing therapy; some adoption of prolonged exposure or cognitive processing therapy or adoption for selected patients; and high adoption, with most to all elements of prolonged exposure or cognitive processing therapy adopted or most to all patients receiving prolonged exposure or cognitive processing therapy.

In an attempt to corroborate our self-report quantitative measure of prolonged exposure and cognitive processing therapy use, we compared it with contemporaneous patient-level receipt of EBPs for PTSD by using electronic medical record templates mandated for EBP documentation in FY 2015. Agreement between therapist self-reported EBP delivery and patient receipt of EBT as measured by progress note templates was good (r=.69–.82) (25).

Data Analysis

Analyses were conducted with SAS, version 9.4. First, we compared the characteristics of the three adoption groups at the time of admission by using chi-square tests for categorical variables, analysis of variance for continuous variables, and Tukey tests for post hoc comparisons. Variables for which a significant baseline difference was noted were entered as covariates into the longitudinal data analysis model. A linear mixed model (PROC MIXED) was used to examine symptom change in a model defined by adoption level, time, and time × adoption level, with the analysis controlling for baseline symptom score and baseline group differences. Time was defined as categorical (i.e., discharge and follow-up, because of variation in time between intake and discharge at each RRTP). Random effects in the model included site (the effect of each RRTP) and individuals who were nested within sites that were nested within implementation group. Finally, because of the differences in the return rates of NEPEC surveys among the three experimental groups and the fact that the total number of the entire population is known (all PTSD RRTP users in FY 2015), we computed population weights corrections. These weights were then entered into the analyses to account for this bias.

Results

In FY 2015, eight RRTPs (N=613 veterans) reported little or no adoption of prolonged exposure or cognitive processing therapy, nine (889 veterans) reported some adoption, and 22 (1,527 veterans) reported high adoption. Table 1 presents the general characteristics of the veterans in each group at admission to the RRTPs. Among clinical variables, groups differed on baseline distress and substance use but not on severity of PTSD symptoms or length of stay. In addition, we examined the proportion of veterans in each of the three adoption groups who completed discharge and follow-up assessments. Groups differed in the percentage of participants who completed discharge assessments (little or no adoption, 81%, N=499; some adoption, 73%, N=651; and high adoption, 84%, N=1,297; χ²=51.3, df=2, p<0.001) and the percentage of participants who completed follow-up assessments (little or no adoption, 18%, N=144; some adoption, 19%, N=226; and high adoption, 22%, N=414; χ²=8.81, df=2, p=0.012), suggesting a higher proportion of assessment completers in the high-adoption group.

Results of the linear mixed-model analyses are presented in Table 2, Table 3, and Table 4. PTSD, distress, and alcohol use symptoms improved over time. There were no effects of adoption group or a group × time interaction for any outcome. Predictors of lower PTSD symptom severity at discharge or follow-up were male gender, African-American race, and lower PTSD and distress at admission (Table 2). Predictors of lower distress at discharge or follow-up were older age and lower distress at admission (Table 3). Predictors of lower alcohol use at discharge or follow-up were serving in the Iraq or Afghanistan theater and lower alcohol use at admission (Table 4). No difference on satisfaction was noted between the groups.

Temporal effects are presented in Figure 1. For all outcomes and all groups, there was a similar pattern of change. Symptoms decreased at discharge and then increased somewhat at follow-up. The effect size for reduction in PTSD symptoms between admission and discharge was large across groups (little or no adoption, d=0.97, 95% confidence interval [CI]=0.88–1.07; some adoption, d=0.92, 95% CI=0.82–1.01; and high adoption, d=0.96, 95% CI=0.90–1.02). The effect size between admission and follow-up was medium (little or no adoption, d=0.70, 95% CI=0.53–0.87; some adoption, d=0.47, 95% CI=0.32–0.62; and high adoption, d=0.54, 95% CI=0.44–0.65).

Discussion

Veterans treated in RRTPs with high levels of adoption of prolonged exposure or cognitive processing therapy did not experience more improvement than veterans treated in RRTPs with less or virtually no adoption. This finding is surprising given the recommendations of PTSD treatment guidelines (13, 26) as well as results of a prior study (16) showing superior outcomes in an RRTP after adoption of cognitive processing therapy, compared with outcomes before adoption. Nevertheless, veterans treated in RRTPs experienced an improvement in PTSD symptom severity in the course of residential treatment. These findings suggest that residential programs are helpful and may have improved over time (5, 8, 27). The improvement in symptoms that veterans experienced, however, was not related to the degree to which the program had adopted EBPs for PTSD.

After discharge, an increase in PTSD symptoms, distress, and substance use was observed in all experimental groups, suggesting that a better understanding of aftercare services provided postdischarge is needed. Such an understanding will help ensure that therapeutic gains are maintained after discharge from these expensive and intensive treatment programs. It appears that the effect of decreased substance use during the residential stay (because of either restrictions on access to substances or the therapeutic intervention) as measured at discharge diminished at 4-month follow-up (no significant differences were noted between the three groups in the level of substance use between admission and 4 months after discharge). This result suggests that the residential program should prepare veterans to better cope with substance use issues on return to the community.

Detecting the impact of EBP use on outcomes may be more difficult in residential settings, compared with outpatient settings, where the studies that have informed PTSD guidelines have been conducted. A prior study found that residential patients had greater PTSD symptom severity and more comorbidity and were less responsive to cognitive processing therapy, compared with outpatients (19). Also, most VA RRTPs offer between 30 and 35 hours per week of programming on weekdays and another 6 hours over the weekend. Even in a high-adoption EBP site, the time spent actually doing cognitive processing therapy or prolonged exposure is small in proportion to the total package of care, which likely made it more difficult to detect the true effects of prolonged exposure and cognitive processing therapy. Many of the programs reported using EBPs for other comorbid conditions common among individuals with PTSD, such as cognitive-behavioral treatment for insomnia. In addition, it is possible that the range of treatments now provided in the course of residential care, as well as the treatment milieu, add to program effectiveness. It is also important to note that some programs that did not offer prolonged exposure or cognitive processing therapy offered another EBP for PTSD, such as eye movement desensitization and reprocessing (17). Notably, although it may be more difficult to detect outcomes of prolonged exposure and cognitive processing therapy in residential settings, this investigation nevertheless provides promising results related to the number of residential programs using these treatments, particularly compared with some VA outpatient settings, where relatively low use has been reported (28).

Another possible explanation for the null effect of implementation is the frequent use of group versus individual cognitive processing therapy. The VA–Department of Defense PTSD guideline (13) states that limited data on the efficacy of group therapy, including group cognitive processing therapy, indicate that group therapy is not as effective as individual therapy (29). However, it is unlikely that this possibility fully explains our findings, because group cognitive processing therapy has evidence of efficacy, albeit a small difference between cognitive processing therapy and present-centered therapy (30).

A number of unavoidable methodological issues may temper inferences drawn from our results. There may have been low fidelity to prolonged exposure and cognitive processing therapy protocols that was not captured by our measure of adoption. Also, some of the 22 programs that reported high use may have overestimated utilization, thereby diluting results. In addition, although some programs had overall high adoption, this is no guarantee that every veteran in that program received or completed a full course of the EBP. A stronger design would involve linking patient outcomes to specific treatments received and assessing the quality and fidelity with which treatment was delivered. Finally, data collection varied across sites, and there was significant loss to follow-up. Veterans in the programs with little or no EBP adoption were less likely to complete assessments at discharge and follow-up. In addition, the low 4-month follow-up rate (even though similar across programs) may also have reduced the ability to detect effects, especially if healthier veterans tended to respond.

The use of intention-to-treat analyses and adjustment for baseline differences among program types may have failed to adequately control bias due to these factors. It is entirely possible that unmeasured differences between participating sites may have masked any effect of delivering an EBP. Unfortunately, because we used existing VA program evaluation data, we were limited to the data that were collected through this national evaluation process. In addition, the primary outcome was based on a self-report measure of PTSD, not a clinician-administered measure. Demand and allegiance effects cannot be ruled out.

These findings suggest a number of opportunities for further study. More carefully controlled and independent assessment protocols and information about the fidelity of EBP adoption are needed in order to yield a more conclusive answer about the relative benefit of EBP use in residential programs. More details about the treatments being provided by programs with little or no adoption could also be helpful in developing treatments for individuals who may not be appropriate for or benefit from EBPs.

Residential treatment for PTSD is a small and distinct part of VA care, often serving patients with the most chronic and severe conditions and with complicated symptom presentations. Despite the lack of differences among EBP adoption groups, our results are encouraging for the PTSD residential programs because they indicate meaningful improvement in this segment of the veteran population.

Conclusions

Although veterans treated in VA PTSD RRTPs experienced a high degree of symptom improvement at discharge, veterans in programs that used prolonged exposure and cognitive processing therapy with most or all their patients did not experience greater reductions in PTSD or depression symptoms or in alcohol use, compared with veterans in programs that did not use prolonged exposure or cognitive processing therapy. Failure to find a treatment effect should not be interpreted definitively as indicating that implementation of these two EBPs in VA RRTPs does not matter. It is possible that the multiple influences in residential programming may overshadow the effects of EBPs. This conclusion, however, seems premature on the basis of program evaluation data, such as those collected here. More carefully controlled research is needed to ensure more complete data capture outside routine clinical care and to more specifically document the treatments actually received, as well as the amount and quality of EBP provided.