Impact of the Seeking Safety Program on Clinical Outcomes Among Homeless Female Veterans With Psychiatric Disorders

As the proportion of women among military personnel has grown in the past three decades, so has concern about the health and well-being of female veterans, who have three to four times greater risk of homelessness compared with their civilian counterparts ( 1 ). Women nevertheless still make up a relatively small proportion of the homeless veteran population ( 2 ). In the Department of Veteran Affairs' (VA's) national Health Care for Homeless Veterans Program, for example, women make up about 3% of the veteran population served, although they generally make up about 5% of most VA patient populations. There is little evidence that homeless female veterans have particular difficulty accessing specialized VA homelessness services. Although there is some evidence that female veterans may choose the VA as a care provider for some services at lower rates than male veterans, particularly for substance abuse services ( 3 ), there is little evidence that women use the VA at proportionately lower rates than men ( 3 ). However, there is a growing consensus that in order to provide the best care to women veterans, it may be important to address their unique health, mental health, and psychosocial needs through specialized programs.

There has been a recent growth in interest in integrating mental health care for clients with multiple psychiatric and substance abuse problems. The Women, Co-occurring Disorders, and Violence Study was conducted at nine sites across the country and found modest but statistically significant effects of integrated treatment over traditional care ( 4 ). One of the intervention strategies employed in that study was a manualized cognitive-behavioral therapy approach titled Seeking Safety ( 5 ). Seeking Safety is an intervention consisting of 25 individual modules that address issues of safe behaviors and relationships, life skills, and relapse prevention. A complete description of the intervention can be obtained at www.seekingsafety.org. It has been evaluated in various studies, including pilot trials ( 6 , 7 , 8 ); dissemination studies ( 9 , 10 , 11 ), two of which involved veterans ( 10 , 11 ); an observational study ( 12 ); and two randomized controlled trials ( 13 , 14 ). In all of these studies, clients appeared to respond positively to the intervention, and Seeking Safety outperformed treatment as usual in all of the controlled and randomized controlled trials ( 12 , 13 , 14 ).

In this article we present data on the one-year client-level outcomes of homeless women veterans who received Seeking Safety services compared with those of women who received care from the Homeless Women Veterans Programs before Seeking Safety became available. The principal hypothesis was that the persons who were offered Seeking Safety services would have superior clinical and functional outcomes, compared with persons in the comparison condition.

Methods

Although double-blind randomized clinical trials provide the least biased assessment of program effectiveness, their generalizability to actual practice is often uncertain. In addition, double blinding cannot be well maintained with behavioral treatments like Seeking Safety, and it is difficult to ensure that raters remain blind to treatment assignment. The VA Seeking Safety initiative was thus evaluated with a pre-post nonequivalent control group design.

Each of 11 VA medical centers was funded to establish a specialized Homeless Women Veterans Programs. Funds were used to hire staff who would conduct outreach, intake, and case management for women in the program and to provide entry into time-limited contract residential treatment services as funding allowed.

Training for the specialized Homeless Women Veterans Programs was conducted in two stages, which coincided with the two phases of the evaluation. In phase I, case managers were oriented to program expectations and the data collection requirements associated with providing specialized clinical services to homeless women within the VA system. Phase I involved no specific intervention other than case management, assessment of the specific needs of clients and referral for treatment, and time-limited transitional residential treatment available through the Homeless Women Veterans Programs. (Transitional residential treatment varied in length and level of services on the basis of client need and each site's specific program design and funding.) Many Homeless Women Veterans Programs also provided substance abuse counseling through the program clinicians directly. Phase I lasted from January 2000 to June 2003. Phase II lasted from June 2003 to December 2005. The intervention lasted for six months. After the baseline interview, follow-up interviews were obtained every three months over the course of one year.

After the basic programs were well established, training for phase II, the Seeking Safety phase, was initiated. New staff were not hired for phase II, rather the same clinicians who worked for the specialized program served in both phases of the study. Seeking Safety training began with a two-day, face-to-face orientation to Seeking Safety principles and practices that was offered to both the staff who were involved with the Homeless Women Veterans Programs and to other mental health staff from each site who would be providing Seeking Safety to homeless women veterans clients. Clinicians on the project were case managers with typically no prior experience in implementing any manualized cognitive-behavioral therapy.

Training continued for the duration of the project through monthly teleconferences that reviewed study progress. In addition, the program had a national clinical supervisor who had been trained by the developer of Seeking Safety (LN) and who consulted with her during the study. This supervisor reviewed audiotapes of sessions from each clinician at least once a month, scored the tapes for fidelity to the model, and had at least monthly calls with individual clinicians to review clinical issues and address any fidelity issues that arose from reviews of the audiotapes. Before clinicians began providing Seeking Safety to their clients, they were required to receive "certification" from the clinical supervisor. This process involved reviews of audiotapes of several client interviews, so that the clinical supervisor was ultimately confident that the clinicians were skilled in the Seeking Safety model. There was also a great deal of focus on basic clinical skills for conducting psychotherapy with this population and on using a treatment manual.

Sample

Veterans were eligible for the study if their case manager determined that they were literally homeless or at high risk of becoming homeless and if they were not receiving VA health services for more than six weeks at the time of program entry, had psychiatric or addiction problems, were cognitively able to complete the interviews, were medically and psychiatrically stable, were interested in participating in available mental health services, and were willing to be interviewed quarterly during a one-year follow-up period. A total of 359 veterans were recruited during phase I and an additional 91 veterans were recruited in phase II. Because structured instruments were not used to determine study eligibility, questions about eligibility were reviewed with the study coordinator (RD) to ensure consistency in eligibility decisions.

Each participant gave written consent to participate in the study, and phase II participants also gave consent to receive Seeking Safety treatment services. Veterans were included in analyses only if they had completed at least one follow-up interview, which consisted of 70% of the baseline sample (450 of 643 participants). There were no statistical differences between those who completed at least one follow-up and those who completed only a baseline interview on demographic characteristics, social support, drug and alcohol use, posttraumatic stress disorder (PTSD) scores, self-esteem, psychiatric distress, income, days homeless, incarceration, and total number of traumas experienced.

Measures

Client characteristics included sociodemographic characteristics; employment status; use of public support programs, such as Medicaid and city welfare; use of services in non-VA settings in the three months before program entry; and military history.

Health status was measured with a variety of standardized self-report measures, including the psychiatric, alcohol, and drug composites from the Addiction Severity Index (ASI) ( 15 ); the 30-item Symptom Checklist Revised (SCL-30-R) ( 16 ); a measure of self-esteem; and the 12-Item Short-Form Survey (SF-12) measures for physical and mental functioning ( 17 ). Extensive trauma histories were obtained on lifetime and recent traumas experienced by each client, and the PTSD Checklist was utilized to assess PTSD symptoms ( 18 ).

Housing status was measured by questions concerning how many days in the past 90 the client has slept in each of 12 types of places. Responses were collapsed into three categories reflecting the number of days the veteran was homeless, living in an institution, or housed.

Participants were also asked how many people they felt close to in each of nine relationship categories. A continuous social support variable was computed indicating the total number of persons to whom the client felt close.

Data analyses

First, participants in phase I and phase II were compared on baseline characteristics by using chi square tests and independent-samples t tests. Factors that were significantly different between phase I and II participants were then included in multivariable models to adjust for potential confounding. Mixed models for longitudinal data were used to compare outcomes over one year, controlling for baseline differences, baseline levels of the outcome of interest, and dropout from the study. Each model included terms representing time, treatment group, and an interaction of time and treatment group. Time was treated as a categorical variable. Adjustment for characteristics of patients lost to follow-up was made with an interaction between the baseline measure and time ( 19 ). The analyses were conducted by using the MIXED procedure in SAS statistical software.

Results

We first compared study participants (N=450) to clients of the Homeless Women Veterans Programs who were not enrolled in the study (N=1,125), to evaluate the representativeness of the sample, using administrative intake data. In general, compared with women not enrolled in the study, those enrolled in the study were older (p=.05), were more likely to be literally homeless (p=.02), had spent more time homeless in their lives (p<.001), were more likely to report alcohol dependence (p<.001) or drug dependence (p<.001), and experienced more symptoms of mental illness (p<.001). As a result, it is likely that those enrolled in the evaluation study represented the more severely ill women enrolled into the Homeless Women Veterans Programs.

During the time of recruitment for phase I of the study, 1,337 women entered the Homeless Women Veterans Programs and 481 (36%) were recruited into the evaluation study. During phase II, 379 women entered the program, and 110 (29%) were recruited into the study. The lower recruitment rate in phase II was significant (p=.04). This lower rate was understandable because in this study, Seeking Safety consisted of up to 25 therapy sessions, and not all women were interested in participating in such a lengthy intervention. The intensiveness of the Seeking Safety intervention was reflected in the fact that women enrolled in the study during phase II had a significantly higher number of case management contacts per month (an average of two) than those enrolled in the study during phase I (average of 1.7) (p<.001). However, women in both phases appeared to stay in the program for roughly equivalent periods of time. The average number of months in the program during the first year for phase I participants was ten months, whereas for phase II participants it was nine months.

Baseline characteristics

As shown in Table 1 , phase I clients were significantly younger than phase II clients. Phase II clients were significantly less likely to have been employed full-time for most of the previous three years and more likely to have been unemployed. There were no baseline differences on other demographic characteristics.

Enlarge table

The rates of reported exposure to lifetime trauma in this sample were high (data not shown), demonstrating that Seeking Safety was likely to be a clinically appropriate intervention. Although there were few significant differences in the experiences of particular types of trauma by phase, it is notable that a majority of women reported a history of rape (N=306, or 68%), many had a history of prostitution that they reported as traumatic (N=158, or 35%), and a few reported a history of combat experience (N=45, or 10%).

It should be noted that the phase I sample (N=359) was four times larger than the phase II sample (N=91) for a number of reasons. First, there were fewer women enrolled into the overall Homeless Women Veterans Programs from which phase II clients could be recruited. Although 1,337 women entered the program during phase I, only 379 entered during phase II. Caseloads had been maximized by phase I, and client turnover was limited. Second, there was a shortened recruiting time for phase II because the certification process for Seeking Safety was lengthier than anticipated, in part because of the inexperience of the clinicians in delivering manualized treatments and cognitive-behavioral therapy.

Table 2 compares the clinical characteristics of phase I and II clients. Phase II clients had significantly lower baseline SCL-30-R scores (p=.03) and significantly higher numbers of days spent in residential treatment in the previous three months (p=.002). These variables were thus entered as covariates in all multivariable longitudinal models of the effects of Seeking Safety.

Enlarge table

Follow-up rates

There was significant dropout from the study over time. At three months follow-up rates were 80%–85%, but at six months they fell to 63%–67%, at nine months they fell to 40%–56%, and at 12 months they fell to 27%–53%. There was significantly less dropout among phase II clients at three months (p=.02), and there was significantly higher dropout among phase II clients at 12 months (p=.001), although there was no significant difference between groups in dropout at six- and nine-month time points.

Outcomes

Table 3 presents the results of multivariate models of outcomes over time. The table presents the means for each group at each time point, the significance of the main effects for phase and time as well as the significance of the interaction between time and phase. In every case, there was a significant effect of time such that for every outcome observed there was significant improvement over time, regardless of phase.

Enlarge table

Participants in phase II evidenced significantly better outcomes than those in phase I on days worked in the past 30 days (p=.002); social support (p=.03); SCL-30-R symptom scores (p=.03); PTSD Checklist scores (p=.03), particularly on the arousal (p=.04) and avoidant (p=.007) subscales; and the ASI psychiatric composite (p=.03). In addition, there were significant interactions between phase and time for social support (p=.04), SCL-30-R scores (p=.04), and PTSD Checklist scores (p=.04), particularly on the arousal (p=.02) and avoidant (p=.03) subscales. The interactions indicated greater rates of improvement for phase II clients. However, Seeking Safety clients were significantly more likely to have used drugs in the past 30 days (p=.04).

Discussion

This study sought to evaluate the effectiveness of a multisite dissemination effort to implement Seeking Safety for homeless female veterans in the VA, using a modestly intensive, but sustained, training effort. The clinicians were case managers with little or no prior training in conducting psychotherapy, in conducting cognitive-behavioral therapy, or in using a manualized intervention. Training was characterized by limited face-to-face contact between trainers and trainees and a heavy reliance on teleconference, audiotape review, and phone contact with the clinical supervisor over almost four years. Training focused on Seeking Safety specifically and also on basic counseling skills. Over a one-year follow-up period, individual veterans who participated in the Homeless Women Veterans Programs after the implementation of Seeking Safety showed modest but significantly greater improvement in psychiatric symptoms, PTSD symptoms (particularly hypervigilance and avoidant behavior), and social support, compared with women in the comparison condition.

Seeking Safety was originally developed to address the concurrent treatment needs of clients with both PTSD and a substance use disorder. The application of Seeking Safety to the Homeless Women Veterans Programs was thus a novel application of the model to a homeless population and more specifically to homeless women veterans. This seemed appropriate because clients in this program reported high levels of traumatic events and had a wide array of problems related to substance use. Moreover, Seeking Safety was designed to allow a high degree of flexibility, and this appeared to be well accepted by clinical staff who reported anecdotally that it worked well with clients. The model was conducted in both group and individual modes, each topic was applicable to a wide range of problems in addition to PTSD and substance abuse, and the topics can be covered in any order as each is freestanding.

Second, even in the absence of a formal diagnosis of either PTSD or substance abuse or dependence, the skills taught in Seeking Safety are appropriate for women who are homeless. The central tenet of this treatment is that safety is the first priority, as operationalized by abstinence from drugs and alcohol (if there is an alcohol problem), avoidance of self-destructive behavior, the establishment of healthy social networks, and general self-protection (for example, HIV protective behavior) and healthy relationships ( 20 ). The treatment focuses on a variety of coping skills (for example, cognitive, behavioral, and interpersonal), and each of the 25 topics addresses one of these skills in a specific way ( 20 ). The high levels of traumatic experiences among women who had found themselves homeless meant they have had a great deal of experience with unsafe situations, whether or not they were related to substance abuse or PTSD, and they could thus benefit from the skills taught by Seeking Safety. Although the mechanism of the treatment effect is not clear from our data, our results indicate that Seeking Safety was associated with positive effects on multiple dimensions, including improved social support.

One area that unexpectedly did not show a significant positive effect of Seeking Safety was substance use. One potential explanation for this is the fact that many of the clinicians worked with clients individually on their substance use issues in phase I, in addition to referring them to other VA substance abuse services. The fact that many clients were in a controlled environment (that is, residential treatment programs) for at least part of the time and could be discharged for substance use was likely to have reduced differences between phases in substance abuse over the course of the study. However, not only were there no positive effects of Seeking Safety, at the follow-up interviews persons in Seeking Safety were significantly more likely to have used drugs in the past 30 days. The explanation for this is unknown.

Strengths and limitations

Because participants were not able to be randomly assigned to Seeking Safety, we conducted a two-phase observational design. Data analyses revealed relatively few differences at baseline across the two phases, and we utilized multiple regression analysis to adjust our assessment of outcomes for baseline differences. However, because unmeasured confounding might have biased the results, we cannot be sure that group differences were entirely attributable to the intervention. The risk of selection bias may be an unavoidable hazard of studying outcomes in real-world system-change projects because simultaneous random assignment to alternative conditions within the same system is often not possible. Two strengths of our design were that there was no contamination effect in the implementation of Seeking Safety (that is, clients sharing Seeking Safety materials across conditions) and there was no confound by clinician or treatment (that is, unintentional implementation of the model in the control condition).

Several additional limitations require comment. First, the VA is a highly integrated health care system with similar rules and procedures at all facilities. This may have facilitated implementation of a multisite dissemination project but leaves it unclear how generalizable this experience is to other, less integrated, health care systems. However, prior studies of Seeking Safety in other frontline settings ( 4 , 9 , 12 ) show comparable results, with Seeking Safety outperforming intensive treatment as usual.

It must also be acknowledged that substance use was assessed only by self-report interviews in this study and was not verified by biochemical testing. This may be particularly problematic because in most cases the client's clinician was the person conducting the research interview. Thus, if the client wanted to create a good impression for her clinician, or if report of substance use may have resulted in eviction from housing, self-report results may not have been unbiased. This was true of both phases, so it should not have introduced a differential bias. However, the likely underreporting of substance use probably reduced our power to detect effects of Seeking Safety on substance use behavior.

Finally, the substantial loss to follow-up in this study limits our ability to make strong conclusions about the long-term effects of Seeking Safety beyond the limits of the six-month treatment period itself. However, the loss to follow-up was equivalent during the first six months of each phase, and several of the outcome differences were greatest in magnitude between the six- and nine-month periods, when follow-up rates were not different across the phases and when Seeking Safety treatment would have been ending. We do not believe that loss to follow-up biased results in favor of phase II participants. We also examined whether participants who dropped out of the study were significantly different from those who continued with follow-up interviews, and we found no statistically significant differences on a wide range of demographic, clinical, and functioning variables.

Given the experience of the clinical staff involved with this project, and the fact that supervision throughout phase II included a focus on developing clinical therapeutic skills, the results found here could potentially be explained by improvements in the skill with which therapy was delivered. These results thus may not be specific to Seeking Safety per se, but rather to the application of cognitive-behavioral therapy interventions more generally. Typical homelessness programs focus on housing, benefits, assessment, and referral for mental health problems, and they generally do not involve the provision of therapy. These results suggest that a variety of cognitive-behavioral therapy interventions may prove to be helpful for improving outcomes among homeless women.

Conclusions

This study provides evidence that Seeking Safety may be a beneficial intervention for homeless women. This was particularly notable because all clients received intensive services as part of their care within the homeless women veterans programs, so that Seeking Safety was compared with intensive services as usual. The study also indicates that case managers without prior training in psychotherapy, cognitive-behavioral therapy, or manualized treatments could learn Seeking Safety and implement it successfully in systems serving homeless women veterans. Finally, results suggest that a beneficial effect may not be specific to Seeking Safety but to the provision of psychotherapeutic services from within a program designed to address homelessness.

Acknowledgments and disclosures

The authors acknowledge the site coordinators and clinical staff of the Homeless Women Veterans Programs for providing not only superior clinical care for their clients but also for recruiting and interviewing for this evaluation study. Staff include Dallas Campbell, Vanessa Gordon-Brown, Maryannie Hughes, Merrlyn Jones, Anita Salamon, and Linda Wright (Atlanta); Heidi Behr, Lauren Dever, Victoria McKeever, and Meredith Powers (Boston); Anne Burke, Franklin Garret, and Mara Kushner (Brooklyn, New York); Alva Reid (Cincinnati); Ruth Fiala and Toni Johnson (Cleveland, Ohio); Desrali Jackson (Dallas); Alicia McCarthy (Houston, Texas); Carole Goguen, Vivian Hines, Florence Hobsen, Brenda Milton, and Patricia Plost (Los Angeles); Lucille Gibbs, Kesa Labanowski, and Joann Pinotti (San Francisco); Katherine Gerard, Sandra Perry, Ann Shahan, Randy Turner, and Charlene Van Meter (Seattle, Washington); Ginny Aubel, Carol Griffiths, and Kathy Palmer (Tampa, Florida). The authors thank Martha Schmitz for her work as clinical supervisor on this project. The authors also acknowledge Gay Koerber and Theresa Hayes in the Department of Veterans Affairs Central Office for their leadership in establishing and supporting this initiative. Finally, the authors thank the women veterans who volunteered to participate in this project.

The authors report no competing interests.