Implementation of Multifamily Group Treatment for Veterans With Traumatic Brain Injury
Abstract
Objective
This study evaluated the initial efficacy and feasibility of implementing multifamily group treatment for veterans with traumatic brain injury (TBI).
Methods
Veterans at two Veterans Affairs medical centers were prescreened by their providers for participation in an open trial of multifamily group treatment for TBI. Enrollment was limited to consenting veterans with a clinical diagnosis of TBI sustained during the Operation Enduring Freedom–Operation Iraqi Freedom era, a family member or partner consenting to participate, and a score ≥20 on the Mini-Mental State Examination. The nine-month (April 2010–March 2011) trial consisted of individual family sessions, an educational workshop, and bimonthly multifamily problem-solving sessions. Interpersonal functioning and symptomatic distress among veterans and family burden, empowerment, and symptomatic distress among families were assessed before and after treatment.
Results
Providers referred 34 (58%) of 59 veterans screened for the study; of those, 14 (41%) met criteria and consented to participate, and 11 (32%) completed the study. Severity of TBI, insufficient knowledge about the benefits of family involvement, and access problems influenced decisions to exclude veterans or refuse to participate. Treatment was associated with decreased veteran anger expression (p≤.01) and increased social support and occupational activity (p≤.05), with effect sizes ranging from .6 to 1.0. Caregivers reported decreased burden (p≤.05) and increased empowerment (p≤.01).
Conclusions
The results supported implementation of a randomized controlled trial, building in education at the provider and family level.
Traumatic brain injury (TBI), affecting an estimated 22% of service members wounded in military operations in Iraq and Afghanistan, is often complicated by comorbid posttraumatic stress disorder (PTSD), depression, and other mental health problems (1,2), having a dramatic impact on the lives of the veteran’s family (3–6).
Veterans with TBI may also have deficits in memory and executive functioning that create challenges in household management and lead to friction between family members if not understood and addressed. Yet veterans and family members often lack information about the veteran’s condition and treatment needs. This gap in information and services erodes family resilience, contributing to frustration, family conflict, child distress, and consequent barriers to community reintegration (1,7–9). Although there is a growing evidence base supporting the efficacy of family and couples treatment in reducing PTSD symptoms (10–13), there is no established family-based treatment for veterans with TBI sustained during the Operation Enduring Freedom–Operation Iraqi Freedom (OEF/OIF) era.
This study evaluated the feasibility, acceptability, and initial efficacy of an intervention developed to address this gap in services. The intervention is an adaptation of multifamily group treatment (14), an evidence-based treatment for serious mental illness that uses support, education, and problem solving to reduce symptoms and improve functional outcomes. This method has been used to treat survivors of civilian TBI and family members and has had positive therapeutic effects (15).
Because underutilization of mental health services has been identified as a problem for the OEF/OIF cohort (16,17), we evaluated problems in both implementation and initial efficacy of the intervention. A companion article presents the qualitative findings of a study of postintervention focus groups about the experience and value to participants (18).
In this article and the companion article, we addressed the following implementation issues using a mixed-methods approach: systems issues or stakeholder concerns influencing buy-in at the veteran, family, and provider level; impact of group composition (level of impairment and family relationship) on group participation and retention; and adaptations of content or structure to optimize responsiveness to problems identified by veterans and family members.
We evaluated treatment efficacy by measuring change in behaviors hypothesized to respond to skills training in group problem solving. Specifically, we hypothesized that participation in the multifamily group would result in improved anger management and greater use of social support among veterans and reduced burden and increased empowerment among family members (primary outcomes). On a more exploratory basis, we hypothesized that veterans would report reduced depressive and PTSD symptoms and increased community reintegration and that greater neurocognitive impairment at baseline would be associated with less improvement in treatment outcomes. These outcomes were designated as exploratory because they were targeted less directly by intervention components or were predictors of treatment response.
Methods
Participants
Veterans with a clinical diagnosis of TBI or a positive Defense and Veteran’s Brain Injury (DVBIC) screen (19) for TBI sustained during the OEF/OIF era were identified by their polytrauma physician at the J. J. Peters Veterans Affairs (VA) Medical Center (VAMC) in the Bronx, New York, and through referral by a clinician or caseworker or examination of electronic medical records at the Durham, North Carolina, VAMC. The diagnosis of veterans identified by DVBIC screen was confirmed by the treating clinician.
Because multifamily group treatment is generally conducted as a closed group (all families begin and complete group sessions in tandem), recruitment of each group was completed within two to three months, in order to retain early recruits. Within this relatively short window, clinicians used the methods described above to identify and prescreen potential candidates and approve and refer those they considered appropriate for inclusion in the study.
Study staff contacted each of the referred veterans to arrange a meeting with the veteran and a family member or partner to provide a full description of the study. All veterans and family members who agreed to meet and gave written informed consent were screened for eligibility. The Mini-Mental State Examination (MMSE), which evaluates basic orientation, memory, and attentional processes, identified veterans whose level of cognitive impairment precluded participation. Possible scores range from 0 to 30, with higher scores indicating more intact cognitive functioning. Eligible veterans had a clinically confirmed diagnosis of TBI, at least one family member or significant other who consented to participate in the study, and a MMSE score ≥20. Veterans were excluded if they or the identified family member had an active psychosis or a severe substance use disorder. The study protocol was approved by the J. J. Peters VAMC Institutional Review Board.
Measures
Veterans and family members were evaluated before and immediately after treatment. Measures were included to test the study hypotheses and to characterize participants by using clinical and subclinical measures of distress and dysfunction frequently employed in psychosocial research to inform future work.
Veterans.
Baseline neurocognitive functioning was evaluated by the MMSE and by measures of attention and mental flexibility (Trail Making Test, parts A and B) (20) and memory (California Verbal Learning Test II) (21). The Patient Health Questionnaire (PHQ) (22) evaluated the presence of depressive, anxiety, and alcohol use disorders. The Sydney Psychosocial Reintegration Scale (23) measured change in occupational activity and interpersonal relationships, and the PTSD Checklist (PCL) (24) quantified level and type of posttraumatic stress disorder (PTSD) symptoms.
Caregivers.
The Caregiver Burden Inventory (25) evaluated the developmental, physical, social, emotional, and time burden associated with the patient’s dependence on the caregiver. The family subscale of the Family Empowerment Scale (26) assessed caregiver perceptions of coping efficacy.
Veterans and caregivers.
Three measures were administered to both veterans and family members. The 24-item AX Scale (27) measured anger management, including anger-in (suppression of angry feelings), anger-out (expression of anger toward property or people), and anger control (the frequency of attempts to control expressions of anger). The Abbreviated Duke Social Support Index (28) quantified growth in the social network (number of weekly contacts) and perceived support. The widely used Center for Epidemiological Studies Depression Scale (CES-D) (29) measured depressed mood.
Intervention
The intervention consisted of three sequential phases. In phase 1, called “joining,” clinicians met with individual families for two or three sessions to evaluate ongoing problems and define treatment goals. In phase 2, all veterans and families attended two three-hour educational workshops about TBI. In phase 3, veterans and their families attended problem-solving multifamily group meetings bimonthly for six months. The group sessions consisted of a 15-minute “socialization” period followed by a “go-round,” when each family described problems or life events experienced since the last meeting and reviewed homework assignments. On the basis of the go-round, a problem or goal was selected for a formal problem-solving exercise involving defining the problem or goal, generating all possible solutions, evaluating pros and cons of each solution, and formulating a solution and plan for implementation.
Group leaders kept a list of all problems identified by group members as candidates for problem solving and a written record of all group-generated solutions. The duration of all three phases of treatment was approximately nine months. Study clinicians attended a two-day training session about the model conducted by highly experienced multifamily group clinicians and supervisors (DN and DD).
All sessions occurred between April 2010 and March 2011 and were audiotaped and supervised by one of the trainers (DN). The trainer rated adherence to the different components of the treatment model for all sessions for the first two groups and 40% of sessions for the third group. Adherence was rated on a 3-point scale according to criteria described by McDonell and others (30), with 0 defined as nonadherence, 1, partial adherence, and 2, full adherence. The mean global adherence score across the three groups was 1.79±.43. Interrater reliability of independent ratings by trainers for six sessions of the Bronx group was adequate (kappa=.80).
Quantitative data analysis
Paired t tests were used to evaluate changes within participants over the nine months of treatment. The association between baseline levels on neurocognitive tests and the degree of change on outcome measures observed over the course of treatment (represented by posttreatment minus pretreatment scores) was evaluated by using Pearson product-moment correlation coefficients (Pearson’s r).
Results
Implementation
Buy-in by providers, veterans, and families.
A total of 59 potential candidates were identified and prescreened by their clinician, and 34 (58%) were approved and referred for study. Twenty-five (42%) of the 59 veterans identified as potential participants were excluded at the provider level because their TBI was judged to be too severe for participation (N=10), the provider was not able to identify at least one family member who was involved with the patient’s care (N=6), or there was a conflict with another treatment or study (N=4). Additionally, some clinic staff felt that adding consideration of the current intervention protocol to the already complex set of evaluation and treatment procedures for new veterans was too demanding, and this position was adopted as clinic policy. Therefore, providers also excluded five new patients.
Of the 34 veterans approved for the study, seven (21%) did not meet eligibility criteria. Two had a severe substance use disorder, and five had spouses who felt too burdened to participate or were not interested in participating (three were mothers of infants or very young children). Of the 27 eligible veterans, 13 either refused (N=6) or were lost to study (N=7). Reasons for refusal included a specific work- or school-related scheduling conflict or feeling too busy. Three of the five who had scheduling conflicts worked a 3–11 p.m. shift, which conflicted with the more popular early evening time for holding the group. Another worked a long daytime shift with a long commute.
A total of 14 veterans (52% of the 27 veterans who were eligible for the study and 41% of the 34 veterans approved for inclusion by providers) met study criteria, gave informed consent, and were enrolled in the study. [A table summarizing the reasons for exclusion or refusal is available online as a data supplement to this article.]
Veterans and family members were treated in three cohorts, one group of four and one group of five veterans and families at the VAMC in Durham and one group of four veterans and families at the VAMC in the Bronx. One couple in Durham’s first cohort withdrew after four group sessions because of the veteran’s medical condition, and two family members in the second cohort dropped out within the first few meetings, one because of a divorce and one because of relapse of a substance use disorder. In both cases the veteran remained in the study. Additionally, one couple in the Bronx withdrew during the joining sessions because of the veteran’s medical complications. Eleven (32%) veterans and nine (27%) family members and intact dyads who were referred to the study completed the treatment and were included in all quantitative analyses. Attendance at group sessions was good—across both sites, participants attended 46 of 59 total possible sessions (78%).
Tables 1 and 2 present the study sample’s demographic and military characteristics and neurocognitive data. Ten veterans reported that their TBI was associated with a loss of consciousness of <30 minutes, characterizing their TBI as mild (19). The duration of loss of consciousness of the 11th veteran was unknown. On neuropsychological testing, as a group they demonstrated intact learning and retention of new material, intact attention, and mild to moderate impairment of executive functioning, with considerable variability observed. Seventy-nine percent of veterans had comorbid PTSD, and 20% to 31% met criteria for other comorbid disorders common in this cohort (Table 2). Overall, the study sample was characteristic of OEF/OIF era veterans with mild TBI and comorbid mental health problems.
Veterans | Family | |||
---|---|---|---|---|
Characteristic | N | % | N | % |
Female | 3 | 21 | 12 | 86 |
Age | 36.1±9.3 | 34.4±12 | ||
Race-ethnicity | ||||
Caucasian | 8 | 57 | 8 | 57 |
African American | 5 | 36 | 4 | 29 |
Hispanic | 1 | 7 | 2 | 14 |
Marital status | ||||
Married or cohabiting | 10 | 71 | 10 | 71 |
Widowed or divorced | 2 | 14 | 2 | 14 |
Never married | 2 | 14 | 2 | 14 |
Highest grade attained | 15.1±2 | 13.8±1.7 | ||
Employment | ||||
Full-time | 7 | 50 | 7 | 50 |
Part-time | 2 | 14 | 3 | 21 |
Retired or on disability | 2 | 14 | 0 | — |
Unemployed | 1 | 7 | 4 | 29 |
Student | 2 | 14 | 0 | — |
Relation to family member | ||||
Adult child | 2 | 14 | ||
Spouse or partner | 10 | 71 | ||
Sibling | 2 | 14 |
Variable | N | % |
---|---|---|
Deployments (M±SD) | 1.8±1.1 | |
Military branch | ||
Army | 12 | 86 |
Navy | 1 | 7 |
Marines | 1 | 7 |
Military status | ||
Separated | 11 | 79 |
National Guard or Reserves | 3 | 21 |
Time since TBI diagnosis (M±SD years) | 1.8±1.7 | – |
Hospitalized for TBI-related injuries | 4 | 29 |
Service-connected disabilitya | 9 | 64 |
Comorbid disorder | ||
Posttraumatic stress disorder | 14 | 100 |
Major depressive disorder | 5 | 19 |
Other depressive disorder | 1 | 4 |
Alcohol use disorder | 8 | 31 |
Trail Making Test, part A (M±SD seconds)b | 34.6±10.6 | – |
Trail Making Test, part B (M±SD seconds)b | 87.9±35.6 | – |
CVLT-II free recall (M±SD words per trial)c | 9.1±3.7 | – |
CVLT-II delayed recall (M±SD words)c | 8.6±3.6 | – |
Group composition and emergent needs.
The problems identified by veterans and family members clustered in three areas: problems related to cognitive deficits, such as memory problems; relationship issues, such as poor communication; and veterans’ self-identity or community interface, such as difficulty accepting limitations or negotiating work and school accommodations. Because most participants were couples, most problems identified in the go-rounds related to disruptions in marital relationships. Although standard problem solving was a good fit for some relationship problems, such as coordinating child care, it proved less ideal for others, such as communicating negative feelings without escalation. Because relationship and communication problems were frequently brought in for group problem solving, we incorporated communication skills training methods from other evidence-based family psychoeducation models, such as behavioral family therapy (31) to facilitate problem solving by couples having relationship problems. Other modifications of the basic model adopted at the trial’s outset are discussed elsewhere (32).
Quantitative analyses
Paired t tests found significant decreases in veteran anger expression and increases in veteran social support and occupational activity (Table 3). Effect sizes ranged from .6 to 1.0. Significant decreases in caregiver burden and increases in family empowerment were observed as well.
Measure | Veterans | Family members | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Pretreatment | Posttreatment | Pretreatment | Posttreatment | |||||||||
M | SD | M | SD | t | Cohen’s d | M | SD | M | SD | t | Cohen’s d | |
Angera | 35.8 | 13.2 | 27.4 | 14.3 | 3.18** | .61 | 29.4 | 10.6 | 20.9 | 11.0 | 1.69 | .79 |
Social supportb | 12.9 | 6.3 | 26.3 | 21.4 | 2.53* | .85 | 16.3 | 7.2 | 18.5 | 7.9 | 1.18 | .30 |
Depressionc | 27.4 | 10.7 | 19.6 | 11.3 | 1.73 | .72 | 19.7 | 12.7 | 10.1 | 6.0 | 2.19 | .96 |
PTSDd | 58.0 | 12.3 | 49.3 | 8.7 | 1.99 | .82 | — | — | — | — | — | — |
Burdene | — | — | — | — | — | — | 30.4 | 19.4 | 12.4 | 15.7 | 2.53* | 1.03 |
Empowermentf | — | — | — | — | — | — | 38.4 | 8.0 | 49.1 | 4.4 | 4.73** | 1.66 |
Occupational activityg | 8.6 | 4.9 | 12.5 | 2.1 | 3.22* | 1.03 | — | — | — | — | — | — |
Interpersonal relationshipsg | 10.1 | 4.0 | 14.0 | 3.2 | 2.11 | 1.07 | — | — | — | — | — | — |
Veterans’ baseline performance on a measure of attention and speed of processing (Trail Making Test, part A) was negatively associated with improvement on outcome measures of anger expression (r=–.72, df=10, p=.020), depression (r=–.63, df=10, p=.051), and PTSD symptoms (r=–.85, df=10, p=.004) (Table 4). Baseline performance on a measure of executive functioning (Trail Making Test, part B) was negatively associated with improvement in occupational activity (r=–.76, df=10, p=.048). These results suggested that neurocognitive processes may affect treatment response.
Neurocognitive functioning | ||||||||
---|---|---|---|---|---|---|---|---|
CVLT-II | Trail Making Test | |||||||
Free recallb | Delayed recall | Part A | Part B | |||||
Measure | rc | p | rc | p | rc | p | rc | p |
AX Scaled | .225 | .531 | –.118 | .745 | –.716 | .020 | –.246 | .494 |
CES-De | .021 | .995 | –.284 | .427 | –.631 | .051 | .018 | .960 |
Duke Social Support Scale | –.060 | .870 | .321 | .365 | .420 | .227 | .779 | .008 |
PTSD Checklistf | .261 | .498 | .003 | .994 | –.847 | .004 | –.272 | .428 |
Occupational activityg | –.226 | .715 | –.458 | .438 | –.632 | .128 | –.759 | .048 |
Interpersonal relationshipsg | –.034 | .948 | –.050 | .925 | –.011 | .980 | –.264 | .528 |
Discussion
This study evaluated the feasibility, acceptability, and initial efficacy of a multifamily group intervention for veterans with TBI, a model adapted from an evidence-based treatment for persons with serious mental illness. The relatively low overall ratio of veterans enrolled in treatment to veterans referred for treatment (41%) was attributable to perceptions and barriers identified by all stakeholders.
Among some providers, discussing with the veteran the potential for family involvement was viewed as incompatible with or as interfering with the primary diagnostic task of the initial visit. Given the complexities involved in diagnosis of TBI, particularly mild TBI (33,34), and the minimal knowledge of staff about the benefits of family involvement, particularly multifamily groups, this view was not surprising and suggested a need to allocate more resources to increase readiness for implementation (35). For example, education demonstrating that family members may facilitate evaluation by providing history not obtainable from the veteran because of cognitive impairment may help to engender a more receptive implementation climate.
Exclusions of veterans by providers related to the severity of the TBI, coupled with pilot data demonstrating a negative association between cognitive impairment and treatment gains on several outcome measures, suggested that implementation might be more feasible and efficacious for veterans with less severe TBI. Although our results were consistent with other research demonstrating limited efficacy of psychosocial treatment for individuals with even relatively mild cognitive impairment (36), recent work suggested the potential for adapting evidence-based psychosocial treatments for individuals with cognitive impairment (36,37).
Refusal or early dropout at the family level related to caring for an infant or young child may relate both to practical considerations, such as finding appropriate child care, and to a high degree of perceived burden. Outreach to spouses and veterans through the VA Intranet to provide education on the potential benefits of such programs, including the use of veteran or family testimonials, might provide a nonintrusive way to tip the costs-benefits ratio sufficiently to interest families in a phone consultation with clinicians, who can begin to engage the family. For these families, conducting joining sessions via teletherapy might help minimize treatment burden while building an alliance. The use of the Internet and teletherapy to extend the boundaries of traditional psychosocial interventions for veterans’ families has grown in recent years (38,39). An example is the recent development of an interactive electronic program called “Families of Heroes,” designed to educate OEF/OIF families to identify behaviors indicating a need for VA services (40).
The major barrier at the veteran level was related to scheduling or time constraints. Many veterans work afternoon shifts or attend evening classes. Offering midday groups would make the treatment accessible to these families. As Dausch and others (41) pointed out, offering effective family services requires flexibility and planning by providers and administrative leadership to accommodate working veterans and relatives.
To address the needs of participants, specifically couples, that emerged during treatment, we introduced communication training methods from individual, skills-based family psychoeducation. Although these methods were introduced within the core problem-solving framework as solutions to specific marital problems of the participants, their use represents an adaptation of the original treatment model to meet the assessed needs at one local site that had enrolled only couples.
According to Damschroder and colleagues (35), the capacity to tailor or refine the “adaptable periphery” of an intervention to meet local needs without altering “core components” is critical to successful implementation. The fact that one couple and two spouses were early dropouts at the site where the groups represented a mix of family relationships lent support to the need to provide couples with more finely-honed tools to address relationship problems within the core problem-solving model. It also suggested that implementation may be smoother with more homogeneous groups.
Our quantitative measures showed medium to large effect sizes for outcomes of all veterans and most family members and supported our hypothesized improvement in anger management and use of social support among veterans and burden and empowerment among families. However, it should be emphasized that because of the lack of a control group, we cannot attribute improvements to the intervention itself rather than to nonspecific treatment effects, such as devoting extra attention to participants. Rather the findings provided support for implementing a randomized clinical trial to evaluate changes observed in multifamily group treatment for veterans with TBI against nonspecific treatment effects. Future work should incorporate additional measures of family and family member coping as well as marital satisfaction. Our preliminary implementation data will allow us to modify our interactions with providers, veterans, and families in approaching this next, critical step.
Conclusions
An open trial of an adaptation of multifamily group treatment for OEF/OIF veterans with TBI demonstrated significant improvement in anger management and use of social supports among veterans and in burden and empowerment among families, providing a rationale for mounting a randomized clinical trial. Challenges to implementation of this intervention observed at the systems, provider, and participant levels suggested the need for building in education for both providers and family members about the benefits of family involvement. Outreach via Web-based products may help promote treatment participation. Adaptation of evidence-based treatments for a new population may require tailoring and refining the model to meet local or specialized needs of the new cohort that emerge during treatment.
1 : Mild traumatic brain injury in US soldiers returning from Iraq. New England Journal of Medicine 358:453–463, 2008Crossref, Medline, Google Scholar
2 : Association of cardiovascular risk factors with mental health diagnoses in Iraq and Afghanistan war veterans using VA health care. JAMA 302:489–492, 2009Crossref, Medline, Google Scholar
3 : Posttraumatic stress symptoms among National Guard soldiers deployed to Iraq: associations with parenting behaviors and couple adjustment. Journal of Consulting and Clinical Psychology 78:599–610, 2010Crossref, Medline, Google Scholar
4 : Developing community capacity to treat postdeployment mental health problems: a public health initiative. Psychological Trauma: Theory, Research, Practice, and Policy 3:283–291, 2011Crossref, Google Scholar
5 : Reintegration problems and treatment interests among Iraq and Afghanistan combat veterans receiving VA medical care. Psychiatric Services 61:589–597, 2010Link, Google Scholar
6 : Welcoming them home: supporting service members and their families in navigating the tasks of reintegration. Professional Psychology, Research and Practice 39:451–458, 2008Crossref, Google Scholar
7 : Serving families who have served: providing family therapy and support in interdisciplinary polytrauma rehabilitation. Journal of Clinical Psychology 64:993–1003, 2008Crossref, Medline, Google Scholar
8 : Combat-injured service members and their families: the relationship of child distress and spouse-perceived family distress and disruption. Journal of Traumatic Stress 23:112–115, 2010Medline, Google Scholar
9 : Program development and defining characteristics of returning military in a VA Polytrauma Network Site. Journal of Rehabilitation Research and Development 44:1027–1034, 2007Crossref, Medline, Google Scholar
10 : Cognitive-behavioral conjoint therapy for posttraumatic stress disorder: application to Operation Enduring and Iraqi Freedom veterans. Journal of Clinical Psychology 64:958–971, 2008Crossref, Medline, Google Scholar
11 : The development of a couple-based treatment for PTSD in returning veterans. Professional Psychology, Research and Practice 42:63–69, 2011Crossref, Google Scholar
12 : Cognitive-behavioral conjoint therapy for PTSD: Initial findings for Operations Enduring and Iraqi Freedom male combat veterans and their partners. American Journal of Family Therapy, in pressGoogle Scholar
13 : Adapting the multifamily group model to the Veterans Affairs system: the REACH program. Professional Psychology, Research and Practice 40:593–600, 2009Crossref, Google Scholar
14 : Multifamily Groups in the Treatment of Severe Psychiatric Disorders. New York, Guilford, 2002Google Scholar
15 : Adapting multiple-family group treatment for brain and spinal cord injury intervention development and preliminary outcomes. American Journal of Physical Medicine and Rehabilitation 86:482–492, 2007Crossref, Medline, Google Scholar
16 : Integrative outpatient treatment for returning service members. Journal of Clinical Psychology 64:928–939, 2008Crossref, Medline, Google Scholar
17 : Combat duty in Iraq and Afghanistan, mental health problems, and barriers to care. New England Journal of Medicine 351:13–22, 2004Crossref, Medline, Google Scholar
18 : Multifamily group treatment for veterans with traumatic brain injury: what is the value to participants? Psychiatric Services (Epub ahead of print, March 1, 2013)Google Scholar
19 : Reliability, sensitivity, and specificity of the VA traumatic brain injury screening tool. Journal of Head Trauma Rehabilitation 26:439–453, 2011Crossref, Medline, Google Scholar
20 : Administration and interpretation of the Trail Making Test. Nature Protocols 1:2277–2281, 2006Crossref, Medline, Google Scholar
21 : The California Verbal Learning Test, 2nd ed: test-retest reliability, practice effects, and reliable change indices for the standard and alternate forms. Archives of Clinical Neuropsychology 5:413–420, 2006Google Scholar
22 : Validation and utility of a self-report version of PRIME-MD: the PHQ Primary Care Study. JAMA 282:1737–1744, 1999Crossref, Medline, Google Scholar
23 : Measuring psychosocial recovery after traumatic brain injury: psychometric properties of a new scale. Journal of Head Trauma Rehabilitation 14:543–557, 1999Crossref, Medline, Google Scholar
24 : Psychometric properties of nine scoring rules for the clinician-administered Posttraumatic Stress Disorder Scale. Psychological Assessment 11:124–133, 1999Crossref, Google Scholar
25 : Application of a multidimensional caregiver burden inventory. Gerontologist 29:798–803, 1989Crossref, Medline, Google Scholar
26 : Measuring empowerment in families whose children have emotional disabilities: a brief questionnaire. Rehabilitation Psychology 37:305–321, 1992Crossref, Google Scholar
27 : The Spielberger Anger Expression Scale: some psychometric data. British Journal of Clinical Psychology 27:279–281, 1988Crossref, Medline, Google Scholar
28 : Abbreviating the Duke Social Support Index for use in chronically ill elderly individuals. Psychosomatics 34:61–69, 1993Crossref, Medline, Google Scholar
29 : The CES-D Scale. Applied Psychological Measurement 1:385–401, 1977Crossref, Google Scholar
30 : Clinician integrity in multiple family groups: psychometric properties and relationship with schizophrenia client and caregiver outcomes. Cognitive Therapy and Research 31:785–803, 2007Crossref, Google Scholar
31 : Behavioral Family Therapy for Psychiatric Disorders. Oakland, Calif, New Harbinger Publications, 1999Google Scholar
32 : Multifamily group treatment for veterans with traumatic brain injury. Professional Psychology: Research and Practice 42:70–78, 2011Crossref, Google Scholar
33 Rehabilitation of Persons With Traumatic Brain Injury. Bethesda, Md, National Institutes of Health, 1998. Available at consensus.nih.gov/1998/1998traumaticbraininjury109program.pdfGoogle Scholar
34 : Verbal learning differences in chronic mild traumatic brain injury. Journal of the International Neuropsychological Society 16:506–516, 2010Crossref, Medline, Google Scholar
35 : Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science 4:1–15, 2009Crossref, Medline, Google Scholar
36 : RESEARCH: cognitive rehabilitation for bipolar disorder: an open trial for employed patients with residual depressive symptoms. CNS Neuroscience and Therapeutics 16:298–307, 2010Crossref, Medline, Google Scholar
37 Schmitter-Edgecombe M, Sanders C, Low C, et al: An integrated cognitive rehabilitation multi-family group intervention for individuals with mild cognitive impairment and their care partners: preliminary data. Presented at the Alzheimer’s Association International Conference, Vancouver, British Columbia, July 14–19, 2012Google Scholar
38 : Web-based psychoeducational intervention for persons with schizophrenia and their supporters: one-year outcomes. Psychiatric Services 61:1099–1105, 2010Link, Google Scholar
39 : Family-focused therapy via videoconferencing. Journal of Telemedicine and Telecare 15:211–214, 2009Crossref, Medline, Google Scholar
40 : Using an avatar-based simulation to train families to motivate veterans with post-deployment stress to seek help at the VA. Games for Health 1:9–14, 2011Medline, Google Scholar
41 : An intervention framework for family involvement in the care of persons with psychiatric illness: further guidance from Family Forum II. American Journal of Rehabilitation 15:5–25, 2012Crossref, Google Scholar