Interventions to Increase Retention in Mental Health Services: A Systematic Review
Abstract
Objective:
Several systematic reviews have evaluated interventions that aim to increase mental health service initiation and engagement as well as adherence to pharmacological treatment. No reviews have focused on evaluating these interventions’ effects on retention in mental health services, however, which was the goal of this systematic review.
Methods:
PubMed, PsycINFO, and Social Services Abstracts were searched for studies that met the inclusion criteria. All studies published prior to March 29, 2015, that compared two or more groups on any measure of retention in mental health services were included. Methodological quality was assessed for each included study. An effect size was calculated for each outcome, although a meta-analysis was not conducted because of heterogeneity across interventions. To facilitate narrative analysis, interventions were categorized by targets—the types of predictors of or barriers to mental health service use that the intervention aimed to address.
Results:
Eleven studies met inclusion criteria. The interventions produced medium to large effects on retention outcomes. Many interventions addressed more than one target. Interventions that targeted mental health knowledge, mental health attitudes, and barriers to treatment all enhanced retention in mental health services compared with control groups. Most interventions with those targets had a large effect on retention and relatively good methodological ratings.
Conclusions:
The most effective retention interventions were comprehensive, addressing mental health knowledge, mental health attitudes, and barriers to treatment. The authors recommend that researchers apply relevant theories to refine these interventions and evaluate the interventions by using rigorous methodology and a range of retention outcomes, mediators, and moderators.
Less than half (41.1%) of Americans with mental disorders initiate any mental health services, and of those, only a third (32.7%) engage in enough treatment visits to be deemed minimally adequate treatment (1). Although psychotropic medication use has increased in recent decades, outpatient psychotherapy use has declined precipitously (2), in what has been described as a “sea change in the provision of mental health services” (3). Persons who engage in psychotherapy receive fewer visits than received in past decades (2). Although medication use is on the rise, a majority of psychotropic prescriptions (59%) are written by general medical practitioners (4), and few individuals receive minimally adequate treatment for psychiatric disorders in general medical settings (12.7%) (1). Even in specialty mental health settings, fewer than half of adults with mental disorders (48.3%) receive the minimum number of visits considered minimally adequate treatment (1).
In order to understand low use of mental health services, many empirical studies have been conducted to identify predictors of mental health service use. Most of this research has focused on predictors of any service use. Such predictors include demographic factors, treatment need, and mental health knowledge, attitudes, and beliefs. Moreover, people are more likely to use mental health services if they do not encounter logistical barriers (limited finances, time commitment, and transportation issues) or normative-influence barriers (stigma concerns, skepticism of treatment, and lack of recognition of problems) that prevent treatment (5,6).
Building upon this empirical literature, some researchers have developed engagement interventions that aim to increase mental health service use. Often occurring before the traditional mental health treatment (psychotropic medication or psychotherapy) begins, engagement interventions aim to increase treatment initiation and retention by addressing various predictors and barriers related to mental health service use. Researchers have extensively studied predictors of mental health service use and published systematic reviews of engagement interventions designed to increase use of mental health services broadly, adherence to psychotropic medication, or use of psychotherapy (7–10). Taken together, these reviews suggest various strategies that may increase engagement, such as education, collaboration and ongoing follow-up with a provider, and appointment reminders.
No reviews, however, have focused solely on the effects of these interventions on retention in treatment, which is an important gap in the literature. In this systematic review, our primary aim was to systematically review interventions designed to increase individuals’ retention in mental health services, with the long-term goal to increase individuals’ retention in a sufficient number of visits to achieve beneficial outcomes. Consistent with the definition of minimally adequate treatment commonly used (four or more medication management visits or eight or more psychotherapy visits), we included any type of mental health visit, not just visits for psychotherapy (1). Moreover, this review focused on individual- and social-level factors that can affect treatment retention rather than structural factors, such as use of a collaborative care model. Structural interventions have been shown to increase service use (11,12) and have been reviewed elsewhere (13).
Methods
Prior to initiating this systematic review, we met to finalize the review procedures, including choosing the databases to search, study inclusion and exclusion criteria, and the data to extract from studies. The methods of the review were drawn from the Cochrane Handbook for Systematic Reviews of Interventions (14), the official document of the Cochrane Collaboration about conducting a systematic review of health care interventions. All procedures (literature search, study selection, and data extraction) were conducted independently by two authors (JG and RB). Discrepancies were identified and discussed among all three authors until consensus was reached.
Search Methods
PubMed, PsycINFO, and Social Services Abstracts were searched by two authors (JG and RB) for studies that met the inclusion criteria. All databases were searched with the following search terms: mental health services, intervention, and (“service use” or “seeking help” or “engagement” or “initiation” or “retention”). The reference lists of all included studies were examined for additional relevant studies. All databases were searched through March 29, 2015. Included studies were published between 1998 and 2014.
Selection Criteria
Studies were included in this review if they evaluated an intervention that aimed to increase retention in any type of outpatient mental health service; studies were excluded if they did not intend to increase retention in outpatient services—for example, studies designed to reduce inpatient service use. Studies of adults, persons age 18 and older, persons who lived in the community, and persons with any type of mental health problem (diagnosed or undiagnosed) were included. All studies that compared two or more groups were included in this systematic review. As such, this review included randomized controlled trials (RCTs), quasi-experimental designs, and pre-post intervention studies. All included studies featured at least one contact with an interventionist (in person or over the telephone) or written materials such as brochures and mailings. Studies that changed the structure of care, provided only appointment scheduling or reminders, or involved participants with a primary diagnosis of substance use disorder were excluded.
Selection of Studies
All abstracts were reviewed in relation to the inclusion and exclusion criteria. Unless the abstract clearly described one or more exclusion criteria, the full article was examined to determine whether it met the inclusion and exclusion criteria. Two authors (JG and RB) independently searched the databases and compiled a list of studies to be considered for inclusion by all authors. Discrepancies between the two lists were discussed among all authors until a consensus was reached about each study. Information about excluded studies is available by request.
Data Extraction
A data extraction spreadsheet was designed and employed to systematically extract data from all included studies. Data extracted included sample size, study conditions, delivery method, duration of intervention, mental health problem addressed, target of the intervention (for example, specific predictors or barriers related to service use), outcomes measured, and results. Two authors (JG and RB) independently extracted data from the included studies.
The methodological quality of each included study was assessed and reported by using a rating table and scoring system for assessing the methodological quality of RCTs of psychological and behavioral interventions. The scoring system was developed by Newell and colleagues (15) and was derived from the recommendations of the Cochrane Collaboration (14). Each study was rated on ten indicators of internal validity: concealment of allocation, random selection of participants, blinding of participants to study group, blinding of care providers, equivalent treatments except for the active intervention, monitoring of care-provider adherence to protocol, loss to follow-up information, percentage of participants not included in analysis, use of intent-to-treat analyses, and presence of objective measures or subjective measures administered by blinded raters. For each study, each of the ten indicators was assigned 0 to 3 points, with 0 indicating not at all fulfilled, 1, mostly not fulfilled, 2, mostly fulfilled, and 3, entirely fulfilled. Therefore, a study could achieve up to 30 points, with 0–10 indicating poor quality, 11–20 indicating fair quality, and 21–30 indicating good quality. Study authors were contacted for clarification if the article lacked sufficient information to make a judgment about quality.
To ensure quality of data extraction, the data extraction spreadsheets of the coding authors (JG and RB) were examined for discrepancies. In the event of a discrepancy between the two coding authors, we discussed the discrepancy until consensus was reached. To ensure impartial ratings of the quality indicators, interrater reliability, a measure of the degree to which different raters agree, was assessed. In addition, intraclass correlation coefficients (ICCs), a statistical measure of interrater reliability similar to Pearson correlation coefficients, were calculated for the ten quality indicators by using a two-way random effects model (consistency type). High ICCs indicate little variation between raters.
A meta-analysis was not undertaken because of the diversity of studies (for example, diverse samples, interventions, and outcomes), as advised by the Cochrane Handbook (14). Instead, the interventions were categorized by their targets—the specific predictors of or barriers to service use that they aimed to address.
Effect sizes (ESs) were calculated on the basis of formulas described by Lipsey and Wilson (16), which can be accessed through an online ES calculator that accompanied the book (www.campbellcollaboration.org/escalc/html/EffectSizeCalculator-SMD-main.php; last accessed on November 6, 2014). An ES of .2 is considered small; .5, medium; and ≥.8, large (17). The specific formulas used to determine ES differed across studies because of differences in types of outcomes and statistics reported.
Results
Between the two authors, the search of PsycINFO yielded 326 to 354 abstracts, the search of PubMed yielded 2,641 to 2,668 abstracts, and the search of Social Services Abstracts yielded 800 to 842 abstracts. Three studies were reviewed on the recommendation of a research colleague. Eight studies from the reference lists of already included studies were also reviewed. From these sources, the authors examined a total of 36 studies. Of these studies, 11 met all inclusion and exclusion criteria.
Description of Interventions
Table 1 contains a description of the included studies. The interventions addressed a variety of mental health problems, including depression (N=4), bipolar disorder (N=3), schizophrenia (N=1), and posttraumatic stress disorder (PTSD) (N=2). Four studies determined eligibility by patient-reported mental distress but did not require a specific diagnosis. Eight of the interventions were delivered in person, one by telephone, one in person with telephone follow-up, and one with written materials and telephone follow-up. The interventions ranged in duration from one 15-minute phone call to 21 psychoeducation sessions over a six-month period, with each session lasting 90 minutes. Approximately half of the interventions took place at a mental health clinic (N=5, 45%), with 36% (N=4) taking place in various types of health clinics. Most studies (N=9) were RCTs comparing an intervention group and a control, or usual care, group. Two studies used nonrandom comparison groups.
| Intervention | Mental health problem | N | ||||||
|---|---|---|---|---|---|---|---|---|
| Study | Name | Target | Delivery | Setting | Design | Intervention | Control | |
| Alegría et al., 2008 (27) | Right Question Project–Mental Health | Patient empowerment and patient activation | Depressive disorder, bipolar disorder, anxiety disorder, psychotic disorder, and other disorder | 3 sessions, 30 minutes each, over approximately 2 months | Community mental health clinic | Nonrandom comparison groups | 141 | 90 |
| Alegría et al., 2014 (29) | DECIDE | Patient activation | Any mental health problem | 3 sessions, 30–45 minutes each, over a 3-month period | Community mental health clinic | Randomized controlled trial | 372 | 352 |
| Alvidrez et al., 2005 (18) | Psychoeducation | Mental health knowledge and mental health attitudes | Any mental health problem | 1 session (15 minutes) | Geriatric health clinic | Nonrandom comparison groups | 32 | 37 |
| Dixon et al., 2009 (19) | Brief critical time intervention | Barriers to treatment, needs assessment, emotional and practical support from clinician, and developing treatment goals | Schizophrenia spectrum disorder, major depression, bipolar disorder, or psychotic disorder not otherwise specified | 1 initial session followed by up to 3 months of contact at the patient’s home and in the community | Veterans Affairs Capitol Healthcare Network | Randomized controlled trial | 64 | 71 |
| Grote et al., 2007 (23) | Pretreatment engagement interview and brief interpersonal psychotherapy (IPT) | Mental health knowledge, mental health attitudes, and barriers to treatment | Depression | 1 engagement session (45–60 minutes) and 8 IPT sessions | Obstetrics clinic | Randomized controlled trial | 25 | 28 |
| McFall et al., 2000 (25) | Outreach | Mental health knowledge, mental health attitudes, and barriers to treatment | Posttraumatic stress disorder | 1 mailing, followed by a 15-minute telephone contact | na | Randomized controlled trial | 302 | 292 |
| McMurran et al., 2013 (20) | Motivational interviewing and Personal Concerns Inventory procedure | Goal attainment via mental health treatment and patient motivation | Any mental health problem | 1 session (1.5 hours) | Personality disorder clinic | Randomized controlled trial | 17 | 24 |
| Scott et al., 2009 (21) | Psychoeducation | Mental health knowledge | Bipolar disorder | 21 sessions, 1.5 hours each, over a 6-month period | Bipolar disorder clinic | Randomized controlled trial | 60 | 60 |
| Sirey et al., 2005 (26) | Treatment Initiation Program | Mental health knowledge, mental health attitudes, barriers to treatment, and developing treatment goals | Depression | 3 sessions, 30 minutes each, during the first 6 weeks and 2 follow-up phone calls at 8 and 10 weeks | Geriatric psychiatry clinic | Randomized controlled trial | 26 | 26 |
| Spooren et al., 1998 (24) | Various educational strategies, depending on the participant | Mental health knowledge, patient motivation, and family involvement | Any mental health problem | 30 minutes to 1.5 hours | Hospital | Randomized controlled trial | 327 | 322 |
| Stecker et al., 2014 (22) | Brief, cognitive-behavioral intervention | Mental health attitudes | Posttraumatic stress disorder | 1 session, (45–60 minutes) via phone | na | Randomized controlled trial | 123 | 151 |
TABLE 1. Characteristics of studies of interventions to increase retention in mental health services
Table 1 also includes each intervention’s targets, which are described in more detail in Table 2. The studies were categorized into 11 intervention targets, which were derived from the studies’ description of the interventions’ methods and goals as well as research literature about predictors of mental health service use. Mental health knowledge, which was addressed by six (55%) interventions, was the most common target. Mental health attitudes and barriers to treatment were also common targets (N=5, 45%, and N=4, 36%, respectively). More than two-thirds (N=8, 73%) of the interventions addressed multiple targets.
| Target | Factors |
|---|---|
| Mental health knowledge | |
| Alvidrez et al., 2005 (18) | Use of medical model of mental health problems to reduce stigma, the specific conditions under which involuntary hospitalization could occur, how therapists could be different from their clients and still be helpful, the receptivity of clinic therapists to discussing issues of religion or spirituality and incorporating them into the treatment, and the importance of the patient’s input in determining therapy goals and session topics |
| Grote et al., 2007 (23) | Psychoeducation about mental health problem (for example, depression) and description of various treatment options |
| McFall et al., 2000 (25) | Information describing available treatment services and assessing awareness of mental health resources |
| Scott et al., 2009 (21) | Each psychoeducation session addresses a different aspect of bipolar disorder and treatment |
| Sirey et al., 2005 (26) | Misconceptions about depression and treatment and cognitive distortions associated with depression |
| Spooren et al., 1998 (24) | General explanation of further outpatient treatment and how patient could benefit from it |
| Mental health attitudes | |
| Alvidrez et al., 2005 (18) | Importance of talking with the therapist about conflicts, misunderstandings, or dissatisfaction with the treatment process |
| Grote et al., 2007 (23) | Understanding patient’s experience of her mental health problem; exploring positive and negative aspects of previous coping mechanisms, especially previous mental health treatments; personal and familial attitudes regarding the stigma of depression and regarding receiving mental health care; and negative past experiences with professionals |
| McFall et al., 2000 (25) | Attitudes toward mental health treatment and the Veterans Health Administration health care system |
| Sirey et al., 2005 (26) | Perceived stigma and perceived need for care |
| Stecker et al., 2014 (22) | Participants identify individual beliefs about mental health treatment during the intervention session; the intervention session addresses a maximum of three beliefs with each participant |
| Barriers to treatment | |
| Dixon et al., 2009 (19) | Assessment of barriers to outpatient mental health care |
| Grote et al., 2007 (23) | Clinician addresses any barrier client raises, such as psychological or cultural barriers, and suggests some barriers, for example practical barriers, such as cost, transportation, child care, and scheduling |
| McFall et al., 2000 (25) | Physical barriers to accessing care |
| Sirey et al., 2005 (26) | Logistical barriers |
| Patient empowerment | |
| Alegría et al., 2008 (27) | Shared patient-provider decision making (empowerment) and preparation for appointments |
| Patient activation | |
| Alegría et al., 2008 (27) | Formulated questions to get information (activation) about patients’ mental health problems, treatments, and relationships with providers |
| Alegría et al., 2014 (29) | Participants identify decisions regarding their mental health care, generate and refine questions for their health care professionals regarding these decisions, and promote interactions with health care professionals that allow for patient needs to be shared and addressed |
| Goal attainment via mental health treatment | |
| McMurran et al., 2013 (20) | Participants identify goals in 11 life areas (for example, relationships, work or education, home, and health) and prioritize 5 goals. Therapists rate goals on scales from 0 to 10 assessing 5 aspects of goal attainment (likelihood of attainment, control over attainment, knowing how to attain it, happiness upon attainment, and commitment to attaining it) |
| Needs assessment | |
| Dixon et al., 2009 (19) | Assess and address individual needs in 9 areas (systems coordination, engagement in psychiatric services, continuation of substance abuse treatment, medication adherence, family involvement and social support network, life skills training and support, integration of medical care, establishment of community linkages and practical needs assistance) |
| Patient motivation | |
| McMurran et al., 2013 (20) | In order to increase motivation to engage in therapy, participants identify obstacles to goal attainment and consider the possibility that therapy could help them overcome these obstacles |
| Spooren et al., 1998 (24) | Counseling directed at increasing motivation and incorporating the patient’s perspective |
| Emotional and practical support from clinician | |
| Dixon et al., 2009 (19) | Clinician maintains a high level of patient contact, conducting home visits, accompanying the patient to initial appointments, and providing emotional and practical support for the patient and family |
| Family involvement | |
| Spooren et al., 1998 (24) | Informing family about the present illness or problems, the need for treatment, and the practical organization of the aftercare |
| Developing treatment goals | |
| Dixon et al., 2009 (19) | Clinician develops individualized treatment goals |
| Sirey et al., 2005 (26) | Identifying treatment goals |
TABLE 2. Factors addressed by interventions to increase retention in mental health services, by target of the intervention
Effects of the Interventions
The ES for every retention outcome was calculated in order to compare the effects of the interventions (16). For continuous outcomes in which means and standard deviations were provided (18–22), the formula below was used, where M1 is the mean of group 1, M2 is the mean of group 2, and Sp is the pooled standard deviation.
For studies in which the outcomes represented proportions of intervention and control groups (23,24), the formula below was used, where p is the proportion of participants.
The study by McMurran and colleagues (20) reported ES for median percentage of treatment sessions attended. McFall and others (25) reported the percentage of participants who attended at least one follow-up treatment session and reported the results of a chi square analysis that was used to compare the proportions of patients who remained in treatment. Therefore, we calculated ES by using the formula below, where χ2 is the chi square statistic and N is the sample size. For Sirey and colleagues (26), the ES was also calculated with a chi square formula; however, the chi square value was imputed from the reported p value.
For Alegría and colleagues (27), the reported odds ratio was converted to an ES by using the formula below (28).
For Sirey and colleagues (26), the formula below was used to calculate the ES for mean number of pharmacotherapy visits and mean number of psychotherapy visits, where t is the t statistic, n1 is the N of group 1, and n2 is the N of group 2.
The ES for Alegría and colleagues (29) was not calculated because proportions of each group retained were not reported. However, the authors stated that there was no intervention effect on retention.
Table 3 describes each study’s follow-up period, outcomes measured, results, and ESs. Time to follow-up ranged from nine weeks to five years, with an average follow-up period of 8.6 months, representing a wide range of follow-up period lengths.
| Study | Follow-up | Retention measure | Outcomes | Effect size |
|---|---|---|---|---|
| Alegría et al., 2008 (27) | 6 months | Retention in treatment (≥4 psychotherapy or psychopharmacology visits) | Intervention participants were over twice as likely as control participants to be retained in treatment (adjusted odds ratio=2.78, 95% confidence interval=1.33–5.79); proportions retained in treatment were not reported | .56 |
| Alegría et al., 2014 (29) | 6 months | Retention in treatment (≥4 psychotherapy or psychopharmacology visits) | Intervention participants were no more likely than control participants to be retained in treatment; proportions retained in treatment were not reported | No effecta |
| Alvidrez et al., 2005 (18) | 3 months | Mean N of psychotherapy sessions attended | 3.5 sessions for the intervention group vs. 1.9 sessions for the control group | .76 |
| Dixon et al., 2009 (19) | 30 days | Mean N of mental health or substance abuse visits in the first 30 days after discharge | 6.67 visits for the intervention group vs. 1.97 visits for the control group | 1.50 |
| 180 days | Mean N of mental health or substance abuse visits in the first 180 days after discharge | 20.80 visits for the intervention group vs. 10.08 visits for the control group | .80 | |
| 180 days | Mean N of 60-day periods with ≥2 mental health or substance abuse visits in the first 180 days after discharge | 2.23 60-day periods for the intervention group vs. 1.62 60-day periods for the control group | .60 | |
| Grote et al., 2007 (23) | na | Proportion of participants completing full course of treatment | 17 of 25 (68%) completers for the intervention group vs. 2 of 28 (7%) completers for the control group | 1.90 |
| McFall et al., 2000 (25) | 6 months | Percentage of participants attending ≥1 follow-up treatment sessionsb | 19.4% for the intervention group vs. 5.8% for the control group | .41 |
| McMurran et al., 2013 (20) | 3 months | Mean N of treatment sessions | 8.18 sessions for the intervention group vs. 6.54 sessions for the control group | .38 |
| 3 months | Of treatment sessions offered, median percentage attended | 83.3% for the intervention group vs. 66.7% for the control group | .44 | |
| Scott et al., 2009 (21) | 6 months | Mean N of outpatient visits | 3.13 visits for the intervention group vs. 2.48 visits for the control group | .23 |
| 6 months | Mean N of therapy sessions | 17.62 sessions for the intervention group vs. 17.25 sessions for the control group | .11 | |
| 5 years | Mean N of outpatient visits | 17.27 visits for the intervention group vs. 8.59 visits for the control group | .64 | |
| 5 years | Mean N of therapy sessions | 17.92 sessions for the intervention group vs. 12.63 sessions for the control group | .86 | |
| Sirey et al., 2005 (26) | 3 months | Proportion remaining in treatment | More intervention participants than control participants remained in treatment (p=.05, two-tailed Fisher’s exact test); proportions remaining in treatment were not reported | .57 |
| 6 months | Proportion remaining in treatment | More intervention participants than control participants remained in treatment (p=.04, two-tailed Fisher’s exact test); proportions remaining in treatment were not reported | .59 | |
| 6 months | Mean N of pharmacotherapy visits | No differences between groups (mean visits were not reported) | .15 | |
| 6 months | Mean N of psychotherapy sessions | No differences between groups (mean sessions were not reported) | .02 | |
| Spooren et al., 1998 (24) | 4 months | Proportion of participants in hospital A who were deemed treatment compliant (continuation of outpatient treatment and keeping appointments on a regular basis) | 52 of 107 (48.6%) intervention participants vs. 23 of 105 (21.9%) control participants | .67 |
| Proportion of participants in hospital B who were deemed treatment compliant (continuation of outpatient treatment and keeping appointments on a regular basis) | 39 of 120 (32.5%) intervention participants vs. 38 of 112 (33.9%) control participants | –.04 | ||
| Proportion of participants in hospital C who were deemed treatment compliant (continuation of outpatient treatment and keeping appointments on a regular basis) | 31 of 99 (30.6%) intervention participants vs. 11 of 105 (10.5%) control participants | .75 | ||
| Stecker et al., 2014 (22) | 1 month | Mean N of treatment sessions | .38 sessions for the intervention group vs. .20 sessions for the control group | .24 |
| 3 months | Mean N of treatment sessions | 1.08 sessions for the intervention group vs. .67 sessions for the control group | .23 | |
| 6 months | Mean N of treatment sessions | 4.06 sessions for the intervention group vs. 2.47 sessions for the control group | .19 |
TABLE 3. Outcomes of interventions to increase retention in mental health services, by study
Mental health knowledge.
Of the six studies that provided psychoeducation to address participants’ mental health knowledge, every intervention had at least one measure of retention, and three had multiple measures. For the three interventions with one retention outcome, all interventions had a medium (ES=.41 [25] and .76 [18]) or large (ES=1.90 [23]) effect on the outcome. In a study by Scott and colleagues (21), the intervention had small effects (ES=.11 and .23) at the six-month follow-up but medium and large effects (ES=.64 and .86) at the five-year follow-up. In the study by Sirey and others (26), the intervention had a medium effect on the proportion of individuals remaining in treatment at three and six months (ES=.57 and .59, respectively) but only small effects on attending psychotherapy and pharmacotherapy visits at six months (ES=.02 and .15, respectively). Last, Spooren and others (24) measured continuation of outpatient treatment at three sites, with varying effects (ES=–.04, .67, and .75) by site.
Mental health attitudes.
Four of the five studies that addressed participants’ attitudes about mental health or mental health treatment also targeted mental health knowledge and produced a range of ESs (.02–1.90) (18,23,25,26). Stecker and colleagues (22) also addressed participants’ attitudes, finding small effects at one, three, and six months (ES=.24, .23, and .19, respectively).
Barriers to treatment.
Three studies addressed practical barriers to mental health treatment and reported a range of effects (ES=.41 [25]; 1.90 [23]; and .02, .15, .57, and .59 [26]). The intervention described in an article by Dixon and colleagues (19) also targeted barriers to treatment and had large effects on increasing mental health visits or visits for substance abuse in both 30-day (ES=1.50) and 180-day (ES=.80) follow-up periods, as well a medium effect on the number of 60-day periods with two or more visits (ES=.60).
Patient empowerment.
One study aimed to increase patient empowerment (27). Alegría and others (27) reported a medium effect (ES=.56) on a dichotomous retention outcome (four or more pyschotherapy or psychopharmacology visits) during a six-month follow-up period.
Patient activation.
Two studies aimed to increase patient activation (27,29). As previously stated, Alegría and others (27) reported a medium effect (ES=.56) on receipt of four or more psychotherapy or psychopharmacology visits during a six-month follow-up period. Using the same outcome, a later study by Alegria and colleagues (29) did not demonstrate a comparable effect. In fact, intervention participants were no more likely than control participants to be retained in treatment.
Other targets.
One intervention helped participants set, prioritize, and work toward attaining goals in 11 life areas (20). This study resulted in medium effects on mean number of mental health visits and median percentage of treatment sessions attended (ES=.38 and .44, respectively). Other interventions targeted conducting a needs assessment (19), increasing patient motivation (20,24), providing emotional and practical support from the clinician (19), encouraging family involvement (24), and developing treatment goals (19,26).
Summary of effects.
Only one intervention had no effect on the retention outcome (29). This could be related to the fact that this intervention was not as comprehensive as the others, given that it addressed only one target, patient activation. Overall, the interventions produced medium or large effects on retention outcomes. Five studies reported a large effect on one or more outcomes, with ES values ranging from .75 to 1.90. These studies (18,19,21,23,24) often addressed multiple targets (N=4, 80%), with mental health knowledge being the most common target (N=4, 80%). Four additional studies reported medium effects on one or more measured outcomes (ES=.41–.59) (20,25–27). All four of these interventions addressed multiple targets, with mental health knowledge (N=2), mental health attitudes (N=2), and barriers to treatment (N=2) being the most common targets.
Methodological Quality
Table 4 provides scores for each methodological quality indicator and total scores for each study. Six study authors were contacted for clarification of articles that lacked sufficient information to make a judgment on one or more criteria. Of those contacted, four responded. The studies were also examined with an additional indicator, random assignment of participants, the hallmark of RCTs. Nine studies (82%) randomly assigned participants.
| Study | Study design | Study score | Quality indicator rating | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| QR | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |||
| Alegría et al., 2008 (27) | Nonrandom comparison groups | 13 | Fair | 0 | 0 | 0 | 0 | 2 | 3 | 2 | 3 | 0 | 3 |
| Alegría et al., 2014 (29) | Randomized controlled trial (RCT) | 20 | Fair | 3 | 0 | 0 | 0 | 3 | 3 | 2 | 3 | 3 | 3 |
| Alvidrez et al., 2005 (18) | Nonrandom comparison groups | 16 | Fair | 3 | 0 | 0 | 0 | 2 | 2 | 3 | 3 | 0 | 3 |
| Dixon et al., 2009 (19) | RCT | 17 | Fair | 0 | 3 | 0 | 0 | 2 | 3 | 1 | 2 | 3 | 3 |
| Grote et al., 2007 (23) | RCT | 15 | Fair | 0 | 0 | 0 | 0 | 2 | 2 | 2 | 3 | 3 | 3 |
| McFall et al., 2000 (25) | RCT | 24 | Good | 3 | 3 | 3 | 0 | 3 | 2 | 3 | 1 | 3 | 3 |
| McMurran et al., 2013 (20) | RCT | 18 | Fair | 3 | 3 | 0 | 0 | 3 | 2 | 3 | 1 | 0 | 3 |
| Scott et al., 2009 (21) | RCT | 20 | Fair | 3 | 0 | 3 | 3 | 3 | 2 | 3 | 0 | 0 | 3 |
| Sirey et al., 2005 (26) | RCT | 18 | Fair | 0 | 3 | 0 | 3 | 3 | 3 | 0 | 0 | 3 | 3 |
| Spooren et al., 1998 (24) | RCT | 17 | Fair | 3 | 3 | 0 | 0 | 3 | 2 | 0 | 3 | 0 | 3 |
| Stecker et al., 2014 (22) | RCT | 19 | Fair | 3 | 3 | 0 | 0 | 2 | 2 | 2 | 3 | 1 | 3 |
TABLE 4. Methodological quality of studies of interventions to increase retention in mental health servicesa
All studies received a methodological rating of good (N=1, 9%) or fair (N=10, 91%). Total points assigned ranged from 13 to 24 (mean±SD=17.6±3.0). Most studies (N=7, 64%) adequately concealed allocation of participants and conducted random selection of the participant population (N=6, 55%). Because of the nature of the interventions under examination, only two studies (18%) were able to blind participants to their treatment group. All studies included equivalent treatment groups (treatments were described for each group and were clearly equivalent) (N=6, 55%) or described the intervention group’s treatment and stated that the control group received usual care (N=5, 45%). All studies included objective outcome measures. Those providing the intervention were monitored in four studies (36%) to ensure that the intervention was delivered with fidelity. The rest of the studies did not describe monitoring procedures but gave detailed descriptions of group interventions. Four studies (36%) provided detailed information about loss to follow-up, such as both the number of participants lost and reasons for attrition by group. Six studies (55%) lost fewer than 10% of participants to follow-up, one (9%) lost 11%−20%, and two (18%) lost 21%−50%. Five studies (45%) included an intent-to-treat analysis.
McFall and colleagues (25), Alegría and colleagues (29), and Scott and colleagues (21) received the highest ratings, with scores of 24, 20, and 20, respectively. McFall and colleagues (25) produced a medium effect (ES=.41) and Scott and others (21) produced small, medium, and large effects on their retention outcomes, whereas Alegría and colleagues (29) reported no effect on the intervention group. However, given that in behavioral research, many ESs are small compared with what may be found in the natural sciences, the presence of even a small difference may be clinically significant (30).
Of all the interventions, those employing the most commonly addressed targets (mental health knowledge, mental health attitudes, and barriers to treatment), had some of the highest mean methodological quality scores (17.8±3.5, 17.8±3.8, and 17.8±4.3, respectively).
Interrater Reliability
No discrepancies were found between the two raters in terms of the extraction of the details of each intervention, for example, sample size, description, outcomes, and results. Some small discrepancies were found related to assessment of the methodological quality of the studies. The scores assigned for each criterion were the same as or within 1 point of each other from 56% to 100% of the time. The ICCs ranged from .37 to 1.00 across all methodological quality indicators, with a median ICC of .86, indicating moderate to high interrater reliability. The ICC for seven of the criteria (adequate concealment of allocation, random selection of patients, care providers blinded to treatment group, treatment equivalence, detailed loss to follow-up information, percentage of patients not in analyses reported, and intention-to-treat analyses reported) were above .80.
Discussion
The findings of this study indicate that many engagement interventions aiming to increase mental health treatment retention address similar factors, most commonly mental health knowledge, mental health attitudes, and barriers to treatment. Moreover, many interventions were comprehensive in that they addressed multiple targets. More comprehensive interventions produced, on average, stronger effects compared with less comprehensive interventions. Specifically, the median ES for interventions addressing only one target was .23, compared with a median ES of .50 for interventions addressing two targets. The median ESs for interventions with three and four targets were even larger (.54 and .59, respectively). On the basis of this evidence, it appears that addressing multiple targets may be necessary in order to have the largest effects on retention.
The targets of the retention interventions reviewed are consistent with existing empirical literature. On the basis of the literature on mental health service use predictors, it logically follows that common elements of retention interventions included addressing mental health knowledge, attitudes, and barriers related to treatment. Notably, most of these interventions appeared to be designed without the benefit of a theoretical model of health behavior or help seeking. Although researchers may have used theories to inform their interventions, only three of the publications explicitly cited one or more theories that informed the intervention development (goal theory, theory of current concerns, and motivational interviewing) (20,23,24).
Potential Biases and Limitations in the Review Process
Some potential limitations of this literature review concerned the studies included in the review. In terms of methodological quality, all studies were rated at the good or fair level. However, only four studies undertook monitoring to ensure that the intervention was delivered with fidelity (19,26,27,29). This presents concerns about the consistency with which the examined intervention was delivered across treatment providers and across sites. Only five studies (19,23,25,26,29) included an intent-to-treat analysis. Moreover, there was remarkable heterogeneity in the reviewed studies in terms of mental health problems addressed, intervention targets and strategies, length and intensity of interventions, service settings, retention variables, methodological quality, and outcomes achieved. This heterogeneity prevented the use of meta-analytic techniques and may obscure the interventions’ effects on retention.
Potential biases and limitations in the review process also existed. The current review can be generalized only to adults in outpatient settings. Every attempt was made to reduce other potential biases by following Cochrane Review methods, such as using two authors to search, select, and review all studies.
Future Directions
The findings of this systematic review indicate that much work remains to retain adults in mental health services. Only 11 studies met criteria for inclusion in the review, indicating a paucity of studies about retention interventions. Our first suggestion, then, is that more research attention and resources be devoted to developing interventions that successfully retain individuals in evidence-based mental health services. To achieve more powerful retention outcomes, we suggest, researchers should incorporate and perhaps adapt theories of health behavior and service use to guide development of retention interventions. At the individual level, dominant theories include the health belief model (HBM) (31) and the theory of planned behavior (TPB) (32). According to the HBM, four types of health beliefs (perceived susceptibility, perceived severity, perceived barriers, and perceived benefits), together with perceived self-efficacy and cues to action (influence from others), affect an individual’s health behavior, in this case, the decision to seek mental health services. The TPB, similarly, focuses on factors (attitude toward the behavior, subjective social norms, and perceived behavioral control) that influence an individual’s intention to perform a health behavior.
Multilevel models that address individual-, social-, and system-level factors affecting utilization of health care also are appropriate paradigms for interventions that target individual behaviors, such as treatment retention, because they consider how individuals interact with their social networks and service systems. Examples of multilevel models include the behavioral model of health service use (33,34) and the network episode model (35). Logic models may be a useful tool for applying theories to explicate theoretical constructs, patient and setting characteristics, intervention targets, intervention strategies, hypothesized mediators of change, retention outcomes, and treatment outcomes.
Use of theories of health behavior and service use could also help develop retention interventions that target the most important factors associated with retention or that more comprehensively target the range of factors involved in retention. Our review suggests that retention interventions should at least address knowledge and attitudes about the mental problems of participants and treatment options as well as barriers to treatment. Perceived benefits and costs of treatment are important components of several theories (31,33–35), and patients likely assess these factors repeatedly during treatment (35); this observation implies that it may be valuable to conceptualize retention strategies as ongoing activities as opposed to pretreatment activities. Thus intervention strategies to improve retention may be integrated into treatment itself in novel ways, such as including retention strategies at the beginning or end of each visit or by conceptualizing participation in treatment as one part of a broader behavior change process, for example, problem solving or seeking social support. As an example of integration, if the treatment involves problem solving or goal setting, then engaging in treatment would be identified explicitly as a goal and as a way to achieve other important goals. It is also likely that development of retention interventions will need to consider the patients’ diagnoses, symptoms, and other personal characteristics as well as the service setting, social network, or treatment modality.
We also suggest that researchers measure a range of retention outcomes as well as investigate how retention outcomes relate to treatment outcomes. Presumably, the overarching purpose of retention interventions is to retain individuals in services until they achieve treatment goals. To consider the full range of retention outcomes, researchers would need to assess more idiographic variables, such as patients’ and providers’ perceptions of whether treatment goals were met. Use of theories of health behavior and service use could help to guide measurement of important outcomes as well as hypothesized mediators and moderators of retention and treatment outcomes. We also recommend that researchers consider methods to measure unintended side effects of retention interventions, which could include overuse of services, stigma, or dissatisfaction. Finally, costs of retention outcomes and their impact on treatment costs deserve greater attention. If costs are assessed consistently across studies, future systematic reviews could identify interventions that are both effective in retaining individuals and cost-effective.
In addition, more research is needed on why retention interventions work. Use of qualitative methods could be used to elicit participants’ views about which intervention components increase retention as well as additional components that should be added. We recommend conducting mediation analyses to identify mechanisms of interventions that improve retention outcomes and moderator analyses to identify which individuals respond to various retention interventions. Such analyses could better integrate theory, mechanisms of interventions, and retention outcomes and potentially lead to more powerful retention outcomes.
When assessing interventions, more rigorous methodology is needed, including ways of monitoring intervention delivery and incorporation of intent-to-treat analyses. When such methods are not included, it is unclear whether attrition bias enhanced or decreased the effect of the intervention. This review also highlights the need for RCTs to evaluate these interventions with the highest level of rigor.
Conclusions
The available evidence suggests that comprehensive interventions addressing mental health knowledge and attitudes and barriers to mental health treatment show the greatest promise for retaining individuals in mental health services. In the future, we recommend that researchers strive to develop retention interventions that are more comprehensive and theoretically informed and to conduct research by using rigorous methodology and a range of retention outcomes, mediators, and moderators. The ultimate goal of adequate retention in mental health services is to improve individuals’ psychiatric symptoms and quality of life as well as achieve significant public health outcomes, including reducing disability and role impairment at the population level.
1 : Twelve-month use of mental health services in the United States: results from the National Comorbidity Survey Replication. Archives of General Psychiatry 62:629–640, 2005Crossref, Medline, Google Scholar
2 : National trends in outpatient psychotherapy. American Journal of Psychiatry 167:1456–1463, 2010Link, Google Scholar
3 : The changing face of US mental health care. American Journal of Psychiatry 167:1419–1421, 2010Link, Google Scholar
4 : Psychotropic drug prescriptions by medical specialty. Psychiatric Services 60:1167, 2009Link, Google Scholar
5 : Barriers to mental health service use in young adulthood; in Young Adult Mental Health. Edited by Grant JE, Potenza MN. New York, Oxford University Press, 2010Google Scholar
6 : Barriers to mental health treatment: results from the WHO World Mental Health surveys. Psychological Medicine 44:1303–1317, 2014Crossref, Medline, Google Scholar
7 : Interventions to increase initial appointment attendance in mental health services: a systematic review. Psychiatric Services 64:1249–1258, 2013Link, Google Scholar
8 : How effective are interventions to enhance adherence to psychiatric medications? Practice implications for social workers working with adults diagnosed with severe mental illness. Journal of Human Behavior in the Social Environment 19:512–530, 2009Crossref, Google Scholar
9 : Improving treatment engagement of underserved US racial-ethnic groups: a review of recent interventions. Psychiatric Services 64:212–222, 2013Link, Google Scholar
10 : Engagement in mental health treatment among adolescents and young adults: a systematic review. Child and Adolescent Social Work Journal 29:241–266, 2012Crossref, Google Scholar
11 : Improving access to geriatric mental health services: a randomized trial comparing treatment engagement with integrated versus enhanced referral care for depression, anxiety, and at-risk alcohol use. American Journal of Psychiatry 161:1455–1462, 2004Link, Google Scholar
12 : Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. JAMA 288:2836–2845, 2002Crossref, Medline, Google Scholar
13 : Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: systematic review and meta-analysis. American Journal of Psychiatry 169:790–804, 2012Link, Google Scholar
14 Higgins J, Green S (eds): Cochrane Handbook for Systematic Reviews of Interventions. Chichester, United Kingdom, Wiley, 2011Google Scholar
15 : Systematic review of psychological therapies for cancer patients: overview and recommendations for future research. Journal of the National Cancer Institute 94:558–584, 2002Crossref, Medline, Google Scholar
16 : Practical Meta-analysis. Thousand Oaks, Calif, Sage, 2001Google Scholar
17 : Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, NJ, Erlbaum, 1988Google Scholar
18 : Psychoeducation to increase psychotherapy entry for older African Americans. American Journal of Geriatric Psychiatry 13:554–561, 2005Crossref, Medline, Google Scholar
19 : Use of a critical time intervention to promote continuity of care after psychiatric inpatient hospitalization. Psychiatric Services 60:451–458, 2009Link, Google Scholar
20 : The addition of a goal-based motivational interview to treatment as usual to enhance engagement and reduce dropouts in a personality disorder treatment service: results of a feasibility study for a randomized controlled trial. Trials 14:50, 2013Crossref, Medline, Google Scholar
21 : Long-term mental health resource utilization and cost of care following group psychoeducation or unstructured group support for bipolar disorders: a cost-benefit analysis. Journal of Clinical Psychiatry 70:378–386, 2009Crossref, Medline, Google Scholar
22 : RCT of a brief phone-based CBT intervention to improve PTSD treatment utilization by returning service members. Psychiatric Services 65:1232–1237, 2014Link, Google Scholar
23 : Engaging women who are depressed and economically disadvantaged in mental health treatment. Social Work 52:295–308, 2007Crossref, Medline, Google Scholar
24 : Strategies to increase compliance with out-patient aftercare among patients referred to a psychiatric emergency department: a multi-centre controlled intervention study. Psychological Medicine 28:949–956, 1998Crossref, Medline, Google Scholar
25 : Effects of an outreach intervention on use of mental health services by veterans with posttraumatic stress disorder. Psychiatric Services 51:369–374, 2000Link, Google Scholar
26 : The Treatment Initiation Program: an intervention to improve depression outcomes in older adults. American Journal of Psychiatry 162:184–186, 2005Link, Google Scholar
27 : Evaluation of a patient activation and empowerment intervention in mental health care. Medical Care 46:247–256, 2008Crossref, Medline, Google Scholar
28 : A simple method for converting an odds ratio to effect size for use in meta-analysis. Statistics in Medicine 19:3127–3131, 2000Crossref, Medline, Google Scholar
29 : Activation, self-management, engagement, and retention in behavioral health care: a randomized clinical trial of the DECIDE intervention. JAMA Psychiatry 71:557–565, 2014Crossref, Medline, Google Scholar
30 : Meta-Analytic Procedures for Social Research. Newbury Park, Calif, Sage, 1991Crossref, Google Scholar
31 : The health belief model; in Health Behavior and Education: Theory, Research and Practice, 3rd ed. Edited by Glanz K, Rimer BK, Lewis FM. San Francisco, Jossey-Bass, 2002Google Scholar
32 : The theory of planned behavior. Organizational Behavior and Human Decision Processes 50:179–211, 1991Crossref, Google Scholar
33 : Societal and individual determinants of medical care utilization in the United States. Milbank Memorial Fund Quarterly 51:95–124, 1973Crossref, Medline, Google Scholar
34 : Revisiting the behavioral model and access to medical care: does it matter? Journal of Health and Social Behavior 36:1–10, 1995Crossref, Medline, Google Scholar
35 : How do people come to use mental health services? Current knowledge and changing perspectives; in A Handbook for the Study of Mental Health: Social Contexts, Theories, and Systems. Edited by Scheid T, Brown T. New York, Cambridge University Press, 1999Google Scholar
