Youth Functioning and Organizational Success for West African Regional Development (Youth FORWARD): Study Protocol

Data from the 2016 Global Burden of Disease Study (1) indicate that globally, 162.5 million disability-adjusted life years (DALYs) were lost because of mental and addictive disorders, 6.8% of all DALYs lost in that year. In sub-Saharan Africa, where the burden from noncommunicable diseases is increasing, total DALYs lost due to mental disorders increased by 113.9% from 1990 to 2017 (2). As much as 78% of psychiatric disorders are untreated among adults in low- and middle-income countries (LMICs), and figures are likely higher for adolescents and youths (3, 4). Among youths, mental health problems resulting from trauma exposure and psychological distress are often expressed as externalizing (e.g., hostility) and internalizing (e.g., withdrawal) behaviors. Emotion dysregulation—difficulties in modulating strong emotional responses—is a major neurobehavioral mechanism operating across a range of mental health issues, including traumatic stress, depressive symptoms, and externalizing problems (5–7). Emotion dysregulation can lead to myriad behavioral and interpersonal problems, which can affect one’s ability to engage in healthy behavior and participate in livelihood opportunities (8–20).

Similar to other postconflict LMICs, Sierra Leone has limited health care infrastructure and faces challenges in the delivery of mental health services (21). Since gaining independence in 1961, the country has experienced several social and political upheavals (a timeline is available in an online supplement). The Ebola outbreak of 2014–2015, which resulted in 3,956 deaths (22), further weakened the health care system and exacerbated the mental health treatment gap (23). The formal and informal structures set up to support mental health services across sectors lack coordination and resources, which has resulted in a disjointed, ineffective system (24). Sierra Leone’s mental health workforce is grossly inadequate (25). The Sierra Leone Psychiatric and Teaching Hospital is the only dedicated mental health facility in the country, and community-level mental health services are scant.

The government of Sierra Leone’s economic development and health policy plans, including the recently launched Mental Health Policy and Strategic Plan for 2019–2023, underscores the importance of health and mental health to advance human capital. The limited reach of mental health services in Sierra Leone, coupled with strong political will, presents an opportunity to deliver evidence-based mental health interventions via the alternative delivery platform of youth employment programming.

Youth Functioning and Organizational Success for West African Regional Development (Youth FORWARD), funded by the National Institute of Mental Health, is an implementation science collaboration focused on scaling out evidence-based mental health interventions for youths exposed to war, community violence, and other adversities. During the course of Youth FORWARD’s work, the term “scaling-out” emerged, referring to the process whereby evidence-based interventions (EBIs) are modified for a new delivery system, a new population, or both (26). We refer to our effectiveness trial as a scale-out study per this recent nomenclature that more accurately captures our approach to delivering a mental health program, the Youth Readiness Intervention (YRI), via youth employment programming.

Youth FORWARD builds on 18 years of research on the effects of war, violence, and postconflict adversity on the mental health of youths in Sierra Leone. The Longitudinal Study of War-Affected Youth (LSWAY; 1R01HD073349–01), launched in 2002, was the first longitudinal study in sub-Saharan Africa to examine trajectories of psychosocial adjustment and social reintegration among a cohort (N=529) of male and female war-affected youths (12, 27–31). The LSWAY demonstrated how trauma and loss influence emotion dysregulation, interpersonal deficits, and impairments in daily functioning (28, 29) while elucidating trajectories of risk and resilience with attention to modifiable factors that could be targeted by behavioral interventions (32). Formative intervention research illuminated a gap in mental health services for youths: programs focused on classic symptoms of posttraumatic stress disorder (PTSD) and offered exposure-based PTSD treatment, but few programs focused on the emotion dysregulation and interpersonal difficulties that impeded success in educational and livelihood programs. Focus groups with community leaders, youths, and professionals led to the YRI and its orientation as a transdiagnostic, common elements–based intervention that could be safely delivered with fidelity in community settings by lay workers receiving strong training and supervision (11, 32).

A randomized controlled trial (RCT) in an educational setting demonstrated that youths who received the YRI reported significantly greater improvements in emotion regulation, prosocial attitudes and behaviors, and daily functioning than did youths in a control group, with YRI participants six times more likely to persist in school (11). Given constraints in Sierra Leone’s health system and high rates of mental health problems within the population, it is critical to test alternative delivery platforms such as schools and youth employment programs for the integration of EBIs such as the YRI.

Youth FORWARD is delivered via a collaborative team approach (CTA) that draws from the implementation science strategy of interagency collaborative teams tested to scale out the SafeCare child protection intervention in California (33) as well as elements of learning collaboratives and quality improvement strategies developed by the Institute for Healthcare Improvement (34). These approaches emphasize the importance of collaboration and reinforce knowledge sharing, problem solving, and intervention oversight, creating a community of practice among stakeholders. The CTA model is being tested as an implementation strategy for scaling out and sustaining quality in the delivery of the YRI.

Implementation of the YRI is guided by the exploration, preparation, implementation, sustainment (EPIS) conceptual model (35). In this four-phased, multilevel framework, implementation determinants in the outer (system) and inner (organization) context, as well as bridging factors that work across the contexts, illuminate and capitalize on the relationships among stakeholders in the implementation process (35). The EPIS framework was used to identify key supports needed within each phase, and the CTA model specified the steps to achieve the supports while addressing challenges (33).

Hypothesis and Aims

By leveraging investments in youth and economic development programs and using CTAs to facilitate integration of mental health interventions within existing delivery platforms, our hypothesis was that LMICs with limited mental health care infrastructure can build capacity to address the mental health treatment gap. Three aims guide Youth FORWARD. Aim 1 (implementation impact evaluation) is to utilize an innovative CTA to scale and sustain the YRI in terms of the feasibility of this approach, facilitator preparedness, sense of satisfaction among facilitators, and impacts on fidelity and sustainment of YRI delivery. Aim 2 (implementation process evaluation) is to identify internal and external factors influencing the integration of the YRI into a youth employment promotion program (EPP) via a process evaluation documenting barriers and facilitators to effective implementation and integration. Aim 3 (clinical effectiveness) is to compare the clinical effectiveness of the YRI when delivered via the EPP platform with results of our previous RCT of the YRI, as measured by improved mental health and reduced functional impairments among high-risk youths. Emotion regulation will be examined as a major mechanism by which the YRI improves behavior of treated youths and their functioning in the EPP.

Methods

Overview of Implementation Science Framework

The EPIS framework draws on five principles of systemwide implementation: generating shared investment in implementation of evidence-based practices, creating a process for incorporating local expertise within multiple organizations to build institutional knowledge, optimizing resources to address known implementation challenges, focusing on quality assurance and appropriate oversight within systems change, and developing an implementation structure that focuses on communication and workload sharing (35–37). Youth FORWARD chose this framework because it illuminates the key outer and inner factors within each phase to guide the implementation and sustainment of EBIs (a figure depicting application of the EPIS framework is available in an online supplement). Models that rely heavily on remote expertise for training and ongoing fidelity monitoring have proven to be an obstacle to scaling and sustaining EBIs in LMICs because such models fail to develop local expertise. To remove the need for remote expertise in training and oversight, Youth FORWARD uses the CTA to scale, sustain, and integrate an evidence-based mental health intervention into existing employment programming.

The CTA is adapted from the EPIS-guided interagency collaborative team approach, which focuses on collaboration as a key element of the implementation and sustainment of evidence-based practices into established service delivery systems (33). Within Youth FORWARD, collaboration primarily involves development of in-country expertise and capacity building directed at local service providers. Specifically, the CTA model creates a community of practice around an EBI through development of a seed team, a local unit of experts, to provide training, coaching, and support as the intervention is scaled out. The seed team monitors and supervises new facilitators as they deliver the EBI while overseeing cross-site collaboration to expand institutional knowledge on best practices in EBI delivery.

Key CTA activities include creation of a cross-site expert seed team; cross-site learning and routine communication; collection of process data for monitoring quality and illuminating barriers and facilitators; structured fidelity monitoring and targeted supervision; and use of plan-do-study-act (PDSA) cycles to identify, analyze, and solve problems (38). Using collaborative teams and scaling across sites where lessons learned from one site can improve delivery in another site are strategies to develop a culture of quality improvement and cross-site learning that can enhance inner and outer factors that reinforce sustainment.

Overview of YRI

The YRI is designed to assist youths facing complex problems by using evidence-based treatments that have been tested and shown to be effective in addressing the mental health of violence-exposed youths in a range of settings and cultures (11, 39, 40). The YRI has three goals: develop emotion regulation skills for healthy coping, develop problem-solving skills to assist with achieving goals, and improve interpersonal skills to enable healthy relationships and effective communication. The intervention integrates six empirically supported practice elements adapted from cognitive-behavioral therapy that have demonstrated effectiveness across disorders ranging from major depressive disorder to anxiety and conduct disorders (40–42). The YRI is organized into 12 weekly 90-minute group sessions, delivered by same-gender lay facilitators to gender-matched groups and supported by a robust training and supervision structure that can sustain best practices (40). Youth FORWARD’s key implementation partner, Caritas Freetown (hereafter Caritas), employs YRI experts who were trained by the YRI developers and worked as YRI facilitators in the prior RCT. YRI delivery is supported by fidelity monitoring via audio-recorded intervention sessions and direct observation to bolster YRI facilitators’ skills and reinforce key YRI components. The fidelity monitoring is then integrated into individual and group supervision.

Because of the challenges facing Sierra Leone and the need for delivery of mental health programming via well-established structures (Box 1), the YRI will be integrated within the youth EPP created by Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ). GIZ has been working in Sierra Leone since 1963 and is keenly aware of the issues facing Sierra Leonean youths (43). The EPP responds to market demand within targeted districts through a needs-oriented approach to ensure that youths can obtain the skills required for employment. The youth capacity development component of GIZ’s EPP includes several module-based focal points, including an entrepreneurship training (ENTR) program. Per input received from local communities, the ENTR program also includes a psychosocial competency module. GIZ uses a competitive bidding process to contract with local service providers to deliver their programs in Sierra Leone. The ENTR training is delivered 5 days per week for 3 weeks.

Study Sites

Youth FORWARD operates in rural areas of Sierra Leone because of the presence of GIZ’s ENTR programming. As such, the pilot was delivered in one rural district (Kailahun), whereas the scale-out study is being conducted in three rural districts in eastern and northern Sierra Leone (Kailahun, Kono, and Koinadugu). Districts in Sierra Leone are divided into chiefdoms, and four chiefdoms per district have been selected to participate in the scale-out study. There is no overlap between chiefdoms from the pilot and scale-out study. Given their rural locations, study districts have limited Internet connectivity, poor road infrastructure that can become impassable during rainy season, and economies that prioritize agricultural production and mining. Youths participating in the YRI may share skills learned with peers and community members, which could make it difficult to determine any differences that exist between the YRI+ENTR and ENTR-only study arms. Thus, study sites are situated at the community level and consider the location of youth clusters to avoid natural diffusion and spillover effects.

Study Procedures

After the exploration phase of the EPIS implementation science framework (35), the Youth FORWARD team negotiated with GIZ to use the evidence-based YRI for the psychosocial competency training module. The team established a memorandum of understanding for use of the YRI curriculum and training package delivered by the Caritas-based YRI experts to the GIZ-contracted ENTR agency. The CTA included the following parties: YRI developers, YRI experts, YRI facilitators, and the Youth FORWARD team. Peripherally, GIZ, the contracted service provider agencies overseeing delivery of the YRI and ENTR program, and Caritas were also part of the CTA.

To prepare for training new YRI facilitators who would deliver the YRI for our pilot and scale-out studies, the YRI experts formed the seed team. As part of seed team responsibilities, the experts were trained in good practice principles to enhance learning; prepare them to facilitate YRI training; and develop strong, competent YRI facilitators. They were also trained on the Youth FORWARD CTA model and EPIS framework. Last, they learned about the key CTA activities they would be expected to carry out and planned how they would do so in the field.

All participants provided informed consent prior to participation. Study approval was granted by the Boston College Institutional Review Board and Sierra Leone Ethics and Scientific Review Committee, with study monitoring oversight by an independent data safety and monitoring board.

Pilot study.

A cluster randomized pilot feasibility trial with a quasi-experimental untreated control group was employed to pretest implementation science aspects of the study (i.e., use of CTA) and refine study assessments. In total, 175 youths ages 18–30 were enrolled; 62% (N=108) were female. Participants were assigned to clusters stratified by chiefdom and training site location; clusters were randomized into YRI+ENTR (N=58) or ENTR only (N=57). A nonrandomized, statistically matched untreated control group (N=60) was recruited from chiefdoms outside of GIZ’s programming in Kailahun. Third-party reporters (N=120) reported on youth participants at two time points. YRI facilitators (N=16), YRI experts (N=4), and agency leaders (N=2) participated in qualitative key informant interviews postintervention, for a total pilot study sample of 317 participants. Results from the pilot study demonstrated that integrated YRI and ENTR delivered within the CTA with robust training and supervision by a seed team of experts is both feasible and acceptable (44). Mixed-effects models indicated improvements in core mental health outcomes (e.g., emotion regulation, functioning) for youths receiving treatment (YRI+ENTR and ENTR only) compared with youths in a matched control group (44). As a precursor to a well-powered scale-out study, the pilot also supported measures refinement; updating of study inclusion criteria and recruitment procedures given the rural location and demographic makeup of study districts; and strengthening of our partnership with GIZ, including adapting our communication and engagement structure for before and during program implementation.

Scale-out study.

For the scale-out study, a hybrid type-2 effectiveness-implementation cluster randomized three-arm trial is being employed, guided by the EPIS framework. A total of 1,151 youths ages 18–30 were enrolled across these three districts (47%, N=544, were female), along with two agency leaders, 12 YRI facilitators, and 618 third-party reporters for a total sample of 1,783 participants. Eligible youths were stratified into clusters on the basis of geographic location and gender. Clusters were then statistically matched into triads on the basis of the following variables: age, sex, marital status, number of dependents, education, previous skill training, income-generating activities, days and hours worked in the past month, World Health Organization Disability Assessment Schedule (WHODAS) score, Difficulties in Emotion Regulation Scale (DERS) score, locality access to a highway, characterization as a hub village, and locality population. Matched clusters were then randomly assigned into the three study arms: 387 youths were clustered and randomly assigned to the control group, 380 youths were clustered and randomly assigned to ENTR only, and 384 youths were clustered and randomly assigned to YRI+ENTR. Data will be collected at baseline, post-YRI, post-ENTR, and 12-month follow-up. Remaining study activities include the collection of data at 12-month follow-up.

Scale-out study outcomes.

Primary outcomes of the scale-out study focus on intervention process and implementation variables according to the EPIS framework, including a costing analysis; fidelity to intervention protocol; and the sustained delivery of the YRI within a CTA to enhance intervention delivery, training, and supervision. Secondary outcomes are emotion regulation skills, interpersonal skills, and functional impairment. A comparison of youths who receive the ENTR with those who receive the YRI+ENTR will determine whether the YRI adds value to the ENTR in terms of positive effects on emotion regulation and interpersonal functioning and whether improvements influence economic self-sufficiency over time (a figure depicting the scale-out study’s aims and primary outcome measures and a detailed data collection plan are available in the online supplement).

Scaling Out the YRI

Three YRI experts comprised the seed team and worked across each study district to ensure processes related to quality improvement (i.e., cross-site learning, fidelity monitoring) occurred as expected. There was a change in the service provision agency contracted by GIZ for the scale-out study, which meant the seed team members added during the pilot study could not be part of the scale-out, and a new cohort of YRI facilitators were trained.

During the preparation phase, GIZ utilized community sensitization and newspaper and radio advertisements to recruit youths to participate in the ENTR. Applications submitted to GIZ’s district-based offices were transferred to Caritas for review and digitization by research assistants. Caritas research assistants then contacted youths who consented to be screened for eligibility. Criteria for youth eligibility included being male or female ages 18–30, disengaged from school, not pregnant, and having elevated scores on the WHODAS and DERS. Locations of participant residence and ENTR training sites were used to determine arrangement of clusters. Statistically matched clusters of youths were randomly assigned into one of three arms: control, ENTR, or YRI+ENTR (a figure describing the study arms is available in an online supplement).

To evaluate post-YRI outcomes, a subsample of youths from the ENTR-only and YRI+ENTR study arms were administered a reduced assessment battery immediately after the YRI. We randomly selected clusters of youths from each treatment arm until we reached our desired sample size of 400 youths (200 from each treatment arm).

Because the scale-out study utilized a staggered YRI rollout across study districts, YRI experts engaged in within-district problem solving and supervision to enhance YRI delivery. Seed team members and other stakeholders, including GIZ, Boston College, and the implementation agency, utilized structured terms of reference to outline roles and responsibilities. This structure supports a community of practice around the YRI that prioritizes the use of problem-solving processes such as PDSA cycles and cross-site learning that allow best practices to be shared and a culture of quality improvement to be institutionalized as part of the YRI scale-out.

As part of a process evaluation, data were collected on the implementation of the CTA and potential for sustaining quality improvement over time. Facilitators worked across training sites and participated in face-to-face and phone-based supervision. Facilitators also collected process data throughout the intervention and utilized supervision to engage in troubleshooting to create a feedback loop that, over time, enhances YRI delivery.

Analysis Strategy

Quantitative data will be analyzed by using a multilevel modeling approach. To determine the effect of the YRI as delivered by facilitators within the ENTR, standardized mean difference between treatment conditions of approximately 0.3 for youth outcomes was assumed. An effect size of 0.3 is similar to what was observed in the YRI RCT (11). The scale-out study includes 1,151 participants clustered and randomly assigned into three arms (control, ENTR only, and YRI+ENTR). Each arm includes 20 clusters, for a total of 60 clusters. The power calculations assume two sex-segregated subgroups of 10 participants per cluster. A multilevel modeling approach (level 1, time point; level 2, individual; level 3, intervention group; level 4, site) will be used to accommodate the potential loss of precision due to attrition (estimated to be a maximum of 20% at last follow-up). Power is estimated by accommodating 10% attrition at the study midpoint. Under intent-to-treat, all participants initially observed will be included in all analyses regardless of their participation in ENTR or YRI+ENTR. Multilevel models will investigate how youth outcomes differ between the YRI+ENTR group and the ENTR-only and control groups over time. A four-level model will be used to compare YRI+ENTR participants with those in the ENTR-only and control groups to assess whether there is greater change in mental and behavioral health outcomes in the YRI+ENTR groups.

Qualitative data will be analyzed by using grounded theory (45) and an analytical strategy derived from thematic content analysis (46). Both qualitative and quantitative data will be synthesized to understand barriers and facilitators to using the CTA to deliver evidence-based mental health programming in Sierra Leone. These methods will allow us to examine areas of convergence or divergence in the data. If contradictions arise, we may examine qualitative and quantitative data on hypothesized associations to establish relationships that may be tested further (for more details, see the online supplement).

Results

Analysis of data from the hybrid type-2 effectiveness-implementation scale-out study includes three primary evaluations. An implementation impact evaluation guided by the EPIS framework will assess the use of the CTA to scale and sustain the YRI with a focus on feasibility, facilitator satisfaction and support, and fidelity to core YRI practices. An implementation process evaluation also guided by EPIS will identify internal and external factors influencing the integration of the YRI into the youth ENTR program via a process evaluation documenting barriers and facilitators to effective implementation and integration. A clinical effectiveness evaluation will compare YRI clinical effectiveness when delivered via the GIZ employment platform with results of a previous RCT of YRI as measured by improved emotion regulation and reduced functional impairments among high-risk youths.

The implementation and testing of the YRI as integrated into the GIZ-supported ENTR program via the CTA strategy is intended to improve the mental health and functioning of vulnerable Sierra Leonean youths while demonstrating the capacity for an EBI to be integrated into an alternative delivery platform. Demonstrating that the YRI can be delivered with fidelity and quality improvement by lay workers linked to ongoing employment and entrepreneurship programming and supported by the CTA strategy will provide evidence for further adoption and scale-out of the YRI using alternative delivery platforms in Sierra Leone and other LMICs.

Next Steps

An opportunity exists to build capacity in fragile and conflict-affected regions by leveraging investments in youth economic development programs to integrate evidence-based mental health interventions therein. In such settings, innovations are needed to address the mental health treatment gap. Study findings will support efforts by the government of Sierra Leone, development actors, local universities, human services agencies, and other partners to address human resources challenges and increase access to mental health services for youths.

Should the YRI as implemented under the CTA and linked to youth entrepreneurship programs prove effective with emotion regulation and functional impairment as well as with ENTR program participation and economic self-sufficiency, this study will strengthen the case for evidence-based mental health interventions in fragile settings. Data from this study can be used to identify elements of the EPIS framework and CTA influencing integration of the YRI into the ENTR and possible pathways for sustainment. Lessons learned can be applied in future efforts to scale out EBIs in postconflict settings.