Technology-Assisted Teachers’ Training to Promote Socioemotional Well-Being of Children in Public Schools in Rural Pakistan

Globally, 10%−20% of children are affected by socioemotional problems, and a majority of them live in low- and middle-income countries (LMICs) (1). Despite this tremendous burden and the widespread recognition of the importance of early identification and intervention, most of these children do not receive mental health services (2). Expanding services requires using a lifecourse approach and integrating interventions into existing social and health systems to which children already have access (1, 2). Schools are part of one such system that serves children and youths across a wide age range and offers the opportunity to promote mental health in the context of promoting academic and social outcomes as well (3–6). School-based programs also offer opportunities to identify and address problems early, when they are easier and less costly to ameliorate (5).

Child mental health has been identified as a priority by the World Health Organization’s (WHO) Eastern Mediterranean Regional Office (EMRO). After consultations with international and regional experts and stakeholders, the WHO EMRO developed a school mental health program (SMHP) based on behavioral and socioemotional learning theories (4) and endorsed by its member countries. The SMHP was designed to train school personnel, primarily teachers, to better identify and respond to the needs of children who demonstrate emotional or behavioral problems. A pilot trial conducted in the Pakistani city of Lahore found that SMHP increased teachers’ mental health literacy and their confidence in addressing student mental health problems. A significant improvement in school psychosocial environment was also observed (7).

Key Challenges and Design Solutions

The federal and provincial health departments in Pakistan, a signatory country of the WHO, made recommendations for a phased implementation of the pilot WHO SMHP in the rural Rawalpindi district of Punjab Province, beginning in 10 public schools in the Kallar Syedan subdistrict in April 2016. This phased implementation provided an opportunity to link our research with conventional implementation plans. The Pakistan School Health Implementation Network: Eastern Mediterranean Region (SHINE) team used the “theory of change” methodology to engage stakeholders in revising the SMHP’s content and methods of delivery (8). A series of meetings and three structured workshops involved 90 stakeholders, including policy makers from education and health departments, mental health specialists, researchers, head teachers, classroom teachers, and representatives of nongovernmental organizations. The theory-of-change process identifies preconditions for, assumptions about, and hypothesized pathways for an intervention’s implementation and effect on desired outcomes. The process identified several potential implementation challenges and elicited suggestions to address them. Broadly, these suggestions included the need to adapt existing components of the intervention to the local context and to develop sustainable mechanisms for delivery of quality training and supervision. Informed by the theory-of-change workshops, SHINE adapted the conventional SMHP (cSMHP) to create an enhanced SMHP (eSMHP). Enhancements occurred at two levels: content enhancements and technological enhancements (Box 1).

Objectives and Hypotheses

The objective of the SHINE trial was to evaluate the effectiveness of the eSMHP in reducing children’s socioemotional difficulties compared with the cSMHP. We hypothesize that enhanced training of teachers will improve an intermediate program outcome (i.e., the teachers’ implementation behavior), which in turn will lead to children’s improved socioemotional well-being. The primary outcome will be reduction in parent-rated Strengths and Difficulties Questionnaire (SDQ) total difficulties scores, assessed 9 months after children have been initially screened (9). Additional hypotheses related to children include that the eSMHP will be superior in reducing externalizing and internalizing problems as well as child absenteeism and self-reported stigma; in improving child functioning, well-being, and academic performance; and in increasing parent-teacher collaboration. Our exploratory implementation hypotheses are that the eSMHP will lead to improved teacher competency and fidelity to the training, greater teacher self-efficacy and well-being, and improvements in school environment and parental involvement. Figure 1 summarizes the model that relates teacher training to subsequent child outcomes.

Methods

Overview

The study has obtained approval from Ethics Review Committee of the University of Liverpool in the United Kingdom and the institutional review board of the Human Development Research Foundation in Islamabad, Pakistan.

Study Sites

In each Pakistani province, education is the responsibility of a provincial ministry of education operating in collaboration with the federal ministry. Within provinces, district education departments are administrative hubs for schools. The school system in Pakistan is organized into three levels: primary (grades 1–5), middle (grades 6–8), and high school (grades 9–10). Federal and provincial health and education departments jointly sponsored a 5-year phased implementation of the cSMHP in two subdistricts (Kallar Syedan and Gujar Khan) of Rawalpindi district, overseen by the Pakistani Institute of Psychiatry WHO Collaborating Center (IoP-WCC). Implementation of the cSMHP included in-person teacher training workshops, supervision of teachers in schools, and program monitoring and evaluation conducted by specialist community mental health professionals from the IoP-WCC. This phased implementation of the cSMHP provided an opportunity to study the eSMHP.

The SHINE trial takes place in Gujar Khan, a semirural subdistrict with an approximate population of 1 million people and an economy largely dependent on subsistence farming. Families receive additional support from the earnings of adult male members employed in public- or private-sector services or industry in nearby Rawalpindi City or serving in the armed forces. The human development indicators of Gujar Khan are typical of a low-income developing economy, with an infant mortality rate of 55 per 1,000 live births and a mortality rate among children under 5 of 82 per 1,000 live births (10).

Gujar Khan has 497 public schools (323 primary, 89 middle, and 85 high schools): 231 schools for boys and 266 schools for girls. Teachers specialize in one or more subjects and teach across multiple grades. For each grade level, one teacher is also responsible for managing administrative matters, such as developing students’ annual academic performance report card and holding meetings with parents/primary caregivers. Studies of school-age children in similar settings in Pakistan have found prevalence rates of socioemotional problems ranging from 25% to 35% (11).

Study Procedures

This study is a parallel-arm, single-blind cluster randomized controlled trial (12). A clustered design is appropriate when the nature of the intervention requires it to be implemented for a group of individuals simultaneously and to avoid the possibility that group members could be exposed to both the intervention and control conditions (13). These circumstances are pertinent to the SMHP, which was designed for use with groups of children in classrooms and at the school level. Single schools are the unit of analysis and are allocated 1:1, stratified by girls’ and boys’ schools. Of the 174 middle schools and high schools, 80 (40 boys’ schools and 40 girls’ schools) will be nominated by the district education department and randomly assigned to either the eSMHP arm or the cSMHP arm. To participate, schools must have 80–150 students ages 8–13 years and seven to 10 teachers.

Participants

Nine hundred sixty children, ages 8–13 years, who screen positive on both the teacher and parent/primary caregiver–rated SDQ will be recruited to evaluate the impact of the two versions of the SMHP. SDQ total difficulties scores of teacher-rated and parent-rated SDQs will be used as screening criteria (≥12 and ≥14, respectively) (9). The SDQ has been translated into the local language and used in the region (14). Children will not be eligible if they are thought to be at high risk of abuse or harm to self or others, as reported by the students themselves, teachers, or parents/primary caregivers, or are identified as such by the assessment team during screening. Children also will not be eligible if they are thought to require immediate or ongoing inpatient medical or psychiatric care, are deaf or blind, or have speech difficulties or developmental disorders as defined by the WHO Mental Health Gap Action Programme intervention guide. Exclusion from the evaluation will not affect care received, because participating schools will target all of their students, regardless of whether the student is enrolled in the trial.

Parents or primary caregivers of the recruited children also will be invited to take part in the trial. Parents or caregivers will not be eligible if they do not live with and have primary caregiving responsibilities for the child.

One head teacher from each of the participating schools (N=80) will be invited to take part in the trial, along with an average of four classroom teachers per school (N=320), who will have appointments for the duration of the study. Classroom teachers will be nominated by head teachers to receive the form of SMHP training to which their school is assigned. Teachers will not be eligible if they report having ongoing health problems that require periods of leave or if they report planning to leave their post prior to the end of the trial.

Intervention

The SMHP is a universal early intervention program that aims to empower educators to understand the importance of mental health in school settings, enhance their understanding of child development, and allow them to identify and support children manifesting socioemotional problems (such as anxiety, reactions to trauma, depression, hyperactivity, disruptive behaviors, and conduct problems) in school settings. The WHO SMHP is informed by developmental, behavioral, social, and cognitive theories (4). The intervention targets teachers’ self-efficacy and their ability to implement and generalize skills at the classroom and school levels. The SMHP also includes practical steps enabling teachers to identify children at risk, provide appropriate support, and refer children for further care when required. These steps include periodically using screening questionnaires in the classroom, comparing socioemotional development of an identified child to his or her age cohort, and observing children’s behavior carefully before referring for further evaluation.

In the SHINE trial, both study arms will involve cascading training and supervision (15). Training for cSMHP (the control arm) will be conducted by trainers based at the IoP-WCC who have been trained by WHO master trainers. The IoP-WCC trainers have a psychology or psychiatry degree. Training for cSMHP involves 3 days of didactic and interactive activities (and follow-up supervision once every 3 months) that address basic theoretical knowledge related to mental health in schools and specific classroom and individual intervention strategies. Trainers will be encouraged to use interactive learning techniques, such as group activities. After training in cSMHP, the nominated teachers from each school are expected to train all other teachers in their school.

The eSMHP (intervention arm) was based on the core content of the cSMHP and modified according to findings from the theory-of-change process. Changes to the training format included converting the initial in-person training into a 4- to 5-hour, self-paced online training course, which participating teachers will be asked to complete within 3 days. Progress from module to module in the online course is conditional upon passing postmodule mental health literacy quizzes (16). Next, the nominated teachers from each school will work as a group on interactive activities and role-plays, supported online and in person by study trainers. These teachers will then apply the training in their respective schools by helping other teachers register for and complete the online training course. While both cSMHP and eSMHP teachers will receive follow-up supervision (once every 3 months and a booster session after the summer break that falls during the intervention period) to reinforce their training, eSMHP teachers will also have continuous access to a smartphone-based “chat bot” that provides information summarizing recommended responses to common child emotional and behavioral problems. For both groups, content of the booster session will include material related to the impact of the COVID-19 epidemic on teachers, students, and families. (As in other parts of the world, schools in Pakistan ended their academic years early as part of infection control.) cSMHP boosters will be delivered in person, and eSMHP boosters will be delivered online.

Content modifications in the eSHMP include enhanced materials on child development, counseling skills, and engaging parents. Whereas the cSMHP trains teachers to identify and respond to types of mental illness, the eSMHP trains them to respond to commonly occurring problems and avoids illness labels that could be stigmatizing. The eSMHP also includes an enlarged module addressing teacher wellness. Table 1 outlines the main areas of contrast between the two SMHP versions.

In both study arms, the trainers will perform live competency rating of the teachers by using an adapted version of Enhancing Assessment of Common Therapeutic Factors (ENACT) for SMHP (ENACT Plus) (17) to measure their knowledge and skills regarding SMHP. Only competent teachers (those achieving a mean score of ≥2 on each domain of ENACT Plus) will be allowed to train and supervise the teachers in their respective schools. In both study arms, trainers will rate 20% of training sessions conducted by nominated teachers in their respective schools using ENACT Plus. Teachers who score less than 2 on any ENACT domain will be provided additional support in their supervision meetings to ensure fidelity of program delivery.

Study trainers will supervise nominated teachers in both study arms every 3 months (2–2.5 hours per session). The structured supervision meetings will cover aspects of the SMHP implementation: review of teachers’ well-being, review of any challenges faced in the implementation of the intervention and discussion to mitigate those challenges, use of the chat bot in managing socioemotional problems of children (eSMHP arm only), and collecting program implementation data and referrals and information on success stories.

Each nominated teacher will provide peer supervision to other teachers at their respective schools every 3 months (2–2.5 hours). The agenda and structure of peer supervision meetings will be similar to those of the supervision meetings conducted by trainers with the nominated teachers. As much as possible, the peer supervision meetings will be integrated into the school routine.

Recruiting and Consent Procedures

The “gatekeeper consenting body” for schools is the District Education Department of Rawalpindi, which has already approved embedding the study within the ongoing implementation of the cSMHP in Gujar Khan. After ethics and data safety monitoring board approval, schools that meet size criteria will be randomly assigned to a treatment arm, and their head teachers will be asked for consent for their participation and the participation of their respective schools, teachers, and students. Head teachers will then nominate four to five teachers from their schools to receive direct training from the study team. The nominated teachers will also be asked to provide consent and will be given the opportunity to discuss their participation in the program with the assessment team.

All children in Gujar Khan schools will be involved with the SMHP, but a sample will be recruited by using a multistage process to assess the differential impact of the two SMHP versions. In the first stage, teacher-level screening, parent/primary caregiver consent will first be obtained. Parent/primary caregiver consent forms will be distributed to all children ages 8–13 by the head teacher in each school. Parents/primary caregivers will have 2 business days to return the signed form to the classroom teachers. The consent form explains the study features, principles, and limits of confidentiality for study participants. Ongoing participation from parents/primary caregivers and children will be voluntary. A decision to withdraw from the study will have no consequence on the child’s usual academic activities in the school. This approach to seeking parental consent is informed by school-based trials (13, 18, 19), where initial parent/primary caregiver consent is obtained by head teachers.

After obtaining parent/caregiver consent for teacher-level screening, the assessment team will seek assent for this stage from children. To prevent possible coercion from parents/primary caregivers, informed assent from children will be sought in privacy, at home or in school. Children who do not provide their assent will not be involved in any research procedure.

If the child screens positive in the first teacher screening, the assessment team will seek parent/primary caregiver consent to carry out a second screening with the parent-rated SDQ. If the child screens positive on both the teacher- and parent-rated SDQ, parent/primary caregiver consent for enrollment in the trial will be sought by the assessment team in a third stage, consent for enrollment in the trial. All assessments with teachers and parents about the child will take place after the child assents to participate.

Measures

Baseline and follow-up assessments of the sample of children, teachers, and head teachers will be conducted by a trained research assessment team blind to the allocation status of the trial participants. The assessment will be conducted in the setting of the participant’s choice (home or school).

Primary outcome: Strengths and Difficulties Questionnaire.

Our primary outcome is reduction in total difficulties scores on the parent-rated SDQ (20) among children 9 months after commencing intervention delivery. The SDQ is a brief, valid (9), and reliable measure of socioemotional competence that is widely used to assess socioemotional difficulties in childhood. The parent-rated version of the SDQ will be used in the study because it has demonstrated greater sensitivity for detecting socioemotional problems of children than do the teacher- and self-rated reports (20). The SDQ has 25 items and consists of subscales to measure emotional symptoms, conduct problems, hyperactivity, peer problems, and prosocial behavior. Items are rated on a 3-point Likert scale (0, not true; 1, somewhat true; 2, certainly true). A total difficulties score is calculated by summing the responses of each item in all domains except for the five prosocial behavior items (9). Table 2 shows the full list of study outcomes, when they are measured, and with which assessments (20–31).

Outcome measures.

Proctor and colleagues’ (32) framework will be used to evaluate acceptability, adoption, appropriateness (including cultural appropriateness), feasibility, penetration, sustainability, and cost in both study arms. The Determinants of Implementation Behavior Questionnaire (DIBQ) (21) will be used to measure the change in teachers’ behavior as they implement the SMHP in school and classroom settings. An adapted version of the DIBQ based on 12 domains (knowledge, skills, social, professional role and identity, beliefs about capabilities, beliefs about consequences, memory, attention and decision processes, environmental context and resources, social influences, emotion, and behavioral regulation and nature of behaviors) will be used pre- and postintervention to assess the change in teachers’ behavior regarding implementation of the SMHP.

Analysis Strategy

All analyses will be detailed in a statistical analysis plan. Primary data analyses will be based on the intention-to-treat principle. For analysis of the primary outcome, a linear mixed model will be employed with treatment, time, and interaction between treatment and time as fixed effects; the baseline value of the SDQ total difficulties score as a covariate; and school cluster as a random effect. In addition, an adjusted linear mixed-model analysis will be performed with the prespecified covariates (age groups 8–10 and 11–13, socioeconomic status, and parents/primary caregivers’ education) measured at baseline added into the mixed model described above. The crude and adjusted mean differences in the primary outcome and their 95% confidence intervals will be derived from the mixed models. In addition, subgroup analysis of the primary endpoint will be performed by using the above prespecified covariates. Secondary outcomes will be analyzed similarly using a linear mixed model or generalized linear mixed model. The per-protocol analyses will also be performed as a supplemental analysis (33). No interim analysis of outcomes is planned.

Mediation analysis.

An exploratory analysis will be conducted to understand the potential mediating effects of teachers’ program implementation behavior on the relationship between the SMHP and SDQ scores 9 months after commencing intervention delivery. We will estimate direct and indirect effects of the intervention using a model-based mediation package by Imai and colleagues (34). All analyses will be exploratory, given the limited power for detecting these interaction effects and difficulties in interpreting mediational effects as causal (35).

Moderator analysis.

An exploratory moderator analysis will be conducted to help clarify for whom and under what circumstances (potential moderators) the SMHP works. We will assess modification of treatment effect with potential, a priori–defined modifiers/covariates (age group, socioeconomic status, parents/primary caregivers’ education, baseline severity of socioemotional problems, school environment, and parents/primary caregivers' interaction with their child’s teacher) by fitting an appropriate interaction term in the linear mixed model.

Process evaluation.

We will conduct mixed-methods research to assess assumptions underlying the intervention and implementation strategies, such as acceptability, feasibility, appropriateness (including cultural appropriateness), and adoption (36). In-depth interviews and focus group discussions will be conducted with participants of both arms, at their preferred setting. Sampling for qualitative interviews will be purposive so that individuals with knowledge of and exposure to the SMHP will be included for each category of respondent. Sampling may be extended on the basis of emerging results. Sampling for in-depth interviews will continue until theoretical saturation has been reached. We anticipate eight to 15 interviews with respondents from each category to reach theoretical saturation. Data analysis will be based on the framework analysis approach (37). Quantitative implementation outcome measures will be used to explore the program’s acceptability, feasibility, appropriateness, adoption, and cost (38).

Results

The trial is expected to yield considerable information about feasible methods for engaging school systems, teachers, and families in implementing mental health programs at scale. Most importantly, it will provide information about how implementation methods in both the planning and training phases relate to the impact of the program on students. In many low-resource communities, it is not possible to employ dedicated mental health professionals in schools, and even then, consultation by students may be limited by stigma, insurance status, or lack of parental consent (39). In these communities, school personnel need to be able to respond therapeutically to students who are experiencing emotional or behavioral problems, but those personnel must come to feel that taking on a therapeutic role will enhance rather than compete with their roles as educators. The SHINE trial will test whether a redesign of the WHO SMHP based on specific input from school personnel will lead teachers to more consistently and effectively help students in distress. Thus, the trial results will have implications not only for specific aspects of the content and training that encourage teacher use but also for the process by which school mental health interventions are introduced into school systems and adapted for those systems’ unique needs.

We expect that suggestions will emerge for further changes to both the SMHP content and implementation process that could enhance its effectiveness. These will be shared with SHINE and WHO EMRO stakeholders, including the SHINE partners from Iran, Egypt, and Jordan, and will influence further use of the SMHP and similar or complementary programs. Some of the most interesting results may be whether the SMHP seems to help some children more than others and what modifications may be necessary to identify and help children for whom the program has limited impact. We also strongly suspect that students’ improvement will be moderated or mediated by parental involvement, a factor that has historically been a barrier to school-based mental health programs (40). In some communities, school mental health programs may require explicit investment in personnel who can encourage family participation in all facets of their children’s educational experiences (41).

Next Steps

The SHINE trial will test whether a careful process of stakeholder collaboration (in this case through a series of theory of change workshops) can help adapt a school mental health program so that it is more completely embraced by teachers and ultimately is more effective in reducing symptoms among students. The trial began in October 2019 and is expected to be complete in March 2021. Outcomes will inform dissemination of the SMHP in Pakistan and other countries.