A Collaborative Process for Evaluating Infant Mental Health Home Visiting in Michigan

Infant mental health home visiting (IMH-HV) is the primary service delivered to at-risk parents and their young children by Medicaid-funded community mental health service providers (CMHSPs) across the state of Michigan. This column describes an innovative collaboration that could serve as a model for meaningful community-university-state partnerships. The collaboration was initiated in response to recent legislation that put funding for the IMH-HV program in jeopardy. This urgent need prompted the formation of this partnership to develop the evidence base and to engage in a quality improvement and quality assurance process in order to sustain this critical program. To develop and maintain this partnership, several key elements were needed to support each partner, and the University of Michigan Department of Psychiatry served as the evaluation hub. The following sections provide a succinct history and overview of the project, detail the elements that stakeholders deem necessary to the success of this partnership, and briefly discuss the success and impact of this community-university-state collaboration.

History

The IMH-HV model is delivered across Michigan through CMHSPs, and it has served thousands of Medicaid-eligible infants, toddlers, and families. The model has a long tradition in Michigan, having been developed in the 1970s by Selma Fraiberg at the University of Michigan and integrated into the state public mental health system as a prevention service for infants at risk of early relationship failure and potentially resultant developmental delays, maltreatment, and behavioral and emotional disorders. With expansion of IMH-HV to all community mental health sites in Michigan, several hundred clinicians now provide these specialized services across the state.

A formal association representing and connecting practitioners across the state was founded in 1977: the Michigan Association for Infant Mental Health (MI-AIMH). MI-AIMH has worked closely with the state to support the practice with workforce development and standards (Endorsement for Culturally Sensitive, Relationship-Focused Practice Promoting Infant Mental Health) to enhance quality in delivery of IMH-HV (1).

IMH-HV is a relationship-based, home-visitation preventive intervention model designed to enhance parental capacities to provide responsive, nurturing care; to reduce the risk of child maltreatment; and to improve social-emotional development. Services are delivered to vulnerable, Medicaid-eligible families during pregnancy and through the child’s age of 36 months. The families served have environmental or familial concerns that place their infant or toddler at risk, and a parent or child has a psychiatric diagnosis. IMH-HV is based on well-established clinical and developmental theory, with refinement based on implementation experiences with thousands of families.

Given this tradition and evidence for effectiveness of other models that have grown out of Michigan’s IMH-HV program (such as child-parent psychotherapy) (2), IMH-HV is an established “promising practice,” as designated in 2015 by the Michigan Department of Health and Human Services (MDHHS) and the Michigan Department of Education (3). However, IMH-HV has not been directly evaluated with the methodological rigor necessary for designation as an evidence-based practice. A pilot evaluation confirmed the high-risk status of families served and indicated likely benefits of the program for child outcomes (Abbey J, unpublished manuscript, 2005). Yet the impact of this study was constrained by high attrition, which was likely attributable to perceived burden from clinicians who collected data and agencies that may not have completely “bought in” to the evaluation. This history underscored the critical need to engage community agencies and clinicians as partners to foster meaningful investment and minimize burden in any future evaluation process.

Recent legislation in Michigan determined that funding for home visiting services will be contingent on meeting clearly specified standards for evidence-based practice. This legislation (Voluntary Home Visitation Programs, Act 291 of 2012) and the federal Affordable Care Act require that evidence-based models be subject to rigorous experimental or quasi-experimental evaluation, including measuring impact on key benchmarks with publication of study findings. Given that Michigan’s IMH-HV has not been evaluated to these standards, its funding and sustainability are in jeopardy. Thus, staff from the MDHHS contacted faculty at the University of Michigan to initiate and lead an evaluation, and together with involvement of the MI-AIMH, the community-university-state collaboration was initiated. A leadership team comprising the authors of this column and representing each branch of the partnership was established to jointly design the study and advise the project (MDHHS, ML; University of Michigan, JML, KLR, and MM; and MI-AIMH, DJW and BT). Additional contributions to study design were made by a statewide faculty research advisory board.

IMH-HV Evaluation Study

Briefly, the IMH-HV evaluation has two concurrent projects. Study 1 is broad, capturing data on every family receiving IMH-HV services in Michigan’s most populous region, Detroit–Wayne County (approximately 300–400 families). IMH clinicians at Detroit-Wayne CMHSP sites are administering clinically relevant assessments to infants and families on a quarterly basis. Included are measures of parent depression, parenting stress, and child development. Measures were kept brief to minimize clinician burden. Data collected over the course of up to two years of intervention will be amassed and analyzed by the research team.

Study 2 is a more in-depth evaluation of a smaller group of infants and families. Twelve programs in seven counties across the state are participating in study 2. The study enrolled 81 families (including 93 parents) as they began receiving IMH-HV services. IMH specialists recruited families to participate in the evaluation and provide biweekly data. Research staff interact with families directly to measure multiple domains of functioning at the beginning of treatment and three, six, nine, and 12 months later. These quarterly assessments include measures of parent and child health, parenting, child-parent relationship quality, child development, and parental stress and will permit evaluation of intervention efficacy across a broad spectrum of outcome domains. Because Medicaid-funded agencies cannot deny service to eligible applicants (thus a randomized controlled trial was not possible), the study team will create a comparison group by utilizing state data on select available measures to contrast outcomes for study participants at the conclusion of the project with an age- and demographically matched group of children entering treatment. For example, a participant who is 36 months old at the conclusion of the study will be compared with a 36-month-old child who was recently referred for services by using data collected by clinicians at the same site.

Identification of positive impact on key outcome domains will allow providers to access state funds for delivery of IMH-HV in the years to come. Both studies are primarily funded by Michigan Medicaid administrative dollars, with matching funds from the Ethel and James Flinn Foundation and the Gerstacker Foundation. Additional financial support was provided by the University of Michigan, MI-AIMH, and children’s mental health block grant funds.

Community Partners

The community partners include CMHSP administrators, supervisors, and clinicians, including the Detroit-Wayne Mental Health Agency Administration (DWMHA), and MI-AIMH. Fourteen agencies across studies 1 and 2 are participating, with approximately 130 IMH clinicians involved.

Key Elements Needed

Engagement and training.

The leadership team contacted CMHSP administrators and invited them to participate in the study. Engagement began with a call between a state agency representative, University of Michigan faculty, and CMHSP administrators to explain the situation and get buy-in for the study. Next, research staff visited each site to further explain the study and to begin discussing logistics and communication. Researchers met with clinicians and administrators to identify evaluation measures that would have clinical utility and that could be integrated into the standard of care for IMH going forward. Thus, measures served both research and clinical quality assurance purposes. In addition, training of IMH clinicians, requiring administrators to release clinicians from duties to give them time to attend, was a key aspect of engagement and mutual benefit. Training was provided by University of Michigan faculty and began with describing the history of and rationale for the study to engage the IMH community. Training also involved review of IMH principles, introduction of a fidelity tool, techniques for clinical use of study measures, and introduction to study procedures. The training allowed IMH clinicians and supervisors to voice their concerns about the study and to discuss potential problems with the research team to mitigate barriers and enhance project success. Research staff also met with administrators and supervisors to establish the site-specific protocols for recruitment, data collection, and communication.

Ongoing support.

Research staff maintain ongoing communication with each site, including monthly check-in calls as well as e-mail and phone contact with IMH clinicians and supervisors. Research staff carefully monitor data collection to ensure that data are captured fully and accurately. For study 2, research team members are in close communication with referring clinicians to schedule in-home assessments. The study hired research staff with previous IMH-HV experience to conduct the assessments. This arrangement increased credibility and trust between clinicians and the research team. Research staff also provided support to DWMHA workers amassing data from multiple agencies.

Active involvement.

The leadership team held monthly calls and identified how partners could contribute to and benefit from the study. Protocol modifications were made in response to feedback from clinicians. For example, the research team had not initially planned on inviting IMH clinicians to the in-home assessments, but clinicians indicated that they could glean relevant information from these visits, as well as possibly build trust between families and the research team. IMH clinicians are now invited to each assessment visit, and most of them attend, which increases participant engagement.

Contributions and Benefits

Community partners were essential for recruitment and data collection. MI-AIMH was instrumental in publishing the third edition of the IMH-HV manual, Infant Mental Health Home Visiting: Supporting Competencies/Reducing Risks (4), as well as in the development of an IMH-HV fidelity tool, which was modified for the study.

The study training offered continuing education credits for IMH clinicians and counted toward the competencies established for IMH endorsement. Many IMH clinicians commented that the training, especially the fidelity tool, was beneficial for improving service delivery and making documentation more efficient. IMH clinicians in study 2 were also able to observe the study assessments, which many found to be clinically useful. The assessments implemented in study 1 will continue to be used beyond the study period to offer IMH clinicians valuable information about maternal depression, parenting stress, and child development.

Agencies received a small monetary incentive for participation. In addition, the project established an infrastructure for accumulating outcome data across IMH sites for continued quality assurance and improvement efforts and provided agencies with a treatment fidelity tool. Finally, the study will create an evidence base for this intervention, which will benefit agencies and clinicians who will continue to be able to be reimbursed for their services.

State Agency Partners

Collaborators at the state level were members of the MDHHS–Mental Health Services to Children and Families.

Key Elements

Support.

The research team supported state agency leaders by collaborating on applications for funding for program evaluation.

Active involvement.

The role of the team member from MDHHS was significant to the success of the collaboration, in that she participated in the leadership team to ensure that the study was meeting the state legislative mandates. She was invited to the community agency training sessions to address relevant questions raised by agency administrators and providers.

Contributions and Benefits

State partners were instrumental in initial engagement of community agencies. A consultant within MDHHS acted as the liaison to connect the research team to community agency administrators. The state was also instrumental in acquiring additional funds through the Medicaid Match process.

The goal of the project is to provide evidence to support the sustainability of the IMH-HV program and continued access to funding at the state level. Researchers raised concerns about research integrity and emphasized that the evaluation could not guarantee positive findings on program effects. To address this issue, we designed careful and thorough assessments to enhance likelihood of detecting an effect. Furthermore, regardless of study outcome, the project will provide a detailed characterization of the experiences of the families served by IMH-HV, and it creates a database for quality assurance and improvement initiatives across multiple sites countywide and statewide.

Research Partners

In addition to the main research lab at University of Michigan, the project also enlisted the support of a research advisory board comprising researchers from across Michigan. Before this collaboration, the principal investigators were part of an informal group of researchers in Michigan with a shared interest in early childhood—the Michigan Infant and Toddler Research Exchange (MITRE). Participating MITRE faculty in the early stages of evaluation included representatives from five universities: Wayne State, Eastern Michigan, Central Michigan, University of Michigan, and Michigan State.

Key Elements

Engagement.

The leadership team approached MITRE members who had an interest in IMH-HV to join the research advisory board. Monthly conference calls were initiated to gain input from advisory board members on all aspects of the research. This branch of the partnership was represented on the leadership team by University of Michigan faculty.

Support.

The University of Michigan provided structure for the research advisory board by scheduling calls and organizing subcommittees. It also provided support for institutional review board (IRB) submission through each member’s university.

Challenges.

A major barrier was the need for IRB review by multiple entities. In addition to review by the University of Michigan Medical School, the study also needed approval from the MDHHS and the DWMHA. In order to have access to data (for coding and analysis), research advisory board members also needed to gain approval from their home IRBs. Given links between participant identifiers and potentially sensitive information, a National Institutes of Health Certificate of Confidentiality was obtained to protect participant confidentiality.

Contributions and Benefits

Research advisory board members have been active collaborators, contributing to study design, coding data, and providing evaluation expertise. They will also be involved in analyses and dissemination of study findings.

The research partners gain access to data on the unique population served by IMH-HV and have the opportunity to explore empirical questions and publish findings. Data collected will provide pilot data for a subsequent proposal for a larger, randomized controlled study to meet federal guidelines for evidence-based practice.

Conclusions

This study represents a unique and productive collaboration across multiple sectors working to evaluate a critical public health service. This community-university-state partnership has served to improve and ensure quality, to address policy demands, and to generate rich evaluation data, and it represents an effective model for multisector collaboration.