Generalizability in the Family-to-Family Education Program Randomized Waitlist-Control Trial

Objective

Randomized controlled trials (RCTs) may have limited generalizability for the community when a high proportion of individuals refuse randomization or otherwise do not participate—a not uncommon phenomenon. A randomized waitlist-control trial of the Family-to-Family (FTF) education program, a 12-week course offered by the National Alliance on Mental Illness for family members of adults with mental illness, was previously reported. This study assessed whether the RCT-derived estimates of effectiveness of FTF were generalizable to individuals who participated in FTF but declined participation in the RCT.

Methods

Propensity score matching was used to create five quintiles, each containing scores for individuals in FTF or waitlist conditions and for decliners; scores were matched on multiple baseline characteristics (N=442) within each quintile. Effectiveness estimates, with standard errors, were derived for the decliner population on the basis of effectiveness estimates derived from participants in the RCT; estimates were weighted to the baseline distribution of quintiles for the decliners.

Results

For each outcome, estimates of the effect sizes observed in the RCT were very similar to the effect sizes observed for the decliner population; confidence intervals also had a high degree of overlap.

Conclusions

This study suggests that the benefits of FTF observed in the RCT are generalizable to the group of individuals who declined RCT participation, providing further evidence of FTF’s effectiveness. Propensity score matching was a useful statistical tool for addressing selection bias resulting from high rates of nonconsent in randomized waitlist-control trials.