Enhancing Generalizability: Stepping Up to the Plate

To the Editor: An article by Braslow and colleagues (1) in the October issue of Psychiatric Services pointed out notable shortcomings in the generalizability of findings from most studies of mental health treatments that were published in four leading psychiatry and psychology journals over a 15-year period. Omitted from the article were studies from Psychiatric Services—the very journal chosen by the authors in which to publish their findings. I was left wondering what the findings would have been had their review included this journal.

Braslow and colleagues spoke of the need for clinical science "to shed light not only on an intervention's efficacy but also on how well efficacious interventions actually work in diverse clinical settings, provider and patient populations, and practice circumstances." I think of that call to action as part of the mission of Psychiatric Services, and thus I would hope that a comparable review of articles published in this journal would yield a better showing than did these authors' review of outcome studies from the American Journal of Psychiatry, Archives of General Psychiatry, Journal of Consulting and Clinical Psychology, and Journal of Abnormal Psychology. This is testable: how well do we who publish in, or review for, this journal monitor ourselves in these domains?

Part of ensuring generalizability has to do with study design and execution, and part has to do with allocating some of the scarce text space to describing the demographic and organizational characteristics that Braslow and colleagues found often went unreported. One can pick at the methods used for their ratings. For example, some studies appropriately exclude one gender or do not collect information on ethnicity; the reliability of the individual ratings appears largely to have gone unassessed; raters may not have been blind to the hypotheses being addressed. However, those are concerns that should not distract from their message. The central point continues to be that much of the research on mental health outcomes after particular interventions continues to be reported in ways that sharply limit its usefulness to those who work in routine practice settings.

Over the years Psychiatric Services has increased its concerns about internal validity and causal inference, which have been the hallmarks of the four journals included in the review. I appreciate Psychiatric Services' emphasis on policy relevance and information to help mental health service providers and administrators make decisions about what works, for whom, and under what circumstances and how to get such services implemented and sustained. The articles in the October issue were good cases in point.

Dr. Essock is affiliated with the department of psychiatry at Mount Sinai School of Medicine in New York City and the Bronx Veterans Affairs Mental Illness Research, Education, and Clinical Center.