Mental Health Outcome Evaluation
The discussion of outcome and outcomes evaluation is ubiquitous in health care. Interest in mental health outcomes research in particular has spawned numerous recent books, and Mental Health Outcome Evaluation by David Speer is one of the latest.
Speer is best known for his contributions in evaluating programs for the elderly, but he has also written on the assessment of clinical change, a pivotal analytic issue in outcome research. His unique combination of practical evaluation experience and analytic sophistication, along with a clear and engaging writing style, results in a primer that is currently one of the best available on the topic of outcome evaluation. The book is relatively free of research jargon from beginning to end, making it accessible to its intended audience of practitioners, administrators, and policy makers.
The requisite initial chapter on the rationale for outcome evaluation is followed by a chapter outlining the author's main, and most unusual, thesis: there is value in "practical outcome evaluation" that is scientifically rigorous yet unapologetic about the lack of control inherent in quasiexperimental, field-based research. In making this argument, Speer reorients the purpose of outcome evaluation from producing new knowledge aimed primarily at the scientific community to influencing policy, funding, and reimbursement decisions.
Despite this orientation, Speer does not provide carte blanche to conduct sloppy, self-serving research. On the contrary, while accepting the fact that experimental designs are not always feasible (although I would assert that they might be more feasible than some think), he insists that the evaluator is still obligated to maintain the highest possible level of rigor. He reviews classic quasiexperimental designs that exemplify most field-based outcome evaluations and offers state-of-the-art strategies for limiting the threats to internal validity associated with these designs. These strategies include developing reasonable comparison groups and analytic approaches to assessing improvement. A later chapter discusses "potential booby traps and landmines" in conducting outcome research, again repeating the theme that outcome evaluations need to retain rigor, especially in the presentation and interpretation of results.
Additional chapters address the practical issues surrounding the selection of outcome domains and instruments. Speer even offers examples of the types of instruments that might be used with different client populations.
Concerns about rigor in "practical outcome evaluation" remain warranted, even given the strategies offered by Speer. Such concerns are especially pertinent in a context in which funding is contingent on positive outcome results. Nonetheless, Mental Health Outcome Evaluation, albeit brief, offers the well-intentioned outcome evaluator motivation and suggestions for conducting outcome research with limited resources as well as some strategies to enhance the quality of the work.
Dr. Salzer is research assistant professor in the Center for Mental Health Policy and Services Research at the University of Pennsylvania in Philadelphia.
