Evaluating Human Resource Development Programs: A Practical Guide for Public Agencies

Massey's book covers each phase of the evaluation process in the order in which the phases ideally occur in human resource development programs: needs assessment, formative evaluation, program selection, process evaluation, and outcome evaluation. The overlap between these activities in real-world situations is also appropriately acknowledged.

The book breaks no new ground as an evaluation textbook, but it may be the first such text to be written from a human resource development perspective. The material is well organized, with each chapter sharing a common format. In each, the basics of the technique are presented—for example, the use of focus groups in needs assessment—followed by a bulleted summary of key points to remember about the technique and a list of the advantages and disadvantages of applying it in the field.

The common format makes the book a handy reference tool as well as a textbook, as students or prospective evaluators can "surf" to a topic and quickly find a brief summary of the issues germane to it. In addition, the many examples from the human resource development literature help to flesh out the abstract concepts.

Technically, the book has some notable omissions. First, the section on surveys covers mail and telephone surveys, but neglects face-to-face surveys. And it all but ignores the subject of response rates; neither the importance of a high response rate for the interpretability of the results nor ways of improving the response rate are discussed.

Second, the table of evaluation designs omits the nonequivalent control group design, perhaps the most common of the quasiexperimental designs. Instead, it is relegated to a subhead of randomized control group designs called "incomplete randomization." The problem goes beyond using nonstandard terminology, as the nonequivalent control group design is not an incomplete randomized design because it is not a randomized design at all. One could imagine other designs being called incomplete randomized designs—mixed designs, or designs in which randomization is in force at only some times or for only some groups or persons within groups—but including this design here only adds to the potential confusion.

Massey discusses interrupted time series designs briefly, but does not mention important technical issues that the student will face when actually attempting to implement one. At minimum, he could have referenced a standard text on the use of this design in evaluation, such as McLeary and Hay's book (1), so the student would have some sense of where to turn. Finally, there is no mention of the regression-discontinuity design (2), whose ability to rule out alternatives in the attribution of intervention effects is second only to that of randomized design.

Readers of Psychiatric Services should be aware that the book is primarily targeted to professionals and students in the human resource development field. Consequently, most of the substantive discussion and examples involve areas such as personnel, employee recruiting, retention, and job satisfaction. Mental health service providers, administrators, policy makers, and researchers might be better served by a more general evaluation text, such as the one by Rossi and Freeman (3), or by the numerous evaluation writings tailored to mental health services.

That said, human resource development students and professionals with little exposure to evaluation should find this book informative and useful to their work.

Dr. Orwin is a senior research scientist at Battelle Centers for Public Health Research and Evaluation in Arlington, Virginia.