Assessing the New York City Involuntary Outpatient Treatment Program

To the Editor: What conclusions can we draw from the findings of the New York City pilot program on outpatient commitment that were reported in the March issue of Psychiatric Services (1)? Dr. Steadman and his coauthors did not state what hypothesis they set out to test. Was it that enhanced outpatient services on their own would provide equivalent clinical outcomes when compared with a combination of enhanced services and outpatient commitment? If so, the methods for an equivalence trial (2) should have been employed. Alternatively, was the hypothesis that outpatient commitment with enhanced services would produce superior clinical outcomes when compared with enhanced services alone?

Dr. Steadman's group used the approach of a comparative study, and hence the reader must presume that they were testing the latter hypothesis. We believe that this is the appropriate hypothesis to test. Concerns about potential limitations of civil liberties associated with the use of outpatient commitment require that it be associated with a superior clinical outcome.

Unfortunately, Dr. Steadman and his colleagues also neglected to identify their primary outcome measure, and the reader is again forced to speculate. Because the amount of hospitalization in the follow-up period is addressed first in the results section, it was probably the primary measure. The authors indicate that there was a "substantial" difference in hospital use between the outpatient commitment group and the control group. The former group spent a mean of 43 days in the hospital during the 11-month follow-up period, compared with 101 days for the control group.

Such a difference is both clinically and administratively relevant, but the key question is, how likely is it that such a difference occurred by chance alone? Dr. Steadman and his colleagues found that this difference did not reach statistical significance. They ask the reader to accept that "the sample was large enough to allow valid comparison of the two groups on all major outcome variables," but they confusingly add, "If we had recruited twice as many subjects for the study, this difference in hospital days would have attained the .05 level of statistical significance." This statement cannot be justified, because it assumes that the distribution of hospitalization in a larger sample would be the same as that found in the smaller sample that was actually studied.

It would have been more informative if the authors had provided a measure of the strength of the trend toward statistical significance. More important, the authors should have provided justification for the sample size. The absence of a clearly identified hypothesis and a primary outcome measure leaves the reader wondering whether calculation of an appropriate sample size was undertaken for this study.

Dr. Steadman and his colleagues must be aware of the contentious nature of the issue they are studying. We would have expected them to meticulously describe their methods and identify any deficiencies that might affect the interpretations of the findings. Unfortunately, the findings as reported in the article will serve only to obfuscate this important area.

The authors are affiliated with the University of Western Ontario in London, Ontario.