Sample for Validation of Jail Mental Health Screen

To the Editor: In their otherwise excellent article reporting on validation of the Brief Jail Mental Health Screen (BJMHS) published in the July 2005 issue, Steadman and colleagues (1) made a subtle but critical analytic error, which, when corrected, leads to different conclusions. They studied the performance of the BJMHS in screening for mental health treatment needs among more than 10,000 prisoners in four jails. From this population, they selected samples, stratified by sex and BJMHS outcome, to create groups of men and women—each group with a ratio of approximately 2:1 of individuals with negative BJMHS outcomes to individuals with positive BJMHS outcomes. These participants were interviewed with the Structured Clinical Interview for DSM-IV (SCID), which was used as the criterion standard. The authors calculated sensitivities, specificities, and other scale statistics by comparing BJMHS results with SCID results. However, they erred when, having stratified by BJMHS outcome, they changed the proportion of negative-to-positive outcomes from about 9:1 in the population to about 2:1 in the SCID sample without making a correction in the final calculations.

The authors present a 2 × 2 table for BJMHS and SCID outcomes among male prisoners that shows 38 true-positive cases, 20 false-negative cases, 117 true-negative cases, and 36 false-positive cases. According to the authors, 9.6 percent of the male population screened positive; thus 90.4 percent screened negative. Assuming a male population of 8,979, and assuming that the proportions of correct and incorrect predictions among positive and negative screened participants remains constant, it is possible to calculate population estimates for the number of true positives (N=443), false negatives (N=1,185), true negatives (N=6,932), and false positives (N=419). This yields an estimated sensitivity of 27.2 percent, in contrast to the reported sensitivity of 65.5 percent. Specificity, recalculated in a similar way, rises to 94.3 percent, and the "hit" rate rises to 82.1 percent. Because the sensitivity declines, the estimated rate of psychopathology in the male jail population as a whole becomes 18.1 percent, which is in accord with many published findings (2).

Inmates who had screened negative on the BJMHS were less likely to receive SCID interviews, which disproportionately reduced the number of false negatives and inflated the sensitivity statistic. The corrected sensitivity, which is based on a cutoff point selected by the authors under different assumptions, is unacceptable for screening purposes. A higher sensitivity can be achieved by adjusting the cutoff point down, although this adjustment may produce a false-positive rate that is too high for the BJMHS to be of practical value.

Dr. Dillon is chief of clinical affairs of the corrections mental health program for the State of Michigan and adjunct clinical assistant professor in the department of psychiatry at the University of Michigan Medical School in Ann Arbor.