Currently, no valid, practical, standardized tools are available for brief, initial screening. A standard screen needs to be brief, because the corrections classification staff have only a limited amount of time to spend with any one inmate. It needs to incorporate explicit decision criteria, because the mental health training and experience of correctional staff is likely to be highly variable and the overall level relatively low. Correctional staff traditionally are confident in their ability to discern overtly psychotic symptoms but are considerably more uncertain about identifying less obvious, but equally serious, signs and symptoms of anxiety and depression.
Because the RDS subscales do not perform well in discriminating among schizophrenia, bipolar disorders, and major depression, the scoring approach for the BJMHS was to develop a single composite scale. Thus a positive score would indicate that an individual has recent or acute symptoms associated with one or more of these three disorders. The total number of items was reduced from the original 14 in the RDS to a smaller set of eight items by eliminating items that had questionable face validity and did not contribute statistically to the composite scale. Several items in the RDS were rephrased to provide clearer wording. Finally, the time frame employed by the RDS was changed from lifetime occurrence to occurrence within the past six months. If detainees indicated that they had such a symptom during the past six months, they were also asked whether they were experiencing that symptom currently.
Our study asked three major research questions: What is the validity of the BJMHS compared with a standard structured clinical interview? How much does the use of this screen add to the workload of classification staff, and is this level acceptable to them? What is the optimal scoring to improve the sensitivity and specificity of the new instrument?
Previous studies with the RDS used its parent measure, the Diagnostic Interview Schedule (DIS), as the standard for its validation. We thought this standard was a weak test of the screen's validity. A more compelling test would involve the use of a different structured clinical interview that would more naturally reflect the clinical assessment phase of the triage process. For that reason, we used the Structured Clinical Interview for DSM-IV (SCID) as our standard.
Participants included jail detainees admitted to one of four county jails—two in Maryland and two in New York—from May 2002 to January 2003. All participants answered questions on two screening instruments on admission to the jails. The two screening instruments were the Brief Jail Mental Health Screen (BJMHS) and the Suicide Prevention Screening Guidelines (SPSG).
The screening data were used to identify a subsample of detainees (approximately 90 from each jail) who were systematically sampled for a detailed clinical assessment conducted by a trained research interviewer using the SCID. This subsample was constructed so that there would be an adequate sample of inmates from each jail who scored positive on the BJMHS, and it was designed to comprise a large enough number of females to enable separate analysis by gender.
BJMHS. The BJMHS consists of eight items from the RDS that can be answered as yes or no. For the purposes of our study we added another item that asked whether the detainee had ever been treated in a jail or prison for emotional or mental health problems. Technically, this additional question was not part of the validation study because it altered the BJMHS. However, we added it because we were interested in whether it would improve the accuracy of the screen. In fact, it did not and was eliminated from our analyses.
The BJMHS is organized into two sections. The first section includes six items that ask about the occurrence of mental health symptoms in the past six months: believing that someone can control your mind by putting thoughts into your head or taking thoughts out of your head, feeling as though other people know your thoughts and can read your mind, having lost or gained as much as two pounds a week for several weeks without even trying, noticing that you are currently much more active than you usually are, feeling as though you have to talk or move more slowly than you usually do, or feeling as though you were useless or sinful for the past few weeks. If a detainee has an affirmative response to one of these items, a subsequent question asks whether the symptom is experienced now. The second section of the BJMHS includes two items that address whether a detainee was ever hospitalized for emotional or mental health problems and whether he or she is currently taking psychotropic medication. Mean administration time of the BJMHS was 2.45±2.0 minutes. The screen is available from the authors at http://gainscenter.samhsa.gov/html/resources/MHscreen.asp.
SPSG. We used the April 2001 version of the SPSG. The SPSG is a screening instrument designed to identify inmates who may be at high risk of suicide during the first 24 to 72 hours of incarceration (
+21). The instrument was developed in New York State, where its use is already a standard part of the booking process in all county jails. The SPSG consists of four sections: observations of the arresting or transporting officer, personal data, behavior and appearance, and action. Total scores on the SPSG can range from 0 to 16, with a score of eight or higher indicating a potential risk of suicide. In addition, a single positive endorsement on six particular SPSG items also indicates a potential risk of suicide. The SPSG takes approximately five to ten minutes to administer.
SCID. The SCID is a semistructured interview designed to assess the presence of selected
DSM-IV axis I diagnoses (
+22). This instrument, which should be administered by a trained clinical interviewer or mental health professional, uses a modular format with skip patterns that allow an interviewer to move out of a given section if the diagnostic criteria in that section are not met. If the diagnostic criteria for a given diagnosis are met, that diagnosis is scored in terms of its lifetime prevalence and in terms of its presence in the past month. We administered a subset of available modules. For the purpose of the validation study, serious mental illness was defined as the presence of one or more of the following SCID diagnoses: major depressive disorder, depressive disorder not otherwise specified, bipolar disorder (I, II, and not otherwise specified), schizophrenia disorder, schizoaffective disorder, schizophreniform disorder, brief psychotic disorder, delusional disorder, and psychotic disorder not otherwise specified. The mean administration time of the SCID was 76±28.5 minutes.
Screening instruments. Correctional classification officers in all four jails participated in information sessions which provided training on administration of the BJMHS. This unstructured training, which took place in the jails, included a brief description of the research project and instructions on completing the BJMHS during the booking process.
Correctional classification officers in one of the Maryland jails and nursing staff in the other Maryland jail received training on administration of the SPSG. For the correctional classification officers, this SPSG training was incorporated into the training for the BJMHS. For the nursing staff, the SPSG training included a brief description of the research project and instructions on completing the SPSG. This training was given immediately after the correctional classification officers completed the BJMHS training. Correctional classification officers in the New York jails did not receive training on administration of the SPSG because this instrument is already a standard part of the booking process in all New York State jails.
Clinical research interviewers. Nine clinical research interviewers were formally trained on administration of the SCID by a clinically trained SCID instructor. This two-day training included a description of the research project, information on conducting interviews in a jail setting, and instructions on completing and scoring the SCID. In addition, all the clinical research interviewers conducted practice interviews on acquaintances and psychiatric patients who volunteered to participate in this aspect of the SCID training process. Reliability results, conducted with the nine interviewers and a trained SCID instructor, were very favorable; alpha=.964 when averaged across the two scored reliability videotapes.
Screening data were collected for all jail detainees who were admitted to the four jails during the eight-month data collection period. In three of the four jails correctional classification officers administered the BJMHS and the SPSG to all incoming jail detainees during the booking process. At the request of administrators of the fourth jail, correctional classification officers administered the BJMHS to all incoming jail detainees during the booking process and nursing staff administered the SPSG to these detainees immediately after the booking process.
As soon as detainees were classified by whether they would have been referred for further mental health assessment, a subject tracking program, programmed in Microsoft Access, was used to identify and generate a list of potential participants for the SCID interview. The SCID interview groups were stratified so that there would be a sufficient number of men and women. This list of potential participants was given to jail administration staff, who verified each detainee's presence in the jail and helped schedule consultation visits between the detainees and the study's trained clinical research interviewers. The clinical research interviewers, who were blind to the detainees' sampling group statuses, approached the detainees on their list of potential participants and completed SCID interviews with those who consented to participate in the study. All interviews occurred within 96 hours of a detainee's admission to the jail.
Participation in this study was voluntary. Informed consent forms were required and obtained for all SCID interview participants. Participants were informed that their decision to participate would not affect their stay in the jail. All human subjects procedures were approved by the institutional review board of the university or organization associated with each data collection site. Detainees in Maryland received $25 for their participation in the SCID interview. At the request of jail administrators in the New York facilities, detainees in New York did not receive compensation for their participation in the SCID interviews. The overall refusal rate was 31 percent.
A total of 11,438 persons were admitted to the jails during the study period. These persons were mostly pretrial detainees (7,882 detainees, or 68.9 percent) and predominately male (9,926 detainees, or 86.8 percent), and slightly more than half were African American (6,621 detainees, or 57.9 percent). The detainees had a mean±SD age of 31.6±10.4 years. All participants answered questions on the BJMHS and the SPSG.
Of the 10,330 detainees with valid BJMHS screening data 1,169 (11.3 percent) were classified as needing a referral for further mental health assessment. Twice as many women (307 women, or 22.4 percent) as men (862 men, or 9.6 percent) were classified as needing a referral.
The screening data were used to identify a subsample of 357 detainees who were administered the SCID for standardized clinical cross-validation—125 detainees (74 males and 51 females) who would have been referred for further mental health assessment on the basis of BJMHS and 232 detainees (137 males and 95 females) who would not have been referred. Among detainees in the subsample who were not classified as a referral by the BJMHS, a small number scored positive on the SPSG. Much like the total sample, the persons in the subsample were mostly pretrial detainees (269 detainees, or 75.4 percent), slightly more than half were African American (183 detainees, or 51.3 percent), and the average age was 32±11.1 years. Slightly more than half of the participants in the subsample were male (211 detainees, or 59.1 percent).
The core research questions of our study were whether the BJMHS met acceptable levels of validity when compared with the standard SCID and whether the scoring method of the BJMHS could be improved. We did this by comparing the SCID results (yes or no for presence of serious mental illness) with the predicted results from the BJMHS (yes or no for referral), performing the analysis separately for men and women.
+Table 1 shows how well the SCID results corresponded with those of the BJMHS. For the BJMHS results, detainees were considered to be referred for further evaluation if they endorsed ever being in a hospital for emotional or mental health problems, or currently taking medication for emotional or mental health problems, or currently having at least two of the six symptoms listed in the BJMHS.
The BJMHS referred 11.3 percent of all screened detainees for follow-up assessment. Among the subgroup that was also given the SCID, 73.5 percent of the men were correctly classified, and there was a false-negative rate of 14.6 percent (20 cases). Among women who were given the SCID, 61.6 percent were correctly classified; however, there was a false-negative rate of 34.7 percent (33 cases).
We next examined the 20 false negatives among the men and 33 false negatives among the women to see whether there were any patterns of their characteristics or symptoms. Two of the 20 men and six of the 33 women were missed because the screen focused solely on current symptoms as opposed to symptoms in the past six months. Another five of the 20 men and four of the 33 women would have been referred on the basis of data from the SPSG. These individuals endorsed symptoms that were not present on the BJMHS but were indicative of needing further mental health evaluation. The remaining cases (13 men and 23 women) would not have been referred by either the BJMHS or the SPSG. The most frequent SCID diagnosis for the missed cases was major depression (13 males and 23 females).
There was an issue with the consistent reporting of symptoms. All the questions asked on the BJMHS were repeated during the SCID interview. They were either part of the SCID or added for this research study. In all but seven of the false-negative cases, the inmates reported different information to the SCID interviewer than they had to the correctional officer. Had they reported the SCID information on the BJMHS, they would have been referred for further mental health assessment and only one male case and six female cases would have been missed. This missed male, who would have been referred on the basis of the additional information provided by the SPSG, was listed as guarded and difficult to interview by the SCID interviewer. The missed females all reported at least one symptom in the SCID interviews but did not report more than one current symptom in order to be classified as a referral by the BJMHS.
The final results for the BJMHS presented in
+Table 1 for men balance accuracy with practicality. The screen lists eight questions, takes just under three minutes to administer, requires very limited training of correction officers, and attains 73.5 percent overall accuracy among men.
Our final area of inquiry regarding the BJMHS was to incorporate feedback from correctional officers in our evaluation of the screen. Debriefing interviews were conducted in New York with six officers at the Albany county jail and four officers at the Rensselaer county jail after the validation study was complete. The officers, who had administered the BJMHS for at least six months, were asked about the length and content of the screen, the training, and problems that arose while administering the screen. Overall reaction to the screen was neutral to positive. The officers reported that the time to administer the BJMHS was three to five minutes. Although time was not a concern, some of the officers were slightly reluctant to add another form to the booking process. However, these officers acknowledged the value of the screen and found that the form reminded them to look for flags and to focus on symptoms.
The officers reported that the last two questions were straightforward (whether a detainee was ever hospitalized for emotional or mental health problems and whether he or she was currently taking psychotropic medication); but they expressed some concern about the first six questions about symptoms. Some indicated that the questions about detainee's symptoms were more difficult to ask and resulted in inmates' asking questions, perceiving questions as "weird," or refusing to cooperate and respond. This observation fits with the earlier observation that respondents changed their responses between the screen and the SCID. It may also partially explain the high rate of false negatives among the women in the sample.
Some correctional officers requested more training on establishing trust and eliciting information. Although no separate standardized training was conducted with all officers who used the screen, informational sessions were conducted with the supervising officers and many of the line staff. Unfortunately, this information did not always get passed to other officers. Two officers had not even seen the instructions on the back of the screening form. On the basis of these feedback sessions, additional training would be recommended to clarify the purpose of the screen and to provide some interviewing and probing techniques. In some cases, nurses at the jails administered the screens as part of the health screening or when inmates were unable to respond during initial booking because of intoxication. Perhaps some consideration might be given to having nursing staff administer the screen to all inmates or to those who are reluctant to respond to the correctional officers, assuming that all nursing staff members are as well trained as those at our study sites.
On the basis of these data we believe that the BJMHS is a powerful tool for screening men booked into U.S. jails. The screen is simple for intake booking officers to use because it requires only modest training. The BJMHS is 73.5 percent accurate for men. Is this enough? Enough is obviously a relative term. Referring 11.3 percent of all inmates for subsequent mental health assessment must be factored into existing jail resources and processes. On the basis of correction officer feedback, the percentage of male detainees with current acute symptoms who were missed by the BJMHS (14.6 percent) could be reduced dramatically by giving the correctional staff additional training to effectively administer the BJMHS. It might also be possible to use a computer-assisted version of the screening questions, which might reduce symptom underreporting to the correctional officers. Of course, any successful use of the BJMHS depends on whether the screener and the detainees speak the same language.
Approximately 10,000,000 men are booked into U.S. jails annually, and 63 percent of those with mental illness are currently not identified (
+8). Thus we believe that the use of the BJMHS would be a substantial improvement that can be absorbed into the resource-strapped jail operations. Early identification of mental illness can facilitate critical treatment interventions and mitigate some of the disruptive behavior associated with detainees with mental illnesses. However, early identification is only a first-step response. Opportunities for identification and referral of inmates with mental disorders after booking are critical. It is by subsequent observation in housing units, cafeterias, recreation yards, and clinics and input from community-based clinicians and family members that the inmates missed by the BJMHS must be identified and treated. Nonetheless, the BJMHS represents a thoroughly researched initial screening instrument that can be used among all adult male detainees.
The BJMHS was not nearly as effective in correctly identifying women. The screen missed 34.7 percent of women with current symptoms, and 45.1 percent of women who were identified for referral did not have a current serious diagnosis. Even though the BJMHS correctly identified serious mental illness less frequently among women detainees than among men detainees, the BJMHS was still more effective at detecting serious mental illness than other screens used in jails. For example, our study found that the BJMHS had a 34.7 percent false-negative rate among women detainees, but Teplin and colleagues (
+8) found that a screen used in Cook County jail had an overall false-negative rate of 63 percent. For the BJMHS to be able to correctly identify 45.9 percent (28 of 61 women) of the true positives among the women is a modest improvement over current practices, although it still leaves much wanting. The lower accuracy of the BJMHS among women may be due to the fact that the BJMHS does not measure symptoms of anxiety that are associated with the high incidence of posttraumatic stress disorders experienced by women detainees (
+23). We recommend that subsequent modifications of the BJMHS for women add questions that capture anxiety symptoms. It may also be that women are less likely to disclose symptoms to correctional officers, who are most often male, on intake. Whatever the explanation, research is needed to create an appropriate jail intake screen for women. In the meantime, U.S. jails can consider introducing the BJMHS as a cost-effective tool for intake screening for male detainees.
The authors thank Roumen Vesselinov, Ph.D., for many of the analyses for this project and Steve Banks, Ph.D., for statistical advice. They also thank the administrators of the jails for participating: Edward Szostak, Robert R. Loveridge, Arthur Wallenstein, and Barry Stanton. The authors also acknowledge the contributions of the Maryland data coordinators: Ericka Oliver, B.A., and Pamela Phillips-Mann. This project was supported by grant 2001-IJ-CX-0030 from the Office of Justice Programs of the National Institute of Justice.
Dr. Steadman, Ms. Agnese, and Ms. Robbins are affiliated with Policy Research Associates, Inc., 345 Delaware Avenue, Delmar, New York 12054 (e-mail, hsteadman@ prainc.com). Dr. Scott is with CSR, Inc., in Arlington, Virginia. Dr. Osher is with the Center for Behavioral Health, Justice, and Public Policy in Baltimore.