As of June 30, 2002, a total of 588,106 men and 77,369 women were incarcerated in approximately 3,300 local jails across the United States (1). This represents an increase of almost 300 percent from the jail population in 1986. Moreover, 11.4 million people are booked into U.S. jails each year (2). As correctional staff struggle to keep up with this rapid influx, while maintaining a secure environment, their efforts are increasingly impacted by individuals with serious mental illnesses who are entering these jails in growing numbers. Teplin (3) and Teplin and colleagues (4) reported that approximately 6 percent of male inmates and 15 percent of female inmates who were admitted to Chicago's Cook County jail displayed acute symptoms of schizophrenia, major depression, or bipolar disorder and required treatment. When these data are extrapolated to the entire U.S. population, approximately 900,000 persons with active symptoms of serious mental disorder are being admitted to U.S. jails annually.
Jails have a substantial legal obligation to provide health and mental health care for inmates (5). Case law and statutes have not provided a clear definition of what constitutes adequate mental health care. However, the American Psychiatric Association has recommended that all jails provide at minimum mental health screening, referral, and evaluation; crisis intervention and short-term treatment (most often medication); and discharge and prerelease planning (6). In a recent national survey of 1,706 American jails, Steadman and Veysey (7) reported that 83 percent of all U.S. jails provide some form of initial screening for mental health treatment needs. However, screening procedures are highly variable. Screening may consist of anything from one or two questions about previous treatment to a detailed, structured mental status examination. One result of this variability is apparent in Teplin's (8) data from Chicago's Cook County jail, which showed that fully 63 percent of inmates who were found to have acute mental symptoms with use of an independently administered standardized clinical instrument were missed by routine screening performed by jail staff and remained untreated.
Currently, no valid, practical, standardized tools are available for brief, initial screening. A standard screen needs to be brief, because the corrections classification staff have only a limited amount of time to spend with any one inmate. It needs to incorporate explicit decision criteria, because the mental health training and experience of correctional staff is likely to be highly variable and the overall level relatively low. Correctional staff traditionally are confident in their ability to discern overtly psychotic symptoms but are considerably more uncertain about identifying less obvious, but equally serious, signs and symptoms of anxiety and depression.
A useful jail mental health screen would also exhibit a low rate of false negatives—that is, it would not miss many inmates who have a serious mental disorder, because the potential costs of not treating an inmate with a serious mental illness could be high. On the other hand, it also would not have a false-positive rate that is too high, because mental health resources in jails are scarce and requiring trained mental health staff to assess many people who do not have a serious mental illness is an inefficient use of staff's time. Because of the high prevalence of co-occurring substance use disorders among detainees with serious mental illnesses (9) and the overlapping behavioral signs and symptoms of these disorders, a brief screen will inevitably capture some individuals with substance use disorders. Thus an effective mental health screening tool would have a high degree of predictive validity, in that most of the people who are flagged by it as being "positive" should, on assessment, be found to have a treatable serious mental illness.
No available screening tool meets all of these criteria. Symptom checklists, like the Symptom Checklist-90 (10) and the Brief Symptom Inventory (11), focus on the recent self-rated experience of specific symptoms within the past week. These checklists have 90 and 53 items, respectively, and require more time to administer than desired. Major drawbacks for the use of the Brief Symptom Inventory are its cost, which is currently more than $1 per administration, and the level of training that is required for reliable use. Rating instruments like the Brief Psychiatric Rating Scale (12) and the Schedule of Affective Disorders and Schizophrenia-Change Version (13,14,15) require independent symptom ratings by a clinically trained interviewer. Although they might be useful as part of a follow-up assessment, these instruments would not be practical for use as a screen by correctional staff, for whom time is of the essence.
One instrument that has been proposed for meeting the key criteria is the Referral Decision Scale (RDS) (16). The RDS was designed to serve as a rapidly administered and easily scored screening tool for use by correctional staff in identifying inmates who are likely to have schizophrenia, bipolar disorder, or major depression. The RDS was meant to flag signs and symptoms of gross impairment associated with each of the three disorders. It consists of three subscales that incorporate 14 items from the Diagnostic Interview Schedule (DIS) that are predictive of these disorders (17). The final published version of the RDS contains three subscales of five items each (one item is used in two subscales) (16). Each of the three subscales contains a cutoff score that, if met or exceeded, should result in a referral for mental health assessment.
Teplin and Swartz (16) provided preliminary evidence of the validity of the RDS by comparing results of the RDS with those of the parent instrument, the DIS. They reported the average sensitivity of the three RDS subscales (how well they detect illness among inmates who are truly ill, as defined by the DIS) as .88, and the mean specificity (how well they detect no illness among inmates who did not have a disorder, as defined by the DIS) as .99 when compared with the DIS-generated definitions for each disorder.
Several subsequent studies have raised questions about the RDS's content and its concurrent and predictive validity. Hart and colleagues (18) examined the validity of the RDS in a sample of 790 male pretrial detainees in Vancouver, British Columbia. The RDS was administered along with two other symptom-rating scales—the Brief Psychiatric Rating Scale and the Diagnostic Profile. As a validation procedure, the full DIS was then administered separately. In this study, the RDS yielded higher prevalence rates than either of the other two scales.
Rogers and colleagues (19) raised additional questions about the validity of the RDS as a screening instrument. The RDS was administered along with the Schedule of Affective Disorders and Schizophrenia and the Personality Assessment Inventory to a sample of 108 male jail detainees who were housed in a specialty unit for inmates with serious mental disorders in Fort Worth, Texas. Rogers and colleagues found supportive evidence for the schizophrenia and depression subscales but not for the bipolar mania scale.
Finally, Veysey and associates (20) challenged the validity of the RDS as a screening instrument on other grounds. Veysey and colleagues questioned the face validity of individual items (that is, several items did not seem to be appropriate for use with incarcerated individuals) and the use of lifetime occurrence of symptoms rather than current symptoms, both of which may overestimate the current need for further mental health services in the resource-poor jail environment. Veysey and colleagues revised the RDS to produce an even briefer and practical tool for use in jails—the Brief Jail Mental Health Screen (BJMHS) (Veysey BM, Beckstead JW, Deane MW, et al, unpublished manuscript, 1999). The study reported here sought to validate the BJMHS.
Because the RDS subscales do not perform well in discriminating among schizophrenia, bipolar disorders, and major depression, the scoring approach for the BJMHS was to develop a single composite scale. Thus a positive score would indicate that an individual has recent or acute symptoms associated with one or more of these three disorders. The total number of items was reduced from the original 14 in the RDS to a smaller set of eight items by eliminating items that had questionable face validity and did not contribute statistically to the composite scale. Several items in the RDS were rephrased to provide clearer wording. Finally, the time frame employed by the RDS was changed from lifetime occurrence to occurrence within the past six months. If detainees indicated that they had such a symptom during the past six months, they were also asked whether they were experiencing that symptom currently.
Our study asked three major research questions: What is the validity of the BJMHS compared with a standard structured clinical interview? How much does the use of this screen add to the workload of classification staff, and is this level acceptable to them? What is the optimal scoring to improve the sensitivity and specificity of the new instrument?
Previous studies with the RDS used its parent measure, the Diagnostic Interview Schedule (DIS), as the standard for its validation. We thought this standard was a weak test of the screen's validity. A more compelling test would involve the use of a different structured clinical interview that would more naturally reflect the clinical assessment phase of the triage process. For that reason, we used the Structured Clinical Interview for DSM-IV (SCID) as our standard.
Participants included jail detainees admitted to one of four county jails—two in Maryland and two in New York—from May 2002 to January 2003. All participants answered questions on two screening instruments on admission to the jails. The two screening instruments were the Brief Jail Mental Health Screen (BJMHS) and the Suicide Prevention Screening Guidelines (SPSG).
The screening data were used to identify a subsample of detainees (approximately 90 from each jail) who were systematically sampled for a detailed clinical assessment conducted by a trained research interviewer using the SCID. This subsample was constructed so that there would be an adequate sample of inmates from each jail who scored positive on the BJMHS, and it was designed to comprise a large enough number of females to enable separate analysis by gender.
BJMHS. The BJMHS consists of eight items from the RDS that can be answered as yes or no. For the purposes of our study we added another item that asked whether the detainee had ever been treated in a jail or prison for emotional or mental health problems. Technically, this additional question was not part of the validation study because it altered the BJMHS. However, we added it because we were interested in whether it would improve the accuracy of the screen. In fact, it did not and was eliminated from our analyses.
The BJMHS is organized into two sections. The first section includes six items that ask about the occurrence of mental health symptoms in the past six months: believing that someone can control your mind by putting thoughts into your head or taking thoughts out of your head, feeling as though other people know your thoughts and can read your mind, having lost or gained as much as two pounds a week for several weeks without even trying, noticing that you are currently much more active than you usually are, feeling as though you have to talk or move more slowly than you usually do, or feeling as though you were useless or sinful for the past few weeks. If a detainee has an affirmative response to one of these items, a subsequent question asks whether the symptom is experienced now. The second section of the BJMHS includes two items that address whether a detainee was ever hospitalized for emotional or mental health problems and whether he or she is currently taking psychotropic medication. Mean administration time of the BJMHS was 2.45±2.0 minutes. The screen is available from the authors at http://gainscenter.samhsa.gov/html/resources/MHscreen.asp.
SPSG. We used the April 2001 version of the SPSG. The SPSG is a screening instrument designed to identify inmates who may be at high risk of suicide during the first 24 to 72 hours of incarceration (21). The instrument was developed in New York State, where its use is already a standard part of the booking process in all county jails. The SPSG consists of four sections: observations of the arresting or transporting officer, personal data, behavior and appearance, and action. Total scores on the SPSG can range from 0 to 16, with a score of eight or higher indicating a potential risk of suicide. In addition, a single positive endorsement on six particular SPSG items also indicates a potential risk of suicide. The SPSG takes approximately five to ten minutes to administer.
SCID. The SCID is a semistructured interview designed to assess the presence of selected DSM-IV axis I diagnoses (22). This instrument, which should be administered by a trained clinical interviewer or mental health professional, uses a modular format with skip patterns that allow an interviewer to move out of a given section if the diagnostic criteria in that section are not met. If the diagnostic criteria for a given diagnosis are met, that diagnosis is scored in terms of its lifetime prevalence and in terms of its presence in the past month. We administered a subset of available modules. For the purpose of the validation study, serious mental illness was defined as the presence of one or more of the following SCID diagnoses: major depressive disorder, depressive disorder not otherwise specified, bipolar disorder (I, II, and not otherwise specified), schizophrenia disorder, schizoaffective disorder, schizophreniform disorder, brief psychotic disorder, delusional disorder, and psychotic disorder not otherwise specified. The mean administration time of the SCID was 76±28.5 minutes.
Screening instruments. Correctional classification officers in all four jails participated in information sessions which provided training on administration of the BJMHS. This unstructured training, which took place in the jails, included a brief description of the research project and instructions on completing the BJMHS during the booking process.
Correctional classification officers in one of the Maryland jails and nursing staff in the other Maryland jail received training on administration of the SPSG. For the correctional classification officers, this SPSG training was incorporated into the training for the BJMHS. For the nursing staff, the SPSG training included a brief description of the research project and instructions on completing the SPSG. This training was given immediately after the correctional classification officers completed the BJMHS training. Correctional classification officers in the New York jails did not receive training on administration of the SPSG because this instrument is already a standard part of the booking process in all New York State jails.
Clinical research interviewers. Nine clinical research interviewers were formally trained on administration of the SCID by a clinically trained SCID instructor. This two-day training included a description of the research project, information on conducting interviews in a jail setting, and instructions on completing and scoring the SCID. In addition, all the clinical research interviewers conducted practice interviews on acquaintances and psychiatric patients who volunteered to participate in this aspect of the SCID training process. Reliability results, conducted with the nine interviewers and a trained SCID instructor, were very favorable; alpha=.964 when averaged across the two scored reliability videotapes.
Screening data were collected for all jail detainees who were admitted to the four jails during the eight-month data collection period. In three of the four jails correctional classification officers administered the BJMHS and the SPSG to all incoming jail detainees during the booking process. At the request of administrators of the fourth jail, correctional classification officers administered the BJMHS to all incoming jail detainees during the booking process and nursing staff administered the SPSG to these detainees immediately after the booking process.
As soon as detainees were classified by whether they would have been referred for further mental health assessment, a subject tracking program, programmed in Microsoft Access, was used to identify and generate a list of potential participants for the SCID interview. The SCID interview groups were stratified so that there would be a sufficient number of men and women. This list of potential participants was given to jail administration staff, who verified each detainee's presence in the jail and helped schedule consultation visits between the detainees and the study's trained clinical research interviewers. The clinical research interviewers, who were blind to the detainees' sampling group statuses, approached the detainees on their list of potential participants and completed SCID interviews with those who consented to participate in the study. All interviews occurred within 96 hours of a detainee's admission to the jail.
Participation in this study was voluntary. Informed consent forms were required and obtained for all SCID interview participants. Participants were informed that their decision to participate would not affect their stay in the jail. All human subjects procedures were approved by the institutional review board of the university or organization associated with each data collection site. Detainees in Maryland received $25 for their participation in the SCID interview. At the request of jail administrators in the New York facilities, detainees in New York did not receive compensation for their participation in the SCID interviews. The overall refusal rate was 31 percent.
A total of 11,438 persons were admitted to the jails during the study period. These persons were mostly pretrial detainees (7,882 detainees, or 68.9 percent) and predominately male (9,926 detainees, or 86.8 percent), and slightly more than half were African American (6,621 detainees, or 57.9 percent). The detainees had a mean±SD age of 31.6±10.4 years. All participants answered questions on the BJMHS and the SPSG.
Of the 10,330 detainees with valid BJMHS screening data 1,169 (11.3 percent) were classified as needing a referral for further mental health assessment. Twice as many women (307 women, or 22.4 percent) as men (862 men, or 9.6 percent) were classified as needing a referral.
The screening data were used to identify a subsample of 357 detainees who were administered the SCID for standardized clinical cross-validation—125 detainees (74 males and 51 females) who would have been referred for further mental health assessment on the basis of BJMHS and 232 detainees (137 males and 95 females) who would not have been referred. Among detainees in the subsample who were not classified as a referral by the BJMHS, a small number scored positive on the SPSG. Much like the total sample, the persons in the subsample were mostly pretrial detainees (269 detainees, or 75.4 percent), slightly more than half were African American (183 detainees, or 51.3 percent), and the average age was 32±11.1 years. Slightly more than half of the participants in the subsample were male (211 detainees, or 59.1 percent).
The core research questions of our study were whether the BJMHS met acceptable levels of validity when compared with the standard SCID and whether the scoring method of the BJMHS could be improved. We did this by comparing the SCID results (yes or no for presence of serious mental illness) with the predicted results from the BJMHS (yes or no for referral), performing the analysis separately for men and women.
t1 shows how well the SCID results corresponded with those of the BJMHS. For the BJMHS results, detainees were considered to be referred for further evaluation if they endorsed ever being in a hospital for emotional or mental health problems, or currently taking medication for emotional or mental health problems, or currently having at least two of the six symptoms listed in the BJMHS.
The BJMHS referred 11.3 percent of all screened detainees for follow-up assessment. Among the subgroup that was also given the SCID, 73.5 percent of the men were correctly classified, and there was a false-negative rate of 14.6 percent (20 cases). Among women who were given the SCID, 61.6 percent were correctly classified; however, there was a false-negative rate of 34.7 percent (33 cases).
We next examined the 20 false negatives among the men and 33 false negatives among the women to see whether there were any patterns of their characteristics or symptoms. Two of the 20 men and six of the 33 women were missed because the screen focused solely on current symptoms as opposed to symptoms in the past six months. Another five of the 20 men and four of the 33 women would have been referred on the basis of data from the SPSG. These individuals endorsed symptoms that were not present on the BJMHS but were indicative of needing further mental health evaluation. The remaining cases (13 men and 23 women) would not have been referred by either the BJMHS or the SPSG. The most frequent SCID diagnosis for the missed cases was major depression (13 males and 23 females).
There was an issue with the consistent reporting of symptoms. All the questions asked on the BJMHS were repeated during the SCID interview. They were either part of the SCID or added for this research study. In all but seven of the false-negative cases, the inmates reported different information to the SCID interviewer than they had to the correctional officer. Had they reported the SCID information on the BJMHS, they would have been referred for further mental health assessment and only one male case and six female cases would have been missed. This missed male, who would have been referred on the basis of the additional information provided by the SPSG, was listed as guarded and difficult to interview by the SCID interviewer. The missed females all reported at least one symptom in the SCID interviews but did not report more than one current symptom in order to be classified as a referral by the BJMHS.
The final results for the BJMHS presented in t1 for men balance accuracy with practicality. The screen lists eight questions, takes just under three minutes to administer, requires very limited training of correction officers, and attains 73.5 percent overall accuracy among men.
Our final area of inquiry regarding the BJMHS was to incorporate feedback from correctional officers in our evaluation of the screen. Debriefing interviews were conducted in New York with six officers at the Albany county jail and four officers at the Rensselaer county jail after the validation study was complete. The officers, who had administered the BJMHS for at least six months, were asked about the length and content of the screen, the training, and problems that arose while administering the screen. Overall reaction to the screen was neutral to positive. The officers reported that the time to administer the BJMHS was three to five minutes. Although time was not a concern, some of the officers were slightly reluctant to add another form to the booking process. However, these officers acknowledged the value of the screen and found that the form reminded them to look for flags and to focus on symptoms.
The officers reported that the last two questions were straightforward (whether a detainee was ever hospitalized for emotional or mental health problems and whether he or she was currently taking psychotropic medication); but they expressed some concern about the first six questions about symptoms. Some indicated that the questions about detainee's symptoms were more difficult to ask and resulted in inmates' asking questions, perceiving questions as "weird," or refusing to cooperate and respond. This observation fits with the earlier observation that respondents changed their responses between the screen and the SCID. It may also partially explain the high rate of false negatives among the women in the sample.
Some correctional officers requested more training on establishing trust and eliciting information. Although no separate standardized training was conducted with all officers who used the screen, informational sessions were conducted with the supervising officers and many of the line staff. Unfortunately, this information did not always get passed to other officers. Two officers had not even seen the instructions on the back of the screening form. On the basis of these feedback sessions, additional training would be recommended to clarify the purpose of the screen and to provide some interviewing and probing techniques. In some cases, nurses at the jails administered the screens as part of the health screening or when inmates were unable to respond during initial booking because of intoxication. Perhaps some consideration might be given to having nursing staff administer the screen to all inmates or to those who are reluctant to respond to the correctional officers, assuming that all nursing staff members are as well trained as those at our study sites.
On the basis of these data we believe that the BJMHS is a powerful tool for screening men booked into U.S. jails. The screen is simple for intake booking officers to use because it requires only modest training. The BJMHS is 73.5 percent accurate for men. Is this enough? Enough is obviously a relative term. Referring 11.3 percent of all inmates for subsequent mental health assessment must be factored into existing jail resources and processes. On the basis of correction officer feedback, the percentage of male detainees with current acute symptoms who were missed by the BJMHS (14.6 percent) could be reduced dramatically by giving the correctional staff additional training to effectively administer the BJMHS. It might also be possible to use a computer-assisted version of the screening questions, which might reduce symptom underreporting to the correctional officers. Of course, any successful use of the BJMHS depends on whether the screener and the detainees speak the same language.
Approximately 10,000,000 men are booked into U.S. jails annually, and 63 percent of those with mental illness are currently not identified (8). Thus we believe that the use of the BJMHS would be a substantial improvement that can be absorbed into the resource-strapped jail operations. Early identification of mental illness can facilitate critical treatment interventions and mitigate some of the disruptive behavior associated with detainees with mental illnesses. However, early identification is only a first-step response. Opportunities for identification and referral of inmates with mental disorders after booking are critical. It is by subsequent observation in housing units, cafeterias, recreation yards, and clinics and input from community-based clinicians and family members that the inmates missed by the BJMHS must be identified and treated. Nonetheless, the BJMHS represents a thoroughly researched initial screening instrument that can be used among all adult male detainees.
The BJMHS was not nearly as effective in correctly identifying women. The screen missed 34.7 percent of women with current symptoms, and 45.1 percent of women who were identified for referral did not have a current serious diagnosis. Even though the BJMHS correctly identified serious mental illness less frequently among women detainees than among men detainees, the BJMHS was still more effective at detecting serious mental illness than other screens used in jails. For example, our study found that the BJMHS had a 34.7 percent false-negative rate among women detainees, but Teplin and colleagues (8) found that a screen used in Cook County jail had an overall false-negative rate of 63 percent. For the BJMHS to be able to correctly identify 45.9 percent (28 of 61 women) of the true positives among the women is a modest improvement over current practices, although it still leaves much wanting. The lower accuracy of the BJMHS among women may be due to the fact that the BJMHS does not measure symptoms of anxiety that are associated with the high incidence of posttraumatic stress disorders experienced by women detainees (23). We recommend that subsequent modifications of the BJMHS for women add questions that capture anxiety symptoms. It may also be that women are less likely to disclose symptoms to correctional officers, who are most often male, on intake. Whatever the explanation, research is needed to create an appropriate jail intake screen for women. In the meantime, U.S. jails can consider introducing the BJMHS as a cost-effective tool for intake screening for male detainees.
The authors thank Roumen Vesselinov, Ph.D., for many of the analyses for this project and Steve Banks, Ph.D., for statistical advice. They also thank the administrators of the jails for participating: Edward Szostak, Robert R. Loveridge, Arthur Wallenstein, and Barry Stanton. The authors also acknowledge the contributions of the Maryland data coordinators: Ericka Oliver, B.A., and Pamela Phillips-Mann. This project was supported by grant 2001-IJ-CX-0030 from the Office of Justice Programs of the National Institute of Justice.
Dr. Steadman, Ms. Agnese, and Ms. Robbins are affiliated with Policy Research Associates, Inc., 345 Delaware Avenue, Delmar, New York 12054 (e-mail, hsteadman@ prainc.com). Dr. Scott is with CSR, Inc., in Arlington, Virginia. Dr. Osher is with the Center for Behavioral Health, Justice, and Public Policy in Baltimore.
Validation of the Brief Jail Mental Health Screen (BJMHS) with the Structured Clinical Interview for DSM-IV (SCID) among 357 detainees, by gender