The authors are affiliated with the Department of Psychiatry, Columbia University, 600 West 168th Street, 5th Floor, New York, NY 10043 (e-mail: bms12@columbia.edu). Data in this study were presented in part in a poster presentation at the annual meeting of the American Psychopathological Association, March 3 to 5, 2005, New York City.

Up to half of patients presenting to psychiatric emergency departments also abuse substances, with many arriving intoxicated (1). For patients experiencing first-episode psychosis, rates of substance-related disorders range from 17 percent to 37 percent (2,3,4). When an individual presents with psychotic symptoms in the context of recent substance use, it is often unclear whether the psychosis is primary and should be considered a first psychotic break or whether it is a direct consequence of the substance use. Although DSM-IV provides guidelines for differentiating between primary psychotic disorders and substance-induced psychotic disorders on the basis of the temporal relationship between the psychotic symptoms and substance use, accurate diagnosis in the emergency department may still be problematic, especially given the limited historical information available and the fact that emergency department clinicians often neglect to inquire about substance use (5).

Although misdiagnosis of psychosis in terms of being primary or substance induced may not have immediate consequences—that is, the agitated patient may need to be calmed regardless of the cause—an incorrect diagnosis of a primary psychotic disorder when the symptoms are substance induced is of particular concern because of its long-term implications. These include the associated stigma, the tendency for the diagnosis to be perpetuated in future encounters with mental health professionals (6,7,8,9), unnecessary inpatient hospitalization (10,11,12), and inappropriate treatment with antipsychotic medication, which has the potential for serious side effects, such as diabetes, tardive dyskinesia, and neuroleptic malignant syndrome (13,14,15,16). Individuals with a misdiagnosis often are not provided with proper follow-up substance abuse treatment (17). In addition, failure to detect substance use problems among patients experiencing a first psychotic episode can have negative clinical outcomes, such as rehospitalizations (18), which may be reduced or prevented with appropriate substance abuse treatment (19).

Therefore, this study had two primary aims: to examine the accuracy of emergency department diagnoses of primary versus substance-induced psychosis, with a best-estimate longitudinal diagnosis (BELD) as a "gold standard," and to examine the long-term implications of the diagnoses in terms of subsequent inpatient hospital admission, antipsychotic medication use, and outpatient follow-up treatment.

The study methods are presented in detail elsewhere (20) and are briefly described here. Patients diagnosed as having early-phase psychosis were recruited from five upper-Manhattan psychiatric emergency departments. Patients spoke English or Spanish, were aged 17 to 45, had at least one psychotic symptom assessed during administration of the research protocol, and had used alcohol or drugs within the previous 30 days. Patients were excluded if their first hospitalization for psychosis occurred more than six months before the index admission or if they had experienced two or more years of continuous psychotic symptoms in the absence of treatment. Each patient gave written informed consent before participation in the study, which took place in July 1998 through December 2002. The institutional review boards of the New York State Psychiatric Institute of Columbia University Medical Center and the other institutions from which study participants were recruited approved the research protocol.

Study participants were interviewed at baseline, six months, and 12 months with the Psychiatric Research Interview for Substance and Mental Disorder (PRISM), a semistructured research interview designed to improve reliability and validity of psychiatric diagnosis of heavy users of alcohol or drugs (21,22,23), as well as with the Positive and Negative Syndrome Scale (PANSS), an instrument that measures psychotic symptomatology (24). Urine toxicology screens were conducted at all time points. Master's-level clinicians and one physician conducted the interviews, and a clinical research supervisor working under the direction of one of the authors (DSH) reviewed the interviews to ensure thoroughness and consistency. Diagnostic narratives were written that summarized the PRISM and PANSS ratings, urine toxicology results, a family member interview (if completed), emergency department records, and inpatient hospital records, including discharge diagnoses, any medications taken, and outpatient follow-up referral information.

BELDs were determined with a method based on Spitzer's (25) longitudinal, expert, all data (LEAD) standard, which has been used as a best-estimate diagnosis method in several studies (26,27,28). To create LEAD diagnoses, expert diagnosticians examine longitudinal diagnostic information from various sources (patients, informants, and so on). For this study, the second author, Michael B. First, M.D., designed a diagnostic decision tree based on DSM-IV criteria that provided a means for distinguishing between primary psychosis and substance-induced psychosis. Diagnostic decisions were based on information gathered over 12 months, which yielded a longitudinal diagnosis—the BELD. Expert diagnosticians reviewed the baseline and six- and 12-month diagnostic narratives as well as timelines detailing weekly symptom severity and substance use. Neither the patients nor the actual hospital charts were available to the BELD reviewers. They could, however, review the complete PRISM interview booklet and confer with the PRISM interviewers if questions remained.

Psychiatry residents and fellows trained by Dr. First served as the expert diagnosticians. To establish the diagnosticians' ability to make correct diagnoses, Dr. First reviewed practice cases in which they used the decision tree. The trained diagnosticians recorded yes-no responses to the decision points in the decision tree and followed the appropriate branches until they arrived at one of the BELDs. Their assessments were randomly reviewed throughout the study period by Dr. First to ensure accuracy.

Of the 400 patients with psychotic symptoms at baseline, 302 (76 percent) were interviewed at all three time points and thus were eligible to receive a BELD. There were no significant demographic or clinical differences between those with completed six- and 12-month follow-up assessments and those not interviewed at all three time points.

For the purposes of this study, BELDs were categorized as follows: substance-induced psychotic disorder, indicating patients who had only substance-induced episodes or whose initial episode was substance induced but who might have had subsequent primary psychotic episodes during the follow-up period; primary psychotic disorder, indicating patients who had only primary psychotic episodes or whose initial episode was primary but who might have had a subsequent superimposed substance-induced episode during the follow-up period; and no psychotic disorder, indicating patients whose baseline psychotic symptoms, on review, did not meet criteria for a diagnosis of psychotic disorder.

Information regarding emergency department discharge diagnoses and medications and outpatient follow-up plans was taken from the PRISM diagnostic narratives. Discharge diagnoses were categorized as primary psychotic disorder, indicating patients with diagnoses of a specific axis I primary psychotic disorder, mood disorder with psychotic features, or primary psychosis not otherwise specified; substance-induced psychotic disorder, indicating patients with diagnoses of a substance-induced psychotic episode or substance-induced mood disorder with psychotic features; and indeterminate, indicating patients with diagnoses that included rule-out diagnoses alongside the listed diagnoses, such as schizophreniform rule-out, cocaine-induced psychosis. Any discharge diagnosis that was questionable or ambiguous was coded as indeterminate to avoid misinterpreting the intent of the emergency department physician.

Medications prescribed at discharge were divided into three categories: no medication, any antipsychotic medication, and other psychotropic medication (mood stabilizers, for example). Outpatient follow-up plans included four types: none, mental health treatment only, substance abuse treatment services or treatment for co-occurring disorders, or nonspecific outpatient referral.

Descriptive statistics were determined for selected covariates. Kappa values were calculated to determine concordance between the emergency department diagnoses and the BELDs. Chi square analyses with a criterion of p<.05 and two-tailed tests were used to determine the statistical significance of the difference between the variables, for example, whether or not patients were hospitalized.

Demographic characteristics and clinical variables

The mean±SD age of the 302 patients was 28.50±8.27 years. Other demographic characteristics are shown in Table 1. The substance use of most patients was severe enough to meet DSM-IV criteria for substance abuse or dependence (Table 1).

Emergency department diagnoses

Of the total sample, 223 patients (74 percent) had a primary diagnosis of psychotic disorder, 53 (18 percent) had a primary diagnosis of substance-induced psychotic disorder, and 26 (9 percent) had indeterminate symptoms.

BELDs

As shown in Table 2, according to the BELDs, symptoms for 195 patients (65 percent) indicated a primary psychotic disorder; symptoms for 101 patients (33 percent) indicated a substance-induced psychotic disorder; and symptoms for six patients (2 percent) indicated no psychotic disorder.

Concordance between diagnosis and the BELD

The overall agreement between the emergency department diagnoses and the BELDs was only fair (κ=.32). As shown in Table 2, 75 percent of patients given a diagnosis of a primary psychotic disorder in the emergency department also received a BELD of primary psychotic disorder (true-positive diagnosis). However, 56 patients (25 percent) given a diagnosis of a primary psychotic disorder in the emergency department were determined by the BELD to have a substance-induced psychotic disorder (52 patients) or no psychotic disorder (four patients) (false-positive diagnosis).

Of the patients diagnosed in the emergency department as having a substance-induced psychotic disorder, 75 percent also received a BELD indicating substance-induced psychotic disorder (true-positive diagnosis). Twenty-one percent of patients with an emergency department diagnosis of substance-induced psychotic disorder had a primary psychotic disorder according to the BELD (false-positive diagnosis).

Diagnosis and hospitalization

The relationship between the diagnosis given in the emergency department and whether the patient was hospitalized on an inpatient unit is shown in Table 2. Of the 223 patients for whom the diagnosis was a primary psychotic disorder, 93 percent were hospitalized, whereas only 66 percent of patients with a diagnosis of substance-induced psychotic disorder were hospitalized. The patients with an indeterminate diagnosis had an intermediate rate of hospitalization, 85 percent. The difference in rates of hospitalization between the groups was statistically significant (χ²=28.32, df=2, p<.001). There was no significant relationship between the severity of clinical presentation, as assessed by the PANSS score, and whether a patient was hospitalized.

Of the 56 patients with a false-positive diagnosis of primary psychotic disorder, 53 patients (95 percent) were hospitalized, a rate similar to that for patients with a true-positive diagnosis. In comparison, of the 40 patients with a true-positive diagnosis of substance-induced psychosis, 24 patients (60 percent) were hospitalized. The difference in hospitalization rates for primary psychotic disorder in the case of false-positive diagnoses and for substance-induced psychosis with true-positive diagnoses was statistically significant (χ²=17.64, df=1, p<.001).

The diagnosis also appears to have influenced the inpatient hospital discharge diagnosis, despite the additional observation time (M±SD length of stay of 12.8±16.7 days). Of the 264 hospitalizations, 201 patients (76 percent) retained the original diagnosis. There was moderate concordance between the emergency department and hospital discharge diagnoses (κ=.46).

Diagnosis and discharge medication

Overall, 70 percent of patients were discharged from the hospital with an antipsychotic medication regimen, 14 percent of patients were discharged with a regimen for other medications, and 16 percent of patients were discharged with no medication (Table 2).

Eighty percent of the patients with a diagnosis of a primary psychotic disorder, 32 percent with substance-induced psychosis, and 58 percent of patients with indeterminate diagnoses were discharged with a prescription for antipsychotic medication. The difference between diagnosis type and discharge medications was statistically significant (χ²=93.81, df=4, p<.001).

Of the 56 patients with a false-positive diagnosis of primary psychotic disorder, 33 patients (59 percent) were discharged with an antipsychotic medication and eight patients (14 percent) were discharged with no prescribed medication. In comparison, of the 40 patients with a true-positive diagnosis of substance-induced psychosis, nine patients (about 20 percent) were discharged with antipsychotic medication, whereas 27 patients (66 percent) were discharged without a prescription. The difference between these groups and whether they were given prescriptions for antipsychotic medications was statistically significant (χ²=28.52, df=2, p<.001).

Diagnosis and outpatient referral

Overall, 184 patients (60 percent) were referred to either a mental health treatment outpatient facility that did not offer substance use treatment or to a nonspecific outpatient clinic (Table 2). Thirty-four (11 percent) of the patients were referred to substance use treatment only, and 34 patients (11 percent) were referred to treatment for co-occurring disorders. No referrals to follow-up treatment were noted for 17 percent of the patients.

The emergency department diagnosis was related to the outpatient treatment plan. Thirty-eight percent of the patients with a diagnosis of primary psychotic disorder were referred to a mental health treatment outpatient facility without any substance use treatment compared with 15 percent who were referred to treatment for either substance abuse or co-occurring disorders. Of the patients diagnosed as having substance-induced psychotic disorders, 6 percent were referred to a mental health treatment outpatient facility without any substance use treatment compared with 53 percent who were referred to treatment for substance abuse or co-occurring disorders. Of patients with an indeterminate diagnosis, 23 percent were referred to mental health services without any substance use treatment and 27 percent were referred to treatment for substance abuse or for co-occurring disorders. The difference between diagnostic groups and outpatient treatment plan was statistically significant (χ²=50.28, df=6, p<.001).

Of the 56 patients with false-positive diagnoses of primary psychotic disorder, 13 patients (25 percent) were referred to treatment for either substance use or for co-occurring disorders. In comparison, of the 40 patients with true-positive diagnoses of substance-induced psychotic disorders, 26 (65 percent) were referred to treatment for either substance abuse or co-occurring disorders. The difference between these groups and outpatient treatment plan was also statistically significant (χ²=18.89, df=3, p<.001).

Despite the adoption of a conservative approach in which an emergency department diagnosis of psychotic disorder was coded as primary only if all diagnoses included in the narratives were primary psychotic disorders, 25 percent of patients diagnosed in the emergency department as having a primary psychotic disorder received a BELD indicating substance-induced psychotic disorder or no psychotic disorder. The misdiagnosis of substance-induced psychotic disorders as primary psychotic disorders raises concerns both because of the pattern of diagnostic error as well as the long-term treatment implications for the patient.

The DSM-IV criteria for substance-induced psychotic disorder were designed to reduce overdiagnosis of primary psychotic disorders. A clinician must establish that the psychotic symptoms persist for a "substantial period of time (e.g., about one month) after the cessation of acute withdrawal or severe intoxication" before establishing a diagnosis of primary psychotic disorder. In an emergency department setting, given the typical lack of data regarding the persistence of psychotic symptoms in the absence of substance use, one might expect that emergency department psychiatrists would err on the side of diagnosing a substance-induced psychotic disorder. Instead, the physicians in this study seemed to consider primary psychotic disorder to be the "default" diagnosis instead of substance-induced psychotic disorder, as is intended by DSM-IV.

There are many reasons why it is difficult to accurately diagnose a substance-induced versus primary psychotic disorder in the emergency department, including patient, physician, and systems-level factors. However, if the error is due to a lack of knowledge of diagnostic criteria, this may highlight an area of psychiatric training that needs improvement. There may need to be a greater emphasis on substance-induced disorders in psychiatric training, especially for those who will be working in an emergency department setting.

Patients given a diagnosis of primary psychotic disorder in the emergency department were significantly more likely to be admitted to the hospital for an inpatient stay, to be started on antipsychotic medications, and to not be referred to outpatient treatment for substance abuse. The opposite was found for patients whose psychosis was diagnosed in the emergency department as substance induced; they were significantly less likely to be hospitalized, to be prescribed antipsychotic medication, and to receive mental health treatment. The power of the diagnosis to influence the patient's disposition beyond the doors of the emergency department highlights the need to be especially circumspect when making diagnoses in challenging clinical situations.

The clinical impact of the emergency department diagnosis was particularly striking in a comparison of the diagnoses in which there was disagreement on primary psychotic disorder (false positives) and agreement on substance-induced psychotic disorder (true positives). The differences in patients' hospitalization rates, discharge diagnoses, medications at discharge, and outpatient referral between these two groups further underscore the need for care when diagnosing the symptoms of a patient in the emergency department. These findings also suggest the need to develop a diagnostic tool that can be used in the emergency department to help improve diagnostic accuracy among patients presenting with psychosis and substance use.

Regardless of the diagnosis, for a population that is actively using substances, there should be higher rates of referral to treatment for substance abuse or co-occurring disorders. On the whole, these patients, especially those with primary psychotic disorders, are not referred to the treatment they need for substance abuse and co-occurring disorders. While physicians await improvements in diagnostic accuracy, active attempts to improve the referral rate to treatment services for this population appear warranted.

The accuracy of the BELD was limited by its sole reliance on a review of the narratives and other information provided by the research assistants. The BELD reviewers did not have access to the patients and thus could not collect additional information for clarification. In addition, although the emergency departments used in this study varied in terms of type of hospital (public and private) and psychiatric staffing (psychiatric residents and attending physicians), all of the emergency departments had special psychiatric emergency areas with access to overnight emergency beds. The results may therefore not be generalizable to other settings.

In today's environment of early intervention and treatment for psychotic disorders as well as decreasing lengths of stay on inpatient psychiatric units, there is increasing pressure on emergency department psychiatrists to definitively diagnose patients during their initial psychotic presentation. This article highlights the challenge of accurately diagnosing a first psychotic episode when it occurs in the context of substance use and underscores the potential for negative consequences if a diagnostic error is made. The results suggest a need both for further study into methods to improve diagnostic accuracy and for circumspection when diagnosing a patient who presents with psychotic symptoms in the context of substance use.

Support for this investigation was provided by research grants R01-DA-10539 and 2R01-10539-06 from the National Institute on Drug Abuse and by K0-AA014223 from the National Institute on Alcohol Abuse. The authors are indebted to Ingrid Ramirez, B.A., and Milagros Ventura, B.A., for their assistance with data processing and preparation.