Because stimulants can mimic and alter schizophrenic symptoms, stimulant abuse can lead to diagnostic error—either a false positive or a false negative diagnosis of schizophrenia. Such diagnostic errors could lead to inappropriate treatment. On the one hand, a false positive diagnosis of schizophrenia for a patient who actually has a substance-induced psychosis could lead to inappropriately prolonged use of antipsychotic medication. On the other hand, a false negative diagnosis of schizophrenia could lead to a failure to use antipsychotic medication. Diagnostic uncertainty and inaccuracy can also lead to exclusion from appropriate treatment programs. For example, substance abuse treatment programs often accept patients with transient toxic psychoses but exclude those with schizophrenia or uncertain diagnoses.
Subjects were recruited on admission to a large, urban Veterans Affairs medical center. Data were drawn from a study, conducted between 1990 and 1995, that compared the efficacy of two experimental treatment programs for cocaine abusers with schizophrenia. One program used case managers to coordinate separate psychiatric and substance abuse services. In the other program, a new treatment unit provided comprehensive integrated services for both disorders. Patients were eligible for the study if the admitting psychiatrist concluded that the patient required psychiatric hospitalization, had a psychotic disorder that might be schizophrenia, and had abused cocaine within the past six months.
The study was approved by the medical center's institutional review board, and all subjects gave informed consent. All patients were hospitalized initially. Each experimental program then attempted to maintain patients in ambulatory care, but success varied. By the 18-month follow-up assessment, patients had received varying durations and intensities of both inpatient and outpatient care.
Altogether, 221 patients were referred to the study by admitting psychiatrists. Forty-nine did not meet study criteria: 13 were too dangerous, ten never had psychotic symptoms, eight never abused stimulants, six had no phone or mailing address, four could not read the consent form, two were too psychotic to consent, two could not participate because it would conflict with current treatment, two planned to move out of the state, one was participating in a conflicting research project, and one had medical problems that were too severe. Seven patients who met study criteria refused to provide consent. Thus a total of 165 subjects completed the initial assessment.
All subjects were males with a mean±SD age of 40±6.4 years. A total of 129 (78 percent) were African American, 28 (17 percent) were Caucasian, and eight (5 percent) were of other races. The mean±SD level of education was 13±1.8 years. Sixty-eight patients (41 percent) had never married, 18 (11 percent) were currently married, and 79 (48 percent) were separated, widowed, or divorced. Seventy-three (44 percent) were homeless.
On average, they had used cocaine regularly for nine years (SD=6.7) and had used cocaine on 13±11.7 days of the 30 days before study entry. A total of 160 patients (97 percent) met DSM-III-R criteria for current cocaine abuse or dependence, 86 (52 percent) for alcohol, 58 (35 percent) for marijuana, 21 (13 percent) for opiates, 17 (10 percent) for sedatives, 12 (7 percent) for amphetamine, and seven (4 percent) for other substances. The mean number of psychiatric hospitalizations for the groups was ten (SD=11).
Thus these patients were severely mentally ill and chronically cocaine dependent individuals. They were characterized by homelessness, many psychiatric hospitalizations, a long duration of cocaine use, and recent extensive use of cocaine as well as other drugs and alcohol.
All subjects were assessed using the Structured Clinical Interview for DSM-III-R (SCID-R) for axis I disorders (
+28), supplemented whenever possible by hospital records and by interviews with family members, friends, and other persons familiar with the patient (collateral interviews). The project was affiliated with the UCLA Clinical Research Center for Schizophrenia and Psychiatric Rehabilitation, which is sponsored by the National Institute of Mental Health. The center maintains reliability of measurement across many projects through standardized training and quality assurance.
Using consensus ratings made by the center's senior diagnosticians as "gold-standard" criteria, five master's-level psychologists were trained to high levels of interrater reliability in the use of the SCID-R (overall kappa of at least .75 for symptoms and 100 percent agreement on diagnosis). Our use of the kappa statistic as a measure of association follows Kraemer's recommendation (
+29) that chance-corrected indexes of association are, in general, more interpretable and meaningful. When using gold standards, Kraemer identifies a family of weighted kappa statistics and notes that the conventional, so-called Cohen's kappa (
+30) is simply the statistic that derives when false positive and false negative errors are weighted equally.
One statistical feature of our weighted kappa is atypical, namely, that the separate units of observation are not independent subjects but rather items from a scale administered to a single individual. Because separate items within a scale are not independent, conventional significance testing of these weighted kappas is not appropriate. However, we use them only descriptively, and they are not tested for statistical significance. The nonindependence of observations does nothing to compromise the statistic as a descriptive index of the degree of chance-corrected association.
The diagnostic interviewers' initial results were discussed in a weekly case conference involving two senior diagnosticians (the first and third authors). The sources of diagnostic uncertainty were discussed in an effort to identify key questions that may not have been asked or new sources of information. Based on the conference, the diagnostic interviewer collected new information and presented the case again until no further sources of diagnostic information could be identified. This strategy, together with the reassessment described below, is in conformance with the "LEAD standard" proposed by Spitzer (
+31). LEAD is an acronym for Longitudinal, Expert and All Data and refers to criterion-based diagnoses made by experts reaching consensus after examining data from a variety of sources and from repeated assessments over considerable time.
Family members or other collaterals were available for interviews in about a quarter of the cases but were rarely able to provide key information, such as whether the patient's psychotic symptoms persisted during abstinence. Medical records were available in nearly three-quarters of cases and occasionally provided important diagnostic information, such as previous incoherence in a patient who reported a nonbizarre delusion as the only psychotic symptom.
The scoring of the SCID-R was modified to allow interviewers to record the source of uncertainty on any item. A list of potential sources of uncertainty was developed. The list included insufficient periods of abstinence, inconsistent responses, poor memory, and disorganized speech, among other sources. If uncertainty on individual items led to uncertainty at decision points in the SCID-R algorithm, interviewers completed questioning along each of the branches from the decision point.
For example, on a number of individual items, the unmodified SCID-R requires that a psychotic symptom be counted toward the diagnosis of schizophrenia only if it is judged to be "not organic." The modified scoring allowed the interviewer to score the symptom in one of three ways: definitely organic, uncertain organicity, or definitely not organic. If organicity was uncertain, the sources of uncertainty were recorded, such as an inconsistent history or insufficient periods of abstinence. Similarly, substance abuse and dependence can raise concern that the mood syndromes, chronicity, and deterioration in functioning may have an organic basis. These criteria were also operationalized to allow for recording uncertainty. (A list of these criteria and their operational characteristics is available from the first author.)
In cases of diagnostic uncertainty, these modifications of SCID-R scoring led to a list of possible diagnoses (differential diagnosis) and a list of sources of uncertainty. The sources of uncertainty were keyed to the criteria for schizophrenia. For example, in
DSM-III-R the organic exclusion criterion for schizophrenia is met if "it cannot be established that an organic factor initiated and maintained the disorder" (
+32). For this study, the statement was operationalized with separate criteria for determining whether psychotomimetic substances initiated or maintained the disorder.
For example, a subject reported that persecutory delusions had first started during a period of amphetamine abuse. This temporal relationship suggested that amphetamine might have initiated the delusions, but it did not provide enough information to conclude that amphetamine was an organic etiologic factor. The close temporal relationship might have been coincidental, or the amphetamine might have precipitated a psychotic disorder that then maintained itself. The interviewer then asked about the persistence of the symptom in the absence of amphetamine use. The interviewer would have concluded that amphetamine was etiologic if both of the following had been true: first, that at least twice, persecutory delusions resolved within six weeks of stopping amphetamine use, and, second, that persecutory delusions had never occurred more than six weeks after stopping amphetamine use.
Conversely, the interviewer would have concluded that persecutory delusions were not due to amphetamine if the subject reported that delusions had occurred more than six weeks after stopping amphetamine use. We required two instances of resolution within six weeks to avoid ascribing etiologic significance to chance coincidence of drug use and psychosis. However, in the example discussed here, the subject had been using amphetamine for several years and had never stopped using the drug for more than three weeks. Although persecutory delusions were less prominent during these brief periods of abstinence, they did not resolve.
Thus the interviewer noted that persecutory delusions were present, that it was uncertain whether they were organic, and that the source of the uncertainty was insufficient periods of abstinence. Rating persecutory delusions as present but their organic basis as uncertain resulted in a differential diagnosis that included, among others, amphetamine delusional disorder and schizophrenia. Similar criteria were developed to specify the relationship between abused substances and three other criteria for schizophrenia±_chronicity, deterioration, and mood syndromes.
This project was designed and conducted before publication of
DSM-IV(
+33). However, early drafts from the
DSM-IV work group suggested that symptoms should not be considered drug induced if they persisted for more than six weeks after cessation of drug use. Consequently, we used the same time frame in our criteria. In the final version of
DSM-IV this time frame was reduced to four weeks.
+
Assessment of drug abuse
Lifetime and current drug use were assessed using the SCID-R section on substance abuse and dependence and the Addiction Severity Index (ASI) (
+34). The ASI is a comprehensive structured interview used widely in substance abuse research, and its validity and reliability have been established among mentally ill substance abusers (
+35,
+36). In addition. results of urine toxicology tests obtained at study entry were used as evidence of substance use even if subjects denied using the detected substances.
+
Diagnostic reassessment
Eighteen months after study entry, each subject for whom the original diagnosis was uncertain was reassessed. An attempt was made to resolve SCID-R items originally coded as uncertain. Diagnostic information was gathered through interviews with subjects and from medical records and collateral interviews whenever possible. Medical records often included many notes by psychiatrists, psychologists, nurses, social workers, and case managers in the two experimental programs. Reported abstinence was verified using urine toxicology tests. These data were reviewed with the two senior diagnosticians as described above until no further sources of diagnostic information could be identified.
Of the 165 patients who consented and completed initial assessments, 27 were not available for the 18-month reassessment: 17 could not be found, six had died, two had been dropped from the study because they had become too dangerous, one was not eligible for VA care, and one elected to drop out.
Cases were classified according to whether a definitive diagnosis was made or diagnosis was uncertain. The frequency with which each diagnosis appeared in the differential diagnoses of the uncertain cases was tabulated, and sources of diagnostic uncertainty were then tabulated based on this frequency. Only uncertainty at the criterion level was tabulated. The frequency with which each criterion for schizophrenia was met, not met, or uncertain was tabulated.
Despite structured diagnostic procedures, urine toxicology tests, review of medical records, and interviews with collateral sources, both initially and prospectively, we reached a definitive diagnosis in only 25 percent of cases. Most of the uncertainty arose because most subjects first developed psychotic symptoms during a period of psychotomimetic drug abuse and rarely stopped using drugs long enough to determine if the drugs were responsible for psychotic symptoms, mood syndromes, or deterioration in social and occupational functioning. Thus the differential diagnoses for the uncertain cases included mood disorders, substance-induced psychotic disorders, and other psychotic disorders.
These data suggest that for most patients with chronic psychoses and extensive psychotomimetic drug abuse and dependence, treatment must begin without a definitive diagnosis. Thus treatment algorithms and psychiatric services that require definitive diagnoses will be useful for only a minority of patients in this population.
It is important not to overgeneralize these findings. Our study involved a unique clinical population. Patients entered the study during a period of exacerbated symptoms and social problems requiring hospitalization. Fewer obstacles to a definitive diagnosis may exist in a less severely ill population, such as one drawn from an outpatient clinic. All our patients had psychotic disorders resembling schizophrenia. A definitive diagnosis may be less problematic for patients with mood and anxiety syndromes. Indeed, a previous study found that current substance dependence did not worsen the one-week test-retest reliability of mood and anxiety diagnoses using the Structured Clinical Interview for DSM-III-R (
+37). Unfortunately, the study included too few subjects with schizophrenia to determine the effect of current substance dependence on the reliability of that diagnosis.
All of our patients were men. Women who met our enrollment criteria might have differed in important ways. For example, if women tend to have more frequent and longer periods of abstinence, this factor would lead to less diagnostic uncertainty. All our patients were veterans of military service. Compared with other chronically psychotic stimulant-abusing persons, they probably had histories of less severe psychiatric disorders and a higher level of social functioning, at least when they entered the military. In other populations, more individuals may experience the early onset of psychotic symptoms before psychotomimetic drug abuse obscures the diagnosis.
Finally, the research setting was a large urban hospital. The results might be different in rural settings where psychotomimetic drugs may be less immediately available. Our study had several other limitations. First, the criteria for the modified SCID-R scoring were biased in favor of diagnostic uncertainty in several ways. Substances were judged to be etiologic based on only two considerations±the temporal relationship between the symptoms and substance use and whether or not the substance in question was known to cause that class of symptoms. Clinicians, on the other hand, may consider how characteristic the symptoms are for substance-induced syndromes.
For example, some subjects in our study had the persisting delusion that their drug use was closely monitored by clandestine government agencies using a variety of electronic devices. Some clinicians would consider this delusion likely to be substance induced because the content is not bizarre and is so closely related to illegal drug use. Substances were judged to be etiologic only if the symptoms always began during periods of substance use and had twice resolved within six weeks of stopping drug use. Less stringent definitions would have produced more diagnostic certainty. Finally, the reliability of diagnoses was inferred from the reliability of training interviewers rather than directly through co-rated interviews of the research subjects.
Despite these limitations, the results do indicate that substance abuse and dependence greatly complicate the diagnosis of psychotic symptoms. Diagnostic uncertainties might be resolved if abstinence could be ensured for many weeks, but ensuring such abstinence is rarely possible. Faced with this dilemma, clinicians may be tempted to pick the most likely diagnosis. However, premature diagnostic conclusions could deprive the patient of appropriate treatment. Clinicians should attempt simultaneous treatment of substance abuse or dependence and primary psychotic disorders even for patients who may later be shown to have had a substance-induced psychotic disorder.
In a substantial proportion of cases, it may not be possible to resolve diagnostic uncertainty even after months of treatment. This lack of resolution can occur if treatment fails to produce abstinence from psychotomimetic substances long enough to eliminate diagnostic uncertainties. Even if psychotic symptoms do resolve during periods of abstinence, it may be unclear whether the resolution was due to abstinence, the effects of antipsychotic medications, or the natural course of the psychotic disorder.
Research on dual diagnosis populations should acknowledge and manage diagnostic uncertainty. One approach is to study only those with definitive diagnoses. Unfortunately, such a strategy would make it difficult to generalize the results to clinical populations characterized by diagnostic uncertainty. Another approach is to classify subjects, as we have, into definitive and uncertain categories pending additional information from other sources or from prospective data on the relationship between abstinence and symptoms. This approach avoids the temptation to reach premature diagnostic conclusions based on a perceived probability that drugs are, or are not, etiologic.
Future research should focus on the effects of stimulants and other abused drugs on symptoms of subjects with and without a primary psychotic disorder. Data from such research could allow refinements in diagnostic criteria for distinguishing substance-induced psychosis from other psychotic disorders.
This research was supported by grant R01-MH48081 from the National Institute of Mental Health (NIMH) and grant IIR-90-033 from the Veterans Affairs Health Services Research and Development Service. The authors thank Robert P. Liberman, M.D., director of the UCLA Clinical Research Center for his support. The authors are indebted to Helena C. Kraemer, Ph.D., for invaluable statistical consultation during the development of the program to train interviewers in diagnosis and symptom assessment within the NIMH-funded Clinical Research Center for Schizophrenia and Psychiatric Rehabilitation.