Diagnostic Stability Four Years After a First Episode of Psychosis

Diagnostic stability is the degree to which a diagnosis remains constant at subsequent patient assessments (1). The diagnosis of a psychotic disorder is based on the presence or absence of characteristic symptoms. However, the presence of such symptoms varies during the course and treatment of these illnesses, which raises the question of how stable a diagnosis of psychosis remains over time.

Diagnostic instability is an important issue, because it has potential implications in terms of service planning and resource allocation. For example, the services required for treating patients with drug-induced psychoses are likely to differ from those required for treating patients with non-substance-related psychotic disorders. Similarly, patients with schizophrenia are likely to require different medical and psychosocial treatments compared with patients who have affective psychoses, in which negative symptoms are less likely to persist after the acute phase of illness.

Several readmission studies have examined the stability of clinical diagnoses over the course of illness (2,3,4,5,6,7,8,9). However, such studies are limited by inadequate reliability of clinical diagnoses and the bias that is inherent in sampling rehospitalized patients (10). Consequently, other authors examined diagnostic stability prospectively (1,11,12,13,14,15,16,17,18) with varying durations of follow-up. These studies concluded that schizophrenia is the most stable initial diagnosis (about 90 percent of initial diagnoses are retained at follow-up), followed by affective psychosis (around 80 percent). Schizophreniform disorder, schizoaffective disorder, and psychosis not otherwise specified are the least stable initial diagnoses.

Even though these studies represented an improvement in methodology compared with earlier studies, they were limited by the fact that they involved first-admission patients only, which raises the question of whether these findings are applicable to all patients with psychosis. The most conclusive way to achieve diagnostic stability is to conduct a prospective longitudinal study of incident cases. However, such studies are difficult to conduct because of the time required to accrue a sample that is large enough for investigating factors related to diagnostic change. Two previous studies (19,20) examined inpatients and outpatients who experienced a first episode of psychosis and reported findings similar to those above. However, those studies did not specifically determine the factors associated with or predictive of diagnostic instability.

Thus we sought to determine diagnostic stability over the first four years of illness in a sample of patients who presented with a first episode of psychosis to a catchment area service in Dublin, Ireland. We examined the stability of the diagnosis at a number of levels. First, we sought to establish how many patients who received a diagnosis at first presentation retained this diagnosis at follow-up. Next, we established how many patients with a given diagnosis at follow-up had also received that diagnosis at first presentation. Finally, we sought to determine what factors predicted a change in diagnosis at four-year follow-up.

Methods

The study was conducted between 1995 and 1999 at Cluain Mhuire Family Centre, a catchment area service that provides community-based psychiatric care for an urban population (approximately 165,000) in County Dublin, and at St. John of God Hospital in County Dublin (21). The ethics committee of the hospital, which serves both institutions, approved the study. First-episode psychosis was defined as a first-ever presentation to any psychiatric service with a psychotic episode. Patients who commenced antipsychotic medication before referral were included in the study, provided that they were being treated for a first psychotic episode and their treatment had not commenced more than 30 days before assessment.

Patients were followed up four years after the initial presentation, and all patients gave informed consent to participate in the initial and follow-up studies. Patients were reassessed across the same clinical measures, which included a diagnostic interview by an investigator who was blinded to the original diagnosis and the initial assessments. Interrater reliability was established by the concomitant examination of ten patients with psychosis or suspected psychosis by study evaluators. The rater who conducted follow-up assessments was blinded to the initial diagnosis, which reduced the chance of test-retest reliability—that is, patients who were given a diagnosis at one time point by a rater are more likely to test the same on subsequent assessments by the same rater.

The follow-up was conducted by means of face-to-face interviews and by contacting treating teams, patients' relatives and carers, and general practitioners as well as reviewing case notes for additional information relevant to diagnosis.

Assessments

All diagnoses were made on the basis of the Structured Clinical Interview for DSM-IV (SCID) (22). In addition, we assessed patients across a number of variables to determine the factors associated with a change in diagnosis. These variables were divided into four groups: background characteristics (age at first presentation, gender, years spent in education, marital status, social class at first presentation, presence or absence of mental retardation, and presence or absence of a personality disorder), clinical history (Global Assessment of Functioning [GAF] score at first presentation, previous treatment with an antipsychotic medication, lifetime history of substance use disorder, and duration of untreated initial psychosis), initial hospitalization characteristics (duration of first hospitalization, scores on the Positive and Negative Symptoms Scale [PANSS] [23] at first presentation, current substance use disorder, admission status [temporary or voluntary], and presence or absence of private health insurance), and outcome characteristics (GAF score at four years, total number of admissions at follow-up, social and occupational functioning at four years, psychopathology at four years, medication status, and insight).

Data analysis

We divided the sample into three diagnostic subgroups: schizophrenia, affective disorders, and other psychoses (drug-induced psychosis, delusional disorder, psychosis not otherwise specified, and schizoaffective disorder). We evaluated diagnostic stability by using two measures in each diagnostic subgroup. The first measure, prospective consistency, refers to the proportion of patients who were in a diagnostic category at presentation who had the same diagnosis at follow-up; this value corresponds to the positive predictive value, assuming follow-up diagnosis as the standard. The second measure, retrospective consistency, refers to the proportion of patients with a given diagnosis at follow-up who had received that diagnosis at first presentation; this value corresponds to sensitivity.

We determined the reasons behind a change in diagnosis by grouping patients into two groups for analysis: those whose diagnosis changed between first presentation and follow-up, and those whose diagnosis was changed to schizophrenia. We compared the patients in each group by using t tests for continuous variables and chi square tests for categorical variables. Only variables that were significantly associated with a change in diagnosis were entered into a direct logistic regression analysis to determine which factors were significantly associated with a diagnostic change at follow-up. All data were analyzed with use of SPSS (24).

Results

A total of 171 patients were assessed at baseline and received diagnoses based on the SCID I interview. At four-year follow-up, five patients had died (four patients by suicide and one from natural causes); these patients' diagnoses were organic psychosis (three patients), bipolar affective disorder (one patient), and drug-induced psychosis (one patient). We excluded cases of organic psychosis (one patient) from further analysis and made new diagnoses for 147 of the initial 165 patients at four years, which resulted in a follow-up rate of 89 percent (Table 1). No statistically significant difference was found between the patients who received new diagnoses and those who did not in terms of age at first presentation, gender, duration of untreated initial psychosis, GAF score at first presentation, psychopathology at first presentation, presence of substance use disorder at first presentation, or initial treatment setting (inpatient or outpatient). Of the 147 patients who were followed up, 131 (89 percent) received their follow-up diagnosis by means of face-to-face interviews.

Schizophrenia

Ninety patients received a diagnosis of schizophrenia or schizophreniform disorder at first presentation. Of these, 87 retained this diagnosis at follow-up. This finding equates to a prospective consistency for a diagnosis of schizophrenia or schizophreniform disorder of 97 percent. We next separated out the subgroup of patients with schizophreniform disorder (N=15). Of the 75 patients who were given a diagnosis of schizophrenia at first presentation, 72 (96 percent) retained this diagnosis at follow-up. Of the 15 patients with schizophreniform disorder, five retained this diagnosis at follow-up, for a prospective consistency rate of 33 percent.

Of the 147 patients we followed up, 106 received a diagnosis of schizophrenia or schizophreniform disorder; of these, 87 received this diagnosis at both first presentation and follow-up, for a retrospective consistency of 82 percent. We repeated the analysis after removing cases of schizophreniform disorder (five patients). Among the 101 patients with schizophrenia, 71 (70 percent) who were given a diagnosis of schizophrenia at follow-up had also received that diagnosis at first presentation. All five patients with schizophreniform disorder received that diagnosis at first presentation as well as at follow-up. Thus the retrospective consistency for a diagnosis of schizophreniform disorder was 100 percent (five of five).

Bipolar affective disorder and affective psychosis

The prospective consistency of a diagnosis of bipolar affective disorder was 80 percent; 16 of 20 patients who were given this diagnosis at first presentation retained it at follow-up. At follow-up, 21 patients received a diagnosis of bipolar disorder, of whom 16 had received that diagnosis at both time points. Thus the retrospective consistency for a diagnosis of bipolar affective disorder was 76 percent (16 of 21). We next added the subgroup of patients who had major depression with psychotic features (N=11). Of the 31 patients with affective psychosis at first presentation, 25 retained that diagnosis at follow-up. Thus the prospective consistency of a diagnosis of affective psychosis was 81 percent (25 of 31). At follow-up 29 patients were given a diagnosis of an affective psychosis. Of these, 24 received that diagnosis at both first presentation and follow-up, for a retrospective consistency of 83 percent.

Other diagnostic subgroups

Of the initial 26 patients with other forms of psychosis, nine received that diagnosis at follow-up, for a prospective consistency of 35 percent. Among patients who were assigned a diagnosis of drug-induced psychosis at first presentation (N=11), two (27 percent) retained that diagnosis at follow-up. Among patients who were given a diagnosis of delusional disorder at first presentation (N=12), five retained that diagnosis at follow-up, for a prospective consistency of 38 percent (five of 13).

The retrospective consistency of a diagnosis of other psychoses was 78 percent (seven of nine). The retrospective consistency for a diagnosis of drug-induced psychosis was 100 percent; the two patients who received that diagnosis at follow-up also received it at first presentation. Of the six patients with delusional disorder at follow-up, five received that diagnosis at first presentation, for a retrospective consistency of 83 percent.

Factors related to diagnostic instability

Total group. Of the 147 patients who were followed up, 37 (25 percent) were given a different diagnosis at follow-up than at first presentation; the diagnoses of 110 patients (75 percent) remained stable. We compared these two patient groups to identify the factors that distinguished one group from the other. The groups did not differ in terms of background characteristics. In terms of clinical history, patients whose diagnosis changed evidenced a statistically significant shorter duration of untreated initial psychosis compared with those whose diagnosis remained stable (t=-2.18, df=145, p=.03) and were more likely to present with a lifetime history of alcohol or substance abuse or dependence (χ²=4.58, df=1, p=.03). In addition, a statistically significant difference in initial hospitalization characteristics was observed between the two groups; patients whose diagnosis changed were more likely to evidence lower total symptom scores (t=-2.40, df=144, p=.02) and to have a comorbid diagnosis of alcohol or substance abuse or dependence (χ²=4.21, df=1, p=.04) at first presentation. No significant differences in outcome characteristics were noted between the two groups.

Logistic regression analysis using the presence or absence of a change in diagnosis as the dependent variable with psychopathology, current and lifetime history of substance abuse or dependence, and duration of untreated psychosis as the predictor variables indicated that a change in diagnosis at follow-up was significantly associated with low levels of psychopathology at first presentation (β=.03, Wald=4.29, p=.04); there was a trend toward significance for a shorter duration of untreated initial psychosis.

Schizophrenia. Of the 101 patients who were given a diagnosis of schizophrenia at follow-up, 29 (20 percent) had not been given that diagnosis initially, whereas for 72 (80 percent) the follow-up diagnosis represented a stable diagnosis. Ten patients evidenced a shift in diagnosis from schizophreniform disorder to schizophrenia and were excluded from further analysis because their cases reflected the natural history of the disorder and not diagnostic change per se.

The 19 patients whose diagnosis changed to schizophrenia evidenced a significantly higher initial GAF score (t=3.24, df=89, p=.003) and completed fewer years of education (t=-.48, df=89, p=.02) compared with the patients with a stable diagnosis of schizophrenia (N=72). In terms of initial hospitalization characteristics, a change to a diagnosis of schizophrenia was associated with significantly fewer positive symptoms (t=-2.93, df=89, p=.01) and negative symptoms (t=-3.93, df=89, p=.001) at first presentation. A change to a diagnosis of schizophrenia was also significantly associated with a current (χ²=9.92, df=1, p=.01) and lifetime (χ²=6.10, df=1, p=.47) diagnosis of alcohol or substance abuse or dependence.

We next performed a logistic regression analysis to determine whether these factors were significant predictors of change to a diagnosis of schizophrenia. This analysis revealed that a shift to a diagnosis of schizophrenia was associated with fewer years of education (β=.39, Wald=5.19, p=.02) and fewer negative symptoms (β=.18, Wald=5.11, p=.02) at first presentation.

Discussion

Main findings

The main finding of this study was that one-quarter of the patients who presented with a first episode of psychosis were given a different diagnosis at four-year follow-up. The most common change was to a diagnosis of schizophrenia. Patients with an initial diagnosis of drug-induced psychosis, psychosis not otherwise specified, or schizophreniform disorder were the most likely to evidence this change. Compared with patients who had a stable diagnosis of schizophrenia, those who were given a diagnosis of schizophrenia as a new diagnosis at follow-up completed fewer years of education and had lower levels of psychopathology at first presentation. These patients were also more likely to have a comorbid diagnosis of alcohol or substance abuse or dependence.

Diagnostic subgroups

Schizophrenia was the most stable diagnosis, followed by affective psychosis. Overall, both disorders displayed high levels of stability, which supports the view that they are distinct clinical entities (25). The most unstable diagnoses were psychosis not otherwise specified, schizophreniform disorder, and drug-induced psychosis. The low prospective consistency of a diagnosis of drug-induced psychosis is of particular interest, because this finding suggests that most people who presented in this fashion had an underlying major psychotic illness. Indeed, almost three-quarters of such patients evidenced a shift in diagnosis into a different category of psychotic illness at follow-up. However, it was beyond the scope of this study to determine the nature of the relationship between psychosis and drug abuse—that is, whether drug abuse causes the psychotic illness or, rather, leads to psychotic symptoms in predisposed or vulnerable patients.

The prospective consistency (positive predictive value) of schizophrenia was higher than the retrospective consistency (sensitivity), which was an encouraging finding. At first presentation it is especially important to make a diagnosis with a high degree of certainty. Clinically, the major concern would be a low specificity (that is, many false-positives), because patients would be incorrectly labeled as having schizophrenia, which would have major implications for patients and their families. The reverse situation—lower sensitivity with many false negatives—is less important, because clinical treatment is initially based on symptoms, not diagnostic categories (19). An alternative method of examining this outcome is to calculate the probability of falsely assigning a diagnosis of schizophrenia, as opposed to another psychotic category, at onset (1 minus positive predictive value). Thus, for patients with schizophrenia, a clinician would make a false-positive diagnosis for one of every 25 patients when using the SCID I interview. Among patients with bipolar affective disorder, the false-positive rate is one in five. Among patients who present with a first episode of psychosis, a clinician will make the wrong diagnosis in one of every four assessments.

Factors related to diagnostic change

A change in diagnosis at follow-up was associated with low levels of psychopathology, poor educational functioning, and presence of substance abuse or dependence at first presentation. This finding suggests that patients whose diagnosis changed to schizophrenia evidenced early warning signs in terms of poor educational functioning, which may reflect a prodromal phase of illness. However, the low levels of psychopathology displayed make it more difficult to correctly differentiate diagnostic subgroups at first presentation, thus leading to uncertainty of diagnosis. An additional complicating factor is the presence of substance abuse or dependence at first presentation, which serves to cause further diagnostic uncertainty. This finding suggests that patients presenting with an initial diagnosis of a drug-induced psychosis need to be followed up carefully, because a significant proportion of these patients will evidence a shift into a different category of psychosis, which may require long-term treatments and interventions.

The main limitation of this study was the small number of patients in the subgroup without schizophrenia, which makes interpretation of diagnostic shift for these patients difficult because of reduced power. However, this was a study that examined outcomes for all psychoses. Future studies of diagnostic stability may focus on specific diagnostic subgroups to increase the power of their conclusions. A further potential limitation of this study is that these findings are applicable only to the relatively early phase of illness, and reassessment at a later time may reveal further patterns of diagnostic instability.

Conclusions

Approximately one-quarter of patients who presented with a first episode of psychosis received a different diagnosis at four-year follow-up. Furthermore, changes in diagnosis over time were associated with substance abuse and low levels of psychopathology at first presentation. The greatest instability occurred in the less frequent diagnostic categories of drug-induced psychosis and psychosis not otherwise specified.

Dr. Whitty, Dr. McTigue, Dr. Kamali, and Professor O'Callaghan are affiliated with Cluain Mhuire Family Centre in Blackrock, County Dublin, Ireland. Professor O'Callaghan is also with St. John of God Hospital in Stillorgan, County Dublin, with which Dr. Clarke and Dr. Larkin are affiliated. Dr. Browne is with Waterford Regional Hospital in County Waterford, Ireland. Send correspondence to Professor O'Callaghan at Stanley Research Unit, Cluain Mhuire Family Centre, Newtownpark Avenue, Blackrock, County Dublin, Ireland (e-mail, [email protected]).

Enlarge table