Characteristics of Participants and Nonparticipants in Medication Trials for Treatment of Schizophrenia

Trials of the efficacy of treatments for mental disorders generally seek to recruit samples of patients as efficiently as possible, usually without systematically sampling an identified target population. Patients who enroll in the trials may constitute a highly select group relative to the larger patient population for whom the treatment is intended.

Furthermore, the number of patients in the target population who are never invited to be screened for participation in efficacy trials is rarely reported. The proportion of screened patients who are excluded and the proportion who do not consent to participate have also seldom been reported. In the 1980s two reviews of psychiatric research concluded that only 6 to 11 percent of studies reported the number of patients who declined to participate (1,2). A more recent review of clinical trials from outside of psychiatry found that only nine of 195 reports published in 1995 gave the consent rate (3).

Several recent psychiatric studies have described the outcomes of the subject selection process in research that has examined the efficacy of treatments for schizophrenia (4), bipolar disorder (5,6), affective psychosis (7), depression (8), and panic disorder (9). In only one of these studies (8) did as many as a third (34 percent) of the patients who screened positive for the target diagnosis actually participate. In the remaining studies, the proportion of screened patients who participated ranged from 10 to 25 percent.

The degree of bias introduced by the sample selection process in efficacy trials has not been thoroughly evaluated, even though trial results are often used to guide the treatment of large patient populations. Little is known about whether the study participants are similar to the general patient population and about whether differences between study participants and the overall clinical population might affect the generalizability of the study results.

Most efficacy trials exclude patients with schizophrenia who have a concurrent substance use disorder (10). Recently the number of "dual diagnosis" efficacy trials that actively select patients with schizophrenia and a concurrent substance use disorder has been increasing. However, we are unaware of reports examining the selection of subjects for dual diagnosis trials.

The study reported here compared patients who enrolled in efficacy trials of treatment for schizophrenia or schizoaffective disorder with the clinical population of patients with these diagnoses who did not participate. Our evaluation considered study populations in trials that selected patients with schizophrenia and no concurrent substance use disorder and in trials that selected patients with dual diagnoses of schizophrenia and a substance use disorder.

Methods

Between March 21, 1991, and October 22, 1997, the authors conducted ten trials of the efficacy of medications for treatment of schizophrenia or schizoaffective disorder at the Connecticut Mental Health Center in New Haven (references are available from the authors). Details about the design of these studies are shown in Table 1.

A total of 165 patients were randomly assigned to treatment or control groups in one or more of the ten trials, and 137 unduplicated individuals participated, including 26 patients who participated in two trials and one patient who participated in three trials. The first six trials listed in Table 1 selected patients with schizophrenia and no concurrent substance use disorder. The last four trials listed in Table 1 selected patients with dual diagnoses. The fifth trial, with six subjects, included some design features typical of treatment effectiveness studies, but it was included in this group of efficacy trials because subject recruitment was conducted in the usual fashion for efficacy trials.

To identify all patients registered at the CMHC who had a clinical diagnosis of schizophrenia or schizoaffective disorder at any time during the study period, we queried the management information system of the State Department of Mental Health and Addiction Services. Patients were eligible for the study reported here if they had a diagnosis designated by DSM-III-R or DSM-IV codes 295.1x, 295.2x, 295.3x, 295.6x, 295.7x, or 295.9x. Of the total of 1,670 patients who were eligible based on diagnosis, 15 patients who participated in a pilot trial of treatment effectiveness but who had not participated in an efficacy trial were excluded from further consideration, leaving 1,655 patients.

Of the 137 unduplicated patients who participated in one or more of the treatment efficacy trials, 119 had a diagnosis of schizophrenia or schizoaffective disorder that was entered into the management information system during the study period. This group of 119 patients constituted the trial participants in the analysis reported here. The remaining 1,536 patients with the target clinical diagnosis who did not participate in the clinical trials constituted the comparison group.

The 18 patients who participated in an efficacy trial but carried no confirming clinical diagnosis in the management information system were excluded from further analysis. These 18 patients included three patients for whom no diagnostic information was included in the management information system; six patients with no psychotic diagnosis except psychotic disorder not otherwise specified, three of whom had a comorbid substance use disorder; five patients with affective disorder but no psychotic diagnoses, all with a comorbid substance use disorder; one patient with posttraumatic stress disorder and a comorbid substance use disorder; two patients with only a substance use disorder; and one patient with intermittent explosive disorder and a comorbid substance use disorder.

Among the 119 unduplicated participants with a confirmed clinical diagnosis of schizophrenia or schizoaffective disorder, 49 had participated in standard efficacy trials that selected patients with schizophrenia and no comorbid substance use disorder, and 70 had participated in trials that selected dual diagnosis patients. No patient participated in both a standard trial and a dual diagnosis trial.

All data were obtained from the department of mental health's management information system. Data collected for this analysis included age, gender, race, marital status (never married versus ever married), educational attainment (less than high school versus high school graduation or more), employment status (full time versus unemployed or part time), number of lifetime inpatient hospitalizations in the state mental health system, number of lifetime inpatient days in the state mental health system, and total number of days registered in the state mental health system during the study period. These data reflected information included in the management information system as of October 22, 1997.

In addition, we gathered several types of data based on information from the last clinical contact when the patient carried a clinical diagnosis of schizophrenia or schizoaffective disorder. These data were presence of an axis I diagnosis of paranoid schizophrenia (DSM-IV code 295.3x), severity of psychosocial stressors as indicated on axis IV, score on the axis V Global Assessment of Functioning, and five estimates of comorbidity, including presence of comorbid axis I mental diagnoses, axis I substance use diagnoses, axis II personality disorder diagnoses, axis III general medical conditions and other medical diagnoses, and any comorbid diagnosis.

Univariate comparisons used Student's t test for continuous measures and chi square tests for categorical measures with alpha set at .05, two tailed. To determine the independent and joint effects of variables, we next performed multivariate analyses using logistic regression modeling (11); we entered variables that had greater than 90 percent data completeness for each group, excluding the measure of any comorbid diagnosis. Significance for an individual variable was considered only if the overall regression analysis resulted in a significant model.

Results

Univariate analyses revealed that participants in both standard and dual diagnosis trials, compared with nonparticipants, were more likely to be male, were more likely never to have married, were six to eight years younger, and had a higher total of days registered in the state mental health system during the study interval (see Table 2). Multivariate logistic regression analysis indicated that the significant relationships between participation or nonparticipation and age and length of service were independent from each other and from the effects of other variables entered into the analysis for both types of trial. Gender became nonsignificant in the multivariate analyses partly because of a correlation with age (full sample r=.24, df=1, N=1,653, p<.001).

Participants in standard trials were more likely to be high school educated and more likely to work full time than either nonparticipants or participants in dual diagnosis trials.

The univariate analyses showed that patients participating in dual diagnosis trials were more likely to be nonwhite and had a higher rate of substance use comorbidity and a higher rate of any comorbidity, compared with either nonparticipants or participants in standard trials. Patients participating in dual diagnosis trials also had a lower rate of medical comorbidity and more lifetime hospitalizations than nonparticipants. Multivariate logistic regression indicated that the relationships between participants versus nonparticipants and all of these measures were independently significant.

Discussion

The study reported here compared characteristics of participants in both standard and dual diagnosis trials of the efficacy of treatments for schizophrenia with the characteristics of all of the other patients with schizophrenia and schizoaffective disorder in the clinical population from which the trial participants had been drawn. The principal findings were that participants in both types of trial were six to eight years younger, were two to four times less likely to have ever married, and had evidence of more extensive service utilization, compared with nonparticipants. Participants in both types of trial were also more likely to be male than were nonparticipants, but these relationships were not independent from those of other variables entered in the analysis, particularly age. Participants in standard trials but not dual diagnosis trials were more likely to be high school graduates, compared with nonparticipants, and were four times more likely to work full time than nonparticipants. Participants in dual diagnosis trials but not standard trials were half as likely to be Caucasian, more likely to have clinical diagnoses indicating substance use comorbidity, and less likely to have medical diagnosis comorbidity, compared with nonparticipants.

The principal strength of this study is that our design accounted for all patients with the target clinical diagnoses in the population from which the samples for the efficacy trials were drawn, not just the patients who were screened for the trials.

However, the study design limits the usefulness of the information in several ways. First, the data were restricted to those available in an administrative database. A design that allows planned prospective data collection would be preferable. Second, the design did not permit determination of why nonparticipants did not participate. We are unable to distinguish which nonparticipants were unaware that the studies were available, which were excluded, and which chose not to participate.

Third, it is not clear to what degree the efficacy trials conducted at our center were typical of efficacy trials for treatment of schizophrenia in general. Relatively few of the participants were enrolled in placebo-controlled trials of a primary antipsychotic medication. Fourth, measures such as employment status, marital status, or education could potentially be considered consequences rather than antecedents of participation in an efficacy trial. Unfortunately, the management information system used in this study did not associate these data with a specific date or record of when or whether they were overwritten.

Fifth, multivariate logistic regression to determine which of the significant univariate correlates of trial participation were independent was complicated by substantial missing data for variables such as education, marital status, and work status. Restricting the multivariate analyses to cases with complete data appeared inappropriate, given the reduction in power and the assumption that data cannot be assumed to be missing at random (12). For example, marital status on examination was not missing at random either with regard to trial participation or with regard to age. No generally accepted technique to handle missing covariates in logistic regression currently exists (12). Thus we cannot determine whether education, marital status, or work status relates to trial participation independently from other measures in the dataset.

Trial design factors as well as patient factors are likely to influence trial participation. Moreover, patient factors would be expected to interact with trial factors. We were able to investigate the influence of only one trial factor: whether the criteria for including patients in the trials restricted or encouraged the participation of patients with a dual diagnosis of a substance use disorder. However, other trial characteristics are confounded with the distinction between standard trials and dual diagnosis trials, including whether the medication examined in the trial was the primary antipsychotic, whether patients with schizoaffective disorder were included, and whether inclusion was restricted to patients with predominantly negative symptoms (see Table 1). The influence of trial design features on participation rates is an important topic for future research.

We believe that this report is the first to compare participants and nonparticipants in dual diagnosis trials. We found similarities between participants in dual diagnosis trials and participants in standard trials, compared with nonparticipants, as well as several differences between the two types of participants, as shown in Table 2.

Separation of the efficacy trials into standard and dual diagnosis trials raises a methodologic concern about the comparison groups. Ideally, we would compare participants in standard trials with the subset of nonparticipants with no dual diagnosis, not with the entire group of nonparticipants. Similarly, we would ideally compare participants in dual diagnosis trials with the subset of nonparticipants with dual diagnoses, not with the entire group of nonparticipants. Unfortunately, diagnoses of the nonparticipants were not assessed by research interview as were the diagnoses of participants. The only data available for the whole clinical population were from the management information system. Use of data from the management information system on substance use comorbidity in the multivariate models suggests that the differences we report between participants in dual diagnosis trials and nonparticipants are independent from substance abuse.

In light of this interpretation, it is interesting that a 1995 survey of clinicians at our center who were treating patients with schizophrenia found some differences between patients who had a dual diagnosis and those who did not that were similar to the differences between the dual diagnosis trial participants and nonparticipants that are reported here. In the earlier survey, dually diagnosed patients with schizophrenia were significantly more likely to be male and younger, less likely to be Caucasian, and less well educated, compared with patients who did not have a dual diagnosis (10). It is difficult to reconcile these two studies because of differing methods for assigning diagnoses and because the clinical groups in the earlier study included trial participants.

Several caveats about the data from the management information system on substance abuse comorbidity are important. Dual diagnosis trials required participants with substance abuse comorbidity, except for the seventh trial listed in Table 1, which permitted but did not require participants to have substance abuse comorbidity. However, the management information system reported substance abuse for only 79 percent of the participants in the dual diagnosis trials. Similarly, 22 percent of the participants in the standard trials, which were designed to exclude patients with substance abuse comorbidity, were identified by the management information system as having a substance use disorder.

Overall, the rate of substance abuse comorbidity among the nonparticipants was 28 percent, which was lower than the rate of 45 percent among patients with schizophrenia in our previous survey (10). Several plausible explanations for these inconsistencies can be proposed. For example, clinicians may not add to the management information system information about substance disorders that are known to them, or they may not specify if substance use disorders go into sustained full remission. Inconsistencies could also be the result of the sampling time frame. The data on comorbidity from the management information system reflect the last time during the study interval when the patient carried a target clinical diagnosis, not the time of participation in the study. This sampling time frame was chosen to facilitate comparison with the nonparticipant group, who have no date of study participation.

Our results for participants in the standard trials can be compared with those from previous studies in which the characteristics of participants in efficacy trials for schizophrenia treatment were compared with those of various nonparticipant groups (4,13,14). Relatively few measures are reported in common across studies. Only our study found that participants were significantly younger than nonparticipants, although participants were two years younger than nonparticipants in one study (14). Like the study reported here, earlier studies have suggested that trial participants are more likely to be male (4,13,14) and employed (4,13), less likely to have ever married (14), and more likely to show evidence of more intensive service utilization (4,13,14) than are nonparticipants. However, in the earlier studies not all of these comparisons were statistically significant.

Previous studies of clinical trials involving hospitalized patients (14) and other clinical research (15,16) have suggested that patients with paranoid schizophrenia are less likely to consent or participate than patients with other schizophrenic diagnoses. The results of our study did not support these findings (see Table 2).

Whether the differences we observed between trial participants and nonparticipants are relevant to treatment response or outcome is not clear. Certain characteristics of the trial participants, such as being less likely ever to have married or more likely to be male, predict poorer response or outcome in some studies (17), although one study of the efficacy of clozapine reported a trend for male patients to be more responsive to the medication (18). Other characteristics—such as younger age (19,20), a greater likelihood of having graduated from high school (21), and a greater likelihood of working full time (22)—have been reported to predict better response or outcome.

Conclusions

Participants in efficacy trials for treatment of schizophrenia at our center differed on several measures from the larger nonparticipant group. Differences were also found between the participants in dual diagnosis trials and the participants in standard trials. Some of the observed differences may predict better treatment response or outcome, and some may predict poorer treatment response or outcome.

To the extent that such predictors could cancel each other out, our study results may provide reassurance about the generalizability of findings in efficacy trials for treatment of schizophrenia. On the other hand, these results can be interpreted to support questions about the degree to which findings from efficacy trials may be routinely generalized to clinical practice. We suggest that results from clinical effectiveness trials that systematically sample specified target populations (23,24,25,26) may be needed to supplement the results of efficacy trials.

Acknowledgments

This work was supported by grants R24-MH-54446 and R01-MH-57292 from the National Institute of Mental Health. The authors thank Roy Money and Cheryl Croll for assistance with analyses using the management information system.

Dr. Woods and Dr. Diaz are affiliated with the Connecticut Mental Health Center, 34 Park Street, New Haven, Connecticut 06519 (e-mail, [email protected]) and with the department of psychiatry at Yale University School of Medicine in New Haven. Dr. Ziedonis is with the division of addiction psychiatry at the Robert Wood Johnson Medical School in Piscataway, New Jersey. Dr. Sernyak and Dr. Rosenheck are with the Veterans Affairs Connecticut Healthcare System in West Haven and the department of psychiatry at Yale University School of Medicine. Dr. Ziedonis and Dr. Sernyak were affiliated with the Connecticut Mental Health Center when this work was done.

Enlarge table

Enlarge table