Self-Report Instruments for Clinical Monitoring of Psychosis Risk States

Evidence that early intervention in the course of psychotic illness leads to better treatment and quality-of-life outcomes has spurred efforts to identify “prodromal” or “clinical high-risk” (CHR) phases of illness. For patients who present with putatively prodromal symptoms, the American Psychiatric Association recommends “careful assessment and frequent monitoring” until symptoms remit or evolve into a “diagnosable and treatable mental disorder” (1). This “watch and wait” approach is also endorsed by the International Early Psychosis Association, Canadian Psychiatric Association, and German Association for Psychiatry, Psychotherapy, and Neurology (2,3). Until high-quality clinical trials have firmly demonstrated the benefits of further interventions, clinical monitoring of CHR symptoms accompanied by evidence-based interventions for comorbid problems or diagnoses will likely continue to constitute the gold standard of care for this population (3). Furthermore, for patients who are receiving mental health interventions and for whom the focus of treatment is either CHR symptoms or comorbid psychiatric disorders, routine outcomes monitoring may constitute an important component of evidence-based care and a valuable mechanism for patient-provider feedback regarding treatment effectiveness (4).

Despite the apparent consensus that patients with CHR symptoms should be closely monitored to detect worsening of symptoms or transition to psychosis, no self-report tool has been validated for this purpose. Studies have used semistructured interviews, such as the Structured Interview for Prodromal Syndromes (SIPS) and the Comprehensive Assessment of At-Risk Mental States, to track symptom course and progression of illness (5), but readministering lengthy interviews at frequent intervals is impractical in clinical settings. Brief, standardized self-report questionnaires may aid substantially in efforts to offer standardized monitoring of CHR symptoms. To that end, this study examined the effectiveness of self-report questionnaires to detect change in symptoms among CHR youths and a naturalistic comparison group receiving treatment over the course of six months.

Methods

Procedures were reviewed and approved by the Institutional Review Boards of the University of Maryland (UM) School of Medicine and University of Maryland, Baltimore County (UMBC). To achieve the study’s aims of contextualizing youths with psychosis risk symptoms within a naturalistic clinical comparison group, participation was open to any teen or young adult (ages 12–22) receiving mental health services at the time of consent. Participants were recruited through clinician referrals and flyers posted at UM and UMBC. Some, but not all, clinician referrals to the study were intended as psychosis consults. Participants provided informed consent and then completed a form soliciting demographic information and the three self-report tools. After encouraging youths to complete the measures on their own, study staff assisted teens with attention or literacy issues by reading items aloud. Participants were then administered the SIPS by trained study staff. Cases were reviewed weekly in team meetings to ensure reliability. As an additional check on reliability, ten audio-recorded interviews were randomly selected for rereview by the assessment team. Interrater agreement for these ten interviews was high (intraclass correlations of .82 for positive symptom ratings and .84 for all symptom ratings; κ=1.00 for diagnosis). Participants were invited to complete a follow-up assessment after six months, at which the self-report tools and the SIPS were readministered. Data were collected between December 2010 and April 2014.

The self-report measures used in this study were developed as psychosis risk “screeners” and included the Prime Screen–Revised (PS-R), the Youth Psychosis At-Risk Questionnaire-Brief (YPARQ-B), and the Prodromal Questionnaire–Brief Version (PQ-B) (6–8). Details about validation and scoring have been published elsewhere (9). [A table with further details about these instruments is included in an online supplement to this report.] The clinician-rated SIPS is a semistructured interview targeting interviewees’ experiences of attenuated symptoms and other indicators of psychosis risk (10). SIPS subscales include positive, negative, disorganization, and general symptoms, with positive symptoms figuring most heavily into the conceptualization of risk syndromes. SIPS diagnostic categories include presence of psychosis, presence of psychosis risk syndromes, and no diagnosis. The positive symptom subscale of the clinician-rated Scale of Psychosis Risk Symptoms (P-SOPS) was used as a criterion measure of positive symptoms; this subscale comprises clinician ratings reflecting five positive symptom domains scored on a 7-point scale.

Descriptive data and histogram visualizations identified three participants as likely “random responders”; these individuals were dropped from the sample before any analyses were performed. Potential ordering effects were examined via a series of analyses of variance (ANOVAs) by comparing means for self-report measures administered first, second, and third in the battery. Data were screened for normality, and statistical tests were chosen based on approximately normal distributions.

No clinician-scored data were missing, and 1% of self-report data were missing. Missing cell values were treated as zeros (indicating that the symptom was not present) for the YPARQ-B and PS-R. For the PQ-B distress scale (which uses symptom distress ratings in addition to presence-absence scores), missing data were scored 3, indicating that the symptom, if present, was considered neutral. Some respondents missed either a page of a measure or a measure entirely, and in these cases the respondent was dropped from the analyses using that measure.

To evaluate the degree of agreement between baseline and six-month follow-up self-report scores, correlations were calculated to reflect the relatedness of baseline and follow-up scores for each measure. On the basis of changes in scores on self-report forms and clinician-rated interviews, participants were assigned a change score reflecting increases or decreases in scores on each measure. Change scores for each self-report measure were tested by using t tests to determine whether mean changes on each measure were significant over time. Self-report change scores were then correlated with P-SOPS change scores to examine screeners’ sensitivity to clinician-rated change over time. These analyses were performed for the full sample and repeated in subsamples of only high-risk (CHR) and only low-risk participants. R to Z tests were run to determine whether differences between correlation magnitudes were statistically significant (11,12). Analyses were conducted with SPSS-22.

Results

The analysis sample included 54 participants who completed both baseline and follow-up evaluations. The 54 participants were from an initial pool of 85 participants who completed baseline evaluations and did not meet criteria for a psychotic disorder on the basis of their baseline SIPS score. Of these 85 participants, 56 (66%) completed six-month follow-up assessments. Completers did not significantly differ from noncompleters with regard to sex, age, or baseline SIPS diagnosis (low risk versus high risk).

Most participants were female (N=35, 65%). The sample was racially diverse (African American, N=28, 52%; white, N=17, 31%; more than one category or other, N=7, 13%; and American Indian, N=2, 4%). Mean±SD age at intake was 16.20±3.07. Most participants (N=28, 52%) reported a family income below $40,000 annually. Eleven participants (20%) were recruited from the UMBC campus, and the remaining 43 (80%) were recruited or referred from community clinics. Data distribution was approximately normal (skew and kurtosis ≤1.1) across all measures except for the PS-R at follow-up, and thus Pearson correlations were chosen for analyses.

At baseline, 33 participants (61%) were considered low risk according to SIPS criteria, and 21 (39%) met criteria for one or more CHR diagnoses. At follow-up, 32 participants (59%) were low risk, 18 (33%) met CHR criteria, and four (7%) met SIPS criteria for new-onset psychosis. Of those who were initially low risk, none transitioned to psychosis, 28 (85%) remained in the low-risk group, and five (15%) met SIPS CHR criteria at follow-up. Of the 21 participants who met SIPS CHR criteria at baseline, four (19%) had transitioned to psychosis by follow-up, four (19%) experienced symptom remission and were low risk at follow-up, and 13 (62%) continued to meet CHR criteria at follow-up.

To examine the degree of stability in self-report scores relative to the stability of clinician-observed symptoms, Pearson correlations between baseline and follow-up measures were calculated. Average change in total score was calculated and compared with zero by using a one-sample t test, and change scores for self-report measures were correlated with P-SOPS change scores. Results are presented in Table 1. ANOVAs exploring the possible impact of ordering effect did not detect significant findings.

Discussion and Conclusions

Despite the small sample size, the data yielded interesting preliminary findings. Self-report measures demonstrated a high degree of stability over the follow-up interval. Notably, clinician ratings were highly consistent from baseline to follow-up for participants who met CHR criteria but not for participants deemed low risk, suggesting a higher level of symptom stability and persistence among those with more severe symptoms at baseline. Both self-report and P-SOPS scores indicated slight decreases in participants’ average symptom severity from baseline to follow-up. This change may reflect regression toward the mean, positive treatment effects (all participants were receiving mental health services), or a tendency among clinicians to refer participants for evaluation at a time of peak concern. Despite this overall pattern of symptom improvement, a few individuals experienced marked symptom exacerbation between time points, with four transitioning from a CHR state to full-threshold psychosis by follow-up. Most CHR participants continued to meet SIPS CHR criteria at follow-up, indicating some persistence of symptoms over time for most individuals in this group.

Of the three self-report measures tested, the PQ-B change score demonstrated the highest correlation with the P-SOPS change score (r=.34 in the full sample and .43 in the CHR subsample), suggesting that the PQ-B may be most sensitive to clinician-detected changes over time. In other words, over repeated administrations, respondents’ endorsements on the PQ-B were most likely to “track” with the interviewing clinicians’ impressions. Thus this instrument may be the best fit for monitoring symptoms at six-month intervals. This may be due to the PQ-B’s inclusion of the distress scale, which gives respondents an opportunity to report how much symptoms are currently bothering them (rather than simply whether symptoms have occurred), similar to how clinicians conceptualize symptom severity scores. The PQ-B’s potentially greater sensitivity to change may also be attributable to the PQ-B’s instructions that respondents think about their experiences over the past month, whereas the other instruments direct respondents to answer items on the basis of their experiences over the past year (PS-R) or “the kind of person you generally are” (YPARQ-B).

When analyses were repeated only for the CHR subsample, the associations between PQ-B and PS-R change scores and P-SOPS change scores became somewhat stronger (that these correlations were not statistically significant likely reflects the small sample size and lack of adequate statistical power for subsample analyses). The stronger associations could indicate that in the CHR group, changes in self-report scores may be more clinically meaningful than in the low-risk group. Especially in the low-risk group, shifts in psychotic-like symptoms may reflect changes in other conditions, such as mood, anxiety, or substance use disorders.

Although several ongoing clinical trials aim to establish treatment protocols for youths with prodromal symptoms (13,14), practice guidelines for CHR individuals emphasize watchful waiting. This study had several limitations, chiefly the small sample and limited duration of follow-up (larger studies of CHR youths have found that conversion to psychosis may take place several years after initial presentation of attenuated positive symptoms, thus necessitating a potentially longer period of clinical monitoring). The sample was primarily female, which is somewhat unusual for samples of CHR youths and may have influenced self-report or interview responses. Despite these limitations, results indicate that questionnaires developed as psychosis risk screeners could be used for the purpose of symptom monitoring as well as to predict transitions to psychosis (15).

Future directions for research that could lead to improvements in accuracy of symptom monitoring include identifying key items useful for detecting and monitoring symptoms over time, validating monitoring tools over longer periods of follow-up, and investigating more comprehensive self-report inventories as alternative tools for monitoring (for example, the Peters Delusional Inventory, the 92-item Prodromal Questionnaire, and the Community Assessment of Psychotic Experiences). Further validation of self-report tools to track the severity of attenuated symptoms over time will be a valuable step toward developing an evidence-based approach for treating high-risk youths.