Online Psychosis Screening: Characterizing an Underexamined Population to Improve Access and Equity

A longer duration of untreated psychosis is associated with poorer long-term functioning and treatment outcomes (1, 2). The World Health Organization has recommended to reduce the duration of untreated psychosis to a maximum of 3 months (3). However, in the United States, specialty psychosis treatment typically begins 1–3 years after psychosis onset (2, 4); moreover, the evidence for the effectiveness of strategies aimed at reducing the duration of untreated psychosis has been mixed (5).

One difficulty in reducing the duration of untreated psychosis includes identifying individuals who may be aware of psychosis-like experiences but who have not yet accessed mental health services. Their needs and characteristics remain largely underresearched because investigators typically focus either on general non–help-seeking populations or on participants within psychiatric care (6). This focus has impeded efforts to promptly respond to the needs of persons with psychotic symptoms.

The Internet has become the primary source of mental health–related information for young people (7, 8), representing one of the earliest steps toward active help seeking (9). Consequently, online outreach strategies should be considered as an important avenue to reducing the duration of untreated psychosis. However, although online early psychosis advertising campaigns have reported high click-through rates and screener completions, these actions have not translated to individuals’ engaging with early psychosis services (10). Going forward, it is critical to understand the wants and characteristics of those seeking information about their experiences online to better serve them and to facilitate earlier treatment engagement.

Mental Health America (MHA) is a nonprofit organization dedicated to promoting mental wellness and addressing the needs of those living with mental illness. Their education and outreach activities include online screening tools, educational materials, and support. In 2017, a total of 152,518 psychosis screens were completed via the MHA platform.

This study’s aims were to characterize who is seeking information about their psychosis-like experiences by completing online screeners, to understand what next steps respondents with high scores intend to take, and to determine whether next steps differ among sociodemographic subgroups. Such work is critical to address barriers to services, particularly among those who have been historically underserved in traditional community care, such as Black and indigenous populations and people of color; Asian and Latinx populations; lesbian, gay, bisexual, transgender, and queer (LGBTQ) individuals; and people with lower income (11). As the largest sample of its kind, this data set represented a unique opportunity to understand symptom experiences, help-seeking intentions, and factors that may affect help seeking among individuals who may be in the earliest phase along the pathway to treatment for psychosis.

Methods

Design

This study was a retrospective, exploratory, cross-sectional analysis examining participants’ self-reported psychosis-like experiences and help-seeking intentions. Data were collected via the MHA online psychosis screening survey (at https://screening.mhanational.org/screening-tools). The construct and ecological validity of using anonymous, Web-based screening tools has been previously established (12–15). Previous work in which researchers examined the factor structure of the Pediatric Symptom Checklist by using MHA-collected data was found to have high internal consistency and a factor structure consistent with other studies (16), further supporting data validity.

Prodromal Questionnaire–Brief (PQ-B)

The PQ-B (17) is a 21-item screening tool for identifying individuals who may meet criteria for a psychosis spectrum disorder. It was designed to detect attenuated psychosis consistent with the clinical high-risk syndrome. It is recommended as the first step in a two-step assessment procedure, in which individuals who score above the threshold qualify for a clinical interview.

The PQ-B records psychosis-like experiences over the past month. Item examples include “Have you had experiences with telepathy, psychic forces, or fortune telling?” and “Do you feel like other people are watching or talking about you?” If the respondent does not endorse an item, it is scored zero. For each endorsed item, responders rate accompanying distress or impairment on a scale ranging from 1 (strongly disagree) to 5 (strongly agree). Incorporating distress is important to improve the specificity of the PQ-B as an indicator of psychosis spectrum disorders (17). The PQ-B can be scored by using the number of items endorsed (PQ-B total score); the sum of distress scores (PQ-B distress score); and the number of distressing items, represented as a 4 or 5 (PQ-B distressing items score).

A distress score threshold of 18–24 has been validated as a cutoff appropriate to identify individuals whose condition merits a more comprehensive assessment for psychosis-like symptoms in nonspecific community mental health care settings (18). Consequently, participants in this study who had a total distress score of ≥20 were defined as high scorers, indicating that their condition may be appropriate for further assessment.

Procedure

MHA first hosted the psychosis screener online in July 2015. Study participants included all individuals who completed the screener between January 1, 2017, and December 31, 2017. Individuals selecting the psychosis screen were directed to an online PQ-B. Participants were required to answer all items before they could proceed to the next page. Next, participants were asked optional sociodemographic questions, the format of which is shown in an online supplement to this article. Race and ethnicity were combined into a single variable, defined as how the respondent self-identified. Finally, users were asked what next step they planned to take and what postscreening support they would consider most helpful. Options for the next step included “Discuss the results with a family member, a friend, or a professional”; “Find additional information online”; “Find treatment”; “Monitor health by taking screens regularly”; and “None at this time.” Only one option could be selected.

After completion, the screening results were presented. Participants with a score <20 were presented with the following text:

Participants with a score ≥20 were presented with the following text:

The date, time, and Internet protocol (IP) address were collected each time the screener was completed. These metadata were reviewed for data integrity, both manually and via plugins. Multiple screens from the same IP address or screens from known suspicious IP addresses were flagged for review and, when necessary, were removed and blocked. The IP address of the device used was anonymized before being included in the data set used in the analyses. To support data integrity and the independence of responses, when multiple surveys from the same anonymized IP hash were completed, we retained only the most recent response. Respondents were notified in the privacy policy linked to the survey (at https://www.mhanational.org/privacy-policy) that their deidentified data may be used for research purposes. The research team did not have access to any identifiers or keys, meaning the study was exempt from institutional review board evaluation.

Analysis Plan

Summary statistics were used to examine the proportion of high scorers and the planned next steps. Logistic regression was used to explore the relationship between the different PQ-B scores (independent variables) and planned next steps (dependent variable). Univariable and multivariable linear regressions were used to explore the association between the demographic variables of age, race-ethnicity, household income, and LGBTQ status, both independently and in the same model, with PQ-B distress score as the dependent variable. We then replicated this analysis as a logistic regression by using the dichotomous planning to seek treatment (yes or no) as the dependent variable. Data from participants with any missing data were excluded from that analysis. Analyses were performed with STATA, version 14.0.

Results

In 2017, 152,517 MHA surveys were completed. Responses from 23,569 participants were removed because of multiple submissions from the same IP address; in addition, a server error between May 23, 2017, and June 12, 2017, led to a loss of 8,011 responses, resulting in a final sample of 120,937 respondents whose characteristics are presented in Table 1. Most participants were female (65.5%), ages 11–24 years (70.1%), and non-Hispanic White (65.9%), with a yearly household income of ≤$39,999 (59.6%); 18.1% identified as LGBTQ. Approximately half (50.5%) reported having never been diagnosed as having a psychiatric disorder.

PQ-B Scores and Plans for Help Seeking

Of the 120,937 participants whose data were included in this study, 82.1% (N=99,314) reported a total distress score of ≥20. Among those who met this threshold, the distress experienced was high (mean±SD distress score=48.5±18.9). Of these participants, 56,597 (57.0%) reported their planned next steps, of whom 30,312 (53.6%) reported that their next step would be to seek more information or advice by discussing their symptoms with family, a friend, or a professional (N=16,384, 29.0%); to look for additional information online (N=11,342, 20.0%); or to conduct ongoing health monitoring by completing regular screens (N=2,586, 4.6%). However, 16,584 participants (29.3%) reported that they planned to take no further action. Only 9,701 (17.1%) reported that their next step would be to seek treatment.

Participants who endorsed more PQ-B items, experienced higher total distress, and experienced more distressing symptoms were significantly more likely to intend to seek treatment, although the strengths of those associations were weak (B=0.009, SE=0.003, 95% confidence interval [CI]=0.004–0.014, pseudo R²<0.01; B=0.009, SE=0.001, 95% CI=0.008–0.010, pseudo R²=0.01; and B=0.055, SE=0.002, 95% CI=0.050–0.059, pseudo R²=0.01, respectively).

PQ-B Scores and Help Seeking by Gender Identity and Sexual Orientation, Race-Ethnicity, and Household Income

PQ-B scores across age, income levels, race-ethnicity, and LGBTQ status are presented in Table 2. The univariable and multivariable associations between sociodemographic variables and PQ-B distress scores are presented in Table 3. In the univariable analysis, younger respondents, those from lower income households, and LGBTQ respondents all reported significantly higher distress from psychosis-like experiences. Asian-Pacific Islander respondents were more likely to report lower distress relative to non-Hispanic Whites, whereas Native Americans and people who identified as more than one race reported significantly higher distress than other racial-ethnic minority groups. These findings were consistent in the multivariable model when all sociodemographic data were entered into one model.

The univariable and multivariable associations between sociodemographic variables and respondents planning to seek treatment among those who screened positively on the PQ-B are presented in Table 4. In the univariable analysis, younger clients, respondents with high household income, and LGBTQ respondents were significantly less likely to plan to seek treatment next. Across race-ethnicity, Hispanic-Latinx and Black-African American respondents who screened positively were more likely to plan to seek treatment relative to all other racial-ethnic minority groups, whereas Asian-Pacific Islander respondents and people who identified as more than one race were less likely to seek treatment next. In the multivariable analysis, these findings were broadly consistent, although differences between individuals who identified as having more than one race and LGBTQ respondents were no longer significant, likely because both groups contained a disproportionately large number of young participants who were much less likely to seek treatment next.

Discussion

Main Findings

Most respondents (82.1%) of the online psychosis screening survey offered by MHA reported distressing psychosis-like experiences at levels sufficient to merit specialty care referral for further evaluation. However, most of these respondents wanted additional information, rather than treatment, as a next step. Additionally, 87.8% of respondents reported being between ages 11 and 34 years, falling within the age range typically used as an eligibility criterion for admission to early psychosis programs (19). This finding suggests that MHA’s online screening program is attracting an important population: young people with significant target symptoms who may be eligible for early psychosis programs but who are still contemplative about seeking care. These findings suggest that online screening programs, when implemented successfully, could play an important part in identifying individuals who may be eligible for services. Additionally, with so many respondents electing not to seek traditional services despite experiencing prominent distressing symptoms, these findings raise important questions around how early psychosis services should be presented and delivered. Finally, discovering significant differences in symptoms experienced and treatment wants across subgroups is informative to help programs better understand and meet the treatment needs of those who reported distressing psychosis-like experiences but who have yet to access care.

Strengths and Limitations

Our sample comprised many participants experiencing distressing psychosis-like experiences who were not necessarily actively receiving mental health care services (>50% reported never having a diagnosis of a behavioral health disorder). At present, much of the literature in this area focuses on non–help-seeking community samples with generally low symptom levels or those already engaged in mental health services (6). Consequently, this study provides valuable insights into an underresearched population about whom more information is critically needed to address the needs of those currently unserved. With >120,000 participants included in this data set, it likely represents the largest compiled data resource on individuals who reported psychosis-like experiences and likely were not receiving mental health care. This study therefore represents an unprecedented opportunity to understand the needs and experiences of this population.

However, this study also had some limitations. First, participants were asked what next steps they planned to take after screening, but follow-up data regarding whether the individuals actually committed to these, or other, steps were unavailable. However, the information about considered next steps we collected in this study can be helpful in addressing the ambivalence regarding help seeking that is a barrier to reducing the duration of untreated psychosis. Additionally, the focus was on the step that respondents planned to take next, rather than all steps they planned to take. Therefore, some individuals could have planned to seek more information first and seek treatment afterward, leading to underreporting of treatment-seeking intentionality. This issue may particularly apply to younger participants who may want parental support or require parental consent before contacting early psychosis services. Second, although respondents reported whether they have ever had a behavioral health diagnosis, information on receipt of services was not collected. Third, race is a social construct, and thus, information gathered on race-ethnicity may have lacked precision and may have limited the ability to interpret these data. Similarly, demographic categories such as Asian-Pacific Islander and LGBTQ populations represent many subgroups, limiting the ability to generalize to other populations. Fourth, we note that a high proportion of sociodemographic data were missing, as detailed in Table 1, which could have affected the association between symptom scores and respondents’ planned next steps. Fifth, some respondents may have fabricated answers out of curiosity. However, the items most frequently endorsed (items related to suspiciousness and persecution as well as something being wrong with their mind) and least endorsed (items related to telepathy, grandiosity, and visual-somatic disturbances) are consistent with data collected in other help-seeking samples (17), suggesting that the data reflect meaningful responses in most cases.

Finally, without follow-up interviews, we could not verify that endorsed items represent clinically meaningful symptoms or, specifically, psychosis-like symptoms. However, when substantial distress was endorsed, we did assume that further psychiatric assessment would be appropriate. Often an assessment is needed, and helpful, to conduct the complex differential diagnosis of the clinical high-risk state for psychosis versus other psychiatric issues. Additionally, we recommend caution in labeling respondents with high scores as “at risk” to avoid undue anxiety and stigma. The website notes that the person “may” be experiencing symptoms that could benefit from further evaluation.

Implications

That most participants who experienced clinically noteworthy psychosis-like experiences did not plan to seek treatment next, and that many planned to do nothing, raises an important challenge to researchers and providers who have struggled to develop effective strategies for reducing the duration of untreated psychosis (5). Furthermore, greater distress was only weakly associated with planning to seek treatment next, indicating that even substantial distress is insufficient to spur help seeking. Notably, this trend was particularly acute among younger participants, who represent the target group for most early psychosis programs.

In the National Comorbidity Study (20), individuals who recognized that they needed help but who elected not to seek treatment cited wanting to solve the problem alone, thinking that the problem would get better without help from others or that treatment would be ineffective. This finding suggests that better education about the effectiveness of early psychosis care may be helpful (2). Alternatively, treatment ambivalence may be related to being in an earlier (i.e., contemplative) stage of change (21). Some individuals may be more receptive to support perceived as less threatening or changed focused (e.g., peer support), which could, in turn, facilitate more change-focused care at a later time (22). Additionally, some individuals may prefer digital support to traditional in-person services (23). It is important to gain a better understanding of which service structure and composition clients prefer.

Given the number of people seeking online screening for mental disorders, expanding digital services for this population may be warranted (24, 25). Although obstacles to telehealth and asynchronous digital services were previously present, the COVID-19 pandemic has reduced service system and payor barriers, expanding future opportunities for digital health (26). Such an approach could minimize cost-related barriers, helping to address health care disparities and inequities worldwide (27). This approach may also help reduce concerns about limited insurance coverage and care costs that can limit help seeking (28).

In the subgroup analysis, African American respondents were most likely to report that they were planning to seek treatment next, independently of age and household income. This finding may appear counterintuitive given persistent disparities and inequities in mental health care utilization between non-Hispanic Whites and African Americans in the United States (29, 30). However, our finding is consistent with recent literature suggesting that although African Americans face greater logistic, provider, and linguistic barriers to accessing community behavioral health services, they may have more positive attitudes toward mental health treatment compared with non-Hispanic Whites (31–33).

Although earlier studies have suggested mistrust of systems of care among African Americans (34), emphasis on this barrier does not adequately consider provider bias and structural factors (including discrimination experienced in health care treatment seeking) that may limit mental health care access (35). Therefore, efforts to address both stigma and structural racism are necessary to improve mental health care access. These efforts would include addressing financial, cultural, and linguistic factors impeding access to care; availability of providers in communities of color; and providers incorrectly assuming that persons from specific minority groups are more resistant to mental health treatment (32).

Conclusions

Many respondents in this online survey reported substantial distress caused by psychosis-like experiences, but most did not plan to seek treatment as a next step. Assuming that some of the respondents are experiencing psychosis or are at clinical high risk for psychosis, we conclude that this inaction could lead to prolonged durations of untreated psychosis and, consequently, to poorer long-term outcomes. Therefore, it is important to consider what early psychosis services individuals want, how such services are presented, and what barriers exist that cause people experiencing psychosis-like symptoms not to seek care. Additionally, providers must address the social and structural barriers that lead to disparities and inequities in service utilization, particularly when the inclination toward seeking care appears to be higher among some historically underserved groups. Most clearly, our findings support the development of targeted online education and support resources for individuals experiencing psychosis-like experiences that engage people outside traditional clinical settings and that meet them where they are.