Acceptability of Psychosis Screening and Factors Affecting Its Implementation: Interviews With Community Health Care Providers

The duration of untreated psychosis predicts future clinical and functional impairment (1,2), affecting outcomes years later (3,4). Starting treatment within three months of psychosis onset is recommended (5), given that outcomes are significantly poorer once this period is exceeded (6). However, initiation of specialist treatment in the United States is delayed, on average, between one and three years (7,8). Early psychosis services have been developed with the aim of reducing the duration of untreated psychosis (5). However, they have not been found to significantly reduce delays (9). The impact of campaigns to provide community education and raise awareness about untreated psychosis has been mixed (10).

Studies examining care pathways to early psychosis clinics suggest that delays within health services account for the greatest contribution to untreated psychosis (11) and are largely due to underdiagnosis in nonspecialty care (12). Consequently, implementing screening for psychosis in nonspecialty settings, as is recommended practice for depression (13), may be one method to reduce the duration of untreated psychosis. In a recent trial (14), screening for psychosis risk detected a threefold higher prevalence of persons at high risk of psychosis in community mental health settings compared with standard care in addition to identifying a significant number of individuals whom they considered “fully psychotic” but who had not been previously identified as having psychosis. Therefore, screening for psychosis may represent a fast, effective solution to identification of early psychosis, without the need for additional expertise. However, the acceptability of psychosis screening, as well as barriers to and facilitators of its implementation, have not been explored.

In this study, we interviewed providers in school counseling services and community mental health clinics who screened for psychosis with a computer tablet–based screening tool as part of a research trial. The community mental health clinics adopted universal screening, and the school counseling services screened all individuals who were evaluated as part of special education assessments or who were seeking behavioral health services. These settings were selected because both have been identified as locations where a comprehensive early psychosis detection strategy could help reduce the duration of untreated psychosis (12,15,16).

Prior to starting the screening program, site staff received training on how to use the tablet, submit the referral, provide answers to commonly asked questions, and introduce the tablet to clients or guardians. Training refreshers were offered on an annual basis. In the school setting, staff social work trainees provided the tablets to the clients. In the community mental health clinics, the tablets were provided by receptionists or clinical staff, dependent on each site’s preference.

On presenting at the referral site, all clients ages twelve to 30 were asked to provide consent prior to completing the Prodromal Questionnaire–Brief (PQ-B) (17) on a tablet. School sites obtained parental consent prior to presenting the individual with the tablet. Each screen was then assigned a unique study identifier, and all data were encrypted on the tablet. If a client’s total distress score was 20 or higher, the screening device instructed the provider to refer the individual to the local early psychosis service for a comprehensive assessment of psychosis-like symptoms. Following consent from the client (or a guardian if the client was a child), the provider faxed a referral to the University of California, Davis, Early Detection and Preventative Treatment early psychosis clinic, and a clinician from this service contacted the client via telephone to conduct an assessment. Clients who refused to continue at any stage of the screening or referral process continued to receive services with their current provider.

In this study, we evaluated staff experiences of implementing this screening procedure in order to determine acceptability of the intervention and to identify barriers to and facilitators of implementation.

Methods

Design

We conducted qualitative, semistructured interviews to examine providers’ experiences of implementing an electronic version of the PQ-B (17) as a screening tool for psychosis. Interviews were conducted at sites implementing a clinical trial evaluating psychosis screening as a method to reduce the duration of untreated psychosis and took place in July and August 2016. We analyzed participants’ responses by using an inductive approach to thematic analysis.

Two investigators were involved in each interview, one with significant experience of the main screening trial (HVS) and one who was more removed from the process (MS or HHG). HVS is the study coordinator for the screening program. MS is a research fellow with experience in running clinical trials evaluating complex interventions and in conducting qualitative interviews. HHG, a psychiatrist and social scientist, has extensive services research experience. Prior to conducting the study, all procedures were approved by the University of California, Davis, Institutional Review Board.

Participants

Investigators recruited participants from screening sites across the Sacramento, California, area. Purposive sampling was adopted, recruiting staff members in a broad range of roles, including clinical, management, administration and support staff. The only eligibility criterion was being involved in, or managing someone involved in, at least one aspect of the screening procedure. Consenting participants were interviewed at their respective clinical sites or schools.

Topic Guide and Data Analysis

Prior to the interviews, we developed a topic guide for conducting the interviews. After the preliminary interviews were completed, the guide was reviewed and refined, with greater focus given to emerging themes as more interviews were completed. [A copy of the staff interview topic guide is available as an online supplement to this article.] Interviews were recorded and transcribed, with identifiable data removed prior to transcription.

Through an iterative process, the investigators conducted the interviews and analyzed the data. After each batch of interviews was completed, the research team met to identify preliminary themes, generate hypotheses, and further develop the topic guide. This process was repeated until saturation of the main topic areas was reached. During analysis, a series of preliminary codes were developed to identify patterns in the data. From these codes, a set of emerging themes was identified and refined, before eventually being finalized. Four researchers were involved in developing and refining the coding framework.

Results

The screening was implemented across two schools and five community health centers, with 584 screens completed between June 2015 and August 2016. Investigators interviewed 17 participants, including two clinical managers, one psychiatrist, six clinicians, one school psychologist, one coordinator of student support services, two senior office assistants, one mental health assistant, two chart room managers, and one AmeriCorps volunteer at a community mental health clinic.

Acceptability of Screening

Participants were generally positive about the screening procedure. Reported advantages included faster identification of clients with psychosis, increased confidence in clinical judgment, and a clearer pathway to specialty services compared with standard care, where referrals are submitted based on clinical judgment alone. Twelve participants reported that implementing the procedure did not significantly affect their overall workload, whereas four reported that it had led to an increase in their workload.

Improved client identification.

Seven participants indicated that screening identified patients whom they would have missed otherwise or detected them faster than would be the case in standard practice (“I feel like it is definitely picking up on it faster, and I feel like it does do a much more thorough job than maybe other clinicians may do during their assessment process.”). Participants reported that psychotic symptoms were previously not typically addressed during the initial assessment, unless they were mentioned in the referral or reported unprompted by the clients themselves.

Increased confidence in clinical judgment.

Three participants reported that a positive screen was helpful in validating the providers’ clinical judgment, supporting their decision to refer a client to specialty services. Some participants appreciated that screening produced quantifiable data, which could be used in conjunction with clinical judgment to initiate referrals more quickly. Others highlighted that the screening provided a more comprehensive evaluation of more subtle symptoms that otherwise may have been missed. For example, one participant, a school psychologist, said, “I think that the tablet is a useful tool in terms of data collection as part of a battery of assessments and just having clear numerical quantitative data to say, ‘Okay, this is what he's saying. . . .This is what she's saying. Here's the numbers to back it up. Let's get the ball rolling.’”

A clearer pathway to specialist services.

Three participants reported that the process helped to establish a clear pathway to specialty care. Being involved in the screening program provided important information on the nature of the services available, the eligibility criteria, and the means to make a referral when necessary. Such experiences highlight the importance of an ongoing partnership between screening sites and specialty care services that can accept referrals following a positive screen. “I think it matches what it's intended for—early detection and giving them that pathway over to you guys that they might not otherwise get,” said participant 2, a clinician.

Negative aspects of screening.

Most participants reported that the screening and referral procedure was not difficult to manage alongside their current work schedule. However, three were concerned that screening added to an already busy assessment process. These participants reported that although the assessments were short, any additional work felt overwhelming. Finally, one participant, a coordinator of student support services, reported that a social work trainee was uncomfortable with administering the tool because of concerns that the individual might be labeled as “crazy.” The participant suggested that the trainee’s concern may have been attributable to a lack of information about psychosis and the study: “A few [trainees] had their own personal biases about the study without having a lot of . . . I don't believe he had a lot of information. But it was kind of categorized as a way to identify students who are ‘crazy.’”

Barriers to and Facilitators of Implementation

Participants reported a number of barriers and facilitators relevant to various stages of the screening process. The various barriers and facilitators, characterized as service-, client-, and program-related factors, are summarized below. Some factors, such as the introduction of the screening tool, were considered to be relevant to only one stage of the procedure. Others, such as the impact of staff training, were considered significant at multiple stages. Figure 1 identifies the various stages of the screening procedure and the points in the process during which clients were more likely to drop out. Table 1 presents a number of solutions to barriers at each stage of the process, as proposed by participants, and quotations supporting the solution. [More supporting quotes on the themes described above are available in the online supplement.]

Service-Level Factors

Support by leadership.

Seven participants reported experiencing strong leadership support for the screening program, and most considered the support important to the program’s successful implementation. Examples of support by management included adding the screening procedure to assessment checklists to improve completion rates, meeting with senior administrative staff to work through implementation issues, tailoring referral forms and procedures at the site level to make it easier for staff to refer patients following a positive screen, and meeting regularly with staff to ensure that the screening was being completed.

Organizational issues.

Organizational factors were identified either as barriers to or facilitators of successful implementation by four participants. At one site, a different staff member was responsible for providing clients with the tablet, completing the referral form, and faxing the form to the specialty care service, and this diffusion of responsibility led to some referrals not being sent. Participants suggested that reducing the number of people involved may improve the likelihood of the referrals being submitted. Another participant reported that high staff turnover and structural changes made consistent implementation of any new procedures difficult. Regarding other potential facilitators, incorporating the screening procedure into existing checklists, which can be reviewed as part of ongoing procedures, was considered helpful.

Client-Level Factors

Symptoms, low functioning, and ongoing life stressors.

Seven participants reported that some clients refused to engage in the screening process because of suspiciousness, anxiety, and poor general functioning. In some cases, clients and families were experiencing a period of crisis, leaving them feeling too overwhelmed to take part. One participant reported significant challenges in trying to engage her clients in a telephone assessment because of their high disorganization and low functioning. In such situations, it was suggested that greater outreach from specialty services, more support from family, or both were necessary to engage clients.

Clients’ wish to avoid changing services.

Nine participants reported working with at least one client who expressed resistance about receiving care from a different service, sometimes leading to refusing a referral. Ambivalence was attributed to a preference for current services, convenience, hesitation about receiving any treatment, concern about the assessment length, and treatment delay. Some participants suggested that this barrier could be mitigated by carefully explaining the nature and reason for referral and ensuring rapid access to the specialty treatment provider.

Program-Level Factors

Introduction to screening.

Twelve participants discussed issues related to how screening was introduced, or by whom. Although it was not necessary for trained clinicians to introduce screening, most participants suggested that the provider who dispenses the tablets should have enough experience in discussing mental health issues to answer the typical questions of clients and guardians. Adopting a personal approach was consistently recognized as an important facilitator for addressing client ambivalence, anxiety, and suspiciousness and was typically preferred over the scripts that were provided during training. Some participants avoided using “psychosis” and similar terms, believing that doing so could lead clients to disengage from the process because of stigma around the illness.

Use of a technology-based device.

Eight participants reported that clients appreciated using a tablet for screening and preferred this method to paper-based questionnaires. The main benefits cited were ease of use, speed of completion, and the novelty factor of using a tablet. One participant reported having difficulty in using the tablet because of a lack of experience with such technology.

Training.

Tablet training and the user manual were well received. Four participants noted that the training and the manual helped with understanding the procedures, learning how to introduce screening, and answering potentially difficult questions. In school settings, role-playing of common client and family concerns during staff training was considered by one participant to be an important tool in learning to address difficult situations. Regarding barriers, in sites that typically experience high staff turnover, such as teaching hospitals, infrequent training was considered problematic, given that newer, untrained staff were less likely to adhere to screening protocols.

Communication and access to information.

One participant in a managerial role reported that collecting data on the number of referrals faxed to the specialty care site by each staff member would help improve screening implementation. Another participant suggested that having more information about the outcome of the full evaluation would be a motivating factor to continuing the screening program.

Discussion

Main Findings

Staff who implemented a screening program for psychosis reported that the program had a number of advantages compared with standard care, including improved detection of psychosis and increased confidence in the clinical judgment of staff. The provision of a clearer pathway to specialty care was also considered important, highlighting the importance of an ongoing partnership between the screening sites and specialty care. Most participants reported that they were able to incorporate screening into existing practice without a significant increase in workload, although one-quarter were concerned about increased workload.

Regular training, leadership support, the novelty of using a tablet-based screening tool, and careful introduction of the screening improved client uptake in the program and reduced dropout at each stage of the process. These findings are consistent with other studies evaluating other screening programs for different conditions (18,19). Some of the barriers identified were related specifically to screening for psychosis, including symptoms such as suspiciousness, stigma, and impairments in functioning, which affect the client’s ability to successfully engage. To address these barriers, it is likely that providers will require more knowledge and expertise in discussing mental health issues relative to screening programs for other, less severe behavioral health concerns.

Although the experiences reported by participants from community mental health clinics and school counseling services were broadly consistent, it is notable that school-based participants raised concerns about a lack of staff knowledge and the need to incorporate role-playing into training. That is perhaps unsurprising, given that school staff would be less likely to come into contact with persons with psychosis and that the school counselors who administered the screening were trainees with limited clinical experience and expertise. As a result, more training or supervision may be necessary for the successful implementation of screening programs for psychosis in school settings compared with sites such as community mental health clinics.

In another significant finding, different barriers and facilitators were implicated in breakdowns at various stages of the screening procedure. Barriers and facilitators considered important in one stage of the screening and referral process may be less important at a different stage. Consequently, these results have important implications for how health screening programs should be evaluated and implemented. Determining the effectiveness of each stage of the screening process separately (for example, completion of screening, agreeing to the referral being sent, and successful engagement with the new service) may be one way to better understand how various factors affect the screening program, allowing for more targeted solutions to improve screening implementation.

Strengths and Limitations

Although studies have examined the barriers to and facilitators of screening for conditions such as perinatal depression (18), trauma (20), developmental and behavioral delays (21), and substance abuse (19), to our knowledge this is the first study to examine the effectiveness of psychosis screening in community settings. We interviewed participants in various clinical, managerial, and nonclinical roles, capturing a broad range of experiences related to screening implementation.

In regard to limitations, it is unclear whether the themes identified would generalize to other settings, such as primary care, to other psychosis screening tools, or to standard clinical practice (where additional research staff support would not be available). Given the interviewers’ association with the screening program, it is possible that participants felt less comfortable highlighting negative aspects. However, participants received no individual or organizational incentives for taking part in the screening project, and some agreed to be interviewed with the expressed aim of highlighting problems. Additionally, one aim of the investigation was to identify problems with implementation, potentially leading the interviewers to focus more on the more negative appraisals of the program. Finally, the clients themselves were not interviewed, meaning their perspectives could not be considered.

Conclusions

The experiences of staff members working in school counseling and community mental health teams suggest that incorporating a technology-based procedure for screening for early psychosis is feasible. Identifying barriers and facilitators at different stages of the screening procedure may help to improve the implementation of such programs by reducing the dropout of clients potentially eligible for early psychosis care. Making changes in who introduces the screening process and how, addressing client ambivalence about changing services, utilizing the support of family members, providing more assertive outreach, and emphasizing the importance of training were all identified as ways to improve screening effectiveness and facilitate successful linkage with specialty services.