Schizophrenia is a relatively common disorder, occurring with a lifetime incidence of about 1% and an annual incidence of ten to 20 per 100,000 in the group aged 15 to 55. This means that all mental health services that provide care for an entire population can expect to provide care for this group. We have little information to measure the quality or outcome of routine services for this group of patients. In the absence of good baseline information on process and outcome measures, attempts have been made to improve outcome through the use of specialized treatment services for first-episode psychosis. Such programs aim to reduce the time between onset of symptoms and treatment—that is, reduce the duration of untreated psychosis. They also aim to offer evidence-based care that optimizes outcomes and minimizes deterioration. This involves reducing relapse and hospital admissions, while offering evidence-based rehabilitation.

There is increasing evidence from randomized controlled studies showing that specialized early psychosis services are more effective than treatment as usual (1). OPUS, the largest study to date, has demonstrated the most robust results (2) and exemplifies the components of such services. Key components of the OPUS specialized services included assertive community treatment teams, evidence-based family intervention offered to all families, and social skills training for those assessed as having impaired social skills. These components were offered as a group-based program focusing on medication, coping with symptoms, building conversation skills, and building problem-solving skills. The World Health Organization and the International Early Psychosis Association have endorsed a consensus statement in the form of an international declaration that articulates core values, goals, and standards for such programs (3). Health jurisdictions in the United Kingdom (4) and in the province of Ontario, Canada, (5) have approved policy frameworks for implementing such programs as part of publicly funded mental health services. One of the challenges of program implementation is the lack of valid and reliable measures to ensure that new programs adhere to the protocols or program models that have been shown to be effective.

This column illustrates some of the tools available for quality improvement and the challenges in their application. The development and application of such tools are required to move first-episode psychosis treatment from innovation to best practice and standard care.

Fidelity scales

A fidelity scale is a measure that assesses the extent to which delivery of an intervention adheres to the protocol or program model originally developed. Such scales have been developed for psychotherapies and for a number of rehabilitation programs. The information from successful research projects and consensus statements provides a useful starting point for defining services. At present there are no published evidence-based fidelity measures for specialized treatment services for first-episode psychosis, although the international declaration on early psychosis provides a useful starting point for such a scale (3).

Clinical practice guidelines

Clinical practice guidelines are defined as systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances. The International Early Psychosis Association has also written clinical practice guidelines (6) that make a number of recommendations. There is also valuable and valid information available in published evidence-based guidelines. For example, the Canadian clinical practice guidelines for the treatment of schizophrenia list a number of service components relevant to treatment services for early psychosis along with the supportive evidence for these components (7). For example, the guidelines assign the highest level of evidence (level A) to family psychoeducation and supported employment, both of which have been tested in first-episode psychosis. Moderate evidence (level B) is assigned to services such as cognitive-behavioral strategies, skills training, and integrated addictions services.

Performance measurement has been defined as "the use of statistical evidence to determine progress towards specific defined organizational objectives" (8). A set of performance measures for treatment services for early psychosis has been identified from the literature and rated for importance using a Delphi consensus method (9). This yielded a large number of measures that covered eight domains that the Canadian Institute of Health Information considers to be important when evaluating health programs: acceptability, accessibility, appropriateness, competence, continuity, effectiveness, efficiency, and safety (10). Challenges in implementing these measures are described below.

Once performance measures have been identified, they can be used to establish standards and benchmarks. Standards have been defined as numerical thresholds for performance and can be established by individuals or groups using arbitrary decisions, consensus, or statistically derived thresholds. For example the international declaration on early psychosis recommends that services should assess the duration of untreated psychosis of all patients and that the mean duration should be under three months (3). In contrast, benchmarks reflect results of the highest-performing organizations. Published results from research studies and quality assessment initiatives provide a basis for preliminary comparisons (11). Once standards have been established it is possible to compare programs and establish benchmarks, particularly if risk adjustment formulas have been developed. These benchmarks can then improve to the point that best practices are defined. The most widely accepted document on standards for first-episode psychosis treatment services is the international declaration on early psychosis (3).

Along with my colleagues, I have applied performance measures in two studies. In the first only one program was examined (12), and in the second two programs were compared (13). The data in Table 1 compare the results from one program (the Calgary Early Psychosis Treatment Service) with the published standards and research results. This is done to demonstrate the practical issues involved in the implementation of performance measures to assess achievement of standards.

Differences in service delivery illustrate another challenge in using performance measures. Family education and support is an important and evidence-based process, which can be delivered either to family groups or to individual families. The identified performance measure for this intervention was the proportion of families receiving a formal family intervention, defined a priori as a minimum of four sessions of family intervention. Although 80% were seen at least once, 44% met the criterion of receiving a minimum of four sessions. This criterion is hard to apply in programs that offer a single large-group educational session followed by informal family education from program staff. In the OPUS study the family intervention started with a seven-hour group session attended by 47% of families.

The difficulties encountered in comparing these performance measures in different settings illustrate the challenge of implementing performance measurement in mental health services more broadly.

In contrast to some of the other measures, hospital admission is an important outcome measure that is universally available, reliable, and easily measured (14). It is a significant event for both patients and families and has significant societal costs. Furthermore, it is influenced by a number of important process measures, such as medication adherence and family education and support (15). A limitation of admission as a performance measure is that its significance may vary over time and in different health care systems (14). The admission rate found in the Calgary program was similar to published readmission rates in a recent randomized controlled study of early psychosis services (16).

The international declaration recommends that the standard for duration of untreated psychosis should be a mean of less than three months (3). Although there is convincing research showing that the duration of untreated psychosis is clinically important, there are statistical and measurement problems in using it as a performance measure (17).

My colleagues and I have previously demonstrated that it is feasible to assess a single program using consensus and evidence-based performance measures (9). By comparing these results with published results, my colleagues and I have also provided data that supports the reasonableness and attainability of some potential standards of care. For example, a two-year hospital admission rate of 35% appears to be attainable, both in published research studies and in our publicly funded clinical services. At the same time, some measures, such as the proportion of participants assessed for positive symptoms, seem less useful. Others, such as the signed care plan, while appealing would require process changes in order to collect relevant data. The collection of performance measures in routine practice can also be facilitated by well-designed information systems; however, current data systems are not established for that purpose.

Standards are most appropriately set by national and international bodies and work best when there is a consensus among consumers, researchers, clinicians, providers, and payers. In order to define best practices for early psychosis treatment, there is a need to develop evidence-based performance measures and standards that reflect attainable benchmarks.

The author reports no competing interests.