Innovations: Evidence-Based Practices: Establishing the Evidence Base for Psychiatric Services: Estimating the Impact on the Population

Evidence-based medicine considers the efficacy of various interventions in order to assess the benefit to individual patients. However, there is a need for a broader approach in terms of benefit to the population of patients and to the population from which they come ( 1 ). Population impact measures have been designed to help prioritize the way in which we spend the health dollar or introduce new services by measuring the maximal gain to the health of the population.

This column describes a six-step method for calculating the impact of a new intervention on a target population. The example used is the introduction of a new assertive community treatment program and the number of hospitalizations that its introduction would prevent. This result can be compared with the impact of alternative interventions, such as increasing adherence to medication in the same population.

Evidence-based medicine and the population

The advent of evidence-based medicine has helped identify the interventions that are most appropriate to provide benefit to patients. In the field of psychiatry, we can turn to many sources to identify the efficacy of different interventions ( 2 , 3 , 4 ). We can assess the benefit of new pharmaceutical agents, using the results of randomized controlled trials and systematic reviews of these trials. We can also assess the benefits of the way that services are organized, although such assessments are less often based on the results of controlled trials or other forms of scientific evaluation; therefore, the evidence base is less secure.

If we must prioritize how we spend the health dollar or introduce new services, we may want to consider the maximal gain to the health of the population of patients in a particular locality and to the disease-specific population within that locality. To do so, we must examine the impact of an intervention on the population rather than on the individual. Population impact measures have been designed to help in such investigations ( 5 , 6 , 7 ).

Assertive community treatment

Let us take the example of assertive community treatment programs, which aim to reduce hospitalization. A study from Indiana published in this journal in 1995 claimed to have found a one-third reduction in frequency of psychiatric hospitalization and a 50 percent reduction in inpatient days ( 8 ). A systematic review of randomized controlled trials published in the Cochrane library in 1998 confirmed the benefit, which was estimated as a reduction of 41 percent in hospital admissions ( 9 ). With this information, the impact of introducing such a program on a specific population can be calculated. Following the six steps outlined in the box on the next page, we find that 14 hospitalizations would be prevented in a year among the 300 people with schizophrenia in a total population of 100,000.

This result can then be compared with other interventions for schizophrenia or with other interventions for different conditions. In a recent publication, which also provides details of these methods ( 10 ), we calculated that various interventions for schizophrenia would prevent between six and 40 hospitalizations and between six and 44 relapses in a year, going from current to best practice in a population of 100,000. Similar calculations for depression interventions led to an estimate of between 100 and 325 relapses prevented. The difference in reductions between schizophrenia and depression is accounted for largely by their different prevalence rates in the population.

Application to a selected population

Having such data for a particular patient population might assist in prioritization of resources. However, policy makers will want to know the cost of the intervention. We might find that development and implementation of assertive community treatment programs is very expensive and that a similar reduction in hospitalization could be obtained at less cost by a cheaper intervention. Once the outcome of an intervention is known, we can then assign priorities to it in relation to other demands on the resource. In another article, we have suggested that producing outcomes in real terms, such as reduction in hospitalization or prevention of relapse, will be of more use to policy makers than the usual outcomes of cost-effectiveness assessments, such as incremental quality-adjusted life years, which depend on an arbitrary assignment of values ( 11 ). The results of population impact measurement could be useful to administrators and policy makers as well as to clinicians. We all want to maximize the benefit to the population of what we do, and here is a way of calculating this benefit.

Of course, the calculations depend on obtaining accurate and appropriate data. Much of the data come from the literature, but for increased accuracy it is best to have and use local data. For instance, in the example above, we assumed that there was no current assertive community treatment service in the area. However, the added benefit of a new program will depend on how many people are already exposed to the intervention. Local data on the extent of exposure are vital. The calculations are also subject to error and variability. Apart from the fact that local data might not reflect the data in the literature, which must be used in lieu of local information, the estimates of benefit might not be robust. The 41 percent reduction in hospitalizations attributable to assertive community treatment is based on a systematic review of the literature of randomized controlled trials. Using this percentage should be appropriate as long as the interventions examined in the literature are similar to those that are to be introduced in practice. Use of this percentage also assumes that the types of patients entered into such trials are similar to the types of patients seen in the population to be targeted. Although the figures used here serve to illustrate methods of calculating population impact, those who wish to use such measures should be aware of the potential for bias and of the importance of using robust values for the prevalence of the condition and its current treatment and the efficacy of the interventions being examined.

Does this approach add value?

How does calculation of the population impact add to the usual ways in which evidence-based medicine can contribute to health care? Many of the principles of evidence-based medicine are used in the calculations presented here. They are based on the best evidence obtained from the literature. The calculations are actually a population extension of a statistic that is used frequently in evidence-based medicine, the number needed to treat (NNT). NNT is the number of people who must be treated to avoid one adverse event. All that we have done here is to set this in the context of the whole population.

A condition with a favorable NNT—that is, one for which the treatment or intervention is highly effective—may not result in much health gain to the population as a whole if the condition being treated is rare in the population. In terms of the NNT for assertive community treatment, ten patients must be exposed to it for the intervention to prevent one hospitalization. In comparison, the NNT for increasing adherence to medication among patients with schizophrenia is such that only five patients need experience the intervention to prevent one hospitalization. Clinicians who see patients might want to focus on this second intervention. However, in the entire population, instead of 14 hospitalizations prevented in the next year through assertive community treatment, only six are prevented by increasing adherence to medication. The difference results from the difference in numbers of patients eligible for the interventions on the basis of current and best-practice estimates.

The use of population impact measurement is in its infancy. We would welcome the opinions of readers involved in psychiatric care as to the potential value of this approach.