The REACT Study: Cost-Effectiveness Analysis of Assertive Community Treatment in North London

In recent years psychiatric care for people with serious mental illness has largely been located in the community. Inpatient admission remains a key component of care but can be associated with disruption to social roles, negative outcomes, and high costs. Therefore, attempts to reduce the need for inpatient care continue ( 1 ). Some clients engage with services only in times of crisis, often resulting in involuntary admission. The assertive community treatment (ACT) model is intended to improve client engagement in treatment through an intensive and flexible team approach ( 2 ) that targets a selected group of clients who are severely ill and poorly engaged and who may pose risks to themselves and to the community.

Since the seminal cost-benefit analysis of the forerunner of modern ACT in Madison, Wisconsin ( 3 ), there have been a number of other economic evaluations of ACT, with most evaluations being from the United States ( 4 , 5 ). Most evaluations have demonstrated that in comparison with conventional community treatment, ACT results in improved outcomes (although symptoms have been largely unaffected) and reduced treatment costs, with the latter resulting from reductions in hospital stays. However, in the United Kingdom a number of studies of ACT-type care have shown less promising results in terms of cost savings ( 6 , 7 , 8 ). The suggestion is that translating evidence regarding ACT from one setting to another is problematic. Recently a randomized controlled trial in the United Kingdom has shown that ACT did not significantly reduce the use of inpatient care ( 9 ). However, clients receiving ACT had significantly higher levels of satisfaction and engagement. This study compared the service use and costs for clients receiving ACT from that trial and estimated with cost-effectiveness analysis the cost of achieving this improved level of satisfaction.

Methods

Setting and sample

Full details of the Randomised Evaluation of Assertive Community Treatment in north London (REACT) study method are available elsewhere ( 9 ), so only a summary is provided here. Participants were recruited between July 1999 and July 2002 from two inner-London boroughs with high levels of socioeconomic disadvantage and mental health need. Inclusion criteria were referral by one of the local community mental health teams (CMHTs); limited engagement with services, reflected in CMHTs that had difficulty maintaining contact with clients and encouraging adherence with care plans; primary diagnosis of a serious mental illness (schizophrenia, schizoaffective disorder, bipolar disorder, or other psychotic illness); and recent high use of inpatient care (at least 100 consecutive inpatient days or at least five admissions within the past two years, or at least 50 consecutive inpatient days or at least three admissions within the past year). Participants not meeting the bed use criteria could be included in the trial if a panel of clinicians and researchers agreed that they met other criteria and their mental health problems were such that they posed a significant risk to themselves or to others.

Interventions and assessments

Participants were assessed at baseline and then randomly assigned to either ACT or usual care. ACT was based on standard principles ( 10 ), including patients seen at home or elsewhere in the community, assertive engagement, commitment to long-term assertive attempts at engagement, maximum caseload of 12, team-based approach, frequent team meetings to discuss clients, and skills originate from the team rather than from outside agencies. Usual care was from CMHTs in which case managers had maximum caseloads of 30. Follow-up assessments took place 18 months after random assignment. Research interviews were conducted at baseline and follow-up with participants who gave their informed consent. Ethics approval was obtained from Camden and Islington Community and Royal Free Hospital local research ethics committees. Computerized data on admissions were also collected for participants who did not consent to be interviewed. Ratings were made of symptoms ( 11 , 12 ), needs ( 13 ), satisfaction ( 14 , 15 ), quality of life ( 16 ), functioning ( 17 ), the Health of the Nation Outcome Scales ( 18 ), compliance with medication ( 19 ), engagement with services ( 20 ), and serious incidents relating to deliberate self-harm, violence, or contact with the criminal justice system.

Service use measurement

Services used during the six-month periods before baseline and follow-up interviews were recorded on the Client Service Receipt Inventory (CSRI), which has been used in many economic evaluations of mental health services ( 21 ). Participants were asked whether they had used specific services during the period and if so, how many times and (where relevant) for how long. Services included supported accommodation, inpatient care, outpatient care, accident and emergency services, day treatment, and support provided by community mental health staff, general practitioners, police, lawyers, courts, probation officers, prisons, and family members and friends. The use of medication was not included in the CSRI. Data on inpatient stays and contacts with mental health workers were also collected from case notes for the 18 months after randomization, and these notes, rather than the participant self-report data, were used to calculate the costs of these services during the follow-up period.

Cost estimations

Service costs were calculated by combining the data on service use with nationally applicable unit costs where available ( 22 ) (a full list is available from the authors). The latter include elements for staff salaries and overhead costs and take into account the ratio of time spent with clients to time spent in other activities. Informal care represents a hidden cost. It was assumed that if informal caregivers (family and friends) were not available, then the same inputs would need to be provided by a home care worker; therefore, the unit cost of the latter was used as a proxy for informal care in such cases. Costs are reported in 2003–2004 British pounds. We report these costs also in U.S. dollars, using an exchange rate of $1.58=£1 ( 23 ).

Total six-month costs at baseline were calculated by summing across individual services. At follow-up, 18-month total costs were calculated by multiplying the cost of community services by 3 and adding this product to the 18-month data on inpatient and mental health worker costs. Although costs of detaining persons in prison and in police cells were calculated for only six months, we did not multiply these by 3, because we assumed that cells would be infrequently used and we wished to avoid overestimation.

Residential care costs were initially measured for the six months before baseline and 18-month follow-up. We assumed that while patients were hospitalized their residential care costs would be zero. To estimate residential care costs for the 18-month follow-up period, we calculated the proportion of time that the client was not hospitalized. We multiplied that by the six-month residential care cost and multiplied again by 3 to obtain the 18-month residential care costs.

Loss to follow-up and lack of consent to complete the research interviews meant that for some participants there were only partial service use and cost data. To address the problem of missing follow-up costs, we divided the cost of services used at baseline (excluding inpatient care, service from mental health workers, and stays in prison or police cells) by the number of days not spent as an inpatient during the baseline period. The median daily cost across the whole sample was used if baseline costs were also missing. This daily cost of community services was then carried forward and multiplied by the number of days that the client was not an inpatient during the 18-month follow-up period and added to the costs of inpatient care and mental health workers.

The significance of the difference in mean mental health worker costs, inpatient costs, and total costs between the two groups was assessed. Cost data are frequently skewed, and therefore nonparametric bootstrapping with 10,000 repetitions was used to produce confidence intervals with the percentile method ( 24 ).

Cost-effectiveness analysis

The principal outcome measure in the REACT study was the use of inpatient care. In this study inpatient costs were combined with the costs of other services as described above. A cost-effectiveness analysis requires that cost be combined with one measure of outcome. The REACT study did not find clinical outcomes to be significantly different between ACT and usual care. Interventions that are similar in effectiveness may differ in terms of process and the acceptability of this process for clients. Other studies have found ACT to be preferred by clients, and to see whether this was the case in our study we included a measure of satisfaction. This was a scale used by Gerber and Prince ( 14 ), which was adapted from a previous scale ( 15 ). Higher scores indicated greater satisfaction.

The findings from the REACT study showed that clients in the ACT group had significantly higher levels of satisfaction than those receiving usual care. Therefore, we used this measure of satisfaction in these analyses to determine how much it would cost to achieve a one-unit improvement in outcome. The satisfaction measure consists of 35 items related to the help and treatment provided by services to the client, of which 18 items are rated between 1, least satisfactory, and 7, most satisfactory. The total score was obtained by summing the scores of these 18 items and combining the total with service cost data by using the net benefit approach, defined by NB= λ E-C, where E is effectiveness (in terms of satisfaction), λ is the hypothetical monetary societal value of a unit of satisfaction, and C is service costs. For example, if an individual in this study has a satisfaction rating of 63 and society values one unit of satisfaction at £20 ($32), then the monetary value of that person's level of satisfaction is £1,260 ($1,991). Supposing that service costs were £1,000 ($1,580), the net benefit would be £260 ($411). Clearly, a positive net benefit (where the monetary value of satisfaction exceeds costs) is to be preferred to a negative net benefit. We do not of course know how much society values a unit of satisfaction (that is, λ ), and therefore we used a range of λ values of £0 to £1,000 ($1,580) in £100 ($158) increments. Using regression analysis, we estimated the difference in net benefit (for each value of λ ) between the usual care group and the ACT group. Bootstrapping was used to generate 1,000 estimated differences for each value, and the proportion that were over zero indicated the probability that ACT was more cost-effective than usual care. These probabilities were plotted against the values of λ to produce a cost-effectiveness acceptability curve.

Results

A total of 251 participants were recruited and randomly assigned to receive ACT (N=127) or usual care from CMHTs (N=124). Baseline characteristics were very similar in both groups ( Table 1 ). Most of the sample was male, and a large proportion were from black or ethnic minority populations. Inpatient service use in the two years before random assignment was similar between the two groups ( Table 2 ). Service use data obtained from interviews were available for 90 ACT group and 76 usual care group (CMHT) participants. At the 18-month follow-up, data on inpatient use and mental health worker contacts were available for 124 clients in the ACT group and 119 in the usual care group. Other follow-up data collected from interviews were available for 91 and 78 clients, respectively.

Enlarge table

Enlarge table

In the six months before the baseline assessment most clients had contacts with mental health workers and had been inpatients at some time ( Table 3 ). There was a relatively high rate of contact with emergency services in both groups. About one-third of participants had day treatment contacts, and slightly less than half had general practitioner contacts. Informal care from friends or family was received by about one-quarter of both groups. For participants who had been inpatients, the mean total length of stay was almost 100 days. Costs were highest for inpatient care followed by informal care and contacts with mental health workers. There were no major differences between the ACT and usual care groups regarding the use or cost of any service in the six months before baseline.

Enlarge table

Medication costs were not included in the analyses. Subsequent analyses revealed that at the 18-month follow-up, 61 (64%) members of the ACT group, compared with 40 (48%) of the usual care group, had received second-generation antipsychotic medication.

In the 18 months before the follow-up interview, most clients had contact with mental health workers ( Table 3 ). However, as would be expected the number of contacts and the cost of these visits were higher for the ACT group. The difference in mental health worker costs for ACT and usual care groups was £1,272 ($2,010), which was statistically significant (95% confidence interval [CI]=£913 to £1,670 [$1,443 to $2,639]). The proportion of participants using inpatient care during the 18 months was similar to that for the six months before baseline. Inpatient costs were higher by £3,161 ($4,994) (equivalent to about 18 days) for the ACT group, but this was not statistically significant (CI=-£3,305 to £9,772 [-$5,222 to $15,440]). There were no other substantial differences in service use or costs between the two groups. Total 18-month costs (including imputed values for missing community costs) were on average £4,031 ($6,369) higher in the ACT group, a difference that was not statistically significant (CI=-£2,592 to £10,690 [-$4,095 to $16,890]).

Data on both 18-month costs and satisfaction at 18 months were available for 91 participants in the ACT group and 75 in the usual care group. The cost difference for this subsample was £3,592 ($5,675), which was 11% less than for the whole sample. The mean±SD total satisfaction score for the ACT group was 79.4±19.1 of 126 possible points, and for the control group the score was 71.7±19.1, a difference of 7.6 points that was statistically significant (CI=1.8 to 13.5) ( 9 ). The ACT group had significantly higher levels of engagement and reduced loss to follow-up, although there were no significant differences in terms of needs, symptoms, quality of life, functioning, compliance with medication, and number of serious incidents.

The incremental cost-effectiveness ratio based on the above difference in satisfaction and the difference in costs was £3,592 ($5,675)/7.6, or £473 ($747) per extra unit of satisfaction produced by ACT. The cost-effectiveness acceptability curve ( Figure 1 ) shows that if the societal value of a one-unit improvement in satisfaction is £0, then there is a 21% chance that ACT is more cost-effective than the control condition. This result increases with higher values placed on improvements in satisfaction, but even if the value were £1,000 ($1,580) there would be only a 78% chance that ACT is more cost-effective than usual care. The point at which ACT becomes more cost-effective than usual care is when a unit of improvement in satisfaction is valued at around £473 ($747) (which is the same as the incremental cost-effectiveness ratio). Although some would see cost-effectiveness acceptability curves as an alternative approach to standard parametric tests, it is interesting to determine the point at which there is a 95% likelihood of the intervention's being most cost-effective. For this to be achieved the value placed on a unit of improvement in satisfaction would need to be over £2,500 ($3,950).

Discussion

This study compared the service use, costs, and cost-effectiveness of ACT with usual community mental health care. There were few differences between the groups at baseline, but during the 18-month follow-up period participants in the ACT group had significantly more conta/cts with mental health workers. A key aim of many ACT teams is to reduce inpatient care. We found that inpatient use and costs were higher in the ACT group, as were overall costs, although these differences were not statistically significant. Outcomes were similar between the groups, but higher satisfaction and engagement suggests that ACT may be the preferred option. This is important because an intervention that does not improve clinical outcome may yet be preferred if the process of care is more acceptable to patients. In fact, satisfaction with care has been one of the more established measures used in studies of ACT, hence its inclusion here. However, although ACT resulted in a significantly higher level of patient satisfaction ( 9 ), this difference would need to be valued very highly for ACT to be more cost-effective than usual community mental health care.

There were limitations to this study. Many of the service use data were collected via participant self-report, so they may not be wholly accurate. However, systematic bias between the two arms of the trial is unlikely, and the research literature suggests that self-report is an acceptable way of collecting such information ( 25 , 26 ). In addition, no alternative method was available given the breadth of information we required and the resources available in terms of researcher time. Data on the two key services (inpatient care and mental health worker contacts), though, were obtained from hospital records for the whole follow-up period. A further limitation is that follow-up service data collected with the CSRI pertained to the six months preceding the 18-month follow-up assessment. The costs for this period were multiplied by 3 to make them compatible with the 18-month data on inpatient and mental health worker costs. Such an extrapolation assumes that these community service costs were representative of the entire follow-up period. Although this may not have been the case, the extrapolated costs formed only a small proportion of the total costs. One cost that was not included in the analyses was medication. As stated earlier, more of the ACT group were taking second-generation antipsychotics as of the 18-month follow-up compared with those in the usual care group. The cost difference reported here is therefore possibly underestimated, although compared with the cost of other services the extra cost would be relatively low. Finally, this study was based in two neighboring areas of inner London; the results are not necessarily generalizable to other (particularly nonurban) areas. It is possible that usual care is more developed than elsewhere, which would reduce the potential gains of ACT.

Conclusions

There were no significant differences in costs for ACT compared with usual care. Although ACT did not result in cost savings over 18 months, it resulted in greater satisfaction for clients and in higher levels of engagement with services. This in itself may justify the expansion of such services. There may have been an expectation that ACT would save money, and in studies from the United States this has been the case. However, as was stated in the introduction, the economic impacts of ACT and of ACT-type care have been less evident in the United Kingdom. There are two likely reasons for the lack of savings. First, standard care services are more developed than they were ten or 20 years ago, so it is more difficult for ACT to achieve additional benefits. Second, engaging more people in services may highlight the need for admission for some. The follow-up period was also probably too short to detect cost savings. Increased engagement and satisfaction that result from ACT may affect service use or reduce the "revolving door" problem in the longer term, and this may result in long-term cost reductions.

Acknowledgments and disclosures

The REACT study was funded by Camden and Islington Health Authority, the King's Fund, and the United Kingdom Department of Health. The authors thank Professor Peter Tyrer for chairing the project's steering committee.

The authors report no competing interests.