0
Get Alert
Please Wait... Processing your request... Please Wait.
You must sign in to sign-up for alerts.

Please confirm that your email address is correct, so you can successfully receive this alert.

1
Brief Report   |    
Assessing Readmission to Substance Abuse Treatment as an Indicator of Outcome and Program Performance
Keith Humphreys, Ph.D.; Kenneth R. Weingardt, Ph.D.
Psychiatric Services 2000; doi: 10.1176/appi.ps.51.12.1568

Managed health care systems often use treatment readmission data as an indicator of psychiatric patient outcome and program performance. This study of 3,018 inpatients being treated for substance abuse in Department of Veterans Affairs medical centers found that across a range of measures and patient subpopulations, patient outcomes and program performance were virtually independent of treatment readmission. These findings suggest that even though readmission for substance abuse treatment may have value as an easily obtainable measure of health care utilization and cost, it cannot serve as a valid substitute for direct assessment of patient outcome or program performance.

Abstract Teaser
Figures in this Article

The potential value of readmission data as an easily gathered indicator of psychiatric patient outcomes and program performance has long been debated (1,2,3,4,5,6). Solomon and Doll (6) argued more than 20 years ago that inpatient readmission might index a plethora of factors other than poor patient outcome and low treatment quality—for example, need for additional care, lack of housing, and community intolerance of persons with serious mental illness. The scope and importance of this debate are increasing as managed health care systems base programming and reimbursement decisions on readmission data.

Lyons and colleagues (7) reported that neither poor hospital outcome nor premature discharge was associated with 30-day or six-month readmission among psychiatric patients in a regional managed care program, leading the authors to question the value of readmission data as a measure of program performance. The study we report here builds on this work by conducting a similar analysis with inpatients in substance abuse treatment. We address two key questions: Are any of a variety of readmission indexes related significantly to patient outcomes, either for all patients or for important subpopulations? Do patients in substance abuse treatment programs that have higher readmission rates have worse outcomes overall than those in programs with lower readmission rates?

This study is part of a large, ongoing evaluation project that has been described in detail elsewhere (8). The sample consists of 3,018 male substance abuse inpatients who were treated for 21 to 28 days between May 1992 and January 1994 at one of 15 Department of Veterans Affairs (VA) facilities and who underwent follow-up one year later. Given an intake sample of 3,612 patients, the follow-up rate was 84 percent.

Participants' mean±SD age at admission was 42.9±9.6 years, and they had a mean±SD of 12.7±1.8 years of education. The sample was primarily composed of African-American veterans (1,468, or 49 percent) and non-Hispanic Caucasian veterans (1,384, or 46 percent). At intake, only 24 percent were employed (N=717), and 18 percent were married (N=547). More than one-third of the participants (1,016, or 34 percent) had been involved in crime, as evidenced by one or more arrests in the previous 12 months. A total of 362 participants (12 percent) were classified as homeless at intake, defined as having spent most of the previous 12 months on the street or in shelters.

According to DSM-III-R diagnostic criteria, 41 percent (N=1,231) had a diagnosis of alcohol dependence or abuse only, 14 percent (N=417) had a diagnosis of drug dependence or abuse only, and 45 percent (N=1,356) were diagnosed as having both alcohol and drug dependence or abuse. In addition, 36 percent (N=1,096) had a comorbid axis I or axis II psychiatric diagnosis.

Patients who agreed to participate in the study completed an inventory at baseline and at follow-up that collected information on demographic characteristics and on four self-reported clinical outcomes, using the past three months as the reference window. The outcomes were any use of drugs or alcohol, coded dichotomously; negative consequences of substance use; depression or anxiety symptoms; and severe psychiatric symptoms. Negative consequences of substance abuse were measured as the sum of positive responses to 18 potential consequences such as family arguments and job problems; depression or anxiety symptoms were measured as the sum of the scores on 12 items from the depression and anxiety scales of the Brief Symptom Inventory (BSI) (9), each ranging from 0, not at all, to 4, extremely. Severe psychiatric symptoms were measured as the sum of the scores on ten items from the BSI paranoid ideation and psychoticism scales, each also ranging from 0 to 4.

Using national VA databases covering all facilities in the system, a dichotomous variable reflecting any readmission with a substance use diagnosis within one year of discharge was created. Because clinical outcome was assessed one year after discharge and thus may better correspond to later readmissions, four additional dichotomous variables were created. They reflected any readmission to inpatient substance abuse treatment within 90 days after discharge, 91 to 180 days after discharge, 181 to 270 days after discharge, and 271 to 360 days after discharge.

Of the 3,018 patients, 1,269 (42 percent) were readmitted at least once within a year after discharge; 788 patients (26 percent) were readmitted at least once within 90 days, 226 patients (8 percent) within 91 to 180 days, 175 patients (6 percent) within 181 to 270 days, and 121 patients (4 percent) within 271 to 360 days.

At one-year follow-up, patients had improved significantly on use of drugs and alcohol (98 percent of participants were using at baseline versus 60 percent at follow-up), negative consequences of substance use (mean±SD= 5.6±4.3 at baseline versus 3.1±3.9 at follow-up), depression and anxiety symptoms (20.1±11.3 at baseline versus 15.6±11.9 at follow-up), and severe psychiatric symptoms (14.2±8.6 at baseline versus 11.6±9 at follow-up). In addition, a higher proportion of patients were employed (38 percent versus 24 percent) and a lower proportion were involved in crime (22 percent versus 34 percent) at follow-up than at baseline. A detailed analysis of outcome data for this sample is presented elsewhere (8).

We considered an observed relationship between readmission and outcome measures significant if it reached Cohen's standard (10) for a "medium" effect size—a correlation of .3 or more, which would explain 9 percent or more of the shared variance. None of the point biserial and fourfold point correlations between readmission and outcome measures even approached this modest standard. Indeed, most were close to zero, and the largest explained only 2.5 percent of variance. We repeated these analyses for important subpopulations within the sample—those who at baseline were homeless, were unemployed, had a dual diagnosis, were involved in crime, were dependent on alcohol only, were dependent on drugs only, and were dependent on both alcohol and drugs. Again, readmission and outcome measures were essentially independent, and none of the 140 correlations we calculated even approached the standard of a medium effect size.

In a second set of analyses, the 15 inpatient programs were ranked according to their rates of readmission and adverse patient outcomes. The association between program rankings on readmission with rankings of abstinence, consequences of substance use, psychological distress, and psychiatric symptoms were all small and not statistically significant.

Readmission to inpatient substance abuse treatment is an inexpensively constructed index of utilization and cost, and it is certainly valuable as such. However, the results of this study indicate that it is essentially independent of one-year patient substance abuse and psychiatric outcomes and therefore should not be used as a substitute for direct assessment of clinical outcome and treatment quality.

The measurement error of readmission—produced by many readmitted patients experiencing good outcomes and many patients who were not readmitted experiencing bad outcomes—was found to be uniformly high. Lack of any observed readmission might indicate not that a patient has had a positive outcome but rather that the patient is incarcerated or is receiving care in a non-VA psychiatric facility. Similarly, rather than indexing poor clinical outcome, readmission might reflect the onset of cold weather, lack of affordable housing, or reductions in VA benefits.

These findings in a sample of substance abuse inpatients echo those of Lyons and colleagues (7) on readmitted psychiatric patients. Taken together, the two studies indicate that across patient populations readmission is not a valid indicator of clinical outcome or program performance.

This work was supported by the Department of Veterans Affairs mental health strategic health care group and health services research and development service. The authors thank John Finney, Ph.D., for his helpful comments on an earlier draft of this paper.

The authors are affiliated with the Program Evaluation and Resource Center at the Veterans Affairs Palo Alto Health Care System (VAPAHCS) and the department of psychiatry at Stanford University School of Medicine. Address correspondence to Dr. Humphreys at the Program Evaluation and Resource Center, VAPAHCS (152-MPD), 795 Willow Road, Menlo Park, California 94025 (e-mail, knh@stanford.edu).

Anthony W: Efficacy of psychiatric rehabilitation. Psychological Bulletin 78:447-456,  1972
[PubMed]
[CrossRef]
 
Aviram U, Syme S, Cohen J: The effects of policies and programs on reduction of mental hospitalization. Social Science and Medicine 10:571-577,  1976
[CrossRef]
 
Caan W, Crowe M: Using readmission rates as indicators of outcome in comparing psychiatric services. Journal of Mental Health (UK) 3:521-524,  1994
[CrossRef]
 
Franklin JL, Kitteridge LD, Thrasher JH: A survey of factors related to mental hospital readmissions. Hospital and Community Psychiatry 26:749-751,  1975
[PubMed]
 
Milne R, Clarke A: Can readmission rates be used as an outcome indicator? British Medical Journal 301:1139-1140,  1990
 
Solomon P, Doll W: The varieties of readmission: the case against the use of recidivism rates as a measure of program effectiveness. American Journal of Orthopsychiatry 49:230-239,  1979
[PubMed]
 
Lyons JS, O'Mahoney MT, Miller SI, et al: Predicting readmission to the psychiatric hospital in a managed care environment: implications for quality indicators. American Journal of Psychiatry 154:337-340,  1997
[PubMed]
 
Ouimette PC, Finney JW, Moos RH: Twelve-step and cognitive-behavioral treatment for substance abuse: a comparison of treatment effectiveness. Journal of Consulting and Clinical Psychology 55:230-240,  1997
 
Derogatis LR, Melisaratos N: The Brief Symptom Inventory: an introductory report. Psychological Medicine 13:595-605,  1983
[PubMed]
[CrossRef]
 
Cohen J: A power primer. Psychological Bulletin 112:155-159,  1992
[PubMed]
[CrossRef]
 
+

References

Anthony W: Efficacy of psychiatric rehabilitation. Psychological Bulletin 78:447-456,  1972
[PubMed]
[CrossRef]
 
Aviram U, Syme S, Cohen J: The effects of policies and programs on reduction of mental hospitalization. Social Science and Medicine 10:571-577,  1976
[CrossRef]
 
Caan W, Crowe M: Using readmission rates as indicators of outcome in comparing psychiatric services. Journal of Mental Health (UK) 3:521-524,  1994
[CrossRef]
 
Franklin JL, Kitteridge LD, Thrasher JH: A survey of factors related to mental hospital readmissions. Hospital and Community Psychiatry 26:749-751,  1975
[PubMed]
 
Milne R, Clarke A: Can readmission rates be used as an outcome indicator? British Medical Journal 301:1139-1140,  1990
 
Solomon P, Doll W: The varieties of readmission: the case against the use of recidivism rates as a measure of program effectiveness. American Journal of Orthopsychiatry 49:230-239,  1979
[PubMed]
 
Lyons JS, O'Mahoney MT, Miller SI, et al: Predicting readmission to the psychiatric hospital in a managed care environment: implications for quality indicators. American Journal of Psychiatry 154:337-340,  1997
[PubMed]
 
Ouimette PC, Finney JW, Moos RH: Twelve-step and cognitive-behavioral treatment for substance abuse: a comparison of treatment effectiveness. Journal of Consulting and Clinical Psychology 55:230-240,  1997
 
Derogatis LR, Melisaratos N: The Brief Symptom Inventory: an introductory report. Psychological Medicine 13:595-605,  1983
[PubMed]
[CrossRef]
 
Cohen J: A power primer. Psychological Bulletin 112:155-159,  1992
[PubMed]
[CrossRef]
 
+
+

CME Activity

There is currently no quiz available for this resource. Please click here to go to the CME page to find another.
Submit a Comments
Please read the other comments before you post yours. Contributors must reveal any conflict of interest.
Comments are moderated and will appear on the site at the discertion of APA editorial staff.

* = Required Field
(if multiple authors, separate names by comma)
Example: John Doe



Web of Science® Times Cited: 8

Related Content
Articles
Books
The American Psychiatric Publishing Textbook of Substance Abuse Treatment, 4th Edition > Chapter 33.  >
The American Psychiatric Publishing Textbook of Geriatric Psychiatry, 4th Edition > Chapter 16.  >
The American Psychiatric Publishing Textbook of Geriatric Psychiatry, 4th Edition > Chapter 33.  >
Dulcan's Textbook of Child and Adolescent Psychiatry > Chapter 39.  >
The American Psychiatric Publishing Textbook of Substance Abuse Treatment, 4th Edition > Chapter 38.  >
Topic Collections
Psychiatric News
APA Guidelines
PubMed Articles