0
Get Alert
Please Wait... Processing your request... Please Wait.
You must sign in to sign-up for alerts.

Please confirm that your email address is correct, so you can successfully receive this alert.

1
Article   |    
Predictors of Admission in First-Episode Psychosis: Developing a Risk Adjustment Model for Service Comparisons
Donald Emile Addington, M.B.B.S.; Cindy Beck, M.D.; JianLi Wang, Ph.D.; Beverly Adams, M.D.; Cathy Pryce, M.N.; Haifeng Zhu, M.Sc.; Jian Kang, M.Sc.; Emily McKenzie, M.Sc.
Psychiatric Services 2010; doi: 10.1176/appi.ps.61.5.483
View Author and Article Information

Dr. Addington, Dr. Beck, Dr. Wang, Dr. Adams, Ms. Zhu, Ms. Kang, and Ms. McKenzie are affiliated with the Department of Psychiatry and Ms. Pryce is affiliated with the Department of Community Health Sciences, all at the University of Calgary, Special Services Bldg., 2nd Floor, Foothills Medical Centre, 1403 29th St., N.W., Calgary, Alberta T2N 2T9, Canada (e-mail: addingto@ucalgary.ca).

Objective: The aim of this study was to develop a risk adjustment model based on hospital admissions that would enable comparison between services for patients with a first episode of psychosis. Methods: Candidate predictor variables for hospital admission were identified in a literature review, from which an expert panel selected 12 potential risk adjustment variables by using a structured process, the Template for Risk Adjustment Information Transfer. Multivariable logistic regression modeling with the 12 variables was used to develop models in one cohort of first-episode psychosis patients (N=297); these models were validated with data from a second cohort (N=309). The C statistic, a measure of model discrimination, was calculated to assess model performance. Results: In the data from the development sample, prior hospitalization was the only significant predictor of hospital admissions within one year of enrollment in the first-episode psychosis program (odds ratio [OR]=1.88, p=.05). Hospital admissions after two and three years from admission to the program were significantly associated with higher levels of initial positive symptoms (OR=1.07, p=.02; OR=1.06, p=.02, respectively), and prior hospitalizations (OR=2.72, p=.001; OR=3.34, p<.001, respectively). The logistic models performed well, with C statistics ranging from .72 to .74 for the three outcomes, where a value of 1.0 implies perfect model discrimination. In the validation data the C statistics were slightly lower, ranging from .67 to .72. Conclusions: According to the C statistic estimates, the model developed provided good discrimination and was relatively robust in predicting hospitalization of first-episode psychosis patients. (Psychiatric Services 61:483–488, 2010)

Abstract Teaser
Figures in this Article

There has been increased interest in the course and outcome of first-episode psychosis, which has been spurred on by both an interest in its underlying neurobiology and the potential for improving clinical outcomes (1). Several clinical trials have been conducted to assess the efficacy of specialized programs for treatment of first-episode psychosis in comparison with treatment as usual (2). The results have been sufficiently encouraging for some government health ministries to implement such services as routine care as a matter of policy (3). However, there is a dearth of published data on the specifics of the care provided or outcomes of treatment as usual in real-world settings (4).

Performance measures for assessing the outcome of first-episode psychosis have been identified (5), have been found to be feasible for individual program evaluation (6), and have been used to provide evidence to support establishing standards of care (7). In this article we focus on hospital admission as the key performance measure because it has been identified as a good proxy outcome in studies of schizophrenia (8). As such it should be suitable for measuring the outcome of any form of service to groups with a first-episode psychosis whatever the service delivery model, including networks of individual providers, standard community mental health services, or specialized first-episode psychosis services. Three key attributes of performance measures are that they should be meaningful, feasible, and actionable (9). Hospital admission is a measure that meets these criteria, particularly for serious mental disorders manifesting as first-episode psychosis. Admission for first-episode psychosis is meaningful in that it is associated with negative outcomes such as relapse, violence, attempted suicide, and increased costs (10,11). It is feasible to measure because it is a concrete measure that is reported in most if not all health services. It is actionable because it is related to processes of care, including inpatient and outpatient care (12). In the outpatient setting preventing admission has been related to medication adherence (13) and family psychoeducation (14). These attributes led us to our study goal, which was to develop a risk adjustment model based on a set of evidence-based predictor variables for hospital admission. We planned to evaluate the model's predictive discrimination using the C statistic, and we considered a value of .7 or higher to signify good discrimination (15).

Risk adjustment is a statistical process for controlling for group differences when comparing nonequivalent groups on outcomes of interest. It can achieve only a partial adjustment and is not equivalent to randomized patient assignment (16). Risk adjustment has been used to compare mental health services (17), but it has not been applied to compare outcomes of either treatment as usual or specialized services for first-episode psychosis. In this study we examined the predictors of hospital admission in first-episode psychosis in order to develop a risk adjustment model for comparing outcomes across services with different patient populations and service delivery models.

We used a retrospective cohort design to assess hospital admission one, two, and three years after patient enrollment into a first-episode psychosis program that serves a population of about one million. The services are provided at no cost to the consumer at the point of service delivery because all of these patients have health insurance. Admissions to local hospitals are recorded in a single database. Hospital admissions are easy to track because the population is relatively isolated and there are few admissions to hospitals in other regions. Unfortunately the exact number of admissions elsewhere is unknown. Potential risk factors for hospital admission were identified by a panel of experts who used the Template for Risk Adjustment and Transfer (TRAIT) (18). Multiple regression analysis was then used to construct a model in which each of the candidate risk factors was entered as a separate variable (19).

The development of the risk adjustment model involved three steps: literature review, expert panel review, and model development. The first step in the research was a literature review that examined potential risk factors for hospital admission. The following terms were searched in MEDLINE and PsycINFO databases in the years 1990–2006: schizophrenia and risk factor, mental illness and risk factor, schizophrenia and risk variable, mental illness and risk variable, predictive factor and outcome, predictor, risk adjustment, outcome, comorbidity, prognosis and outcome, and prognostic factor.

The next step in the research was to have an expert panel reduce the resulting set of 17 candidate variables to a parsimonious set of potential risk adjustment variables with the assistance of TRAIT. The expert panel comprised psychiatric epidemiologists and clinical and health services researchers, all with expertise in first-episode psychosis research. A full description of the method is outlined in a document developed for the Center for Mental Health Services of the Substance Abuse and Mental Health Services Administration (18). Members of the expert panel ranked the level of evidence for individual risk factors according to three categories—published research evidence, clinical experience, or other experience or evidence. The TRAIT was originally developed to contribute to closing the knowledge gap between developers of quality measures and developers of models to adjust results from quality measures for level of risk. TRAIT is a template that guides measure developers in the identification of patient factors that are potentially useful for risk adjustment of an individual quality measure.

The next step involved identification of two cohorts that each represented an incidence cohort of first-episode psychosis patients from the same catchment area of about one million people. The first sample consisted of 297 patients admitted between February 2003 and December 2005, and the validation sample comprised 309 consecutive admissions between January 1997 and December 2000. All patients received similar treatment in a program for individuals experiencing a first episode of psychosis (20). Participants met DSM-IV (21) criteria for schizophrenia, schizophreniform disorder, schizoaffective disorder, or other psychotic disorders. At the time of enrollment they had received less than 16 weeks of adequate treatment for their first psychotic episode (22).

The power of our available sample was considered to be adequate based on the estimate that five to ten events are required to test each risk adjustor (23). Our planned sample size of 300 (270 at two years) provided an estimated 80 events (admissions), which was considered adequate because we did not plan to test more than 15 predictors.

Exclusion criteria included evidence of an organic central nervous system disorder (such as epilepsy or traumatic brain injury), mental retardation, or being under age 16 or over age 65. The research project was approved by the local conjoint research ethics committee. In accordance with the local Health Information Protection Act and ethics committee guidelines, individual patient consent was not required because the project used deidentified data from administrative and clinical databases.

In the third stage of developing our risk adjustment model, we used multivariate logistic regression to model hospital admission as a function of the 12 potential risk adjustment variables identified by the TRAIT. The model was first used in the development cohort and then externally validated in a second cohort (24). Model reduction was conducted using methodology consistent with that of Harrell and colleagues (15). Model assumptions of logistic regression were tested, and necessary modifications were made. For hospital admission, model discrimination (ability to predict differences in the outcome on the basis of the risk factors) was crudely assessed by plotting covariance graphs. Next, the C statistic was calculated (25). This equals the area under the curve for the receiver operating characteristic (sensitivity versus 1 - specificity) of the predictive model for the observed data; C=.5 indicates no predictive discrimination, whereas C=1.0 implies perfect discrimination. Bootstrap methods were used to construct a 95% confidence interval for C as detailed by Efron (26). This is to allow comparisons across models and with the validation data. Bootstrap methods allow calculation of confidence intervals for variables such as the C statistic, which do not have known underlying distributions. The data were analyzed with Stata 10.0 (27).

Selected terms were searched in MEDLINE and PsycINFO for the years 1990–2006 and produced over 45,000 hits: schizophrenia and risk factor, 1,592; mental illness and risk factor, 25,886; schizophrenia and risk variable, 90; mental illness and risk variable, 980; predictive factor and outcome, 17; predictor, 382; risk adjustment, 39; outcome, 3,958; comorbidity, 1,023; prognosis and outcome, 2,451; and prognostic factor, 11. Therefore, the search strategy was refined to include at least one of the following combinations: Positive and Negative Syndrome Scale (PANSS) and predictors; Global Assessment of Functioning (GAF) and predictors; relapse, risk factor, and schizophrenia; rehospitalization; remission and risk factor and schizophrenia; and predictor and outcome. A review of 18 identified studies produced 17 potential risk factors.

The TRAIT analysis identified 12 of these 17 variables as potential predictors of hospitalization. For this study only one diagnostic category was considered: first-episode psychosis. This meant that no specific diagnoses were considered to be predictors of hospitalization. The TRAIT analysis revealed potential variables, including sociodemographic, clinical, and other attributes. The variables and their definitions are presented in the box on this page.

Risk factors used in a risk adjustment model and their definitions

Gender

  Male or female

Marital status

  Married or common-law or other (baseline)

Education level

 At least high school education (baseline)

Ethnicity

  Self-report

Admission age

  Age at enrollment in the first-episode program

Duration of untreated psychosis

  Assessed by a psychiatrist and based on all sources of information

Prior hospitalization

  Hospitalized before enrollment in a first-episode program (based on linkage with hospital discharge data)

Global Assessment of Functioning

  DSM-IV axis V (baseline)

Positive symptoms of psychosis

  Score on positive syndrome subscale of Positive and Negative Syndrome Scale (baseline)

Negative symptoms of psychosis

  Score on negative syndrome subscale of Positive and Negative Syndrome Scale (baseline)

Comorbid major depression

 Major depression at baseline based on the Structured Clinical Interview for DSM-IV (baseline)

Comorbid substance use disorder

  Drug or alcohol misuse at baseline based on the Structured Clinical Interview for DSM-IV (baseline)

The clinical and sociodemographic features of the development and validation samples are reported in Table 1. The samples were broadly similar, but there were significant differences between the two samples in enrollment age for the first-episode program (t=2.9, df=604, p=.004) and PANSS negative symptom score at baseline (t=4.0, df=595, p<.001), proportion of Caucasians (z=-3.1, df=1, p=.002), marital status (z=2.0, df=1, p=.05), and major depression.

The high percentage of missing data (N=98) in the validation database for high school education necessitated removal of the variable from the modeling process. The one-, two-, and three-year hospitalization outcomes were similar in both samples, with no significant differences between cohorts (Table 2).

The results of the multiple regression analyses are presented in Table 3. Prior hospitalization was a common significant predictor of admission in all three time periods in both the development and validation samples, and the associated odds ratios were comparable for the two samples. In the validation sample, comorbid substance use disorder was a significant predictor of admissions by years 1 and 2 (OR=2.50, p=.01; OR=1.84, p=.05, respectively), and GAF score was a significant predictor of admissions by years 2 and 3 (OR=.97, p=.03; OR=.97, p=.047, respectively). The initial PANSS score for positive symptoms was a significant predictor of admission by years 2 and 3 in the development sample (OR=1.07, p=.02; OR=1.06, p=.02, respectively). Finally, the C statistics for the model in the development data were .72 or higher for admissions in all periods and ranged from .67 to .72 in the validation data. The 95% bootstrap confidence intervals had lower bounds of .61 or higher and upper bounds of .80 or lower in all cases. We also examined the strength of the model using only prior admission as a variable. This reduced the strength of the risk adjustment model as assessed by a C statistic of .59–.64.

These results represent the first report of a risk adjustment model for a valid, available, and actionable outcome for patients with a first episode of psychosis. The model will require replication, and perhaps modification and external validation with larger samples, but the model serves as a tool that can be used to enhance comparisons of outcome across first-episode psychosis programs, which could lead to changes that improve the quality of care within a program (28). The sample size met the requirements of the power calculations; however, it was relatively small for the development of risk adjustment models, which are commonly applied to large populations of patients. With some planning for the data for both the outcome measure and the predictor variables, data could be collected on a scale that would allow for application to large samples and enhance comparison of performance across programs. We have used regression techniques that represent the preferred approach, compared with stratification techniques and an approach that has been used for examination of hospitalization for mental illness (29). Risk adjustment is an important tool that can be used for quality improvement, more specifically for setting standards and norms for service performance.

As mentioned above, a C value of .5 indicates no predictive discrimination, whereas a value of 1.0 implies perfect prediction (15). With C=.7 representing good discrimination (30), the logistic regression models for both of our samples performed well (C=.67–.74). Moreover, no confidence intervals for the C statistic had a bound lower than .61. We also examined the impact of using only prior admissions as a predictor and found that C dropped to the range of .55–.64, which demonstrates that inclusion of other predictor variables strengthened the prediction model. The point estimates of the C statistics for each model in the validation sample were less than in the development sample, as would be expected (15). However, examination of the confidence intervals suggests that in fact these C statistic estimates were not significantly different. Therefore, the model remained robust in the validation sample, which had baseline characteristics different from those in the development sample. The implication is that this risk adjustment model may continue to discriminate when used to compare outcomes of other programs with diverse patient characteristics.

A limitation of the study is that it examined data from only one region serving a population of about one million. Therefore, it is important to test this risk adjustment model in larger and more diverse populations served by a range of types of services. Many individuals with a first episode of psychosis do not receive specialized first-episode psychosis services, but rates of hospitalization were appropriate for evaluating such general mental health services. Larger studies could also examine other predictor variables, such as financial status and health coverage, in populations where these may vary. Another limitation is that we used secondary data and were therefore unable to include some measures identified by the TRAIT as potential risk factors, such as socioeconomic status. Despite this limitation, the model achieved good discrimination as defined by the C statistic. A further limitation lies in the use of hospital admission as a key performance measure for first-episode psychosis services. Much of the data supporting its use as a performance measure were drawn from studies of multiepisode schizophrenia. There is a need for prospective studies in the first-episode population linking processes of care to this key outcome measure.

We have developed a risk adjustment model that corrects for baseline patient characteristics when comparing hospitalization outcomes after initiation of treatment for first-episode psychosis. The model was robust and was validated in a second sample. The next step should be to test the model in larger populations and across a range of services. Such analysis would require programs to collect baseline data on the variables identified in this project as predictors of hospitalization.

This research was supported by a grant to Dr. Addington from the Alberta Heritage Foundation for Medical Research.

The authors report no competing interests.

Keshavan MS, Amirsadri A: Early intervention in schizophrenia: current and future perspectives. Current Psychiatry Reports 9:325–328, 2007
 
Malla AK, Norman RM, Joober R: First-episode psychosis, early intervention, and outcome: what have we learned? Canadian Journal Psychiatry 50:881–891, 2005
 
Program Policy Framework for Early Intervention in Psychosis. No 7610-2242965. Toronto, Ontario Ministry of Health and Long Term Care, 2004
 
Garety PA, Rigg A: Early psychosis in the inner city: a survey to inform service planning. Social Psychiatry and Psychiatric Epidemiology 36:537–544, 2001
 
Addington D, Mckenzie E, Addington J, et al: Performance measures for early psychosis treatment services. Psychiatric Services 56:1570–1582, 2005
 
Addington D, Mckenzie E, Addington J, et al: Performance measures for evaluating services for people with a first episode psychosis. Early Intervention in Psychiatry 1:157–167, 2007
 
Addington D, Norman R, Mckenzie E, et al: A comparison of early psychosis treatment services using consensus and evidenced-based performance measures: moving towards setting standards. Early Intervention in Psychiatry 2:A33, 2008
 
Burns T: Hospitalisation as an outcome measure in schizophrenia. British Journal of Psychiatry 50(suppl):s37–s41, 2007
 
Hermann RC, Palmer RH: Common ground: a framework for selecting core quality measures for mental health and substance abuse care. Psychiatric Services 53:281–287, 2002
 
Almond S, Knapp M, Francois C, et al: Relapse in schizophrenia: costs, clinical outcomes and quality of life. British Journal of Psychiatry 184:346–351, 2004
 
Weiden PJ, Olfson M: Cost of relapse in schizophrenia. Schizophrenia Bulletin 21:419–429, 1995
 
Craig TJ, Bromet EJ, Jandorf L, et al: Diagnosis, treatment, and six month outcome status in first-admission psychosis. Annals of Clinical Psychiatry 9:89–97, 1997
 
Wunderink L, Nienhuis FJ, Sytema S, et al: Guided discontinuation versus maintenance treatment in remitted first-episode psychosis: relapse rates and functional outcome. Journal of Clinical Psychiatry 68:654–661, 2007
 
Dixon LB, Adams C, Lucksted A: Update on family psychoeducation for schizophrenia. Schizophrenia Bulletin 26:5–20, 2000
 
Harrell FE Jr, Lee KL, Mark DB: Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in Medicine 15:361–387, 1996
 
Hendryx MS, Beigel A, Doucette A: Introduction: risk-adjustment issues in mental health services. Journal of Behavioral Health Services and Research 28:225–234, 2001
 
Hermann RC, Rollins CK, Chan JA: Risk-adjusting outcomes of mental health and substance-related care: a review of the literature. Harvard Review of Psychiatry 15:52–69, 2007
 
Methods Working Group Forum on Performance Measures in Behavioural Health Care: Template for Risk Adjustment Information Transfer (TRAIT). Rockville, Md, US Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, 2003
 
Elixhauser A, Steiner C, Harris DR, et al: Comorbidity measures for use with administrative data. Medical Care 36:8–27, 1998
 
Addington J, Addington D: Early intervention for psychosis; the Calgary Early Psychosis Treatment and Prevention Program. Canadian Psychiatric Association Bulletin 33:11–13, 2001
 
Diagnostic and Statistical Manual of Mental Disorders, 4th ed. Washington DC, American Psychiatric Association, 1994
 
Larsen TK, McGlashan TH, Moe LC: First-episode schizophrenia: 1. early course parameters. Schizophrenia Bulletin 22:241–256, 1996
 
Peduzzi P, Concato J, Kemper E, et al: A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology 49:1373–1379, 1996
 
Steyerberg EW, Harrell FE Jr, Borsboom GJ, et al: Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. Journal of Clinical Epidemiology 54:774–781, 2001
 
Shwartz M, Ash AS: Evaluating risk adjustment models empirically; in Risk Adjustment for Measuring Health Care Outcomes, 3rd ed. Edited by Iezzoni LI. Washington, DC, Academy Health, 2003
 
Efron B: Bootstrap methods: another look at the jackknife. Annals of Statistics 7:1–26, 1979
 
Stata Version 10.0. College Station, Tex, Stata Corp, 2007
 
Addington D: Improving quality of care for patients with first-episode psychosis. Psychiatric Services 60:1164–1166, 2009
 
Hendryx MS, Moore R, Leeper T, et al: An examination of methods for risk-adjustment of rehospitalization rates. Mental Health Services Research 3:15–24, 2001
 
Liu K, Dyer AR: A rank statistic for assessing the amount of variation explained by risk factors in epidemiologic studies. American Journal of Epidemiology 109:597–606, 1979
 
Table 1  Baseline characteristics of first-episode psychosis patients included in development and validation samples for a risk adjustment model
Table 2  Hospital admission as outcome measure for assessing a risk adjustment model with development and validation samples
Table 3  Performance of a risk adjustment model for predicting hospital admission among patients in development and validation samples who had a first episode of psychosis
Table 1  Baseline characteristics of first-episode psychosis patients included in development and validation samples for a risk adjustment model
Table 2  Hospital admission as outcome measure for assessing a risk adjustment model with development and validation samples
Table 3  Performance of a risk adjustment model for predicting hospital admission among patients in development and validation samples who had a first episode of psychosis
+

References

Keshavan MS, Amirsadri A: Early intervention in schizophrenia: current and future perspectives. Current Psychiatry Reports 9:325–328, 2007
 
Malla AK, Norman RM, Joober R: First-episode psychosis, early intervention, and outcome: what have we learned? Canadian Journal Psychiatry 50:881–891, 2005
 
Program Policy Framework for Early Intervention in Psychosis. No 7610-2242965. Toronto, Ontario Ministry of Health and Long Term Care, 2004
 
Garety PA, Rigg A: Early psychosis in the inner city: a survey to inform service planning. Social Psychiatry and Psychiatric Epidemiology 36:537–544, 2001
 
Addington D, Mckenzie E, Addington J, et al: Performance measures for early psychosis treatment services. Psychiatric Services 56:1570–1582, 2005
 
Addington D, Mckenzie E, Addington J, et al: Performance measures for evaluating services for people with a first episode psychosis. Early Intervention in Psychiatry 1:157–167, 2007
 
Addington D, Norman R, Mckenzie E, et al: A comparison of early psychosis treatment services using consensus and evidenced-based performance measures: moving towards setting standards. Early Intervention in Psychiatry 2:A33, 2008
 
Burns T: Hospitalisation as an outcome measure in schizophrenia. British Journal of Psychiatry 50(suppl):s37–s41, 2007
 
Hermann RC, Palmer RH: Common ground: a framework for selecting core quality measures for mental health and substance abuse care. Psychiatric Services 53:281–287, 2002
 
Almond S, Knapp M, Francois C, et al: Relapse in schizophrenia: costs, clinical outcomes and quality of life. British Journal of Psychiatry 184:346–351, 2004
 
Weiden PJ, Olfson M: Cost of relapse in schizophrenia. Schizophrenia Bulletin 21:419–429, 1995
 
Craig TJ, Bromet EJ, Jandorf L, et al: Diagnosis, treatment, and six month outcome status in first-admission psychosis. Annals of Clinical Psychiatry 9:89–97, 1997
 
Wunderink L, Nienhuis FJ, Sytema S, et al: Guided discontinuation versus maintenance treatment in remitted first-episode psychosis: relapse rates and functional outcome. Journal of Clinical Psychiatry 68:654–661, 2007
 
Dixon LB, Adams C, Lucksted A: Update on family psychoeducation for schizophrenia. Schizophrenia Bulletin 26:5–20, 2000
 
Harrell FE Jr, Lee KL, Mark DB: Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in Medicine 15:361–387, 1996
 
Hendryx MS, Beigel A, Doucette A: Introduction: risk-adjustment issues in mental health services. Journal of Behavioral Health Services and Research 28:225–234, 2001
 
Hermann RC, Rollins CK, Chan JA: Risk-adjusting outcomes of mental health and substance-related care: a review of the literature. Harvard Review of Psychiatry 15:52–69, 2007
 
Methods Working Group Forum on Performance Measures in Behavioural Health Care: Template for Risk Adjustment Information Transfer (TRAIT). Rockville, Md, US Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, 2003
 
Elixhauser A, Steiner C, Harris DR, et al: Comorbidity measures for use with administrative data. Medical Care 36:8–27, 1998
 
Addington J, Addington D: Early intervention for psychosis; the Calgary Early Psychosis Treatment and Prevention Program. Canadian Psychiatric Association Bulletin 33:11–13, 2001
 
Diagnostic and Statistical Manual of Mental Disorders, 4th ed. Washington DC, American Psychiatric Association, 1994
 
Larsen TK, McGlashan TH, Moe LC: First-episode schizophrenia: 1. early course parameters. Schizophrenia Bulletin 22:241–256, 1996
 
Peduzzi P, Concato J, Kemper E, et al: A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology 49:1373–1379, 1996
 
Steyerberg EW, Harrell FE Jr, Borsboom GJ, et al: Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. Journal of Clinical Epidemiology 54:774–781, 2001
 
Shwartz M, Ash AS: Evaluating risk adjustment models empirically; in Risk Adjustment for Measuring Health Care Outcomes, 3rd ed. Edited by Iezzoni LI. Washington, DC, Academy Health, 2003
 
Efron B: Bootstrap methods: another look at the jackknife. Annals of Statistics 7:1–26, 1979
 
Stata Version 10.0. College Station, Tex, Stata Corp, 2007
 
Addington D: Improving quality of care for patients with first-episode psychosis. Psychiatric Services 60:1164–1166, 2009
 
Hendryx MS, Moore R, Leeper T, et al: An examination of methods for risk-adjustment of rehospitalization rates. Mental Health Services Research 3:15–24, 2001
 
Liu K, Dyer AR: A rank statistic for assessing the amount of variation explained by risk factors in epidemiologic studies. American Journal of Epidemiology 109:597–606, 1979
 
+
+

CME Activity

There is currently no quiz available for this resource. Please click here to go to the CME page to find another.
Submit a Comments
Please read the other comments before you post yours. Contributors must reveal any conflict of interest.
Comments are moderated and will appear on the site at the discertion of APA editorial staff.

* = Required Field
(if multiple authors, separate names by comma)
Example: John Doe



Related Content
Articles
Books
Textbook of Traumatic Brain Injury, 2nd Edition > Chapter 28.  >
DSM-5™ Handbook of Differential Diagnosis > Chapter 2.  >
DSM-5™ Clinical Cases > Chapter 2.  >
DSM-5™ Clinical Cases > Chapter 2.  >
DSM-5™ Clinical Cases > Chapter 3.  >
Topic Collections
Psychiatric News