Measures of Personal Recovery: A Systematic Review

Personal recovery is “a deeply personal, unique process of changing one’s attitudes, values, feelings, goals, skills and/or roles. It is a way of living a satisfying, hopeful, and contributing life even within the limitations caused by illness” (1). An orientation toward personal recovery is central to mental health policy in Australia (2), Canada (3), England (4), New Zealand (5), the United States (6), and other nations (7).

If recovery is the policy orientation, then measures to assess the impact of interventions on recovery are needed. However, definitions and conceptualizations of recovery vary (8). This variety makes it difficult to make decisions about what should be measured. To measure recovery as an outcome in mental health, a conceptual framework must be developed that encompasses the best available evidence on recovery processes. A conceptual framework is a network of interlinked concepts, each playing an integral role, that provides a comprehensive understanding of a concept (9). Frameworks for understanding how recovery unfolds, such as the personal recovery framework (10) and the RECOVER framework (11), have been developed.

More recently, the CHIME framework for personal recovery was developed through a systematic review and narrative synthesis of recovery (12). The framework consists of three superordinate categories of recovery: characteristics of the recovery journey, recovery processes, and recovery stages. The acronym CHIME derives its name from the recovery processes identified by the framework, namely connectedness, hope and optimism, identity, meaning and purpose, and empowerment. This conceptual framework allows measures to be evaluated for the extent to which they measure recovery rather than other aspects of good practice in mental health services.

Mental health problems and recovery are influenced by social factors (13). According to some, this means that mental health services should focus on the core business of treating illness. Others argue that, in the future, mental health professionals will need to better support access to community resources (14), becoming social activists involved in challenging discrimination in wider society (15). Whatever the future, adequate measures of recovery are needed. Recovery has two distinct components—recovery as experienced by the individual and the support for recovery offered by the mental health system. This study focuses on recovery as an experience. Notably, a systematic review of measures of recovery orientation found an absence of usable measures (16).

Several compendia of outcome measures have been published. Some have a broad focus on outcome (17), and others relate to specific populations, such as persons with addictions (18) and schizophrenia (19). Compendia with a specific focus on recovery are also available (20). Three recent reviews of recovery measures have been published. The first looked at the suitability of 33 recovery measures for an Australian setting (21) and recommended four measures for future consideration: the Illness Management and Recovery (IMR) Scale (22), the Recovery Assessment Scale (RAS) (23), the Recovery Process Inventory (RPI) (24), and the Stages of Recovery Instrument (STORI) (25). The review concluded that further work was needed to determine which, if any, of the measures is suitable for routine use. The second review looked at measures of recovery from psychosis (26). It identified six measures for consideration: the IMR, the Psychosis Recovery Inventory (PRI) (27), the Questionnaire About the Process of Recovery (QPR) (28), the RAS, the RPI, and the STORI. The RAS was identified as most published, and the QPR was the most favored by service user consultants. Further research for investigating reliability and utility within clinical settings is needed.

The third review identified 13 measures of recovery for schizophrenia and judged the RAS to be the best of the reviewed measures (29). To date, no review of recovery measures has used a full range of data sources (including conferences and Web site repositories), included ratings of articles’ eligibility by two researchers to improve quality assurance, included a flow diagram consistent with reporting guidelines (30), or operationalized the definition of recovery.

This literature review aimed to identify measures of personal recovery, evaluate the extent to which the identified measures focus on aspects of recovery defined by the CHIME framework, and characterize the psychometric properties of each identified measure.

Methods

Search strategy

Searches were conducted by using six data sources. First, eight databases were searched from date of inception to May 2012: MEDLINE, PsycINFO, EMBASE, CINAHL, CSA Illumina, TRIP, CDSR, and DARE. The search terms were divided into four domains—personal recovery, mental illness, measure or instrument, and psychometric properties. The terms were identified from the title, abstract, key words, and medical subject headings (MeSH). The search terms were amended for each database as necessary. A copy of the full list of search terms is available from the first author.

Second, 11 Web-based repositories were searched by using the terms “personal recovery,” “mental health,” and “measure.” The repositories were the Department of Health (www.dh.gov.uk), the Scottish Recovery Network (www.scottishrecovery.net), the Centre for Mental Health (www.centreformentalhealth.co.uk), Recovery Devon (www.recoverydevon.co.uk), the Repository of Recovery Resources (www.bu.edu/cpr/repository/index.html), Mind (www.mind.org.uk), Rethink (www.rethink.org), the National Mental Health Development Unit (www.nmhdu.org.uk), the Mental Health Commission of New Zealand (www.mhc.govt.nz/), the Mental Health Commission of Ireland (www.mhcirl.ie/), and the Mental Health Commission of Canada (www.mentalhealthcommission.ca/).

Third, a search of Google Scholar (www.scholar.google.co.uk) was conducted by using the terms “personal recovery,” “mental health,” and “measure.” Fourth, abstracts from three international conference series were searched by using the terms “personal recovery” and “measure.” The conferences were biennial conferences of the European Network for Mental Health Service Evaluation (1994–2010), annual meetings of the American Psychiatric Association (1999–2009), and Refocus on Recovery conferences (2010 and 2012). Fifth, the table of contents was searched by hand for the following journals: Psychiatric Services, International Journal of Methods in Psychiatric Research, and Psychiatric Rehabilitation Journal. Sixth, reference lists of included papers were assessed for further measures.

Aim 1: identification of measures

Eligibility criteria.

Articles were included if they involved the use or validation of a measure of personal recovery, were published in either peer-reviewed or non–peer-reviewed publications and were Web accessible, and involved a population of working-age (ages 16 to 65) adults with a diagnosis of any mental illness other than an eating or substance use disorder.

Data extraction.

Data were extracted by the first author. Results from the search were stored in Endnote × 4. Duplicates were removed. The results were then sifted to exclude papers by title or by language. The abstracts of papers that were included by title were sifted further, allowing for exclusion by abstract. Excluded abstracts were also rated by a second rater (JW) to assess reliability. A concordance level of 98.2% was achieved. If the abstract was judged to be relevant, the full paper was reviewed and a copy of the measure was obtained. A decision was made whether to include the measure.

Aim 2: recovery relevance

To evaluate the extent to which the identified measures focus on aspects of recovery, four raters independently compared the measures to the CHIME framework. Each item on the measure was mapped to one of the five CHIME framework categories, if possible. Items covering more than one domain were assigned to the domain that it represented most. Items that did not map, and, therefore, did not assess personal recovery, were counted as not mapping. A concordance in allocation to CHIME category of over 70% was achieved between raters.

Aim 3: psychometric properties

Each measure was evaluated for nine psychometric properties by using criteria modified from several authors (19–21). For each property, the measure was rated positive (adequate demonstration), indeterminate (inconclusive evidence), negative (inadequate demonstration), or not enough information (no evidence available).

Results

Aim 1: identification of measures

From 336 abstracts reviewed, the search identified for inclusion 35 articles, which described 13 measures. Eight articles described measure development, nine were reports of psychometric properties, 16 articles described studies that used a measure as part of outcome assessment, and one article compared measures of personal recovery and clinical recovery. One measure, the Stages of Recovery Scale, was unobtainable and was excluded from analysis. The 12 measures are described in Table 1. [A flow diagram of the identification of the measures is available online as a data supplement to this article.]

All measures are rated by service users, and the IMR also has a clinician-rated version. The two measures cited most widely in the search were the RAS and the IMR. The RAS appeared in 13 articles, including four describing psychometric properties, eight describing use as an outcome measure (three studies each from the United States and Australia and one each from Canada and Sweden), and one comparing consumer-defined and clinically defined recovery measures. The IMR appeared in eight articles, including two describing psychometric use and six describing studies that used it as an outcome measure (one each from Sweden and Israel and four from the United States). The only other measure used as an outcome assessment, by two studies from the United States, was the Recovery Markers Questionnaire (RMQ) (31).

Aim 2: recovery relevance

The mapping of items from each measure to the CHIME framework is shown in Table 2. The QPR was the only measure to have every item map to the CHIME conceptual framework and the only measure to have at least 10% of items in each category.

Aim 3: psychometric assessment

Three measures (the Mental Health Recovery Measure [32], the RMQ, and the Self-Identified Stage of Recovery [33]) were not included in any psychometric or scale development articles and, therefore, were excluded from psychometric analysis. Three measures were included in more than one psychometric article (the RAS, four articles, and the IMR and the STORI, two each). Findings from all articles were included in the evaluation. The psychometric evaluation of measures is shown in Table 3.

The measure with positive ratings for the highest number of the nine investigated properties was the STORI (N=6 properties), followed by the Maryland Assessment of Recovery (MARS) (34) (N=5); the QPR and the RAS (N=4); the IMR, Recovery Star (35), and the RPI (N=3); and the Short Interview to Assess Stages of Recovery (33) (N=2). The psychometric properties demonstrated by the most measures were content validity (N=9 measures), internal consistency (N=8) and test-retest reliability (N=7). The least demonstrated properties were criterion validity (N=0), responsiveness (N=1), reading age (N=1), and feasibility (N=2).

Discussion

A total of 13 measures of recovery were identified. The QPR had the strongest match with recovery, the RAS was the most widely published, and the STORI, the MARS, the QPR, and the RAS demonstrated the widest range of psychometric properties. All measures were also identified in at least one of the three recent reviews of recovery measures mentioned earlier (21,26,29).

Although recovery has become a widely used term in health services research, inconsistency in its meaning and operational definition has obscured the clarity of the concept. It has been said that its scope can “make a cow catcher on the front of a road train look discriminating” (36). This review has found that the CHIME framework could be used to evaluate the extent to which a measure assesses recovery rather than other aspects of good practice.

In general, the findings of this review concur with other work in this area, namely that no measure has been subject to a substantial and robust psychometric evaluation. The content validity of all the measures had been addressed during development through the use of service users, supporting the findings of Campbell-Orde and colleagues (20) that measures were largely based on consumers’ views and experiences. However, as shown in Table 3, the psychometric evaluation showed a less consistent focus on other aspects of validity and on reliability and feasibility.

The lack of measurement of responsiveness is an issue for self-report measures. It is crucial that measures demonstrate the ability to detect change over time, for both clinical or research purposes. The findings indicate the need for a gold standard measure to assess criterion validity.

Strengths and limitations

This review had three strengths. First, the methodological rigor advances the field by using more sources of data than existing reviews (21,26,29) and double ratings of eligibility as a quality check, by including a flow diagram, and by operationalizing the definition of recovery. Second, although existing reviews included staff-rated measures and measures of recovery orientation, there was substantial overlap between the measures identified in this review and earlier reviews. Third, the distribution of country of development for the measures (Table 1) supports the view that recovery conceptualizations—at least as present in English-language sources—have primarily emerged from the English-speaking world (37).

The review had three limitations. First, it excluded non–English-language papers, possibly missing some measures. Second, the term “personal recovery” is not a MeSH heading in databases, and, therefore, the search may have lacked some specificity and missed some measures. The use of the large number of search criteria aimed to counter this possibility. Finally, applying the CHIME framework is a subjective process that depends on individual interpretations of the measures and the meaning of each item. Each CHIME category contains many subcategories (12), so a measure may not adequately assess a category if only a small number of items map to it. A measure’s uneven coverage of the CHIME categories means that the summary score emphasizes different components of recovery. CHIME also represents only one conceptualization of recovery, and the use of other frameworks or conceptual backgrounds may have resulted in different findings. Several measures have items not mapping to CHIME, possibly because these items represent nonrecovery elements of best practice or because CHIME needs to be extended to incorporate other recovery processes. Although the CHIME framework captures an understanding of recovery within the English-speaking world (37), its wider applicability is unknown.

Implications

The challenge in mental health services is developing a system that is based on the views of service users about how their recovery can be supported. Problems with conceptual clarity and inadequate psychometric testing of recovery measures are barriers to meeting this challenge. Having a clear understanding of the conceptual basis of recovery is necessary to enable service providers to provide recovery-focused interventions. If a service is to be recovery focused, the interventions need to be oriented toward this goal. Whether mental health services should be judged in relation to their recovery orientation or by change in service recipients’ experience of recovery is an important scientific and clinical question. Stronger measurement tools are needed for either approach.

It is also necessary for service providers to be able to identify the resource implications of using recovery measures in routine outcome assessment. The development of consensus on how recovery is measured will allow investigation of the resource consequences of transformation toward a recovery orientation. In order for recovery measurement to advance to the next stage of scientific development, it will be necessary to refine theoretically defensible (38) and psychometrically adequate measurement tools that address recovery-specific evaluation issues (39).

Conclusions

This review has identified three knowledge gaps to inform future research. First, no measure has yet had a complete psychometric evaluation, although the RAS and the QPR have the strongest evidence base. Future research of recovery measures must specifically include sensitivity to change. Measuring change will involve an empirically defensible conceptualization of recovery as a construct—for example, whether it is a continuous process or occurs in discrete stages—and the methodological rigor to ensure best practice in evaluation (40). Second, there is a clear need to identify a “gold standard” measure for assessing criterion validity. Finally, measures need to be evaluated for a range of service settings, clinical populations, and languages.