Systematic Review of Symptom Assessment Measures for Use in Measurement-Based Care of Bipolar Disorders

Even while engaging in treatment, many individuals with a bipolar disorder experience symptoms of mania and depression that fluctuate or occur concurrently (1–7). Failure to systematically assess symptoms and compare them with prior clinical status can lead to inaccurate detection of nonresponse and uncertainty about when to make treatment changes (8). Likewise, the presence of residual depressive or hypomanic symptoms is associated with poor outcomes, including recurrence of a mood episode (7)—highlighting the need for ongoing symptom assessment and treatment to target (i.e., remission).

Measurement-based care (MBC) is a clinical strategy involving regular measurement of symptom frequency and severity, side effects, and treatment adherence and use of those findings to inform clinical decision making (9–11). Existing literature demonstrates that MBC is effective for treating patients with most psychiatric disorders and that adoption of MBC has been recommended in the treatment of individuals with a range of psychiatric illnesses (9).

In the past decade, several organizations have recommended the adoption of MBC specifically for the treatment of bipolar disorder. In 2009, the International Society for Bipolar Disorders (12) recommended using symptom measures at baseline and at follow-up clinical visits to aid clinicians in determining clinical response and remission for individuals with bipolar disorder. The report also noted that symptom measurement can provide additional clinical insights, such as determining the predominant polarity of a mixed episode (12). Guidelines published by the U.S. Department of Veterans Affairs (VA) and the Department of Defense (DoD) also recommend using symptom measures to monitor treatment of bipolar disorder, but unlike VA/DoD guidelines for depression (13), they do not provide specific instructions about which measures to use, how to interpret results of any specific measure, or frequency of measurement (14). The absence of clear guidance in MBC of bipolar disorder—as well as limited clinician understanding of available measures that could be used as options—may have contributed to low adoption of MBC for this clinical population.

Prior reviews of bipolar disorder measures, published in 2009 (15) and 2013 (16), included measures of bipolar disorder symptoms and screening tools and other instruments that are not used for serial symptom assessment. Neither review used a comprehensive systematic review methodology that included searching multiple databases, assessed a full range of psychometric properties, or evaluated clinical utility. We sought to extend prior reports by conducting a systematic review of instruments that could be used for MBC of bipolar disorder. In particular, we sought to answer the following questions, What patient-reported and clinician-observed measures of bipolar disorder symptoms exist? What are the psychometric properties and clinical utility of the existing measures?

Methods

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) method to conduct and report the results of this review (17).

Search Strategy

Search strategies were developed to capture articles about bipolar disorders, symptom measures, and measurement psychometrics. (Complete search strategies are available in the online supplement). Searches were constructed by using a combination of keywords and standardized terms in PubMed, Embase, PsycINFO, and the Cochrane Register of Controlled Trials. Gray literature sources were also searched, including ClinicalTrials.gov, ProQuest Dissertations and Theses, and the World Health Organization’s International Clinical Trials Registry Platform. Results were filtered for English articles, adults, and years 1990 to the present. Searches were conducted in June 2017. Citations were managed in EndNote, including removal of duplicates, and the excluded and included citations were organized by using the Rayyan Web application for systematic reviews. Reference lists of selected studies, including the literature reviews (15, 16), were hand-searched to identify additional scales, including those described in articles published before 1990.

Eligibility and Exclusion

Article titles found in the search were screened for relevance to the topic by the first author. Selected abstracts were then screened. Articles were eligible if they described symptom measures for adults with a bipolar disorder, were published in English, and addressed measurement psychometrics (e.g., validity or reliability).

Studies that did not report on individuals with a bipolar disorder, did not include adult populations, did not measure bipolar disorder symptoms, or did not include primary data (e.g., review articles) were excluded. We also excluded reports on instruments that would not be appropriate for use in MBC, such as screening measures used for case identification and instruments assessing only one symptom (e.g., suicidal ideation).

Data Abstraction

We developed a data abstraction tool. Abstracted information included study author and year; study population and clinical setting; and other details about the measure, including the number of items, time frame assessed, and scoring of items. One author (JMC) reviewed eligible studies to complete the data abstraction tool. All eligible studies were reviewed by two additional authors (SBG and JCF) to assess information required for computing the clinical utility scores described below. Disagreements were resolved through discussion among authors and through consultation with a multidisciplinary group of researchers within the Department of Psychiatry and Behavioral Sciences at the University of Washington School of Medicine. Corresponding authors were contacted for missing data.

To quantify the clinical utility of the included measures, we adapted a method developed by Zimmerman et al. to describe the clinical utility of symptom measures (18). Clinical utility was assessed based on 11 items related to content, use, or psychometrics of the measure for individuals with bipolar disorder, including three items of validity and two items of reliability. Items (with cutoff criteria when applicable) included whether the instrument was brief (≤18 items); assessed suicidal thoughts; was easy to score (total score computed by adding individual item responses); was publicly available (determined by author report or identified through Internet search); reported a remission indicator in included study (a score suggesting clinical remission); and was adequate in internal consistency (Cronbach’s α ≥.7), test-retest reliability (Pearson correlation coefficient ≥.6), content validity (proportion of assessed DSM-5 symptoms of depression and mania), concurrent validity (Pearson or Spearman’s correlation coefficient ≥.6), construct validity (either convergent or discriminant validity; p<.05), and sensitivity to change (p<.05). Cutoffs for psychometric properties were based on prior reports (19–21).

Reliability items assessed the degree to which an instrument consistently measures a construct, across both items and time points. Internal consistency assessed whether the instrument consistently measures the construct across items in the scale. Test-retest reliability assessed whether the instrument measures the construct consistently across time. Content validity assessed the extent to which the instrument measures all facets of a given construct. Concurrent validity assessed whether the instrument measures the same construct as a validated instrument when administered at the same time. In most cases, construct validity was assessed by whether the scale distinguishes between patients diagnosed as having (convergent validity) or not having (discriminant validity) the relevant diagnosis. Discriminant validity assessed whether the instrument does not measure unrelated constructs. Sensitivity to change assessed whether the instrument captures variation in symptoms over time.

Each item was initially given a score of 0 if the item was absent or did not meet cutoff criterion and a score of 1 if the item was present and met cutoff criterion, with two exceptions. The test-retest reliability item was scored 0.5 if the item was found to meet cutoff criterion only in a control group. The content validity item was given a score from 0 to 1 based on the proportion of DSM symptoms included in the measure, depending on whether the measure was intended to assess depression, mania, or depression and mania. A score of 1 indicated that all DSM-5 symptoms from the relevant category (or categories) are included in the measure, and a score of 0 indicated that no DSM-5 symptoms from the relevant category (or categories) are included (22). For measures that do not include all DSM-5 symptoms, the total number of DSM-5 symptoms included was divided by the number of relevant symptoms listed in DSM-5 for the respective condition (nine for measures assessing only depressive symptoms, seven for measures assessing only manic symptoms, and 16 for measures assessing both). Given that the sample size required to assess a measure’s reliability and validity is partially dependent on the length (i.e., number of items) of the measure (23), scores for psychometric items were adjusted based on the ratio of sample size to number of items in the measure. Ratios of sample size to item were calculated based on the sample size used in the analysis of the measure’s psychometric property. If a measure was evaluated in multiple studies, we added the analytical sample sizes together. The initial score was multiplied by one if the ratio of sample size to item was excellent (≥10), by 0.75 if the ratio was very good (≥5 and <10), by 0.5 if the ratio was good (≥3 and <5), by 0.25 if the ratio was fair (≥2 and <3), and by 0 if the ratio was poor (<2). A clinical utility score ranging from 0 to 11 was determined for each measure by summing the values for each of the 11 components, with higher scores reflecting higher utility.

Results

Our search resulted in 4,617 citations, and 14 citations were identified through other sources such as hand-searching of bibliographies. After the removal of 417 duplicate citations, 4,214 unique citations remained and were assessed through title and abstract review. Seventy-three studies were assessed for eligibility through full-text review, and 39 studies (24–62) were included in the qualitative synthesis and clinical utility scoring. Search results are shown in a flow diagram (see figure in online supplement). A summary of included studies is shown in Table 1.

Twenty-eight symptom measures were identified in 39 studies, including 10 measures of manic symptoms (four patient-reported and six clinician-observed), eight measures of depressive symptoms (three patient-reported and five clinician-observed), and 10 measures of both manic and depressive symptoms (six patient-reported and four clinician-observed). One measure, the Observer-Rated Scale for Mania (36), was developed to help nonclinicians communicate with clinicians, although the measure could be used over time by clinicians to monitor treatment. For this study it was classified as a clinician-observed measure of manic symptoms.

Measures Assessing Manic Symptoms

Thirteen studies (24–36) described 10 instruments assessing manic symptoms only. Four instruments were patient-reported, and six were clinician-observed. Seven measures (27, 30–36) of manic symptoms, including all six clinician-observed measures, were initially tested among hospitalized patients with bipolar disorder who were receiving treatment for mania. Most clinician-observed measures assessed current symptoms, whereas all patient-reported measures assessed symptoms over the preceding week to month. The number of items per measure ranged from five to 49, with two patient-reported measures (28, 29) having a variable number of items contingent on patient response. Four studies (26, 28, 33, 36) (evaluating three different measures) included at least 100 patients with bipolar disorder.

Overall clinical utility scores and scores for each item are shown in Table 2 (see online supplement for expanded information). For measures of manic symptoms, clinical utility scores ranged from 3 to 9.6 for patient-reported measures and from 3 to 6.55 for clinician-observed measures. The measure with the highest clinical utility score was the patient-reported Altman Self-Rating Mania Scale (27). The clinician-reported measure with the highest clinical utility score, the Bech-Rafaelsen Mania Rating Scale (31, 32), included tests of internal consistency but lacked tests of test-retest reliability and discriminant validity.

Recent innovations included two studies (28, 29) that evaluated use of computerized technology to improve administration efficiency and clinical accuracy by focusing more specifically on relevant symptom areas. One measure used adaptive testing technology in which a variable number of items from a bank of 89 were selected for administration on the basis of the prior responses of the patient (29). This scale generates a severity score within a fixed range, regardless of how many items are administered.

Measures Assessing Depressive Symptoms

Eight studies (37–44) described eight instruments assessing depressive symptoms only. Three measures were tested in samples including hospitalized patients who were diagnosed as having a bipolar disorder and who were experiencing depressive or mixed symptoms (38, 42, 44). The number of items per measure range from five to 52. All five studies (40–44) evaluating clinician-observed measures included at least 100 patients with bipolar disorder.

Clinical utility scores for clinician-observed measures ranged from 5.5 to 10. The clinician-observed Quick Inventory of Depressive Symptoms (40, 41) and the five-item Hamilton Depression Rating Scale (HAMD-5) (44) had the highest clinical utility scores (10 and 8.3, respectively). A patient-reported version of the Quick Inventory of Depressive Symptomatology had a relatively high clinical score (6.75).

Five measures (38, 40, 41, 43, 44) were originally developed for use among individuals with major depression before being studied among individuals with bipolar disorder. One measure, the 24-item, clinician-observed Bipolar Depression Rating Scale, was developed specifically for use among individuals diagnosed as having a bipolar disorder on the basis of observed differences in the phenomenology of depression between individuals with bipolar disorder or major depressive disorder (42).

Measures Assessing Both Manic and Depressive Symptoms

Nineteen studies (26, 45–62) described 10 instruments assessing both manic and depressive symptoms. Six instruments were patient-reported and four were clinician-observed. The number of items per measure ranges from two to 44. Nine studies included at least 100 individuals with bipolar disorder and evaluated six different measures (26, 50–53, 57, 59, 61, 62).

Clinical utility scores ranged from 1.4 to 7.73 for patient-reported measures and from 5.65 to 8 for clinician-observed measures. The instruments with the highest clinical utility scores included the patient-reported Internal State Scale (26, 45–48) and the clinician-observed Bipolar Inventory of Symptoms Scale (60–62).

Three patient-reported measures (47, 48, 53–55), each with up to six items, assess symptoms daily and require individuals with bipolar disorder to complete assessments outside the context of a clinical encounter.

Discussion and Conclusions

This systematic review of measures for assessing symptoms of bipolar disorder identified numerous candidates for use in MBC. Across the 28 measures we identified, approximately half were patient-reported and half were clinician-observed. Ten measures assessed depressive and manic symptoms, whereas the remaining measures assessed either depressive or manic symptoms. On the whole, considerable variability was found regarding the strength of the psychometric properties and clinical utility of the measures reviewed (scores ranged from 1.4 to 10).

Our results also revealed a temporal trend in the type of measures being developed. Measures developed more recently focus on the assessment of depressive symptoms or depressive and manic symptoms among outpatients, whereas earlier studies primarily assessed manic symptoms among hospitalized individuals. This trend is consistent with a growing understanding of the clinical course of patients who experience chronic depressive or mixed symptoms and of the effort to focus more on outpatient care of individuals with bipolar disorder (2, 4–6, 63).

How might a clinician or practice choose which measure to use? As guidance to clinicians on how to choose among the high number of depression measures, Kroenke (64) recently suggested that measure selection could be informed by clinical utility features—such as ease of scoring, brevity, and degree of uptake by other clinicians. Following this advice, we suggest that the clinical utility scores reported in our study may similarly help to guide clinicians in choosing which measure to use when caring for individuals with bipolar disorder. Choice of a measure may also depend on how each clinician or practice weighs each category of the clinical utility scale (e.g., for some, scale brevity may be more highly valued than test-retest reliability) and on the clinical setting and presenting problem.

This review suggests that a variety of measures have promising clinical utility for use in MBC of bipolar disorder. One patient-reported mania scale, the Altman Self-Rating Mania Scale (27), had a high clinical utility score, with strengths such as being brief and easy to score and having good reliability and validity. Because this measure assesses manic symptoms only, general use in certain settings (e.g., outpatient clinics) would likely require combining it with a depression symptom measure, such as the Quick Inventory of Depressive Symptomatology Self-Report (39). Use of these two patient-reported measures together was described in a report on an MBC program for adults diagnosed as having bipolar disorder (65). Regarding clinician-observed measures, two clinician-observed depression scales, the Quick Inventory of Depressive Symptomatology (40, 41) and the HAMD-5 (44), had high clinical utility scores. However, the clinician-administered mania scales all had lower clinical utility scores than the Altman Self-Rating Mania Scale.

For clinicians and systems that prefer using either the patient-reported method or the clinician-observed method, but not both methods together, the scales assessing both mania and depression may have the most utility. Two patient-reported mania and depression scales—the Internal State Scale (26, 45, 46) and the Affective Self-Rating Scale (49)—had moderately high clinical utility scores (7.73 and 7.15, respectively). The Internal State Scale (26, 45, 46) assesses a range of symptoms consistent with the clinical course of many individuals with bipolar disorder and its psychometric properties have been evaluated in depth, although it is more difficult to score than other measures. The Affective Self-Rating Scale (49) also assesses a range of symptoms, including increased and decreased sleep and thought speed, and is scored by summing item responses, although much of the psychometric evaluation was conducted on a smaller sample size compared with studies of the Internal State Scale.

Two clinician-observed mania and depression scales had moderately high clinical utility scores. The Bipolar Inventory of Symptoms Scale (60), with 44 items and a clinical utility score of 8, has been the focus of a psychometric evaluation in three studies. Clinicians with experience caring for individuals with bipolar disorder or those who work in bipolar disorder specialty settings may be better able to appreciate the detail and subtleties of this measure. The Life Chart Methodology–Clinician, with a score of 7.6, tracks symptoms on a chart, permitting rapid evaluation of an individual’s clinical course over time.

Two studies explored psychiatrist-reported barriers to use of MBC in general (66, 67). Psychiatrists reported not using symptoms measures for reasons including uncertainty about which measure to use. We found that most measures have been examined to some extent for reliability or validity, and numerous measures have moderately high to high clinical utility. In addition, the available patient-reported measures do not require clinician time or expertise in administration and, therefore, address concerns about use of clinician time and the level of familiarity required to administer measures. Given adequate clinic infrastructure, patients could complete the measure before an encounter with a clinician. Furthermore, some measures are intended for patients to complete outside of clinic settings, which may be appropriate if clinic infrastructure cannot support administration of measures. Clinic kiosks or home-based administration can allow patients to complete measures prior to a clinical encounter, allowing results to inform clinical decision making during a subsequent face-to-face visit (64).

The multisite Systematic Treatment Enhancement Program for Bipolar Disorder network of studies included use of the Clinical Monitoring Form, a measure that assesses depressive and manic symptoms, helps clinicians to assess clinical status, and guides decision making at clinic visits (58, 68). Additionally, reports from mood disorder specialty settings and general psychiatry clinics demonstrated the feasibility of using measures to monitor treatment. These reports demonstrated that enhanced treatment programs including symptom measurement for bipolar disorder is associated with better outcomes compared with usual care (69–75).

Limitations of the current study included comparing clinical utility scores for measures assessing depressive or manic symptoms only versus measures assessing both symptom domains. Additionally, if a measure lacked testing of a psychometric property, we applied an item score of 0 in the clinical utility score, although it is possible that the property is present and adequate. Although most measures were developed prior to DSM-5, we applied symptoms listed in DSM-5 to assess content validity for all measures, given that this classification reflects current practice. Our database search included results from 1990 to 2017, which includes four years prior to the publication of DSM-IV. It is possible that measures not included in our study were published prior to 1990; however, we identified measures published prior to 1990 through a citation review of included studies and an assessment of two prior reviews of bipolar disorder symptom measures. An expanded literature review of older instruments revealed no additional psychometric testing.

A potential next step could include determining which measures are most acceptable to clinicians and to patients. Additionally, the Patient Health Questionnaire–9 (PHQ-9), a patient-reported measure of depression symptoms in wide use for monitoring treatment of depression, is notably absent from our results because of a lack of studies meeting inclusion criteria. Because the PHQ-9 is commonly used and acceptable to many clinicians, a future direction could be to evaluate the psychometric properties of the PHQ-9, possibly in combination with a measure of manic symptoms, among individuals with bipolar disorder.