Serious mental illness is the second-leading cause of disability in the United States among persons ages 15 to 44 (1) and accounts for between 5,000 and 10,000 disability-adjusted life years lost worldwide per year per one million population. In 2002, the costs of serious mental illness were estimated to exceed $100 billion in health care expenditures, $193 billion in lost earnings, and $24.3 billion in disability benefits, for a total of over $317 billion (2).

Serious mental illness represents the largest diagnostic category for people receiving Supplemental Security Income and Social Security Disability Insurance payments. More than 30% of the U.S. population suffers from a mental illness each year (3). Of this group, 22% of mental illnesses are classified as serious, 37% as moderate, and 40% as mild (4). In sum, about 17.8 million people (5.8% of the U.S. population) live with serious mental illness in any given year.

The results of three large comparative effectiveness trials have been sobering, arguably highlighting the limitations of our current ability to help many patients afflicted with serious mental illnesses, such as schizophrenia, bipolar disorder, and depression (3,5,6). Reports from the Schizophrenia Patient Outcomes Research Team underscore that even when effective treatments are available, implementation and uptake are suboptimal (7).

The Agency for Healthcare Research and Quality (AHRQ) supports a variety of patient-centered outcomes research (PCOR) activities through its Effective Health Care (EHC) program (8). To contribute to AHRQ’s agenda for PCOR activities related to mental health, we engaged a broad and representative group of stakeholders to discuss issues related to serious mental illness in a series of three meetings called an Issues Exploration Forum (IEF). This article is a summary of a report prepared for AHRQ about the results of the IEF (9).

The IEF’s overarching purpose was to engage a diverse group of stakeholders to help establish priorities to guide PCOR in mental health, an area with significant unmet need despite available interventions. Conducting PCOR is likely to have an impact on reducing variation and uncertainty in clinical practice and outcomes, methodological and conceptual uncertainty, and disease burden. The IEF also sought to identify knowledge gaps and to generate and prioritize topics for future PCOR, including topics for evidence synthesis (systematic reviews).

The IEF process was limited to adults with serious mental illness who had psychotic or mood disorders because these disorders account for a majority of serious mental illnesses and because the forum’s time and resources were limited. [A figure illustrating the overlap of serious mental illness with mood and psychotic disorders is available online as a data supplement to this article.] Alcohol- and substance-related, developmental, anxiety, and personality disorders were excluded unless they co-occurred with a psychotic or mood disorder.

Several working definitions of serious mental illness, severe mental illness, or severe and persistent mental illness have been used (10–18). We used the following criteria: adults ages 18 or older who had currently or at any time during the past year a diagnosable mental, behavioral, or emotional disorder of sufficient duration to meet DSM-IV diagnostic criteria (or ICD-9 equivalent) and functional impairment that substantially interfered with or limited one or more major life activities (9,18).

We sought a broad array of stakeholder opinions to minimize bias and ensure that diverse perspectives were reflected. Stakeholders represented a cross-section of patients, family members, consumers, advocates, practicing clinicians, members of professional societies, policy makers, public and private payers, representatives of federal agencies, researchers, and methodologists. Once confirmed, participants were offered an orientation on AHRQ and the EHC program by using a Web-hosted presentation.

Topic generation

To generate topics, three meetings were held; the first and third meetings were held by conference call with Web-hosted presentations, and the second was an all-day, in-person meeting at AHRQ. A professional facilitator was used during all meetings.

At the first meeting, participants provided feedback in the areas related to research on serious mental illness that they identified as having greatest interest. We invited participants to continue the discussion and submit research ideas by e-mail, phone, or an online forum hosted by AHRQ. The suggestions by participants were grouped into three main themes: patient-centered care—improving outcomes that matter to patients; conceptual frameworks for research; and reducing disparities for subpopulations.

The objectives of the second, in-person meeting were to discuss broad issues in serious mental illness, identify gaps, and generate and prioritize research topics that can address these issues and gaps. Participants were preassigned to one of three work groups corresponding to the three main themes identified at the first meeting. Each work group reviewed a list of previously submitted topics in its theme and generated additional topic ideas through facilitated group discussion. The work groups presented brief overviews of their discussions to the full stakeholder group. All topic ideas were recorded and displayed in the meeting room. Following a facilitated discussion by the full group, participants used a rolling feedback process to offer amendments.

Topic prioritization

After final review and discussion, a nominal group process was used to identify initial priorities. Stakeholders used stickers to indicate the topics that should be given highest priority for research. Participants were allotted 20 stickers and allowed to place up to five stickers on any one topic idea.

Identification of common themes

Following the in-person meeting, we sorted and organized the topics generated by the work groups by theme. Some topics were generated by more than one work group. After the topics were qualitatively assessed by teams of investigators, we organized them into common topic themes.

The objectives of the final meeting were to review the prioritized topic themes, gather feedback on the process used, and get a preliminary sense of which topics were appropriate for evidence synthesis, evidence generation, and other research activities and products.

The first Web-based meeting and related discussion, attended by 38 stakeholders, yielded 82 topics and ideas. These topics and ideas formed the basis for the three small-group discussions during the in-person meeting. The 33 stakeholders participating in the second meeting generated about 80 additional topics and ideas.

During initial prioritization, stakeholders assigned at least one sticker to nearly half of the topics and 12 or more stickers to 15 topics. After eliminating duplicate topics and ideas, the result was roughly 140 topics subsumed within 21 main themes. Table 1 lists the 21 themes prioritized by total number of stickers assigned during the second meeting.

During the final meeting, the stakeholders identified 11 of the 21 prioritized themes as potentially appropriate for evidence synthesis (Table 1). [Additional description of the main themes and related topics and ideas is available online as a data supplement to this article.]

This article describes priorities to guide PCOR for people with serious mental illness. Although this project aimed to contribute to the AHRQ’s agenda for PCOR activities related to mental health, it was also intended to reach a broad audience, including others who are conducting, funding, or utilizing PCOR. With the recent establishment of the PCOR Institute, a research organization dedicated to the support and promotion of comparative clinical effectiveness research (19), there has been a growing focus on the establishment of future PCOR priorities.

One of the three main themes of the forum was patient-centered care—improving outcomes that matter to patients—and one of the final 21 priorities (priority 7) focused on strategies to personalize or individualize treatment. This priority included comparing strategies to individualize psychotropic medication treatment and identifying variations in response and predictors of response or tolerance that could inform the treatment of specific groups, for example, by age, race or ethnicity, culture, common comorbidities, biomarkers, or genomics. It also included targeting specific interventions to individuals’ goals and wishes or integrating decision analysis into comparative effectiveness trials.

Approach to clinical research

On the basis of multiple prior initiatives (20–24), we anticipated that stakeholders would prioritize head-to-head comparisons of available evidence-based interventions, faster dissemination of these findings into clinical practice, and studies, particularly with patient-centered outcomes, to fill literature gaps. Instead, three of the top four priorities focused on how research was conducted, namely developing consensus measurement and outcomes definitions; improving infrastructure for research, including longitudinal studies, new data sets, and new investigators; and developing PCOR methodology.

Related to measurement and outcomes (priority 1), stakeholders emphasized the need to develop assessment tools and metrics for use in a chronic care model to replace the current inadequate metrics, which are based on an acute-care model. This process includes identifying priority outcomes for consumers, defining more appropriate outcomes criteria, and improving the standardization and consistency of documentation.

Stakeholders identified steps to further develop PCOR methodology (priority 4), including using methods that adjust for confounding during follow-up; making greater use of performance-based measures for improved validity and reliability; exploring and expanding new and different study designs and data analysis strategies, for example, illness trajectories, propensity scores, and time series; using longer-term studies; using research design strategies to optimize long-term follow-up while avoiding attrition bias and confounding; boosting consumer and patient involvement; developing a taxonomy to define and describe psychosocial and psychotherapeutic interventions; and integrating biological and psychosocial research.

These findings highlighted a feeling expressed by many stakeholders—a need to fundamentally change the way research on serious mental illness is conducted by improving research methodology and required infrastructure to conduct meaningful and informative real-world studies of treatment over the lifetime of a serious mental illness. Many individuals with serious mental illness are initially diagnosed in their twenties, or younger, and live most of their lives with considerable disabilities. Consequently, the current treatment model may need to shift to be more consistent with a chronic illness model of care. This shift has occurred with certain medical illnesses, with some benefits reported (25).

Stakeholders emphasized the value of the therapeutic relationship in the treatment of chronic mental illness and the need for further PCOR to clearly consider or further evaluate this relationship (priority 6). A good therapeutic relationship is critical for many aspects of successful treatment and is consistent with a chronic illness model of care. Stakeholders emphasized that the therapeutic alliance explains a large degree of variance in psychotherapy treatment outcomes and should be assessed in evaluating these outcomes. This could include comparing the effectiveness of psychotherapy versus usual care on long-term functional outcomes, such as a continuous therapeutic relationship with a coach, therapist, or other clinician. Stakeholders also supported research assessing the features of health professionals that support the development of a continuous healing relationship and factors associated with better patient outcomes.

Evidence-based interventions, knowledge gaps

Service delivery, treatment settings, and the structure of delivery of care were identified as a high priority (priority 3), which seems to follow a trend. Close to half of all mental health topics in Initial National Priorities for Comparative Effectiveness Research (CER) (26), a report by the Institute of Medicine, were related to systems and delivery of care. Further, the Federal Coordinating Council for CER (27) concluded that “the emphasis on pharmacologic treatments has meant fewer resources for other interventions, including behavioral, procedures, prevention, and delivery system interventions that can have major impacts on health outcomes.”

Stakeholders described the need to use treatment settings and service delivery systems in experiments to compare the effectiveness of different systems for structuring the delivery of care, such as psychiatric medical homes versus usual care; variations across settings, systems, and states that affect disparities for specific groups, for example, rural residents or members of racial and ethnic minority groups; state-to-state or smaller-area comparisons; public or various models of private insurance; group versus individual practice models; separate versus integrated care; delivery tools, such as prior authorization, and their impacts; and methods to support care of specific racial and ethnic groups.

Comparisons of treatment approaches to avoid early morbidity and mortality were highlighted (priority 8). A point of emphasis at the in-person meeting was that something must be done to improve the early mortality of people with serious mental illness (28,29). For example, this effort could include research on integrating mental health care and primary care (30), improving consumer self-care, or combining integration of mental health and primary care and self-care. It could also include comparing various types of integrated care, such as colocating mental health care providers in primary care versus colocating primary care providers in mental health treatment settings; various approaches to improving communication between primary care and mental health care providers; use of medications alone versus medications plus psychosocial treatments; various versions of assertive community treatment; use of older, less expensive psychotropic medications versus newer medications; combinations of psychotropic medications; bundled programs of antipsychotic medications and various psychosocial interventions; and interventions to educate patients and their families about potential harms of medications.

Comparisons of strategies to increase adherence to evidence-based guidelines and treatment regimens (priority 11), including the role of electronic medical records and decision support, were also identified. A related priority (priority 15), perhaps, is work to compare the effectiveness of strategies to disseminate and implement research findings, which has received little attention (31,32). This includes comparing strategies to advance the use of evidence-based practices and new discoveries in community practice.

Priority populations

Stakeholders identified two populations as priority targets for PCOR—people with serious mental illness and comorbid medical illness or substance use disorders and people with serious mental illness in the criminal justice system (priorities 12 and 13). A need for research focused on specific ethnic and racial communities and a need to identify and explain the reasons for disparities related to access, systems issues, institutional racism, and lack of services among people with limited English skills (priority 5) highlight additional targets for PCOR.

Limitations

Resource constraints limited the number and scope of participants. Similarly, although the most productive part of the process was the in-person meeting, resources limited face-to-face interaction to a single meeting.

Many participants were not initially familiar with the EHC program and PCOR and required the context provided by the optional orientation sessions about the EHC program and PCOR and about how PCOR might benefit them. This critical step allowed participants, especially those who were not clinicians or researchers, to participate fully. At the end of the process, most participants indicated a basic understanding of PCOR and the EHC program and a willingness to continue participating in similar processes. Consequently, investigators and individual stakeholders are working together to advance priority topics for consideration in the EHC program.

Stakeholders with a variety of decisional needs identified their priorities and preferences, clearly indicating a need to change the way research on serious mental illness is conducted. Most current PCOR involving patients with serious mental illness has been limited to relatively short-term outcomes and provides little guidance for long-term treatment over a person’s lifetime. There is a critical need for well-designed, long-term studies and creative alternative strategies that better define the course and effective treatment of chronic mental illness. To meet this challenge, new research methodologies and infrastructure must be developed, including the identification and assessment of long-term outcomes important to individuals. Only then can evidence-based decisions be made about how to integrate pharmacologic and psychosocial treatment approaches over the lifetimes of those with serious mental illness.

This project was supported by American Recovery and Reinvestment Act contract 290 2007 10056 I from the Agency for Healthcare Research and Quality. The authors thank Carol Woodell, B.S.P.H., Lynn Rosenberg, M.S.W., Loraine Monroe, and Laura Small, B.A. They acknowledge the participation of the following stakeholders in the Issues Exploration Forum: Jeffrey A. Buck, Ph.D., Chris Collins, M.S.W., Kurt Cousins, M.D., M.B.A., Steven Crystal, Ph.D., Timothy Cuerdon, Ph.D., King Davis, Ph.D., M.S.W., Lisa Dixon, M.D., M.P.H., Kenneth Duckworth, M.D., Laura J. Fochtmann, M.D., Daniel Galper, Ph.D., Tobias Gerhard, Ph.D., Howard H. Goldman, M.D., Ph.D., Carrie Gray, M.B.A., Rachel G. Guerrero, M.S.W., L.C.S.W., Kevin Hennessy, Ph.D., Michael F. Hogan, Ph.D., Gayle Jordan-Randolph, M.D., Jeff King, Ph.D., Thomas Laughren, M.D., Hunter McQuistion, M.D., Don Mordecai, M.D., Eve Mościcki, Sc.D., M.P.H., Katherine Nordal, Ph.D., Ruth Perou, Ph.D., Harold Pincus, M.D., Rhonda Robinson Beale, M.D., Agnes Rupp, Ph.D., Ron Schraiber, M.A., Steven Sharfstein, M.D., David Shern, Ph.D., Jennifer Skeem, Ph.D., Pluscedia Williams, Sharon Yates, Alexander Young, M.D., M.S.H.S., and Judy Zerzan, M.D., M.P.H.

The authors report no competing interests.