The Need for Implementing Evidence-Based Practices

The biggest lesson to be learned, however, from our 40 year old experiment with deinstitutionalization, is that we simply have not learned our lessons…. [W]e must first begin by implementing what we know and do it now.

Although published ten years ago, the tag line of the book chapter reprinted here identifies a current, major concern of consumers, providers, and policy makers—the failure of evidence-based practices to be offered in routine care. In this book chapter John Talbott struggles to discern and distill the major clinical messages that are embedded in the available scientific evidence. His approach is sensible and grounded.

If we examine Dr. Talbott's visionary message and approach today, what do we see? We scrutinize his work with his blessing, because an implicit part of his exhortation is to keep reexamining the evidence. We first see an unfortunate confirmation that, even today, less has been learned than has been applied in practice. Studies by the Schizophrenia Patient Outcomes Research Team (PORT) and other systematic efforts to examine the quality of care for persons with severe mental illness have suggested a huge gap between what we know and what we do (1,2,3). This inconsistency is particularly true with psychosocial treatments, which by and large have no industrial sponsor to do their marketing. Notably, the deficits are apparent not only in the mental health field but also across many medical specialties (4,5,6). And it is fitting that a key message delivered by the 1999 Surgeon General's report on mental health—under the scientific leadership of Dr. Talbott's successor, Howard H. Goldman—is that treatment works, but treatments that are most likely to work are often not delivered (7).

At the same time, the science of evidence-based practice has been both challenged and advanced. What are some of the challenges? Several national leaders in the movement to promote the dissemination of evidence-based practices convened to attempt to articulate the substantial concerns of six stakeholder groups: consumers, family members, practitioners, administrators, policy makers, and researchers. Consumers and families, whose self-help interventions have not been subjected to rigorous research, worry that the absence of evidence for these interventions will be mistaken as evidence for the absence of effectiveness. Such interventions were created outside the biomedical infrastructure and its brand of science and hypothesis testing. Consumers and families also worry that an effective program could be closed for lack of evidence of its effectiveness. Consumers and providers worry that poorly conceived treatment algorithms that are based on evidence-based practice could lead to the absence of individualized treatment plans and lead to "cookbook" medicine. Consumers also worry that the outcomes that have driven the studies that underpin evidence-based practice—symptoms and relapse—do not sufficiently weigh the outcomes that consumers value, such as quality of life. Practitioners worry that research findings have limited relevance to practice—that is, the efficacy-effectiveness gap. For example, the findings of randomized clinical trials that screen out patients with substance abuse may not be relevant to the patients they routinely treat, because up to 50 percent of patients abuse substances. Providers also worry that they have limited time for new training. Finally, administrators and policy makers often worry about resources. Overall, the group expressed concerns about the limits of science, funding shifts that would eliminate or reduce valuable practices, increased costs, feasibility, and unintended shifts in power and control (8).

The PORT update by Lehman and colleagues cited ambiguities and limitations in the scientific evidence, the absence of studies in critical areas of practice, valid disagreements about interpretation of the evidence, and practitioners' skepticism about the need to change practices (5,9,10). Dr. Talbott would probably agree with all these concerns, and I wonder how he will accommodate these concerns and derive new lessons in a future article.

What else do we see if we examine Dr. Talbott's visionary message and approach today? My own participation in the development of practice guidelines for the PORT and the American Psychiatric Association revealed that the science of combining studies to understand the meaning of the primary data and to create valid statements about what we know has advanced markedly. Statistical techniques have tackled the challenges of performing meta-analyses that are more refined. Complex techniques have been created to deal with the nonindependence of subjects and outcomes when randomized studies are not possible and patients are clustered within treatment teams, clinics, or systems of care. Propensity scoring, instrumental variables, and innovative cost-effectiveness techniques are being applied to large databases to get population-relevant answers as to how well a program works and how much a program would cost. The Department of Veterans Affairs has been a vigorous participant in such studies with its large electronic database.

The use of mixed designs that apply rigorous qualitative and quantitative approaches has been extended. In these studies consumers, families, and providers are all heard to ensure the inclusion of their voices in the development and testing of interventions. Finally, the science of learning about the dissemination of evidence-based practices has become a major funding priority of the National Institutes of Health. We must learn how to implement what we know and do it now.

In short, the science of evidence-based practice has been challenged and is trying to address the challenges.

The most striking lesson that I learned from writing this commentary pertains to a most remarkable and enduring aspect of Dr. Talbott's talents. He tells a story by using words and language that show concern for individuals with mental illness. This empathy is one of his singular gifts as a writer and communicator. Perhaps the most important lesson from Dr. Talbott is that research in a vacuum gets us nowhere. We need to take the time to tell the stories we have by using language that is respectful and humble and that will encourage the readers and listeners—consumers, families, providers, administrators, and the public—to hear our stories.

Dr. Dixon is affiliated with the department of psychiatry at the University of Maryland School of Medicine and with the Department of Veterans Affairs Capitol Health Care Network Mental Illness Research, Education, and Clinical Care Center, 22 South Greene Street, Baltimore, Maryland 21201 (e-mail, ldixon@ psych.umaryland.edu). This commentary is part of a tribute to John A. Talbott, M.D., Editor Emeritus, who served as Editor of Hospital and Community Psychiatry and Psychiatric Services from 1981 to 2004.