In their commentary in the September 1997 issue, Hogarty and associates (1) present a strong defense of efficacy research, or randomized controlled trials, and imply that such research is unrelated and noncomplementary to effectiveness research, which focuses on the more naturalistic implementation of interventions. They also imply that current interest in effectiveness research runs the risk of causing managed care organizations, and possibly clinicians, to withhold beneficial treatments that have been studied only in efficacy trials. The authors believe that efficacy studies have been unfairly characterized as having limited relevance to clinical practice because of their methodological constraints.
The authors do not appear to see any value in effectiveness research, believing that the impetus behind the National Institute of Mental Health's commitment to it is misplaced. We see efficacy and effectiveness research as crucial parts of the same question: will this intervention benefit the recipient? Both internal validity (efficacy) and external validity (effectiveness) are critical aspects of evaluating interventions. The major difference is that effectiveness research relaxes exclusionary criteria and broadens the scope of outcome measurement. Because the results can be generalized, effectiveness research has the added capability of informing the policy process (2).
Cost-effectiveness studies are the best example of how efficacy and effectiveness studies differ in generalizability and relevance to policy. Interventions with demonstrated benefit in efficacy studies need to be transported into the effectiveness arena. Hogarty and associates suggest that by integrating innovative statistical and theoretical models within efficacy studies, results can be produced that are relevant both to policy and to clinical care. But even with such techniques, contextual and human factors, such as clinic prescribing practices and staff training, cannot be modeled adequately.
We share the authors' view that controlled clinical trials should be the foundation of clinical knowledge and practice. Such research is the initial testing ground for interventions and provides maximal opportunity to demonstrate their benefits. In pharmacotherapy, for example, information about the benefits and safety of one drug compared to another is essential where the differences between treatments may be relatively small.
However, we also recognize important ways in which effectiveness research not only can complement efficacy research but also can provide important information that is not otherwise available. In this regard, we note that effectiveness research does not always mean uncontrolled, retrospective, or correlational research. Well-planned studies can use random assignment procedures within naturalistic settings to produce unbiased estimates of the benefits of interventions (3,4). Additionally, quasiexperimental designs (5) have gained prominence in community-based research because of the balance between scientific rigor and clinical ideology and practice.
Our message is that both efficacy and effectiveness research are essential. Ideally, the evaluation of interventions has two stages. The first is to study interventions in a controlled trial where the subject pool and the intervention delivery are tightly controlled. The second is to transport studies with demonstrated efficacy into naturalistic community settings where mediating and moderating factors can be identified.
Dr. Summerfelt and Dr. Meltzer are affiliated with the division of psychopharmacology in the department of psychiatry at Vanderbilt University School of Medicine in Nashville, Tennessee.