Weighing Scientific Evidence: Let's Keep the Blindfold Off
Lady Justice—or Justitia, the Roman goddess of justice—holds a double-edged sword in her right hand and a scale in her left, and sometimes she is blindfolded. The blindfold symbolizes the ideal of weighing evidence with impartiality. However, in the case of scientific evidence, scrutiny and deliberation are required to find the balance point in measuring the strength of evidence and its applicability to a clinical decision.
The findings of a study reported in this issue, "Evidence-Based Use of Second-Generation Antipsychotics in a State Medicaid Pediatric Population, 2001–2005," require careful scrutiny because they could be misinterpreted as indicating overuse or inappropriate use of these agents to treat children. Conclusions about the appropriateness of medication treatment that are based on claims data are limited because the validity of the diagnosis cannot be established from such data and the clinical rationale for prescribing a particular drug is missing. The authors of the report acknowledge that they were unable to link the diagnoses recorded in the Medicaid claims to the index prescription of the antipsychotic.
Nevertheless, on the basis of the design and sample size of published studies and diagnoses from claims data, these investigators rated the strength of evidence for each initial prescription of a second-generation antipsychotic for 11,700 children between 2001 and 2005. During this period none of these agents were approved for pediatric use by the Food and Drug Administration (FDA), and the majority of published reports that included children were open-label studies involving risperidone. One medication, aripiprazole, was approved for adult use in November 2002. Was the study premature to apply ratings of evidence strength? Or by applying the ratings at a time when the evidence base was not strong, did the study simply reflect the fact that prescribers were exercising their clinical judgment in making medication decisions for high-risk children?
Either way, the study illuminates the larger debate about what constitutes substantial evidence of the safety and effectiveness of psychotropic medications prescribed to children. In 1962 the proposed FDA criterion for "substantial evidence" of effectiveness was "at least two adequate and well-controlled studies, each convincing on its own, to establish effectiveness." With the FDA Modernization Act of 1997, the criterion was relaxed to "one adequate and well-controlled clinical investigation and confirmatory evidence," and rules were revised to allow for extrapolation from adult efficacy data to pediatric patients. National efforts have also been made to standardize approaches to rating scientific evidence. These approaches acknowledge a subjective component in the assessment of research methods to "weigh" scientific evidence. Given the need for more long-term studies to establish the safety and efficacy of antipsychotics for children in real-world settings, let's keep Lady Justice's blindfold off during our deliberations.
