Cochrane Systematic Reviews for the Mental Health Field: Is the Gold Standard Tarnished?

Objective

This study assessed the appropriateness of the Cochrane review system for providing guidance to the mental health field regarding evidence-based practice.

Methods

Excluding entries for substance misuse and dementia, all entries from the three mental health review groups in the 2005–March 2012 Cochrane database were included (N=552). Entries were coded for review group, type (protocol versus review), design (randomized controlled trials or not), datedness, percentage of studies excluded, eventual withdrawal, and nature of the review’s conclusions.

Results

A majority (N=322, 58%) of entries focused on psychotic, mood, and anxiety disorders. The average Cochrane review was last considered up to date in 2006. The reviews excluded over twice as many studies as they included, and inclusion of “gray literature” was infrequent. A total of 159 (44%) reviews reported that there was insufficient evidence to form a conclusion. A finding of insufficient evidence to form a conclusion was not related to the entry’s design but was related to the ratio of excluded to total studies and to the use of the Grading of Recommendations Assessment, Development, and Evaluation system of evaluating the quality of evidence.

Conclusions

The Cochrane Collaboration, an established review system respected globally for its methodological rigor, is poised to enlarge its role in guiding the evidence-based mental health field. However, addressing issues regarding datedness, diagnostic breadth, limited types of evidence, and amount of guidance provided in the conclusions would help maximize its contribution to the mental health field.