Identification and impact of outcome selection bias in meta‐analysis

Abstract

The systematic review community has become increasingly aware of the importance of addressing the issues of heterogeneity and publication bias in meta‐analyses. A potentially bigger threat to the validity of a meta‐analysis appears relatively unnoticed. The within‐study selective reporting of outcomes, defined as the selection of a subset of the original variables recorded for inclusion in publication of trials, can theoretically have a substantial impact on the results. A cohort of meta‐analyses on the Cochrane Library was reviewed to examine how often this form of within‐study publication bias was suspected and explained some of the evident funnel plot asymmetry. In cases where the level of suspicion was high, sensitivity analysis was undertaken to assess the robustness of the conclusion to this bias. Although within‐study selection was evident or suspected in several trials, the impact on the conclusions of the meta‐analyses was minimal. This paper deals with the identification of, sensitivity analysis for, and impact of within‐study selective reporting in meta‐analysis. Copyright © 2004 John Wiley & Sons, Ltd.