Systematic Review: Why sources of heterogeneity in meta-analysis should be investigated

Abstract

Although meta-analysis is now well established as a method of reviewing evidence, an uncritical use of the technique can be very misleading. One common problem is the failure to investigate appropriately the sources of heterogeneity, in particular the clinical differences between the studies included. This paper distinguishes between the concepts of clinical and statistical heterogeneity and exemplifies the importance of investigating heterogeneity by using published meta-analyses of epidemiological studies of serum cholesterol concentration and clinical trials of its reduction. Although not without some dangers of speculative conclusions, prompted by overzealous inspection of the data to hand, a sensible investigation of sources of heterogeneity should increase both the scientific and the clinical relevance of the results of meta-analyses. * This paper was presented at a meeting on Systematic Reviews organised jointly by the BMJ and the UK Cochrane Centre and held in London in July 1993; it is the last in this series The purpose of a meta-analysis of a set of clinical trials is rather different from the specific aims of an individual trial. For example, a particular clinical trial investigating the effect of serum cholesterol reduction on the risk of ischaemic heart disease tests a particular treatment regimen, given for a specified duration to participants fulfilling certain selection criteria, using a particular definition of outcome measures. The purpose of a meta-analysis of cholesterol lowering trials is broader - that is, to estimate the extent to which serum cholesterol reduction, achieved by a variety of means, generally influences the risk of ischaemic heart disease. A meta- analysis also attempts to gain greater objectivity, generalisability, and precision by including all the available evidence from randomised trials that pertain to the issue.1 Because of the broader aims of a meta- analysis, the trials included usually encompass a substantial variety of specific …