Design and analysis of studies evaluating smoking cessation interventions where effects vary between practices or practitioners

Abstract

Background. Patients grouped together in practices may share characteristics that cause them to have similar responses to an intervention. Sampling from such groups means that the power of a trial is less than when subjects are selected from the population at random. Knowledge of likely variation in outcome at the practice level is necessary to calculate the extent to which sample size would need to be inflated to maintain statistical power in the face of ‘cluster effects’. To plan sample size and precision requirements of a clinical trial, we examined reports of primary care smoking cessation trials for information on outcomes at the level of clusters, and found them unhelpful. We therefore constructed hypothetical scenarios to quantify the potential importance of this effect. Scenarios of moderate and large inter cluster variation were compared with a sample where there was no difference in effect size at the level of practices. A study with 80% power to detect a difference of 20% versus 10% at a 5% significance level would need 200 patients in each arm in the absence of cluster effects. With moderate variation in outcome between clusters, over a thousand patients would be needed in the study to maintain this precision. With larger inter-cluster variation, close to 4000 subjects would be required. In the absence of detailed data from previous studies, hypothetical models can give insight into the statistical implications of possible cluster effects on study design and analysis. With even moderate intercluster variation, sample size will have to be inflated considerably to maintain the same statistical precision. Workers in this field will greatly assist those planning future research if they publish details of variation in outcome at the level of clusters.