Estimating Sample Size in Computing Simulation Experiments

Abstract

A method is described for estimating and collecting the sample size needed to estimate the mean of a process (with a specified level of statistical precision) in a simulation experiment. Steps are also discussed for incorporating the determination and collection of the sample size into a computer library routine that can be called by the ongoing simulation program. We present the underlying probability model that enables us to denote the variance of the sample mean as a function of the autoregressive representation of the process under study and describe the estimation and testing of the parameters of the autoregressive representation in a way that can easily be “built into” a computer program. Several reliability criteria are discussed for use in determining sample size. Since these criteria assume that the variance of the sample mean is known, an adjustment is necessary to account for the substitution of an estimate for this variance. It is suggested that Student's distribution be used as the sampling distribution, with “equivalent degrees of freedom” determined by analogy with a sequence of independent observations. A bias adjustment is described that can be applied to the beginning of the collected data to reduce the influence of initial conditions on events in the experiment. Four examples are presented using these techniques, and comparisons are made with known theoretical solutions. One unfortunate shortcoming of the proposed procedure is that its performance is directly linked to the initially chosen sample size. Our results show that as this sample size increases, the procedure gives results which agree more closely with predicted results.