Evaluation of startup policies in simulation experiments

Abstract

This paper describes a procedure for evaluating pro posed startup policies that define the initial condi tions and the truncation point in a simulation ex periment. The evaluation procedure involves the tabulation of bias, variance, and mean square error of the sample mean over a fixed range of truncation points and initial conditions. For each policy, the tabulated values are averaged with respect to an empirical truncation-point distribution based on independent model runs. These results are used to construct confidence intervals for the steady-state mean with the same average length for all policies; the corresponding average confidence interval cover age can then be used to compare policies. To illustrate the procedure, several well-known policies are evaluated for two finite-state Markovian queueing systems—α single-server queue with a capac ity of 15 and a machine-repair system with 3 repair- men and 14 machines. The results for both systems indicate that the best initial condition is the most frequently observed value (the mode) of the steady- state distribution of the number-in-system process, and that the judicious selection of an initial condi tion is more effective than truncation in improving the performance of the sample mean as an estimator of the steady-state mean.