Estimated Sampling Distributions: The Bootstrap and Competitors

Abstract

Let $X_1, X_2, \cdots, X_n$ be i.i.d random variables with d.f. $F$. Suppose the $\{\hat{T}_n = \hat{T}_n(X_1, X_2, \cdots, X_n); n \geq 1\}$ are real-valued statistics and the $\{T_n(F); n \geq 1\}$ are centering functionals such that the asymptotic distribution of $n^{1/2}\{\hat{T}_n - T_n(F)\}$ is normal with mean zero. Let $H_n(x, F)$ be the exact d.f. of $n^{1/2}\{\hat{T}_n - T_n(F)\}$. The problem is to estimate $H_n(x, F)$ or functionals of $H_n(x, F)$. Under regularity assumptions, it is shown that the bootstrap estimate $H_n(x, \hat{F}_n)$, where $\hat{F}_n$ is the sample d.f., is asymptotically minimax; the loss function is any bounded monotone increasing function of a certain norm on the scaled difference $n^{1/2}\{H_n(x, \hat{F}_n) - H_n(x, F)\}$. The estimated first-order Edgeworth expansion of $H_n(x, F)$ is also asymptotically minimax and is equivalent to $H_n(x, \hat{F}_n)$ up to terms of order $n^{- 1/2}$. On the other hand, the straightforward normal approximation with estimated variance is usually not asymptotically minimax, because of bias. The results for estimating functionals of $H_n(x, F)$ are similar, with one notable difference: the analysis for functionals with skew-symmetric influence curve, such as the mean of $H_n(x, F)$, involves second-order Edgeworth expansions and rate of convergence $n^{-1}$.