Ill-posed inverse problems arise in many branches of science and engineering. In the typical situation one is interested in recovering a whole function given a finite number of noisy measurements on functionals. Performance characteristics of an inversion algorithm are studied via the mean square error which is decomposed into bias and variability. Variability calculations are often straightforward, but useful bias measures are more difficult to obtain. An appropriate definition of what geophysicists call the Backus-Gilbert averaging kernel leads to a natural way of measuring bias characteristics. Moreover, the ideas give rise to some important experimental design criteria. It can be shown that the optimal inversion algorithms are methods of regularization procedures, but to completely specify these algorithms the signal to noise ratio must be supplied. Statistical approaches to the empirical determination of the signal to noise ratio are discussed; cross-validation and unbiased risk methods are reviewed; and some extensions, which seem particularly appropriate in the inverse problem context, are indicated. Linear and nonlinear examples from medicine, meteorology, and geophysics are used for illustration.