Small sample size effects in statistical pattern recognition: recommendations for practitioners and open problems

Abstract

The authors discuss the effects of sample size on the feature selection and error estimation for several types of classifiers. In addition to surveying prior work in this area, they give practical advice to today's designers and users of statistical pattern recognition systems. It is pointed out that one needs a large number of training samples if a complex classification rule with many features is being utilized. In many pattern recognition problems, the number of potential features is very large and not much is known about the characteristics of the pattern classes under consideration: thus, it is difficult to determine a priori the complexity of the classification rule needed. Therefore, even when the designer believes that a large number of training samples has been selected, they may not be enough for designing and evaluating the classification problem at hand. It is further noted that a small sample size can cause many problems in the design of a pattern recognition system.