Avoiding false local minima by proper initialization of connections

Abstract

The training of neural net classifiers is often hampered by the occurrence of local minima, which results in the attainment of inferior classification performance. It has been shown that the occurrence of local minima in the criterion function is often related to specific patterns of defects in the classifier. In particular, three main causes for local minima were identified. Such an understanding of the physical correlates of local minima suggests sensible ways of choosing the weights from which the training process is initiated. A method of initialization is introduced and shown to decrease the possibility of local minima occurring on various test problems.