Perceptron beyond the limit of capacity

Abstract

An input-output map in which the patterns are divided into classes is considered for the perceptron. The statistical mechanical analysis with a finite number of classes turns out to give the same results as the case of only one class of patterns ; the limit of capacity and the relevant order parameters are calculated in a mean field approach. The analysis is then extended to the Derrida Gardner canonical ensemble in which the perceptron can be studied beyond the limit of capacity. We complete the analysis with numerical simulations with the perceptron learning rule. The relevance of those results to the possible emergence of spontaneous categorization is finally discussed