Nonlinear neural networks near saturation

Abstract

Nonlinear neural networks are studied near saturation, when the number $q$ of stored patterns is proportional to the system size $N$, i.e., $q=\alpha N$. The statistical mechanics is obtained for arbitrary nonlinearity. For a wide class of models, including the original Hopfield model and clipped synapses, it is shown that there exists a critical ${\alpha }_c$ above which the system looses its memory completely. Furthermore, ${\alpha }_c$ never exceeds ${\alpha }_c^{\mathrm{hopfield}}$ and is determined by a universal expression. A moderate dilution of the bonds may improve the memory function.