Neural networks with dynamical thresholds

Abstract

We incorporate local threshold functions into the dynamics of the Hopfield model. These functions depend on the history of the individual spin (= neuron). They reach a maximal height if the spin remains constant. The resulting one-pattern model has ferromagnetic, paramagnetic, and periodic phases. This model is solved by a master equation and approximated by simplified systems of equations that are substantiated by numerical simulations. When several patterns are included as memories in the model, it exhibits transitions—as well as oscillations—between them. The latter can be excluded by known methods. By introducing threshold functions which affect only spins which remain positive, thus mimicking fatigue of the individual neurons, one can obtain open-ended movement in pattern space. Using couplings which form pointers from one pattern to another, our system leads to self-driven temporal sequences of patterns, resembling the process of associative thinking.