We investigate self-sustaining stable states (attractors) in networks of integrate-and-fire neurons. First, we study the stability of spontaneous activity in an unstructured network. It is shown that the stochastic background activity, of 1-5 spikes/s, is unstable if all neurons are excitatory. On the other hand, spontaneous activity becomes self-stabilizing in presence of local inhibition, given reasonable values of the parameters of the network. Second, in a network sustaining physiological spontaneous rates, we study the effect of learning in a local module, expressed in synaptic modifications in specific populations of synapses. We find that if the average synaptic potentiation (LTP) is too low, no stimulus specific activity manifests itself in the delay period. Instead, following the presentation and removal of any stimulus there is, in the local module, a delay activity in which all neurons selective (responding visually) to any of the stimuli presented for learning have rates which gradually increase with the amplitude of synaptic potentiation. When the average LTP increases beyond a critical value, specific local attractors (stable states) appear abruptly against the background of the global uniform spontaneous attractor. In this case the local module has two available types of collective delay activity: if the stimulus is unfamiliar, the activity is spontaneous; if it is similar to a learned stimulus, delay activity is selective. These new attractors reflect the synaptic structure developed during learning. In each of them a small population of neurons have elevated rates, which depend on the strength of LTP. The remaining neurons of the module have their activity at spontaneous rates. The predictions made in this paper could be checked by single unit recordings in delayed response experiments.