Emergent activation functions from a stochastic bit-stream neuron

Abstract

The authors present the results of experimental work that demonstrates the generation of linear and sigmoid activation functions in a digital stochastic bit-stream neuron. These activation functions are generated by a stochastic process and require no additional hardware, enabling the design of an individual neuron to be extremely compact.