Simple sigmoid-like activation function suitable for digital hardware implementation

Abstract

A simple sigmoid-like second-order piecewise activation function suitble for direct digital hardware implementation is presented. Simulation results on the uses of the second-order function and the bipolar sigmoid function for training multilayer feedforward networks using the backpropagation algorithm show that they have similar generalisation properties while the second-order function has a slight advantage in convergence speed.