Formation of disconnected decision regions with a single hidden layer

Abstract

The authors show that two-layer nets with a single hidden layer are capable of forming disconnected decision regions. In addition to giving examples of the phenomenon, they explain why and how disconnected decision regions are formed. Assuming neural nodes with threshold elements, an expression is derived for the number of cells formed in the input space by the hyperplanes associated with the first (hidden) layer. This expression can be useful in deciding how many nodes to have in the first layer. Each hyperplane in the second layer then determines a decision region in the input space, which consists of a number of cells that are typically connected to each other. However, through the hypothesization of the existence of additional virtual cells formed by the first layer, the authors show how the decision regions formed by the second layer can indeed be disconnected. It is shown that, far from being isolated examples, the number of such disconnected regions can be very large.