MULTISENSOR IMAGE RECOGNITION BY NEURAL NETWORKS WITH UNDERSTANDABLE BEHAVIOR

Abstract

Recently, a kind of structured neural networks (SNNs) explicitly devoted to multisensor image recognition and aimed at allowing the interpretation of the "network behavior" was presented in Ref. 1. Experiments reported in Ref. 1 pointed out that SNNs provide a trade-off between recognition accuracy and interpretation of the network behavior. In this paper, the combination of multiple SNNs, each of which has been trained on the same data set, is proposed as a means to improve recognition results, while keeping the possibility of interpreting the network behavior. A simple method for interpreting the "collective behaviors" of such SNN ensembles is described. Such an interpretation method can be used to understand the different kinds of "solutions" learned by the SNNs belonging to an ensemble. In addition, as compared with the interpretation method presented in Ref. 1, it is shown that the knowledge embodied in an SNN can be translated into a set of understandable "recognition rules". Experimental results on the recognition of multisensor remote-sensing images (optical and radar images) are reported in terms of both recognition accuracy and network-behavior interpretation. An additional experiment on a multisource remote-sensing data set is described to show that SNNs can also be effectively used for multisource recognition tasks.