Input variable selection for multi-layer neural networks

A method is proposed for selecting relevant input variables to multi-layer neural networks. A minimal set of inputs is selected which is necessary to obtain a network with a good generalization ability and some insight into the input-output relationship. The inputs of network are selected automatically by a combination of constructive and destructive algorithms. The constructive algorithm starts with a minimal input set and adds new inputs if necessary, while the destructive algorithm deletes unnecessary inputs. The main issue addressed here is the measure of input significance used in the constructive algorithm. Some measures are proposed based on mutual infomation and linear correlation paying much attention to the structural constraint imposed on the networks. The experimental results show that the measures are valid and that the derived network with the selected inputs has a good generalization ability.