Evolution of neural network training set through addition of virtual samples

Abstract

Using an oversized neural network or too small a training sample set results in overfitting. In order to improve generalization capability, either the network should be reduced or additional training samples have to be collected. Obtaining additional training samples, however, can be often very expensive or impossible. Here we propose an evolutionary approach where new virtual samples are added to the training sample set as a population of MLPs evolve over generations. At each generation, these newly added virtual samples are used to retrain the MLPs. This approach is in contrast to previous evolutionary neural network approaches where connection weights, network architectures, learning rules, or their mixtures evolve. A preliminary result obtained from a robot arm kinematics problem is promising. The generalization error was reduced more than 50%. The approach can be applied in various practical situations where additional training samples are expensive or impossible.