Competitive learning and map formation in artificial neural networks using competitive activation mechanisms

Abstract

Inhibitory effects in artificial neural networks have usually been achieved via direct inhibitory connections between competing nodes. This mechanism is limited by the large number of inhibitory connections that are sometimes necessary, and the difficulty of designing competitive interactions with inhibitory connections for some applications. Because of these limitations competitive activation mechanisms have been introduced to provide competitive interactions between nodes using strictly excitatory connections. Competitive activation mechanisms have been successfully applied to problems in Al, cognitive modeling, and computational neuroscience, sometimes producing effects which are difficult or impossible to achieve with noncompetitive activation mechanisms. However, applications using competitive activation mechanisms have been limited by the absence of effective learning methods. This dissertation develops the first unsupervised learning method for artificial neural networks using competitive activation mechanisms. The learning method, a variant of competitive learning, is shown to be effective through both computer simulations and mathematical analysis. Competitive learning can be used for classification tasks involving the separation of input pattern clusters; analysis shows that a typical competitive activation model produces a different classification than a typical noncompetitive activation model using competitive learning. The unsupervised competitive learning rule is extended to include reinforced and supervised versions which are also shown to function effectively. Competitive learning has been used successfully with noncompetitive activation mechanisms in the past for feature map formation in many applications (speech recognition, robotic control, optimization, brain modelling, etc.). Computer simulations show that competitive activation models can also produce computational map formation with different structural characteristics than comparable noncompetitive activation models. Further, competitive activation models can generate more rapid and extensive map reorganization following network damage than noncompetitive activation models. Competitive activation models also support topographic map formation/refinement and map reorganization in response to changes in the structure of the input stimuli. Evaluating topographic map formation necessitated the development of new measurement and plotting techniques which are presented here. This work shows that competitive learning using competitive activation mechanisms is a powerful approach for artificial neural networks.