Bioinspired triboelectric-driven multisensory framework with autonomous cross-modal adaptation

Abstract

The human multisensory neural network supports advanced cognitive functions through cross-modal integration, recognition, and imagination by synergistically processing visual, tactile, auditory, olfactory, and gustatory stimuli. This biological mechanism generates comprehensive environmental representations through dynamic sensory interactions rather than isolated processing. In this study, a bioinspired multisensory framework is developed, integrating triboelectric sensors with artificial vision, tactile receptors, auditory interfaces, and simulated olfactory/gustatory modules. The system employs a distributed multisensory framework for biomimetic hierarchical processing of multimodal data perception, storage, and fusion. Through cross-modal learning, the system establishes effective associations among different sensory inputs, achieving 97.12% accuracy in tactile-visual recognition and 94.62% accuracy in auditory-visual-olfactory-gustatory reconfiguration. Beyond empirical learning, the framework also demonstrates non-empirical human-like cognitive functions, such as association, inference, and creative pattern generation. The proposed multisensory cross-modal system establishes a versatile framework with significant technological advantages of energy-efficient cognition, adaptive processing, and cognitive scalability. The bioinspired cross-modal reconfiguration combining with triboelectric sensing provides technical innovation and methodological impact to establishing new paradigm for energy-autonomy robotic perception. • A bioinspired multisensory framework integrating tactile, visual, auditory, olfactory, and gustatory modalities is developed. • The system enables biomimetic hierarchical perception, cross-modal association, and integrated information fusion in an energy-efficient manner. • The system achieves 97.12% accuracy in tactile-visual recognition and 94.62% in auditory-visual-olfactory-gustatory cross-modal reconfiguration. • It supports both empirical learning and non-empirical cognitive functions including inference, association, and generative pattern creation. • This framework provides a scalable and adaptive platform toward energy-autonomous robotic perception with broad applicability in intelligent systems.