An Efficient Deep Reinforcement Learning Approach for Autonomous Ultrasound Scanning Robot Based on Multimodal Sensing and Distance ProbSparse Self-Attention

摘要

Medical ultrasound is an essential noninvasive diagnostic tool across various disciplines, yet its dependence on skilled practitioners presents significant challenges to achieving efficient autonomous imaging. This article presents an autonomous robotic ultrasound scanning method enhanced with multimodal sensing and distance probSparse self-attention (DPSA). By integrating ultrasound images, dual-view cameras, tactile feedback, and robotic action sequences, the system achieves comprehensive environmental perception. The 6-D pose decision-making task for the robot is formulated as a deep reinforcement learning (DRL) problem, and a hybrid reward function is designed to conform to professional sonographers. The proposed DPSA mechanism is designed to capture critical information from the current multimodal sensory data by allocating greater attention to important time steps. In addition, this work employs the discrete soft actor–critic (DSAC) algorithm, prioritized experience replay (PER), and a pretrained ResNet-18 model, significantly reducing training time. Evaluation in real-world environments using soft, movable, and unmarked kidney phantoms demonstrates that our approach outperforms existing baseline models in terms of scanning success rate, accuracy, and training efficiency, while maintaining robust stability under interference conditions.