Fusion of vision and IMU to track the racket trajectory in real time

Abstract

This paper presents a novel approach on identification of human habits and behaviors when playing with the table tennis robot. The main idea of this approach lies in tracking the trajectory of the racket on player's hand by fusion of vision and IMU(Inertial measurement unit) sensors' data. In order to uniform the data from the vision and IMU sensors, the non-linear algorithm is applied to accomplish the calibration. The visual viewable range could be broadened by the method to switch the vision systems between the monocular and binocular vision system. The fusion approach of the vision and IMU sensors is based on the EKF (Extended Kalman Filter) for obtaining accurate and robust racket pose. Taking advantage of the racket pose, the player's habits and behaviors can be represented when he playing with the table tennis robot. Experiments and results showed that the proposed method is effective and real-time.