Hand Palm Tracking in Monocular Images by Fuzzy Rule-Based Fusion of Explainable Fuzzy Features With Robot Imitation Application

Abstract

This article proposes a new method for the tracking of three-dimensional (3-D) hand palms from the whole human standing body using fuzzy rule-based fusion of explainable fuzzy features from a monocular video. The characteristics of this method include visually and linguistically explainable fuzzy features and rules and computational efficiency. This article first tracks the 2-D palms using the following four fuzzy features: optical flows; the degree of a pixel in the foreground; skin color information; and the search area around a hand palm candidate from a segmented body. Afterward, a fuzzy system (FS) is proposed to fuse the four fuzzy features to estimate the 2D- palm positions. Localization of the elbows is based on the estimated palm locations, human body skeletons, and body contour. The 2-D palms and elbows are tracked using a modified particle filter. To estimate the depth of each palm, the locations of the palm and elbow are fed as inputs to a neural FS. The 3-D palm tracking result is applied to a robot upper-body imitation system. Experiments with comparisons of different hand palm tracking methods are performed to verify the real-time computational ability and accuracy of the proposed method.