A high-definition human face tracking system using the fusion of omni-directional and PTZ cameras mounted on a mobile robot

Abstract

The omni-directional cameras providing 360 degrees field of view (FOV) are widely used in video surveillance and robot vision applications. However, the omni-directional cameras have an obvious drawback; that is, only low-resolution images captured. Therefore, the objects are not able to be correctly identified if they are far from the omni-directional cameras. To overcome this problem, we propose a high-definition human face tracking system using the fusion of omni-directional and pan-tilt-zoom (PTZ) cameras. Our system first detects and tracks human faces in the panoramic images received from an omni-directional camera, and then controls the PTZ camera to fixate at a target face for capturing a high-resolution image. At the beginning, the human face detection procedure obtains moving human faces by means of temporal differencing together with skin and hair color filtering. The detected human faces are subsequently fed to the face tracking procedure which employs a particle filter for iteratively tracking human faces. Once a target human face is selected, the PTZ camera is directed to stare at the target and zoom in it speedily. Then the face tracking procedure turns to use the images received from the PTZ camera for continuously tracking the target face until it is outside the FOV of the camera.