Vision‐based robotic person following under light variations and difficult walking maneuvers

Abstract

Abstract The paper addresses the problem of robotic person following indoors and outdoors with significant light variation such as moving from sunny areas into shade and difficult walking maneuvers such as jumping to the side or making sharp turns. These are achieved through certain image processing techniques and by real‐time control of camera parameters affecting the light exposure, i.e., camera gain and shutter speed. Several strategies are also formulated and integrated to quickly recognize and segment the person from the environment during difficult walking maneuvers. The centroid of a person's image, which is a critical quantity for person following, is determined using a new method based on aggregation of the detected regions and their color characteristics. In addition to the camera exposure control, the camera pan and tilt and the robot speed and steering are controlled using the characteristics of the detected person's image such as mass (area) and centroid. Although the emphasis of the paper is on person following, the same techniques can be adapted for following moving objects such as another robot or a car. The system has been implemented on a Segway robotic platform that has been equipped with a camera, an onboard computer, loudspeakers, game pad, etc. Extensive trials have been carried out both indoors and outdoors, and a video has been prepared that shows person following under a variety of conditions. © 2010 Wiley Periodicals, Inc.