Stereo vision guided telerobotics system for autonomous pick and place operations

摘要

In this work, we manipulate an industrial robot from a remote place through a Local Area Network using a force feedback enabled Phantom haptic device as a master, and a stereo vision system for viewing as well as for depth estimation. We propose and implement a stereo vision guided telerobotics system for autonomous pick and place operations. The robot gets the location of pick up from a pre-specified marker, whose position is determined by the stereo vision system. We make use of a pair of IP cameras as the stereo camera, a 3D enabled monitor for viewing stereo contents and an NVIDIA 3D Vision Pro system for managing our stereo vision system. We perform real-time stereo rectification of a stereo image stream for comfortable stereoscopic viewing. We design object markers to support stereo matching up to sub-pixel level. The coordinate systems of the robot and the stereoscopic system are transformed to a pre-defined world coordinate system using a planar chessboard calibration pattern. We evaluate accuracy and repeatability of our stereo vision system on the chessboard plane as well as within the working volume of the robot. Our stereoscopic system is highly accurate with errors less than 2mm along X and Y directions, and below 5mm along Z direction which is direction of viewing.