Ruby: A Robotic Platform for Real-time Social Interaction

摘要

The majority of our waking hours are spent engaging in social interactions. Some of these interactions occur at the level of long-term strategic planning while others\ntake place at faster time scales, such as in conversations or card games. The abilityto perceive subtle gestural, postural, and facial cues, in addition to verbal language,\nin real-time is a critical component. An understanding of the underlying perceptual\nprimitives that support this kind of real-time social cognition is key to understanding\nsocial development.\nRobots present an ideal opportunity to study the development of social interaction\nin infants [Fasel,Deak,Triesch,Movellan 2002]. It is possible to create robots that\nexhibit precisely controlled contingency structures. By observing how infants interact\nwith these robots we gain an opportunity to understand how infants identify the\noperating characteristics of the social agents with whom they interact.\nWe have recently developed a social interaction robot, "Ruby", designed to communicate\nwith children. Ruby is endowed with the following real-time perceptual\nprimitives to facilitate social interaction: face tracking, motor control and speech\ndetection. It communicates via head and eye movements and we have recently run\npilot studies indicating that Ruby is fun and non-threatening to children.\nRuby's face tracking system consist of 3 cues taken from 3 inputs. The first 2 inputs\nare high-resolution pan-tilt-zoom color cameras which are the "eyes". The third\ninput is an omni-directional camera acting as Ruby's peripheral vision. Each eye\nuses the MPLab's contrast-feature based frontal face finder [Fasel et al CVIU2004]\nand adaptive color-based tracker [Ishiguro et al 2003] [Hershey et al CVPR2004].\nRuby combines both of these to find both frontal and rotated faces at more than\n30 frames per second. Ruby's motor control system currently has 3 components;\nneck control, eye control, and control of external objects for experiments. Ruby\nalso features speech detection [Pellom 2004] and response with variable delay parameters.\nWe are now adding eye and eye-blink detection[Fasel et al CVIU2004],\nexpression recognition[Littlewort-Ford, Bartlett et al 2004], recognition of common\ncommunicative words in English, arm movements, finger pointing, and touch sensors.\nWe hope to use Ruby to collect and analyze data on social interaction and contingency\nand on the development of social interaction in infants.