Deep Learning of Augmented Reality based Human Interactions for Automating a Robot Team

Abstract

Getting a team of robots to achieve a relatively complex task using manual manipulation through augmented reality (AR) is interesting. However, the true potential of such an approach manifests when the system can learn from humans. We propose a system comprising a team of robots that performs a previously unseen task—a variant, to be specific—by learning from the sequences of actions taken by multiple human beings doing this task in various ways using deep learning (DL). The training inputs can be through actual manipulation of the team of robots using an augmented-reality tablet or through a simulator. Results indicate that the system is able to fulfill the specified variant of the task more than 80% of the time, inaccuracies mainly owing to unrealistic specifications of tasks. This opens up an avenue of training a team of robots, instead of crafting a rule base.