Overtrust of Robots in High-Risk Scenarios

Abstract

From personal robot assistant to self-driving vehicles, artificial intelligence (AI) is the backbone underlying millions of future advanced applications. As robots become increasingly pervasive in daily life, it is expected that robots will augment human laborers in many domains in the near future. When robots are deployed in the real world, the underlying assumption is that they are capable of accomplishing their given tasks. However, researchers have shown that robots made mistakes, and in several cases, humans tend to overtrust robotic systems (Abney 2017; Borenstein et al. 2017; Robinette, Howard, and Wagner 2017). Overtrust of a robot happens in scenarios where (1) a person accepts risk because that person believes the robot can perform a function that it cannot or (2) the person accepts too much risk because the expectation is that the system will mitigate the risk." (Abney 2017). In particular, we are interested in two emerging domains where an appropriate amount of trust is a minimal requirement and overtrust could cause harm: 1) healthcare scenarios and 2) self-driving car (i.e. autonomous driving) scenarios. Both healthcare and autonomous driving scenarios often involve high risks, and the negative outcomes could be detrimental to the user. The objective of our research focuses on 1) investigating the causes that contribute to human overtrust of these robots systems 2) developing a behavior-based computational model to predict overtrust, and 3) developing techniques to mitigate outcomes caused by the overtrust.