Virtual Humans: A New Toolkit for Cognitive Science Research.

Abstract

Virtual Humans: A New Toolkit for Cognitive Science Research Jonathan Gratch ([email protected]), Arno Hartholt ([email protected]), Morteza Dehghani ([email protected]), Stacy Marsella ([email protected]) Institute for Creative Technologies, University of Southern California 12015 Waterfront Drive, Playa Vista, CA 90094, USA Social Cognition: People interact socially through their bodies and VHs allow researchers to systematically examine and model the cognitions underlying social interaction. VHs can act as “virtual confederates” (Blascovich et al., 2002), allowing systematic manipulation of visual appearance, speech type, and contextual graphical environments. This makes VHs a convenient platform to isolate unique socio- cultural characteristics and realize them through simulation. Along with enhanced experimental control, ease of manipu- lations, consistency and controlled measurements, these features make VHs useful and reliable tools for studying social cognitions. In the tutorial, we will review several ex- amples, including how expressions of emotion by VHs can influence decision making in negotiations tasks and social dilemmas (e.g. de Melo et al., 2012; Dehghani et al., 2012); the role of accent in cultural cognition (Dehghani et al., 2012) and the role of rapport and gender in enhancing par- ticipants’ performance (Karacora et al., 2012). Keywords: Virtual humans, embodied cognition, social cog- nition, virtual confederates Tutorial Objectives Virtual humans (VHs) are digital anthropomorphic charac- ters that exist within virtual worlds but are designed to per- ceive, understand and interact with real-world humans. Alt- hough typically conceived as practical tools to assist in a range of application (e.g., HCI, training and entertainment), the technology is gaining interest as a methodological tool for studying human cognition. VHs not only simulate the cognitive abilities of people, but also many of the embodied and social aspects of human behavior more traditionally studied in fields outside of cognitive science. By integrating multiple cognitive capabilities (e.g., language, gesture, emo- tion, and the control problems associated with navigating and interacting with a simulated virtual world) and requiring these processes to support real-time interactions with peo- ple, VHs create a unique and challenging environment with- in which to develop and validate cognitive theories. In this tutorial, we will review recent advances in VH technologies, demonstrate examples of use of VHs in cognitive science research and provide hands on training using our Virtual Human Toolkit (http://vhtoolkit.ict.usc.edu/). The Virtual Human Toolkit The University of Southern California’s Institute for Crea- tive Technologies (ICT) is recognized as a leader in the de- velopment of VH technology (Gratch et al., 2002) and in applying this research to application domains including “vir- tual role players” for interpersonal-skills training (e.g., Campbell et al. 2011), informal science education (e.g. Swartout et al., 2010), intelligent tutoring, (e.g. Lane et al., 2011), and as “virtual confederates” to study cognitive and social processes (e.g. de Melo et al., 2012; Dehghani et al., 2012). One goal of the institute is to foster research in VH by making this technology freely available for research pur- poses through the Virtual Human Toolkit. The research underlying the toolkit draws heavily on cognitive science research. For example, VH “brains” are inspired by psychological theories of human cognition (e.g. Swartout, Gratch et al., 2006), language (e.g. Traum, 2008) and emotion (Gratch & Marsella, 2005), VH bodies are in- formed by knowledge of physiological and biomechanical processes (e.g. Honglun, et al. 2007; Thiebaux et al., 2008) and the relation between the VH’s brain and body is in- formed by social psychology research (Lee & Marsella, 2006, Wang et al, 2013). Translating these theories and findings into working software