The ACT-R Unity Interface: Integrating ACT-R with the Unity Game Engine

Abstract

Cognitive architectures are computational frameworks that support the development of computational models of human cognitive processes. They have typically been used to advance our understanding of human cognition in specific task environments; however, they have also been used to support the development of a variety of intelligent systems and agents (e.g., cognitive robots). There are a broad range of reasons to motivate the effort to integrate cognitive architectures with virtual environments. These include the development of intelligent virtual characters for the purposes of training simulations, enhanced gameplay experiences and the modelling of user/gamer behaviour. Virtual environments also support simulations of actual real-world environments (e.g., using physics engines and advanced lighting models) that can be used to perform computational simulations into embedded, extended, and embodied cognition. In the current report, we describe the effort to develop an integration framework that enables the ACT-R cognitive architecture to be used in conjunction with the Unity game engine. The resulting framework, referred to as the ACT-R Unity Integration (ACT-R UI) framework, enables individual ACT-R models to control the behaviour of virtual non-player characters that inhabit 3D virtual environments built on top of Unity. We first provide an overview of the ACT-R architecture and the Unity game engine. We describe the key features of both systems and discuss why they provide such a compelling target for integration. We then go on to describe the nature of the integration solution itself. We outline the extensions to the ACT-R architecture that enable ACT-R models to exchange information with Unity, and we also present the Unity components that enable virtual characters to be controlled or influenced by ACT-R models. Finally, we provide a concrete example of the use of the ACT-R UI. In particular, we show how an ACT-R model can be used to control the behaviour of a virtual robotic character that inhabits a Unity-based virtual environment.