Video Game Interface Design Patterns to Facilitate Human-Agent Teaming

Abstract

AbstractUser interface design patterns that document Human-Agent Team (HAT) interface best practices were developed by analyzing team video games with different types of agents, ranging from virtual players in sports games to advisors contributing to strategy and planning in combat games. One factor that may contribute to the enormous success of some games is the cohesive team interactions between humans and agents. A design pattern collection was developed by identifying 25 design patterns under interface themes most relevant to HATs. The design patterns describe how multiple successful game interfaces handle challenging aspects of HAT interaction, addressing contexts including planning, spatial orientation, task management, communication, and more. Based on the complex and well-tested interface solutions in video games, these design patterns can provide HAT user interface developers practical guidance for interface development and a starting point for developing a HAT design pattern language.Keywords: Design patternshuman-autonomy teaminggame analysisuser interface design AcknowledgmentsWe want to thank Iowa State University students Desmond Bonner, Austin Boling, Hayden Havelka, and John Chandara for their work analyzing games. We want to thank Air Force Research Lab and Infoscitex Corporation personnel who met regularly with the team and provided feedback: Kyle Behymer, Gloria Calhoun, Elizabeth Frost, Jacob Ehrenstrom, Hunter Rogers, and Heath Ruff.Disclosure statementThe authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.Additional informationFundingThis research is funded by a grant from the United States Air Force Research Laboratory under the direction of InfoSciTex [Contract number FA8650-20-D-6207, Subcontract FPH14-S012].Notes on contributorsNathan SepichNathan Sepich received his BS in Industrial Engineering and his MS in Industrial Engineering with a co-major in HCI from Iowa State University. His MS thesis work focused on the impact of task workload on cybersickness. Nate is an operations engineering lead at Collins Aerospace on the AI team.Mitchell TalyatMitchell Talyat is a student at Iowa State University pursuing a bachelor's degree in Software Engineering. He has been the president of the Game Development Club since the summer of 2021. Recent work includes research on human-autonomy teaming frameworks.Elmin DidicElmin Didic is an undergraduate Software Engineering student at Iowa State University. His work has applied his Esports and game development background to research human‐agent teams and teaming frameworks. Elmin is a Software Engineering Student Council Leader and a member of the Iowa State University Varsity Valorant team.Michael C. DorneichMichael C. Dorneich is the Joseph Walker Professor of Engineering at Iowa State University in the Industrial and Manufacturing Systems Engineering Department. His research interests focus on creating joint human-autonomy systems that enable people to be effective in complex and stressful environments in aviation, robotic, and space applications.Stephen B. GilbertStephen B. Gilbert is an Associate Professor in Industrial and Manufacturing Systems Engineering at Iowa State University. He is the Associate Director of the Virtual Reality Application Center and Director of its HCI graduate program. His research interests include technology to advance cognition, intelligent tutoring systems, human-autonomy teaming, and XR usability.