QoE Evaluation of Real-Time Remote Operation with Network Constraints in a System-of-Systems

Abstract

With the advent of robotics, remote operation gained momentum as it allows for the control and feedback of distant systems, thus reducing human logistics and risk. Despite the growing popularity of fully autonomous systems, remote operation still retains a significant relevance due to operative requirements like manual overrides in critical situations. Such a requirement calls for a robust and reliable remote operation experience. We propose an evaluation system following a system-of-systems architecture to assist in the development of remotely operated applications with tested resilience against network effects. The system has an extensible architecture allowing the swap of real and virtual components and enables the evaluation of remote operation applications in unreliable network environments. In a proof-of-concept study, the system is implemented and used to evaluate the impact of network constraints on a teleoperated high-speed driving scenario. Our results show that the baseline setup leads to approximately equal (± 0.9%) end-to-end driver performances. Thus, other evaluations are primarily affected by additional constraints, enabling our system for extensive analyses. Comprehensive drive trials show that network latency steadily mitigates driving performance, whereas packet loss can be compensated well up to a specific value. In addition, our Quality of Experience (QoE) surveys show the subjective ratings of participants exposed to network impacts.