Papers
6
Total Citations
119
H-Index
4
About
Hengshuai Yao is a leading researcher in reinforcement learning (RL) and robotics, whose work bridges the gap between theoretical algorithms and real-world autonomous navigation. His most impactful contribution addresses the critical challenge of mapless collision avoidance in crowded human environments. In his 2020 paper (71 citations), Yao introduced a novel framework that distinguishes between "ego-safety" (the robot's own collision risk) and "social-safety" (the robot's impact on surrounding pedestrians), enabling robots to navigate dynamically without pre-built maps. This work has become a cornerstone for socially-aware autonomous systems. Yao has also made significant theoretical advances in RL exploration. He proposed the Quantile Option Architecture (QUOTA), which leverages distributional RL to make decisions based on the full quantile distribution of values rather than just the mean—opening a new dimension for efficient exploration. Additionally, his ACE algorithm (Actor Ensemble for Continuous Control) uses multiple actors to search for global maxima in continuous action spaces, achieving state-of-the-art performance in complex control tasks. With over 100 total citations and foundational contributions to both practical robotics and algorithmic RL theory, Yao’s research is essential reading for anyone working at the intersection of safe autonomous systems and deep reinforcement learning.
Research Focus
Key Achievements
Top Papers
- 1
- 2QUOTA: The Quantile Option Architecture for Reinforcement Learning20 citations · 2019
- 3ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search19 citations · 2019
- 4QUOTA: The Quantile Option Architecture for Reinforcement Learning4 citations · 2018
- 5ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search3 citations · 2018
- 6Practical Issues of Action-Conditioned Next Image Prediction2 citations · 2018