We propose a framework for training competitive robots that maintain safety without sacrificing performance, by separating safety synthesis from task learning in a two-stage reinforcement learning process. We formulate interactions as safety-critical games, prove that safety filtering preserves non-exploitability when players maintain safe maneuvers, and demonstrate superior performance over existing safe RL methods in quadruped touchdown games and hardware tests.

We study strict reach-avoid-stay (sRAS) control, which requires safely reaching a target and remaining there indefinitely after first entry. We present a Q-control barrier function filter for high-dimensional black-box systems with bounded uncertainty, merging a stay value encoding safe permanent residence with a reach-avoid value encoding safe target reachability. Our approach requires no knowledge of system dynamics, structural properties, or hand-crafted barriers, and we validate it on quadruped gap-jumping experiments (simulated and physical) and simulated autonomous racing scenarios.

We consider a coaching setting where an expert AI teaches a novice human to acquire motor skills for high-speed drone racing. We formulate this problem as a general-sum game, where the AI's goal is to promote long-term human skill development and independence. By strategically reducing AI assistance and providing demonstrations after the human has experienced failures, the AI coach can foster more effective learning than with static, full assistance. In this case, the AI is no longer a passive guardian angel but an active mentor that deliberately orchestrates challenging scenarios as structured learning signals.

We propose a human-centered safety filter (HCSF) for shared autonomy that significantly enhances system safety without compromising human agency. Our HCSF is built upon a novel state–action control barrier function (Q-CBF) safety filter, which does not require any knowledge of the system dynamics for both synthesis and runtime safety enforcement.

In this paper, we present a novel adversarial reinforcement learning algorithm that guarantees local convergence to a minimax equilibrium solution. We then build on this approach to provide local convergence guarantees for a general deep RL-based robot safety synthesis algorithm.

We consider the multi-agent spatial navigation problem of computing the socially optimal order of play, i.e., the sequence in which the agents commit to their decisions, and its associated equilibrium in an N-player Stackelberg trajectory game. We model this problem as a mixed-integer optimization problem over the space of all possible Stackelberg games associated with the order of play’s permutations.

In this work, we investigate a principled methodology to blend a data-driven reference policy with an optimization-based game-theoretic policy. We formulate KLGame, an algorithm for solving non-cooperative dynamic game with Kullback-Leibler (KL) regularization with respect to a general, stochastic, and possibly multi-modal reference policy. Our method incorporates, for each decision maker, a tunable parameter that permits modulation between task-driven and data-driven behaviors.

To systematically reconcile safety and performance in designing human-centered autonomy, our key idea is to equip a safety filter with a task policy that predicts a wide range of possible interactions, induced by factors such as the human’s goal or alertness, between the robot and nearby humans. These multi-modal predictions enable the robot to preempt future costly safety filter interventions and, where possible, adjust its course of action to avoid the risk of having to apply an inefficient last-minute maneuver. This approach transforms conventional probabilistic safety (i.e., gambits) into prediction-centric performance tuning under strict safety guarantees.