EN.601.497/697 Human-Centered Robotics: Models and Algorithms

Fall 2026 · 3 credits

Time and Location: MW 3-4:15 PM, Hodson 305

Office Hours: M 4:15-5 PM, Malone 327

Level: Graduate & Undergraduate

Prerequisites: Linear algebra, calculus, probability, and basic programming (e.g., EN.601.220, 601.226).

Signup Forms: Debates · Paper Discussions

If you're presenting a paper at the next lecture, please check in with Prof. Hu at the end of the preceding lecture (your presentation doesn't need to be complete at that point).

Report Template: RSS Format (LaTeX preferred)

Course Description

In this course, we will study fundamental concepts in human-centered robotics, with an emphasis on mathematical models of human-robot interaction and decision-making algorithms for safely deploying robots in human-populated environments. We will ground these ideas in applications such as autonomous vehicles, aerial robots, and home robots.

We will start by introducing the main technical tools for robot planning and control in interactive, safety-critical settings through a game-theoretic lens. After that, we will turn our attention to human-centered robot learning and try to answer two tightly coupled questions: how can robots learn to safely work with humans and help humans learn. The course will combine seminar-style discussions of research papers and whiteboard-style lectures to introduce the key theoretical concepts, and the class project will give you an opportunity to explore the approaches covered in class and possibly combine them with your own research.

After this class, you will be familiar with the state of the art and open challenges in safe and performant human-robot interaction, and you will understand the guarantees and tradeoffs offered by different algorithmic frameworks for human-centered robotics.

Grading

Component	Percentage
Homework assignments (2 × 15%, mix of theoretical and programming tasks)	30%
Paper discussion	5%
In-class debate	5%
Project lightning talk	5%
Midterm project report	5%
Final project presentation	20%
Final project report	30%

Late policy: Late submissions incur a penalty of 20% points per day.

Letter grade conversion (click to expand)

Score Range	Grade	Score Range	Grade	Score Range	Grade
95-100%	A+	79-82%	B	66-70%	C-
90-95%	A	76-79%	B-	63-66%	D+
85-90%	A-	73-76%	C+	60-63%	D
82-85%	B+	70-73%	C	Below 60%	F

Final scores may be curved.

Generative AI Policy (click to expand)

Allowed

Helping you understand a paper.
Helping you understand a homework assignment.
Helping with literature reviews (you are responsible for verifying any hallucinated references).
Brainstorming research ideas.
Generating illustrative figures and polishing text in presentations and reports.

Not Allowed (can result in losing all points on the assignment)

Generating entire slides in a presentation.
Generating entire solutions or paragraphs in assignments and reports.
Generating core coding solutions (e.g., all of main.py).

Reference Textbooks

(There is no required textbook)

Dimitri Bertsekas, Reinforcement Learning and Optimal Control.
Tamer Başar, Geert Jan Olsder, Dynamic Noncooperative Game Theory, 2nd Edition.
David Fridovich-Keil, Smooth Game Theory.
Somil Bansal, Jaime Fernández Fisac, Safe Neurosymbolic Learning and Control.
Christopher Bishop, Pattern Recognition and Machine Learning.
Andrea Thomaz, Guy Hoffman, Maya Cakmak, Computational Human-Robot Interaction.

Learning Objectives

You will learn to formulate and solve physical human-robot interaction problems using a set of interdisciplinary tools from game theory, machine learning, dynamical systems, and cognitive science.
You will explore how different mathematical models can help inform robot decision-making in safety-critical, human-interactive settings.
You will learn principles and algorithms for ensuring safety when deploying robots in human-populated environments, including safety filters, dynamic games, planning under uncertainty, and learning-based approaches, and understand different guarantees and tradeoffs associated with various decision-making frameworks.
You will study how robots can support and accelerate human learning through strategic interactions, as well as the algorithmic principles behind it, including shared autonomy, assistance games, and active teaching methods that go beyond reactive assistance.
Through debate sessions, inspired by the ICRA Robotics Debates, you will develop critical thinking about emerging challenges in modern robotics, and learn to articulate, defend, and critique competing perspectives.
Through a 5-minute lightning talk, modeled after common practices at major robotics conferences, you will learn to communicate technical insights concisely and prepare for future conference engagement.
Through a semester-long project, you will learn the full pipeline of conducting robotics research, including dissecting research papers, identifying limitations in existing approaches, formulating problems with suitable models, developing algorithmic solutions, and deploying and verifying them on hardware platforms with real human participants. The Alliance AI Lab is committed to providing ongoing mentorship and equipment support for students wishing to advance their projects toward publication.

Syllabus

Week	Date	Lecture	Topic	Type	Notes
Module I: Foundations of Interactive Robotics
1	Aug 31	1	Introduction: Why should we study human-centered robotics? [Slides]	Lecture	Recommended Podcast: Can We Trust Robots?
1	Sep 2	2	Elements of Human-Centered Robotics: Dynamical systems, control policies, uncertainty types, safety guarantees, runtime inference [Slides] [Slides-annotated]	Lecture	Required Reading: All models are wrong. George Box (1976)
2	Sep 7		Labor Day 🏖️
2	Sep 9	3	Robotic motion planning I: Optimal control and dynamic programming [Slides] [Slides-annotated]	Lecture Signup Forms Due	Optional Reading: A Tour of Reinforcement Learning: The View from Continuous Control. Recht (2019)
3	Sep 14	4	Robotic motion planning II: ILQR, MPC, RL, Search Algorithms [Slides] [Slides-annotated]	Lecture	Optional Reading 1: Predictive Control for Linear and Hybrid Systems. Borrelli, Bemporad, Morari (2016) Optional Reading 2: Planning Algorithms. LaValle (2006)
3	Sep 16	5	Robot safety I: Operational design domain, safety filters, HJ reachability [Slides]	Lecture	Required Reading: The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systems. Hsu et al. (2024) Optional Reading: Handbook on Safety Certificates: Techniques from Hamilton-Jacobi Reachability Analysis and Control Barrier Functions. Wang et al. (2026) [Link available in October]
4	Sep 21	6	Robot safety II: CBFs, rollout/gameplay filters, unified safety filter theory [Slides]	Lecture Debate	Debate Proposition: A robot that fails less often than humans performing the same type of task should be considered safe enough for deployment. Required Reading: Control Barrier Functions: Theory and Applications. Ames et al. (2019) Optional Reading: Safe Reinforcement Learning with Nonlinear Dynamics via Model Predictive Shielding. Bastani (2020)
4	Sep 23	7	Emerging topics in robot safety and risks	Paper discussion	Paper 1: Generalizing Safety Beyond Collision-Avoidance via Latent-Space Reachability Analysis. Nakamura et al. (2025) Paper 2: Safe Exploration for Optimization with Gaussian Processes. Sui et al. (2015) Paper 3: How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics. Majumdar et al. (2017)
5	Sep 28	8	Duy P. Nguyen (Waymo): Scaling safe RL for high-dimensional robotic systems	Guest Lecture HW release	Homework 1: Motion planning, robot safety
Module II: The Game Theory of Human-Robot Interaction
5	Sep 30	9	Introduction to dynamic game theory	Lecture	Required Reading: Efficient Iterative Linear-Quadratic Approximations for Nonlinear Multi-Player General-Sum Differential Games. Fridovich-Keil et al. (2020) Optional Reading: Blending Data-Driven Priors in Dynamic Games. Lidard et al. (2024)
6	Oct 5	10	Allen Z. Ren (Physical Intelligence): Learning Generalist Robot Policies	Guest Lecture Project proposal due
6	Oct 7	11	Human-robot interaction as a dynamic game	Lecture Debate	Debate Proposition: Learning-based methods (e.g., RL) are more useful than optimization-based methods (e.g., MPC) for deploying robots around people. Required Reading: Planning for Autonomous Cars that Leverage Effects on Human Actions. Sadigh et al. (2018) Optional Reading 1: Understanding the Intentions of Others: Re-Enactment of Intended Acts by 18-Month-Old Children. Meltzoff (1995) Optional Reading 2: Active uncertainty reduction for safe and efficient interaction planning: A shielding-aware dual control approach. Hu et al. (2024)
7	Oct 12	12	Safety filtering around humans	Lecture	Required Reading 1: Deception Game: Closing the Safety-Learning Loop in Interactive Robot Autonomy. Hu et al. (2023) Required Reading 2: Safety with Agency: Human-Centered Safety Filter with Application to AI-Assisted Motorsports. Oh et al. (2025) Optional Reading: On Infusing Reachability-Based Safety Assurance within Planning Frameworks for Human-Robot Vehicle Interactions. Leung et al. (2020)
7	Oct 14	13	Cooperative games and value alignment	Lecture Debate	Debate Proposition: We can never enforce absolute (i.e., 100%) safety for human-robot interaction. Required Reading 1: Cooperative Inverse Reinforcement Learning. Hadfield-Menell et al. (2016) Required Reading 2: Pragmatic-Pedagogic Value Alignment. Fisac et al. (2017) Optional Reading: AssistanceZero: Scalably Solving Assistance Games. Laidlaw et al. (2025)
8	Oct 19	14	Planning with human internal states	Paper discussion	Paper 1: Contingency Games for Multi-Agent Interaction. Peters et al. (2024) Paper 2: Improving Automated Driving through POMDP Planning with Human Internal States. Sunberg and Kochenderfer (2022) Paper 3: Probabilistically Safe Robot Planning with Confidence-Based Human Predictions. Fisac et al. (2018)
Module III: Robots that Learn to Safely Work with Humans
8	Oct 21	15	Safe robot learning I: Multi-agent RL, safe RL, learning-based HJ reachability, adversarial RL	Lecture	Required Reading 1: Bridging Hamilton-Jacobi Safety Analysis and Reinforcement Learning. Fisac et al. (2019) Required Reading 2: DeepReach: A Deep Learning Approach to High-Dimensional Reachability. Bansal and Tomlin (2020) Required Reading 3: Robust Adversarial Reinforcement Learning. Pinto et al. (2017) Optional Reading 1: Learning Control Barrier Functions from Expert Demonstrations. Robey et al. (2020) Optional Reading 2: MAGICS: Adversarial RL with Minimax Actors Guided by Implicit Critic Stackelberg. Wang et al. (2024)
9	Oct 26	16	Safe robot learning II: RL with safety filters, verification of learned policies	Lecture HW release	Required Reading 1: Provably Optimal Reinforcement Learning under Safety Filtering. Oh et al. (2025) Required Reading 2: Verification of Neural Reachable Tubes via Scenario Optimization and Conformal Prediction. Lin and Bansal (2024) Required Reading 3: Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning. Brunke et al. (2022) Optional Reading: Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming. Fazlyab et al. (2020) Homework 2: Dynamic games, safe learning, verification
9	Oct 28	17	Nandan Tumu (NEC Labs): Conformal Prediction for Trustworthy Robot Learning Abstract: In this lecture, we will cover the basics of Conformal Prediction, and its' applications in robotics. We will address approaches to address uncertainty in robotic sensing, perception, and safety, and the conditions required to achieve safety guarantees with conformal methods. By the end of this lecture, the student should come away with an understanding of the potential and limitations of Conformal Prediction in robotics.	Guest Lecture
10	Nov 2	18	Inverse RL and games	Lecture Debate	Debate Proposition: Safe (reinforcement) learning is fundamentally Mission Impossible for robots in real-world deployment. Required Reading: Maximum Entropy Inverse Reinforcement Learning. Ziebart et al. (2008) Optional Reading: Inferring Objectives in Continuous Dynamic Games from Noise-Corrupted Partial State Observations. Peters et al. (2021)
10	Nov 4	19	Robot safety in the LLM era	Paper discussion	Paper 1: Safety Guardrails for LLM-Enabled Robots. Ravichandran et al. (2025) Paper 2: Real-Time Anomaly Detection and Reactive Planning with Large Language Models. Sinha et al. (2024) Paper 3: Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners. Ren et al. (2023)
Module IV: Robots that Help Humans Learn
11	Nov 9	20	Legibility and predictability	Lecture Debate	Debate Proposition: Zero-sum games are a useful abstraction of safe robot operation in uncertain, human-populated environments. Required Reading 1: Legibility and Predictability of Robot Motion. Dragan et al. (2013) Required Reading 2: “Data Will Solve Robotics and Automation: True or false?”: A Debate. Amato et al. (2025)
11	Nov 11	21	Midterm "mini-conference": 2-min lightning talks	Lightning talks Midterm report due
12	Nov 16	22	AI coaching	Lecture Debate	Debate Proposition: Explicit representations of human skill level and productive failures are both essential for a robot coach to effectively teach humans. Required Reading: AI Coaching for Accelerating Human Skill Development with Reinforcement Learning. Wang et al. (2026) Optional Reading 1: Learning from Errors. Metcalfe (2017) Optional Reading 2: The Mundanity of Excellence. Chambliss (1989)
12	Nov 18	23	Kaiqu Liang (Princeton/Anthropic): RLHF, Human-AI safety	Guest Lecture	Required Reading 1: Deep Reinforcement Learning from Human Preferences. Christiano et al. (2017) Required Reading 2: RLHS: Mitigating Misalignment in RLHF with Hindsight Simulation. Liang et al. (2026) Optional Reading 1: Human-AI Safety: A Descendant of Generative AI and Control Systems Safety. Bajcsy and Fisac (2024) Optional Reading 2: From Refusal to Recovery: A Control-Theoretic Approach to Generative AI Guardrails. Pandya et al. (2025) Optional Reading 3: Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback. Casper et al. (2023)
13	Nov 23		Fall Recess 🏖️
13	Nov 25		Fall Recess 🏖️
14	Nov 30	24	Robots that assist and teach	Paper discussion	Paper 1: Dreaming to Assist: Learning to Align with Human Objectives for Shared Control in High-Speed Racing. DeCastro et al. (2024) Paper 2: Proximal State Nudging: Reducing Skill Atrophy from AI Assistance. Srivastava et al. (2026) Paper 3: Spatio-Temporal Retrieval-based Priors for Adaptive Computational Teaching in Driving. Gopinath et al. (2026)
14	Dec 2	25	Course wrap-up	Lecture Debate	Debate Proposition: With enough data we can solve safety and alignment in human-centered robotics.
15	Dec 7	26	Project Final Presentations 1	Project presentation
15	Dec 9	27	Project Final Presentations 2	Project presentation

Acknowledgement

This course is inspired by and partially builds on the following courses:

Safety-Critical Robotics and AI (taught by Jaime Fernández Fisac), Princeton.
Interactive Robotics (taught by Andrea Bajcsy), CMU.
Physical Intelligence (taught by Antonio Loquercio), Penn.
Model Predictive Control (taught by Manfred Morari), Penn.