Simulate to Learn, Learn to Simulate for Dexterous Robot Control

Abstract: Simulation enables robots to learn and evaluate behaviors at scale before real-world deployment. Yet the mismatch between simulation and the physical world remains a fundamental obstacle. This is particularly challenging for dexterous manipulation, where contact-rich interactions and dynamics variations across objects and robot embodiments are difficult to model. In my thesis research, I explore how robot learning can scale through simulation and how learned models can make simulation more accurate to the physical world, through two complementary directions. Part I: Differentiable simulation for scalable robot learning. First, I present a GPU-parallel differentiable multiphysics simulation and a first-order reinforcement learning algorithm that pairs simulation gradients with entropy regularization, for smoother policy optimization on locomotion and manipulation tasks. Next, I introduce hybrid analytic differentiability, combining implicit differentiation, auto-differentiation, and custom analytic Jacobians to compute gradients through contact without modifying forward dynamics. With it, I develop a production-ready differentiable simulation and show how its gradients support initial value problems, trajectory optimization, and system identification. Part II: Aligning simulation with the real world across diverse embodiments and tasks. First, I introduce an algorithm for iterative real-to-sim alignment. Alongside, I present a hybrid neural dynamics model that combines learned dynamics correction with analytical inverse dynamics while retaining the simulator's contact resolution, to produce physically consistent simulation trajectories. Next, I build flexible real-time robot I/O infrastructure for synchronized data collection and policy deployment across different robots, sensors, and interfaces. In my proposed work, I will explore how differentiable simulation and differentiable rendering can support real-to-sim reconstruction of simulation environments from multimodal real-world data. In my final project, I will study how neural dynamics can scale real-to-sim-to-real learning to dexterous hands and humanoid robots requiring high-dimensional continuous control. Together, these directions aim to establish a feedback loop in which robot policies and simulators continually improve one another. By turning physical experience into better simulators and using better simulators to train more capable robots, this loop could scale robot learning across tasks and embodiments in ways that neither simulation nor real-world data can achieve alone. Thesis Committee: Jean Oh (co-chair) Guanya Shi (co-chair) Jeff Ichnowski Miles Macklin (NVIDIA) Thesis Link