Efficient cross-embodiment transfer via world models and policy steering

Abstract: Learning robot manipulation policies typically requires large amounts of high-quality demonstration data, which is expensive to collect on real robots. While large robot and human datasets exist, embodiment gaps make transferring knowledge between platforms challenging. This thesis investigates cross-embodiment transfer--how robots can learn from experience collected on different robots and humans to reduce the need for new demonstrations. I present Latent Policy Steering (LPS), a framework that pretrains a world model across diverse embodiments using embodiment-agnostic visual dynamics represented by optical flow. The pretrained model can be adapted to an unseen robot using less than an hour of teleoperation data. At deployment, the world model enables the policy to anticipate the consequences of its actions, identify likely mistakes, and steer itself back toward safer behaviors through latent-space search. Experiments on both real-world manipulation tasks and simulation benchmarks demonstrate that this approach consistently improves policy performance and outperforms methods relying on embodiment-specific representations. In summary, LPS demonstrates an effective paradigm for efficient cross-embodiment transfer, leveraging diverse, cost-effective robot and human data to adapt to an unseen robot with minimal demonstrations, reducing data collection costs and enabling more scalable robot learning. Committee: Jeff Schneider (advisor) Christopher Atkeson Tejus Gupta