How can we train humanoid mobile manipulators to perform complex manipulation tasks in the real world? Humanoids must be able to coordinate their arms and legs to perform complex tasks, but learning such skills via reinforcement learning is challenging. In R2S2, Li Yi tells us how we can learn a robot action space based on reinforcement learning which can transfer from simulation to the real world and supports manipulation tasks like picking up a box.
Watch Episode 30 of RoboPapers to find out more!
Abstract:
Humans possess a large reachable space in the 3D world, enabling interaction with objects at varying heights and distances. However, realizing such large-space reaching on humanoids is a complex whole-body control problem and requires the robot to master diverse skills simultaneously-including base positioning and reorientation, height and body posture adjustments, and end-effector pose control. Learning from scratch often leads to optimization difficulty and poor sim2real transferability. To address this challenge, we propose Real-world-Ready Skill Space (R2S2). Our approach begins with a carefully designed skill library consisting of real-world-ready primitive skills. We ensure optimal performance and robust sim2real transfer through individual skill tuning and sim2real evaluation. These skills are then ensembled into a unified latent space, serving as a structured prior that helps task execution in an efficient and sim2real transferable manner. A high-level planner, trained to sample skills from this space, enables the robot to accomplish real-world goal-reaching tasks. We demonstrate zero-shot sim2real transfer and validate R2S2 in multiple challenging goal-reaching scenarios.
