Retargeting—mapping human hand trajectories to robot joint commands—is fundamental to dexterous manipulation, bridging the embodiment gap for both offline learning and online teleoperation. Recent work leaves gaps: kinematics-based methods prioritize mimicry over task completion, while object-centric methods train separate models per demonstration, limiting generalization.
HOVER builds on object-centric learning but introduces a virtual operator that simulates human teleoperation adaptively. This enables goal-directed behaviors that generalize across tasks while preserving human-like motion, with direct hardware deployment capability.
Key Contributions
- Generalizable Retargeting Policy: Single policy across diverse tasks achieving both high task fidelity and anthropomorphism—metrics that existing methods trade off.
- Virtual Operator Methodology: Novel training paradigm simulating closed-loop human control for offline demonstrations, maintaining generalization through minimal interface design.
- Comprehensive Evaluation: Improved success rates across diverse manipulation tasks in simulation and successful real hardware deployment with DexHand021.
