General In-Hand Object Rotation with Vision and Touch
Haozhi QiBrent YiSudharshan SureshMike LambetaYi MaRoberto CalandraJitendra Malik
Haozhi QiBrent YiSudharshan Suresh
...+3
Jitendra Malik
Sep 2023
0被引用
1笔记
开学季活动火爆进行中，iPad、蓝牙耳机、拍立得、键盘鼠标套装等你来拿
摘要原文
We introduce RotateIt, a system that enables fingertip-based object rotation along multiple axes by leveraging multimodal sensory inputs. Our system is trained in simulation, where it has access to ground-truth object shapes and physical properties. Then we distill it to operate on realistic yet noisy simulated visuotactile and proprioceptive sensory inputs. These multimodal inputs are fused via a visuotactile transformer, enabling online inference of object shapes and physical properties during deployment. We show significant performance improvements over prior methods and the importance of visual and tactile sensing.
