Ken Goldberg

Learning Large Touch-Vision-Language Models Using Self-Supervised Robot Learning

Abstract

Humans depend on the integration of multiple sensory inputs, including but not limited to vision, language, audio, and tactile, to successfully carry out daily tasks. Giving robots an analogous ability to perceive and process information from different sensory modalities enables a richer understanding of the physical...

Learning Robust Robot Policies to Manipulate and Fill Deformable Bags

Rearranging and manipulating deformable objects such as cables, fabrics, and bags is a long-standing challenge in robotic manipulation. The complex dynamics and high-dimensional configuration spaces of deformables, compared to rigid objects, make manipulation difficult not only for multi-step planning, but even for goal specification. Goals cannot be as easily specified as rigid object poses, and...

X-Ray for Lateral Access Mechanical Search

Efficiently finding an occluded object with lateral access arises in many contexts such as warehouses, retail, healthcare, shipping, and homes. We introduce LAX-RAY (Lateral Access maXimal Reduction of occupancY support Area), a system to automate...

Offline Recovery RL: Offline Reinforcement Learning with Safe Online Adaptation

Safety remains a central obstacle preventing widespread use of RL in the real world: learning new tasks in uncertain...