Efficiently finding an occluded object with lateral access arises in many contexts such as warehouses, retail, healthcare, shipping, and homes. We introduce LAX-RAY (Lateral Access maXimal Reduction of occupancY support Area), a system to automate the mechanical search for occluded objects on shelves. For such lateral access environments, LAX-RAY couples a perception pipeline predicting a target object occupancy support distribution with a mechanical search policy that sequentially selects occluding objects to push to the side to reveal the target as efficiently as possible. Within the context of extruded polygonal objects and a stationary target with a known aspect ratio, we explore three lateral access search policies: Distribution Area Reduction (DAR), Distribution Entropy Reduction (DER), and Distribution Entropy Reduction over Multiple Time Steps (DER-MT) utilizing the support distribution and prior information. We evaluate these policies using the First-Order Shelf Simulator (FOSS) in which we simulate 800 random shelf environments of varying difficulty, and in a physical shelf environment with a Fetch robot and an embedded PrimeSense RGBD Camera. Average simulation results of 87.3% success rate demonstrate better performance of DER-MT with 2 prediction steps. When deployed on the robot, results show a success rate of at least 80% for all policies, suggesting that LAX-RAY can efficiently reveal the target object in reality. Both results show significantly better performance of the three proposed policies compared to a baseline policy with uniform probability distribution assumption in non-trivial cases, showing the importance of distribution prediction.
Update
Researchers
- Michael Danielczuk, UC Berkeley, link
- Huang Huang, UC Berkeley
- Jeffrey Ichnowski, UC Berkeley, link
- Kate Sanders, UC Berkeley
- Anelia Angelova, Google, link
- Vincent Vanhoucke, Google, link
- Ken Goldberg, UC Berkeley, link
Overview
See the project website links below for overviews, videos, and paper links. We are also planning on open-sourcing code in the future. The X-Ray paper was presented at IROS 2020 and the LAX-RAY paper will be presented at IROS 2021.
In future work we are pursuing (1) combinations of grasping and pushing actions using a suction cup gripper and (2) pushing multiple objects and leveraging inter-object contacts, among other ideas.
Links
- Project Pages (includes videos and paper links):
- Feel free to contact me at mdanielczuk@berkeley.edu with any questions, comments, or opportunities/requests for collaboration!
- arXiv Links: