Overview
Online shopping has been made easier with Augmented Reality. Through Amazon’s View in your room, you can see how products fit and look in your home before you bring them home. We propose to develop scene understanding techniques that could lead to even more advanced shopping features such as: virtually redecorating a home by removing or replacing existing objects, suggesting items based on the room style, suggesting different furniture layout styles.
Specifically, we leverage our expertise on 2D and 3D scene parsing, sketch-to-photo synthesis, and unsupervised representation learning to generate re-stylized photorealistic images given a scene sketch of a room with walls and furniture. Such an image allows the user to re-imagine their room with new items with greater freedom (Fig. 1). As an application extension, we would limit objects to indoor furniture products from a catalog of Amazon items.
Figure. Our goal is to learn a model that can synthesize a photorealistic scene given a freehand sketch of a room without such paired training data across two domains. Top) We study scene synthesis by leveraging our expertise on object-level sketch-to-photo synthesis, 2D/3D scene parsing, and unsupervised representation learning. Bottom) As an application extension, such an image could help re-imagine a room with realistic products available from a 3D object catalogue.
Researchers
Stella Yu (stellayu@berkeley.edu)
Peter Wang (peterwg@berkeley.edu)
Utkarsh Singh (s.utkarsh@berkeley.edu)
Himanshu Arora (arorah@amazon.com)
Brian Zhang (xizhn@amazon.com)
Michael Lou (ylou@amazon.com)
Acknowledgment
We thank AWS for providing computation credits.