Active

Long Term Video Understanding

While comprehending the long range temporal structure of events in a video stream is a fundamental problem in computer vision, little progress has been made towards this goal beyond short range comprehension. The roadblocks to progress include (1) Severe memory constraints imposed by on device RAM on GPUs (2) Lack of effective learning techniques that can avoid...

Low-Data Learning for Assistive Video Description

Automatic video captioning aims to train models to generate text descriptions for all segments in a video, however, the most effective approaches require large amounts of manual annotation which is slow and expensive. Unsupervised Learning, Semi Supervised Learning, and Active Learning are methods which are designed to improve the learning efficacy of models in low-data domains. In this project...

Large-scale 3D Reconstruction from Multi-view Image Datasets

This ongoing project attempts at using large scale multi-view datasets available online to build a multi-view 3D reconstruction approach that works on wide-baseline images.

Researchers Shubham Goel, UC Berkeley, https://...

Robustness for Deep Learning/Ethical AI Through Human Value Modeling

Despite the recent advances in adversarial training based defenses, deep neural networks are still vulnerable to adversarial attacks outside the perturbation type they are trained to be robust against. In this project, we propose Protector, a two-stage pipeline to improve the robustness against multiple perturbation types. We demonstrate that...

Generalizing Domain Randomization for Zero-Shot Transfer

The goal of this project is to use reinforcement learning to build pixel-based agents that successfully zero-shot transfer from a simulated environment to reality, which is...

Towards Robust Neural Networks with Conditional Generative Models

We aim to build neural networks that are intrinsically robust against adversarial attacks. We focus on classifying images in real-world scenarios with complex backgrounds under unforeseen adversarial attacks. Previous defenses lack interpretability and have limited robustness against unforeseen attacks, failing to deliver trustworthiness to users. We will study Bayesian models, which are more interpretable and have intrinsic robustness. We will explore two directions: extending an existing Bayesian classifier with better models and building new Bayesian models from discriminative...

Never Decrypt Data Lake

Opaque enables a Never Decrypt Data Lake running computation on top of existing cloud infrastructure. In particular, Opaque offers rich analytics built on top of Apache Spark that computes only on encrypted data. Opaque leverages secure enclaves to ensure that the entire software stack (outside of the enclave) cannot access plaintext, decrypted data, promising greater...

Compressing High Capacity Models with Implicit Neural Networks and Frank-Wolfe

Reducing parameter footprint and inference latency of machine learning models is being driven by diverse applications like mobile vision and on-device intelligence [Choudary 20], and it is increasingly important, as models become increasingly large. In the proposed work, we will develop an alternative to the current train/compress paradigm, and instead we will train sparse high-capacity models from scratch, simultaneously achieving low training cost and high sparsity. We will also explore the robustness of such models.

To do so, we are building an optimization...

Adding Safety and Robustness to Learning for Robots by Learning on Robots

Safety and robustness of robotic systems are crucial for deploying robots in the real world. Machine learning has emerged as a promising tool for enabling robots to perform complex tasks or operate under uncertainties in dynamics and environment. However, learning techniques used often do not take safety into account, which could damage the robot or its environment, hindering deployment of learning-based methods. In contrast, in control theory, several methods such as model-predictive control, Hamilton-Jacobi (HJ) reachability, control barrier functions, and Lyapunov-...

Learning-based Safe Navigation for Dynamic Legged Robots

Legged systems have the potential to navigate over a wide variety of difficult terrain, such as cluttered indoor environments, as illustrated in the figure above. Enabling legged robots to autonomously step on, over and around obstacles make them better suited for mobility in the real-world for...