David Wagner

Self-supervised Open-World Segmentation

Overview

Standard benchmarks in image segmentation assume a "closed-world" setting, in which a pre-determined set of non-overlapping object categories is exhaustively segmented and labeled in all training and evaluation images. This significantly increases the difficulty of data collection, requiring either complex quality control and post-processing schemes if using crowd-sourced labeling or...

Better Visual Representations through Language Supervision

Overview

CLIP [1] demonstrates the power of simple contrastive learning on a large-scale dataset of image and caption pairs collected directly from the internet, without expensive manual labeling. In our project we seek to improve the data efficiency of CLIP and performance when trained on uncurated datasets, as well as explore additional capabilities beyond classification.

Data Efficiency

CLIP achieves impressive results when trained on WIT400M, a dataset of 400M image and caption pairs collected by searching for a curated list of...

Robust Image Classification via Parts and Disentangled Attributes

Robustness is steadily becoming a real concern for machine learning models, especially when deployed in security or safety-critical settings. Significant research has demonstrated the fragility of neural networks when the i.i.d. assumption does not hold, e.g., natural corruptions and adversarial perturbation. Many application domains...

Realistic Large-Scale Benchmark for Adversarial Patch

Problem

The goal of our study is to make machine learning models robust against patch attacks. In particular, we will develop the first standardized benchmark for security against patch attacks, under realistic threat models. Our benchmark will cover two important aspects often ignored in past work: (1) realistic patches that must work under...

Disentangling Input Signals for Robust Computer Vision

High-dimensional real world imagery presents an embarrassment of riches to powerful, overparameterized neural networks; it is possible to train image classification models to surprising levels of accuracy on high-frequency or low-frequency features alone. The dominant paradigm of training and evaluating deep neural networks on independent and identically distributed (IID) data splits has obscured a significant weakness of current models, namely a lack of robustness to distribution shifts. One class of explanations posits that powerful models can learn spurious correlations, including...

Towards Robust Neural Networks with Conditional Generative Models

We aim to build neural networks that are intrinsically robust against adversarial attacks. We focus on classifying images in real-world scenarios with complex backgrounds under unforeseen adversarial attacks. Previous defenses lack interpretability and have limited robustness against unforeseen attacks, failing to deliver trustworthiness to users. We will study Bayesian models, which are more interpretable and have intrinsic robustness. We will explore two directions: extending an existing Bayesian classifier with better models and building new Bayesian models from discriminative...