Yi Ma

Towards Learning and Auditing Private Foundation Models

Abstract

Foundation models (e.g., DALL-E, GPT-3, CLIP, MAE) – pre-trained on vast amounts of diverse data through self-supervised learning – have emerged as an important building block for artificial intelligence (AI) systems [BHA+2021]. These models can be simply adapted to various downstream applications (e.g., language, vision, robotics) via fine-tuning, prompting, linear probing, etc. Despite foundation models having been extensively deployed, there is a significant lack of understanding regarding the privacy risks associated with training...

Towards a Unified Understanding of Privacy and Generalization for Better Algorithm Design

Abstract

Machine learning and deep learning have emerged as important technologies, which enable a wide range of applications including computer vision, natural language processing, healthcare, and recommendation. However, in order to responsibly deploy these machine learning algorithms in society, it is critical to design them to conform to ethical values such as privacy, safety, fairness, etc. For instance, researchers have found that information about training data can be extracted from a released machine learning model which raises important privacy concerns, and adversarial attacks or...

Learning Dexterous In-Hand Manipulation with Vision and Touch

Overview

Consider the task of stacking LEGO bricks or assembling IKEA furniture in Figure. Given a goal image configuration, humans can rapidly figure out a plan to accurately manipulate the LEGO bricks or furniture parts to achieve the goal. This is mainly due to: 1) humans are already equipped with a good mental dynamics model through daily interaction with objects,...