Kurt Keutzer

Factorized language representations with knowledge and logic

Knowledge and logical reasoning are essential constituents of spoken and written language. Despite the availability of more compact and accessible representations, such as knowledge graphs (KG) and logic forms, modern NLU technologies predominantly encapsulate knowledge and logical reasoning in vector representations along with other linguistic patterns. Despite its successes, this encapsulation erects barriers between NLU and symbolic technologies developed for knowledge representation and reasoning, which are inherently more transparent, robust, and scalable than their...

Addressing Challenges in Large-scale Distributed AI Systems

Training Neural Network models is becoming increasingly more expensive, requiring scaling to thousands of processes. This problem is becoming more challenging, as the training data is growing exponentially as well, especially in light of recent unsupervised learning methods. This has made it difficult to apply NN models to large scale problems....

Echo State Transformers

We demonstrate that transformers obtain impressive performance even when some of the layers are randomly initialized and never updated. Inspired by old and well-established ideas in machine learning, we explore a variety of non-linear “reservoir” layers interspersed with regular transformer layers, and show improvements in wall-clock compute time until convergence, as well as overall performance, on various machine translation.

Updates

Closing...

Hardware Software Co-Design for NLP and Recommendation Systems

This project investigates the co-design of Deep Neural Nets and their hardware support in Neural Net Accelerators.

Project Updates

Project updates. ...