Background: Federated Learning (FL) has emerged as a powerful paradigm for distributed, privacy-preserving machine learning over a large network of devices [1]. Most existing works on FL focus on learning a single model that is deployed to all devices. Given the diverse characteristics of the users and application scenarios, personalization is highly desirable and inevitable in the near future. Personalized Federated Learning (PFL) aims to improve the experience of individual users by training personalized on-device models that overcome the limitations of a common...