Machine learning is increasingly being used in applications involving sensitive data, such as healthcare and finance. This necessitates approaches that incorporate secure and private use of data. Differential privacy is the main framework for addressing these needs. However its adoption has been rife with barriers especially for distributed data. One reason for this is that theoretical guarantees often consider extreme cases where the data is fully distributed across agents (one data point per agent). This has led to impractical privacy guarantees, e.g., some methods...