PRISM: Privacy-Preserving Improved Stochastic Masking For Federated Generative Models

TL;DR — PRISM is a new federated framework for generative models, identifying optimal stochastic binary mask to enhance performance, reduce communication costs, ensure robust privacy preservation, and yields a lightweight generator.

Abstract

While training generative models in distributed settings has recently become increasingly important, prior efforts often suffer from compromised performance, increased communication costs, and privacy issue. To tackle these challenges, we propose PRISM: a new federated framework tailored for generative models that emphasizes not only strong and stable performance but also resource efficiency and privacy preservation. The key of our method is to search for an optimal stochastic binary mask for a random network rather than updating the model weights; i.e., identifying a “strong lottery ticket”: a sparse subnetwork with competitive generative performance. By communicating the binary mask in a stochastic manner, PRISM minimizes communication overhead while guaranteeing differential-privacy (DP). Unlike traditional GAN-based frameworks, PRISM employs the maximum mean discrepancy (MMD) loss, ensuring stable and strong generative capability, even in data-heterogeneous scenarios. Combined with our weight initialization strategy, PRISM also yields an exceptionally lightweight final model with no extra pruning or quantization, ideal for environments such as edge devices. We also provide a hybrid aggregation strategy, PRISM-$\alpha$, which can trade off generative performance against communication cost. Experimental results on MNIST, CelebA, and CIFAR10 demonstrate that PRISM outperforms the previous methods in both IID and non-IID cases, all while preserving privacy at the lowest communication cost. To our knowledge, we are the first to successfully generate images in CelebA and CIFAR10 with distributed and privacy-considered settings.