FENNs: A Resource-Efficient, Adaptive, Privacy-Preserving Decentralized Learning Framework

TL;DR — A novel neural architecture that uses the resource constraint measurements and the task complexity to modify the seed neural architecture for a more efficient network.

Abstract

Deep neural networks have demonstrated exceptional performance in various tasks; yet, their resource-intensive nature and ongoing data privacy concerns remain key obstacles. In response, we introduce Federated Ephemeral Neural Networks (FENNs), a pioneering architecture that ingeniously addresses both challenges. FENNs rely on the concept of ephemeral neural networks (ENNs), a novel paradigm where neural networks exhibit dynamic adaptability in their architecture based on available computing resources. FENNs seamlessly blend the flexibility of ENNs with the privacy-preserving features of federated learning to tailor their structures to task complexity while ensuring data privacy within a decentralized learning environment. Rigorous tests conducted on resource-constrained devices within federated environments validate the effectiveness of FENNs. We also introduce a novel metric for evaluating the efficacy of resource-constrained learning and/or machine learning in resource-constrained environments. The proposed architecture shows significant prospects in the domains of edge computing and decentralized artificial intelligence applications.