《2-1 联邦学习的核心挑战和对策.pdf》由会员分享,可在线阅读,更多相关《2-1 联邦学习的核心挑战和对策.pdf(30页珍藏版)》请在三个皮匠报告上搜索。
1、Federated Learning:Challenges and Solutions吴方照 微软亚洲研究院 主管研究员|2Privacy is Important for AI AI relies on data for model training and online serving Highly privacy sensitive in many scenarios Strict laws on user privacy protection3Federated Learning Collaboratively learn a shared model while keeping da
2、ta on device Decouple the ability of learning from the need of data centralizationCommunication-Efficient Learning of Deep Networks from Decentralized Data,AISTATS 20174Applications of Federated Learning Examples Gboard text prediction Siri personalization5Federated Learning:Key ChallengesHeterogene
3、ityFederatedLearningEfficiencyPrivacySecurity6Federated Learning:Our WorksHeterogeneityEfficiencyPrivacySecurity FedKD Efficient-FedRec FedX InclusiveFL FedGNN FedCTR UA-FedRec FedPrompt PrivateFL FedAttack RobustFLFederatedLearning7Federated Learning:Our WorksHeterogeneityEfficiencyPrivacySecurity
4、FedKD Efficient-FedRec FedX InclusiveFL FedGNN FedCTR UA-FedRec FedPrompt PrivateFL FedAttack RobustFLFederatedLearning AI models are bigger and bigger Communication cost between client and server is huge8FedKD:Motivation9FedKD:ModelCommunication-efficient federated learning via knowledge distillati
5、on,Nature Communications10FedKD:Experiments News recommendation11FedKD:Experiments Medical text classification12Federated Learning:Our WorksHeterogeneityEfficiencyPrivacySecurity FedKD Efficient-FedRec FedX InclusiveFL FedGNN FedCTR UA-FedRec FedPrompt PrivateFL FedAttack RobustFLFederatedLearning B
6、ig AI models are expensive to learn Clients usually have weak computing capability 13Efficient-FedRec:Motivation Sub-models may have different privacy and computing requirements Split learning14Efficient-FedRec:Motivation15Efficient-FedRec:ModelEfficient-FedRec:Efficient Federated Learning Framework
7、 for Privacy-Preserving News Recommendation,EMNLP 2021 News Recommendation16Efficient-FedRec:Experiment Efficiency of computation and communication17Efficient-FedRec:Experiment18Federated Learning:Our WorksHeterogeneityEfficiencyPrivacySecurity FedKD Efficient-FedRec FedX InclusiveFL FedGNN FedCTR U
8、A-FedRec FedPrompt PrivateFL FedAttack RobustFLFederatedLearning19InclusiveFL:Motivation Heterogeneous client devices have different computing capabilities Use small model for all clients?Exclude weak clients for big model?20InclusiveFL:ModelNo One Left Behind:Inclusive Federated Learning over Heter
9、ogeneous Devices,KDD 202221InclusiveFL:Experiments Better performance due to contribution from all heterogeneous clients with affordable computing overhead22Federated Learning:Our WorksHeterogeneityEfficiencyPrivacySecurity FedKD Efficient-FedRec FedX InclusiveFL FedGNN FedCTR UA-FedRec FedPrompt Pr
10、ivateFL FedAttack RobustFLFederatedLearning23FedPrompt:Motivation Federated learning cannot provide strict privacy protection guarantee Solution:DP/LDP Challenge:lower accuracy24FedPrompt:Model LDP+Prompt-tuning25FedPrompt:Experiments NLP tasks26Federated Learning:Our WorksHeterogeneityEfficiencyPri
11、vacySecurity FedKD Efficient-FedRec FedX InclusiveFL FedGNN FedCTR UA-FedRec FedPrompt PrivateFL FedAttack RobustFLFederatedLearning27FedAttack:Motivation Federated learning is vulnerable Data poison attack Model poison attack28FedAttack:ModelFedAttack:Effective and Covert Poisoning Attack on Federated Recommendation via Hard Sampling,KDD 202229FedAttack:Experiments非常感谢您的观看|