1、Federated Learning:Challenges and Solutions吴方照 微软亚洲研究院 主管研究员|2Privacy is Important for AI AI relies on data for model training and online serving Highly privacy sensitive in many scenarios Strict laws on user privacy protection3Federated Learning Collaboratively learn a shared model while keeping da

2、ta on device Decouple the ability of learning from the need of data centralizationCommunication-Efficient Learning of Deep Networks from Decentralized Data,AISTATS 20174Applications of Federated Learning Examples Gboard text prediction Siri personalization5Federated Learning:Key ChallengesHeterogene

3、ityFederatedLearningEfficiencyPrivacySecurity6Federated Learning:Our WorksHeterogeneityEfficiencyPrivacySecurity FedKD Efficient-FedRec FedX InclusiveFL FedGNN FedCTR UA-FedRec FedPrompt PrivateFL FedAttack RobustFLFederatedLearning7Federated Learning:Our WorksHeterogeneityEfficiencyPrivacySecurity

4、FedKD Efficient-FedRec FedX InclusiveFL FedGNN FedCTR UA-FedRec FedPrompt PrivateFL FedAttack RobustFLFederatedLearning AI models are bigger and bigger Communication cost between client and server is huge8FedKD:Motivation9FedKD:ModelCommunication-efficient federated learning via knowledge distillati

5、on,Nature Communications10FedKD:Experiments News recommendation11FedKD:Experiments Medical text classification12Federated Learning:Our WorksHeterogeneityEfficiencyPrivacySecurity FedKD Efficient-FedRec FedX InclusiveFL FedGNN FedCTR UA-FedRec FedPrompt PrivateFL FedAttack RobustFLFederatedLearning B

6、ig AI models are expensive to learn Clients usually have weak computing capability 13Efficient-FedRec:Motivation Sub-models may have different privacy and computing requirements Split learning14Efficient-FedRec:Motivation15Efficient-FedRec:ModelEfficient-FedRec:Efficient Federated Learning Framework

7、 for Privacy-Preserving News Recommendation,EMNLP 2021 News Recommendation16Efficient-FedRec:Experiment Efficiency of computation and communication17Efficient-FedRec:Experiment18Federated Learning:Our WorksHeterogeneityEfficiencyPrivacySecurity FedKD Efficient-FedRec FedX InclusiveFL FedGNN FedCTR U

8、A-FedRec FedPrompt PrivateFL FedAttack RobustFLFederatedLearning19InclusiveFL:Motivation Heterogeneous client devices have different computing capabilities Use small model for all clients?Exclude weak clients for big model?20InclusiveFL:ModelNo One Left Behind:Inclusive Federated Learning over Heter

9、ogeneous Devices,KDD 202221InclusiveFL:Experiments Better performance due to contribution from all heterogeneous clients with affordable computing overhead22Federated Learning:Our WorksHeterogeneityEfficiencyPrivacySecurity FedKD Efficient-FedRec FedX InclusiveFL FedGNN FedCTR UA-FedRec FedPrompt Pr

10、ivateFL FedAttack RobustFLFederatedLearning23FedPrompt:Motivation Federated learning cannot provide strict privacy protection guarantee Solution:DP/LDP Challenge:lower accuracy24FedPrompt:Model LDP+Prompt-tuning25FedPrompt:Experiments NLP tasks26Federated Learning:Our WorksHeterogeneityEfficiencyPri

11、vacySecurity FedKD Efficient-FedRec FedX InclusiveFL FedGNN FedCTR UA-FedRec FedPrompt PrivateFL FedAttack RobustFLFederatedLearning27FedAttack:Motivation Federated learning is vulnerable Data poison attack Model poison attack28FedAttack:ModelFedAttack:Effective and Covert Poisoning Attack on Federated Recommendation via Hard Sampling,KDD 202229FedAttack:Experiments非常感谢您的观看|