1、姜碧野/高级算法专家伯努利：结构化的工业级流式机器学习系统阿里妈妈Bernoulli,An Industrial Streaming System for Machine Learning with Structured DesignsIntroIntroDesignDesignUseUse CasesCasesSummarySummary#1#2#3#4#1#1IntroIntro ofof InternetInternet ApplicationsApplicationsInternet as IR(Information Ranking)Core applications of Inte

2、rnet:Search Engine/Recommendation/AdvertisingInternet is providing information services to users by ranking candidate itemsRanking requires predictionRanking requires predicting user behavior and preference(Click Through Rate etc.)Deep Learning is widely used for predicting CTRUpdating models in min

3、utes is very important!屠龙少年与龙：漫谈深度学习驱动的广告推荐技术发展周期2021Framework Algorithm Co-EvolutionThe success of DL has driven the evolution of the frameworkALGORITHMFRAMEWORKThe Hardware Lottery.Sara Hooker 2020屠龙少年与龙：漫谈深度学习驱动的广告推荐技术发展周期2021Design for ProductionFrameworks like Tensorflow/PyTorch work very well

4、in researchBut how does a“Framework”in Internet industry(Search Engine/Recommendation/Ads)look like?|min!(!,)Sample generation andfeature extraction fromstreaming data can be TB scale forCTR modelsIncremental updatesOptimization may failRepeatedexperimentationCloud NativeLimited ResourceVersion Cont

5、rolInference uses!()onlyIndustrial DL pipelineThe success of DL has driven the evolution of the industrial pipelineSampleGenerationTrainingServingXDL:An Industrial Deep Learning Framework for High-dimensional Sparse Data.Jiang et al.DLP-KDD 2019DCAF:A Dynamic Computation Allocation Framework for Onl

6、ine Serving System.Jiang et al.DLP-KDD 2020What Do We Need for Industrial Machine Learning Systems?Bernoulli,A Streaming System with Structured Designs.Luo et al.DLP-KDD 2021在线算力效能技术体系阿里定向广告 2020https:/ Talk:BernoulliXDLXDLDynaDynamicmic ComputationComputationAllocationAllocation FrameworkFrameworkT

7、hese ML System Papers are published on DLP-KDD:The Workshop on Deep Learning Practice for High-Dimensional Sparse Data with KDD#2 2DesignDesign ofof thethe StreamingStreaming SystemSystemSample Generation for CTR modelsPossible designs of training pipeline:How to generate？Re-generating samples can e

8、nable faster experimentationsThe Hardware Lottery.Sara Hooker 2020屠龙少年与龙：漫谈深度学习驱动的广告推荐技术发展周期2021FeatureGenerationXDL-BlazeXpctrResult SelectionPage view&User feedbackXDLyOfflineFeatureGenerationXFeatureKV-StoreModel updateOnline prediction serviceExperimentation driven designSample ExperimentationFe

9、ature ExperimentationModel ExperimentationStrong demand for experimentationswith Limited Resource BudgetBernoulli:Streaming Sample FactorySamples and features come as a flow,Training Engine is the final consumerUse Blink(Internal version of Flink)as infrastructureWhat Do We Need for Industrial Machi

10、ne Learning Systems?Bernoulli,A Streaming System with Structured Designs.Luo et al.DLP-KDD 2021Streaming Sample GenerationBlink:The Engine to merge data from different sourcesWindow Join:Trade-off between latency and success rateWhat Do We Need for Industrial Machine Learning Systems?Bernoulli,A Str

11、eaming System with Structured Designs.Luo et al.DLP-KDD 2021Feature Generation using Blink UDFFeatureKV-StoreFeatureFetchFeatureParsingUser_ID User Behavior SequenceUser_ID User Profile FeatureItem_ID Item FeatureCombine FeatureSampleAssembleRemote Procedure CallUser real-time FeatureWindow JoinLabe

12、lStreaming-Batch UnificationHow to perform Sample-Feature K-V join?What Do We Need for Industrial Machine Learning Systems?Bernoulli,A Streaming System with Structured Designs.Luo et al.DLP-KDD 2021StreamingBatchJoin as KV query via RPCReal-Time processingGlobal Join via Map-ReduceWait until everyth

13、ing finishUnified viaSample ReplayStructured data pipelineSample pool is used to cache intermediate representationsDifferent sample streams can share common featuresStructured data pipelineSamples have structure:One user usually viewed several itemsCommon features could be compressedModular training

14、Sample stream and embeddings can be reused by different tasks#3 3UseUse CasesCasesCase Study:Data FusionWith Bernoulli,we can easily fuse data from different business scenariosWhat Do We Need for Industrial Machine Learning Systems?Bernoulli,A Streaming System with Structured Designs.Luo et al.DLP-K

15、DD 2021One Model to Serve All:Star Topology Adaptive Recommender for Multi-Domain CTR Prediction.Sheng et al.CIKM 2021Case Study:Near-line RankingBy near-line ranking using Blink,we can rank tens of thousands of itemsTruncation is avoidedTruncation-Free Matching System for Display Advertising at Ali

16、baba.Li et al.DLP-KDD 2021Case Study:Near-line RankingBy removing hard time constrains,inference efficiency can be largely improvedUseful for pre-ranking stageTruncation-Free Matching System for Display Advertising at Alibaba.Li et al.DLP-KDD 2021Online User requestFeatureGenerationXDL-BlazeXpctrRes

17、ult SelectionPage view&User feedbackXDL-BlazeOfflineFeatureGenerationXFeatureKV-StoreOnline prediction serviceXDLNear-line prediction servicepctrNear-lineResult SelectionNear-line requestResultStoreage20msIn minuates#4 4SummarySummarySummary of BernoulliExperimentation DrivenStructured data pipelineModular TrainingNear-line Ranking20212021-1212-1010THANKSContact:Contact:biye.jbyalibababiye.jbyalibaba-