《02基于元数据和配置驱动的 eBay 交易风控 AI 模型管理和部署实践--王兵.pdf》由会员分享，可在线阅读，更多相关《02基于元数据和配置驱动的 eBay 交易风控 AI 模型管理和部署实践--王兵.pdf（30页珍藏版）》请在三个皮匠报告上搜索。

1、Model Spec Driven AI Model Management&Deployment at eBay Payments RiskBing WangeBay Payments&RiskAgendaeBay Payments Risk AI Model Lifecycle and Model SpecUnified Context for Model Training&Model ServingModel Integration&Deployment by Model SpecModel Serving Observability&Monitoring1234eBay Payments

2、 Risk AI Model Lifecycle and Model SpecFeature EngineeringModel TrainingModel DeploymentPerformance ValidationBusiness UsageModel Refresh(Refit)OfflineOnlinePayments Risk AI Model Lifecycle Feature EngineeringModel TrainingModel DeploymentPerformance ValidationBusiness UsageModel Refresh(Refit)Offli

3、neOnlineMetadata in AI Model Lifecycle Raw Features MetadataTraining Dataset Metadata,Pipeline Metadata,Model Object Metadata,Model Service API Metadata,Features Fetching Metadata,Feature Preprocessing Metadata,Model Prediction Metadata,Model Output Post-processing MetadataFeature EngineeringModel T

4、rainingModel DeploymentPerformance ValidationBusiness UsageModel Refresh(Refit)OfflineOnlineRaw Features MetadataTraining Dataset Metadata,Pipeline Metadata,Model Object Metadata,Model Service API Metadata,Features Fetching Metadata,Feature Preprocessing Metadata,Model Prediction Metadata,Model Outp

5、ut Post-processing MetadataMetadata in AI Model Lifecycle Model Spec(Model Specification)Metadata Group-Model SpecModel Spec(Model Specification)Basic Model Information:owners,model type,scenario,refresh frequency,.Feature Fetching:feature name,data source,value type,default value,.Model Object:mode

6、l type,framework and version,parameters,target SLA,Inference Preprocessing&Post-processing:dependent raw features,feature preprocessing expression,model output mapping logics,Multi-models Inference:pipeline definition,model routing definitionMonitoring and Logging:schema definition,metrics,event/tab

7、le information,Unified Context for Model Training&Model ServingModel DeploymentTraining codesModel files(pkl,txt,json,bin)Training OutputsModel Training PipelineModel ServiceModel Application CodesModel files(pkl,txt,json,bin)translationcopyDeployModel DeploymentTraining codesModel files(pkl,txt,jso

8、n,bin)Training OutputsModel Training PipelineModel ServiceModel Application CodesModel files(pkl,txt,json,bin)translationcopyDeploy Much Manual Effort Vulnerable to discrepancy between model training and inference Model IntegrationModel ServiceBusiness Domain ServicedeployFeaturesFetchingModel API C

9、allFeatures&Inference Result MonitoringrequestInference requesttranslationModel IntegrationModel ServiceBusiness Domain ServicedeployFeaturesFetchingModel API CallFeatures&Inference Result MonitoringrequestInference requesttranslation Much Manual Effort Vulnerable to discrepancy between model traini

10、ng and inference Different ContextModel Training(Data Scientists)Model Deploying(Data Engineers)Model Integrating(Domain Engineers)own contextown contextown contextUnified Context Model SpecModel Training(Data Scientists)Model Deploying(Data Engineers)Model Integrating(Domain Engineers)Model SpecMod

11、el Integration&Deployment by Model SpecFeature PreprocessingThe traditional way to move feature preprocessing logics from model training to inference is serializing/deserializing object by pickleFeature Preprocessing*.pklobjectobjectProblem I:Forcing dependency on libraries in different environments

12、Solution in Model Spec:Reproduce preprocessing by Logics RepresentationDumping&SavingLoading&ParsingLogics RepresentationLogics RepresentationFeature PreprocessingProblem II:Data processing performance for singleton inference is not optimalOptimization in Model Spec:concurrency by multiprocessing,st

13、d:threadBatch ProcessingSingleton ProcessingMulti-threadingMulti-processingFeature PreprocessingMove feature preprocessing logics from model training to inference by Model SpecCodes to Representations in Model Spec DumpingParsingLogics RepresentationLogics RepresentationFeature FetchingModel Feature

14、 PreprocessingModel ObjectModel Output Post-processingModel Inference RoutingConfigurations Snapshotting from Model SpecModel Spec StoreFeature Fetching ConfigurationModel Object ConfigurationFeature Preprocessing ConfigurationModel Ouput Post-preprocessing ConfigurationsnapshottingversioningConfigu

15、ration DeploymentConfiguration SyncConfiguration ValidationFeature Fetcher Object BuildingCanary ChangeEvent DroppingConfiguration Deployment in Business Domain Service Model Inference Session BuildingCanary ChangeEvent DroppingConfiguration ValidationConfiguration SyncConfiguration Deployment in Mo

16、del Service Configuration DeploymentConfiguration SyncConfiguration ValidationFeature Fetcher Object BuildingCanary ChangeEvent DroppingConfiguration Deployment in Business Domain Service Model Inference Session BuildingCanary ChangeEvent DroppingConfiguration ValidationConfiguration SyncConfigurati

17、on Deployment in Model Service Metadata and Configuration Driven,Few Code Changes Needed Model Integration&Deployment by Model SpecModel Spec StoreModel Spec LibraryModel Training PipelineDomain Service Model Service Model Spec LibraryModel Spec LibraryRequestModel Inference RequestRead/Update Model

18、 Spec Read/Update Model Spec Read/Update Model Spec Model Serving Observability&Monitoring4The ML Test Score:A Rubric for ML Production Readiness and Technical Debt ReductionEric Breck,Shanqing Cai,Eric Nielsen,Michael Salib,D.Sculley Proceedings of IEEE Big Data(2017)ML System MonitoringML System O

19、bservability&MonitoringModel Output Monitoring:-default result rate-score distributionModel Features Monitoring:-null/empty rate-value distributionModel System Monitoring:-latency-error rateModel Outputs&Features Observability&MonitoringModel Spec StoreApache FlinkEvents Consumer&ProcessorMonitoring MetadataMonitoring MetadataKafka ClusterAggregated EventsNRTMetricsLog EventsHadoopHadoopHDFSLog EventsOffline MetricsMonitoring MetadataThankThank YouYou