《在 Apache Flink 中使用 GPU 来完成机器学习任务.pdf》由会员分享,可在线阅读,更多相关《在 Apache Flink 中使用 GPU 来完成机器学习任务.pdf(40页珍藏版)》请在三个皮匠报告上搜索。
1、在Apache Flink中使用GPU来完成机器学习任务 郭旸泽阿里巴巴/ 开发 1 陈戊超阿里巴巴/ 开发 01 Background 2 Apache Flink An unified batch and stream processing engine. Widely used in recommendation, ETL, real-time risk management and other scenarios. 3 Machine learning workflow Feature Engineering Model Training Model Inference Model S
2、erving Analysis Flink Flink TensorFlow TensorFlow Serving TensorFlow 4 Problems Users do feature engineering, model training and model prediction with two framework. Distributed programs often run in clusters but its not friendly to use TensorFlow for distributed training to determine IP and port fi
3、rst. 5 Goal Feature Engineering Model Training Model Inference Model Serving Analysis Flink Flink TensorFlow Serving Flink Flink 6 What we need? Add GPU support in Flink. Integrates Flink with TensorFlow to run machine learning tasks and support using GPU. 7 02 External Resource Framework 8 Motivation Supporting the Machine Learning scenarios is one of Flinks roadmap targets. Machine Learning mode