1、 2022,Amazon Web Services,Inc.or its Affiliates.王敏捷资深应用科学家亚马逊云科技上海人工智能研究院开源图深度学习框架的机遇与挑战 2022,Amazon Web Services,Inc.or its Affiliates.图数据无处不在药物和分子结构用户产品交互网络社交网络知识图谱 2022,Amazon Web Services,Inc.or its Affiliates.图机器学习任务节点预测例：分析一个用户的兴趣爱好链接预测例：判断一个交互行为的类型例：知识图谱补全，推荐系统社区（子图）预测例：检测是否存在可疑的金融欺诈行为图性质预测例：

2、预测分子或化合物性质例：图生成模型 2022,Amazon Web Services,Inc.or its Affiliates.图+深度学习=图神经网络(GNN)用于学习点、边或者整张图的向量表示的一类深度神经网络图神经网络分类器 2022,Amazon Web Services,Inc.or its Affiliates.图神经网络基于消息传递消息函数更新函数累和函数 2022,Amazon Web Services,Inc.or its Affiliates.图神经网络有多火 2022,Amazon Web Services,Inc.or its Affiliates.Graph Neu

3、ral Networks are the next BIG thing!re:MARS 2022 keynote by Swami Sivasubramanian 2022,Amazon Web Services,Inc.or its Affiliates.Deep Graph Library(DGL)面向图结构数据的专用深度学习框架。2018年12月在Neurips大会上宣布开源。开发团队最初主要来自NYU和NYU Shanghai，由张峥教授发起。目前主要开发团队为张峥教授带领的亚马逊云科技上海人工智能研究院。项目上线初就获得广泛关注和好评。Github Stars:9.8K,Forks:

4、2.3K,贡献者：206 DGL论文引用数 600+在学界，DGL是全球领先的图深度学习框架之一；在业界，DGL在使用率上更是全面领先。2022,Amazon Web Services,Inc.or its Affiliates.2022,Amazon Web Services,Inc.or its Affiliates.DGL开源社区建设广泛的开源合作伙伴每月定期组织用户群分享会。邀请学界和业界的研究者分享图神经网络的最新成果。在学术顶会上举办DGL手把手教程(GTC19,KDD19,WWW20,KDD20,GTC20,WSDM21)（所有材料都公开在 https:/ Web Service

5、s,Inc.or its Affiliates.DGL知识图谱嵌入复杂图GNN评测标准结构化自然语言处理生命科学数据库可视化 2022,Amazon Web Services,Inc.or its Affiliates.201820192020First prototypeDevelopment startedV0.2Sampling APIsV0.3Fused message passingMulti-GPU/-coreV0.3.1NN modulesDGL-LifeSciV0.4Heterogeneous graphDGL-KEV0.4.3TF supportV0.1(NeurIPS18)

6、V0.5Distributed TrainingUser guideNew API doc20212022V0.7GPU SamplingCPU kernel updateV0.6GNN Model PushDGL三年开发历程V0.8Enhanced sampling pipeline,heterograph,data transforms 2022,Amazon Web Services,Inc.or its Affiliates.易用性大规模图高性能开源图机器学习系统的核心挑战 2022,Amazon Web Services,Inc.or its Affiliates.易用性+高性能图神

7、经网络入门门槛较高编写高效代码不容易Message passing in three stagesMessage creation:Message aggregation:Feature update:2022,Amazon Web Services,Inc.or its Affiliates.Graph Attention Network(GAT)User-defined Function(UDF)Specialized GNNPrimitives Intuitive The system converts the irregular-shaped graph computation int

8、o fixed-shaped tensor computation by data duplication,sharding,etc.(Less efficient)Suitable for quick prototyping Each primitive directly maps to a low-level CUDA kernel(Very efficient)Less intuitive Suitable for performance critical scenariosPerformance gap can be 10 x 100 x!2022,Amazon Web Service

9、s,Inc.or its Affiliates.Graphiler:Optimizing GNN with Message Passing Data Flow GraphCompile!Performance FlexibilityUDFPrimitives利用编译器对用户代码进行无缝转换。2022,Amazon Web Services,Inc.or its Affiliates.OptimizationsPerform program optimization by pattern substitution.Two broadly applicable pattern substituti

10、on rules:broadcast reordering and broadcast fusionUnify the optimization space for homogeneous and heterogeneous GNNs.Check out our paper(MLSys22)for more details!MP-DFG of GAT 2022,Amazon Web Services,Inc.or its Affiliates.大规模图学术圈越来越关注大规模图数据工业界图在百亿甚至千亿量级 2022,Amazon Web Services,Inc.or its Affiliat

11、es.巨图训练基于子图采样GNN的小批次训练（mini-batch training）基于子图采样1.对目标节点随机选取部分邻居节点，并迭代拓展。2.抽取采样的边形成子图。3.抽取子图特征。4.在子图上训练网络并更新参数。5.重复步骤 1直至收敛 2022,Amazon Web Services,Inc.or its Affiliates.单GPU训练流程Subgraph ExtractionGraph structureNode/edge featuresSubgraph TransformationFeature FetchingGNNGPUCPU数据拷贝成为训练瓶颈 2022,Amazo

12、n Web Services,Inc.or its Affiliates.单GPU训练流程(v0.8)Subgraph ExtractionGraph structureNode/edge featuresSubgraph TransformationFeature FetchingGNNGPUCPU but visible to GPU使用CUDA UVA技术利用GPU加速子图采样Pre-fetch features of next mini-batch 2022,Amazon Web Services,Inc.or its Affiliates.单GPU训练流程(v0.8)用户只需添加几行

13、代码在ogbn-papers100M训练GraphSAGE有将近4x性能提升 2022,Amazon Web Services,Inc.or its Affiliates.多GPU训练对图进行分割并存储在多块GPU的内存中。利用多GPU进行并行采样和训练。如何设计高效的多GPU采样算法？如何利用多GPU间的高速带宽？(Paper under submission)2022,Amazon Web Services,Inc.or its Affiliates.训练成本MAG240M：图结构30GB，节点特征200GB。CPU+GPU混合训练：g4dn.metal,384GB CPU RAM,$7.

14、824/hr全GPU训练：4x g4dn.metal,32x T4 GPU,$31.296/hrWeb graph:4.66B web pages(10 x!)2022,Amazon Web Services,Inc.or its Affiliates.训练成本GNN模型推理代价也很高。2022,Amazon Web Services,Inc.or its Affiliates.GNN推理 2022,Amazon Web Services,Inc.or its Affiliates.GNN推理实现高效的GNN推理需要大幅修改原生模型实现。2022,Amazon Web Services,Inc

15、.or its Affiliates.全自动GNN推理通过编译手段分析用户模型并对用户模型进行改写。自动生成高效逐层推理代码。自动搜索推理超参数，自适应底层硬件优化推理速度。(Paper under submission)2022,Amazon Web Services,Inc.or its Affiliates.总结DGL作为全球领先的开源图神经网络系统的技术特点开源社区介绍图神经网络系统仍然面临着诸如易用性、高性能和大规模图方面的挑战。在编译、分布式等领域的最新研究成果。2022,Amazon Web Services,Inc.or its Affiliates.欢迎使用并贡献DGL用户论坛，Slack，微信群，知乎专栏或者加入我们！实习岗位常年开放！cn-ai- 2022,Amazon Web Services,Inc.or its Affiliates.谢谢！