《CUDA Graph 在 TensorFlow 中的应用.pdf》由会员分享,可在线阅读,更多相关《CUDA Graph 在 TensorFlow 中的应用.pdf(28页珍藏版)》请在三个皮匠报告上搜索。
1、Jiajie Yao, Dec 2020 CUDA GRAPH IN TENSORFLOW 2 CONTENT Whats CUDA Graph How to Use CUDA Graph Launch Overhead in TensorFlow Integrate CUDA Graph into TensorFlow Performance 3 WHATS CUDA GRAPH Reduce Launch Overheads What Problem CUDA Graph Solves Time Launch A ABCDE Idle Idle graph launch ABCDE Lau
2、nch B Launch C Launch D 4 WHATS CUDA GRAPH Stream Launch Stream Launch vs Graph Launch https:/ 5 WHATS CUDA GRAPH Graph Launch (Pre-Ampere) Stream Launch vs Graph Launch CUDA Graph Launch cudaGraphLuanch(g1, s1) Stream Queues Grid Management Execution A B Block A0 SM 0 Block A1 SM 1 ABCD C D Grid Co
3、mpletionOther Dependencies https:/ 6 WHATS CUDA GRAPH Graph Launch (Ampere) Stream Launch vs Graph Launch CUDA Graph Launch cudaGraphLuanch(g1, s1) Stream Queues Grid Management Execution A B Block A0 SM 0 Block A1 SM 1 A B C D C D Full Graph Completion Graph Upload Grid Upload Kernel Upload https:/ 7 WHATS CUDA GRAPH Launch overhead comparison (test using empty kernel) A100 GPU * Graph with 32 no