1、1 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheAn Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheZhiyong Li,Sangjin Kim,Dongseok Im,Donghyeon
2、 Han,and Hoi-Jun Yoozhiyong_likaist.ac.krSemiconductor System Lab.School of EE,KAIST2 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheSuper-Resolution on Mobile Platform Improve User Experience with High Quality
3、Images Expansion of image feature channels and maintain higher image resolution Enhance quality of streaming media/camera shot for better QoS*QoS:Quality of Service720p SR ON!1080p SR ON!3 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision a
4、nd Energy-efficient CacheCharacteristic of SR CNN Large and Non-zero Feature Resolution Maintaining or enlarging feature resolution 9x input image resolution Non-ReLu act.func.remove sparsity zero skipping is impossible Classification CNNSuper-Resolution CNN640 x360 x31920 x1080 x3224x224x31000 x1Bu
5、tterflyTiger Flower*Measured VGG-16,*Measured FSRCNN(Set 5)4 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheHigh Memory Requirement Large Data Transactions Total 11x data transaction than classification CNN HW d
6、esign issue of on-chip cache units and processing unitsExt.MemOn-chipCacheExt.MemOn-chipCacheMore PUMore MemT0T1T2Mem0Mem1Mem2LargeEMALowThoughputReuseReuse11TimesClassification CNN*91MBData Transaction(MB/Frame)20406080Feature Map Weight8.3MBSuper-resolution CNN*Challenge 1)High Memory Requirement5
7、 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheCharacteristic of SR CNN Large Bit Precision for QoS Inefficient high bit data transactions and computations Efficient Processing algorithm&HW is needed x1 x3.8 x1
8、2.3Relative Energy Cost1FXP8 MACFXP16 MAC16b on-chip Mem Access38.69 dBFXP16PrecisionResults on Set5 PSNR18.85 dBFXP8Challenge 2)High Bit Precision1:A.Raha et al.VLSID 20216 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-effic
9、ient CacheFeatures of Proposed SR-SoC 2-Part Optimization with Heterogeneous Hybrid-precisionHeterogeneous Hybrid-PrecisionFor Data Compression and Enhance HeterogeneityHeterogeneous Accel.ArchitectureFor Energy-EfficiencyHierarchical Line Cache SubsystemFor Smaller Cache Footprint and Less EMASW-HW
10、Co-designProcessUnitMicro-Arch.Level CacheUnitHeterogeneous L1 Line Cache For Smaller L1 Cache and low powerProposed Approach7 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheHeterogeneous Hybrid-Precision Archit
11、ecture SW-HW Co-designed Architecture Heterogeneous Hybrid Precision for less data transaction&sparsity High parallelism&Skipping architecture for high computing efficiency Heterogeneous hierarchical cache w/Tiled exec.for high mem efficiencyProposed ApproachOutliersProcess UnitsCaching UnitsAlgorit
12、hmSkippingSkipping Ctrl.UnpredictableTag check=?PsumTagPsumTagPsumTagHigh Parall.Hierarchical$ShortShortShortLong-termLong-termLong-termL2 L1 TiledLineRegular AccessInput Distribution of SRInliersHeterogenietyStream Acc.Acc.1D-PEsDenseSparse0003x3x54 2D-PEsSkip012012idididProblemsLarge&Non-sparseDat
13、a Transactions 1)Large EMA2)Large on-chipMem PowerNon-sparse 16b Data 3)Inefficient HighBit Computation8 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheProposed Heterogeneous Hybrid Precision Divide Data into Sp
14、arsity-biased 2 Groups Thresholding with Probability Density Function(p.d.f.)lower 90%100%non-zero dense FXP8 group and 10%non-zero sparse FP8 grouplog2|X|in FSRCNN*Dataset from Set5,Set14 SR dataset,measurement includes input and intermediate activationLSBMSBOutlierTh=pdf-1(90%)log2(Th)8b represent
15、InliersInliersDense FXP8Sparse FP8SSkipSkip9 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheResult of Heterogeneous Hybrid-Precision Maintaining Quality(0.5dB loss)while Reducing 47%External Memory Access*Datase
16、t from Set5,Set14 SR dataset,measurement includes input and intermediate activation0.620%Outlier Ratio0.0(FXP16)0Data Transaction(%)FXP16=FXP8+FP843%10%30%0.547%5%(FXP8+5%FP8)100PSNR(dB)Activation Precision(bit)3736816S.VLSI1935S.VLSI21TCSVT18This Work*More energy-efficientHigher qualityISSCC2110 of
17、 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheProposed Super-resolution SoC ArchitectureL2 Line$(30KBx4)TLBL2 Line$(30KBx4)TLBL2 Line$(30KBx4)TLBFXPU 1SwitchFXPU 1SwitchFXPU 1SwitchOff-chip MemoryHost PCRISC-VL1
18、I$+D$(4KB)L2 Line$(30KBx4)TLBInterconnection NetworkUARTSPII2CGPIOFXPU 0FXPU 0FXPU 0FXPU 0Input AllocaterL1 Line$(5KB)Weight Spad(4KB)Loop Ctrlr.3x18 WSSystolic PE Array3x18 WSSystolic PE Array6-ch 3x3 WSSystolic PE ArrayAdder TreeAdder TreeAdder Tree.x4FXPU 0SwitchRouterFP2FxPPLUTFPUInputL1$(2KB)Ou
19、tputL1$(0.5KB)IDX$Lane 17Lane 0W SPad(4KB)Inst.$AXI 64-bitx41.FiXed-point Processing Units-For Non-zero FXP8 Convolution-For Short-term Line$2.Floating-point Processing Unit-For Sparse FP8 Convolution-For Short-term Tagged Line$3.Global L2 Line Cache-For top-level threshold biasing-For Long-term Ove
20、rlapped Line$11 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheHeterogeneityMotivations of Heterogeneous Arch.Sparsity Heterogeneity of HHP Data 100%non-zero FXP8 and 10%non-zero FP8 group Inefficient in previou
21、s homogeneous architecture 100%Non-zeroRegular Mem.AccessLarge portion10%Non-zero w/indexUnpredictable Mem.AccessSmall portionDense FXP8Sparse FP812 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheData Compressio
22、n for Non-Sparse SR Previous Mixed-Precision Hardware Divide data into 16b large value group and 8b small value group Low compression ratio with non-zero SR data Additional zero data indexing bits as overhead FP16-GroupFP8-GroupPartial Use of FP8(Ratio:01.0)FP8FP16Raw dataFP16zero data index as over
23、head Non-zerodataFP16+MetadataFP 16+zero data index FP 162 Lee et al.ISSCC 2019;13 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheZero-Skipping HWZero-Gating HW090%30%60%TFLOPS/W101Previous Mixed-Precision Archi
24、tecture Homogeneous Mixed-Precision Accelerator Homogeneous skipping arch.for higher energy-efficiency,BUT Dense low precision group Skipping Overhead dominant(90%of SR data)Sparse high precision group Benefits from small portion(10%)of total dataFP8FP16Sparsity lie here!zFP8/FP16 Cfg.MAC Array Col-
25、bufDMEMCtrlr FP8 ModeSkippingOverheadFP16-GroupFP8-Group2 Lee et al.ISSCC 2019;14 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CachePrevious Mixed-Precision Architecture Homogeneous Mixed-Precision Accelerator Homo
26、geneous skipping arch.for higher energy-efficiency,BUT Dense low precision group Skipping Overhead dominant(90%of SR data)Sparse high precision group Benefits from small portion(10%)of total data2 Lee et al.ISSCC 2019;Zero-Skipping HWZero-Gating HW090%30%60%TFLOPS/W101Sparsity lie here!FP16-GroupFP8
27、-GroupFP8FP16FP8/FP16 Cfg.MAC Array Col-bufDMEMCtrlr FP16 Mode15 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheHeterogeneous Accelerating Architecture Efficient Skipping-and Parallelism-exploit AcceleratorFPUSp
28、arse L1$SIMDALU laneIdx$Inst.$SboradFXPUDense L1$SystolicPE arrayFunc.LUTFXPUDense L1$SystolicPE arrayFunc.LUTFXPUDense L1$SystolicPE arrayFunc.LUTFXPUDense L1$SystolicPE arrayFunc.LUTHeterogenousHigh ParallelismRegular Mem.AccessHigh ThroughputUnpredictable Mem.Access16 of 22HOTCHIPS 2022An Efficie
29、nt High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheProposed High Parallelism FXPU High Parallelism FXPU for Diverse Convolution Kernels3x3 2D PE Array:MAC operationOutput StationarySystolic Data Reuse3x3 OS Systolic ArrayInput TensorAdder Tree
30、Input Allocater.X2X1X0.X2X1X0.X2X1X0.X2X1X0T0 x9Input Allocator w/Dense L1$:Receive input dataReform&feed to PE ArrayAdder Tree w/LUTFor Activation Func.17 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheProposed
31、 Zero-skipping Architecture Zero-skipping FPU for Sparse Convolutions4 3x1 1D PE Lane:Sparse Multiply&Act.Func.operationScoreboard based parallelismSkip Ctrlr.w/Sparse L1$:Receive input data&skip indexSelect weight w/zero indexAccumulator&Reorder L1$Sparse AccumulateScoreboardInput VectorSkip contro
32、llerIdx$01.X7X3.X7X3.X7X3.X7X3V0Sparse AccumulatorReorder bufferLane 1Lane 2Lane 318 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheResults of Heterogeneous Accelerating Arch.*Under grouping threshold of 95%3 J.
33、Lee et.al S.VLSI 2019 Energy-efficiency(mJ/frame)2.38BaselineFXP16ProcessorHybridFP-FXPArch.1.249.6%Normalized Power(%)100BaselineFXP16Processor100HybridFP-FXPArch.46%47%CoreMemoryPreviousSRNPU3PreviousSRNPU319 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC w
34、ith Hybrid-precision and Energy-efficient CacheLevel 2Level 1LRFMain MemProposed Hierarchical Cache Subsystem 2-level Hierarchical Line Cache Tile-based hierarchical cache to reduce cache size&power Multi-core layer fusion to reduce EMA Cached FeatureTotal Feature SpaceFXPU0FXPU1FXPUnLayer fusionTil
35、e-based Hierarchical$20 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheProposed Hierarchical Cache Subsystem Tile-based Execution Fetch small tile of entire input each time Shorter reuse distance between lines x
36、12 smaller on chip cache Level 2Level 1LRFMain MemCached FeatureTotal Feature SpaceFeedfor Conv.PEArrayx12 smaller 21 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheProposed Hierarchical Cache Subsystem 2-level
37、Hierarchical Line Cache Tile-based hierarchical cache to reduce cache size&power Multi-core layer fusion to reduce EMA Level 2Level 1LRFMain MemCached FeatureTotal Feature SpaceFXPU0FXPU1FXPUn22 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-preci
38、sion and Energy-efficient Cache4.394.44.891.8This workPreviousworkExt.Mem/FMemFmem/PEL1$/PEL2$/L1$Ext.Mem/L2$91.8Data Transactions(MB)No fusionPreviouswork91EMA(MB)4.397004.39358ProposedHHP47%4.4014866%ProposedHier.$Subsys.On-chip mem footprint(KB)Result of Hierarchical Cache Subsystem Additionally
39、reduce 66%global memory footprint Reduce 18x L2 cache(Large memory)access4 K.Goetschalckx et al.S.VLSI 2021;4423 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheResult of Hierarchical Cache Subsystem With smaller
40、 cache subsystem 58.4%power With fused layer 61.8%energy Normalized Power(%)100PreviousSR NPU100HybridFP-FXPArch.5354HierarchicalLine Cache41.65446%47%58.4%CoreMemoryEnergy/Frame(mJ)2.38PreviousSR NPUHybridFP-FXPArch.HierarchicalLine Cache1.20.9161.8%49.6%3 J.Lee et.al S.VLSI 2019 3324 of 22HOTCHIPS
41、 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheChip Photography and SummaryFPURISC-VL1 Line Buf.Enc./PLUTFXPU0,1L1 Line Buf.4,0002,500Dec./Aggr.FXPU2,3L1 Line Buf.L1 Line Buf.L2 Line BufferInst.cacheInput$Resolution270p1080p
42、Method540p4kFSRCNNClassSRFramerate107 fps41 fpsGANOMGD86 fpsProcessActivationPrecisionSupplyVoltage(V)65nm FXP8+510%FP81.0Frequency(MHz)200SR AlgorithmFSRCNN/ClassSRFrame Rate*(fps)SR Energy*(mJ/frame)1070.92*x4 FSRCNN on Set5,Set14 Dataset25 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-res
43、olution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient Cache Energy-Efficient Non-sparse High-quality SRCNN Heterogeneous Accelerating w/Hybrid-precision 46%Processing Power&47%Mem Access Power&47%EMA Data Lifetime-aware Hierarchical Line cache 53.7%Mem Access Power&71.8%EMA Concl
44、usionA 0.92 mJ/frame Super-resolution SoC for Resource-limited Mobile Applications26 of 22HOTCHIPS 2022An Efficient High-quality FHD Super-resolution Mobile Accelerator SoC with Hybrid-precision and Energy-efficient CacheThank You!Questions?Feel Free to Contact Me!E-mail:Zhiyong_likaist.ac.kr Zoom Meeting:https:/zoom.us/xxxx(Password:HC_GANPU)Acknowledgement TBD.
sgpjbg002
工作日 8:30 - 17:30
会员动态：
报告分类
新质生产力
ChatGPT
新能源汽车
大模型
Sora
机器人
微短剧
芯片
薪酬
白皮书
低空经济
数据要素
创新药
人工智能
AI产业
信息科技
互联网
消费经济
汽车交通
电商零售
传媒娱乐
医药健康
投资金融
能源环境
地产开发
传统产业
全球化
其它
扫码登录
手机快捷登录
账号登录
