1、In the last 5 years,we have witnessed worldwide efforts to develop innovative integration solutions that respond to the needs of power-efficient high-performance computing at cost compatible with the semiconductor industry.Chiplet enable the integration of specialized components,such as high-speed i

2、nterconnects,memory,and accelerators,leading to improved system-level performance and efficiency.Fueled by the fast adoption of generative AI,2.5D/3D architectures,the industry has geared up to chiplet adoption for years and has step by step removed the technical obstacles.Its not only about increas

3、ing computing performance in making CPU/GPU closer to the HBM memory through a silicon interposer,this has also been already available on the market for more than 10 years,but offering flexibility,modularity and re-use in design to achieve the most performant system.This presentation will make a ret

4、rospective of the emergence of chiplet in datacenters and consumer markets and draw the outlines of its widest adoption in the coming years.Why chiplet will transform the semiconductor ecosystem from design to packaging?Emilie JOLIVET,More Moore business line director,Yole Group Why chiplet will tra

5、nsform the semiconductor ecosystem from design to packaging?SUSTAINABLE SCALABLE COMPUTATIONAL INFRASTRUCTUREOPEN CHIPLETThe explosion of generative AI applications,such as ChatGPT and Dall-E,combines the trend of exponential model size growth with massive acceleration of user demand.As such,the com

6、puting hardware requirements for these generative AI applications are substantial,with an acute impact to GPUs.For training on GPT-3 with 175 billion parameters,for example,we estimate that between 6,000 and 8,000 A100 GPUs would have required up to a month to complete.There are dozens of other LLMs

7、 being trained and deployed per year,a number that will only grow as the potential of generative AI achieves wider realization.Hardware implications of generative AI3OCP Summit Regional 2024|Yole Group|Chiplet|Emilie JOLIVET Courtesy of NvidiaTraining(across many servers)Previous cloud-based AIGener

8、ative AI eraserverCPUCPUGPUGPUGPUGPUserverCPUCPUserverCPUCPUGPUGPUGPUGPUGPUGPUGPUGPUserverCPUCPUGPUGPUGPUGPUGPUGPUGPUGPUInference(across many servers)DATACENTER AI PROCESSORS TIMELINE LEADING COMPETITOR FOCUS200024202320252026InferentiaIntel acquires Habana for$2BGaudi 1M&AComm

9、ercial launchesGPU A100GPU H100GPU T4GPU V100InvestmentsMicrosoft invests$1B in OpenAIMicrosoft invests$10B in OpenAIMAIA 100OpenAI to launch its own chip?TrainiumInferentia 2Trainium 2TPU v2TPU v3TPU v4TPU v5GPU MI300GPU MI250GPU MI100Kunlun 1Gaudi 2Gaudi 3Alibaba founds T-headTesla DojoMTIA v2MTIA

10、 v1Zixiao v1Hanguang 8004GPU B100Kunlun IIAscend 910Ascend 910BOCP Summit Regional 2024|Yole Group|Chiplet|Emilie JOLIVET The massive growth that the datacenter GPU and AI ASIC market experienced in 2023(167%YoY)is expected to continue in 2024,before stabilizing in the year following.We expect this

11、stabilization as the number of companies able to massively buy GPUs and AI ASIC is limited,and as the life cycle of these components is also growing on average.The total market is expected to reach more than$150B in 2025 and more than$230B in 2029.It represents a CAGR23-29 of 29%.Datacenter GPU and

12、AI ASIC forecast5OCP Summit Regional 2024|Yole Group|Chiplet|Emilie JOLIVET Datacenter GPU and AI ASIC revenue forecast in B$Nvidia H100-CoWoS-S technology from TSMC6OCP Summit Regional 2024|Yole Group|Chiplet|Emilie JOLIVET Nvidia H100 package top viewSource:SPR23722 Yole System Plus,2023NVIDIA H10

13、0 GPU with HBM Cross Section viewSource:SPR23722 Yole System Plus,2023 6 HBMs6 HBMsHBM2e or HBM3 HBM2e or HBM3 optionoptionSK SK hynixhynixSamsungSamsung GPUGPUTSMC 4NTSMC 4N 80B transistors80B transistorsNVIDIA announced its Hopper Tensor Core GPU in 2022.It uses a Si Interposer for its package to

14、interconnect the GPU(4N from TSMC)with 6 HBMs leveraging CoWoS-S technology from TSMC.It comes in both HBM2e and HBM3 configurations upto 80GB and is the first ever product on the market to use HBM3 offering two times more DRAM bandwidth than its predecessor A100.In addition,the Hopper GPU is paired

15、 with the Grace CPU using NVIDIAs ultra-fast chip-to-chip interconnect,delivering 900GB/s of bandwidth,7X faster than PCIe Gen5.This innovative design will deliver up to ten times higher performance for applications running terabytes of data for the HPC,AI,and gaming markets.Nvidia has also designed

16、 A800 and H800 for the Chinese market with reduced capabilities of A100 and H100 also leveraging CoWoS-S from TSMC.DieINTERPOSERIC SubstrateHBMCoWoS package architectureYole SystemPlus tear down available on this productSi InterposerGoogle collaborated with BroadcomBroadcom for its TPU AI accelerato

17、r ASIC for neural network machine learning using Googles Tensor Flow software.TPUs are designed for a high volume of low precision computation and these ASICs are mounted in a heatsink assembly which can fit within a data center rack.Introduced in 2021,TPUvTPUv4 4 ASICASIC is based on TSMCTSMC 7 7nm

18、nm node which are interconnected to 4 4 HBMHBM2 2 (with a total memory capacity of 32GB)using TSMCsTSMCs CoWoSCoWoS-S S packagingpackaging technologytechnology.The die size of TPUv4 is 400 mm2 and the HMB bandwidth is 1228 GB/s.A single TPUv4 pod has 10 x interconnect bandwidth per chip scale compar

19、ed to any other networking technology and according to Google is 1.2x-1.7x faster than NVIDIA A100.Announced in 2023,TPUvTPUv5 5 is the successor of TPUv4,also utilizing TSMCs process node(exact node unstated).The ASIC is interconnected to 6 6 HBMHBM2 2e e with a total capacity of 95GB and having a

20、bandwidth of 2765 GB/s.The die size of TPUv5 is 325 mm2 and is also assumed to use TSMCsTSMCs CoWoSCoWoS-S S packagingpackaging technologytechnology.The TPUv5 can train large-language models 2.8 times faster than TPUv4.TPUv5 package schematic is not yet available.GOOGLE TENSOR PROCESSING UNIT(TPU)TP

21、Uv4&TPUv5 AI accelerator ASIC in collaboration with BroadcomGoogle TPUv4 package top viewSource:Google4 HBMs4 HBMsHBM2HBM21228 GB/s1228 GB/sSK SK hynixhynixSamsungSamsung ProcessorProcessorTSMC 7nmTSMC 7nmSingle dieSingle dieDieINTERPOSERIC SubstrateHBMCoWoS-S package architecture7OCP Summit Regiona

22、l 2024|Yole Group|Chiplet|Emilie JOLIVET HBM GenerationHBMHBM2HBM2EHBM3HBM3E/HBM3P(1)HBM4/HBMNext(2)(expected)Players with market productsPlayers with market productsYear of first product releaseYear of first product release201420182026Typical number of dies per stackTypical number

23、of dies per stack(main packaging approach)(main packaging approach)4Hi(TSV&Microbumps)4-8Hi(TSV&Microbumps)4-8Hi(TSV&Microbumps)8-12Hi(TSV&Microbumps)8-12Hi(3)(TSV&Microbumps)12-16Hi(Cu-Cu hybrid bonding for 16Hi)Max capacity per stackMax capacity per stack1GB4-8GB8-16GB16-24GB24-36GB36-64GBDie dens

24、ityDie density(Typical process)(Typical process)2Gb(2x)8-16Gb(2y,2z)16Gb(1y,1z)(4)16Gb(1z)24Gb(1a,1b/1)(5)24-32Gb(1b/1,1c/1)Max data rateMax data rate1Gbps2-2.4Gbps3.2-3.6Gbps5.6-6.4Gbps8.0-9.8Gbps 9GbpsEffective Bus WidthEffective Bus Width1,0241,0241,0241,0241,0242,048Max bandwidth per stackMax ba

25、ndwidth per stack128GB/s205-307GB/s460GB/s819GB/s1.2TB/s 2TB/sHBM PRODUCT DEVELOPMENT OVERVIEWFlashboltFlareboltAquaboltAquabolt-XL(PIM)IceboltShineboltNotesNotes:(1)HBM3P is Samsungs name for HBM3E;(2)HBMNext is Microns name for HBM4,while its HBM3E is sometimes referred to as HBM3 Gen 2.(3)HBM3 te

26、chnically supports 16Hi stacks,but so far,no manufacturer is using it.(4)Different from Samsung and SK hynix,Micron started manufacturing HBM2E with 1z DRAM.(5)Micron and SK hynix are expected to implement HBM3E with 1/1b hynix)process,while Samsung is likely to start from the 1a process.Major updat

27、e!8OCP Summit Regional 2024|Yole Group|Chiplet|Emilie JOLIVET$2.0$5.4$14.3$20.5$26.4$31.6$37.1$41.80%10%20%30%$-$20$40$60200222023202420252026202720282029HBM revenue($M)HBM share of DRAM market(%)High Bandwidth Memory A market driven by generative AI9OCP Summit Regional 2024|Yole Group|Ch

28、iplet|Emilie JOLIVET CAGR23-2940%Revenue($B)Revenue($B)($B)Yole Group March 202410WHAT COUNTS AS A CHIPLET?The answer is Disaggregation and DuplicationSOC packageSOC dieSOC packageComputedieI/OdieSOC packageComputedieI/OdieCachedieGfxdieSOC packageSOC dieSOC packageSOC dieSOC dieDisaggregation:the S

29、oC monolithic die is partitioned in smaller chips with different functions,then interconnected in the same package.Duplication:two or more SoC monolithic dies are interconnected in the same package forming a bigger SoC.AMD RyzenApple M1 UltraIntel Meteor LakeMonolithic SoCModularized SoC(Chiplet)Adv

30、anced PackagingHeterogeneous IntegrationMonolithic SoC2x or more Monolithic SoC using Advanced Packaging PlatformChipletChipletOCP Summit Regional 2024|Yole Group|Chiplet|Emilie JOLIVET 11Chiplets add costChiplets add costTotal area of silicon larger to enable die-to-die interconnectDie-to-die inter

31、connect scheme,advanced packaging techniquesChiplets reduce costChiplets reduce costSmaller individual dies have higher fab yield(save fab cost)Opportunity to use lower cost nodes for certain IC functionsOpportunity to create new products by mixing dies already designed(save design cost)ECONOMICS OF

32、 CHIPLETS286 sq mm467 sq mm182 sq mm0050060000500600700Cost per Unit($)Monolithic Die Area(mm2)Cost per unit by silicon areamonolithic mfg costchiplet mfg costmonolithic all-in cost(1M)chiplet all-in cost(1M)critical Si area-mfg costcritical Si area-all-in costManufacturing cos

33、t cross-over:where manufacturing cost of chiplet approach is equal to monolithic approachAll-in cost cross-over:where cost of chiplet approach is equal to monolithic approach,with design cost and unit volumes factoredMonolithic more expensiveChiplet more expensiveMonolithic more expensiveChiplet mor

34、e expensiveOCP Summit Regional 2024|Yole Group|Chiplet|Emilie JOLIVET 12EXAMPLES OF CHIPLET-BASED PRODUCTSCommercialized&announced products12CompanyStrategyProducts*Chiplet and heterogeneous integration is focus of IDM 2.0 strategy.AMD has now shipped several generations of PC and server processors

35、using its chiplet interconnect platform Infinity Fabric(IF).AMD uses organic substrate to integrate chiplets.Apple recently released its M1 Ultra product which is a combination of two M1 Max dies interconnected using TSMCs LSI technology.Amazon launched their Graviton3 data center processor during t

36、heir AWS in 2021.It is a chiplet product with seven dies and a total of 55 billion transistors.Dojo D1 is Teslas in-house supercomputer for Machine Learning.25 D1 chiplets packaged together.BR100 GPU consists of 2 chiplets(dual die GPU)processed on TSMCs 7nm node.Sapphire Rapids(EMIB)Launched Q1 202

37、3Ponte Vecchio(Co-EMIB)Available 2023Meteor Lake(Foveros)Available 2023Graviton3(with Intels EMIB technology)Launched 2022AMD Ryzen SeriesRyzen 3000 Launched 2019AMD Radeon Instinct MI200 Series Launched 20223D V-Cache in AMD Ryzen and EPYC Gen 3 Launched 2022 M1 Ultra(with TSMC InFO_LSI)-launched 2

38、022*Non-exhaustive listFoveros DirectAvailable 2024AMD Radeon RX7900 Launched 2022Arrow Lake(Foveros)Available 2024TESLA DOJO(TSMCs integrated fan-out system on wafer(InFO_SoW)package)Launched 2022Biren Technology BR100(with TSMC CoWoS)Launched 2022OCP Summit Regional 2024|Yole Group|Chiplet|Emilie

39、JOLIVET HIGH-BANDWIDTH MEMORY FOR ADVANCED(X)PUsDevice structure and supply chain xPU2.5D INTERPOSERIC SubstratePCB BoardHBMFinal Assembly:OSAT|Foundry|IDMThe final assembly(HBM,GPU,interposer,interposer on PCB,passives assembly and BGA balls)is performed by OSAT and foundry(possibly IDM).HBM Packag

40、ing:IDM|Memory SupplierThe HBM stack(memory dies,logic die)is made by a memory manufacturer.xPU Supplier:Foundry|IDMThe CPU or GPU die is manufactured by the wafer foundry.2.5D Interposer:Foundry|IDMThe Interposer die is manufactured by the wafer foundry.IC Substrate:Substrate SupplierThe PCB packag

41、e substrate is made by a substrate maker.Source:“High-End Performance Packaging 2023”Report13OCP Summit Regional 2024|Yole Group|Chiplet|Emilie JOLIVET HIGHHIGH-END PERFORMANCE PACKAGING:ALL PLATFORMSEND PERFORMANCE PACKAGING:ALL PLATFORMSYoles classification for 2.5D and 3DHigh-End Performance Pack

42、agingUHD FOSi Interposer14Molded die+Thin film RDL+IC substrate TSMC:InFO_(X)CoWoS-RASE:FoCoSSPIL:FO-MCMJCET:XDFOIAmkor:SWIFTMold Interposer3D SoCD2W or W2W Hybrid BondingTSMC:SoIC (3D Fabric)Intel:Foveros DirectSamsung:X-CubeTSMC:InFO-LSICoWoS-LSamsung:I-CubeEASE:FoCoS-BridgeSPIL:FOEBJCET:XDFOI-EBA

43、mkor:S-ConnectMicrobumps&TSV/HBW2W Hybrid bondingYMTC3 DSHBM3D MemorySamsung,SK hynix&Micron3D NAND StackMold compound+Thin film RDL+IC substrate Intel:Sapphire RapidsEmbedded in IC substrateEmbedded Si BridgeIntel:FoverosIntel:Ponte VecchioEmbedded in mold compoundSi BridgeEMIBActiveCo-EMIBInterpos

44、er die+TSVs+Microbumps+FC Bumps+IC substrate TSMC:CoWoS-SSamsung:H-CubeI-CubeSUMCPassive2.5D3D3D DRAMSamsung,SK hynix&MicronSource:High-End Performance Packaging 2024OCP Summit Regional 2024|Yole Group|Chiplet|Emilie JOLIVET GPU AND AI ASIC ADVANCED PACKAGING FORECASTRevenue forecast for 2.5D packag

45、ing,by technology,in$M,2023-2026-2029$107M,88%$4M,3%$2M,2%$7M,6%$1M,1%2023$169M,61%$26M,10%$53M,19%$8M,3%$20M,7%2026$195M,59%$32M,9%$71M,21%$12M,4%$24M,7%2029$121M$276M$333MThe revenue associated to the 2.5D/3D packaging are expected to grow from$121 in 2023 up to$333M in 2029,which represents a CAG

46、R23-29 of 18%.CAGR23-26=32%CAGR26-29=7%15OCP Summit Regional 2024|Yole Group|Chiplet|Emilie JOLIVET GLASS CORE SUBSTRATES A Better Long-term Option For The Data CentersGlass core substrates30%60%15%70%65%Organic core substrates70%40%85%30%35%InvestmentsThermal managementApplication rangeCompatibilit

47、y for AI acceleratorsInterconnect densityMain criteria for Data centersFor the organic core substrates,they maintain cost-effectiveness and versatility across various applications but require additional requirements such as form factor,higher layer count,improved signal routing,and good thermal mana

48、gement.The industry of organic substrates is expected to keep pace with the requirements for form factor and layer count,especially from ASP and yield perspectives.One approach might take the advanced substrates to the next level is about adopting glass as a new core material.Glass core substrates c

49、an offer smaller L/S,fewer layer count,and superior thermal conductivity,making them a promising candidate to resolve the thermal management issues,expected in the next-gen high-performance computing,data centers,and AI products.Although the supply chain is currently limited and under consolidation

50、with major challenges in terms of high yield,glass handling and processing.16OCP Summit Regional 2024|Yole Group|Chiplet|Emilie JOLIVET 17AS A CONCLUSIONAS A CONCLUSIONHigher yield Optimized cost02Finer bump or pad pitch Higher density03Optimization node per chiplet04Partitioned die more dies per wa

51、fer 01Faster time-to-market05Chiplet&Heterogeneous Integration AdoptionHow to realize chiplet&heterogeneous integration?An Advanced Packaging platform such as 2.5D/3D,fan-out or high-density flip-chip is needed.FC on organic substrateFC on organic with RDL(UHD FO)Mold Interposer+Si BridgesSi Interpo

52、serSi bridgesD2W/W2W hybrid bonding(3D SoC)Source:YINTR23336-High-end Performance Packaging 2023OCP Summit Regional 2024|Yole Group|Chiplet|Emilie JOLIVET Thank you!Copyright Yole Group 2024.All rights reserved.|YOLE GROUP KEY FIGURES2525Years in the semiconductor Years in the semiconductor industry

53、industry900+900+ProductsProducts150+150+Trade shows&Trade shows&conferencesconferences5,000+5,000+Interactions with the industryInteractions with the industry180+180+Worldwide collaboratorsWorldwide collaborators15,000+15,000+Followers on LinkedinFollowers on Linkedin180,000+180,000+Contacts in our

54、databaseContacts in our database10,000+10,000+Mentions in MediaMentions in Media(EETIMES,Forbes,Thomson Reuters)(EETIMES,Forbes,Thomson Reuters)19Copyright Yole Group 2024.All rights reserved.|20We provide industrial companies,financial investors and R&D organizations,with market research and market

55、ing analysis,technology,supply chain and cost analysis,as well as performance evaluation,to help our customers in their decision-making about their future business and manufacturing strategy in the semiconductor,photonic and electronic sectors.YOLE GROUPvvvMarket,technology,and strategy consultingM&

56、A,Due Diligence and evaluation of companiesTechnology,process&cost analysisTeardown and reverse engineeringComparative analysisTest of electric and electro-optical performance of devicesComparison of performances and related technical choice3 CORE ACTIVITIES BASED ON DEEP SYNERGIES3 CORE ACTIVITIES

57、BASED ON DEEP SYNERGIESMarket&TechnologyTeardown Reverse engineering and costingPerformance analysisCopyright Yole Group 2024.All rights reserved.|13 different monitors 13 different monitors quarterly updatedquarterly updated115+reports per year115+reports per year190 custom projects per year190 cus

58、tom projects per year315+Single Teardowns Tracks per 315+Single Teardowns Tracks per year year-Daily updatesDaily updates21OUR PRODUCTS&SERVICESQUARTLERY QUARTLERY MONITORSMONITORSINSIGHTINSIGHTQuarterly analysis with market data and technology trends in units,value at wafer level Direct access to t

59、he analyst(except for Wafer Data Monitor)FORMATFORMATExcel files with dataPDF files with analyses graphs and key factsWeb accessTOPICSTOPICSAdvanced PackagingCompound semiconductor:RF electronics,power electronics,optoelectronicComputing:Microcontroller,ProcessorMemory:DRAM,NANDSemiconductor Test:eq

60、uipment,consumables,wafer fab,subsystemsWafer DataREPORTSREPORTSINSIGHTINSIGHTYearly published reports Market,technology and strategy analysis Reverse costing and reverse engineeringPerformance analysisFORMATFORMATPDF files with analysesExcel files with graphics and dataWeb accessTOPICSTOPICSBattery

61、,Compound Semiconductor,Power Electronics,Radio FrequencyComputing&Software,Memory,Semiconductor PackagingDisplay,Imaging,Photonics and Lighting,Sensing and ActuatingSemiconductor Equipment and MaterialsGlobal Semiconductor TrendsCUSTOM SERVICESCUSTOM SERVICESINSIGHTINSIGHTSpecific and dedicated pro

62、jectsStrategic,financial,technical,supply chain,market and other semiconductor-related fieldsReverse costing&reverse engineeringFORMATFORMATDirect work with the analyst team PDF files with analysesExcel files with graphics and dataTYPES OF ANALYSISTYPES OF ANALYSISCompany profilingCost model/designF

63、inance project:DD,M&A buy side,M&A sell sideMarket analysisMarketing analysis,market segmentationPatent analysisStrategy analysisTechnology analysisWorkshopTEARDOWN TRACKSTEARDOWN TRACKSINSIGHTINSIGHTTeardowns of phones,smart home,wearables and automotive modules and systemsBill-of-MaterialsBlock di

64、agramsInteractive zoomingFORMATFORMATWeb accessPDF and Excel filesHigh-resolution photosTOPICSTOPICSConsumer:Phones,smart home,wearables,tablet,computing and gamingAutomotive:ADAS,infotainment,telematics,electrification and other ECUsTelecom:Baseband unit,active antenna unit,CPE and othersCopyright

65、Yole Group 2024.All rights reserved.|22OUR EXPERTISESEMICONDUCTOR INDUSTRYImagingBatteryDisplayElectronic SystemsComputing and SoftwareCompound SemiconductorGlobal Semiconductor TrendsMemoryPhotonics and LightingPower ElectronicsRadio FrequencySemiconductor EquipmentSemiconductor PackagingSensing an

66、d ActuatingCopyright Yole Group 2024.All rights reserved.|23MOBILE MOBILE&CONSUMER&CONSUMERAUTOMOTIVEAUTOMOTIVE&MOBILITY&MOBILITYTELECOM&TELECOM&INFRASTUCTUREINFRASTUCTUREMEDICALMEDICALDEFENSEDEFENSE&AEROSPACE&AEROSPACEINDUSTRIALINDUSTRIALACROSS THE SEMICONDUCTOR SUPPLY CHAIN&MARKETSAcademic/Academi

67、c/researchresearchDesign&Design&engineeringengineeringMaterial&Material&equipmentequipmentFrontFront-endendmanufacturingmanufacturingBackBack-endendmanufacturingmanufacturingOEMs&OEMs&system system integratorintegratorConsulting&Consulting&financefinanceCopyright Yole Group 2024.All rights reserved.

68、|24YOLE GROUP A WORLDWIDE PRESENCESeoulShanghaiTokyoHsinchuMelakaBostonPhoenixAustinRaleighLyonLyonFrankfurtNanteNantes sLondon180+collaborators in 9 different countriesGeneral terms General terms and conditions and conditions of salesof salesFOLLOW US ONCopyright Yole Group 2024.All rights reserved

69、|25GLOBAL OPERATIONSREPORTS,MONITORS&TRACKSCUSTOM SERVICESCUSTOM SERVICES MARKETINGMARKETING PUBLIC RELATIONS&EXTERNAL PUBLIC RELATIONS&EXTERNAL COMMUNICATIONSCOMMUNICATIONS GENERAL INQUIRIES GENERAL INQUIRIES |+33 4 72 83 01 80NORTH AMERICANORTH AMERICA+1 833 338 4999 EMEAEMEA +49 69 9621 7675 JAPAN,KOREA,REST OF ASIAJAPAN,KOREA,REST OF ASIA +81 3 4405 9204 GREATER CHINA GREATER CHINA +886 979 336 809+86 136 6156 6824 CONTACTS