1、 APRIL 2020 AI Chips: What They Are and Why They Matter An AI Chips Reference AUTHORS Saif M. Khan Alexander Mann Center for Security and Emerging Technology | 2 Table of Contents Introduction and Summary 3 The Laws of Chip Innovation 7 Transistor Shrinkage: Moores Law 7 Efficiency and Speed Improve

2、ments 8 Increasing Transistor Density Unlocks Improved Designs for Efficiency and Speed 9 Transistor Design is Reaching Fundamental Size Limits 10 The Slowing of Moores Law and the Decline of General-Purpose Chips 10 The Economies of Scale of General-Purpose Chips 10 Costs are Increasing Faster than

3、 the Semiconductor Market 11 The Semiconductor Industrys Growth Rate is Unlikely to Increase 14 Chip Improvements as Moores Law Slows 15 Transistor Improvements Continue, but are Slowing 16 Improved Transistor Density Enables Specialization 18 The AI Chip Zoo 19 AI Chip Types 20 AI Chip Benchmarks 2

4、2 The Value of State-of-the-Art AI Chips 23 The Efficiency of State-of-the-Art AI Chips Translates into Cost-Effectiveness 23 Compute-Intensive AI Algorithms are Bottlenecked by Chip Costs and Speed 26 U.S. and Chinese AI Chips and Implications for National Competitiveness 27 Appendix A: Basics of S

5、emiconductors and Chips 31 Appendix B: How AI Chips Work 33 Parallel Computing 33 Low-Precision Computing 34 Memory Optimization 35 Domain-Specific Languages 36 Appendix C: AI Chip Benchmarking Studies 37 Appendix D: Chip Economics Model 39 Chip Transistor Density, Design Costs, and Energy Costs 40

6、Foundry, Assembly, Test and Packaging Costs 41 Acknowledgments 44 Center for Security and Emerging Technology | 3 Introduction and Summary Artificial intelligence will play an important role in national and international security in the years to come. As a result, the U.S. government is considering

7、how to control the diffusion of AI-related information and technologies. Because general-purpose AI software, datasets, and algorithms are not effective targets for controls, the attention naturally falls on the computer hardware necessary to implement modern AI systems. The success of modern AI tec

8、hniques relies on computation on a scale unimaginable even a few years ago. Training a leading AI algorithm can require a month of computing time and cost $100 million. This enormous computational power is delivered by computer chips that not only pack the maximum number of transistors basic computa

9、tional devices that can be switched between on (1) and off (0) statesbut also are tailor-made to efficiently perform specific calculations required by AI systems. Such leading-edge, specialized “AI chips” are essential for cost-effectively implementing AI at scale; trying to deliver the same AI appl

10、ication using older AI chips or general-purpose chips can cost tens to thousands of times more. The fact that the complex supply chains needed to produce leading-edge AI chips are concentrated in the United States and a small number of allied democracies provides an opportunity for export control po

11、licies. This report presents the above story in detail. It explains how AI chips work, why they have proliferated, and why they matter. It also shows why leading- edge chips are more cost-effective than older generations, and why chips specialized for AI are more cost-effective than general-purpose

12、chips. As part of this story, the report surveys semiconductor industry and AI chip design Center for Security and Emerging Technology | 4 trends shaping the evolution of chips in general and AI chips in particular. It also presents a consolidated discussion of technical and economic trends that res

13、ult in the critical cost-effectiveness tradeoffs for AI applications. In this paper, AI refers to cutting-edge computationally-intensive AI systems, such as deep neural networks. DNNs are responsible for most recent AI breakthroughs, like DeepMinds AlphaGo, which beat the world champion Go player. A

14、s suggested above, we use “AI chips” to refer to certain types of computer chips that attain high efficiency and speed for AI-specific calculations at the expense of low efficiency and speed for other calculations.* This paper focuses on AI chips and why they are essential for the development and de

15、ployment of AI at scale. It does not focus on details of the supply chain for such AI chips or the best targets within the supply chain for export controls (CSET has published preliminary results on this topic1). Forthcoming CSET reports will analyze the semiconductor supply chain, national competit

16、iveness, the prospects of Chinas semiconductor industry for supply chain localization, and policies the United States and its allies can pursue to maintain their advantages in the production of AI chips, recommending how this advantage can be utilized to ensure beneficial development and adoption of

17、 AI technologies. This report is organized as follows: Industry Trends Favor AI Chips over General-Purpose Chips From the 1960s until the 2010s, engineering innovations that shrink transistors doubled the number of transistors on a single computer chip roughly every two years, a phenomenon known as

18、Moores Law. Computer chips became millions of times faster and more efficient during this period. (Section II.) * Our definition of “AI chips” includes graphics processing units (GPUs), field-programmable gate arrays (FPGAs), and certain types of application-specific integrated circuits (ASICs) spec

19、ialized for AI calculations. Our definition also includes a GPU, FPGA, or AI-specific ASIC implemented as a core on system-on-a-chip (SoC). AI algorithms can run on other types of chips, including general-purpose chips like central processing units (CPUs), but we focus on GPUs, FPGAs, and AI-specifi

20、c ASICs because of their necessity for training and running cutting-edge AI algorithms efficiently and quickly, as described later in the paper. Center for Security and Emerging Technology | 5 The transistors used in todays state-of-the-art chips are only a few atoms wide. But creating even smaller

21、transistors makes engineering problems increasingly difficult or even impossible to solve, causing the semiconductor industrys capital expenditures and talent costs to grow at an unsustainable rate. As a result, Moores Law is slowingthat is, the time it takes to double transistor density is growing

22、longer. The costs of continuing Moores Law are justified only because it enables continuing chip improvements, such as transistor efficiency, transistor speed, and the ability to include more specialized circuits in the same chip. (Section III and IV.) The economies of scale historically favoring ge

23、neral-purpose chips like central processing units have been upset by rising demand for specialized applications like AI and the slowing of Moores Law-driven CPU improvements. Accordingly, specialized AI chips are taking market share from CPUs. (Section V.) AI Chip Basics AI chips include graphics pr

24、ocessing units (GPUs), field-programmable gate arrays (FPGAs), and application-specific integrated circuits (ASICs) that are specialized for AI. General-purpose chips like central processing units (CPUs) can also be used for some simpler AI tasks, but CPUs are becoming less and less useful as AI adv

25、ances. (Section V(A).) Like general-purpose CPUs, AI chips gain speed and efficiency (that is, they are able to complete more computations per unit of energy consumed) by incorporating huge numbers of smaller and smaller transistors, which run faster and consume less energy than larger transistors.

26、But unlike CPUs, AI chips also have other, AI-optimized design features. These features dramatically accelerate the identical, predictable, independent calculations required by AI algorithms. They include executing a large number of calculations in parallel rather than sequentially, as in CPUs; calc

27、ulating numbers with low precision in a way that successfully implements AI algorithms but reduces the number of transistors needed for the same calculation; speeding up memory access by, for example, storing an entire AI algorithm in a single AI chip; and using programming languages built specifica

28、lly to efficiently translate AI computer code for execution on an AI chip. (Section V and Appendix B.) Center for Security and Emerging Technology | 6 Different types of AI chips are useful for different tasks. GPUs are most often used for initially developing and refining AI algorithms; this proces

29、s is known as “training.” FPGAs are mostly used to apply trained AI algorithms to real- world data inputs; this is often called “inference.” ASICs can be designed for either training or inference. (Section V(A).) Why Cutting-Edge AI Chips are Necessary for AI Because of their unique features, AI chi

30、ps are tens or even thousands of times faster and more efficient than CPUs for training and inference of AI algorithms. State-of-the-art AI chips are also dramatically more cost-effective than state-of-the-art CPUs as a result of their greater efficiency for AI algorithms. An AI chip a thousand time

31、s as efficient as a CPU provides an improvement equivalent to 26 years of Moores Law-driven CPU improvements. (Sections V(B) and VI(A) and Appendix C.) Cutting-edge AI systems require not only AI-specific chips, but state-of-the-art AI chips. Older AI chipswith their larger, slower, and more power-h

32、ungry transistorsincur huge energy consumption costs that quickly balloon to unaffordable levels. Because of this, using older AI chips today means overall costs and slowdowns at least an order of magnitude greater than for state-of- the-art AI chips. (Section IV(B) and VI(A) and Appendix D.) These

33、cost and speed dynamics make it virtually impossible to develop and deploy cutting-edge AI algorithms without state-of-the-art AI chips. Even with state-of-the-art AI chips, training an AI algorithm can cost tens of millions of U.S. dollars and take weeks to complete. In fact, at top AI labs, a larg

34、e portion of total spending is on AI-related computing. With general-purpose chips like CPUs or even older AI chips, this training would take substantially longer to complete and cost orders of magnitude more, making staying at the research and deployment frontier virtually impossible. Similarly, pe

35、rforming inference using less advanced or less specialized chips could involve similar cost overruns and take orders of magnitude longer. (Section VI(B).) Implications for National AI Competitiveness State-of-the-art AI chips are necessary for the cost-effective, fast development and deployment of a

36、dvanced security-relevant AI systems. The United States and its allies have a competitive advantage in several semiconductor industry sectors necessary for the production of these chips. U.S. firms dominate AI Center for Security and Emerging Technology | 7 chip design, including electronic design a

37、utomation (EDA) software used to design chips. Chinese AI chip design firms are far behind and are dependent on U.S. EDA software to design their AI chips. U.S., Taiwanese, and South Korean firms control the large majority of chip fabrication factories (“fabs”) operating at a sufficiently advanced l

38、evel to fabricate state-of-the-art AI chips, though a Chinese firm recently gained a small amount of comparable capacity. Chinese AI chip design firms nevertheless outsource manufacturing to non-Chinese fabs, which have greater capacity and exhibit greater manufacturing quality. U.S., Dutch, and Jap

39、anese firms together control the market for semiconductor manufacturing equipment (SME) used by fabs. However, these advantages could disappear, especially with Chinas concerted efforts to build an advanced chip industry. Given the security importance of state-of-the-art AI chips, the United States

40、and its allies must protect their competitive advantage in the production of these chips. Future CSET reports will analyze policies for the United States and its allies to maintain their competitive advantage and explore points of control for these countries to ensure that the development and adopti

41、on of AI technologies increases global stability and is broadly beneficial for all. (Section VII.) The Laws of Chip Innovation All computer chipsincluding general-purpose CPUs and specialized ones like AI chipsbenefit from smaller transistors, which run faster and consume less energy than larger tra

42、nsistors. Compared to CPUs, AI chips also gain efficiency and speed for AI applications through AI-optimized designs. However, at least while transistor shrinkage came at a fast rate and produced large speed and efficiency gains through the late 2000s, the value of specialized designs remained low a

43、nd CPUs were the dominant chip. However, Moores Law is close to driving transistors to fundamental size limits at atomic scales. For a basic introduction to chips, see Appendix A. Transistor Shrinkage: Moores Law Moores LawMoores Law states that the number of transistors in a chip doubles about ever

44、y two years. Technical innovations that shrink transistors allow increased transistor density. Moores Law was first observed in the 1960s, and it held until the 2010s, when improvements in transistor density began slowing. Today, leading chips contain billions of transistors, but they have 15 times

45、fewer transistors than they would have if Moores Law had continued.2 Center for Security and Emerging Technology | 8 Transistor density increases occur in generations, or “nodes.” Each node corresponds to the transistor size (expressed in terms of length) that allows a doubling of transistor density

46、 relative to the previous node. Fabs began “risk production,” i.e. experimental production, of the latest node of 5 nanometers (“nm”) in 2019, with mass production expected in 2020.3 The previous leading nodes were 7 nm and 10 nm.4 A companion principle to Moores Law says that because smaller transi

47、stors generally use less power than larger ones, as transistor density increases, power consumption per unit chip area remains constant.5 However, transistor power reduction rates slowed around 2007.6 Efficiency and Speed Improvements CPU speed has improved prodigiously since the 1960s due in large

48、part to Moores Law. Greater transistor density improved speed primarily via “frequency scaling,” i.e. transistors switching between ones and zeros faster to allow more calculations per second by a given execution unit. Because smaller transistors use less power than larger ones, transistor switching

49、 speeds could be increased without increasing total power consumption.7 Figure 1 shows transistor density, speed, and efficiency improvements since 1979. Between 1978 and 1986, frequency scaling drove 22 percent annual increases in speed. Then, between 1986 and 2003, speed increased by 52 percent annually, due to frequency sca