1、1 2 Steering Committee Yoav Shoham (Chair) Stanford University Raymond Perrault SRI International Erik Brynjolfsson MIT Jack Clark OpenAI James Manyika McKinsey Global Institute Juan Carlos Niebles Stanford University Terah Lyons Partnership On AI John Etchemendy Stanford University Barbara Grosz Ha

2、rvard University Project Manager Zoe Bauer AI INDEX 2018 3 How to cite this Report: Yoav Shoham, Raymond Perrault, Erik Brynjolfsson, Jack Clark, James Manyika, Juan Carlos Niebles, Terah Lyons, John Etchemendy, Barbara Grosz and Zoe Bauer, The AI Index 2018 Annual Report”, AI Index Steering Committ

3、ee, Human-Centered AI Initiative, Stanford University, Stanford, CA, December 2018. AI INDEX 2018 Our Mission is to ground the conversation about AI in data. The AI Index is an effort to track, collate, distill, and visualize data relating to artificial intelligence. It aspires to be a comprehensive

4、 resource of data and analysis for policymakers, researchers, executives, journalists, and the general public to develop intuitions about the complex field of AI. Welcome to the AI Index 2018 Report (c) 2018 by Stanford University, “The AI Index 2018 Annual Report” is made available under a Creative

5、 Commons Attribution-NoDerivatives 4.0 License (International) https:/creativecommons.org/licenses/by-nd/4.0/legalcode 4 Table of contents AI INDEX 2018 Introduction to the AI Index 2018 Report Overview Volume of Activity Research Published Papers Course Enrollment Participation Robot Software Indus

6、try Startups / Investment Jobs Patents AI Adoption Survey Earnings Calls Robot Installations Open Source Software GitHub Project Statistics Public Interest Sentiment of Media Coverage Government mentions Technical Performance Vision Natural Language Understanding Other Measures Derivative Measures G

7、overnment Initiatives Human-Level Performance Milestones Whats Missing? Acknowledgements Appendix 5 6 8 9 21 26 29 31 33 35 36 38 41 42 43 44 47 50 55 57 59 63 66 69 5 We are pleased to introduce the AI Index 2018 Annual Report. This years report accomplishes two objectives. First, it refreshes last

8、 years metrics. Second, it provides global context whenever possible. The former is critical to the Indexs mission grounding the AI conversation means tracking volumetric and technical progress on an ongoing basis. But the latter is also essential. There is no AI story without global perspective. Th

9、e 2017 report was heavily skewed towards North American activities. This reflected a limited number of global partnerships established by the project, not an intrinsic bias. This year, we begin to close the global gap. We recognize that there is a long journey ahead one that involves further collabo

10、ration and outside participation to make this report truly comprehensive. Still, we can assert that AI is global. 83 percent of 2017 AI papers on Scopus originate outside the U.S. 28 percent of these papers originate in Europe the largest percentage of any region. University course enrollment in art

11、ificial intelligence (AI) and machine learning (ML) is increasing all over the world, most notably at Tsinghua in China, whose combined AI + ML 2017 course enrollment was 16x larger than it was in 2010. And there is progress beyond just the United States, China, and Europe. South Korea and Japan wer

12、e the 2nd and 3rd largest producers of AI patents in 2014, after the U.S. Additionally, South Africa hosted the second Deep Learning Indaba conference, one of the worlds largest ML teaching events, which drew over 500 participants from 20+ African countries. AIs diversity is not just geographic. Tod

13、ay, over 50% of the Partnership on AIs members are nonprofits including the ACLU, Oxfords Future of Humanity Institute, and the United Nations Development Programme. Also, there is heightened awareness of gender and racial diversitys importance to progress in AI. For example, we see increased partic

14、ipation in organizations like AI4ALL and Women in Machine Learning (WiML), which encourage involvement by underrepresented groups. Introduction to the AI Index 2018 Annual Report AI INDEX 2018 6 The report has four sections: 1.Data: Volume of Activity and Technical Performance 2.Other measures: Rece

15、nt Government Initiatives, Derivative measures, and Human-Level Performance 3.Discussion: Whats Missing? 4.Appendix DATA The Volume of Activity metrics capture engagement in AI activities by academics, corporations, entrepreneurs, and the general public. Volumetric data ranges from the number of und

16、ergraduates studying AI, to the percent of female applicants for AI jobs, to the growth in venture capital funding of AI startups. The Technical Performance metrics capture changes in AI performance over time. For example, we measure the quality of question answering and the speed at which computers

17、 can be trained to detect objects. The 2018 AI Index adds additional country-level granularity to many of last years metrics, such as robot installations and AI conference attendance. Additionally, we have added several new metrics and areas of study, such as patents, robot operating system download

18、s, the GLUE metric, and the COCO leaderboard. Overall, we see a continuation of last years main takeaway: AI activity is increasing nearly everywhere and technological performance is improving across the board. Still, there were certain takeaways this year that were particularly interesting. These i

19、nclude the considerable improvement in natural language and the limited gender diversity in the classroom. OTHER MEASURES Like last year, the Derivative Measures section investigates relationships between trends. We also show an exploratory measure, the AI Vibrancy Index, which combines trends acros

20、s academia and industry to quantify the liveliness of AI as a field. We introduce a new qualitative metric this year: Recent Government Initiatives. This is a simplified overview of recent government investments in artificial intelligence. We include initiatives from the U.S., China, and Europe. The

21、 AI Index looks forward to including more government data and analysis in future reports by collaborating with additional organizations. The Human-Level Performance Milestones section of the report builds on our timeline of instances where AI shows human and superhuman abilities. We include four new

22、 achievements from 2018. AI Index Report Overview AI INDEX 2018 7 Finally, to start a conversation in the AI community, the Whats Missing? section presents suggestions from a few experts in the field, who offer ideas about how the AI Index could be made more comprehensive and representative. APPENDI

23、X The Appendix supplies readers with a fully transparent description of sources, methodologies, and nuances. Our appendix also houses underlying data for nearly every graph in the report. We hope that each member of the AI community interacts with the data most relevant to their work and interests.

24、SYMBOLS We earmark pages with the globe symbol below when discussing AIs universality. This includes country comparisons, deep dives into regions outside of the U.S., and data on diversity in the AI community. AI Index Report Overview (continued) AI INDEX 2018 8 AI INDEX 2018 VOLUME OF ACTIVITY The

25、graph below shows growth in annual publishing rates of academic papers, relative to their rates in 1996. The graph compares the growth of papers across All fields, Computer Science (CS), and Artificial Intelligence (AI). The growth of annually published papers in AI continues to outpace that of annu

26、ally published papers in CS, suggesting that growth in AI publishing is driven by more than a heightened interest in computer science. See Appendix 1 for data and methodology. AI outpaces CS AI papers on Scopus have increased 8x since 1996. CS papers increased 6x during the same timeframe. VOLUME OF

27、 ACTIVITY RESEARCH Published Papers: Papers by topic 9 Note: This visual uses the Scopus query search term “Artificial Intelligence,” not the Elsevier keyword approach. See more details in the appendix. Growth of annually published papers by topic (19962017) Source: Scopus AI PapersCS PapersAll Pape

28、rs Growth in papers (relative to 1996) 2000200520152010 1x 3x 5x 7x 9x The graph below shows the number of AI papers published annually by region. Europe has consistently been the largest publisher of AI papers 28% of AI papers on Scopus in 2017 originated in Europe. Meanwhile, the number of papers

29、published in China increased 150% between 2007 and 2017. This is despite the spike and drop in Chinese papers around 2008. See Appendix 2 for data and methodology. Europe is the largest publisher of AI papers In 2017, 28% of AI papers on Scopus were affiliated with European authors, followed by Chin

30、a (25%) and the U.S. (17%). VOLUME OF ACTIVITY RESEARCH Published Papers: AI papers by region 10 Note: We speculate that the increase in AI papers in China around 2008 is a result of The National Medium- and Long-Term Program for Science and Technology Development (2006 2020), and other government p

31、rograms that provide funding and a range of incentive policies for AI research. Similarly, FP7 (20072013) and other science and technology research programs in Europe may have contributed to the small uptick in papers around 20082010. Annually published AI papers on Scopus by region (19982017) Sourc

32、e: Elsevier China Number of papers United StatesEuropeRest of World 2000200520152010 0 5,000 10,000 15,000 20,000 The graph below shows the number of AI papers on Scopus, by subcategory. Categories are not mutually exclusive. 56 percent of papers fell into the Machine Learning and Probabilistic Reas

33、oning category in 2017, compared to 28% in 2010. For most categories below, papers were published at a faster rate during the period 20142017 than in the period 20102014. Most notably, Neural Networks had a compound annual growth rate (CAGR) of 3% from 20102014, followed by a CAGR of 37% from 201420

34、17. See Appendix 2 for data and methodology. VOLUME OF ACTIVITY RESEARCH Published Papers: AI papers by subcategory 11 The number of Scopus papers on Neural Networks had a CAGR of 37% from 2014 to 2017 Number of AI papers on Scopus by subcategory (19982017) Source: Elsevier Number of papers Machine

35、Learning and Probabilistic Reasoning Search and Optimization NLP and Knowledge Representation Computer Vision Fuzzy Systems Planning and Decision Making Neural Networks Total 60,000 40,000 20,000 2000200520102015 0 The graph below shows the number of AI papers on arXiv, by each papers primary subcat

36、egory. The right axis refers the sum of all AI papers on arXiv (indicated by the grey dashed line). The number of AI papers on arXiv is increasing overall and in a number of subcategories. This points to AI authors tendency to disseminate their research, regardless of whether it is peer reviewed or

37、has been accepted into AI conferences. This also points to the fields competitive nature. Computer Vision (CV) and Pattern Recognition has been the largest AI subcategory on arXiv since 2014; prior to 2014, growth in this category closely tracked Artificial Intelligence and Machine Learning. In addi

38、tion to showing a growing interest in Computer Vision (and its general applied applications), this also indicates the growth in other AI application areas, such as Computation and Language and Robotics. See Appendix 3 for data and methodology. VOLUME OF ACTIVITY RESEARCH Published Papers: AI papers

39、on arXiv “.Aside from the increase in publications, its important to note the adoption of arXiv by these communities for disseminating results. Weve seen many times how establishing some critical mass then catalyzes ever higher levels of participation within a community.” Paul Ginsparg, Cornell 12 N

40、umber of AI papers on arXiv by subcategory (20102017) Source: arXiv Artificial IntelligenceComputation about half of all companies had embedded AI into a corporate business process. However, its still early; most had not yet adopted the complementary practices necessary to capture value from AI at s

41、cale.” -Michael Chui, McKinsey North America: N = 479; Developing markets (incl. China): N = 189 (China N = 35); Europe: N = 803 North America Developing markets (incl. China) Europe Robotic process automation Machine learning Conversational interfaces Computer vision NL text understanding NL speech

42、 understanding NL generation Physical robotics Autonomous vehicles Percent of respondents Capabilities embedded in at least one company function (2018) Source: McKinsey AsiaPacific: N = 263; India: N = 197; Middle East and North Africa: N = 77; Latin America: N = 127 India Middle East and North Afri

43、ca Latin America Robotic process automation Machine learning Conversational interfaces Computer vision NL text understanding NL speech understanding NL generation Physical robotics Autonomous vehicles Asia Pacific Percent of respondents Capabilities embedded in at least one company function (2018) S

44、ource: McKinsey Telecom: N = 77; High tech: N = 215; Financial services: N = 306; Professional services: N = 221; Electric power and natural gas: N = 54; Healthcare systems and services: N = 67; Automotive and assembly: N = 120; Retail: N = 46; Travel, transport, and logistics: N = 55; Pharma and me

45、dical products: N = 65. Telecom High tech Financial services Professional services Power Telecom: N = 77; High tech: N = 215; Financial services: N = 306; Professional services: N = 221; Electric power and natural gas: N = 54; Healthcare systems and services: N = 67; Automotive and assembly: N = 120

46、; Retail: N = 46; Travel, transport, and logistics: N = 55; Pharma and medical products: N = 65. See description and data from Service operations, Product / service development, and Marketing / sales on the previous page. Organizations adopt AI in business functions that provide the most value withi

47、n their industry This implies that the rate of AI progress for specific applications will likely correlate to uptake in industries where that specialization is particularly important. Supply-chain management ManufacturingRisk Telecom High tech Financial services Professional services Power see appen

48、dix Accuracy ImageNet competition test set accuracyImageNet 2012 validation set accuracy Human performance ImageNet competition ends in 2017. 20018 70% 2010 80% 90% 100% 48 TECHNICAL PERFORMANCE VISION Object detection: ImageNet training time The graph below shows the amount of time it takes to train a network to