报告预览

ONLINE：AI 全景报告 2020（英文版）（177页）.pdf

编号：20692

PDF 177页 28.87MB 下载积分：VIP专享

下载报告请您先登录！

ONLINE：AI 全景报告 2020（英文版）（177页）.pdf

1、State of AI Report October 1, 2020 #stateofaistateof.ai Ian HogarthNathan Benaich About the authors Nathan is the General Partner of Air Street Capital, a venture capital fi rm investing in AI-fi rst technology and life science companies. He founded RAAIS and London.AI, which connect AI practitioner

2、s from large companies, startups and academia, and the RAAIS Foundation that funds open-source AI projects. He studied biology at Williams College and earned a PhD from Cambridge in cancer research. Nathan BenaichIan Hogarth Ian is an angel investor in 60+ startups. He is a Visiting Professor at UCL

3、 working with Professor Mariana Mazzucato. Ian was co-founder and CEO of Songkick, the concert service used by 17M music fans each month. He studied engineering at Cambridge where his Masters project was a computer vision system to classify breast cancer biopsy images. He is the Chair of Phasecraft,

4、 a quantum software company. stateof.ai 2020 Introduction | Research | Talent | Industry | Politics | Predictions#stateofai Artifi cial intelligence (AI) is a multidisciplinary fi eld of science and engineering whose goal is to create intelligent machines. We believe that AI will be a force multipli

5、er on technological progress in our increasingly digital, data-driven world. This is because everything around us today, ranging from culture to consumer products, is a product of intelligence. The State of AI Report is now in its third year. New to the 2020 edition are several invited content contr

6、ibutions from a range of well-known and up-and-coming companies and research groups. Consider this Report as a compilation of the most interesting things weve seen with a goal of triggering an informed conversation about the state of AI and its implication for the future. We consider the following k

7、ey dimensions in our report: -Research: Technology breakthroughs and their capabilities. - Talent: Supply, demand and concentration of talent working in the fi eld. -Industry: Areas of commercial application for AI and its business impact. -Politics: Regulation of AI, its economic implications and t

8、he emerging geopolitics of AI. -Predictions: What we believe will happen in the next 12 months and a 2019 performance review to keep us honest. Collaboratively produced by Ian Hogarth (soundboy) and Nathan Benaich (nathanbenaich). stateof.ai 2020 Introduction | Research | Talent | Industry | Politic

10、 Nu (Claire) Wang. Artifi cial intelligence (AI): A broad discipline with the goal of creating intelligent machines, as opposed to the natural intelligence that is demonstrated by humans and animals. It has become a somewhat catch all term that nonetheless captures the long term ambition of the fi e

11、ld to build machines that emulate and then exceed the full range of human cognition. Machine learning (ML): A subset of AI that often uses statistical techniques to give machines the ability to learn from data without being explicitly given the instructions for how to do so. This process is known as

12、 “training” a “model” using a learning “algorithm” that progressively improves model performance on a specifi c task. Reinforcement learning (RL): An area of ML concerned with developing software agents that learn goal-oriented behavior by trial and error in an environment that provides rewards or p

13、enalties in response to the agents actions (called a “policy”) towards achieving that goal. Deep learning (DL): An area of ML that attempts to mimic the activity in layers of neurons in the brain to learn how to recognise complex patterns in data. The “deep” in deep learning refers to the large numb

14、er of layers of neurons in contemporary ML models that help to learn rich representations of data to achieve better performance gains. Defi nitions stateof.ai 2020 Introduction | Research | Talent | Industry | Politics | Predictions#stateofai Algorithm: An unambiguous specifi cation of how to solve

15、a particular problem. Model: Once a ML algorithm has been trained on data, the output of the process is known as the model. This can then be used to make predictions. Supervised learning: A model attempts to learn to transform one kind of data into another kind of data using labelled examples. This

16、is the most common kind of ML algorithm today. Unsupervised learning: A model attempts to learn a datasets structure, often seeking to identify latent groupings in the data without any explicit labels. The output of unsupervised learning often makes for good inputs to a supervised learning algorithm

17、 at a later point. Transfer learning: An approach to modelling that uses knowledge gained in one problem to bootstrap a different or related problem, thereby reducing the need for signifi cant additional training data and compute. Natural language processing (NLP): Enabling machines to analyse, unde

18、rstand and manipulate language. Computer vision: Enabling machines to analyse, understand and manipulate images and video. Defi nitions stateof.ai 2020 Introduction | Research | Talent | Industry | Politics | Predictions#stateofai Research -A new generation of transformer language models are unlocki

19、ng new NLP use-cases. -Huge models, large companies and massive training costs dominate the hottest area of AI today: Natural Language Processing. -Biology is experiencing its “AI moment”: From medical imaging, genetics, proteomics, chemistry to drug discovery. -AI is mostly closed source: Only 15%

20、of papers publish their code, which harms accountability and reproducibility in AI. Talent -American institutions and corporations further their dominance of major academic conference papers acceptances. -Multiple new institutions of higher education dedicated to AI are formed. - Corporate-driven ac

21、ademic brain drain is signifi cant and appears to negatively impact entrepreneurship. -US AI ecosystem is fuelled by foreign talent and the contribution of researchers educated in China to world-class papers is clear. Industry - The fi rst trial of an AI-discovered drug begins in Japan and the fi rs

22、t US medical reimbursement for AI-based imaging procedure is granted. -Self-driving car mileage remains microscopic and open sourcing of data grows to crowdsource new solutions. -Google, Graphcore, and NVIDIA continue to make major advances in their AI hardware platforms. -NLP applications in indust

23、ry continue to expand their footprint and are implemented in Google Search and Microsoft Bing. Politics -After two wrongful arrests involving facial recognition, ethical risks that researchers have been warning about come into sharp focus. - Semiconductor companies continue to grow in geopolitical s

24、ignifi cance, particularly Taiwans TSMC. -The US Military is absorbing AI progress from academia and industry labs. -Nations pass laws to let them scrutinize foreign takeovers of AI companies and the UKs Arm will be a key test. Executive Summary stateof.ai 2020 Introduction | Research | Talent | Ind

26、0M), Chorus.ai ($45M), Ironscales ($23M), ComplyAdvantage ($50M), Rasa ($26M), HyperScience ($60M), ASAPP ($185M), Cresta ($21M), Eigen ($37M), K Health ($48M), Signal ($25M), and many more! No autonomous driving company drives 15M miles in 2019. YesWaymo (1.45M miles), Cruise (831k miles), Baidu (1

27、08k miles). Privacy-preserving ML adopted by a F2000 company other than GAFAM (Google, Apple, Facebook, Amazon, Microsoft). Yes Machine learning ledger orchestration for drug discovery (MELLODY) research consortium with large pharmaceutical companies and startups including Glaxosmithkline, Merck and

28、 Novartis. Unis build de novo undergrad AI degrees.Yes CMU graduates fi rst cohort of AI undergrads, Singapores SUTD launches undergrad degree in design and AI, NYU launches data science major, Abu Dhabi builds an AI university. Google has major quantum breakthrough and 5 new startups focused on qua

29、ntum ML are formed. Sort of Google demonstrated quantum supremacy in October 2019! Many new quantum companies were launched in 2019 but only Cambridge Quantum Computing, Rahko and Xanadu.ai are explicitly working on quantum ML. Governance of AI becomes key issue and one major AI company makes substa

31、ion Date 2017 2018 2019 2020 25% 20% 15% 10% 0% 5% Research paper code implementations are important for accountability, reproducibility and driving progress in AI. The fi eld has made little improvement on this metric since mid-2016. Traditionally, academic groups are more likely to publish their c

32、ode than industry groups. Notable organisation that dont publish all of their code are OpenAI and DeepMind. For the biggest tech companies, their code is usually intertwined with proprietary scaling infrastructure that cannot be released. This points to centralization of AI talent and compute as a h

33、uge problem. AI research is less open than you think: Only 15% of papers publish their code stateof.ai 2020 Introduction | Research | Talent | Industry | Politics | Predictions#stateofai Hosting 3,000 State-of-the-Art leaderboards, 750+ ML components, and 25,000+ research along with code. Papers Wit

34、h Code tracks openly-published code and benchmarks model performance stateof.ai 2020 Introduction | Research | Talent | Industry | Politics | Predictions#stateofai % PyTorch Papers of Total TensorFlow/PyTorch Papers % of total framework mentions 100% 75% 50% 25% 0% Of 20-35% of conference papers tha

35、t mention the framework they use, 75% cite the use of PyTorch but not TensorFlow. Of 161 authors who published more TensorFlow papers than PyTorch papers in 2018, 55% of them have switched to PyTorch. The opposite happened in 15% of cases. Meanwhile, the authors observe that TensorFlow, Caffe and Ca

36、ffe2 are still the workhorse for production AI. Facebooks PyTorch is fast outpacing Googles TensorFlow in research papers, which tends to be a leading indicator of production use down the line stateof.ai 2020 Introduction | Research | Talent | Industry | Politics | Predictions#stateofai 47% of these

37、 implementations are based on PyTorch vs. 18% for TensorFlow. PyTorch offers greater fl exibility and a dynamic computational graph that makes experimentation easier. JAX is a Google framework that is more math friendly and favored for work outside of convolutional models and transformers. PyTorch i

38、s also more popular than TensorFlow in paper implementations on GitHub stateof.ai 2020 Repository Creation Date Share of implementations 100% 75% 50% 25% 0% 2017 2018 2019 2020 Introduction | Research | Talent | Industry | Politics | Predictions#stateofai Huge models, large companies and massive tra

39、ining costs dominate the hottest area of AI today, NLP. Language models: Welcome to the Billion Parameter club 2018 (left) through 2019 (right)2020 onwards 11B 175B 9.4B 17B 1.5B 8.3B 2.6B 1.5B 66M355M340M330M665M465M340M110M94M 1.5B stateof.ai 2020 Note: The number of parameters indicates how many

40、different coeffi cients the algorithm optimizes during the training process. Introduction | Research | Talent | Industry | Politics | Predictions#stateofai Empirical scaling laws of neural language models show smooth power-law relationships, which means that as model performance increases, the model

41、 size and amount of computation has to increase more rapidly. Bigger models, datasets and compute budgets clearly drive performance stateof.ai 2020 Introduction | Research | Talent | Industry | Politics | Predictions#stateofai Tuning billions of model parameters costs millions of dollars Based on va

42、riables released by Google et al., youre paying circa $1 per 1,000 parameters. This means OpenAIs 175B parameter GPT-3 could have cost tens of millions to train. Experts suggest the likely budget was $10M. stateof.ai 2020 Introduction | Research | Talent | Industry | Politics | Predictions#stateofai

43、 This sparse transformer-based machine translation model has 600B parameters. To achieve the needed quality improvements in machine translation, Googles fi nal model trained for the equivalent of 22 TPU v3 core years or 5 days with 2,048 cores non-stop stateof.ai 2020 Introduction | Research | Talen

44、t | Industry | Politics | Predictions#stateofai Without major new research breakthroughs, dropping the ImageNet error rate from 11.5% to 1% would require over one hundred billion billion dollars! Many practitioners feel that progress in mature areas of ML is stagnant. Were rapidly approaching outrag

45、eous computational, economic, and environmental costs to gain incrementally smaller improvements in model performance stateof.ai 2020 Introduction | Research | Talent | Industry | Politics | Predictions#stateofai This has implications for problems where training data samples are expensive to generat

46、e, which likely confers an advantage to large companies entering new domains with supervised learning-based models. A larger model needs less data than a smaller peer to achieve the same performance stateof.ai 2020 Introduction | Research | Talent | Industry | Politics | Predictions#stateofai Google

47、 made use of their large language models to deliver higher quality translations for languages with limited amounts of training data, for example Hansa and Uzbek. This highlights the benefi ts of transfer learning. Low resource languages with limited training data are a benefi ciary of large models s

48、tateof.ai 2020 Introduction | Research | Talent | Industry | Politics | Predictions#stateofai Since 2012 the amount of compute needed to train a neural network to the same performance on ImageNet classifi cation has been decreasing by a factor of 2 every 16 months. Even as deep learning consumes mor

49、e data, it continues to get more effi cient Training effi ciency factorTwo distinct eras of compute in training AI systems stateof.ai 2020 Introduction | Research | Talent | Industry | Politics | Predictions#stateofai PolyAI, a London-based conversational AI company, open-sourced their ConveRT model (a pre-trained contextual re-ranker based on transformers).

友情提示

1、下载报告失败解决办法
2、PDF文件下载后，可能会被浏览器默认打开，此种情况可以点击浏览器菜单，保存网页到桌面，就可以正常下载了。
3、本站不支持迅雷下载，请使用电脑自带的IE浏览器，或者360浏览器、谷歌浏览器下载即可。
4、本站报告下载后的文档和图纸-无水印,预览文档经过压缩，下载后原文更清晰。

本文（ONLINE：AI 全景报告 2020（英文版）（177页）.pdf）为本站（Mercury）主动上传，三个皮匠报告文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若此文所含内容侵犯了您的版权或隐私，请立即通知三个皮匠报告文库（点击联系客服），我们立即给予删除！

温馨提示：如果因为网速或其他原因下载失败请重新下载，重复下载不扣分。

上海品茶

ONLINE：AI 全景报告 2020（英文版）（177页）.pdf

ONLINE：AI 全景报告 2020（英文版）（177页）.pdf