1、1 浅谈人工智能的下个十年 Jie Tang Computer Science Tsinghua University 2 人工智能的第三次浪潮 3 人工智能历史人工智能历史 4 人工智能领域发展趋势人工智能领域发展趋势 Powered by Claude Shannon Shannon, Claude E. XXII. Programming a computer for playing chess. Philosophical magazine 41.314 (1950): 256-275. Alan Turing Turing, Alan M. Solvable and unsolvab

2、le problems. Science News-ens. fr 39 (1954). 1950计算机象棋博弈 1954图灵测试 5 人工智能领域发展趋势人工智能领域发展趋势 Powered by John McCarthy McCarthy, J., et al. Dartmouth Conference. Dartmouth Summer Research Conference on Artificial Intelligence. 1956 1956达特茅斯会议 1959一般问题解决器 Marvin Minsky Nathan Rochester Claude Shannon Herb

3、ert Simon J.C. ShawAllen Newell Newell, A.; Shaw, J.C.; Simon, H.A. (1959). Report on a general problem-solving program. Proceedings of the International Conference on Information Processing. pp. 256264. 6 人工智能领域发展趋势人工智能领域发展趋势 Powered byDaniel Bobrow Bobrow, Daniel G. Natural language input for a co

4、mputer problem solving system. (1964) Joseph Weizenbaum Weizenbaum, Joseph. ELIZAa computer program for the study of natural language communication between man and machine. Communications of the ACM 9.1 (1966): 36-45. 1964 理解自然语言输入 1966 ELIZA人机对话 7 人工智能领域发展趋势人工智能领域发展趋势 Powered by Buchanan, Bruce, Ge

5、orgia Sutherland, and Edward A. Feigenbaum. Heuristic DENDRAL: a program for generating explanatory hypotheses in organic chemistry. Defense Technical Information Center, 1968. 1968 世界首个专家系 统DENDRAL Edward Feigenbaum 8 人工智能领域发展趋势人工智能领域发展趋势 Powered byRandall Davis Applications of meta level knowledge

6、 to the construction, maintenance and use of large knowledge basesM. Stanford University, Computer Science Department, AI Laboratory, 1976. Drew McDermott, Jon Doyle McDermott D, Doyle J. Non-monotonic logic IJ. Artificial intelligence, 1980, 13(1): 41-72. 1976 大规模知识库构建 与维护 1980 非单调逻辑 9 人工智能领域发展趋势人工

7、智能领域发展趋势 Powered by Berliner H J. Backgammon computer program beats world championJ. Artificial Intelligence, 1980, 14(2): 205- 220. 1980 计算机战胜双陆 棋世界冠军 Hans Berliner 10 人工智能领域发展趋势人工智能领域发展趋势 Powered by Brooks R. A robust layered control system for a mobile robotJ. Robotics and Automation, IEEE Journa

8、l of, 1986, 2(1): 14-23 1987 基于行为的机器 人学 Rodney Brooks 11 人工智能领域发展趋势人工智能领域发展趋势 Powered by Tesauro G. TD-Gammon, a self-teaching backgammon program, achieves master-level playJ. Neural computation, 1994, 6(2): 215-219. 1987 自我学习双陆棋 程序 Gerry Tesauro 12 人工智能领域发展趋势人工智能领域发展趋势 Powered byTim Berners-Lee Ber

9、ners-Lee, Tim. Semantic web road map. (1998). McGuinness, Deborah L., and Frank Van Harmelen. OWL web ontology language overview. W3C recommendation 10.2004- 03 (2004): 10. 1998 语义互联网路线图 2004 OWL语言 13 人工智能领域发展趋势人工智能领域发展趋势 Powered by Geoffrey Hinton Hinton, Geoffrey E., Simon Osindero, and Yee-Whye T

10、eh. A fast learning algorithm for deep belief nets. Neural computation 18.7 (2006): 1527-1554. Le, Quoc V., et al. Building high- level features using large scale unsupervised learning. arXiv preprint arXiv:1112.6209 (2011). 2006 深度学习 2011 高层抽象特征构建 14 人工智能领域发展趋势人工智能领域发展趋势 Powered by Markoff, John. G

11、oogle cars drive themselves, in traffic. The New York Times 10 (2010): A1. 2009 谷歌自动驾驶汽车 Sebastian Thrun 15 人工智能领域发展趋势人工智能领域发展趋势 Powered by IBMs Watson Markoff, John. Computer program to take on Jeopardy!. The New York Times (2009). Apples Siri Sadun, Erica, and Steve Sande. Talking to Siri: Learnin

12、g the Language of Apples Intelligent Assistant. Que Publishing, 2013. 2011 沃森获得 Jeopardy冠军 2011 自然语言问答 16 人工智能领域发展趋势人工智能领域发展趋势 Powered by 17 人工智能近人工智能近10年年 18 AI趋势：从感知到认知 From perceptron to cognition ComputingPerceptionCognition Storage & Computing Recognize text, images, objects, voices Organize an

13、d generate knowledge, reasoning 19 Artificial Intelligence AlphaGoAlphaGo Image recognitionImage recognition Self-drivingSelf-driving 20 DDPG(2015)A3C(2016) Perceptron(1958) Frank Rosenblatt Cornell University psychologist BPNN/MLP(1986) Hopfield Network(1982) recurrent & feedback Geoffery Hinton Un

14、iversity of Toronto deep learning Neocognitron(1980) convolution & pooling LeNet/CNN(1998) Yann Lecun New York University deep learning AlexNet(2012) Relu, dropout & bigger VGG(2014) GoogLeNet(2015) ResNet(2016) Kaiming He MSRA = FAIR computer vision DenseNet(2017) RBM(1986/2006) Deep Belief Nets(20

15、06) stack AutoEncoder(1989/2006) Denosing Autoencoder(2008) VAE(2013) Variational Inference Max Welling University of Amsterdam statistical learning GAN(2014) DCGAN(2014) WGAN(2017) PGGAN(2017) Ian Goodfellow Google Brain deep adversarial learning RNN/LSTM(1997) Jrgen Schmidhuber IDSIA Universal AI

16、Seq2Seq(2014) RNN in Speech Recognition(2013) Yoshua Bengio University of Montreal Deep learning Neural Probabilistic Language Model(2003) word2Vec(2013) SeqGAN(2017) LeakGAN(2018) Character CNN(2015) self-attention(2017) Deep Q- learning(2013) AlphaGo(2016) Double DQN(2015) Dueling Net(2016) David

17、Silver DeepMind Reinforcement learning Alpha Zero(2017) Capsule Nets(2017) 算法 21 BERT Pre-train Fine tune Beat all state-of-the-arts on 11 NLP tasks in 2018 https:/arxiv.org/pdf/1810.04805.pdf 22 XLNet Autoregressive Model Beat BERT in 2019 https:/arxiv.org/pdf/1906.08237.pdf 23 ALBERT A Lite BERT P

18、arameter-reduction techniques Beat XLNet and all the others https:/arxiv.org/pdf/1909.11942.pdf 24 Video-to-Video Synthesis The best video synthesis performance https:/arxiv.org/abs/1808.06601 25 graph_net By DeepMind https:/arxiv.org/abs/1806.01261 26 MoCo Unsupervised visual representation learnin

19、g Momentum contrastive learning Outperform its supervised pre-training counterparts https:/arxiv.org/abs/1911.05722 27 SimCLR Simplified contrastive learning framework Outperform previous self-supervised and semi- supervised methods on ImageNet https:/arxiv.org/abs/2002.05709 28 人工智能未来人工智能未来 29 第三代人

20、工智能的理论体系 早在2015年，张钹老师就提出第三代人工智能体系的雏形； 2017年DARPA发起XAI项目，从可解释的机器学习系统、人机 交互技术以及可解释的心理学理论三个方面，全面开展可解释性 AI系统的研究 2018年底，正式公开提出第三代人工智能的理论框架体系第三代人工智能的理论框架体系 建立可解释、鲁棒性可解释、鲁棒性的人工智能理论和方法 发展安全、可靠、可信安全、可靠、可信及可扩展及可扩展的人工智能技术 推动人工智能创新创新应用 具体实施路线图 与脑科学脑科学融合，发展脑启发的人工智能理论 数据与知识融合数据与知识融合的人工智能理论与方法 第三代人工智能的理念在国内外 获得广泛影

21、响力 30 认知图谱 (Cognitive Graph) 知识图谱, 认知推理, 逻辑表达 31 知识图谱 “Knowledge graph”由Google于2012年提出 知识工程，专家系统 CYC: 世界上历史最长的AI项目 (1985) Tim Berners Lee Father of WWW Turing Award Edward Feigenbaum Father of KB Turing Award 32 认知图谱：算法与认知的结合 The Quality Cafe is a now-defunct diner in Los Angeles, California. The re

22、staurant has appeared as a location featured in a number of Hollywood films, including Old School, Gone in 60 Seconds, . Quality Caf Los Angeles is the most populous city in California, the second most populous city in the United States, after New York City, and the third most populous city in North

23、 America. Los Angeles Old School is a 2003 American comedy film released by Dream Works Pictures and The Montecito Picture Company and directed by Todd Phillips. Old School Todd Phillips is an American director, producer, screenwriter, and actor. He is best known for writing and directing films, inc

24、luding Road Trip (2000), Old School (2003), Starsky & Hutch (2004), and The Hangover Trilogy. Todd Phillips Alessandro Moschitti is a professor of the CS Department of the University of Trento, Italy. He is currently a Principal Research Scientist of the Qatar Computing Research Institute (QCRI) Ale

25、ssandro Moschitti Tsinghua University Tsinghua University is a major research university in Beijing and dedicated to academic excellence and global development. Tsinghua is perennially ranked as one of the top academic institutions in China, Asia, and worldwide. 33 算法: BIDAF, BERT, XLNet 目标：理解整个文档，而

26、不仅仅是局部片段 但仍然缺乏在知识层面上的推理能力 BiDAFBERTXLNet 34 挑战 : 可解释性 大部分阅读理解方法都只能看做黑盒黑盒: 输入: 问题和文档 输出: 答案文本块 (在文档中的起止位置) 如何让用户可以验证答案的对错: 推理路径或者子图 每个推理节点上的支撑事实 用于对比的其他可能答案和推理路径 35 认知图谱: 知识表示, 推理和决策 36 和认知科学的结合 Dual Process Theory (Cognitive Science) System 1 Intuitive System 2 Analytic 371. From Bengios NIPS2019 Ke

27、ynote 38 Reasoning w/ Cognitive Graph System 1: Knowledge expansion by association in text when reading System 2: Decision making w/ all the information System 1 Intuitive System 2 Analytic 39 CogQA: Cognitive Graph for QA An iterative framework corresponding to dual process theory System 1 extract

28、entities to build the cognitive graph generate semantic vectors for each node System 2 Do reasoning based on semantic vectors and graph Feed clues to System 1 to extract next-hop entities Question Quality caf Todd Phillips Gone in 60 seconds Old school Los Angeles Dominic Sena System 1 System 2 Cogn

29、itive Graph location featured in a number of Hollywood films, including Old School, Gone in 60 Seconds input input clues predict extract 40 Cognitive Graph: DL + Dual Process Theory 1.M. Ding, C. Zhou, Q. Chen, H. Yang, and J. Tang. Cognitive Graph for Multi-Hop Reading Comprehension at Scale. ACL19

30、. ? System 1: implicit knowledge expansion System 2: explicit decision 41 Cognitive Graph: DL + Dual Process Theory 1.M. Ding, C. Zhou, Q. Chen, H. Yang, and J. Tang. Cognitive Graph for Multi-Hop Reading Comprehension at Scale. ACL19. System 1: implicit knowledge expansion System 2: explicit decisi

31、on 42 Cognitive Graph: Representation, Reasoning, and Decision ? 43 认知与推理 Trillion-scale common-sense knowledge graph Tim Berners Lee Turing Award Winner Edward Feigenbaum Turing Award Winner * AI = Knowledge + Intelligence 1. J. Tang, J. Zhang, L. Yao, J. Li, L. Zhang, and Z. Su. ArnetMiner: Extrac

32、tion and Mining of Academic Social Networks. KDD08. pp.990-998. Big DataKnowledgeIntelligence 44 Related Publications Ming Ding, Chang Zhou, Qibin Chen, Hongxia Yang, and Jie Tang. Cognitive Graph for Multi-Hop Reading Comprehension at Scale. ACL19. Jie Zhang, Yuxiao Dong, Yan Wang, Jie Tang, and Mi

33、ng Ding. ProNE: Fast and Scalable Network Representation Learning. IJCAI19. Yukuo Cen, Xu Zou, Jianwei Zhang, Hongxia Yang, Jingren Zhou and Jie Tang. Representation Learning for Attributed Multiplex Heterogeneous Network. KDD19. Fanjin Zhang, Xiao Liu, Jie Tang, Yuxiao Dong, Peiran Yao, Jie Zhang,

34、Xiaotao Gu, Yan Wang, Bin Shao, Rui Li, and Kuansan Wang. OAG: Toward Linking Large-scale Heterogeneous Entity Graphs. KDD19. Qibin Chen, Junyang Lin, Yichang Zhang, Hongxia Yang, Jingren Zhou and Jie Tang. Towards Knowledge-Based Personalized Product Description Generation in E-commerce. KDD19. Yif

35、eng Zhao, Xiangwei Wang, Hongxia Yang, Le Song, and Jie Tang. Large Scale Evolving Graphs with Burst Detection. IJCAI19. Yu Han, Jie Tang, and Qian Chen. Network Embedding under Partial Monitoring for Evolving Networks. IJCAI19. Yifeng Zhao, Xiangwei Wang, Hongxia Yang, Le Song, and Jie Tang. Large

36、Scale Evolving Graphs with Burst Detection. IJCAI19. Jiezhong Qiu, Yuxiao Dong, Hao Ma, Jian Li, Chi Wang, Kuansan Wang, and Jie Tang. NetSMF: Large-Scale Network Embedding as Sparse Matrix Factorization. WWW19. Jiezhong Qiu, Jian Tang, Hao Ma, Yuxiao Dong, Kuansan Wang, and Jie Tang. DeepInf: Model

37、ing Influence Locality in Large Social Networks. KDD18. Jiezhong Qiu, Yuxiao Dong, Hao Ma, Jian Li, Kuansan Wang, and Jie Tang. Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec. WSDM18. Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. ArnetMiner: Extraction and Mining of Academic Social Networks. KDD08. For more, please check here 45 Jie Tang, KEG, Tsinghua U Download all data & Codes Thank you！