《经合组织:2023科学研究中的人工智能:挑战、机遇和未来报告(英文版)(300页).pdf》由会员分享,可在线阅读,更多相关《经合组织:2023科学研究中的人工智能:挑战、机遇和未来报告(英文版)(300页).pdf(300页珍藏版)》请在三个皮匠报告上搜索。
1、Artificial Intelligence inScienceCHALLENGES,OPPORTUNITIES ANDTHEFUTURE OFRESEARCHArtificial Intelligence in ScienceCHALLENGES,OPPORTUNITIES AND THE FUTURE OF RESEARCHThe Executive Summary and Chapter entitled“Artificial intelligence in science:Overview and policy proposals”wereapproved by the Commit
2、tee on Scientific and Technological Policy at its 122nd Session on 22-24 March 2023 andprepared for publication by the OECD Secretariat.The essays set out in Parts I to IV of this document are under the responsibility of the authors named and the opinionsexpressed and arguments employed therein are
3、their own.The essays benefited from input and comments from theOECD Secretariat and CSTP delegates.The essays should not be reported as representing the views of the OECD orof its member countries.This document,as well as any data and map included herein,are without prejudice to the status of or sov
4、ereignty overany territory,to the delimitation of international frontiers and boundaries and to the name of any territory,city or area.Please cite this publication as:OECD(2023),Artificial Intelligence in Science:Challenges,Opportunities and the Future of Research,OECD Publishing,Paris,https:/doi.or
5、g/10.1787/a8d820bd-en.ISBN 978-92-64-44154-5(print)ISBN 978-92-64-44621-2(pdf)ISBN 978-92-64-92820-6(HTML)ISBN 978-92-64-33228-7(epub)Photo credits:Cover PopTika/S.Corrigenda to OECD publications may be found on line at:www.oecd.org/about/publishing/corrigenda.htm.OECD 2023The use of this work,wheth
6、er digital or print,is governed by the Terms and Conditions to be found at https:/www.oecd.org/termsandconditions.PREFACE 3 ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 Preface Rarely a week passes without announcements that artificial intelligence(AI)has achieved new capabilities.Since the arrival
7、of generative AI,ChatGPT and subsequent large language models after many of the contributions to this book were written-discussion of AIs proliferating uses and their implications is increasingly visible in mainstream media.The economic,business,labour market and societal ramifications of AI now occ
8、upy the attention of firms,professional bodies,governmental and non-governmental organisations.Indeed,most governments in OECD countries have national AI strategies.Amid these developments,and except for specialised journals,less consideration has been given to the role of AI in research.This may be
9、 inevitable,as science is a specialised field.However,raising the productivity of research may be the most valuable of all the uses of AI.Being able to discover more scientific knowledge,helping science become more efficient,and doing this more quickly,will strengthen the foundations critical to add
10、ressing global challenges.Applying AI to research could be as transformative as the rise of systematised and institutionalised research and development in the post-war era.Preparing for new contagions,generating technologies that elevate living standards,countering the diseases of ageing,producing c
11、lean energy,creating environmentally benign materials,and other overarching goals,all require technologies and innovations that emerge from science.In this context,it gives us great pleasure to present this publication,Artificial Intelligence in Science:Challenges,Opportunities and the Future of Res
12、earch.Gathering the views of leading practitioners and researchers,but written in non-technical language,this publication is addressed to a wide readership,including the public,policymakers,and stakeholders in all parts of science.Among other topics examined are:AIs current,emerging and possible fut
13、ure uses in science,including a number of rarely discussed applications;where progress in AI is needed to better serve science;changes in the productivity of science;and,measures to expedite the uptake of AI in developing-country research.A distinctive contribution is the books examination of polici
14、es for AI in science.Policymakers and actors across research systems can do much to maximise the society-wide benefits of AI in science,deepening AIs use in science,while also addressing the fast-changing implications of AI for research governance.This publication is the fruit of a collaboration bet
15、ween our two organisations.The OECDs Directorate for Science,Technology and Innovation undertook the substantive work,under the aegis of its Committee for Scientific and Technological Policy.The publication and the wider project of which it is a part have been made possible thanks to financial and o
16、ther support from the Fondation IPSEN(https:/ works to improve living conditions by disseminating scientific knowledge to the public and promoting exchanges within the scientific community.James A.Levine,President,Fondation IPSEN Andrew Wyckoff,Director,OECD Directorate for Science,Technology and In
17、novation 4 FOREWORD ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 Foreword In late 2019 the OECD concluded an agreement with the Fondation IPSEN,which would provide financial support to work on artificial intelligence(AI)and the productivity of science.The context was one in which some scholars had a
18、rgued that the productivity of science may be stagnating,or even in decline.One aim of the project was to update and significantly expand previous work on AI in science conducted under the aegis of the Committee on Scientific and Technological Policy(CSTP).This prior work included a chapter in the 2
19、018 edition of the OECD Science,Technology and Innovation Outlook,titled“Artificial intelligence and machine learning in science”.A session on the growing importance of AI in science was also organised on 23 February 2022 the second OECD AI WIPS Conference.The first output of the project was a works
20、hop “AI and the Productivity of Science”held from 29 October to 5 November 2021.The workshop gathered over 80 leading experts to explore topics highlighted in this book.The workshop was filmed and can be viewed here https:/ project update was discussed at the 120th Session of the CSTP on 6-7 April 2
21、022.Analysis of numerous issues underpinning a discussion of policies for AI in science necessarily draws on prior CSTP examinations of topics bearing on data-intensive science.These topics include,among others:The changing demand for and nature of digital skills in the scientific workforce(see,in p
22、articular,the report“Building digital workforce capacity and skills for data-intensive science”,https:/doi.org/10.1787/e08aa3bb-en).Access to public research data(see the Recommendation of the Council concerning Access to Research Data from Public Funding,https:/legalinstruments.oecd.org/en/instrume
23、nts/OECD-LEGAL-0347).Many of the issues raised in this publication are also relevant to CSTPs current and upcoming work streams,especially in connection with the role of science and technology in sustainable transitions,as well as technology governance,skills,and citizen engagement in science.Work o
24、n AI in science is one among a wide set of AI-related topics being examined by the OECD,overviews of which can be found at the OECD AI Policy Observatory.ACKNOWLEDGEMENTS 5 ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 Acknowledgements This publication was edited by Alistair Nolan from the OECD Direc
25、torate for Science Technology and Innovation.Alistair Nolan also wrote the opening overview and synthesis of policy recommendations.Special thanks are due to several experts who provided ideas and advice through much of the process of preparing this book,and for commenting on various of the essays.A
26、mong this group are Marjorie Blumenthal,William Clements,Jeremy Frey,Aishik Ghosh,Dominique Guellec,Ross King,Isabelle Ryle,and Hector Zenil.Valuable comments on parts of the book were had from Jesus Anton,Jonathan Brooks,Alessandra Colecchia,Diogo Machado,Daniel Opalka,Carthage Smith,and Pierre War
27、nier.Thanks are due to all the authors of the papers in this publication,who gave freely of their time and insights.Some of the essays also benefitted from the assistance of third parties,as follows.For helping to develop the Elicit model described in the essay“Elicit:Language models as research too
28、ls”,the authors thank Ben Rachbach,Amanda Ngo,Eli Lapland,Justin Reppert,Luke Stebbing,Melissa Samworth,and James Brady.As concerns the essay“AI in drug discovery”,Kristof Zsolt Szalay extends thanks to Andreas Bender,Krishna Bulusu,Abraham Heifets and Aviad Tsherniak for insights into the state of
29、the field,as well as to Andreas Bender and Daniel V.Veres for expert reviews.With respect to the essay“Declining R&D efficiency Evidence from Japan”,Tsutomu Miyagawa expresses gratitude to Takayuki Ishikawa.With regards to the essay“The end of Moores Law?Innovation in computer systems continues at a
30、 high pace”,Henry Kressel acknowledges valuable discussions with William Janeway.Sylvain Fraccola helped generate graphics,and Mark Foss copy edited the entirety of the text,with support from Cline Colombier-Maffre.Thanks are due to all the participants and contributors to the workshop“AI and the Pr
31、oductivity of Science”,held from 29 October to 5 November 2021.Celia Valeani managed the organisation of that event.Angela Gosmann,Beatrice Jeffries and Blandine Serve kindly made the text ready for publication.Lastly,the essay“Quantifying the cognitive extent of science and how it has changed over
32、time(and across countries)”was made possible in part thanks to support from the United States Air Force Office of Scientific Research,and a Grant Thornton Fellowship.6 TABLE OF CONTENTS ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 Table of contents Preface 3 Foreword 4 Acknowledgements 5 Executive s
33、ummary 10 Artificial intelligence in science:Overview and policy proposals by Alistair Nolan 13 Part I Is science getting harder?49 Are ideas getting harder to find?A short review of the evidence by Matt Clancy 51 The end of Moores Law?Innovation in computer systems continues at a high pace by Henry
34、 Kressel 58 Is technological progress in US agriculture slowing?by Matt Clancy 62 Erooms Law and the decline in the productivity of biopharmaceutical R&D by Jack W.Scannell 70 Is there a slowdown in research productivity?Evidence from China and Germany by Philipp Boeing and Paul Hnermund 79 Declinin
35、g R&D efficiency:Evidence from Japan by Tsutomu Miyagawa 85 Quantifying the“cognitive extent”of science and how it has changed over time and across countries by Staa Milojevi 89 TABLE OF CONTENTS 7 ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 What can bibliometrics contribute to understanding resear
36、ch productivity?by Giovanni Abramo and Ciriaco A.DAngelo 95 Part II Artificial intelligence in science today 101 How can artificial intelligence help scientists?A(non-exhaustive)overview by Aishik Ghosh 103 A framework for evaluating the AI-driven automation of science by Ross King and Hector Zenil
37、113 Using machine learning to verify scientific claims by Lucy Lu Wang 121 Robot scientists:From Adam to Eve to Genesis by Patrick Courtney,Ross King and Oliver Peter 129 From knowledge discovery to knowledge creation:How can literature-based discovery accelerate progress in science?by Gus Hahn-Powe
38、ll,Dimitar Hristovski,Yakub Sebastian and Neil R.Smalheiser 140 Advancing the productivity of science with citizen science and artificial intelligence by James Bibby,Luigi Ceccaroni,Paul Flemons,Alexis Joly,Katina Michael,Jessica L.Oliver and Erin Roger 148 What can artificial intelligence do for ph
39、ysics?by Sabine Hossenfelder 155 AI in drug discovery by Kristof Z.Szalay 158 Data-driven innovation in clinical pharmaceutical research by Joshua New 167 Applying AI to real-world health-care settings and the life sciences:Tackling data privacy,security and policy challenges with federated learning
40、 by Mathieu Galtier and Darius Meadon 171 Part III The near future:challenges and ways forward 179 Artificial intelligence in scientific discovery:Challenges and opportunities by Ross King and Hector Zenil 181 8 TABLE OF CONTENTS ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 Machine reading:Successes
41、,challenges and implications for science by Jesse Dunietz 188 Interpretability:Should and can we understand the reasoning of machine-learning systems?by Hugh M.Cartwright 200 Combining collective and machine intelligence at the knowledge frontier by Aleks Berditchevskaia and Eirini Malliaraki 206 El
42、icit:Language models as research tools by Jungwon Byun and Andreas Stuhlmller 214 Democratising artificial intelligence to accelerate scientific discovery by Joaquin Vanschoren 224 Is there a narrowing of AI research?by Joel Klinger and Juan Mateos-Garcia 230 Lessons from shortcomings in machine lea
43、rning for medical imaging by Veronika Cheplygina and Gal Varoquaux 238 Part IV Artificial intelligence in science:Implications for public policy 243 Artificial intelligence for science and engineering:A priority for public investment in research and development by Tony Hey 245 The importance of know
44、ledge bases for artificial intelligence in science by Ken Forbus 251 High-performance computing leadership to enable advances in artificial intelligence and a thriving compute ecosystem by Mallikarjun Shankar,Georgia Tourassi and Feiyi Wang 257 Improving reproducibility of artificial intelligence re
45、search to increase trust and productivity by Odd Erik Gundersen 262 AI and scientific productivity:Considering policy and governance challenges by Kieron Flanagan,Priscila F.De Oliveira and Barbara Ribeiro 271 TABLE OF CONTENTS 9 ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 Part V Artificial intelli
46、gence,science and developing countries 279 Artificial intelligence and development projects:A case study in funding mechanisms to optimise research excellence in sub-Saharan Africa by Davor Orli and John Shawe-Taylor 281 Artificial intelligence for science in Africa by Gregg Barrett 287 Artificial i
47、ntelligence,developing-country science and bilateral co-operation by Peter M.Addo 294 10 EXECUTIVE SUMMARY ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 Executive summary Accelerating the productivity of research could be the most economically and socially valuable of all the uses of artificial intel
48、ligence(AI).While AI is penetrating all domains and stages of science,its full potential is far from realised.Policy makers and actors across research systems can do much to accelerate and deepen the uptake of AI in science,magnifying its positive contributions to research.This will support the abil
49、ity of OECD countries to grow,innovate and address global challenges,from climate change to new contagions.Ambitious multidisciplinary programmes can promote progress Broad multidisciplinary programmes are needed that bring together computer and other scientists with engineers,statisticians,mathemat
50、icians and others to solve challenges using AI.Among other measures,dedicated government funding is required.It needs to be allocated using processes that encourage broad collaboration,rather than siloed funding for individual disciplines.One priority is to foster interaction between roboticists and
51、 domain experts.Laboratory robots could revolutionise some domains of science,lowering the cost and hugely increasing the pace of experimentation.Governments can encourage and support visionary initiatives with long-term impact.Initiatives such as the Nobel Turing Challenge to build autonomous syste
52、ms capable of world-class research can inspire collaboration and co-ordination in science,to help focus efforts on global challenges,drive agreement on standards and attract young scientists to such ambitious endeavours.It is important to increase access to high-performance computing(HPC)and softwar
53、e for advances in AI and science.The provision of computing resources by large tech companies is helpful,but this has important gaps,and less well-funded research groups could fall behind.For academics to be competitive using state-of-the-art HPC/AI computing resources from commercial cloud provider
54、s is in most cases unrealistically expensive.National laboratories and their computing infrastructures,in collaboration with industry and academia,could address the gaps and help to develop training materials for institutions of tertiary education.Countries at the forefront of the field,including th
55、e United States and leaders in the European Union,may also collaborate on policy frameworks to make resources available from a shared pool.Updating curricula could assist.For example,using already proven AI-enabled techniques,students could be taught how to search for new hypotheses in existing scie
56、ntific literature.The standard biomedical curriculum provides no such training.New integrative PhD programmes and/or industry research programmes based on knowledge synthesis aided by AI could also help.Governments can take steps to increase the availability of open research data and to harness the
57、power of data across various fields,from health to climate.Examples include Europes Health Data Space,and GAIA-X,which aims to build a federated data infrastructure for Europe.Research centres can be helped to adopt systems such as federated learning that can apply AI to sensitive data held by multi
58、ple parties without compromising privacy.Another challenge is to make laboratory instruments EXECUTIVE SUMMARY 11 ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 more interoperable via standardised interfaces.Governments could bring laboratory users,instrument suppliers and technology developers togeth
59、er and incentivise them to achieve this goal.Public R&D can be used to advance the field Public research and development(R&D)can target areas of research where breakthroughs are needed to deepen AIs uses in science and engineering.Research goals include going beyond current models based on large dat
60、asets and high-performance computing,and to find ways to automate the large-scale creation of findable,accessible,interoperable and reusable(FAIR)data.Another target could be to advance AutoML automating the design of machine-learning models to help address the scarcity and high cost of AI expertise
61、.Research challenges could be organised around AutoML for science,and research could be funded that involves applying AutoML in AI-driven science.Support should also be given for the development of open platforms(such as OpenML and DynaBench)that track which AI models work best for a wide range of p
62、roblems.Public support is needed to make such platforms easier to use across many scientific fields.Public R&D could help foster new,interdisciplinary,blue-sky thinking.For instance,natural language processing(NLP)can help to work with the enormous growth of scientific literature.However,current per
63、formance claims are overstated.Todays research in NLP also offers limited incentives for the sort of high-risk,speculative ideation that breakthroughs may need.Research centres,funding streams and/or publication processes could be set up to reward novel methods even if these are at a nascent stage.K
64、nowledge bases organise the worlds knowledge by mapping the connections between different concepts,drawing on information from many sources.Governments should support an extensive programme to build knowledge bases essential to AI in science,a need that will not be met by the private sector.Research
65、 could work towards creating an open knowledge network to serve as a resource for the whole AI research community.Relatively small amounts of public funding could help bring together AI scientists,scientists from multiple domains and professional societies along with volunteers to build the foundati
66、ons for AI to utilise and communicate professional and commonsense knowledge.The thematic diversity of research on AI appears to be narrowing and is increasingly driven by the compute-and data-intensive approaches that dominate in large tech companies.Bolstering public R&D might make the field more
67、diverse and help to grow the talent pool.Funders could pay special attention to projects that explore new techniques and methods separate from the dominant deep-learning paradigm.Meanwhile,policy makers could support research to examine and quantify losses of technological resilience,creativity and
68、inclusiveness brought about by a narrowing of AI research and the possible implications of the increasing dominance of industry in AI research.Much of AI in science involves teaming with people,but funders could also help develop specialised tools to enhance collaborative human-AI teams,and to integ
69、rate these tools into mainstream science.Combining the collective intelligence of humans and AI is important,not least because science is now carried out by ever-larger teams and international consortia.Investment in this field of research has lagged other topics in AI.Among other fields,progress is
70、 needed in applying machine learning to medical imaging.Failures during COVID-19 were considerable.As in other uses of machine learning in science,incentives are needed to encourage research on methods with greater validation.Funding should involve more rigorous evaluation practices.12 EXECUTIVE SUM
71、MARY ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 Research governance matters Policy bodies should systematically evaluate the impacts of AI on everyday scientific practice,including on human-AI teaming,work,career trajectories and training where important changes could occur.Funding calls could req
72、uire such assessments,and funders and policy makers should establish response mechanisms to act on the insights gathered.Among other measures,funders and policy makers could establish and support new independent fora for ongoing dialogue about the changing nature of scientific work and its impacts o
73、n research productivity and culture.The deployment of large language models(LLMs),such as ChatGPT,demands attention from policy makers as their consequences are currently uncertain.LLMs could lead to more shallow work by making this easier,blur concepts of authorship and ownership,and possibly creat
74、e inequalities between speakers of high-and low-resource languages.However,LLMs and other forms of AI could also aid governance processes,for instance in supporting peer review a possibility that requires more study and testing.Policy should address the potential dangers entailed in dual use of AI-p
75、owered drug discovery.Little attention has been paid to the imminent dangers of being able to automate the design,testing and making of extremely lethal molecules(and there will be other dual use research to consider,too).Policy makers and other actors in the research system need to assess which of
76、the possible governance arrangements will best protect the public good.Policy makers and their staff need more know-how to help decide what sort of technology initiatives to support Existing social networks and platforms could be used to help spread emerging practices.Social platforms such as Academ
77、ia.edu and the Loop community could be used as testbeds for experimenting with combined human-AI knowledge discovery,idea generation and synthesis,and for propagating and evolving such approaches as literature-based discovery.Steps are likewise needed to improve the reproducibility of AI research.Am
78、ong other actions,public funding agencies can require code,data and metadata to be shared freely with third parties,allowing them to run experiments on their own hardware.There is a strong case for sub-Saharan Africa,and possibly other developing regions,to receive much greater funding for AI in sci
79、ence.Development co-operation can help countries to advance open science,frame data protection legislation,improve digital infrastructures,strengthen overall AI readiness and support Africas own emerging initiatives,including indigenous development of data,software and technology.Projects with devel
80、oping countries for AI in science can be mutually beneficial,and low-cost models of support have been proven.Development co-operation can also help create and support centres of research excellence.ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS 13 ARTIFICIAL INTELLIGENCE IN SCIENCE
81、 OECD 2023 A.Nolan,Organisation for Economic Co-operation and Development Introduction This book addresses the current and emerging roles of artificial intelligence(AI)in science.Accelerating the productivity of research could be the most economically and socially valuable of all AIs uses.AI and its
82、 various subdisciplines are pervading every field and stage of the scientific process.Advances in AI have led to an outpouring of creative uses in research.However,AIs potential contribution to science is far from realised,and the impact of some widely hailed achievements may be less than is general
83、ly thought.AI,for instance,contributed little to research and treatment during the COVID-19 pandemic.Moreover,policy makers and other actors in research systems can do much to speed and broaden the uptake of AI in science,and to magnify its positive contributions to science and society.The books mai
84、n contributions are to:Describe,in terms amenable to non-technical readers,AIs current and possible future uses in science.Help raise awareness of the roles that public policy could play in amplifying AIs positive impact on science,while also managing governance challenges.Draw attention to applicat
85、ions of AI in science and related topics that may be unfamiliar to some lay readers.Such applications include,among others,AI and collective intelligence,AI and laboratory robotics,AI and citizen science,developments in scientific fact-checking,and the emerging uses of AI in research governance.Rela
86、ted topics include the thematic narrowing of AI research and the reproducibility of AI research.Assess what AI cannot yet do in science,and areas of progress still required.Examine empirical claims of a slowdown in the productivity of science,engaging the views of domain experts and economists.Consi
87、der the implications of AI in science for developing countries,and the measures that could be taken to expedite uptake in developing-country research.This chapter proceeds as follows:the opening sections discuss why raising research productivity is important,whether through using AI or other means.T
88、he key issues concern economic effects,addressing critical knowledge gaps,summarising the evidence for and countering possible sources of drag on research productivity.In so doing,the text outlines why some scholars have argued that the productivity of science may be stagnating.To be clear,the claim
89、 is not that progress in science is slowing,but that it is becoming harder to achieve.The chapter continues with summaries of the books 34 essays.The summaries are presented under five broad headings.These correspond to the five parts of the book:Is science getting harder?Artificial intelligence in
90、science:Overview and policy proposals 14 ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 Artificial intelligence in science today The near future:Challenges and ways forward Artificial intelligence in science:Implications for public polic
91、y Artificial intelligence,science and developing countries.The salient policy implications and suggestions are highlighted in text boxes.AI and the productivity of science:Why does this matter?The productivity of science is of critical interest for many reasons.Three are described here:economic;the
92、need to close gaps in significant areas of scientific knowledge;and claims of slowing research productivity.Economic implications of research productivity Economists have established a fundamental relationship between innovation,which draws from basic research,and long-term productivity growth.The e
93、conomic effects of COVID-19,sluggish macro-economic conditions in most OECD countries,burgeoning public debt and population ageing have all added urgency to the quest for growth.The sheer scope of sciences role in modern economies is easily underestimated.By one assessment,industries reliant just on
94、 physics research,including electrical,civil and mechanical engineering,as well as computing and other industries,contribute more to Europes economic output and gross value added than retail and construction combined(European Physical Society,2019).The scope of any feedthrough from changes in resear
95、ch productivity will be correspondingly broad.Recent analysis by the International Monetary Fund(IMF)based on patents data suggests that basic scientific research diffuses to more sectors in more countries and for a longer time than commercially oriented applied research(IMF,2021).Theory also sugges
96、ts that growth stemming from more productive R&D will be more lasting than that spurred by automation in final goods production,which can yield a one-time increase in the rate of growth(Trammell and Korinek,2020).Much basic and essential scientific knowledge is lacking In many domains,science is adv
97、ancing rapidly.In 2022,there was widely publicised progress in fields as diverse as astronomy,with unprecedented images from the James Web telescope,the development of a nasal vaccine for COVID-19 and the first laboratory-based controlled fusion reaction.However,it is also the case that both old sci
98、entific questions endure and new ones arise continually.To take just three examples:After decades of climate modelling,uncertainty persists.Important uncertainties exist on such issues as tipping points(e.g.inversion of the flows of cold and hot oceanic waters),when changes could become irreversible
99、(e.g.melting of West Antarctic or Greenland ice-shelves),and the quantitative role of plants and microbes in the carbon cycle(plants and microbes cycle some 200 billion tons of carbon a year,compared to anthropogenic production of around 6 billion tons).Many elementary cellular processes are not und
100、erstood.For instance,the process by which Escherichia coli(a bacterium)consumes sugar for energy is one of the most basic biological functions.It is also important for industry in designing microbial biocatalysts that use carbohydrates in biomass.However,how the process operates has not been fully e
101、stablished(even though research on the subject was first published over 70 years ago).ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS 15 ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 Around 55 million people worldwide currently suffer from Alzheimers disease or other dementias.While
102、studies have identified several risk factors for Alzheimers disease from age,to head injury,to high cholesterol the cause of the disease is still unknown(and treatments are missing).More productive science will also set foundations for breakthroughs in innovation,especially in some crucial fields.Fo
103、r instance,many of the antibiotics in use today were discovered in the 1950s,and the most recent class of antibiotic treatments was discovered in 1987.Innovation in the energy sector is also essential for achieving low-emission economic growth.But todays leading energy generation technologies were m
104、ostly invented over a century ago.The combustion turbine was invented in 1791,the fuel cell in 1842,the hydro-electric turbine in 1878 and the solar photo-voltaic cell in 1883.Even the first nuclear power plant began operating over 60 years ago(Webber et al.,2013)(although the performance of these t
105、echnologies has of course improved over time).By accelerating science and innovation,AI could help to find solutions to global challenges such as climate change(Boxes 1 and 2),and the diseases of ageing.Box 1.Artificial intelligence,materials science and net zero Materials science is central to new
106、technologies needed to address climate change.Among many possibilities,new materials promise more efficient solar panels,better batteries,lightweight metal alloys for more fuel-efficient vehicles,carbon-neutral fuels,more sustainable building materials and low-carbon textiles.Progress in materials s
107、cience may also create substitutes for materials with fragile supply chains,including rare earth elements.Assisted by an open-source research community and open-access databases,AI is ushering in a revolution in materials science,quickly and efficiently exploring large datasets for arrangements of a
108、toms that yield materials with user-desired properties,while optimising aspects of experimentation.Materials discovery has traditionally been slow and uncertain,based on trial-and-error examination of many sometimes millions of candidate samples.The research sometimes takes decades.However,the new c
109、ombinations of high-performance computing,AI and laboratory robots can greatly accelerate discovery(later essays in this book explore robotics in science).Service(2019)describes some materials discovery processes being compressed from months to just a few days.One lab robot conducts 100 000 experime
110、nts a year,producing five years of experiments in just two weeks(Grizou et al.,2020).The urgency of achieving net zero underscores the importance of accelerating materials discovery.Faster discovery can also encourage the private sector to invest in materials R&D,as returns are more likely to be had
111、 within commercial timeframes.Lowering costs per experiment can encourage more creative research,as the risk of failure is mitigated if a broad and fast-running portfolio of experiments is possible.In addition,faster discovery might help junior researchers to establish themselves(Correa-Baena et al.
112、,2018).These advances in materials science require contributions from many disciplines,including computer scientists,roboticists,electronics engineers,physical scientists and materials researchers.Policies and approaches that facilitate cross-disciplinary research and exchange of ideas could help.16
113、 ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 Box 2.Catalysing research at the intersection of climate change and machine learning Climate Change AI(CCAI)1 is a not-for-profit organisation bringing together volunteers from academia and
114、 industry.One of its most significant offerings is a catalogue2 of numerous research questions across many areas in science,engineering,industry and social policy where AI could make a dent in climate problems.CCAI also cultivates a community of many researchers,engineers,policy makers,investors,com
115、panies and non-governmental organisations,many of which are applying AI techniques to scientific problems.1.See https:/www.climatechange.ai/.2.See https:/www.climatechange.ai/summaries.AI also matters because science itself may be becoming harder Claims of a slowdown in science are not new.More than
116、 50 years ago,Bentley Glass,former President of the American Academy for the Advancement of Science,asserted that“There are still innumerable details to fill in,but the endless horizons no longer exist”(Glass,1971).Recently,attention to a purported stagnation in research productivity has been spurre
117、d by Bloom et al.(2020)and other papers.Matt Clancy,in this book,reviews the relevant economic and technology-specific studies,and concludes that while quantification of research productivity is conceptually and methodologically complex,and not uncontentious,science has by some measures become harde
118、r.If science were indeed to become harder then,other conditions unchanged,governments would be forced to spend more to achieve existing rates of growth of useful scientific output.Timeframes could be lengthened for achieving scientific progress needed to address todays global challenges.And for inve
119、stments in science equivalent to todays,ever-fewer increments of new knowledge will be available with which to counter unforeseen events with negative global ramifications,from new contagions to novel crop diseases.It is helpful to consider the arguments made by the scholars who contend that science
120、 is getting harder.These are summarised in Box 3.Examining the explanations why this might be can help to pinpoint how AI could help.Essays in this book examine various issues relevant to the effects of bad incentives in science systems,argument(1)in Box 3.Those essays explore such issues as AI in s
121、cientific fact-checking,and AI in governance processes(see the contributions of Varoquaux and Cheplygina;Flanagan,Ribeiro and Ferri;and Gundersen Wang).In connection with argument(2)in Box 3 a more limited involvement of the private sector in basic research AI can incentivise some areas of private r
122、esearch and development.This is because AI can help conduct some parts of science more rapidly,better aligning with commercial investment horizons.AI has also spurred the creation of firms specialised in doing basic science for larger corporates(see essays by Szalay;Ghosh;and by King,Peter and Court
123、ney).AI in science is also relevant to argument(3)the economic limits on discovery as it can lower costs in some stages of science,especially laboratory experimentation.In addition,potentially large savings of scientists time could come from compressing the duration of research projects for instance
124、 by using increasingly capable AI-driven research assistants(the subject of the essay by Byun and Stuhlmller).Argument(4)in Box 3 relates to the need for larger teams in science.The essay on AI and collective intelligence by Malliaraki and Berditchevskaia considers how to harness the capabilities of
125、 such teams,as does the essay on AI and citizen science by Ceccaroni and his colleagues.Furthermore,arguments relating to the burden of knowledge arguments(5)and(6)are explored from different viewpoints in essays on natural language processing applied to scientific texts(see the contributions of Dun
126、ietz;Wang;Byun and Stuhlmller;and Smalheiser,Hahn-Powell,Hristovski and Sebastian).ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS 17 ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 Box 3.Why might science get harder?Researchers have posited reasons for an alleged decline in the produc
127、tivity research.While not exhaustive,the main arguments concern the following:1.Changes in scientific incentives.Among others,Bhattacharya and Packalen(2020)explore the role of citations in performance measurement and in shifting scientists rewards and behaviour toward incremental science,with high
128、rates of retraction,non-replicability and even fraud.2.A more limited engagement of the private sector in basic science(Arora et al.,2019).3.Economic limits on discovery.For example,the cost of the next generation LHC supercollider is estimated at EUR 21 billion.To generate energies needed to probe
129、smaller subatomic phenomena would be orders of magnitude more costly.4.As more prior and diverse science must be absorbed to make new breakthroughs,larger teams are needed.But larger teams seem less prone to make fundamental discoveries than small teams(Wu,Wang and Evans,2019).5.Scientists have reac
130、hed“peak reading”.By one account,100 000 articles on COVID-19 were published in the first year of the pandemic.Tens of millions of peer-reviewed papers exist in biomedicine alone.However,the average scientist reads about 250 papers a year(Noorden,2014).6.The sheer size of the corpus of scientific li
131、terature in different fields.In larger corpora,potentially important contributions cannot garner field-wide attention through gradual processes of diffusion(Chu and Evans,2021).7.As science progresses,it branches into new disciplines.Some breakthroughs require more inter-disciplinarity,but there is
132、friction at the boundaries between disciplines.8.There are a finite number of scientific laws.Once a law or artefact is discovered,science has to proceed to the next challenge.DNA,for example,can only be discovered once.Is science getting harder?Are ideas getting harder to find?A short review of the
133、 evidence Reviewing multiple studies,Matt Clancy concludes that,using diverse methodological and conceptual approaches,a constant supply of research effort(such as numbers of scientists)does not lead to a constant proportional increase in various proxies for technological capabilities(e.g.doubling t
134、he number of transistors on an integrated roughly every two years).There are few exceptions to the general finding that a constant proportional increase in metrics of interest has tended to require an increasing supply of research effort.Clancy also points to other measurement approaches based on th
135、e idea that progress is not just about squeezing the last drop of possibility from each technology,it is also,and perhaps mostly,about the creation of entirely new branches of technology.However,acknowledging this perspective,Bloom et al.(2020)showed that,at least in health,despite successive waves
136、of new technologies,from antibiotics to mRNA vaccines,etc.,saving a year of life has needed increasing research effort measured by the number of clinical trials or biomedical articles.Another measure of the effects of R&D relates to performance outcomes in private sector companies.Bloom et al.(2020)
137、examine sales,number of employees,sales per employee and market capitalisation 18 ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 and find here,too,that on average it takes more and more R&D effort by firms to maintain growth in these mea
138、sures.Clancy likewise discusses total factor productivity(TFP)the efficiency with which an economy combines inputs to create outputs as a broad measure of technological progress.Bloom et al.(2020)found that for the US economy,going back to the 1930s,growing R&D effort has been required to keep TFP i
139、ncreasing at a constant exponential rate.Miyagawa,in this book,arrives at a similar result for Japan,as do Boeing and Hnermund for Germany and the Peoples Republic of China(hereafter“China”).Another way to examine research productivity is to look at measures from science.Clancy discusses one approac
140、h which looked at the share of Nobel Prize winning awards that go to discoveries described in papers published in the preceding 20 years.Across all fields,this has fallen significantly.Clancy also describes studies that show a steady decline since the 1960s in the share of citations to more recent p
141、apers(those published in the preceding five or ten years),possibly suggesting a declining impact of recent scientific output.Patents share this pattern,and increasingly cite older scientific work.Clancy also explains why conceptual and methodological caveats apply to all the analyses.TFP,for instanc
142、e,can vary for reasons unrelated to science and technology,such as changes in the geographic mobility of workers.However,many papers employing diverse approaches arrive at converging conclusions.Nevertheless,Clancy closes by acknowledging that even if ideas are getting harder to find,society also se
143、ems to be trying harder to find them,causing science to advance.Other essays in this volume summarised below examine three fields of technology where Bloom et al.(2020)compared performance metrics with measures of research input and thereby argued for a decline in research productivity:namely Moores
144、 Law,agriculture and the biopharmaceuticals sector.However,the picture that emerges in the essays below is not quite as clear-cut as Bloom et al.(2020)suggest.The end of Moores Law?Moores Law,which has held since the 1960s,posits that transistor chip density doubles roughly every two years,with a co
145、rresponding decline in unit transistor cost.Bloom et al.(2020)suggest that an apparent slowing of Moores Law indicates a decline in the pace of innovation in electronics.Such a decline would have serious consequences,as microelectronics are central to practically all industrial products and systems.
146、However,Henry Kressel shows that while the ability to shrink transistors is reaching physical limits,fears of stagnation or decline in the power of computing systems are premature.He shows that other innovations additional to those tracked by Moores Law continue to improve the economic and technical
147、 performance of electronic systems.For instance,manufacturers are findings ways to improve energy efficiency,and developing three-dimensional architectures that make better use of the chip area.Good ideas are not running out.Nor is there evidence of declining interest in such research.At base,Kresse
148、ls essay contains an important generalisable message:measuring the progress of a technology-driven field with a single metric can mislead.Indeed,at present,while non-specialists focus on Moores Law,no reliable general metric of progress is available today because computing systems range so greatly i
149、n scale and functionality.Is technological progress in US agriculture slowing?Matt Clancy examines innovation in US agriculture and concludes that the case for a slowdown seems to hold whether measured with growth in yields over time or using more sophisticated methods,such as changes in TFP.The slo
150、wdown may stem from agriculture-specific factors,such as stagnating levels of R&D through much of the late 20th century.It may also be influenced by broader forces,such as slowing technological progress in non-farm domains that supply critical inputs to agriculture.Moreover,while this ARTIFICIAL INT
151、ELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS 19 ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 essay examines US agriculture,Clancy cites research suggesting that global productivity growth in agriculture fell from an average of 2%per year over the 2000s to 1.3%per year over the 2010s.Echoing Kr
152、essels point on the need for care in selecting metrics of progress,Clancy observes that changes in agricultural yield a focus of Bloom et al.has drawbacks.For example,almost all of US corn is genetically modified to confer resistance to a key pesticide(glyphosate).This helps farmers by making it les
153、s costly to control weeds,a benefit not captured in measures of yield.Similarly,an important dimension of agricultural innovation not typically included in TFP is the environmental sustainability of agricultural production,which may be improving.Erooms law and the decline in the productivity of biop
154、harmaceutical R&D Jack Scannell explores Erooms law,the observation that drug development becomes slower and more expensive over time.Scannell examines various metrics that show a significant decline in the productivity of biopharmaceutical R&D since the late 1990s(although with a slight uptick sinc
155、e 2010).He points out that DNA sequencing,genomics,high-throughput screening,computer-aided drug design and computational chemistry,among other advances,were widely adopted and/or became orders of magnitude cheaper between 1950 and 2010.However,over the same period,the number of new drugs approved b
156、y the US Food and Drug Administration(FDA)per billion US dollars of inflation-adjusted R&D fell roughly a hundredfold.Scannell suggests that levels of innovation in biopharma have fallen for several reasons.Arguably of greatest importance is the progressive accumulation of an inexpensive pharmacopoe
157、ia of effective generic drugs.When drugs patents expire,they become much cheaper but no less effective.An ever-expanding catalogue of cheap generic drugs progressively raises the competitive bar for new drugs in the same therapy area,eroding incentives for R&D.Such therapy areas hold meagre returns
158、for investment in“new ideas”,even if the ideas themselves have not become harder to find(there are many unexploited drug targets and therapeutic mechanisms and a vast number of chemical compounds).Scannell explains that R&D investment has been squeezed towards diseases where R&D has for long been le
159、ss successful,such as advanced Alzheimers,some metastatic solid cancers,etc.He observes that novel chemistry where AI can play a big role-is the most investible form of biopharmaceutical innovation because it can be protected by strong patents.However,the lack of good screening and disease models is
160、 a key constraint on drug discovery(a disease model is a biological system in the laboratory that mirrors a disease and its processes).A major reason for this shortage is economic:once the mechanism identified by a new disease model is publicly proven in trials in human patients,the information beco
161、mes freely available to competitors.AI will be incrementally helpful but not revolutionary in drug discovery Scannell considers that AI will help in drug R&D.However,its overall impact on industry-level productivity will likely be modest in the near term.This is because the areas with the most progr
162、ess in using AI such as drug chemistry are rarely relevant to the rate-limiting steps in drug development.Meanwhile,AI is less likely to yield solutions where gains in R&D productivity are most needed.A main reason for this is that much of the critical data is of insufficient quality.For example,too
163、 much of the published biomedical literature is false,irrelevant or both.Generating better biological data will help take advantage of AI,but doing so is costly and takes time.Is there a slowdown in research productivity?Evidence from China and Germany Philipp Boeing and Paul Hnermund provide eviden
164、ce for a decrease in research productivity in recent decades for China and Germany,following the methodology developed by Bloom et al.(2020)where it 20 ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 was argued that R&D efficiency,measure
165、d by economic productivity growth divided by the number of researchers,has declined in the United States.For Germany,R&D expenditures increased by an average of 3.3%per year during the period 1992-2017.Averaged over firm-level outcome measures,research productivity fell by 5.2%per year.This number i
166、s similar to that reported by Bloom et al.(2020)for the United States.These negative compound average growth rates imply that research effort must be doubled every 13 years to support constant rates of economic growth.The authors find that research productivity in China has declined much faster.The
167、effective number of researchers employed by publicly listed firms in the sample used increased by,on average,21.9%per year between 2001 and 2019.This significant expansion is not matched by increases in economic growth.The findings entail a drop in research productivity of 23.8%per year.However,if a
168、nalysis is restricted to the most recent decade(when China began large-scale R&D activities)research productivity fell by only 7.3%a year,a number closer to those found for Germany and the United States.Declining R&D efficiency:Evidence from Japan Tsutomu Miyagawa notes that while Japan has maintain
169、ed a ratio of R&D to gross domestic product(GDP)of around 3%for some time,R&D efficiency growth appears to have slowed.Adopting the methodology used in Bloom et al.(2020),Miyagawa and Ishikawa(2019)found that the efficiency of R&D in Japanese manufacturing and information services had fallen.Using m
170、ore recent data,Miyagawas essay in this volume examines two measures of R&D efficiency.The first is derived from a simple production function in which productivity depends on the stock of R&D.The second again follows the method of Bloom et al.(2020).Both measures show that R&D efficiency in Japan in
171、 the 2010s declined compared to the 2000s.Quantifying the“cognitive extent”of science and how it has changed over time and across countries Staa Milojevi approaches the measurement of research productivity in an entirely different way.She discusses trends in the“cognitive extent”of knowledge in scie
172、ntific literature.Milojevi quantifies the cognitive extent of scientific fields by using information on the number of unique phrases contained in the titles of journal articles.In a given body of literature,a smaller number of unique phrases would indicate a lot of repetition,and a smaller cognitive
173、 extent.A larger number of unique phrases suggests a wider range of concepts and a greater cognitive extent.Milojevi finds stagnation in cognitive extent since the mid-2000s.She also examines individual fields of research,showing that cognitive extent in physics,astronomy and biology is expanding,wh
174、ereas medicine is stagnating or even contracting.In addition.Milojevi compares cognitive extent across countries.She finds that while China was the biggest producer of scientific publications in 2019,its papers covered a smaller cognitive extent than many individual West European countries and Japan
175、.What can bibliometrics contribute to understanding research productivity?Giovanni Abramo and Ciriaco Andrea DAngelo discuss the strengths and weaknesses of the most popular bibliometric indicators used to assess research performance.They describe the well-known limits of evaluative bibliometrics:1)
176、publications may not be representative of all knowledge produced;2)bibliographic repertories do not cover all publications;and 3)citations are not always a certification of use.However,the authors underscore that bibliometrics is primarily concerned with research outputs.Understanding changes in res
177、earch productivity also requires measures of the associated research inputs,namely labour and capital.ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS 21 ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 Abramo and Andrea DAngelo present a proxy bibliometric indicator of research producti
178、vity that includes data on research inputs.They describe the first results of a longitudinal analysis of academic research productivity at a national level using such an indicator.This shows that productivity is increasing over time for Italian academics in most research fields.The authors call on g
179、overnments to support more useful national and international research productivity assessments by establishing mechanisms by which bibliometricians are provided with data on labour and capital inputs to research institutions.Artificial intelligence in science today How can artificial intelligence he
180、lp scientists?A(non-exhaustive)overview Aishik Ghosh observes that AI is being taken up in every domain and stage of science,from hypothesis generation to experiment design,monitoring and simulation,all the way to scientific publication and communication.In the future,AI may optimise many scientific
181、 workflows end-to-end from data collection to final statistical analysis(see the essay on laboratory robots by King,Peter and Courtney).Nonetheless,Ghosh explains that the potential impact of AI on science is a long way from being realised.The author sets out the main categories of AIs use in scienc
182、e.While typical machine-learning models are difficult to interpret a point repeated in other essays in the book they remain useful for tasks such as hypothesis generation,experiment monitoring and precision measurements.Models that create new data generative AI can assist with simulations,removing u
183、nwanted features from data and converting low-resolution,high-noise images into high-resolution,low-noise images,with many useful applications.In materials science,for example,AI can correctly enhance cheaper,low-resolution electron microscopic images into otherwise more expensive high-resolution im
184、ages.Unstructured data(e.g.satellite images,global weather data)have traditionally been a challenge because dedicated algorithms need to be developed to handle them.Deep learning(a class of machine learning,or ML)has been enormously effective in handling such data to solve unusual tasks.Innovations
185、in developing causal models to disentangle correlation from causation will provide huge benefits for the medical and social sciences.AI can also keep track of multiple uncertainties that accumulate through long scientific pipelines.One benefit of this is to make data acquisition more efficient by pr
186、ioritising data gathering where there is uncertainty.AI is also benefiting science in indirect ways,for instance by advancing mathematics.For example,towards the end of 2022 DeepMind announced it had used a technique known as reinforcement learning to discover how to multiply matrices more rapidly.B
187、eyond the main stages of research,AI is also more broadly useful to science.For example,some AI models have been developed to summarise research papers and a few popular Twitter bots regularly tweet these automated summaries.Ghosh also points to recent research on an AI-based method to present exper
188、imental measurements in physics to theoretical physicists more effectively.Box 4 considers AI in peer review.Box 4.AI and peer review:Semi-automating time-consuming processes Peer review consumes enormous scientific resources.By one estimate,just in the United States,and in 2020 only,the time cost o
189、f peer review was USD 1.5 billion(Aczel,Szaszi and Holcombe,2021).Experiments are underway to assess potential uses of AI in multiple aspects of research governance.Checco et al.(2022)describes one such study of AI-assisted peer review.The authors trained an AI 22 ARTIFICIAL INTELLIGENCE IN SCIENCE:
190、OVERVIEW AND POLICY PROPOSALS ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 model on 3 300 past conference papers and the associated review evaluations.When shown unreviewed papers the AI model could often predict the peer review outcome.Semi-automated peer review raises ethical and institutional cha
191、llenges.One possible problem is bias,for instance in propagating cultural and organisational features in the papers on which the AI is trained.However,AI can also reveal biases already operating in human-only peer review.Some uses of AI in peer review would be time saving and relatively uncontrovers
192、ial,such as in pre-peer review screening to detect early superficial problems in papers.This could be helpful to authors.In addition,removing such problems could lower the impact of first-impression bias and help peer reviewers to focus on papers scientific content.As Checco et al.explain,more study
193、 is needed of AI-enabled decision support.However,as the volume of scientific literature rapidly expands,the practical benefits of emerging AI systems could outweigh their potential disbenefits.Ghosh also describes possible dangers raised by AI in science.AI models sometimes malfunction in different
194、 ways than do traditional algorithms.Using deep learning,a robot trained to work with red,blue and green bottles in a laboratory,for example,may not generalise correctly to black bottles.Deep-learning models pick up subtle patterns in training data,including biases in simulations.And some bias mitig
195、ation techniques can lead to further unintended harm.In addition,the trend has been to develop large AI models that require enormous computing resources to train.As other authors in this book also note,this can create problems for research groups with smaller budgets.In November 2022,following Ghosh
196、s essay,OpenAI released ChatGPT.Many professions are now debating how ChatGPT and other large language models(LLMs)will affect their futures.Uses to increase the productivity of knowledge work are many:quickly and automatically writing diverse materials,from presentations to essays;improving the qua
197、lity of written language;reducing language barriers for non-native speakers;rapid summarisation;writing computer code;and fostering creativity through dialogue.Evidently,such benefits are also available to science.However,as Byun and Stuhlmller discuss later in this book,LLMs like ChatGPT and Galact
198、ica often gets things wrong.These authors emphasise the need for processes of evaluation to ensure accuracy as applications are scaled up.They also observe that LLMs risk making superficial work more abundant,as well as creating inequalities,for instance between English-speaking and other users.In a
199、 commentary in Nature,van Dis et al.(2023)draw attention to the need for research systems to address governance challenges posed by LLMs(Box 5).Box 5.What do ChatGPT and future LLMs imply for the research community?Van Dis et al.(2023)call for an international forum on the development and use of LLM
200、s for research.The goal would be to answer questions essential to research governance.Among the questions they highlight are the following:Which academic skills remain essential for researchers,and in what ways might scientists training need to change?Which steps in an AI-assisted research process s
201、hould require human verification?How should research integrity and other policies change?(for example,ChatGPT does not reliably cite original sources,and researchers might use it without giving credit to earlier work.This might be unintentional).Most LLMs are proprietary products of large tech compa
202、nies.Should this spur public investment in open-source LLMs?How could this best be done,given the much larger resources available to tech companies?ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS 23 ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 What quality standards should be expect
203、ed of LLMs(such as source crediting and transparency)?Which stakeholders should be responsible for the standards?How should LLMs be used to enhance principles of open science?How can researchers ensure that LLMs do not create inequities in research?What legal implications do LLMs have for scientific
204、 practice(for example,laws and regulations related to patents,copyright and ownership)?A framework for evaluating the AI-driven automation of science Ross King and Hector Zenil hold that the future of science,especially experimental science,lies in AI-led closed-looped automation systems.Automation
205、has accelerated productivity in many industries,and could do so again in science.Citing a prediction of the physics Nobel Laureate Frank Wilczek that in 100 years the best physicist would be a machine,the authors underscore the importance of developing autonomous systems to improving human welfare(K
206、ing himself co-developed the robot scientist“Adam”,the first machine to autonomously discover scientific knowledge,generating a hypothesis which it then tested using laboratory automation,King et al.2009).Robotic systems are already accelerating science in genetics and drug discovery(the essay by Ki
207、ng,Peter and Courtney explores the role of robot scientists in greater depth).The authors describe a possible future in which human scientists will decide how to work with the AI scientists and how much scope AI will have to define its own problems and solutions.Synergies could arise in which AI ide
208、ntifies research where humans have been biased or else highlights areas of research that human scientists have failed to explore.A progressive scale of automation in science King and Zenil set out a framework of automation levels in science based on the quantity and quality of input and execution re
209、quired from human scientists.An analogy they draw is to the 1 to 5 classification of automation in cars set by The Society of Automotive Engineers.In science,at Level 1,humans still describe a problem in full,but machines do some data manipulation or calculation.A case might be made for dating the a
210、chievement of Level 1 to the 1950s and 1960s,with the advent of the first theorem provers.Level 5 corresponds to full automation,covering all levels of discovery with no human intervention.Today,in certain areas of laboratory-based science,some systems have reached Level 4.This is the stage where sc
211、ience can be greatly accelerated.For instance,a robot chemist developed at the University of Liverpool moves about the laboratory guided by Lidar and touch sensors.An algorithm lets the robot explore almost 100 million possible experiments,choosing which to do next based on previous test results.The
212、 robot can operate for days,stopping only to charge its batteries.For such machines,there is almost no human intervention except for providing consumables.The authors are part of the“Nobel Turing Challenge”.This challenge is exploring how to develop AI systems capable of making Nobel-quality scienti
213、fic discoveries highly autonomously by 2050.As they report,participants at the first workshop on the Turing Challenge,in 2020,estimated that widespread uptake of Level 2 and Level 3 systems will happen within the following five years.Level 4 systems could become widespread in the next 10-15 years,an
214、d Level 5 in the next 20-30 years.Concluding,King and Zenil cite the example of a fully automated experiment that recently tested systematic research reproducibility from literature papers for the first time,illustrating progress towards Levels 4 and 5.Using machine learning to verify scientific cla
215、ims Lucy Wang explores the current state and limitations of ML systems for scientific claim verification.She notes that there is a renewed urgency to successfully automate claim verification,driven by the significant 24 ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS ARTIFICIAL INTE
216、LLIGENCE IN SCIENCE OECD 2023 extent of misinformation spread on line during the COVID-19 pandemic,the sensitivity of topics such as climate change and the sheer abundance of scientific output.Platforms like Twitter,Facebook and others engage in both manual and automated fact-checking.These companie
217、s may employ teams of fact-checkers and ML models.However,Wang notes that scientific claims pose a unique set of challenges for fact-checking due to the abundance of specialised terminology,the need for domain-specific knowledge and the inherent uncertainty of findings at the knowledge frontier.Auto
218、mated scientific claim verification has made significant advances in recent years,but technical and other challenges require further progress.Wang describes areas where more work is needed,including integrating external sources of information into veracity prediction,such as information on funding s
219、ources and sources historical trustworthiness;how to generalise specific domains(scientific claim verification datasets are limited to a few select domains,most notably biomedicine,public health and climate change);widening the space of potential evidence documents,for example expanding from a sampl
220、e of trusted scientific articles to all peer-reviewed scientific documents;and,achieving claim verification that accounts for the beliefs and needs of users.Wang notes that questions remain around how to integrate the outputs of claim verification models with the decisions of human fact-checkers.In
221、addition,there is little study so far on the social issues or consequences of automated scientific claim verification.For example,that the outputs of models built to assist manual fact-checking might have to be different from models built to increase the ability of lay people to engage in scientific
222、 discourse.Robot scientists:From Adam to Eve to Genesis Ross King,Oliver Peter and Patrick Courtney discuss the rapid pace of development in combining robotics with AI to automate aspects of the scientific process.Materials scientists,chemists and drug designers have increasingly taken up integratio
223、n of AI with laboratory automation.AI systems and robots can work more cheaply,faster,more accurately and longer than human beings(i.e.24/7).But they have other advantages besides.As the authors explain,robot scientists can do the following:Flawlessly collect,record and consider vast numbers of fact
224、s.Systematically extract data from millions of scientific papers.Perform unbiased,near-optimal probabilistic reasoning.Generate and compare a vast number of hypotheses in parallel.Select near-optimal(in time and money)experiments to test hypotheses.Systematically describe experiments in semantic det
225、ail,automatically recording and storing results along with the associated metadata and procedures employed,in accordance with accepted standards,at no additional cost,to help reproduce work in other labs,increase knowledge transfer and improve the quality of science.Increase the transparency of rese
226、arch(fraudulent research is more difficult),standardisation and exchangeability(by reducing undocumented laboratory bias).Furthermore,once a working robot scientist is built,it can be easily multiplied and scaled.Robotic systems are also immune to a range of hazards,including pandemic infections.All
227、 of these capabilities remain complementary to the creativity of human scientists.Emerging laboratories in the“cloud”King,Peter and Courtney also describe new experimentation services in the biopharmaceutical industry whereby researchers access automated labs through a user interface or an API,desig
228、ning and executing ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS 25 ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 their experiments remotely.Such services could enable biopharmaceutical enterprises to operate without needing to own a laboratory.However,global cross-platform standar
229、ds for cloud-based laboratories must be adopted.The authors suggest various roles for public support for robotics in science(Box 6).Box 6.Laboratory automation:Suggestions for policy Foster interaction between roboticists and domain experts.Industrial robotics has developed rapidly but not always in
230、 ways that meet the needs of science.Collaborative research programmes and centres could help to bridge these needs by bringing together materials scientists,chemists,AI experts and roboticists to help,for example,develop next-generation battery materials.Collaborative programmes could also facilita
231、te road-mapping across disciplines to identify gaps,opportunities and funding priorities.Governments are best placed to create such programmes,bringing together players that otherwise rarely co-ordinate their activities.Strengthen data governance.Laboratory instruments need to become interoperable v
232、ia standardised interfaces.At present the controls and data produced are presented in a proprietary format and lack the digital metadata around an experiment.This stifles exchange and re-use of data.Laboratory users,suppliers and technology developers could be brought together and incentivised to co
233、-operate from the moment when data are generated by funders and publishers.This might take place under open science initiatives,such as the European Open Science Cloud,that support data curation and sharing through the FAIR principles.Support long-term collaboration across scientific disciplines.The
234、 development of cross-disciplinary research and development centres can serve as a focus for such collaboration,setting medium-term goals and providing formal training that combines engineering(robotics,AI,data,etc.)and science.For example,engineers are seldom exposed to modern,data-rich life scienc
235、e.When linked together,such centres(often national in reach)can also support common interests such as training and evolving research practice.OECD(2020)reviews good practice in designing and implementing cross-disciplinary research.The Centre for Rapid Online Analysis of Reactions(ROAR),at Imperial
236、College London,is an example of such an approach.ROAR aims at digitising chemistry,providing the missing cross-disciplinary exposure and training.Similarly,the CAT+centre is an open-access facility for Swiss scientists combining cutting-edge high-throughput and automated experimentation equipment,as
237、 well as AI,to develop sustainable catalysts.The centre also provides training and enables collaborative work.Support visionary initiatives with long-term impact.Initiatives such as the Nobel Turing Challenge(see the essay by King and Zenil)can galvanise and inspire collaboration and co-ordination i
238、n science and should be supported at an international level.This could help focus efforts on addressing global challenges.It could help to drive agreement on standards and attract young scientists to such ambitious endeavours.From knowledge discovery to knowledge creation:How can literature-based di
239、scovery accelerate progress in science?Neil Smalheiser,Gus Hahn-Powell,Dimitar Hristovski and Yakub Sebastian describe prospects for generating new scientific insight from“undiscovered public knowledge”(UPK)and literature-based discovery(LBD).UPK refers to scientific findings,hypotheses and assertio
240、ns that exist within the published literature without anyone being aware of them.They may be undiscovered for many reasons.Perhaps,for instance,they were published in obscure journals or lack Internet indexing.Or perhaps multiple types of 26 ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PRO
241、POSALS ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 evidence exist across different studies that address the same issue but are not integrated readily with each other(e.g.epidemiologic studies vs.case reports).Entirely new,plausible and scientifically non-trivial hypotheses can be found by combining
242、 findings or assertions across multiple documents.If one article asserts that“A affects B”and another that“B affects C”,then“A affects C”is a natural hypothesis.LBD differs from AI data mining efforts to identify explicitly stated findings or associative trends in the data.LBD attempts to identify u
243、nknown knowledge that is implicitly rather than explicitly stated.The problems that LBD tools are solving(generating potentially novel hypotheses)are inherently more difficult and specialised than searching the research literature(as done by PubMed and Google Scholar).And LBD is distinct from to met
244、a-analysis,which attempts to collate comparable studies.To date,most research on LBD has come from practitioners in computer science,information science and bioinformatics.Indeed,the authors note that LBD launched the entire field of drug repurposing.But LBD can be used much more widely.The authors
245、show that less than 6%of all LBD publications can be mapped to at least one of the United Nations Sustainable Development Goals,even though the techniques could facilitate progress in relevant fields.The next-generation LBD systems are also likely to use information in non-natural language forms,suc
246、h as numerical tables,charts and figures,programming codes,etc.The authors suggest that advances in AI are key to improving LBD systems.Proposals for better exploiting LBD in science are set out in Box 7.Box 7.Better utilising LBD systems in science:Suggestions for policy Train students to search sy
247、stematically for new hypotheses.The biomedical curriculum,for example,provides no such training.LBD analyses should be undertaken in dialogue or partnership between biomedical end-users and informatics consultants in response to specific research questions.For example,what molecular pathways are mos
248、t promising to study in Alzheimers disease?Increase the availability of open research data.Platforms such as Figshare(https:/)and Zenodo(https:/zenodo.org)provide open access to research data as figures,datasets,images or videos.Cloud-based bibliography management solutions(Mendeley,Zotero)and acade
249、mic social networking sites(ResearchGate,Academia.edu)could open exciting possibilities for more author and community-centric LBDs.Such sites could serve as platforms for new initiatives and/or co-ordination mediated by research funders and/or policymaking bodies.Help integrate LBD analyses into eve
250、ryday science.There is no LBD tool similar to Google Scholar used by the general scientific community.Instead,LBD tools are more specialised and require some training,not unlike the training required to use statistics packages or computer programming environments.Perhaps the best way forward is not
251、to require bench and clinical investigators to become LBD experts themselves but rather to create partnerships and collaborations with informatics consultants fluent with LBD tools.One might also envision holding workshops and conferences that address specific problems(e.g.climate change)and carry o
252、ut brainstorming in conjunction with domain experts assisted by LBD analyses.Advancing the productivity of science with citizen science and artificial intelligence Luigi Ceccaroni,Jessica Oliver,Erin Roger,James Bibby,Paul Flemons,Katina Michael and Alexis Joly explain how AI can enhance citizen sci
253、ence.Advances in communication and computing technologies have enabled the public to collaboratively participate in new ways in science projects.To date,the most significant impacts of citizen science have been in data collection and processing,such as classifying ARTIFICIAL INTELLIGENCE IN SCIENCE:
254、OVERVIEW AND POLICY PROPOSALS 27 ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 photographic images,video and audio recordings.However,citizen scientists are engaged in projects across scientific domains such as astronomy,chemistry,computer science and environmental science.The authors describe how ci
255、tizen science systems in combination with AI are advancing science by increasing the speed and scale of data processing;collecting observations in ways not achievable with traditional science;improving the quality of data collected and processed;supporting learning between humans and machines;levera
256、ging new data sources;and diversifying engagement opportunities.Future applications,emerging now,will include more accessible ways for non-experts to use AI techniques,along with autonomous systems of all types,such as drones,self-driving vehicles,and other robotic and remote sensing instrumentation
257、 integrated with AI.All these and other emerging applications will aid data collection and the automatic detection and identification of items in images,audio recordings or videos.More generally,citizen science needs to find ways to break complex research projects into discrete tasks that citizen sc
258、ientists can then undertake.AI might assist in this partitioning of tasks.It is also foreseeable that AI could help ensure adherence to the scientific method and assist in quality assessment(concerns over data quality remain prevalent in citizen science).The authors also describe how policy makers c
259、an help advance the use of AI in citizen science(Box 8).Box 8.AI to help raise the productivity of science using citizen science:Suggestions for policy Develop guidance on proper application of AI.Each use of AI in citizen science needs to carefully consider risks,traceability,transparency and upgra
260、dability.Traceability is essential to reproduce,qualify and revise the data generated by AI algorithms(e.g.through version control and accessibility of the AI models).Transparency is crucial for understanding and correcting biases in AI models(e.g.by making training data fully accessible).Without ap
261、propriate transparency,errors by AI algorithms cannot be understood or,in some cases,even detected.Upgradability the ability of AI algorithms to be upgraded over time is necessary to accommodate new inputs and corrections made by experts and citizen scientists.What can artificial intelligence do for
262、 physics?Sabine Hossenfelder observes that ML has spread to every part of physics.Furthermore,physicists themselves have been at the forefront developments in ML.The behaviour of magnets,to take one example,sheds light on some properties of machines that learn.Hossenfelder groups the applications of
263、 AI in physics into three main categories:Data analysis.For example,achieving fusion power requires AI-enabled solutions to the challenge of suspending super-hot unstable plasma in a ring of powerful magnets.Modelling.For instance,simulating some physical systems such as how subatomic particles scat
264、ter takes a long time.However,ML can learn to extrapolate from existing simulations without re-running the full simulation each time.Model analysis.For example,the theory for materials atomic structure is known in principle.However,many calculations needed to operationalise the theory are so vast th
265、at they have exceeded computational resources.ML is beginning to change that.Hossenfelder reiterates what other contributors to this volume also draw attention to,namely that current algorithms are not a scientific panacea.They rely heavily on humans to provide suitable input data and cannot yet for
266、mulate their own goals.28 ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 AI in drug discovery Kristof Szalay explains that ML has been integral to parts of the process of drug development for decades.Recent improvements in AI have allowe
267、d it to enter other areas in the drug discovery.As major pharmaceutical companies have adopted a business model aimed at decreasing risk in the early parts of drug discovery by in-licensing trial-ready compounds from smaller biotech companies it is in small biotechnology companies where an explosion
268、 in the use of AI technologies has happened.Szalay observes,in line with Jack Scannells essay in this volume,that the main challenge of bringing a new drug to market is that a lot of time and money are needed before a drugs efficacy is determined by testing on patients.AIs main impact will be in sel
269、ecting experiments with the best chance of yielding drugs that pass clinical testing.However,predicting which patients will respond well enough to a drug is a challenge for AI.Each patient is unique,with slightly different biochemistry.In addition,each patient can be dosed only once.If they return t
270、o the clinic,whether the drug has worked or not,their condition may have changed,essentially rendering them for training purposes a different patient.Szalay also highlights a tension between the dynamic creativity of software development and the safety needs of the drug industry.Explainable AI could
271、 address this problem,and help with others,for instance in detecting biases against ethnic minorities in the composition of genomic databases.However,the leading AI models deep-learning systems are not explainable,and other AI approaches are not yet good enough.AI infrastructure and the financial bu
272、rden on smaller academic groups Szalay explains that large modern AI set-ups must move all the pieces of data and the code together at large scales.AI companies have a dedicated team of engineers building the necessary scaffolding(data processing pipelines,orchestrating compute resources,database pa
273、rtitioning,etc.).In this way,every piece of code and data is in the right place at the right time on all the dozens of machines training the AI.This requires expertise and human resources that only make sense to gather if AI is a main focus of a business.Early discovery requires large AI systems and
274、 many training runs,with costs running from hundreds of thousands to millions of US dollars.Szalay suggests a role for policy in addressing the infrastructure challenges(Box 9).Box 9.Access to computational infrastructure for small academic groups:Suggestions for policy Academic groups would need a
275、stronger AI backbone like,for example,that proposed by the National Artificial Intelligence Research Resource Task Force in the United States(NAIRR Task Force,2022).Similar consortia such as the European Open Science Cloud(EC,n.d.)have been established recently in the European Union to support colla
276、boration in the field.However,they are mostly focused on sharing data and tools rather than solving the problem of scaling AI in academia.One step might be to offer research grants that require universities to pool their AI resources into one single effort.Access to supercomputing centres possibly s
277、ubsidised should include the involvement of data engineers who could help researchers get their data through the computing system.Data-driven innovation in clinical pharmaceutical research Joshua New explains that a major barrier to developing new treatments is the cost of evaluating candidate drugs
278、 for safety and efficacy.He cites estimates that,as of 2018,the average cost of an individual clinical trial was USD 19 million.A promising way to reduce costs is through improved use of data and AI in clinical ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS 29 ARTIFICIAL INTELLIGEN
279、CE IN SCIENCE OECD 2023 trial design,particularly to increase patient recruitment and engagement.Selecting a site to perform a clinical trial can be a significant financial commitment.To minimise this risk,some companies have developed AI systems that can guide site-selection decisions.Several compa
280、nies are using AI to improve patient recruitment directly.They analyse structured and unstructured clinical data to better identify patients that match trial criteria,allowing trial organisers to conduct more targeted recruitment.In some cases,patients may end their participation in a trial due to t
281、he negative side effects of a treatment.Therefore,researchers have developed ML algorithms that can identify the fewest and smallest doses of a treatment,to reduce overall toxicity.The author suggests,among other recommendations,that policy makers should expand access to institutional and non-tradit
282、ional data.For example,they could reduce regulatory barriers to data sharing,better enforce publication of clinical trial results and promote data sharing with international partners.Applying AI to real-world health-care settings and the life sciences:Tackling data privacy,security and policy challe
283、nges with federated learning Mathieu Galtier and Darius Meadon explain that ML in health care will not successfully transition from research settings into everyday clinical practice without large,diverse and multimodal data(i.e.digital pathology,radiology and clinical).However,patient and other impo
284、rtant data are usually stored in silos,for instance in different hospitals,companies,research centres,and across different servers and databases.Health data are also tightly regulated.While necessary,this can also hinder research.For instance,completely removing information on a patients identity ca
285、n decrease the performance of an algorithm.The authors discuss how federated learning(FL)can overcome the challenge of fragmented health data.With FL,algorithms are dispatched to different data centres where they train locally.Once improved,the algorithms return to a central location.The data themse
286、lves do not need to be shared(FL is one part of broader family of“privacy-enhancing technologies”that can be applied to AI.Other examples include differential privacy,homomorphic encryption,secure multiparty computation and distributed analytics).Many start-ups now provide FL platforms,but few have
287、managed to apply these in real-world settings at scale.The public sector has started to become active.The UK government,for example,has outlined a plan to set up a federated infrastructure for managing UK genomics data.The authors set out suggestions for policy(Box 10).Box 10.Expanding the use of fe
288、derated learning across research centres:Suggestions for policy Governments can assist through public financing,especially in helping research centres to adopt a decentralised approach and to create shared infrastructure.Public funding is important because the level of co-operation needed would othe
289、rwise emerge slowly.Any funding should be conditional on the recipient infrastructure being governed on the basis of a shared set of rules and protocols for,for example,interoperability,data portability and security.More broadly,governments can take steps to harness the power of data across various
290、fields,from health to climate.For example,in 2022 the European Commission presented its Health Data Space(HDS)(EC,2022).The HDS aims to create a trustworthy and efficient context for the use of health data for research,innovation,policy making and regulation.More broadly,the OECD Recommendation of t
291、he Council concerning Access to Research Data from Public Funding provides guidance to governments on enhancing access to research data(OECD,2021).30 ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 AI and science in the near future:Challe
292、nges and ways forward Artificial intelligence in scientific discovery:Challenges and opportunities Hector Zenil and Ross King consider challenges and opportunities in using AI for science.Their key insights concern the differences between the two main forms of ML learning:statistical ML,the most use
293、d and successful form,which is based upon complex pattern learning,and model-driven ML.As the authors explain,the ability of human scientists to reason rationally,to do abstract modelling and to make logical inferences(deduction and abduction)is central to science.However,these abilities are handled
294、 poorly by statistical ML.Statistical ML operates differently from the human mind.Humans build abstract models of the world that allow mental simulations on the fly of how an object can be modified.They can also generalise even if they have never encountered the same situation before.Humans do not n
295、eed to drive millions of miles to pass a driving test,for example.Model-driven methods can explain more observations with less training data,just as human scientists do when they derive models from sparse data.For instance,Newton and others derived the classical theory of gravitation from relatively
296、 few observations.Pointing to limitations in statistical ML the authors draw attention to the large amounts of data it requires,which are often unavailable in some realms of science;problems associated with data annotation and labelling(for example,it takes time and resources to label large database
297、s by hand,and those doing the labelling might have different levels of competence);variation in features of the data across some areas of science,which may not allow generalisation across fields;and,the black-box character of statistical ML approaches.No matter how abundant the supply of data,the pr
298、oblem of understanding and transfer learning(generalisation)cannot be solved simply by applying ever-more powerful statistical computation.Too little attention,research effort,conference venues,journals and funds are available to AI approaches that differ from statistical ML,such as deep learning.Th
299、is is a consequence of the dominant role of some academic actors and corporate AI research and development(see the essay in this volume by Mateos-Garcia and Klinger).Computers are still unable to formulate interesting research questions,design proper experiments,and understand and describe their lim
300、itations.More resources are needed to develop the methodological frameworks most relevant to the AI required for further progress in scientific discovery.Machine reading:Successes,challenges and implications for science Jesse Dunietz examines the capabilities of state-of-the-art natural language pro
301、cessing(NLP).NLP,researchers hope,could assist scientists by automating some of the reading of scientific papers.Dunietz lays out a variety of reading comprehension tasks that NLP systems might perform on scientific literature,placing these on a spectrum of sophistication based on how humans compreh
302、end written material.The author shows that current NLP techniques grow less capable as tasks require more sophisticated understanding.For example,todays systems excel at flagging names of chemicals.However,they are only moderately reliable at extracting machine-friendly assertions about those chemic
303、als,and they fall far short of,say,explaining why a given chemical was chosen over plausible alternatives.The fundamental problem is that NLP techniques lack rich models of the world to which they can ground language(the essay by Ken Forbus explains the importance of knowledge bases and graphs in ad
304、dressing this problem).They have no exposure to the entities,relationships,events,experiences and so forth that a text speaks about.As a result,even the most sophisticated models still often generate fabrications or outright nonsense.ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS 3
305、1 ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 The author observes that a surprisingly large fraction of research on NLP applied to science has focused only on the surface structure of texts,such as finding key words.Research policies may be able to facilitate progress towards machines capable of so
306、phisticated comprehension of what they read,including scientific papers.To that end,Dunietz proposes two possible ways forward(Box 11).Box 11.Making progress in machine reading of scientific texts:Suggestions for policy Foster new,interdisciplinary,blue-sky thinking:NLP research is often driven by t
307、he pursuit of standardised metrics,by expectations of quick publications and by the allure of the low-hanging fruit from the past decades progress.This environment produces much high-quality work,but it offers limited incentives for the sort of high-risk,speculative ideation that breakthroughs may n
308、eed.Research centres,funding streams and/or publication processes could be set up to reward novel methods even if at a nascent stage.These steps could be taken without prioritising publishing speed,performance metrics and immediate commercial applicability.Support under-studied research:Policy maker
309、s can fund specific areas of under-studied research.To this end,prioritising and funding selected techniques may prove less important than funding aimed at achieving specific tasks.The most sophisticated forms of machine reading seem likeliest to emerge where systems must communicate with humans to
310、perform tasks in a real or simulated physical environment.Interpretability:Should and can we understand the reasoning of machine learning systems?Hugh Cartwright examines the inability of the most powerful ML systems to explain their output,and what means for science,where elucidating the link betwe
311、en cause and effect is fundamental.He notes that not all forms of AI lack interpretability:tools,such as decision trees or reverse engineering offer some insight into their own logic.However,most scale poorly with software complexity and are of value only to experts.Cartwright describes why interpre
312、tation in science poses particular conceptual challenges,even if ML could explain its own logic.As science continues to evolve,some topics may become so intellectually demanding that no one can understand them(he gives an example from the mathematics of string theory,understandable perhaps to only a
313、 few specialists).If an AI system were to discover such knowledge,it is unclear what an explanation for human scientists would look like.Similarly,translating into human-digestible form what an AI system has learnt in a hugely dimensional data space may yield hard-to-understand lines of reasoning,ev
314、en if individual parts of the argument are clear.In some cases,explanations need to be illustrated by images.However,Cartwright points out that while image recognition applications have progressed,it is challenging for AI systems to construct images to assist explanation.In addition,explanation mech
315、anisms may not port well from one application area to another.A risk exists,in Cartwrights view,that the demand for useful,commercially valuable,AI may outstrip progress on explanation.Combining collective and machine intelligence at the knowledge frontier Eirini Malliaraki and Aleks Berditchevskaia
316、 highlight that while AI has greatly advanced,humans have unique abilities such as intuition,contextualisation and abstraction.Consequently,novel AI and human collaborations could advance science in new ways.Properly orchestrated,the capabilities of collaborating 32 ARTIFICIAL INTELLIGENCE IN SCIENC
317、E:OVERVIEW AND POLICY PROPOSALS ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 individuals can exceed the sum of the capabilities of the same individuals working in isolation.This is“collective intelligence”.Malliaraki and Berditchevskaia observe that a robust understanding of how to make the most of
318、collective intelligence in science is only beginning to emerge.In addition,progress in combining human collective intelligence and AI is important because science is now carried out by ever-larger teams and international consortia.The authors describe how AI-human collaborations can improve upon cur
319、rent approaches to mapping the knowledge frontier in a number of ways,including those described below.Encoding and discovering knowledge Todays science communication infrastructure does not help researchers make the best use of predominantly document-centric scholarly outputs.For example,words and s
320、entences may be searched for,but images,references,symbols and other semantics are mostly inaccessible to current machines.Recent advances in language models can help but do not work well outside the domains where they are developed.Harnessing complementary expertise from among scientists and policy
321、 makers would assist.Connecting and structuring knowledge Once relevant public knowledge is encoded and discovered it needs to be organised and synthesised.With recent advances in knowledge representation and human-machine interaction,scholarly information can be expressed as knowledge graphs(see Ke
322、n Forbus essay on knowledge bases and graphs).Current automatic approaches to create these graphs have limited accuracy and coverage.Hybrid human-AI systems help.Oversight and quality control A knowledge synthesis infrastructure will not be complete without ongoing curation and quality assurance by
323、domain experts,librarians and information scientists.Automated systems to check scientific papers are helpful,but they require augmentation by distributed peer review or the crowdsourced intelligence of experts.Malliaraki and Berditchevskaia suggest how policy could accelerate the integration of com
324、bined AI-human systems into mainstream science(Box 12).Box 12.Integrating combined AI-human systems into mainstream science:Suggestions for policy Develop tools to enhance AI and collective intelligence combinations:Co-operative human-AI systems will have to navigate problems where the goals of diff
325、erent actors and organisations are in tension with one another,as well as those where actors have common agendas.For instance,some academic groups are in competition.They may not be incentivised to share for fear of being scooped or may simply have conflicting approaches to a method or a problem.Whi
326、le there has been some research in this area such as in this field of research has lagged other topics in AI.Make use of existing social networks to experiment with human-AI collaboration:Social platforms such as Academia.edu and the Loop community support knowledge exchange between academics and pr
327、ovide an infrastructure for literature discovery.Some of these platforms already use AI-enabled recommendation systems.Such platforms could become testbeds for experimenting with combined human-AI knowledge discovery,idea generation and synthesis.The benefit of these platforms is that they already h
328、ave an engaged community united around a common interest/purpose.An extended ARTIFICIAL INTELLIGENCE IN SCIENCE:OVERVIEW AND POLICY PROPOSALS 33 ARTIFICIAL INTELLIGENCE IN SCIENCE OECD 2023 functionality would need to align with or enhance that common purpose.Working together with researchers,fundin
329、g and/or incentives provided by research funders might catalyse progress.Such investment could also be connected to mission-oriented research agendas.Re-think incentives for knowledge mapping and synthesis:Several institutional and educational conditions inhibit work on knowledge integration.Existin
330、g measures of publishability motivate discoveries built on individual disciplines rather than knowledge synthesis.New integrative PhD programmes and/or industry research programmes based on knowledge synthesis might help.Research councils and academic institutions should experiment with these propos
331、als and support new roles and career paths.They could support the development of expertise in curating and maintaining information infrastructure,which could also help to build bridges between the public,academia and industry.Elicit:Language models as research tools Jungwon Byun and Andreas Stuhlmll
332、er examine how ML could change research over the next decade.Intelligent research assistants could increase the productivity of science,for instance by enabling qualitatively new work,making research accessible to non-experts,and reducing what can be extraordinary and sometimes fruitless calls on sc
333、ientists time(for example,one study in Australia found that 400 years of researchers time was spent preparing unfunded grant proposals for support from a single health research fund,Herbert,Barnett and Graves,2013).Byun and Stuhlmller observe that existing research tools are not designed to direct the researcher quickly and systematically to research-backed answers.In response,the authors have hel