1、大模型智能体的自主规划学习张宁豫浙江大学 代表案例：AutoGPT，GPT-Engineer，Voyager,RT-2，什么是大模型驱动的自主智能体 定义：由大型语言模型驱动的自治代理，它们可以遵循语言指令并在真实世界或模拟环境中执行各种复杂任务背景1 A Survey on Large Language Model based Autonomous Agents大模型驱动自主智能体的飞速发展背景1 Artificial Intelligence:A Modern Approach2 Multiagent Reinforcement Learning:Rollout and Policy It

2、eration大模型驱动的自主智能体本质 智能体（Agent）的概念自人工智能领域诞生之日就存在，并同时受到面向对象系统、人机交互、分布式学习、强化学习等领域关注图引用自1图引用自2背景1 https:/lilianweng.github.io/posts/2023-06-23-agent/2 Reasoning with Language Model Prompting:A Survey大模型驱动的自主智能体本质图引用自1图引用自2 智能体（Agent）的概念自人工智能领域诞生之日就存在，并同时受到面向对象系统、人机交互、分布式学习、强化学习等领域关注 大模型驱动的自主智能体有什么区别？本质

3、的区别在于语言的运用 人工智能代理（Agent）将语言作为思维和沟通的工具，这是人类独有的能力背景大模型驱动的自主智能体关键技术背景1 Graph of Thoughts:Solving Elaborate Problems with Large Language Models大模型驱动的自主智能体的任务规划大模型驱动的自主智能体通过任务解耦和思维链（图）的方式进行任务规划智能体自主规划大模型驱动的自主智能体的任务规划智能体自主规划1 Voyager:An Open-Ended Embodied Agent with Large Language Models9大模型驱动的自主智能体的任务规划

4、智能体自主规划智能体自主规划 挑战二：大模型智能体如何根据已有的推理路径自主校准并优化规划过程Towards A Unified View of Answer Calibration for Multi-Step Reasoning(2023)大模型智能体的任务自主规划关键挑战 挑战一：大模型智能体如何根据任务的内容自主规划并选择和使用外部工具Making Language Models Better Tool Learners with Execution Feedback(2023)开源工具：Agents:An Open-source Framework for Autonomous La

5、nguage Agents(2023)智能体自主规划Project:https:/zjunlp.github.io/project/TRICE/Code:https:/ Making Language Models Better Tool Learners with Execution Feedback大模型驱动的自主智能体的工具使用智能体自主规划智能体使用工具的步骤:1.智能体什么时候候用？用什么工具?2.智能体需要提供工具哪些信息?3.智能体如何根据工具的结果做出反馈?大模型驱动的自主智能体通过外部工具和环境反馈实现自主规划完成任务智能体自主规划1 Making Language Mode

6、ls Better Tool Learners with Execution Feedback先基于指令微调学习如何使用工具，后基于工具执行反馈进一步习得何时使用工具的自主规划能力智能体自主规划1 Making Language Models Better Tool Learners with Execution Feedback智能体自主规划1 Making Language Models Better Tool Learners with Execution Feedback智能体自主规划1 Making Language Models Better Tool Learners with

7、Execution Feedback智能体自主规划1 Making Language Models Better Tool Learners with Execution Feedback智能体自主规划1 Making Language Models Better Tool Learners with Execution Feedback该方法可以在一定程度上降低智能体对工具依赖，让智能体知道何时靠自己何时找帮手智能体自主规划1 Making Language Models Better Tool Learners with Execution Feedback智能体自主规划 挑战二：大模型智

8、能体如何根据已有的推理路径自主校准并优化规划过程Towards A Unified View of Answer Calibration for Multi-Step Reasoning(2023)大模型智能体的任务自主规划关键挑战 挑战一：大模型智能体如何根据任务的内容自主规划并选择和使用外部工具Making Language Models Better Tool Learners with Execution Feedback(2023)开源工具：Agents:An Open-source Framework for Autonomous Language Agents(2023)智能体自

9、主规划1 Towards A Unified View of Answer Calibration for Multi-Step Reasoning智能体自主规划大模型规划路径的自主校准问题：何时校准？校准到什么粒度？1 Towards A Unified View of Answer Calibration for Multi-Step Reasoning智能体自主规划大模型智能体规划路径自主校准的统一视角，提供何时校准及如何校准的经验性原理=0和=1是self-verification和self-self-consistency特例1 Towards A Unified View of A

10、nswer Calibration for Multi-Step Reasoning智能体自主规划大模型智能体规划路径自主校准的统一视角，提供何时校准及如何校准的经验性原理1 Towards A Unified View of Answer Calibration for Multi-Step Reasoning智能体自主规划1 Towards A Unified View of Answer Calibration for Multi-Step Reasoning智能体自主规划两个阈值区间内智能体的规划推理性能达到最优，验证了所提出的统一视角的原理智能体自主规划 挑战二：大模型智能体如何根据

11、已有的推理路径自主校准并优化规划过程Towards A Unified View of Answer Calibration for Multi-Step Reasoning(2023)大模型智能体的任务自主规划关键挑战 挑战一：大模型智能体如何根据任务的内容自主规划并选择和使用外部工具Making Language Models Better Tool Learners with Execution Feedback(2023)开源工具：Agents:An Open-source Framework for Autonomous Language Agents(2023)1 Agents:A

12、n Open-source Framework for Autonomous Language Agents智能体自主规划Agents:An Open-source Framework for Autonomous Language Agents智能体自主规划Controllability via Symbolic Control(SOP)An SOP is a graph of states each describing a sub-goal of the agent system.Language agents acts according to the specification of

13、 the state and transit according to the an LLM-based controller1 Agents:An Open-source Framework for Autonomous Language Agents智能体自主规划In the Agents Framework,a(multi-)agent system is defined by the agents,the SOP,and the environment.All these components can be described in a single config file using

14、 natural language 1 Agents:An Open-source Framework for Autonomous Language Agents智能体自主规划Running loop in the Agent framework is straightforward1 Agents:An Open-source Framework for Autonomous Language Agents智能体自主规划Single AgentsMulti-Agents1 Agents:An Open-source Framework for Autonomous Language Age

15、nts智能体自主规划Agents can help user generate a config file for a language agents system by simply inputting a short description of it.How?Retrieval-based generation using agent-hub that stores various config files.“The more you share,the more you can create”1 Agents:An Open-source Framework for Autonomou

16、s Language Agents展望 大模型智能体如何根据已有的推理路径自主校准并优化规划过程Towards A Unified View of Answer Calibration for Multi-Step Reasoning(2023)大模型智能体如何根据任务的内容自主规划并选择和使用外部工具Making Language Models Better Tool Learners with Execution Feedback(2023)开源工具：Agents:An Open-source Framework for Autonomous Language Agents(2023)大模

17、型智能体的自主规划学习展望 Why multi-agents?Goodharts Law-The better on object A,the worse on many other objects B What do agents interact with?Knowledge boundary,Brain in a Vat What is the preferred method of communication among agents?Natural language or Code How to communicate between agents?Roles,Society,Behaviors1 Exploring Collaboration Mechanisms for LLM Agents:A Social Psychology View2023/11/3036ACCEPTMYENDLESSGRATITUDE敬请各位专家批评指正