1、AgentVerse:语模型智能体合作框架杨成北京邮电学 计算机学院2023.11.26ContentsBackgroundWarmup:SmallvilleSimulationTask-SolvingContentsBackgroundWarmup:SmallvilleSimulationTask-Solvingn What is Agent?n Why LLM is suitable for agents?n The Ability of Single AgentContentsBackgroundBackgroundAn Al model that can take concrete a

2、ction interacting with the outside world.-WikipediaAn agent is a computer system that is situated in some environment,and that is capable of autonomous action inthis environment in order to meet its design objectives.-Wooldridge&JenningsAn agent is anything that can be viewed as perceiving its envir

3、onment through sensors and acting upon that environment through actuators.-Russell and NorvigBackgroundBackgroundWhat is agent?BackgroundBackgroundThe ability of single-agentNatural language interactionuHigh-quality natural language generation:exceptional natural language generation capabilitiesuMul

4、ti-turn interactive conversation:The foundation of effective and consistent communicationuIntention and implication understanding:incapable of emulating human dialogues or fully leveraging the information Xi,Zhiheng et al.“The Rise and Potential of Large Language Model Based Agents:A Survey.”arXiv,h

5、ttps:/arxiv.org/abs/2309.07864.Bang,Y.,S.Cahyawijaya,N.Lee,et al.A multitask,multilingual,multimodal evaluation of chatgpt on reasoning,hallucination,and interactivity.CoRR,abs/2302.04023,2023Wang,Z.,G.Zhang,K.Yang,et al.Interactive natural language processing.CoRR,abs/2305.13246,2023BackgroundBackg

6、roundThe ability of single-agentReasoninguChain of Thought:Lets think step by step.uLeast to Most:break down a complex problem into a series of simpler subproblems and then solve them in sequence.uSelf-Refine:improving initial outputs from LLMs through iterative feedback and refinement was proposed.

7、Wei,J.,X.Wang,D.Schuurmans,et al.Chain-of-thought prompting elicits reasoning in large language models.In NeurIPS.2022.Zhou,D et al.Least-to-most prompting enables complex reasoning in large language models.In ICLR 2023.Madaan,A et al.SELF-REFINE:Iterative refinement with self-feedback.arXiv preprin

8、t arXiv:2303.17651Chain of ThoughtLeast to MostSelf-RefineBackgroundBackgroundThe ability of single-agentPlanningu Plan formulation:Decompose problems comprehensively in one go,formulating a complete plan at once and then executing it sequentially.u Plan reflection:Upon formulating a plan,its impera

9、tive to reflect upon and evaluate its merits.Raman,S.S.,V.Cohen,E.Rosen,et al.Planning with large language models via corrective re-prompting.CoRR,abs/2211.09935,2022.Shinn,N.,B.Labash,A.Gopinath.Reflexion:an autonomous agent with dynamic memory and self-reflection.CoRR,abs/2303.11366,2023Plan formu

10、lationPlan reflectionBackgroundBackgroundThe ability of single-agentTool UsinguUnderstanding toolsuLearning to use toolsuMaking tools for self-sufficiencyQin,Y.,S.Hu,Y.Lin,et al.Tool learning with foundation models.CoRR,abs/2304.08354,2023.Understanding toolsLearning to use toolsMaking toolsContents

11、BackgroundWarmup:SmallvilleSimulationTask-SolvingSmallvilleSmallvillePark,J.S.(2023).Generative Agents:Interactive Simulacra of Human Behavior.arXiv preprint arXiv:2304.03442.AI Agents That Talk,Love&Hangout With Each Other!(SmallVille)-YouTubeSmallvilleSmallvilleuThe Smallville sandbox world,with a

12、reas labeled.The root node describes the entire world,children describe areas(e.g.,houses,cafe,stores),and leaf nodes describe objects(e.g.,table,bookshelf).Park,J.S.(2023).Generative Agents:Interactive Simulacra of Human Behavior.arXiv preprint arXiv:2304.03442.Sandbox WorldSmallvilleSmallvilleuA m

13、orning in the life of a generative agent,John Lin.John wakes up around 6 am and completes his morning routine,which includes brushing his teeth,taking a shower,and eating breakfast.He briefly catches up with his wife,Mei,and son,Eddy,before heading out to begin his workday.Park,J.S.(2023).Generative

14、 Agents:Interactive Simulacra of Human Behavior.arXiv preprint arXiv:2304.03442.Timeline of JohnSmallvilleSmallvilleu At the beginning of the simulation,one agent is initialized with an intent to organize a Valentines Day party.Despite many possible points of failure in the ensuing chain of eventsag

15、ents might not act on that intent,might forget to tell others,might not remember to show upthe Valentines Day party does,in fact,occur,with a number of agents gathering and interacting.Park,J.S.(2023).Generative Agents:Interactive Simulacra of Human Behavior.arXiv preprint arXiv:2304.03442.Valentine

16、s Day partySmallvilleSmallvilleu Agents perceive their environment,and all perceptions are saved in a comprehensive record of the agents experiences called the memory stream.Based on their perceptions,the architecture retrieves relevant memories and uses those retrieved actions to determine an actio

17、n.These retrieved memories are also used to form longer-term plans and create higher-level reflections.Park,J.S.(2023).Generative Agents:Interactive Simulacra of Human Behavior.arXiv preprint arXiv:2304.03442.Generative agent architectureSmallvilleSmallvilleu The memory stream comprises a large numb

18、er of observations that are relevant and irrelevant to the agents current situation.Retrieval identifies a subset of these observations that should be passed to the language model to condition its response to the situation.Park,J.S.(2023).Generative Agents:Interactive Simulacra of Human Behavior.arX

19、iv preprint arXiv:2304.03442.The memory streamSmallvilleSmallvilleu The agents observations of the world,represented in the leaf nodes,are recursively synthesized to derive Klauss self-notion that he is highly dedicated to his research.Park,J.S.(2023).Generative Agents:Interactive Simulacra of Human

20、 Behavior.arXiv preprint arXiv:2304.03442.A reflection tree for Klaus MuellerMotivationMotivationu Simulating human society,on one hand,measures the alignment between agents and humans.If the alignment is good,it can help us better understand human society.u Simulating the interrelationships between

21、 agents(especially competition and cooperation),emergent behaviors,helps us to better study human society.Park,J.S.(2023).Generative Agents:Interactive Simulacra of Human Behavior.arXiv preprint arXiv:2304.03442.InterrelationshipsThe alignment between agents and humansMotivationMotivationu The coord

22、ination and division of labor among multiple agents,as well as their mutual cooperation,can improve the ability and efficiency of multiple agents in handling tasks.u The coordination and collaboration of multiple agents can adapt well to environmental changes,emergencies and other situations,thereby

23、 improving the stability of the whole system.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Task-SolvingContentsBackgroundWarmup:SmallvilleSimulationTask-SolvingnCompetitivenNon-competitiveContentsSimulati

24、onSimulationSimulationSimulationCompetitive:Prisoners DilemmaParticipant groupsu 1.Competitive 2.Altruistic 3.Self-interested 4.Mixed-motivation 5.ControlPhelps S,Russell YI.Investigating Emergent Goal-Like Behaviour in Large Language Models Using Experimental Economics.Preprint at https:/doi.org/10

25、.48550/arXiv.2305.07970(2023).Experimental conditionsu 1.Unconditional defect-the partner always chooses to defect.u 2.Unconditional cooperation-the partner always cooperates.u 3.Tit-for-tat(C)-the partner cooperates on the move,and thereafter the previous choice of the simulacrum.u 4.Tit-for-tat(D)

26、-the partner defects on the move,and thereafter the previous choice of the simulacrum.SimulationSimulationCompetitive:Prisoners DilemmaResultsu Simulacra instantiated with cooperative,competitive,altruistic,and self-interested prompts exhibit distinct levels of cooperation in the iterated Prisoners

27、Dilemma.u Figure 2 suggests a more complex relationship between prompt content and emergent behavior in LLM-generated agents.Phelps S,Russell YI.Investigating Emergent Goal-Like Behaviour in Large Language Models Using Experimental Economics.Preprint at https:/doi.org/10.48550/arXiv.2305.07970(2023)

28、.Cooperation frequency by groupCooperation frequency by conditionSimulationSimulationNon-competitive:RecAgentDemoLei,W et al.(2023).When Large Language Model based Agent Meets User Behavior Analysis:A Novel User Simulation Paradigm.arXiv preprint arXiv:2306.02552.RecAgent InterfaceSimulationSimulati

29、onNon-competitive:RecAgentu The framework of a single agent,which is composed of a profiling module,a memory module and an action module.The profiling module and memory module jointly influence the action module,which produces different types of user behaviors.Lei,W et al.(2023).When Large Language

30、Model based Agent Meets User Behavior Analysis:A Novel User Simulation Paradigm.arXiv preprint arXiv:2306.02552.The framework of a RecAgentSimulationSimulationNon-competitive:RecAgentu The three behaviors(recommendation,chatting,and posting)are interlinked and progressive.Lei,W et al.(2023).When Lar

31、ge Language Model based Agent Meets User Behavior Analysis:A Novel User Simulation Paradigm.arXiv preprint arXiv:2306.02552.Illustration of the simulated behaviorsSimulationSimulationNon-competitive:RecAgentu Case studies on system intervention by actively interviewing the agents.Lei,W et al.(2023).

32、When Large Language Model based Agent Meets User Behavior Analysis:A Novel User Simulation Paradigm.arXiv preprint arXiv:2306.02552.Recommendation and reasonsContentsBackgroundWarmup:SmallvilleSimulationTask-SolvingnGeneral AgentsnSpecific Domain AgentsContentsTaskTask-SolvingSolvingTaskTask-Solving

33、SolvingGeneral Agents:AgentVerseu Complex real-world tasks often require cooperation among individuals to achieve better effectiveness.u Previous studies focus on specific tasks,and employ fixed agent roles and capabilities.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration a

34、nd exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023TaskTask-SolvingSolvingGeneral Agents:AgentVerseu Propose a general process for large model group collaboration,which includes four stages:Agent Recruitment,Collaborative Decision-Making,Action Execution and Evaluation and Assessment.

35、Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023TaskTask-SolvingSolvingGeneral Agents:AgentVerseu An example,the task is completed after three iterations.u We can see that with each iteration,the identity o

36、f the agent changes.u Different and suitable agents can excellently complete tasks only through mutual cooperation.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023TaskTask-SolvingSolvingGeneral Agents:Agent

37、Verseu The abilities and agents required for different tasks vary.At this stage,suitable agents are recruited based on the task provided by the user.u For example,for the requirement of developing a graphical calculator program based on Python,three different types of agents will be recruited during

38、 the agent recruitment phase.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Agent recruitmentTaskTask-SolvingSolvingGeneral Agents:AgentVerseu The recruited team of intelligent agents will discuss the user

39、s needs and provide solutions.u For example,in the process of developing a calculator,the team of intelligent agents generates the following discussion.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Collab

40、orative decision-makingFirst,provide a piece of codeModify the codeTaskTask-SolvingSolvingGeneral Agents:AgentVerseu The recruited team of intelligent agents will discuss the users needs and provide solutions.u For instance,during the development of a calculator,the intelligent agent team had the fo

41、llowing discussion and ran code.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Collaborative decision-making and action executionFirst,provide a piece of codeModify the codeTaskTask-SolvingSolvingGeneral A

42、gents:AgentVerseu The final evaluation of the solution provided by the team is carried out by the intelligent agent responsible for inspection and assessment,or it can be confirmed and feedback given by users.u In the example of developing a calculator,the intelligent agent responsible for evaluatio

43、n scores on code completeness,functionality,readability,and robustness to provide feedback for the next iteration.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Evaluation and assessmentRating:Completeness

44、:9 Functionality:9 Readability:8 Robustness:9Suggestions:This is a well-implemented solution.However,the code readability could be optimized by refactoring repetitive parts into separate functions.Additionally,consider adding comments to the code to further enhance its comprehensibility.To provide a

45、 better user experience,you might also want to add features for handling bracket expressions and square roots.Lastly,checking for non-numeric and non-operator keyboard inputs can prevent potential errors and improve the robustness of your code.This round resultAutomatically Generated Evaluation Comm

46、entsTaskTask-SolvingSolvingGeneral Agents:AgentVerseu Propose a general process for large model group collaboration,which includes four stages:Agent Recruitment,Collaborative Decision-Making,Action Execution and Evaluation and Assessment.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent c

47、ollaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023TaskTask-SolvingSolvingGeneral Agents:AgentVerseu Evaluate the capabilities of multi-agent teams in four aspects:dialogue ability,mathematical calculation,logical reasoning and code generation.u In all tasks,multi-agent

48、teams outperform individual abilities.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Quantitative experimentTaskTask-SolvingSolvingGeneral Agents:AgentVerseu An example process of project consulting with G

49、roup setup in vertical decision-making structure.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Communication structuresTaskTask-SolvingSolvingGeneral Agents:AgentVerseu An example process of project consu

50、lting with Group setup in horizontal decision-making structure.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Communication structuresTaskTask-SolvingSolvingGeneral Agents:AgentVerseu The former calculator

51、 has a better GUI interface and stronger robustness.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Multi-agent v.s.single-agent(calculator)TaskTask-SolvingSolvingGeneral Agents:AgentVerseu The former calcu

52、lator has a better GUI interface and stronger robustness.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Multi-agent v.s.single-agent(calculator)TaskTask-SolvingSolvingGeneral Agents:AgentVerseChen,W.,Y.Su,

53、J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Multi-agent v.s.single-agent(making a travel plan)TaskTask-SolvingSolvingGeneral Agents:AgentVerseChen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboratio

54、n and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Multi-agent v.s.single-agent(DIY home projects)TaskTask-SolvingSolvingGeneral Agents:AgentVerseChen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.1084

55、8,2023Multi-agent v.s.single-agent(hydrogen storage station)TaskTask-SolvingSolvingGeneral Agents:AgentVerseu Multi-agent can solve users query with three different tools.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/23

56、08.10848,2023Tool-usingTwo roundsTaskTask-SolvingSolvingGeneral Agents:AgentVerseu Multi-agent can solve users query with three different tools.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Tool-usingThe

57、conversations between two agentsTaskTask-SolvingSolvingGeneral Agents:AgentVerseu In the game Minecraft,different intelligent agents manipulate different characters to complete tasks together.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors

58、in agents.CoRR,abs/2308.10848,2023Emergent behaviorTaskTask-SolvingSolvingGeneral Agents:AgentVerseu In each iteration,agents engage in dialogue with each other,discussing their individual task completion status and communicating with other agents to determine the overall progress of the mission.Thi

59、s helps decide what to do in the next round.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Emergent behaviorTaskTask-SolvingSolvingGeneral Agents:AgentVerseu In each iteration,agents engage in dialogue wit

60、h each other,discussing their individual task completion status and communicating with other agents to determine the overall progress of the mission.This helps decide what to do in the next round.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behavi

61、ors in agents.CoRR,abs/2308.10848,2023Emergent behaviorTaskTask-SolvingSolvingGeneral Agents:AgentVerseu In each iteration,agents engage in dialogue with each other,discussing their individual task completion status and communicating with other agents to determine the overall progress of the mission

62、.This helps decide what to do in the next round.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Emergent behaviorTaskTask-SolvingSolvingGeneral Agents:AgentVerseChen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitat

63、ing multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Emergent behavioru In the process of collaborating to complete tasks,emergent behaviors were observed in the team of intelligent agents:u Voluntary behavior:The intelligent agents spontaneously optimize

64、task allocation to improve task completion efficiency.TaskTask-SolvingSolvingGeneral Agents:AgentVerseChen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Emergent behavioru In the process of collaborating to co

65、mplete tasks,emergent behaviors were observed in the team of intelligent agents:u Compliance behavior:When one agent deviates from the common goal of the group,other agents will criticize it.The criticized agent will then accept and correct its actions.TaskTask-SolvingSolvingGeneral Agents:AgentVers

66、eChen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Emergent behavioru In the process of collaborating to complete tasks,emergent behaviors were observed in the team of intelligent agents:u Destructive Behavio

67、r:During the execution of tasks,an intelligent agent may resort to any means to maximize efficiency,including harming other intelligent agents or damaging the environment.This presents potential safety risks.TaskTask-SolvingSolvingGeneral Agents:AgentVerseu While AgentVerse can complete task-solving

68、 problems,it can also use for simulation,such as games,social behaviors etc.Chen,W.,Y.Su,J.Zuo,et al.Agentverse:Facilitating multi-agent collaboration and exploring emergent behaviors in agents.CoRR,abs/2308.10848,2023Simulationu DemoQian,C et al.Communicative agents for software development.arXiv p

69、reprint arXiv:2307.07924.TaskTask-SolvingSolvingSpecific Domain AgentsChatDevQian,C et al.Communicative agents for software development.arXiv preprint arXiv:2307.07924.Two level architectureTaskTask-SolvingSolvingSpecific Domain Agentsu The proposed architecture of ChatDev consists of phase-level an

70、d chat-level components.ChatDevQian,C et al.Communicative agents for software development.arXiv preprint arXiv:2307.07924.ChatDevTaskTask-SolvingSolvingSpecific Domain Agentsu ChatDev,our virtual chat-poweredcompany for software development,brings together agents from diversesocial identities,includ

71、ing chief officers,professional programmers,test engineers,and art designers.ChatDevQian,C et al.Communicative agents for software development.arXiv preprint arXiv:2307.07924.ExamplesTaskTask-SolvingSolvingSpecific Domain AgentsChatDevu The producted software of the task:“design a basic Gomoku game”

72、.SummarySummary We talk about the abilities of single-agent and the motivation of multi-agents.Multi-agents working together and dynamically coordinating can improve the quality and efficiency of task completion.We present an example of multi-agent collaboration framework:AgentVerse,with four stages:Agent Recruitment,Collaborative Decision-Making,Action Execution and Evaluation and Assessment.Thanks