《叶昌-开发者大会.pptx》由会员分享,可在线阅读,更多相关《叶昌-开发者大会.pptx(15页珍藏版)》请在三个皮匠报告上搜索。
1、AIAI时代,嵌入式工程师如何玩转大模型时代,嵌入式工程师如何玩转大模型?叶昌2024.06.011 Gholami,Amir,Zhewei Yao,Sehoon Kim,Coleman Hooper,Michael W.Mahoney,and Kurt Keutzer.“Ai and memory wall.”arXiv preprint arXiv:2403.14123(2024).AI vs.HWAI vs.HWFast AI vs.Slow Fast AI vs.Slow HardwareHardwareCompare to the Moores law,LLMs have an al
2、most crazy increasing speed on memory requirement;For embedded software engineers,CNN is OK,but Transformer is too far;TimeTimeParameter countParameter countcomputing power gapCNNTransformerEmbedded engineeringAI2021(ChatGPT)AI vs.Embedded engineeringAI vs.Embedded engineeringARE WE REALLY STUCKED A
3、T THE CNN PHASE IN NEW AI ERA?ARE WE REALLY STUCKED AT THE CNN PHASE IN NEW AI ERA?Opportunity for embedded software engineersOpportunity for embedded software engineers2 Liang Y,Wu C,Song T,et al.Taskmatrix.ai:Completing tasks by connecting foundation models with millions of apisJ.arXiv preprint ar
4、Xiv:2303.16434,2023.3 Zhao W X,Zhou K,Li J,et al.A survey of large language modelsJ.arXiv preprint arXiv:2303.18223,2023.OpenAI,Microsoft,Google,Meta,Alibaba,Baidu,LLMLLMdesign,trainuseSmart homeRobotManufacturingAuto drivingMedical assistantOpportunity for embedded software engineers:from applicati
5、on viewpointOpportunity for embedded software engineers:from application viewpoint4 https:/ relatedHardware relatedOpportunity for embedded software engineers:from chip viewpointOpportunity for embedded software engineers:from chip viewpoint5 https:/market.us/report/ai-chip-market/AI chip type:CPU,G
6、PU,FPGA,ASIC;All kinds of forms of AI chip will keep fast developing in the coming years synchronously;CPU is flexible and low cost,and ASIC is efficient and power saving.For embedded software engineers,and for RT-ThreadFor embedded software engineers,and for RT-Thread OS,OS,how to make all devices?
7、how to make all devices?RT-Thread OS:make all devices smartRT-Thread OS:make all devices smartIntelligentizehuman machine interactionhuman machine interactionhuman machine interactionIntelligent control for air-conditionerIntelligent control for washing machineIntelligent control for fridgeCommon fu
8、nctionsHigh computing costVarious functionsLow computing costSolution 2:Distributed AI Multiplexing human machine interactionIntelligent control for Solution 1:Independent AIhuman machine interaction+Intelligent controlHigh costAI resource waste Low costAI high efficiency95%AI computing 95%AI comput
9、ing powerpowerRT-Thread AIOS:Software architectureRT-Thread AIOS:Software architectureASCII:Function_name_1 Function_name_1 params_1 prarms_2;Function_name_2 Function_name_2 prarms_3 params_4;pythonCHuman Machine Interaction layerOnline LLM:-chatgptPrivate LLM:-Llama-ChatGLMInformation transfer laye
10、rROS/DDSserialWIFIRT-Thread AIOS componentRT-Thread AI_Thread(Func)Library 2Library 1Kernel layerRT-Thread AIOS:AI function componentRT-Thread AIOS:AI function componentRT-Thread OS 1RT-Thread OS 2RT-Thread AIOS:HardwareRT-Thread AIOS:HardwareLow cost MCULow cost MCUMiddle and Middle and h highigh c
11、ost MCU cost MCUMPU/MPU/MulticoreMulticoreHigh performance High performance MPU/SOCMPU/SOCFPGAFPGART-Thread Smart:Typical resource requirement:ROM 32K-1M Bytes RAM 10K-1M BytesTypical architecture:Cortex M/Cotex A/Cortex R series MIPS/LoongArch series SPARC series RISC-V seriesRT-Thread OS:the lowes
12、t resource requirement:ROM 3K Bytes RAM 2K BytesTypical architecture:Cortex M series SPARC seriesWith RT-ThreadWith RT-Thread OS,OS,we can make all devices smart!we can make all devices smart!RT-Thread AIOS:DemoRT-Thread AIOS:DemoTest board:RT-Thread spark RT-Thread spark Stm32f4 series MCU without
13、AI acceleratorRTX 3090TI nvidia GPUTest server:ChatGLM3ChatGLM3RT-Thread AIOS:DemoRT-Thread AIOS:DemoRT-Thread AIOS:DemoRT-Thread AIOS:DemoContinuously call 26 times 26 times of RT-Thread OS API by LLM.(rgb_led_on)RT-Thread AIOS:call more rt-thread APIsRT-Thread AIOS:call more rt-thread APIsRT-Threa
14、d AIOS:call more rt-thread APIsRT-Thread AIOS:call more rt-thread APIsWe can call three various RT-Thread APIs,but we can not control the LLMs output detailly.ConclusionsConclusionsDistributed AI is a better pathway to realize embedded AI products.RT-Thread OS can help embedded software engineers easily develop new AI products with AI function module.With LLMs,RT-Thread OS can execute complex function clusters when we want smart devices do something without coding.Thanks!Thanks!Q&A