《利用 SLM 结合边缘设备构建 AIoT Agent-卢建晖.pdf》由会员分享,可在线阅读,更多相关《利用 SLM 结合边缘设备构建 AIoT Agent-卢建晖.pdf(32页珍藏版)》请在三个皮匠报告上搜索。
1、利用 SLM 结合边缘设备构建 AIoT Agent卢建晖 微软高级云技术布道师模型发展模型继续发展文本语音视频图片向量闭源模型 云端 开源模型 本地企业最终选择Cloud LLMLocal SLMs混合模型GPUNPUCPU云+本地的算力架构微调微调RAG行业数据SLM什么是 SLM(Small Language Model)LLMSLM厂商的 SLM 角力模型选择 LLM vs SLM性能1.更少的算力需求2.部署在更小的设备甚至边缘计算场景上无障碍1.更多开发者和组织可以使用2.具备一定的业务能力,便于企业开发人员使用定制化1.针对特定领域和任务进行微调2.所有权不同的开源小模型应用场景
2、Qianwen-chatOpenELMGemma-2bPhi3知识/文本/聊天NanoDBjina-embeddings-v2-base-cn向量模型图片,代码LLaVAAudioCraftNanoVLM数据算法评估部署如何组织数据企业内部多个数据源的整理如何有效组织数据数据安全Qlora vs Lora参数调整计算本地或云端模型性能提示工程有效性效能存储边缘设备的部署模型压缩回应迭代影响构建行业模型的四大要素Azure AIAzure AI Content SafetyAzure OpenAI ServiceAzure AI TranslatorAzure AI DocumentIntell
3、igenceAzure AI VisionAzure AI SearchAzure AI LanguageAzure AI SpeechAzure AI ServicesPre-trained,turnkey solutions for intelligent applicationsCutting-Edge ModelsAccess to the latest foundation and open-source modelsAzure AI StudioOne place for building and deploying AI solutionsResponsible AI Model
4、Fine TuningModel TrainingAzure Machine LearningFull-lifecycle tools for designing and managing responsible AI modelsPrompt FlowOrchestrationModel CatalogSolutionEvaluationModelBenchmarkingContinuous MonitoringCode-First ExperienceAzure AI InfrastructureState-of-the-art silicon and systems for AI wor
5、kloadsAzure Maia SiliconMicrofluidic CoolingHigh-Bandwidth NetworkingMicrosoft FabricUnified data platformLake House can build data based on businessCloud+local data integrationPrompt flowResult EvaluationCompare modelsbusiness flow integrationONNX RuntimeCross-platformResponseInt4,float32,float16 m
6、ulti-format compatibleAzure Machine LearningFull-lifecycle tools for designing and managing responsible AI modelsPrompt FlowOrchestrationDeploymentDatastoreModel CatalogComputeNvidiaSLM+Azure MLSLM OpsSLMSLMF Fineine-tuning-tuningModelModeldatasetEvaluationDeploymentAzure Machine Learning/Azure AI S
7、tudioFull-lifecycle tools for designing and managing responsible AI modelsMicrosoft OliveMicrosoft OliveMicrosoft Olive是一款非常易用的开源模型优化工具,既可以涵盖Generative AI领域的微调,也可以引用。只需简单的配置,结合使用开源SLM和相关运行时环境(AzureML/本地GPU、CPU、DirectML),通过自动优化即可完成模型的微调或参考,让您找到 最佳模型部署到云或边缘设备。Windows AI Studio 上引入了 Microsoft Olivemicr
8、osoft/Olive:Olive is an easy-to-use hardware-aware model optimization tool that composes industry-leading techniques across model compression,optimization,and compilation.()Microsoft Olive makes SLMOps engineering easier for development teamsMicrosoft Olive-Making the SLMOps process easy混合解决方案混合解决方案
9、SLMSLMF Fineine-tuning-tuningModelModeldatasetEvaluationDeploymentSLMOpsComputeEndpointOlive.config as ServiceOlive.config Model Catalog Olive.config-Compute SettingsOlive.config Qlora&Lora Olive.config ConvertONNX Model Convert(Float16,Float32,Int4)Provides a base for the Copilot StackDemoRunning Phi3 in Jetson+Running Phi3 in AIPCRunning Phi3 in Mobile