《9. 基于 Apache Doris 构建多场景极速分析体验.pdf》由会员分享，可在线阅读，更多相关《9. 基于 Apache Doris 构建多场景极速分析体验.pdf（30页珍藏版）》请在三个皮匠报告上搜索。

1、基于 Apache Doris 构建多场景极速分析体验王天宜飞轮科技 资深解决方案架构师Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit A

2、sia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023目录2.Apache Doris 核心能力与特点3.Apache Doris 多场景实时分析方案1.Apache Doris 在大数据生态中的定位Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 202

3、3Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit

4、Asia 20231Apache Doris在大数据生态中的定位Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Dori

5、s Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Apache Doris 在数据分析中的定位CDCFlinkSparkRDBMSDTSLOGKafkaIOT报表分析即席分析湖仓联邦分析日志检索分析ETL/ELT高并发数据服务大屏，驾驶舱自助 BI 平台订单、运单分析广告、营销分析用户行为分析AB 实验平台日志分析（替 ES）时序数据分析统一湖仓平台SourceIntegrationData

6、LakeApplicationApache DorisHiveIcebergHudiDoris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asi

7、a 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Apache Doris 主要特性统一 结构化与半结构化统一 在线与离线数据统一 湖仓统一 多元数据建模统一简单高效 极速的分析性能 高效的数据更新能力 丰富的数据集成能力 极致弹性与存算分离 高可用与高可靠 多租户管理能力 可视化管理工具 丰富周边工具Doris Summit Asia 2023Doris Summit

8、Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris

9、 Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 20232Apache Doris实时AdHoc分析场景Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia

10、 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023实时报表痛点问题ODS 源数据层DWD 明细层DWM 汇总层DWS 主题层数据质量管理数据入库及更新问题数据导入时效性要求高，秒级别数据可见基于上游 TP 库的主键需要实时更新数据需要不丢不重，重要数据原子性导入数据查询时效性问题使用人数

11、众多，并发量告查询延迟要求高，延迟控制在秒级别基于场景不同，要求多样的建模方式Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2

12、023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Apache Doris 高性能更新能力MemTableKeyC1C2KeyC1C210000001V6V7KeyC1C200010010V6V8KeyC1C210000001V7V8KeyC1C20001V8SegmentDeleteBitmapRowSet1version 0-5RowSet3version 7-7RowSe

13、t2version 6-6RowSet3version 8-81.Flush to File2.Lookup rowkey in batch,Then update the bitmap forcurrent versionMerge on Write 模型Merge on Write 更新能力实现原理：主键索引+Delete Bitmap 实现导入过程成数据删除标记 delete bitmap基于 mow 模型实现查询谓词下推适用场景：适用于小批量实时高频导入，基于主键做高频数据更新支持 UPSERT、条件更新、条件删除、部分列更新、分区覆盖100 并发无 Cache有 CacheMoR23

14、.314.3MoW5.60.3提升倍数4.2x47.6xTPCH 标准测试集，使用 MOW 模型，性能提升近 50 倍Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 202

15、3Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Apache Doris 高性能更新能力游戏行为分析场景存量数据 300 亿，单副本 80TB、70字段15 并发 Upsert，TPS 40w/s游戏行为分析场景半年物流订单分析，200 字段宽表8 并发 Upsert，TPS 6w/s游戏行为分析场景千亿数据月度统计，万亿数据年度统计1

16、0 并发 Upsert，TPS 10w/s消费金融场景宽表更新场景，基于 MOW 的部分列更新200 列，20 并发，每个并发更新 10 列数据可见性明显降低，30s-10s某客户 PoC 压力测试40 并发 Upsert，10s CheckpointMOW 表稳定导数，TPS 7.5w/sDoris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023

17、Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Apache Doris 数据建模能力CREATE TABLE doris_agg_tab(store_id INT

18、,date INT,amount BIGINT SUM DEFAULT 0,)AGGREGATE KEY(store_id,date)DISTRIBUTED BY HASH(store_id)BUCKETS 32store_iddateamount-151store_iddateamount-151500store_iddateamount-154000原表新增结果AGGREGATE KEY报表统计、指标计算order_iddatestatus102-14完成支付202-15完成支付order

19、_iddatestatus102-14待支付202-15完成支付order_iddatestatus102-14完成支付原表新增结果CREATE TABLE doris_uni_tab(order_id INT,date INT,status VARCHAR(20)SUM DEFAULT 0,)UNIQUE KEY(order_id)DISTRIBUTED BY HASH(order_id)BUCKETS 32UNIQUE KEY订单状态、用户状态user_iddateaction102-14浏览202-15收藏user_iddateaction102-16购买user_iddateactio

20、n102-14浏览202-15收藏102-16购买原表新增结果CREATE TABLE doris_dup_tab(user_id INT,date INT,action VARCHAR(60)SUM DEFAULT 0,)UNIQUE KEY(order_id)DISTRIBUTED BY HASH(order_id)BUCKETS 32DUPLICATE KEY明细数据、行为数据Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2

21、023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Apache Dori

22、s 极速分析能力100 并发V1.1.5V1.2.4V2.0.0(倒排)SQL1 平均执行时间0.1300.1400.023SQL1 平均执行时间1.1780.3970.048SQL1 平均执行时间0.6530.2840.028SQL1 平均执行时间4.6010.9720.055SUM(单位ms)6.5621.7930.51006date2005206storeX01X02X03X04X05X06prodCUACUBCUCCUDCUECUFcustdate:1001,store:201,prod:X01.date:1002,st

23、ore:202,prod:X02.date:1003,store:203,prod:X03.date:1004,store:204,prod:X04.date:1005,store:205,prod:X05.date:1006,store:206,prod:X06.行存Storage EngineBackendFrontendSQL ParserAnalyzerShort-Circuit PlanRPCSQL1 cache short-circuit plan1SQLN cache short-circuit planNPrepareStatement Map高并发查询痛点：以点查 SELEC

24、T*FROM user_table WHERE id=xxx 为例宽表带来 IOPS 放大的问题执行引擎和优化器对于点查操作过重SQL 解析规划由 FE 负责，高并发易形成瓶颈Apache Doris 面向高并发数据服务优化方案：引入行列混存，解决 IOPS 瓶颈引入 PrepareStatement，解决解析瓶颈规划点查短路径优化，减少框架开销高并发数据服务某手机厂商实时行报表查询场景，使用倒排索引，性能提升 45 倍Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023

25、Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit A

26、sia 2023Apache Doris 解决方案收益极致查询性能向量化 MPP 查询Pipeline 执行引擎多维度分析多表物化视图基于 AGG 模型的预聚合不丢不重事务性导入 2PC数据多副本存储高并发查询分区分桶剪裁行存储加速某头部短视频公司面向广告主在线报表平台，2000 台高配物理机，并发 QPS 数万每秒。Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Su

27、mmit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 20233Apache Doris用户行为分析及人群圈选Doris Summit Asia 20

28、23Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit

29、 Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023用户行为分析业务特点行为数据属性数据交易数据事件分析漏斗分析留存分析归因分析路径分析分布分析人群预估人群洞察人群圈选人群查看A/B 测试产品迭代精细化运营人群吸 引 力转化路径中断路径画像数据采集与处理数据存储与分析数据应用行为分析标签结果事件画像业务特点：标签生成时间不统一，有实时更新需求快速圈选、生成人群包及 TGI 计算行为分析业务特点：业务属性频繁变动SQL 复杂，效率低数据量大，交互式分析，要求秒级延迟Doris Summit Asi

30、a 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Su

31、mmit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023画像和行为分析 Apache Doris 解决方案实时数据同步离线数据同步活跃标签行为标签消费标签会员标签模型标签活跃标签会员维度行为变更移动埋点PC 埋点消费数据会员数据分析函数正交为图管理查询存储存储A/B 测试产品迭代精细化运营人 群转化路径中断路径人 群MQADSDWODS数据变更表结构变更Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Dori

32、s Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia

33、2023Doris Summit Asia 2023丰富的行为分析函数Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023D

34、oris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Apache Doris 极速分析能力倒排索引CREATE INDEX idx_request ON httplogs(request)USING INVERTED;-search for request contains word login SELECT*FROM httplogs WHERE request MATCH login

35、;-search for request contains word error and error SELECT*FROM httplogs WHERE request MATCH_ALL login error;倒排索引实现多维度快速检索持定义分词，实现全检索，加速志场景100 并发V1.1.5V1.2.3V2.0.0(倒排)SQL1 平均执行时间1SQL1 平均执行时间218610763SQL1 平均执行时间68715821127SQL1 平均执行时间6611181977SUM(单位ms)0862Doris Summit Asia 2023Do

36、ris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asi

37、a 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Apache Doris 部分列更新能力uidnameagetag1tag2tagN主键活跃标签StreamLoad 部分列更新：curl -location-trusted-u root:-H partial_columns:true-H column_separator:,-H columns:uid,tag2-T/tmp/update.csv http:/127.0.0.1:48037/api/db1/user_profile/_stream_l

38、oadInsertInto 部分列更新：SET enable_unique_key_partial_update=true;INSERT INTO user_profile(uid,tag2)values(1,xxx);Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit A

39、sia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Apache Doris 数据建模能力CREATE TABLE advertiser_view_record(click_timeDATE,advertiser VARCHAR(60),c

40、hannel VARCHAR(20),user_id INT)DISTRIBUTED BY HASH(click_time)BUCKETS 32;CREATE MATERIALIZED VIEW advertiser_uv ASSELECTadvertiser,channel,bitmap_union(to_bitmap(user_id)FROMadvertiser_view_recordGROUP BYadvertiser,channel;基于物化视图的 UV 统计Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 20

41、23Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit

42、 Asia 2023Doris Summit Asia 2023Apache Doris 标签宽表CREATE TABLE orders_based_wide(user_id INT,tag_dateDATE,age INT,gender CHAR(2),city VARCHAR(50),region VARCHAR(50),tag1 BIGINT,tag2 BIGINT,tag3 BIGINT,tag5 BIGINT,.tagN BIGINT)UNIQUE KEY(user_id)DISTRIBUTED BY HASH(user_id)BUCKETS 64;宽表圈人：SELECT user_

43、idFROM orders_based_table_wideWHERE tag1=23 AND tag2=18 AND tag3=27;宽表人群预估：SELECT COUNT(user_id)FROM orders_based_table_wideWHERE tag1=23 AND tag2=18 AND tag3=27;Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris

44、Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Apache Doris 标签高表CREATE TABLE user_tag_hig

45、ht_bitmap(tag VARCHAR(50),tag_valVARCHAR(50),date_timeDATE,user_idsBITMAP BITMAP_UNION)AGGREGATE KEY(tag,tag_val,date_time)DISTRIBUTED BY HASH(tag)BUCKETS 8;判断用户是否在人群包内：SELECTbitmap_contains(user_ids,13643)FROMuser_segmentsWHEREseg_name=seg_1;高表圈人：SELECT orthogonal_bitmap_intersect_count(user_ids,ta

46、g,tag1,tag2,.)FROMuser_tag_bitmapWHEREtag IN(tag1,tag2);Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Do

47、ris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Apache Doris 高表宽表结合圈人WITH user_segment_act AS(SELECT orthogonal_bitmap_intersect(user_ids,tag,tag01,tag02,.,tag102)FROMuser_tag_bitmap WHEREtag IN(t

48、ag01,tag02,.,tag102)SELECTuser_id,age,gender,city,region,phone,.FROMorders_based_table_wide t0JOIN(SELECT uid FROM user_tag_bitmapLATERAL VIEW explode_bitmap(user_ids)tmp AS uid)as t1ONt0.user_id=t1.uidSTEP1：高表通过正交函数圈人STEP2：位图展开后行转列STEP3：高表结果 join 宽表结果Doris Summit Asia 2023Doris Summit Asia 2023Dori

49、s Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia

50、2023Doris Summit Asia 2023Doris Summit Asia 2023Apache Doris 解决方案收益用户画像千亿数据，秒级人群预估，秒级 10 标签圈人，10 秒 100 标签圈人多维度建模宽表圈人Bitmap 高表圈人丰富分析函数留存、漏斗、归因函数Bitmap 正交函数高性能导入活跃标签部分列更新多数据源导入方案Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit A

51、sia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 20234Apache Doris综合湖仓统一方案Doris S

52、ummit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 202

53、3Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023数据湖与数据仓库能力数据仓库能力数据湖能力 开放的生态 灵活的数据访问方式 可扩展性 高性价比 高数据质量 极致查询性能 数据治理及模型分析 高实时性Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Dori

54、s Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Apache Doris 湖仓一体方案主要应用场景湖仓查询加速数据导入和集成统一

55、查询网关ETL/ELT 加速，写回开放湖仓存储格式便捷的元数据和数据打通元数据映射、cache 和 自动刷新支持几乎所有开放湖仓格式 和 meta store支持 ES 和关系型数据库，并且插件扩展支持外表的认证鉴权，如 keberos,ranger分析加速利用 Doris 高效的分析引擎加速热数据 Cache 到本地支持弹性计算节点，实现计算弹性加速外表处理结果可写入内表，形成加速视图湖仓一体架构Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summi

56、t Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023基于

57、DataLake 的实时业务改造ODSDWDDWSADSKafkaODSDWDDWSADSODSDWDDWSADSVIEWVIEWVIEW行为数据业务数据系统日志爬虫数据实时分析实时大屏实时推荐实时查询湖仓一体方案数据下沉到 Hive、Iceberg、Hudi 等湖产品中Doris 作为计算引擎，进行查询加速离线数据入库方案创建 Catalog 链接INSERT INTO doris_tab SELECT*FROM Catalog_tab分析加速用户生产环境Hive 查询 56minDoris Hive Catalog 加速值 7min通过 Doris 内表查询 30sDoris 实时数仓改造

58、方案Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit

59、 Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Apache Doris 存算分离架构BECACHEBECACHEBECACHECluster 1BECACHEBECACHEBECACHECluster 2DISKFEMETAFEDISKMETAFEDISKMETAS3/OSS/HDFS/MinIO元数据服务层计算集群层共享存储层FEDISKFEDISKFEDISKFEDISKFEDISKFEDISKFEDISK存算一体架构轮科技基于 Apache

60、 Doris 实现了存算分离模式的 SelectDB CloudSelectDB Cloud 将在 V2.1 贡献给 Apache Doris存储资源和计算资源分离，各弹性，更极致性价依赖够稳定、吞吐的共享存储，通常公有云上才有可以通过多 cluster 机制实现负载隔离，读写分离等机制存算分离架构简单易部署，易运维，适合绝多数FE 和 BE节点都可以灵活扩缩容存储持冷热数据分层，将冷存储下沉到对象存储或HDFS可以持弹性计算节点，快速实现计算弹性Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Su

61、mmit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023

62、Doris Summit Asia 2023Apache Doris 解决方案收益Hive 查询 56min，Doris Hive Catalog 加速值 7min，通过 Doris 内表查询 30s弹性扩展高性能存算分离方案计算层弹性可扩展简单部署无需数据迁移简化导入流程统一查询接口支持多源联邦查询避免数据孤岛产生Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Sum

63、mit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023获取更多社区动态与最佳实践Doris Summit 峰会官网：doris- Doris S

64、ummit 峰会回放：https:/ Doris 官网：doris.apache.orgApache Doris GitHub： Doris 官方平台：Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023