1、腾讯云TCHouse-D：基于Apache Doris打造全托管企业级实时数仓平台沈彪腾讯云 TCHouse-D 产品负责人Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia

2、2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023目录2.客户需求及痛点简析3.打造稳定易用的全托管Doris服务1.腾讯云 TCHouse-D 产品概述Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Do

3、ris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asi

4、a 20231腾讯云 TCHouse-D 产品概述Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summi

5、t Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023腾讯云 TCHouse-D 的产品架构及定位源端腾讯云 TCHouse-D 的产品架构HadoopMySQLKafka日志文件对象存储DataInlongOceanusStream loadBroker loadS3 load接入腾讯云 TCHouse-D企业级高可用多维实时分析企业级管控业务应用集群管理集群监控慢查询监控参数配置安全审计扩容变配参数配置多

6、内核版本支持写入性能提升BI工具敏捷BI运营系统实时分析运维监控实时监控广告归因分析业务分析在腾讯云产品体系中的定位聚焦实时数据仓库场景，联动腾讯云Wedata、DataInlong、Oceanus、BI、弹性MapReduce等产品，为客户提供一站式数据开发、治理、分析服务。腾讯云数据仓库 TCHouse-D 基于 Apache Doris 内核构建，兼容 MySQL 协议，与云上大数据生态体系融合，提供丰富的集群管控能力，及完善的巡检告警体系，为客户提供简单易用、极简运维的云上全托管服务。Doris Summit Asia 2023Doris Summit Asia 2023Doris S

7、ummit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 202

8、3Doris Summit Asia 2023Doris Summit Asia 20232客户需求及痛点简析Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Dor

9、is Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023腾讯云 TCHouse-D 在多个行业及场景均有落地案例覆盖 20+行业的落地覆盖多种应用场景实时在线业务广告归因分析实时报表用户行为分析业务经营查询实时大屏游戏汽车教育通用SaaS电商智慧零售社交娱乐金融医疗能源政法工业云交通运智慧建筑Doris Summit Asia 2023Doris Su

10、mmit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023

11、Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023客户来源及主要诉求l 关注端到端整体链路的时效性（分钟/秒级）l 数据实时写入、大数据量OLAP实时数据分析OLTP数据分析l 专家支持，平台兜底l 降低运维成本自建转化l Hive查询加速l 简化技术栈（替代Presto、Impala）l 充分兼容现有大数据系统Hadoop查询加速l 友商迁移降本l 日志场景降本一些资源降本诉求Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Do

12、ris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asi

13、a 2023Doris Summit Asia 2023客户的常见痛点自建 DorisHive湖仓查询加速IcebergRanger兼容Hadoop版本兼容KerberosJar包兼容性字段格式兼容性问题 TIMESTAMP时区问题 VARCHAR精度问题等数据迁移/接入效率 成千上万张表的迁移 分区、分桶设置等联动稳定性问题 失败重试机制 企业级稳定性保障MySQL实时OLAP分析场景l 缺乏简便的数据备份和跨集群迁移方案l 从其他系统迁移至Doris，缺乏应用改造的经验和专家支持l 问题被动发现，缺乏主动预警及解决问题的能力Doris Summit Asia 2023Doris Summi

14、t Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Dor

15、is Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 20233打造稳定易用的全托管Doris服务Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 202

16、3Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023联动腾讯云上生态，快速接入数据并分析展示实时链路MySQL源端数据集成DataInlong数据处理Oceanus腾讯云TCHouse-D腾讯云 BIOLAP实时分析实时报表自动建表自动字段类型适配一键单表/整库同步智能限流限速拉取源端schema

17、，并自动批量建表基于表数据量自动分桶字段类型自动映射自动精度对齐一键时区转换支持库表数据同步支持DML过滤支持DDL同步监控集群负载情况，在合理的区间内自动限流，保障集群健康运行Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit

18、Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023一键数据备份恢复，一键跨集群迁移支持配置一次性、周期性任务支持全量、按库、按表备份及恢复支持精细化任务及实例管理备份实例永久保留可追溯，备份进度可查看n 备份恢复 支持跨Doris集群一键数据/元数据迁移 全量、按库、按表迁移 支

19、持COS数据迁移n 跨集群数据迁移Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2

20、023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023充分适配 Hadoop 生态，简化 Multi Catalog 配置简单进行外部Catalog注册支持on HDFS、COS多种存储方式充分适配2.7、3.5的Hadoop版本支持kerberos认证n Catalog可视化注册支持源、库、表、视图预览支持针对需要的SQL打开Profilen SQL工作区便捷查询Doris Summit Asia 2023Doris Su

21、mmit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023

22、Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023慢查询监控，可视化查询分析 Profile，高效业务调优慢查询监控n 问题感知慢查询清单n 初步分析慢查询日志下载慢查询告警n 深入分析相关SQL打开Profile查看查询计划&执行详情Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Sum

23、mit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023丰富的集群+业务监控告警指标，常用指标一键关注集群监控，直观定位问题健康检查，风险高亮提示

24、实时巡检，问题提前发现群聊/工单，问题快速解决120+监控指标，全面覆盖BE、FE、及业务监控指标；五大分类，查找更方便；重点指标一键关注，更聚焦；Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris

25、Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023集群健康检查，主动进行预警，并给出处理建议集群监控，直观定位问题健康检查，主动提示风险实时巡检，问题提前发现群聊/工单，问题快速解决资源预警：集群负载持续超过80%，磁盘使用率持续超过90%状态告警：节点失活；指标预警：核心指标数值异常等Doris Summi

26、t Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Dor

27、is Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023异常告警+实时巡检，降低 80%+问题升级，减轻对业务影响集群监控，直观定位问题健康检查，主动提示风险实时巡检，问题提前发现群聊/工单，问题快速解决问题主动识别和解决，客户侧+腾讯云共同进行风险控制重要指标可在控制台配置告警客户侧主动实时巡检腾讯云企业微信内部告警提醒高危问题直接拉起工单值班人员主动介入，联系客户解决BE、FE负载数据/系统盘使用率节点存活情况compaction_score等指标集群创建耗时售后及运维人员定位

28、并解决问题站内信短信邮件企业微信Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 20

29、23Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023客户专项支持群+工单体系，保障问题解决效率集群监控，直观定位问题健康检查，主动提示风险实时巡检，问题提前发现群聊/工单，问题快速解决小时级问题响应，资源不足一键扩容/升配，配置优化一键下发生效，Bugfix快速发版解决客户工单/巡检工单客户咨询VIP客户专项支持群咨询建议调优建议集群故障集群故障SRE解答处理疑难问题专家支持群聊助手+值班人员支持Doris Summit A

30、sia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris

31、Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023一些后续规划 支持直接读取对象存储的数据，并通过缓存等策略达到和本地盘相近的性能 支持冷数据自动写入对象存储支持存算分离 产品容器化 支持分时弹性、按集群负载自动弹性资源弹性扩缩容 基于 CCR 跨集群数据同步能力，提升在线服务数据可用性，实现读写负载分离以及多机房容灾备份产品化支持集群读写分离 支持用户配额、和SQL任务配额管理，资源隔离避免任务间互相影响用户及作业资源隔离Doris Summit Asia 2023Doris S

32、ummit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 202

33、3Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023获取更多社区动态与最佳实践Doris Summit 峰会官网：doris- Doris Summit 峰会回放：https:/ Doris 官网：doris.apache.orgApache Doris GitHub： Doris 官方平台：Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023