1、分布式数据库与分布式事务处理技术演讲人：李海翔CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CONTENT目录分分布布式式事事务务型型数

2、数据据库库的的理理念念与与背背景景分布式事务处理技术第三代分布式数据库强一致性+高性能=鱼和熊掌兼得CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek

3、 20231.1 分布式数据库的基础高可靠（high reliability）定义为一个服务连续无故障运行的时间，无故障运行的时间越长，可靠性就越高。99.999%高可用定义为在足够长的时间里，一个服务可用的时间，服务可用时间越长越好高性能低隔离别+高性能可串行化+低性能易用性例例如如：1 MySQL可可串串行行化化，性性能能差差2 MySQL是是单单机机系系统统推推测测：分分布布式式的的可可串串行行化化，是是不不是是性性能能更更差差？CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek

4、 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20231.1 分布式数据库的基础数据的正确性并发的性能事事务务处处理理领领域域的的怪怪现现象象1 有有多多少少个个数数据据异异常常？-不不知知道道2 为为什什么么性性能能变变低低？-不不知知道道事事务务处处理理技技术术的的两两个个核核心心问问题题1 没没有

5、有任任何何数数据据异异常常2 并并发发导导致致性性能能变变差差事事务务处处理理技技术术CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20231.

6、2 怪现象-强一致+低性能弱弱一一致致性性强强一一致致性性缺点不能100%确保数据一致性性能低（额外判断，耗时，如SSI）开发者需要在数据库上层消除数据不一致认知8种有限个数的不一致性现象（数据异常）需要认知所有的不一致性现象优点100%确保数据的准确性，用户编程简单用户/开发者编程简单主流观点够用了，Oracle的RC是日常选择因性能不好而被弃用新问题如果性能足够好，将极大简化用户的开发负担隔隔离离级级别别，与与一一致致性性的的关关系系？-不不知知道道陷陷入入如如下下具具体体的的问问题题，就就事事论论事事CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWe

7、ek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20231.2 怪现象-强一致+低性能用户：要求高性能开发者：用弱隔离级别用户：要求数据正确开发者：主动加锁数据库：性能降低SelectFor Update数数据据库库使使用用怪怪圈圈灵

8、灵魂魂问问题题：Q1：我们能精准地说出，什么情况下应该用For Update、什么情况下不用吗？现现实实问问题题：Q2：为什么我们需要精准地知道什么情况下应该用For Update、什么情况下不用？CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2

9、023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20231.2 怪现象-强一致+低性能测试目的：验验证证用用户户主主动动加加锁锁、不不加加锁锁的的性性能能差差异异背景：TPC-C，数据集只有30GB，MySQL 的page全部缓存在buffer pool中R RC C隔隔离离级级别别不加锁用户加锁并并发发数数tpmctpmc加锁影响1 16 65-3.68%328-7.69%640-9.52%6824-11.04%2561-14.35

10、%5096-18.28%04113-21.25%RR（可可重重复复读读）不加锁用户加锁并并发发数数tpmctpmc加锁影响1 16 68-6.05%321-9.33%644-10.34%4165-12.70%2563-18.61%5418-23.17%8620-23.12%用用户户主主动动加加锁锁，有有性性能能损损失失CoreJavaWeek 2023CoreJavaWeek 2023CoreJav

11、aWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20231.2 怪现象-强一致+低性能测试目的：验验证证用用户户主主动动加加锁锁、不不加加锁锁的的性性能能差差异异背景：测试数据集只有30GB，MySQL 的page全部缓存在bu

12、ffer pool中，TPC-C性能差异并发数RC+用户加锁RR+用户加锁S088806564468247256380969862098637用用户户主主动动加加锁锁，性性能能接接近近于于可可串串行行化化CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023Cor

13、eJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20231.2 怪现象-强一致+低性能测试目的：验验证证用用户户主主动动加加锁锁、不不加加锁锁的的性性能能差差异异背景：Percona Server 8.0.18-9：验验证证用用户户主主动动加加锁锁、不不加加锁锁的的性性能能差差异异背景：测试的recordcount设置为

14、4000w，数据实际容量为90GB，稍大于buffer pool的大小，YCSB的只读场景只读场景，用户加锁性能损失很大只读场景，系统内部加锁，性能损失小CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023Co

15、reJavaWeek 2023CoreJavaWeek 20231.2 怪现象-强一致+低性能测试目的：验验证证用用户户主主动动加加锁锁、不不加加锁锁的的性性能能差差异异Pg14tpmCRC164.9W可串行化67W可串行化+For Update40.4W结结论论：【可可串串行行化化】优优于于【低低隔隔离离级级别别+用用户户主主动动加加锁锁】1 基基于于数数据据库库的的开开发发变变得得简简单单 2 数数据据库库的的性性能能有有保保证证3 分分布布式式数数据据库库应应该该更更加加慎慎用用锁锁CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023Co

16、reJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CONTENT目录分分布布式式事事务务型型数数据据库库的的理理念念与与背背景景分分布布式式事事务务处处理理技技术术第三代分布式数据库强一致性+高性能=鱼和熊掌兼得CoreJavaWeek 2023

17、CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20232 分布式事务处理技术序值获取异常消除（执行阶段）异常消除（提交前冲突判定）日志（WAL）日志分散（分布式）原子提交/回滚（

18、2PC）提交序值（可见性依据）全全局局读读一一致致性性C全全局局写写一一致致性性C+弱弱隔隔离离级级别别I全全局局写写一一致致性性C+可可串串行行化化I原原子子性性A持持久久性性D与与一一致致性性C持持久久性性D与与一一致致性性C与与原原子子性性A恢复全全局局读读一一致致性性CCoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJava

19、Week 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20232 分布式事务处理技术全局读一致性全局写一致性提交的原子性编编号号各各种种方方案案缺缺点点案案例例1全局事务管理器非去中心化、低效Pg XC2基于封锁的并发访问控制算法+全局可串行化低效某些系统SS2PL+MVCC3全局可串行化+线性一致性所有事件全序排序=所有事务全局排序，低效SpannerSS2PL+MVCC4全局可串行化+混合逻辑时钟+全局事务提交标志数据是否可读，需要通过全局事务提交状态验证，增加通讯

20、次数CockroachDBSSI+MVCC52次读Scalable atomic visibility with ramp transactions增加了通讯轮数，且只能解决读半已提交数据异常学术界的解决方式CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaW

21、eek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20232 分布式事务处理技术全局读一致性全局写一致性提交的原子性写一致性执行阶段写一致性验证阶段快快照照技技术术止止步步于于此此可可串串行行化化需需要要验验证证阶阶段段确确保保CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023Cor

22、eJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20232 分布式事务处理技术分分布布式式事事务务：S SS SI I，基基于于M MV VC CC C的的可可串串行行化化开开始始阶阶段段：要获取全局快照（全局读一致性）执执行行阶阶段段：不要全局加锁（WW冲突通常采取锁机制）验验证证阶阶段段：需需要要进进行行全全局局可可串串行行化化验验证证（可可串串行行化化的的关关键键）提提交交阶阶段段：要获取全局的提交时间戳 2 2P PC C提提交交结结束束阶阶段段

23、：不需要释放全局锁（WW的写锁是局部的）分分布布式式事事务务:快快照照隔隔离离开开始始阶阶段段：要获取全局快照（全局读一致性）执执行行阶阶段段：不要全局加锁（WW冲突通常采取锁机制）验验证证阶阶段段：不需要可串行化验证提提交交阶阶段段：要获取全局的提交时间戳 2 2P PC C提提交交结结束束阶阶段段：不需要释放全局锁（WW的写锁是局部的）CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWe

24、ek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20232 分布式事务处理技术分分布布式式事事务务提提交交阶阶段段，保保证证提提交交操操作作的的原原子子性性全局读一致性全局写一致性提交的原子性CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023Core

25、JavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CONTENT目录分分布布式式事事务务型型数数据据库库的的理理念念与与背背景景分布式事务处理技术第三代分布式数据库强强一一致致性性+高高性性能能 =鱼鱼和和熊熊掌掌兼兼得得CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2

26、023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20233 第三代分布式数据库极简的易用性只能做强一致性的正确性可控时延的“永远在线”的高可用性CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20

27、23CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20233 第三代分布式数据库CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20

28、23CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20233 第三代分布式数据库SpannerCRDBOBxxDB架构SN+文件级共享存储SNSNSN+日志级共享存储事务强一致性严格可串行化因果+可串行化SI严格可串行化、可串行化多写架构-可串行化多写SI多写可串行化多写可扩展-扩展性较弱千台级规模扩展存储无限扩展计算千台级扩展高可用数据多副本提

29、供系统级的高可用，事务级瞬时闪断数据可灵活分布，数据多副本提供系统级的高可用数据多副本提供系统级的高可用数据级高可用：日志服务器；内存级高可用：主主互备、RO切换高可靠-更资料截取（Change Data Capture，CDC）、备份与恢复、管理API，以及99.99%正常运行时间数据备份和日志备份库、表级备份恢复日志级备份恢复数据块级备份恢复可控时延超大时延超低性能高时延（全球级、时延未做特定设计）低性能高时延、扩展节点获高性能低时延高性能、扩展节点获更高性能CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20

30、23CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023感谢聆听演讲人：嘉宾名CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023