1、Checkpoint原理浅析杨向博 DBACheckpoint 定义Checkpoint 触发条件Checkpoint 会做什么Checkpoint skipped机制Checkpoint 过程记录Checkpoint 定义Checkpointsarepointsinthesequenceoftransactionsatwhichitisguaranteedthattheheapandindexdatafileshavebeenupdatedwithallinformationwrittenbeforethatcheckpoint.Atcheckpointtime,alldirtydatapa

2、gesareflushedtodiskandaspecialcheckpointrecordiswrittentothelogfile.(ThechangerecordswerepreviouslyflushedtotheWALfiles.)Intheeventofacrash,thecrashrecoveryprocedurelooksatthelatestcheckpointrecordtodeterminethepointinthelog(knownastheredorecord)fromwhichitshouldstarttheREDOoperation.Anychangesmadet

3、odatafilesbeforethatpointareguaranteedtobealreadyondisk.Hence,afteracheckpoint,logsegmentsprecedingtheonecontainingtheredorecordarenolongerneededandcanberecycledorremoved.(WhenWALarchivingisbeingdone,thelogsegmentsmustbearchivedbeforebeingrecycledorremoved.)简单来说，checkpoint就是一个事务顺序的记录点。checkpoint主要是进

4、行刷脏页，redo时会参考checkpoint进行日志回放。除了刷脏之外还会更新一些位点信息，清理一些不再需要的wal。Checkpoint 触发条件在PostgreSQL中Checkpoint是由checkpointer进程执行的。Checkpointer进程的主流程是一个无条件的for循环，在未触发checkpoint时一直在WaitLatch中sleep，也就是在epoll_wait中观察list链表，查看是否有事件句柄已经就绪（某个条件在触发checkpoint）；如果已经存在就绪事件，则wakeup（通过SetLatch中writepipe的方式wakeup），执行checkpoin

5、t。Checkpoint 触发条件触发checkpoint的Flags:/*ThesedirectlyaffectthebehaviorofCreateCheckPointandsubsidiaries*/#defineCHECKPOINT_IS_SHUTDOWN0 x0001/*Checkpointisforshutdown*/#defineCHECKPOINT_END_OF_RECOVERY.0 x0002/*Likeshutdowncheckpoint,but*issuedatendofWALrecovery*/#defineCHECKPOINT_IMMEDIATE0 x0004/*Do

6、itwithoutdelays*/#defineCHECKPOINT_FORCE0 x0008/*Forceevenifnoactivity*/#defineCHECKPOINT_FLUSH_ALL0 x0010/*Flushallpages,includingthose*belongingtounloggedtables*/*TheseareimportanttoRequestCheckpoint*/#defineCHECKPOINT_WAIT0 x0020/*Waitforcompletion*/#defineCHECKPOINT_REQUESTED0 x0040/*Checkpointr

7、equesthasbeenmade*/*Theseindicatethecauseofacheckpointrequest*/#defineCHECKPOINT_CAUSE_XLOG0 x0080/*XLOGconsumption*/#defineCHECKPOINT_CAUSE_TIME0 x0100/*Elapsedtime*/以手动触发checkpoint为例：caseT_CheckPointStmt:if(!superuser()ereport(ERROR,(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),errmsg(mustbesuperuserto

8、doCHECKPOINT);RequestCheckpoint(CHECKPOINT_IMMEDIATE|CHECKPOINT_WAIT|(RecoveryInProgress()?0:CHECKPOINT_FORCE);break;Checkpoint 会做什么以FlushDatafile为例：1)open打开文件；2)pwrite写入修改内容，DBBLCLKSZ为8k，OS为4k，非原子写，FPW特性兜底；3)fsync将缓冲块立即落盘；1、Flush dirty pageCheckpoint 会做什么/*Updatethecontrolfile.*/LWLockAcquire(Contr

9、olFileLock,LW_EXCLUSIVE);if(shutdown)ControlFile-state=DB_SHUTDOWNED;ControlFile-checkPoint=ProcLastRecPtr;ControlFile-checkPointCopy=checkPoint;ControlFile-time=(pg_time_t)time(NULL);/*crashrecoveryshouldalwaysrecovertotheendofWAL*/ControlFile-minRecoveryPoint=InvalidXLogRecPtr;ControlFile-minRecov

10、eryPointTLI=0;/*PersistunloggedLSNvalue.Itsresetoncrashrecovery,sothisgoes*unusedonnon-shutdowncheckpoints,butseemsusefultostoreitalwaysfordebuggingpurposes.*/SpinLockAcquire(&XLogCtl-ulsn_lck);ControlFile-unloggedLSN=XLogCtl-unloggedLSN;SpinLockRelease(&XLogCtl-ulsn_lck);UpdateControlFile();LWLockR

11、elease(ControlFileLock);/*Updateshared-memorycopyofcheckpointXID/epoch*/SpinLockAcquire(&XLogCtl-info_lck);XLogCtl-ckptFullXid=checkPoint.nextFullXid;SpinLockRelease(&XLogCtl-info_lck);2、Update some pointsCheckpoint 会做什么3、Remove old wal/*Updatetheaveragedistancebetweencheckpointsifthepriorcheckpoint

12、*exists.*/if(PriorRedoPtr!=InvalidXLogRecPtr)/*根据ptr偏移量，预估出两次checkpoint间产生的wal量CheckPointDistanceEstimate*/UpdateCheckPointDistanceEstimate(RedoRecPtr-PriorRedoPtr);/*Deleteoldlogfiles,thosenolongerneededforlastcheckpointto*preventthediskholdingthexlogfromgrowingfull.*/XLByteToSeg(RedoRecPtr,_logSeg

13、No,wal_segment_size);/*根据minwal_keep_segments,min(replication_slot.restart_lsn)计算出_logSegNo，比_logSegNo早的日志后续将会被清理掉*/KeepLogSeg(recptr,&_logSegNo);_logSegNo-;/*首先根据CheckPointDistanceEstimate结合一套公式，计算出开始回收重用的recycleSegNo，从这个日志开始回收重用（wal_recycle默认开启，主要是保留日志并rename为新的序列号，回收一个序列号加一）*/*然后将_logSegNo之前并已经归档

14、（如果开启归档）的wal都清理掉*/RemoveOldXlogFiles(_logSegNo,RedoRecPtr,recptr);Checkpoint skipped机制/*Ifthisisntashutdownorforcedcheckpoint,andiftherehasbeenno*WALactivityrequiringacheckpoint,skipit.Theideahereisto*avoidinsertingduplicatecheckpointswhenthesystemisidle.*/if(flags&(CHECKPOINT_IS_SHUTDOWN|CHECKPOINT

15、_END_OF_RECOVERY|CHECKPOINT_FORCE)=0)if(last_important_lsn=ControlFile-checkPoint)WALInsertLockRelease();LWLockRelease(CheckpointLock);END_CRIT_SECTION();ereport(DEBUG1,(errmsg(checkpointskippedbecausesystemisidle);return;当非停库、redo完成、强制触发checkpoint时，如果数据库没有写入操作，则直接return不进行Flushdirtypage等操作Checkpoin

16、t skipped机制wal_keep_segments配置256，不存在复制槽，归档无报错的情况下保留17000+个wal打开debug日志后，checkpoint_timeout触发checkpoint后发现是进入了checkpointskipped机制手动强制触发checkpoint后，wal日志清理成功Checkpoint 过程记录1)当log_checkpoints=on时，pglog会记录checkpoint过程；2)使用命令行工具pg_controldata解析pg_control文件；3)登陆数据库查询系统函数pg_control_checkpoint()；THANKS谢谢观看