《SNIA-SDC23-Klein-What-10-Years-Drive-Stats.pdf》由会员分享,可在线阅读,更多相关《SNIA-SDC23-Klein-What-10-Years-Drive-Stats.pdf(37页珍藏版)》请在三个皮匠报告上搜索。
1、1|2023 Backblaze Inc.All Rights Reserved.Virtual ConferenceSeptember 28-29,2021What 10 Years of Drive Stats Data Can Tell UsAndy Klein|Backblaze Inc.2|2023 Backblaze,Inc.All Rights Reserved.Andy Klein8 years-Drive Stats Guy25 years-Marketing6 years-Sys Admin8 years DeveloperAgenda The data weve coll
2、ected Hard drive failure rates SSDs:Failure rates and SMART stats A roundup on hard drive stats AI/ML and drive stats3|2023 Backblaze Inc.All Rights Reserved.240,940Active data drives(Psst:256,310 as of 9/1/2023)3 ExabytesActive data storage capacity5 Data Centers-Rancho Cordova,CA-Phoenix,AZ-Amster
3、dam,Netherlands-Stockton,CA(Nautilus)*-Reston,VAAs of 6/30/2023The Hard Drive Environment4|2023 Backblaze Inc.All Rights Reserved.The Nautilus Data CenterWater cooled No A/CClosed loop cooling30%savings in electricNo adverse effect on wildlifeLocated in a DHS protected site(Port of Stockton)5|2023 B
4、ackblaze Inc.All Rights Reserved.The Drive Stats DataCollected and stored since April 2013Use smartmontools package to collect datahttps:/www.smartmontools.org/All drives in operation each day to create a CSV file for each dayAbout 3,600 files,400 million records,17GB of ZIP data,113GB of raw dataTh
5、e data is open Backblaze Inc.All Rights Reserved.Gross Revenue Retention91%Drive Data Collected Each Daydateserial_numbermodelcapacity_bytesfailureSmart_1_normalizedSmart_1_raw3/15/23Z305B2QNST4000DM000400078703001609827663/15/23PL1331LAHG1S4HHGST HMS5C4040ALE6404000787030016010003/15/23ZACH007ST800
6、0NM0055800811390153/15/23ZA130TTWST8000DM002800831009013/15/23ZA18CEBFST8000NM0055800811405513/15/23PL2331LAH3WYAJHGST HMS5C4040BLE640400078703001601000SMART Stats:Collected using Smartmontools.There are 255 pairs of values per drive.SMART Stat Attributes,e.g.Smart_1
7、:Read Error Rate Smart_5:Reallocated Sector CountSmart_9:Power On HoursMore SMART statsMore Drivesen.wikipedia.org/wiki/S.M.A.R.T.Drive Day:The data collected for one drive for one day.Schema as of 3/31/20237|2023 Backblaze Inc.All Rights Reserved.More information is on the wayNew data fields in Q2:
8、Vault ID and Pod IDComing Soon:Data Center&Server TypeThings we can learnFailure Rates by Storage Server TypeFailure Rates by Data CenterTemperature by Storage Server TypeTemperature by Data Center8|2023 Backblaze Inc.All Rights Reserved.Whats a Drive FailureReactive FailureThe drive will not spin u
9、p or connect to the OS.The drive will not sync or stay synced in a storage array.Proactive FailureTriggered by ATA,FSCK,etc.Combined with SMART dataReviewed by Backblaze before action is takenData Center:Sac0Pod:pod-000-1113-01Drive:drive_0057Tasks:Replace Data Drive Action:ProactiveReason:High Offl
10、ine Uncorrectable(SMART)Brand:HGSTModel:HGST HUH721212ALN604Serial:8AJK007BHSize:12TB Drive Notes:5 Reallocated_Sector_Ct-82197 Current_Pending_Sector-276198 Offline_Uncorrectable-266199 UDMA_CRC_Error_Count-09 Power_On_Hours-23422Found ATA error that is 2 hours old-CONSIDER REPLACING THIS DRIVE9|20
11、23 Backblaze Inc.All Rights Reserved.Computing Annualized Failure Rate(AFR)This method accounts for drives with different drive days within the period.1.Define AFR cohort and period:a.Cohort=Drive Model:All models active as of 6/30/2023b.Period=Q2 20232.Obtain Drive Days and Drive Failures for the c
12、ohort and period.a.Drive Days=21,408,175b.Drive Failures=1,339c.Drive Count=240,9403.Apply Formula:AFR=(Drive Failures/(Drive Days/365)*100AFR=(1,339/(21,408,175/365)*100=2.28%10|2023 Backblaze Inc.All Rights Reserved.Drive Stats ReportsQuarterly ReportsHDDs report each quarterSSDs report twice a ye
13、arResults for Most recent quarterAnnualLifetimeFind reports at for drive stats11|2023 Backblaze Inc.All Rights Reserved.more models(31 this quarter)Backblaze Hard Drives Annualized Failure Rates for Q2 2023Reporting period:4/1/2023 thru 6/30/2023 for drives models active as of 6/30/2023MFGModelDrive
14、SizeDriveCountAvg.Age(months)DriveDaysDriveFailuresAFRHGSTHMS5C4040ALE6404TB3,621 83.2326,504 4 0.45%HGSTHMS5C4040BLE6404TB11,934 80.11,083,231 22 0.74%HGSTHUH728080ALE6008TB1,115 6299,279 9 3.31%HGSTHUH728080ALE6048TB90 71.18,094 3 13.53%HGSTHUH721212ALE60012TB2,606 44.8232,974-0.00%HGSTHUH721212AL
15、E60412TB13,203 271,181,748 42 1.30%HGSTHUH721212ALN60412TB10,537 50.7941,603 164 6.36%SeagateST4000DM0004TB17,899 91.91,607,828 167 3.79%SeagateST6000DX0006TB883 98.380,411 3 1.36%SeagateST8000DM0028TB9,354 80.6842,239 114 4.94%SeagateST8000NM000A8TB153 11.612,088-0.00%SeagateST8000NM00558TB14,118 6
16、8.81,270,271 215 6.18%12|2023 Backblaze Inc.All Rights Reserved.Backblaze Hard Drives Lifetime Annualized Failure RatesReporting period:4/20/2013 thru 6/30/2023 for drives models active as of 6/30/2023MFGModelDriveSizeDriveCountDriveDaysDriveFailuresAFRConfidence IntervalHGSTHMS5C4040BLE6404TB11,934
17、 35,700,373 413 0.42%0.40%0.50%HGSTHUH728080ALE6048TB90 44,529 8 6.56%2.80%12.90%HGSTHUH721212ALE60412TB13,203 10,730,974 172 0.59%0.50%0.70%SeagateST4000DM0004TB17,899 76,254,477 5,425 2.60%2.60%2.70%SeagateST6000DX0006TB883 4,025,712 99 0.90%0.70%1.10%SeagateST8000DM0028TB9,354 24,112,992 930 1.41
18、%1.30%1.50%SeagateST8000NM000A8TB153 53,743-0.00%0.00%3.80%SeagateST12000NM001G12TB13,029 11,876,363 270 0.83%0.80%1.00%SeagateST14000NM001G14TB10,790 9,351,776 312 1.22%1.10%1.40%SeagateST16000NM001G16TB27,255 11,827,250 276 0.85%0.80%1.00%ToshibaMG07ACA14TA14TB38,101 37,163,688 987 0.97%0.90%1.00%
19、ToshibaMG08ACA16TEY16TB5,289 2,910,620 36 0.45%0.40%0.80%WDCWUH721414ALE6L414TB8,432 7,763,187 71 0.33%0.30%0.40%WDCWUH721816ALE6L416TB14,099 3,699,632 32 0.32%0.20%0.50%13|2023 Backblaze Inc.All Rights Reserved.Lifetime Gold Medal Winners as of Q2 2023(1)-Not available for retail sale in the US and
20、 Canada(2)Rebranded as Western Digital MFGModelDriveSizeAFRConfidence IntervalWDCWUH721816ALE6L0(1)16TB0.13%0.10%0.30%WDCWUH721816ALE6L416TB0.32%0.20%0.50%ToshibaMG08ACA16TEY16TB0.45%0.40%0.80%WDCWUH721414ALE6L414TB0.33%0.30%0.40%ToshibaMG07ACA14TA14TB0.97%0.90%1.00%HGSTHUH721212ALE600(2)12TB0.33%0.
21、20%0.50%HGSTHUH721212ALE604(2)12TB0.46%0.40%0.60%SeagateST12000NM001G12TB0.83%0.80%1.00%HGSTHUH728080ALE600(3)8TB0.94%0.70%1.20%SeagateST6000DX000(3)6TB0.90%0.70%1.10%HGSTHMS5C4040BLE640(3)4TB0.42%0.40%0.50%Criteria 2M Drive Days=0.5 Confidence Interval(3)Available as rehab drives only14|2023 Backbl
22、aze Inc.All Rights Reserved.Lets Talk About SSDs15|2023 Backblaze Inc.All Rights Reserved.SSDs versus HDDsSpeed-SSDElectricity-SSDCost/GB-HDDReliability-?The Tale of the Tape16|2023 Backblaze Inc.All Rights Reserved.SSD versus HDD ReliabilityBoot Drives Boot the server Regular daily activity to read
23、/write/delete log files for system access and diagnostics.SDD Count2,558HDD Count 1,50517|2023 Backblaze Inc.All Rights Reserved.Life SSD versus HDD Annualized Failure Rates thru 2021Will the SSD Failure Rate?A.Rise like the HDDsB.Rise,but slowerC.Level out18|2023 Backblaze Inc.All Rights Reserved.L
24、ife SSD versus HDD Annualized Failure Rates thru 2022Will the SSD Failure Rate?A.Rise like the HDDsB.Rise,but slowerC.Level outD.Drop19|2023 Backblaze Inc.All Rights Reserved.SSD versus HDD Reliability Year 5HDD Annualized Failure Rate(Lifetime)3.55%SSD Annualized Failure Rate(Lifetime)0.92%Adrian!2
25、0|2023 Backblaze Inc.All Rights Reserved.2022|20How SMARTare SSDs?A look at SSD SMART StatsMaking Sense of SSD SMART Statshttps:/ Backblaze Inc.All Rights Reserved.Lets Get SMARTedSelected three SSDs from our collection all are 250GB modelsSeagate BarraCuda 120 SSD ZA250CM10003 20 SMART attribute pa
26、irsWDC WDS250G2B0A 25 SMART attribute pairsCrucial CT250MX500SSD1 23 SMART attribute pairCommon Attributes for all three modelsSMART 9:Power-On HoursSMART 12:Power Cycle CountSMART 194:TemperatureSMART 173:SSD Wear LevelingSMART 174:Unexpected Power Loss Count22|2023 Backblaze Inc.All Rights Reserve
27、d.Lifetime PercentageSMART 169:Remaining Lifetime Percentage(WDC)The approximate life left from a combination of program-erase cycles and available reserve blocks of the device.A new SSD starts at“100”for the Normalized value and decrease to“0”.SMART 231:Life Left(Seagate)The approximate life left f
28、rom a combination of program-erase cycles and available reserve blocks of the device.A new SSD starts at“100”for the Normalized value and decreases with a threshold value of“10”for replacement.A“0”value means the drive is in read-only mode.SMART 202:Percentage of Lifetime Used(Crucial)How much of th
29、e drives projected lifetime has been used at any point in time.A new SSD will report“0”for the Normalized value and rise to“100,”as it being used.23|2023 Backblaze Inc.All Rights Reserved.Interesting SSD AttributesSMART 230:Drive Life Protection Status(WDC)A comparison of the SSDs usage trajectory v
30、ersus the expected life curve Yes,there is a usage trajectory calculation and value(SMART 169)Yes,there is a defined expected life curveSMART 232:Endurance Remaining(Seagate and WDC)Percentage of physical erase cycles completed versus the maximum number the drive is designed to endureDoes not consid
31、er available reserved blocks but can be useful given that erasing SSD blocks at an accelerated rate often leads to having to utilize available reserved blocks laterSMART 210:RAIN Successful Recovery Page Count(Crucial)Redundant Array of Independent NAND(RAIN)is similar data redundancy using RAIDRAIN
32、 redundancy is accomplished on the SSD itself and is user transparent24|2023 Backblaze Inc.All Rights Reserved.How Long Can You Store an SSD Without Power?Current thinkingAbout a year for new drives,decreasing from there as drives ageUsing SMART Attributes to compute storage timeCrucial(a)SMART 202N
33、ormalizedSeagate(b)SMART 231NormalizedWDC(b)SMART 169NormalizedMaximum Storage Period01001001 year5050506 months1001001 month(a)Documented by Crucial(b)No documentation found 25|2023 Backblaze Inc.All Rights Reserved.What About Hard DrivesSome random bits about what we know about hard drives26|2023
34、Backblaze Inc.All Rights Reserved.Average Age of Drive FailureSOURCEFAILED DRIVE COUNTAVERAGE FAILED AGESecure Data Recovery2,0072 years 10 monthsBackblaze17,155(all models)2 years 6 monthsBackblaze3,379(models no longer in production)2 years 7 months27|2023 Backblaze Inc.All Rights Reserved.Average
35、 Age of Drive Failure by Drive SizeDriveFailureAverage Age of FailureDrivesAvg Age of RemainingSizeCountYearsMonthsRemainingYearsMonths1 TB76 4701.5 TB704 3902 TB309 3603 TB2,168 2104 TB6,098 3234,614715 TB2 0806 TB168 21884818 TB2,160 3525,16351110 TB120 3101,1595412 TB3,579 1860,24021114 TB1,368 1
36、759,4022416 TB399 01055,4311228|2023 Backblaze Inc.All Rights Reserved.Drive Failure Over Time:202229|2023 Backblaze Inc.All Rights Reserved.Drive Failure Over Time:202230|2023 Backblaze Inc.All Rights Reserved.Predicting Hard Drive Failure2016Paper:Predicting Disk Replacement towards Reliable Data
37、CentersAuthors:Botezatu,Mirela&Giurgiu,Ioana&Bogojeska,Jasmina&Wiesmann,Dorothea.(2016).Location:https:/dl.acm.org/doi/10.1145/2939672.29396992021Paper:Interpretable Predictive Maintenance for Hard DrivesAuthors:Maxime Amram,Jack Dunn,Jeremy J.Toledano,Ying Daisy ZhuoLocation:https:/ others31|2023 B
38、ackblaze Inc.All Rights Reserved.Using Machine Learning to Predict Hard Drive FailureBackblaze Drive Stats Data+Optimal Decision Trees+Survival Curves=Predictions on long-term&short-term drive healthDrive Stats Data:https:/ Backblaze Inc.All Rights Reserved.Optimal Decision TreesInterpretable AIwww.
39、interpretable.ai33|2023 Backblaze Inc.All Rights Reserved.Predicting Short-term Drive HealthExample survival curves for selected cohorts5(raw)123(norm)=29.57(raw)=359546190.5Healthy DrivesInterpretable AIwww.interpretable.aiNode 1534|2023 Backblaze Inc.All Rights Reserved.Predicting Short-term Drive
40、 HealthExample survival curves for selected cohorts5(raw)=12187(norm)=17.57(raw)=492733657.5Unhealthy DrivesInterpretable AIwww.interpretable.aiNode 2435|2023 Backblaze,Inc.All Rights Reserved.Summary The data weve collected Hard drive failure rates SSDs:Failure rates and SMART stats A roundup on hard drive stats AI/ML and drive stats36|2023 Backblaze Inc.All Rights Reserved.Questions37|2023 Backblaze Inc.All Rights Reserved.Please take a moment to rate this session.Your feedback is important to us.