《夏锐航-PromQL Got a Boost.pdf》由会员分享,可在线阅读,更多相关《夏锐航-PromQL Got a Boost.pdf(39页珍藏版)》请在三个皮匠报告上搜索。
1、第三届中国 Rust 开发者大会PromQL Got a BOOST:用 Rust 重写 Prometheus 的查询引擎Ruihang Xia Ruihang GitHub:waynexia Losing hair at Greptime Wanna sleep 10 hrs/day Learn JS every year Healing the HyruleABOUTBackground01CONTENTSthe Parser02the Engine03Rust Gives04Outlook0501BackgroundAbout Prometheus and PromQL Why rewr
2、ite?What are boosted?PROMETHEUS AND PROMQLThe de facto standard in cloud-native monitor scenario Has its own query language called“PromQL”rate(api_request_durationhost=“foo”5m)More than a monitor:alert,visualize with many integrationsPROMETHEUS AND PROMQLrate(api_request_durationhost=“foo”5m)Operato
3、rDataFilterRangeSELECT rate(val)OVER(ORDER BY timestamp RANGE BETWEEN 5 minutes PRECEDING AND CURRENT ROW )AS rateFROM api_request_durationWHERE host=fooOURSGreptimeTeam/greptimedb Time-Series Database(could-naive,hybrid-workload,distributed)SQL and table modelWHY REWRITETarget to be the most powerf
4、ul drop-in replacement PromQL has a good UX for time-series data And it can be better if combine with SQL RIIR,super cooooooool!WHAT ARE BOOSTEDPromQL is extended multi-field support new operators&UDF better performance larger dataset more computation resources SQL is extended introducing the new co
5、ncept we have just made SQLWHAT ARE BOOSTEDSQLT(elemetry)QLPromQL is extended multi-field support new operators&UDF better performance larger dataset more computation resources SQL is extended introducing the new concept we have just made PROMQL VIA SQL(1)prepare data using SQL(2)write your PromQL!0
6、2the ParserTech stack Our choiceOUR CHOICElrpar-generator,yacc compatible PromQL has replaced its hand-written parser with yacc But there are still many slight differencesYACC FILEHOW TO CHOOSEBased on the scenario and pre-requirement E.g.:sqlparser-hand-written combinator nom81,000,000+combinatorpe
7、stcombinelalrpop37,500,000+21,700,000+5,500,000+generatorcombinatorgeneratorDownloadsMethodCrateERROR HANDLINGTerminal v.s.text chumsky,nom,winnow?Still exploring 03the EngineWhich way to rewrite?Apache Arrow&DataFusion Diff between modelsREWRITEAround 1/7 of Prometheus is PromQL A straightforward 3
8、-layer structure Translate to Rust?Or rebuild itDRAWBACK OF PROMETHEUSBad efficiency:calculates point by point Directly interpreters over AST Hard to scale and distributing Hard to extendTRANSLATE OR REBUILDTranslate Pros.:Easy to write Compatibility Cons.:Hard to extend Same performanceRebuild Pros
9、.:Controllable Best fit Cons.:A big workREBUILDBased on the table-oriented model Works like an extension Powered by REBUILDOptimize before executeThe same executor with existing SQL partMultiple formats from one resultBENEFITS OFBuilt-in vectorized operators A complete planner,optimizer and executor
10、 Warm community 04Rust GivesAdvantages from Rust Easy to extend Easy to distribute computationBY USING RUSTNo too much troubles/burdens Powerful static check by Rust Fast feedback&debug with UT Prometheus:almost no unit testPOWERFUL ABSTRACTIONMACRO!Make use of arrow operator Focus only on the logic
11、 Macro takes care of all other thingsDISTRIBUTE ITMake distributed PromQL in ONE DAYKNOW PROMQL BETTERTrack your query with TQL ANALYZEBY REWRITINGMore coming soon!Metric engine More UDF in PromQL Dashboard,alert center .05OutlookRust has joined the game In cloud native areaJOIN THE GAMEPrometheus c
12、ompliance test (last updated on 2022.05)And maybe the first Rust projectJOIN THE GAMEOur parser has supported another project to hack with PromQL Our implementation is also easy to integrateCLOUD NATIVEGolang+.io?From CNCF landscape,the number of Rust projects has exceeded 1/10 of Golangs With some
13、key infrastructure like WebAssemblyWEBASSEMBLYwaynexia/datafusion-playgroundINTEROPERABILITYcoprocessor(returns=value)def add(*params)-vectori64:a=paramsa b=paramsb return int(a)+int(b)Available in PromQL soon!IMAGINE THE FUTUREMonitor data collected&transformed in Vector()Then store and analyze with GreptimeDB()Happy evolve your system()Thanks for listening Q&A