《【万凯】如何用 Grafana 构建 Apache SkyWalking UI.pdf》由会员分享,可在线阅读,更多相关《【万凯】如何用 Grafana 构建 Apache SkyWalking UI.pdf(41页珍藏版)》请在三个皮匠报告上搜索。
1、如如何何用用 G Gr ra af fa an na a 构构建建A Ap pa ac ch he e S Sk ky yWWa al lk ki in ng g U UI I万万凯凯Engineer Tetrate Apache SkyWalking PMC MTetrate工程师万万凯凯contents使使用用 G Gr ra af fa an na a 构构建建 S Sk ky yWWa al lk ki in ng g U UI I 的的场场景景和和集集成成要要点点01如如何何使使用用 P Pr ro ommQ QL L S Se er rv vi ic ce e04S Sk ky y
2、WWa al lk ki in ng g 中中的的基基本本概概念念和和 mme et tr ri ic cs s 的的特特性性03S Sk ky yWWa al lk ki in ng g 的的 P Pr ro ommQ QL L S Se er rv vi ic ce e 是是什什么么,能能够够做做什什么么02S Sk ky yWWa al lk ki in ng g G Gr ra af fa an na a P Pl lu ug gi in ns s05构构建建 G Gr ra af fa an na a d da as sh hb bo oa ar rd ds s 实实践践G Gr r
3、a af fa an na a U UI I 作为 S Sk ky yWWa al lk ki in ng g 原生 U UI I 的扩展需要将SkyWalking中的一部分内容和其他系统内容集成在一起混合展现希望使用Grafana的一些生态集成和特有功能喜欢Grafana的展现风格S Sk ky yWWa al lk ki in ng g 与 G Gr ra af fa an na a 集成O OA AP P S Se er rv ve er rPromQL ServiceGraphQL ServiceG Gr ra af fa an na a S Se er rv ve er rProme
4、theus Data Source PluginSkyWalking Data Source PluginG Gr ra ap ph hQ QL L P Pr ro ommQ QL L S Sk ky yWWa al lk ki in ng g 和 P Pr ro ommQ QL LP Pr ro ommQ QL L (P Pr ro omme et th he eu us s Q Qu ue er ry y L La an ng gu ua ag ge e)lets the user select and aggregate time series data in real time.The
5、 result of an expression can either be shown as a graph,viewed as tabular data in Prometheuss expression browser,or consumed by external systems via the HTTP API.(https:/prometheus.io/docs/prometheus/latest/querying/basics/#querying-prometheus)P Pr ro omme et th he eu us s S Se er rv ve er rGrafanaT
6、hird-party API clientsPrometheus web UIP Pr ro ommQ QL L S Sk ky yWWa al lk ki in ng g v ve er rs si io on n =9 9.4 4.0 0P Pr ro ommQ QL L P Pr ro ommQ QL L S Se er rv vi ic ce e i in n S Sk ky yWWa al lk ki in ng gHTTP APIExpressions ParserMetrics TransformationExpressions CalculationG Gr ra ap ph
7、hQ QL L S Se er rv vi ic ce eO OA AP PMMe et ta aD Da at ta aO OA AP P S St to or ra ag ge e a an nd d Q Qu ue er ry y E En ng gi in ne eP Pr ro ommQ QL L S Se er rv vi ic ce e遵循 PromQL 的协议和语法,无缝接入已经支持PromQL的系统基于SkyWalking 原生 GraphQL 查询服务之上提供查询SkyWalking数据的服务支持基于PromQL 表达式的附加查询阶段计算MMe et tr ri ic cs
8、 s 和P Pr ro ommQ QL L表达式endpoint_slaservice=songs,endpoint=GET:/songs/top,layer=MESH 9800,1679020888Metric nameLabelsValueTimestampendpoint_slaservice=songs,endpoint=GET:/songs,layer=GENERAL /100Match nameMatch labelendpoint_slaservice=songs,endpoint=GET:/songs,layer=MESH 10000,1679020888endpoint_sl
9、aservice=songs,endpoint=GET:/songs/top,layer=GENERAL 9900,1679020888endpoint_slaservice=songs,endpoint=GET:/songs,layer=GENERAL 10000,1679020888endpoint_slaservice=songs,endpoint=GET:/songs,layer=GENERAL 100,1679020888Additional CS Sk ky yWWa al lk ki in ng g 中的基本概念L La ay ye er rRepresents an abstr
10、act framework in computer science,such as Operating System(OS_LINUX layer),Kubernetes(k8s layer).This layer would be the owner of different services detected from different technologies.S Se er rv vi ic ce eS Se er rv vi ic ce e I In ns st ta an nc ce eRepresents a set/group of workloads which provi
11、des the same behaviors for incoming requests.An individual workload in the Service group.E En nd dp po oi in nt tA path in a service for incoming requests.P Pr ro oc ce es ss sAn operating system process.In some scenarios,a service instance is not a process,such as a pod Kubernetes could contain mul
12、tiple processes.S Se er rv vi ic ce e/I In ns st ta an nc ce e/E En nd dp po oi in nt t/P Pr ro oc ce es ss s R Re el la at ti io on nRepresents the relation between Service/Instance/Endpoint/PMMe et tr ri ic cs s 和 T Tr ra af ff fi ic cS So ou ur rc ce es sA Ag gg gr re eg ga at ti io on n /P Pr re
13、 e-c ca al lc cu ul la at ti io on nD Do owwn ns sa ammp pl li in ng gMMe et tr ri ic cs sMinuteHourDayxx_cpmxx_sla.xx_xxxOAL/MALTimeBucketNameEntityValue.T Tr ra af ff fi ic cTraffic dispatcherInstance EndpointService.Processs se er rv vi ic ce e_ _c cp pmm=from(Service.*).cpm();-name:mme et te er
14、r_ _o oa ap p_ _i in ns st ta an nc ce e_ _c cp pu u_ _p pe er rc ce en nt ta ag ge e exp:(process_cpu_seconds_total*100).sum(service,host_name).rate(PT1M).instance(service,host_name,Layer.SO11Y_OAP)O OA AL L:MMA AL L:E En nt ti it ty y 和 S Sc co op pe eRepresents the metrics belonging and used for
15、the query.Represents the metrics level and in query stage represents the Scope catalog,Scope catalog provides high-dimension classifications for all scopes as a hierarchy structure.E En nt ti it ty yS Sc co op pe eS Sc co op pe eE En nt ti it ty y I In nf fo oServiceService(include layer info)Servic
16、eInstanceService,ServiceInstanceEndpointService,EndpointServiceRelationService,DestServiceServiceInstanceRelationServiceInstance,DestServiceInstanceEndpointRelationEndpoint,DestEndpointProcessService,ServiceInstance,ProcessProcessRelationProcess,ServiceInstance,DestPS Sk ky yWWa al lk ki in ng g P P
17、r ro omme et th he eu us sE Ex xa ammp pl le e Metric nameMetric nameservice_instance_cpmLayerLabellayer=GENERALServiceLabelservice=agent:songsServiceInstanceLabelservice_instance=agent:songs_instance_1EndpointLabelCommon Value(long)Value 6Labeled Value(Text)Label.,Value.TimeBucketTimestamp168128580
18、0S Sk ky yWWa al lk ki in ng g MMe et tr ri ic cs s 与 P Pr ro omme et th he eu us s 对应关系service_instance_cpm service=agent:songs,service_instance=agent:songs_instance_1,layer=GENERAL 6 1681285800endpoint_percentileservice=agent:songs,endpoint=GET:/songs,layer=GENERAL,label=P50 3 1681285800endpoint_p
19、ercentileservice=agent:songs,endpoint=GET:/songs,layer=GENERAL,label=P90 6 1681285800C Co ommmmo on n V Va al lu ue e (6 6)L La ab be el le ed d V Va al lu ue e (P P5 50 0,3 3|P P9 90 0,6 6)H HT TT TP P A AP PI IP Pr ro ommQ QL L H HT TT TP P A AP PI I说说明明示示例例/api/v1/query查询实时metric数据/api/v1/query?q
20、uery=service_cpmservice=agent:songs,layer=GENERAL/api/v1/query_range查询时间范围metric数据,step根据查询时间范围自动返回/api/v1/query_range?query=service_cpmservice=agent:songs,layer=GENERAL&start=1677479336&end=1677479636/api/v1/series查询某个metric的label name/value 组合的集合查询有哪些服务:/api/v1/series?match=service_trafficlayer=GE
21、NERAL&start=1677479336&end=1677479636/api/v1/labels查询某个metric的label name的集合查询instance_jvm_cpu 有哪些label:/api/v1/labels?match=instance_jvm_cpu/api/v1/label/values查询某个label value的集合查询所有metric的名称:/api/v1/label/_name_/values/api/v1/metadata查询metrics的类型、单位、帮助说明/api/v1/metadata?metric=service_cpm或/api/v1/m
22、etadata?limit=查查询询表表达达式式编编写写步步骤骤选取metrics根据Scope设置metrics的匹配Label:service/service_instance/endpoint和layer根据metric value类型判断是否需要匹配Label:labels和relabels根据查询类型判断是否需要匹配Label:parent_service,top_n,order添加额外计算endpoint_percentileendpoint_percentileservice=agent:songs,endpoint=GET:/songs,layer=GENERALendpoin
23、t_percentileservice=agent:songs,endpoint=GET:/songs,layer=GENERAL,labels=0,1,2,3,4,relabels=P50,P75,P90,P95,P99endpoint_percentileservice=agent:songs,endpoint=GET:/songs,layer=GENERAL,labels=0,1,2,3,4,relabels=P50,P75,P90,P95,P99+P Pr ro ommQ QL L S Se er rv vi ic ce e 相关配置在在O OA AP P端端a ap pp pl li
24、 ic ca at ti io on n.y ymml l 或或者者环环境境变变量量中中设设置置,默默认认为为开开启启:#This module is for PromQL API.p pr ro ommq ql l:s se el le ec ct to or r:$SW_PROMQL:default d de ef fa au ul lt t:#For HTTP server r re es st tH Ho os st t:$SW_PROMQL_REST_HOST:0.0.0.0 r re es st tP Po or rt t:$SW_PROMQL_REST_PORT:9090 r r
25、e es st tC Co on nt te ex xt tP Pa at th h:$SW_PROMQL_REST_CONTEXT_PATH:/r re es st tMMa ax xT Th hr re ea ad ds s:$SW_PROMQL_REST_MAX_THREADS:200 r re es st tI Id dl le eT Ti imme eO Ou ut t:$SW_PROMQL_REST_IDLE_TIMEOUT:30000 r re es st tA Ac cc ce ep pt tQ Qu ue eu ue eS Si iz ze e:$SW_PROMQL_REST
26、_QUEUE_SIZE:S Sk ky yWWa al lk ki in ng g T To op po ol lo og gy S Sk ky yWWa al lk ki in ng g T To op po ol lo og gy y G Gr ra ap ph hQ QL Lt ty yp pe e T To op po ol lo og gy y n no od de es s:N No od de e!c ca al ll ls s:C Ca al ll l!#Node in Topologyt ty yp pe e N No od de e#The service ID of th
27、e node.i id d:I ID D!#The literal name of the#id.n na amme e:S St tr ri in ng g!#The type name may be#1.The service provider/middleware tech,such as:Tomcat,SpringMVC#2.Conjectural Service,e.g.MySQL,Redis,Kafka t ty yp pe e:S St tr ri in ng g#It is a conjecture node or real node,to represent a servic
28、e or endpoint.i is sR Re ea al l:B Bo oo ol le ea an n!#The Call represents a directed distributed call,from the source to the target.t ty yp pe e C Ca al ll l s so ou ur rc ce e:I ID D!#The protocol and tech stack used at source side in this distributed call#No value in instance topology and endpoi
29、nt dependency.s so ou ur rc ce eC Co ommp po on ne en nt ts s:I ID D!t ta ar rg ge et t:I ID D!#The protocol and tech stack used at target side in this distributed call#No value in instance topology and endpoint dependency.t ta ar rg ge et tC Co ommp po on ne en nt ts s:I ID D!i id d:I ID D!#The det
30、ect Points of this distributed call.d de et te ec ct tP Po oi in nt ts s:D De et te ec ct tP Po oi in nt t!S Sk ky yWWa al lk ki in ng g G Gr ra af fa an na a P Pl lu ug gi in ns sgrafana-cli plugins install skywalking-datasourcee en nv v:-n na amme e:GF_INSTALL_PLUGINS v va al lu ue e:skywalking-da
31、tasourcehttps:/ Gi it th hu ub b:C CL LI I 安安装装:通通过过环环境境变变量量:e en nv v:-n na amme e:GF_INSTALL_PLUGINS v va al lu ue e:$skywalking.plugin.url;skywalking-datasource指指定定安安装装包包地地址址:S Sk ky yWWa al lk ki in ng g G Gr ra af fa an na a P Pl lu ug gi in ns s provide extensions for Apache SkyWalking to visu
32、alize telemetry data on Grafana.The plugins are focusing on visualizing more telemetry data,e.g.Service Map Topology,as a new Grafana data source G Gr ra af fa an na a -设置 D Da at ta a S So ou ur rc ce G Gr ra af fa an na a -设置 D Da at ta a S So ou ur rc ce e P Pr ro ommQ QL LG Gr ra af fa an na a -
33、设置 D Da at ta a S So ou ur rc ce e G Gr ra ap ph hQ QL LG Gr ra af fa an na a -设置 V Va ar ri ia ab bl le es G Gr ra af fa an na a -V Va ar ri ia ab bl le es s -D DS S_ _S Sk ky yWWa al lk ki in ng g (P Pr ro ommQ QL L)G Gr ra af fa an na a -V Va ar ri ia ab bl le es s -P Pl lu ug gi in n_ _S Sk ky y
34、WWa al lk ki in ng g (G Gr ra ap ph hQ QL L)G Gr ra af fa an na a -V Va ar ri ia ab bl le es s -l la ay ye er 相关API和表达式:/api/v1/series?match=service_trafficlayer=GENERAL&start=1677479336&end=1677479636variable service=label_values(service_trafficlayer=$layer,service)variable endpoint=label_values(en
35、dpoint_trafficlayer=$layer,service=$service,keyword=$endpoint_keyword,limit=$endpoint_limit,endpoint)variable service_instance=label_values(instance_trafficlayer=$layer,service=$service,service_instance)G Gr ra af fa an na a -V Va ar ri ia ab bl le es s -s se er rv vi ic ce e/i in ns st ta an nc ce
36、e/e en nd dp po oi in nt tH HT TT TP P A AP PI I R Re es sp po on ns se e:s st ta at tu us s:s su uc cc ce es ss s,d da at ta a:_ _ _n na amme e_ _ _:s se er rv vi ic ce e_ _t tr ra af ff fi ic c,s se er rv vi ic ce e:a ag ge en nt t:s so on ng gs s,s sc co op pe e:S Se er rv vi ic ce e,l la ay ye e
37、r r:G GE EN NE ER RA AL L ,_ _ _n na amme e_ _ _:s se er rv vi ic ce e_ _t tr ra af ff fi ic c,s se er rv vi ic ce e:a ag ge en nt t:r re ec co ommmme en nd da at ti io on n,s sc co op pe e:S Se er rv vi ic ce e,l la ay ye er r:G GE EN NE ER RA AL L ,_ _ _n na amme e_ _ _:s se er rv vi ic ce e_ _t t
38、r ra af ff fi ic c,s se er rv vi ic ce e:a ag ge en nt t:a ap pp p,s sc co op pe e:S Se er rv vi ic ce e,l la ay ye er r:G GE EN NE ER RA AL L ,_ _ _n na amme e_ _ _:s se er rv vi ic ce e_ _t tr ra af ff fi ic c,s se er rv vi ic ce e:a ag ge en nt t:g ga at te ewwa ay y,s sc co op pe e:S Se er rv vi
39、 ic ce e,l la ay ye er r:G GE EN NE ER RA AL L ,_ _ _n na amme e_ _ _:s se er rv vi ic ce e_ _t tr ra af ff fi ic c,s se er rv vi ic ce e:a ag ge en nt t:f fr ro on nt te en nd d,s sc co op pe e:S Se er rv vi ic ce e,l la ay ye er r:G GE EN NE ER RA AL L G Gr ra af fa an na a -V Va ar ri ia ab bl le
40、 es s -e en nd dp po oi in nt t_ _k ke ey ywwo or rd G Gr ra af fa an na a -V Va ar ri ia ab bl le es s -e en nd dp po oi in nt t_ _l li immi it G Gr ra af fa an na a -P Pa an ne el l 设置 T To op po ol lo og gy G Gr ra af fa an na a -P Pa an ne el l 设置 T To op po ol lo og gy y n na amme e:s se er rv
41、vi ic ce e_ _c cp pmm,l la ab be el l:L Lo oa ad d,u un ni it t:c cp pmm ,n na amme e:s se er rv vi ic ce e_ _s sl la a,c ca al lc cu ul la at ti io on n:p pe er rc ce en nt ta ag ge e,l la ab be el l:S Su uc cc ce es ss s R Ra at te e,u un ni it t:%,n na amme e:s se er rv vi ic ce e_ _r re es sp p_
42、 _t ti imme e,l la ab be el l:L La at te en nc cy y,u un ni it t:mms s ,n na amme e:s se er rv vi ic ce e_ _a ap pd de ex x,l la ab be el l:A Ap pd de ex x,c ca al lc cu ul la at ti io on n:a ap pd de ex x n na amme e:s se er rv vi ic ce e_ _r re el la at ti io on n_ _s se er rv ve er r_ _c cp pmm,l
43、 la ab be el l:”S Se er rv ve er r L Lo oa ad d,u un ni it t:c cp pmm,t ty yp pe e:S SE ER RV VE ER R ,n na amme e:s se er rv vi ic ce e_ _r re el la at ti io on n_ _s se er rv ve er r_ _r re es sp p_ _t ti imme e,l la ab be el l:S Se er rv ve er r L La at te en nc cy y,u un ni it t:mms s,t ty yp pe
44、 e:S SE ER RV VE ER R ,n na amme e:s se er rv vi ic ce e_ _r re el la at ti io on n_ _c cl li ie en nt t_ _c cp pmm,l la ab be el l:C Cl li ie en nt t L Lo oa ad d,u un ni it t:c cp pmm,t ty yp pe e:C CL LI IN NE ET T ,n na amme e:s se er rv vi ic ce e_ _r re el la at ti io on n_ _c cl li ie en nt t
45、_ _r re es sp p_ _t ti imme e,l la ab be el l:C Cl li ie en nt t L La at te en nc cy y,u un ni it t:mms s,t ty yp pe e:C CL LI IN NE ET T E Ed dg ge e mme et tr ri ic cs s:N No od de e mme et tr ri ic cs s:G Gr ra af fa an na a -P Pa an ne el l -选择指标相相关关A AP PI I和和表表达达式式:/api/v1/query_range?query=se
46、rvice_apdexservice=agent:songs,layer=GENERAL/10000&start=1681456800&end=1681457700查查询询时时间间范范围围 1 1h h 或或者者 1 1d d ,A AP PI I将将返返回回 h ho ou ur r/d da ay y s st te ep p 的的指指标标数数据据MMi in n i in nt te er rv va al l 也也可可以以根根据据时时间间范范围围选选择择1 1mm/h h/d dG Gr ra af fa an na a -P Pa an ne el l-C Co ommmmo on
47、n V Va al lu ue e MMe et tr ri ic cs sH HT TT TP P A AP PI I R Re es sp po on ns se e:s st ta at tu us s:s su uc cc ce es ss s,d da at ta a:r re es su ul lt tT Ty yp pe e:mma at tr ri ix x,r re es su ul lt t:mme et tr ri ic c:_ _ _n na amme e_ _ _:s se er rv vi ic ce e_ _a ap pd de ex x,l la ay ye e
48、r r:G GE EN NE ER RA AL L,s sc co op pe e:S Se er rv vi ic ce e,s se er rv vi ic ce e:a ag ge en nt t:s so on ng gs s ,v va al lu ue es s:1 16 68 81 14 45 56 68 80 00 0,0 0.9 99 9 ,1 16 68 81 14 45 56 68 86 60 0,1 1 ,相相关关A AP PI I和和表表达达式式:/api/v1/query_range?query=service_percentileservice=agent:son
49、gs,layer=GENERAL,labels=0,1,2,3,4,relabels=P50,P75,P90,P95,P99&start=1681456800&end=1681457700l la ab be el ls s:用于过滤需要查询的labelsr re el la ab be el ls s:用于重命名返回的labels,列表顺序需要一致G Gr ra af fa an na a -P Pa an ne el l-L La ab be el le ed d V Va al lu ue e MMe et tr ri ic cs sH HT TT TP P A AP PI I R Re
50、 es sp po on ns se e:s st ta at tu us s:s su uc cc ce es ss s,d da at ta a:r re es su ul lt tT Ty yp pe e:mma at tr ri ix x,r re es su ul lt t:mme et tr ri ic c:_ _ _n na amme e_ _ _:s se er rv vi ic ce e_ _a ap pd de ex x,l la ay ye er r:G GE EN NE ER RA AL L,s sc co op pe e:S Se er rv vi ic ce e,s
51、 se er rv vi ic ce e:a ag ge en nt t:s so on ng gs s ,v va al lu ue es s:1 16 68 81 14 45 56 68 80 00 0,0 0.9 99 9 ,1 16 68 81 14 45 56 68 86 60 0,1 1 ,相相关关A AP PI I和和表表达达式式:/api/v1/query_range?query=service_instance_cpmparent_service=agent:songs,layer=GENERAL,top_n=10,order=DES&start=1681456800&end
52、=1681457700p pa ar re en nt t_ _s se er rv vi ic ce e:metric父级的service的名字t to op p_ _n n:用于过滤排序结果的条数o or rd de er r:排序规则G Gr ra af fa an na a -P Pa an ne el l-S So or rt t MMe et tr ri ic cs s/S Sa ammp pl le ed d R Re ec co or rd ds sH HT TT TP P A AP PI I R Re es sp po on ns se e:s st ta at tu us
53、s:s su uc cc ce es ss s,d da at ta a:r re es su ul lt tT Ty yp pe e:mma at tr ri ix x,r re es su ul lt t:mme et tr ri ic c:_ _ _n na amme e_ _ _:s se er rv vi ic ce e_ _i in ns st ta an nc ce e_ _c cp pmm,l la ay ye er r:G GE EN NE ER RA AL L,s sc co op pe e:S Se er rv vi ic ce eI In ns st ta an nc
54、ce e,s se er rv vi ic ce e_ _i in ns st ta an nc ce e:e e6 6c cd d9 9e e4 47 7f f8 86 60 04 45 51 12 2b bd db b6 6a a1 16 6a a2 2d df fd d4 4e ec c2 21 10 0.6 68 8.1 1.2 22 23 3 ,v va al lu ue es s:1 16 68 81 14 45 56 68 80 00 0,1 17 7 ,.,mme et tr ri ic c:_ _ _n na amme e_ _ _:s se er rv vi ic ce e
55、_ _i in ns st ta an nc ce e_ _c cp pmm,l la ay ye er r:G GE EN NE ER RA AL L,s sc co op pe e:S Se er rv vi ic ce eI In ns st ta an nc ce e,s se er rv vi ic ce e_ _i in ns st ta an nc ce e:9 91 17 7a a8 83 3e e4 46 63 3a a0 04 40 07 75 59 93 35 56 65 5f fa a0 0c c0 0a ab bd db be e8 81 10 0.6 68 8.1
56、1.2 22 26 6 ,v va al lu ue es s:1 16 68 81 14 45 56 68 80 00 0,1 16 6 ,.,构建 G Gr ra af fa an na a d da as sh hb bo oa ar rd d设置Data Source设置Variables作为查询表达式的参数化匹配条件创建dashboard,建议按照Layer创建创建panel,metrics使用PromQL表达式,topology设置SkyWalking Plugins验证表达式,设置panel等全全新新构构建建:从从S Sk ky yWWa al lk ki in ng g S S
57、h ho owwc ca as se e 导导入入(建建议议):设置Data Source从SkyWalking Showcase 下载或者从Demo导出dashboard文件,通过dashboard导入功能导入根据需要调整或者新增metrics和panelhttps:/skywalking.apache.org/#demoD De emmo o:https:/ Sh ho oww c ca as se e:最终效果-S Se er rv vi ic ce e t to op po ol lo og gy y 最终效果-S Se er rv vi ic ce e mme et tr ri ic cs s 最终效果-S Se er rv vi ic ce e I In ns st ta an nc ce e mme et tr ri ic cs 最终效果-S Se er rv vi ic ce e I In ns st ta an nc ce e J JV VMM mme et tr ri ic cs 最终效果-E En nd dp po oi in nt t mme et tr ri ic cs T TH HA AN NK KS S