1、测试基础架构演进之路测试基础架构演进之路测测试试架架构构的的演演变变qGUI Automation Test Framework的的演演变变q Test Data Platform的的演演变变q API Automation Test Framework的的演演变变q Test Execution Environment的的演演变变q Test Execution and Management Platform的的演演变变q Test Report Platform的的演演变变测测试试架架构构的的演演变变GUI Automation Test Framework测测试试架架构构的的演演变变-G

2、UI Automation Test Framework最最原原始始的的GUI测测试试Business RequirementFeature RequirementTest RequirementTest CaseDesign?manually?execute according to test case design测测试试架架构构的的演演变变-GUI Automation Test Framework基基于于录录制制回回放放的的GUI自自动动化化测测试试UI AutomationTest Case?Replay test case to execute testTest CaseDesig

3、nBusiness RequirementFeature RequirementTest RequirementCreate test case by UI recording测测试试架架构构的的演演变变-GUI Automation Test Framework基基于于可可重重用用测测试试代代码码片片段段构构成成GUI测测试试用用例例Operation-based reusable script snippetTest Case?Replay test case to execute testTest CaseDesignBusiness RequirementFeature Require

4、mentTest RequirementUI recording测测试试架架构构的的演演变变-GUI Automation Test Framework基基于于Component Abstract构构成成GUI测测试试用用例例Reusable script snippetTest Case?Replay test case to execute testTest CaseDesignBusiness RequirementFeature RequirementTest RequirementComponent AbstractComponent Operation Abstract测测试试架架

5、构构的的演演变变-GUI Automation Test Framework基基于于Page Abstract构构成成GUI测测试试用用例例Reusable script snippetTest Case?Replay test case to execute testTest CaseDesignBusiness RequirementFeature RequirementTest RequirementComponent AbstractComponent Operation AbstractPageAbstractPageOperation Abstract测测试试架架构构的的演演变变-

6、GUI Automation Test Framework基基于于Business Flow Abstract构构成成GUI测测试试用用例例Reusable script snippetTest Case?Replay test case to execute testTest CaseDesignBusiness RequirementFeature RequirementTest RequirementComponent AbstractComponent Operation AbstractPageAbstractPageOperation AbstractBusiness FlowAb

7、stract测测试试架架构构的的演演变变-GUI Automation Test Framework测测试试数数据据数数据据驱驱动动的的测测试试+测测试试数数据据准准备备Reusable script snippetTest Case?Encapsulated utiliites which call API or directly DB access?Test CaseDesignBusiness RequirementFeature RequirementTest RequirementComponent AbstractComponent Operation AbstractPageAb

8、stractPageOperation AbstractBusiness FlowAbstract?Test Date ProviderTest Date Preparation ToolReplay test case to execute test测测试试架架构构的的演演变变-GUI Automation Test Framework测测试试数数据据使使用用Out-of-box Test Data/Golden Data SetReusable script snippetTest Case?Encapsulated utiliites which call API or directly

9、 DB access?Test CaseDesignBusiness RequirementFeature RequirementTest RequirementComponent AbstractComponent Operation AbstractPageAbstractPageOperation AbstractBusiness FlowAbstract?Test Date ProviderTest Date Preparation ToolReplay test case to execute test?Use OOB/GDS test data测测试试架架构构的的演演变变-GUI

10、Automation Test Framework基基于于Unified Flow Framework实实现现Flow Branch控控制制Reusable script snippetTest Case?Encapsulated utiliites which call API or directly DB access?Test CaseDesignBusiness RequirementFeature RequirementTest RequirementComponent AbstractComponent Operation AbstractPageAbstractPageOpera

11、tion AbstractUnified Business FlowAbstract?Test Date ProviderTest Date Preparation ToolReplay test case to execute test?Use OOB/GDS test dataUnified Flow FrameworkAdoptBase on UF测测试试架架构构的的演演变变-GUI Automation Test Framework基基于于Page Encapsulation Code Generator提提高高Page Abstract的的效效率率Reusable script sn

12、ippetTest Case?Encapsulated utiliites which call API or directly DB access?Test CaseDesignBusiness RequirementFeature RequirementTest RequirementComponent AbstractComponent Operation AbstractPageAbstractPageOperation AbstractUnified Business FlowAbstract?Test Date ProviderTest Date Preparation ToolR

13、eplay test case to execute test?Use OOB/GDS test dataUnified Flow FrameworkAdoptBase on UFPage Encapsulation Code Generator引引入入Test Data Service，提提供供统统一一的的测测试试数数据据准准备备服服务务Reusable script snippetTest Case?Test CaseDesignBusiness RequirementFeature RequirementTest RequirementComponent AbstractComponen

14、t Operation AbstractPageAbstractPageOperation AbstractUnified Business FlowAbstract?Test Date ProviderTest Date AdapterReplay test case to execute testCall TDS to prepare(create/search/update)test data on SUTUnified Flow FrameworkAdoptBase on UFPage Encapsulation Code GeneratorTest Date Service测测试试架

15、架构构的的演演变变-GUI Automation Test Framework测测试试架架构构的的演演变变Test Data Platform测测试试架架构构的的演演变变 Test Data PlatformTest Data Service的的雏雏形形Reusable script snippetTest Case?Test CaseDesignBusiness RequirementFeature RequirementTest RequirementComponent AbstractComponent Operation AbstractPageAbstractPageOperatio

16、n AbstractUnified Business FlowAbstract?Test Date ProviderTest Date AdapterReplay test case to execute testCall TDS to prepare(create/search/update)test data on SUTUnified Flow FrameworkAdoptBase on UFPage Encapsulation Code GeneratorTest Date ServiceUse OOB/GDS test dataWeb Service?Test DataPrepara

17、tion Tool?Test Date AdapterEncapsulated utiliites which call API or directly DB access?引引入入Test Data Core Service和和RecipeWeb Service?Test DataPreparation Tool?Test Date AdapterEncapsulated utiliites which call API or directly DB access?Test Data Core Service Offering data Meta data management Recipe

18、 data managementTest Data Meta DBRecipe DefinitionMeta data for business objectsOut-of-box test data测测试试架架构构的的演演变变 Test Data PlatformReusable script snippetTest Case?Test CaseDesignBusiness RequirementFeature RequirementTest RequirementComponent AbstractComponent Operation AbstractPageAbstractPageOp

19、eration AbstractUnified Business FlowAbstract?Test Date ProviderTest Date AdapterReplay test case to execute testCall TDS to prepare(create/search/update)test data on SUTUnified Flow FrameworkAdoptBase on UFPage Encapsulation Code GeneratorTest Date Service引引入入Data Quantity/Quality管管理理Web Service?Te

20、st DataPreparation Tool?Test DateAdapterEncapsulated utiliites which call API or directly DB access?Test Data Core Service Offering data Meta data management Recipe data management Data quantity management Data quality managementTest Data Meta DBRecipe DefinitionMeta data for business objectsOut-of-

21、box test dataMeta data for data quantityMeta data for data qualityUI Interface for managementUnified Test DataController Test DataValidation Tool测测试试架架构构的的演演变变 Test Data PlatformReusable script snippetTest Case?Test CaseDesignBusiness RequirementFeature RequirementTest RequirementComponent AbstractC

22、omponent Operation AbstractPageAbstractPageOperation AbstractUnified Business FlowAbstract?Test Date ProviderTest Date AdapterReplay test case to execute testCall TDS to prepare(create/search/update)test data on SUTUnified Flow FrameworkAdoptBase on UFPage Encapsulation Code GeneratorTest Date Servi

23、ce引引入入Unified Controller接接入入不不同同Test Data ToolsWeb Service?Test DataPreparation Tool?Test DateAdapterTest Data Core Service Offering data Meta data management Recipe data management Data quantity management Data quality managementTest Data Meta DBRecipe DefinitionMeta data for business objectsOut-of

24、-box test dataMeta data for data quantityMeta data for data qualityUI Interface for managementUnified Test DataController Test DataValidation ToolDEV Test DataPreparation ToolPerformance Test Data Preparation Tool测测试试架架构构的的演演变变 Test Data PlatformReusable script snippetTest Case?Test CaseDesignBusine

25、ss RequirementFeature RequirementTest RequirementComponent AbstractComponent Operation AbstractPageAbstractPageOperation AbstractUnified Business FlowAbstract?Test Date ProviderTest Date AdapterReplay test case to execute testCall TDS to prepare(create/search/update)test data on SUTUnified Flow Fram

26、eworkAdoptBase on UFPage Encapsulation Code GeneratorTest Date Service测测试试架架构构的的演演变变APIAutomation Test Framework测测试试架架构构的的演演变变API Automation Test Framework最最原原始始的的API测测试试API Design SpecificationAPI Test CaseDesignAPI Test Tool(POSTMAN/SOAPUI)CollectionsTeam Library?Re?Response引引入入API Test Executor实实

27、现现Code-based API自自动动化化测测试试API Design SpecificationAPI Test CaseDesignAPI Test Executor?Re?ResponseAPI Test Case with Test Data测测试试架架构构的的演演变变API Automation Test FrameworkTest Case和和Test Data分分离离实实现现Data-Driven TestAPI Design SpecificationAPI Test CaseDesignAPI Test Executor?Re?ResponseAPI Test Case D

28、ata-driven Test Data 测测试试架架构构的的演演变变API Automation Test FrameworkAPI Design SpecificationAPI Test CaseDesignAPI Test Executor?Re?ResponseAPI Test Case Data-driven Test Data Automatic Test Data GeneratorAPI Parameters Type测测试试架架构构的的演演变变API Automation Test Framework引引入入Data-Driven Test Data的的自自动动生生成成引引

29、入入Test Data Service生生成成测测试试数数据据API Design SpecificationAPI Test CaseDesignAPI Test Executor?Re?ResponseAPI Test Case Data-driven Test Data Automatic Test Data GeneratorAPI Parameters TypeTest Data Service测测试试架架构构的的演演变变API Automation Test Framework引引入入Parallel Execution Controller实实现现API并并发发测测试试API D

30、esign SpecificationAPI Test CaseDesignAPI Test Executor?Re?ResponseAPI Test Case Data-driven Test Data Automatic Test Data GeneratorAPI Parameters TypeTest Data ServiceParallel Execution Controller测测试试架架构构的的演演变变API Automation Test Framework引引入入Load Generator Cluster实实现现API高高并并发发和和压压力力测测试试API Design

31、SpecificationAPI Test CaseDesignAPI Test Executor?Re?ResponseAPI Test Case Data-driven Test Data Automatic Test Data GeneratorAPI Parameters TypeTest Data ServiceParallel Execution ControllerLoad Generator Cluster 测测试试架架构构的的演演变变API Automation Test Framework引引入入2R History Database实实现现API Diff Identif

32、icationAPI Design SpecificationAPI Test CaseDesignAPI Test Executor?Re?ResponseAPI Test Case Data-driven Test Data Automatic Test Data GeneratorAPI Parameters TypeTest Data ServiceParallel Execution ControllerLoad Generator Cluster?测测试试架架构构的的演演变变API Automation Test Framework微微服服务务架架构构下下的的API测测试试挑挑战战

33、q API的的种种类类数数量量多多q API测测试试高高覆覆盖盖率率的的代代价价大大q微微服服务务架架构构之之间间的的耦耦合合q第第三三方方服服务务的的耦耦合合?Re?ResponseRe?Response测测试试架架构构的的演演变变API Automation Test Framework基基于于消消费费者者契契约约的的API测测试试基基于于契契约约的的测测试试与与验验证证?Record Proxy Consumer A Request/Response SuiteConsumer B Request/Response SuiteService T Contract SuiteTest Ex

34、ecutionTest Validation测测试试架架构构的的演演变变API Automation Test Framework基基于于消消费费者者契契约约的的API测测试试基基于于契契约约的的Mock Service实实现现API依依赖赖解解耦耦?Service TAPI Test测测试试架架构构的的演演变变API Automation Test Framework基基于于消消费费者者契契约约的的API测测试试基基于于契契约约的的Mock Service实实现现API依依赖赖解解耦耦Service X Contract SuiteService Y Contract Suite?Servi

35、ce TAPI TestMock Service YMock Service X测测试试架架构构的的演演变变API Automation Test Framework测测试试架架构构的的演演变变Test Execution Environment测测试试架架构构的的演演变变-Test Execution Environment基基于于Jenkins触触发发测测试试执执行行Test Case?Test CaseDesignBusiness RequirementFeature RequirementTest Requirement?Locally develop/debug/runJenkins

36、Pull BuildRun?Push基基于于Test Runner/Test Execution SystemTest Case?Test CaseDesignBusiness RequirementFeature RequirementTest RequirementTest Requests?Locally develop/debug/runTest Runner Test Execution SystemJenkins Pull BuildRun?Push测测试试架架构构的的演演变变-Test Execution Environment基基于于Selenium Grid提提高高测测试试并

37、并行行执执行行能能力力Test Case?Test CaseDesignBusiness RequirementFeature RequirementTest RequirementTest Requests?Locally develop/debug/runTest Runner Test Execution SystemJenkins Pull BuildRun?Push?Test RequestsRundispatch Rundispatch dispatch 测测试试架架构构的的演演变变-Test Execution Environment基基于于Jenkins Cluster提提高高

38、测测试试并并行行执执行行能能力力Jenkins Slave n Jenkins Slave 2 Test Case?Test CaseDesignBusiness RequirementFeature RequirementTest RequirementTest Requests?Locally develop/debug/runTest Runner Test Execution SystemJenkins Master Pull BuildRun?Push?Test RequestsRundispatch Rundispatch dispatch Jenkins Slave 1 disp

39、atch 测测试试架架构构的的演演变变-Test Execution Environment基基于于测测试试负负载载，用用Docker实实现现Selenium Grid的的动动态态扩扩展展与与收收缩缩Jenkins Slave n Jenkins Slave 2 Test Case?Test CaseDesignBusiness RequirementFeature RequirementTest RequirementTest Requests?Locally develop/debug/runTest Runner Test Execution SystemJenkins Master P

40、ull BuildRun?Push?Test RequestsRundispatch Rundispatch dispatch Jenkins Slave 1 dispatch Dispatch and auto scale servicedispatch Dynamicallty add/delete selenium node in Grid accoriding to test load?Run测测试试架架构构的的演演变变-Test Execution Environment基基于于Docker实实现现Jenkins Cluster的的动动态态扩扩展展与与收收缩缩Jenkins Slav

41、e n Jenkins Slave 2 Test Case?Test CaseDesignBusiness RequirementFeature RequirementTest RequirementTest Requests?Locally develop/debug/runTest Runner Test Execution SystemJenkins Master Pull BuildRun?Push?Test RequestsRundispatch Rundispatch dispatch Jenkins Slave 1 dispatch Dispatch and auto scale

42、 servicedispatch Dynamicallty add/delete selenium node in Grid accoriding to test load?RunJenkins auto scale serviceDynamicallty add/delete?测测试试架架构构的的演演变变-Test Execution Environment引引入入Test Report Service生生成成各各种种测测试试报报告告Test Case?Test CaseDesignBusiness RequirementFeature RequirementTest Requirement

43、?Locally develop/debug/runAuto ScaleJenkins ClusterTest Execution Report Meta DataTest Requests?PushTest Report Service Story Board Test Report for GUI Full Trace Test Report Test Summary ReportTest RunnerTest Execution ServiceDispatchPull BuildFeature Coverage CalculationBusiness Coverage Calculati

44、onTest Coverage Calculation测测试试架架构构的的演演变变-Test Execution Environment测测试试架架构构的的演演变变Test Report Service测测试试架架构构的的演演变变-Test Report ServiceTest Report Service生生成成的的各各种种测测试试报报告告q Full Trace Test Report for DEV/QEq GUI Video-based Test Report for PM/POq Story Board Test Report for PM/POq Multi-Site Story

45、Board Comparison Test Report for LQAq Test Summary Report for management teamq Test Trend Report for management team 典典型型的的Test Report Platform架架构构Test Case?Test CaseDesignBusiness RequirementFeature RequirementTest Requirement?Locally develop/debug/runAuto ScaleJenkins ClusterTest Execution Report

46、Meta DataTest Requests?PushTest Report Service Story Board Test Report for GUI Full Trace Test Report Test Summary ReportTest RunnerTest Execution ServiceDispatchPull BuildFeature Coverage CalculationBusiness Coverage CalculationTest Coverage CalculationTest Report Service GUI Story Board Test Repor

47、t Full Trace Test Report Test Summary ReportRequirement Management System(JIRA/DOORS/ALM)?GUI Video-based Test Report?测测试试架架构构的的演演变变-Test Report Service引引入入Test Analysis Service提提高高Defect分分类类效效率率Test Report TraceAnalysis Service?Defect Management System(JIRA?)Test Report Service GUI Story Board Test

48、 Report Full Trace Test Report Test Summary ReportRequirement Management System(JIRA/DOORS/ALM)?GUI Video-based Test Report?Defect AutomaticClassificate and SubmitFailed Test CasesDefect close then send root cause back to Analysis Service,it will be used as machine learning sample set?测测试试架架构构的的演演变变

49、-Test Report ServiceTest Case?Test CaseDesignBusiness RequirementFeature RequirementTest Requirement?Locally develop/debug/runAuto ScaleJenkins ClusterTest Execution Report Meta DataTest Requests?PushTest Report Service Story Board Test Report for GUI Full Trace Test Report Test Summary ReportTest R

50、unnerTest Execution ServiceDispatchPull BuildFeature Coverage CalculationBusiness Coverage CalculationTest Coverage Calculation引引入入Multi-Site Comparison Report提提高高LQA测测试试效效率率Test Report TraceAnalysis Service?Defect Management System(JIRA?)Test Report Service GUI Story Board Test Report Full Trace Te

51、st Report Test Summary ReportRequirement Management System(JIRA/DOORS/ALM)?GUI Video-based Test Report?Defect AutomaticClassificate and SubmitFailed Test CasesDefect close then send root cause back to Analysis Service,it will be used as machine learning sample setMulti-Site Story Board Comparison Re

52、port Generator Service Multi-Site Story Board Comparison ReportLQA submit translation and layer out defect?测测试试架架构构的的演演变变-Test Report ServiceTest Case?Test CaseDesignBusiness RequirementFeature RequirementTest Requirement?Locally develop/debug/runAuto ScaleJenkins ClusterTest Execution Report Meta D

53、ataTest Requests?PushTest Report Service Story Board Test Report for GUI Full Trace Test Report Test Summary ReportTest RunnerTest Execution ServiceDispatchPull BuildFeature Coverage CalculationBusiness Coverage CalculationTest Coverage Calculation引引入入Test Statistics ServiceTest Report TraceAnalysis

54、 Service?Defect Management System(JIRA?)Test Report Service GUI Story Board Test Report Full Trace Test Report Test Summary ReportRequirement Management System(JIRA/DOORS/ALM)?GUI Video-based Test Report?Defect AutomaticClassificate and SubmitFailed Test CasesDefect close then send root cause back t

55、o Analysis Service,it will be used as machine learning sample setMulti-Site Story Board Comparison Report Generator Service Multi-Site Story Board Comparison ReportLQA submit translation and layer out defectTest ReportStatistics Service Test Trend Report?测测试试架架构构的的演演变变-Test Report ServiceTest Case?T

56、est CaseDesignBusiness RequirementFeature RequirementTest Requirement?Locally develop/debug/runAuto ScaleJenkins ClusterTest Execution Report Meta DataTest Requests?PushTest Report Service Story Board Test Report for GUI Full Trace Test Report Test Summary ReportTest RunnerTest Execution ServiceDispatchPull BuildFeature Coverage CalculationBusiness Coverage CalculationTest Coverage CalculationTHANK YOU