1、www.valtrix.inTony Wang王浩为王浩为August 23-25,2023RISC-V Summit ChinaAddressing the Complexitiesof RISC-V Functional Verification解决RISC-V功能验证中的复杂性www.valtrix.in1.Valtrix-EDA company headquartered in Bangalore,India;2.Developing products/solutions for CPU/SoC verification3.STING-First commercially availa

2、ble and most advanced design verification solution for RISC-V4.Software-driven portable self-checking architecturally correct stimulus generator5.Used by several RISC-V CPU/SoC companies such as Google,Seagate,Sifive and Tenstorrent for verifying functional correctness and architectural compliance o

3、f designsValtrix-An Introduction 简介www.valtrix.in01Challenges in RISC-V Design VerificationRISC-V 设计验证中的挑战02Challenges imposed by DUT ConfigurationDUT配置带来的挑战03Popular Test Generation Methodologies现行有效的测试生成方法04Functional Correctness and Compliance功能正确性和合规性功能正确性和合规性Agenda Of The Talk 议程05Test Generati

4、on over Design Life-Cycle在设计周期不同阶段的测试生成方法06Case Studies for Test Generation案例研究07Conclusion结论www.valtrix.inCPU/SoC Verification Challenges传统芯片验证中会遇到的挑战1.How many tests to run to ensure comprehensive verification?2.How to distribute the testing across simulation,emulation,FPGA and silicon for best th

5、roughput?3.How to write test stimulus which is portable across all DUT environments?4.How to debug failures from silicon quickly on simulations?5.How to develop a test which runs on simulation and silicon alike?6.How does test generation scale from top-level simulation test benches all the way to a

6、complete SoC on silicon?www.valtrix.inUnique Challenges in RISC-V VerificationRISC-V验证中独有的挑战1.Plug and play extension architecture leads to huge number of designs which need to be verified即插即用的扩展架构导致大量验证需求RV32I+MRV32IM+MRV32IMC+MRV32IMAC+MRV32IMAFDC+MRV32IMAFDCV+MRV32IMAFDCV+MRV32IMAFDCV+MURV32IMAFD

7、CV+MSURV32IMAFDCV+MSU+MMURV32IMAFDCV+MSU+MMU+PMPand many many many more.www.valtrix.inTEST GENERATION FRAMEWORKFUNCTIONAL COVERAGE IMPACTBEHAVIORAL MODELINGINTEROPERABILITYUnique Challenges in RISC-V VerificationRISC-V验证中独有的挑战1.Plug and play extension architecture leads to huge number of designs whi

8、ch need to be verified2.Custom extensions and features自定义指令和特性ARCHITECTURAL COMPLIANCECUSTOM EXTENSIONS/FEATURESwww.valtrix.inUnique Challenges in RISC-V VerificationRISC-V验证中独有的挑战1.Plug and play extension architecture leads to huge number of designs which need to be verified2.Custom extensions and

9、features 3.Continuously evolving specifications不断进化的规格SEP 20190.6 DraftOCT 20190.7 DraftNOV 20190.7.1 DraftJUNE 20200.8 DraftJULY 20200.9 DraftJUNE 20201.0 rc1 DraftDEC 20181.0 DraftSEP 20211.0 FrozenTIMELINEVECTOR SPECEVOLUTIONReview of vector specificationBasic support for vector specification in

10、STING;Support for few instructions addedAdded support for all vector instructions in STINGNewer instructions were added;Changes in opcodes;Some instructions removed;Vsetvli functionality modified Major change in spec;EEW and EMUL introduced;Fractional LMUL introduced;Existing stimulus rendered usele

11、ss;Test generator redesignSupport for Half Precision FP with SEW=16;More instruction modifications;Overlap constraints updatedSupport for latest modifications in vector Spec;Adding optimizations to handle tradeoffs in test generatorwww.valtrix.inChallenges imposed by DUT ConfigurationDUT配置带来的挑战1.Lim

12、ited verification cycles in simulations2.Emulation,FPGA and silicon based DUT platforms faster but suffer from lack of debug visibility3.Debug features to reduce time taken to isolate failures4.Different test methodologies(based on SV/UVM,OS based tools and so on)across different DUT environments in

13、creases redundancy and reduces reuse5.Portability of stimulus6.Test generator design to address challenges imposed by DUT www.valtrix.inChallenges imposed by DUT ConfigurationDUT配置带来的挑战Software Driven Functional Verification MethodologySiliconFPGAPrototypeIn-CircuitEmulationsPre-SiliconSimulationsDe

14、sign Life CycleIncrease in CPU FrequencySV/UVM TestsOS Based TestsEffectiveness of TestsVerification Coverage Provided1.Allow consistent execution on all DUT environments2.Reuse of test stimulus3.Failures hit on silicon can be migrated to an earlier stage for faster debug4.Early enabling of software

15、 driven stimulus increases the chances of hitting complex bugs early5.Save on duplicate efforts spent on design verification and system validationwww.valtrix.inChallenges imposed by DUT ConfigurationDUT配置带来的挑战SLOWSIMSFPGA/SILICONSTING TEST GENERATIONPRE-TEST SETUPKERNEL BOOTTEST EXECUTIONPOST-TEST C

16、HECKSOffline System(x86)Offline System(x86)DUTSTING TEST GENERATIONKERNEL BOOTFPGA/EMULATIONSTING TEST GENERATIONPRE-TEST SETUPKERNEL BOOTTEST EXECUTIONPOST-TEST CHECKSOffline System(x86)DUTPOST-TEST CHECKSDUTPRE-TEST SETUPTEST EXECUTIONNN1.Modular test generator design to achieve maximum test throu

17、ghput as per the DUT environment2.Self generating mode on FPGA/silicon enables long runtimeswww.valtrix.inPopular Test Generation Methodologies测试生成方法1.Constrained RandomRandomize within fixed constraintsAdvantageous in sweeping through many random combinationsNot very good in targeting fixed use cas

18、esflwswaddivle16fmulcustomflwswaddivle16fmulcustomflwswaddivle16fmulcustomvle16customaddiflwfmulswaddivle16flwcustomswfmulSingle Instruction GroupMany Instruction Stream Possibilitieswww.valtrix.inPopular Test Generation Methodologies测试生成方法1.Constrained Random2.Dynamic RandomSimulate as the test is

19、getting generatedAvailability of complete state during test generation helps in targeting more complex scenariosVery slow because of simulation overhead and frequent re-evaluations of choices made during test generationflwswaddivle16simulatesimulatesimulateswsdmulsimulatesimulatesimulatesimulatewww.

20、valtrix.inPopular Test Generation Methodologies测试生成方法1.Constrained Random2.Dynamic Random3.Directed TestingSTING Snippet Programming Frameworkresource blocksnippet initsnippet runsnippet checkCPU 0TESTINITPHASETESTRUNPHASETESTCHECKPHASESTART OF TEST BARRIEREND OF TEST BARRIERCPU 1Highly directed sce

21、narios can be coveredAdvantageous in covering architectural compliance testsHigh on manual effort and does not scale very well across multiple projects/designs if not written properlywww.valtrix.inPopular Test Generation Methodologies测试生成方法1.Constrained Random2.Dynamic Random3.Directed Testing04.Gra

22、ph BasedTEST STARTTEST ENDOPGRP_AOPGRP_B501TESTOPGRPStimulus graph based test generation gives greater control on the shape of the testEasily cover scenarios which are otherwise very difficult to achieve using constrained random or are not random enough for directed test caseswww.valtrix.inPopular T

23、est Generation Methodologies测试生成方法1.Constrained Random2.Dynamic Random3.Directed Testing04.Graph Based05.Use Case Testing/Real World ScenariosInterruptsMemory CopyPagingPerformanceBenchmarksCache HarasserPointer ChasingAuto VectorizationLoad BalancingCoverage for OS use cases/real world scenariosRun

24、ning them early on simulations help hitting the complex bugs earlywww.valtrix.inVerifying Functional Correctness and Compliance验证功能正确性和合规性1.Lockstep Checkinga.Run golden model along with RTL simulationb.Check model result vs RTL resultc.ADVANTAGES:Easy debug,extensive checking of memory and register

25、sd.DISADVANTAGES:Simulation only,Hard to implement for MP2.Model Reference Checkinga.Model used to generate golden datab.Checking at end of testc.Portable programs which can be run on any DUT environmentd.ADVANTAGE:No infrastructure work on TB,Fast,Portable(Sim,Emu.)e.DISADVANTAGE:Slightly higher de

26、bug effort,granularity of check is lower3.Directed Checksa.Covers compliance and corner case scenarios,designer concerns.b.Checks will be part of testc.ADVANTAGE:Coverage of a given scenario is guaranteedd.DISADVANTAGE:Difficult to cover all scenarios,Labour intensive4.Multi-Pass Checkinga.Effective

27、 for timing bugsb.Consistency checks across multiple passesc.ADVANTAGE:Very Fast;Instruction simulation is not required.d.DISADVANTAGE:Not good at finding functional bugs www.valtrix.inTest Generation over Design Life Cycle1.Verify basic functionality of each instructiona.Fixed data pattern testsb.R

28、andom data pattern testsc.Check whether reads and writes can access memories2.Test interaction between different instructionsa.Pipeline dependenciesb.Control dependenciesc.Address dependencies3.Test interaction with different eventsa.Exceptionsb.External interrupts4.More complex use casesa.Interacti

29、on with MMU-page size switching,address bits change,change of memory typeb.Algorithmic tests-Memory Copy,pointer chasingc.Microarchitectural tests-Cache fill/evictions,self modifying code,cross modifying code5.Add traffic from other peripherals in a SoC test bencha.Test concurrent traffic from multi

30、ple sourceswww.valtrix.inConstrained RandomGraph Based TestsDirected SequencesASM-like Directed TestsC+-based TestsConfigurationsMicro-KernelTest GeneratorLibraryDevice DriversSTING ELFSIMULATIONEMULATIONFPGASILICONWorkloadsOS/Real life SW ScenariosBenchmarksSTING Design Verification Tool1.Bare meta

31、l tool using a software driven methodology for RISC-V design verification2.Self-checking architecturally correct stimulus portable across simulation,emulation,FPGA and silicon3.Highly scalable and quick test generation;Compatible with any system configuration/memory map;IoT/embedded to server class;

32、MP-ready4.Complete support for 32-bit and 64-bit RISC-V base integer extensions along with all standard extensions;Supports RVA23 profile5.Comprehensive coverage of privilege specification areas such as MMU,PMP,Hypervisorwww.valtrix.inCase Study-Support for any RISC-V configuration对所有RISC-V配置的支持1.JS

33、ON-like rich configuration mechanism for input specification2.Every RISC-V extension/feature can be controlled using configuration overrides3.The configuration overrides determine many things such as the valid directed tests and instructions to select in the test,toolchain options to compile and gen

34、erate the ELF file and so on4.Placement of text/data sections randomized based on physical memory attributes specified in the memory map5.Any RISC-V implementation can be tested without worrying about tool compatibility issuesSTING ELFRV_PARAMSHAS_RV64I:.HAS_RV32I:.HAS_SUPERVISOR:.HAS_USER:.HAS_HYPE

35、RVISOR:.HAS_M_EXT:.HAS_A_EXT:.HAS_ZICBOM_EXT:.BOARD_CONFRAMS:BASE:.SIZE:.ATTR:R:.W:.X:.UARTS:BASE:.SIZE:.STING META DATAwww.valtrix.inCase Study-Directed Test Development定向测试开发resource sectionsnippet initsnippet runsnippet checksnippet handlersCPU 0CPU 1START OFTESTBARRIEREND OFTESTBARRIERmtvecstvec

36、utvecKERNELSNIPPET STRUCTURETESTINITPHASETESTRUNPHASETESTCHECKPHASE1.STING Snippets-a programming framework for development of directed stimulus2.Language constructs to make development of complex test cases seamless and easy3.Integrates with random instruction streams to produce interesting cross p

37、roducts every time its rendered into the test streamwww.valtrix.inCase Study:Graph Based Test Generation基于图形的测试生成方式1.Graph based instruction stream generation gives extremely high control on the shape of the test2.Scheduling of directed sequences/drivers inside the test very useful for use case veri

38、fication3.Have been used to mix elements of constrained random and directed testing methodologies in the same test resulting in extremely high coverageTEST STARTTESTTEST ENDABC213DE45LM5050NO5020P30XY21AABCCCTEST STARTLMMLNONXXYNTESTDDDDEEEEEPLEGENDOp Group SeqOp Group RandOpOp Groupwww.valtrix.inCa

39、se Study-Configurable Test GeneratorJ1J2J3SYSTEM CONFIGURATION SPECIFICATION IN ANY FORMAT FOR e.g.JSON.SCC(STING Configuration Creator)STING TEST GENERATORDUT 1DUT 2DUT 3ELF1_TEST_PROPSJnRV_PARAMSBOARD_CONF_TEST_PROPSRV_PARAMSBOARD_CONF_TEST_PROPSRV_PARAMSBOARD_CONFELF2ELF3RTL GENERATOR1.Configurab

40、le stimulus for configurable cores;Very useful for CPU vendors2.Reusability of valid stimulus across multiple generation of cores3.Reduces manual effort and improves verification efficiency4.Automatically generate STING configurations for any address map and SoC device configurationwww.valtrix.inCas

41、e Study:Enabling External InterruptsTESTINITPHASETESTRUNPHASETESTENDPHASEINTERRUPTVECTORTABLESTING KERNELCPU 0CLINTCPU 1STING TESTDEVICE1Initialize the CLINT deviceat the starting of the test;Register handlers with IRQ ID for the CLINT interrupts2Program CLINT device to deliver an interrupt3CLINT de

42、livers interrupt to CPU 5Checker at the end of test to determine if test sequence ran alright4Control goes to handler in kernel 1.STING snippet framework for development of device drivers for INTC2.Graph based stimulus generation mechanism to render the driver instantiations in different points of i

43、nterest in the test3.Customizable kernel allows registration of interrupt handlers for each IRQ ID4.Snippet based checkers at the end of the test determine if the interrupts were received;In case the intent was not met,a failure is flaggedwww.valtrix.inConclusion1.RISC-V provides great flexibility t

44、o CPU designers but increases the complexity of verification2.Design of test generators very crucial to address the unique verification needs of RISC-V3.Combining different test generation methodologies and execution mechanisms achieves best possible verification throughput4.Scale test stimulus and

45、add more complex blocks as the design matures5.Consistent execution on all DUT environments important for portability of stimuluswww.valtrix.inTHANK YOUFor more information please visit www.valtrix.in/write to us at contactvaltrix.inTony Wang王浩为王浩为twangvaltrix.inAddressing the Complexitiesof RISC-V Functional Verification解决RISC-V功能验证中的复杂性