CN113742202A - AI chip verification system, method, device and storage medium - Google Patents
AI chip verification system, method, device and storage medium Download PDFInfo
- Publication number
- CN113742202A CN113742202A CN202010476115.3A CN202010476115A CN113742202A CN 113742202 A CN113742202 A CN 113742202A CN 202010476115 A CN202010476115 A CN 202010476115A CN 113742202 A CN113742202 A CN 113742202A
- Authority
- CN
- China
- Prior art keywords
- verification
- case
- template
- verified
- use case
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012795 verification Methods 0.000 title claims abstract description 337
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000013461 design Methods 0.000 claims abstract description 127
- 238000012360 testing method Methods 0.000 claims abstract description 71
- 238000003062 neural network model Methods 0.000 claims abstract description 52
- 238000005070 sampling Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000013528 artificial neural network Methods 0.000 description 18
- 238000013527 convolutional neural network Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 230000006399 behavior Effects 0.000 description 5
- 230000007613 environmental effect Effects 0.000 description 3
- 230000005284 excitation Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012827 research and development Methods 0.000 description 3
- 238000011176 pooling Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000012942 design verification Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000006386 memory function Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Neurology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Test And Diagnosis Of Digital Computers (AREA)
- Debugging And Monitoring (AREA)
- Storage Device Security (AREA)
- Testing Or Measuring Of Semiconductors Or The Like (AREA)
Abstract
The present disclosure provides an AI chip verification system, method, device and storage medium, wherein the system includes: the verification case generator is used for generating a verification case of the object to be verified based on information of the object to be verified and a preset case template, wherein the verification case meets a format defined by the preset case template, and the object to be verified comprises a target operator in a neural network model or the neural network model; the design to be tested module is used for executing the verification case through the design to be tested to obtain a test result; and the comparison module is used for comparing the test result with the reference result to obtain a verification result of the to-be-verified design on the to-be-verified object.
Description
Technical Field
The present disclosure relates to the field of AI chip verification, and in particular, to an AI chip verification system, method, device, and storage medium.
Background
Verification is an important stage in the chip development process, and the correctness of the chip function needs to be ensured through verification before the chip is delivered to a manufacturer for production.
In the research and development engineering aiming at the AI chip, due to the introduction of various types of neural networks, the chip is required to have stronger adaptability and robustness, so that the functional verification of the chip is correspondingly required to be capable of adapting to the neural networks of various types. However, the conventional chip verification method requires a special verification environment for the neural network to be verified, and cannot be adapted to verify various neural networks.
Disclosure of Invention
The present disclosure provides an AI chip verification system, method, device, storage medium, and computer program product.
According to a first aspect of the embodiments of the present disclosure, there is provided an AI chip verification system, the system including: the verification case generator is used for generating a verification case of the object to be verified based on information of the object to be verified and a preset case template, wherein the verification case meets a format defined by the preset case template, and the object to be verified comprises a target operator in a neural network model or the neural network model; the design to be tested module is used for executing the verification case through the design to be tested to obtain a test result; and the comparison module is used for comparing the test result with the reference result to obtain a verification result of the to-be-verified design on the to-be-verified object.
In the embodiment of the disclosure, the verification case is generated according to the preset case template with the predefined format, so that the AI chip verification system provided in the embodiment of the disclosure can adaptively verify neural network models of various types.
In some optional embodiments, the preset use case template comprises at least one of: the instruction stream template file comprises an instruction stream template of the verification case; initializing a data template file, wherein the initialized data template file comprises the initialized data template of the verification case; and the register configuration template file comprises a register configuration template of the verification use case.
In some optional embodiments, the system further comprises: and the environment driver is used for driving the verification use case to the to-be-tested design module based on at least one template file included in the preset use case template so as to execute the verification use case by the to-be-tested design.
In some optional embodiments, the preset use case template includes: a result template file containing an output result template of the verification case; the system further comprises: and the environment sampler is used for sampling the test result from the design module to be tested based on the result template file and transmitting the test result obtained by sampling to the comparison module.
In some optional embodiments, the use case generator comprises: and the orientation generator is used for acquiring the parameter file of the object to be verified and generating a verification case of the object to be verified by analyzing the object parameters contained in the parameter file of the object to be verified.
In the embodiment of the disclosure, the hierarchy of the neural network model can be verified, the whole neural network model can also be verified, and the verification depth and flexibility of the design to be tested on the neural network are improved.
In some optional embodiments, the parameter file comprises: a fixed-point parameter file and a network structure file.
In some optional embodiments, the preset use case template includes: a reference result file containing a reference result of the verification case; the reference result obtained by the comparison module is obtained from the preset case template.
In the embodiment of the disclosure, the reference result of the verification case can be directly obtained from the verification case, and the verification case is not required to be executed by a reference model, so that the execution efficiency of the comparison device is improved, and the verification efficiency of the design to be tested is improved.
In some optional embodiments, the use case generator comprises: and the random generator is used for acquiring the configuration file of the object to be verified, generating the object parameter of the object to be verified based on the configuration file, and generating the verification case of the object to be verified based on the generated object parameter.
In the embodiment of the disclosure, the random generator generates different verification cases by using a random technology based on a specific verification case according to the configuration file, so that the verification scene coverage is more comprehensive, and the verification completeness is effectively improved.
In some optional embodiments, the system further comprises: and the reference model is used for executing the verification use case generated by the use case generator to obtain a corresponding reference result.
In some optional embodiments, the preset use case template includes: a dispatching core instruction stream template file which contains a dispatching core instruction stream template of the verification case; the system further comprises: the scheduling instruction compiler is used for generating the scheduling core instruction stream of the design to be tested based on the scheduling core instruction stream template file; the use case generator is used for generating the verification use case of the object to be verified based on the scheduling core instruction stream generated by the scheduling instruction compiler.
In the disclosed embodiments, other modules of the AI chip verification system for level 1 DUTs are multiplexed. The chip verification workload is reduced, the verification efficiency of the design to be tested is improved, and the verification resources are saved.
In some optional embodiments, the system further comprises a performance statistics engine, connected to the design under test module, and configured to: and acquiring performance parameters of the to-be-tested design for executing the verification case, and acquiring a performance statistical result based on the performance parameters.
In the embodiment of the disclosure, the performance statistics device can perform statistics on the operation time and the bandwidth of the verification case of the hierarchy of the neural network model to be designed and executed, and also can perform statistics on the operation time and the bandwidth of the verification case of the network hierarchy of the whole neural network model to be designed and executed, thereby enriching the performance statistics on the verification of the design to be tested.
According to a second aspect of the embodiments of the present disclosure, there is provided an AI chip verification method, including: generating a verification use case of an object to be verified based on information of the object to be verified and a preset use case template, wherein the verification use case meets a format defined by the preset use case template, and the object to be verified comprises a target operator in a neural network model or the neural network model; executing the verification case through the design to be tested to obtain a test result; and comparing the test result with a reference result to obtain a verification result of the design to be tested on the object to be verified.
In the embodiment of the disclosure, the verification use case generated according to the information of the object to be verified is generated based on a format predefined in a preset use case template. The verification cases generated by the objects to be verified corresponding to the neural networks of different types have a uniform format, so that the AI chip verification system provided by the embodiment of the disclosure can adaptively verify the neural networks of various types.
In some optional embodiments, the preset use case template comprises at least one of: the instruction stream template file comprises an instruction stream template of the verification case; initializing a data template file, wherein the initialized data template file comprises the initialized data template of the verification case; and the register configuration template file comprises a register configuration template of the verification use case.
In some optional embodiments, after the generating the verification use case of the object to be verified based on the information of the object to be verified and a preset use case template, the method further includes: and driving the verification case into the to-be-tested design module based on at least one template file included in the preset case template so as to execute the verification case by the to-be-tested design.
In some optional embodiments, the preset use case template includes: a result template file containing an output result template of the verification case; after the verification case is executed through the design to be tested to obtain a test result, the method further comprises the following steps: and sampling the test result from the design module to be tested based on the result template file, and transmitting the test result obtained by sampling to the comparison module.
In some optional embodiments, the generating a verification use case of the object to be verified based on the information of the object to be verified and a preset use case template includes: and acquiring a parameter file of the object to be verified, and generating a verification case of the object to be verified by analyzing object parameters contained in the parameter file of the object to be verified.
In some optional embodiments, the parameter file comprises: a fixed-point parameter file and a network structure file.
In some optional embodiments, the preset use case template includes: and the reference result file comprises a reference result of the verification case.
In some optional embodiments, the generating a verification use case of the object to be verified based on the information of the object to be verified and a preset use case template includes: acquiring a configuration file of the object to be verified, generating object parameters of the object to be verified based on the configuration file, and generating a verification use case of the object to be verified based on the generated object parameters.
In some optional embodiments, the method further comprises: and executing the verification case through a reference model to obtain a corresponding reference result.
In some optional embodiments, the preset use case template includes: a dispatching core instruction stream template file which contains a dispatching core instruction stream template of the verification case; the method further comprises the following steps: generating a dispatching core instruction stream of the design to be tested based on the dispatching core instruction stream template file; and generating a verification use case of the object to be verified based on the dispatching core instruction stream.
In some optional embodiments, the method further comprises: and acquiring performance parameters of the to-be-tested design for executing the verification case, and acquiring a performance statistical result based on the performance parameters.
According to a third aspect of the embodiments of the present disclosure, there is provided a computer apparatus including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the AI chip verification method of any one of the second aspects when executing the program.
According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, the program implementing the AI chip verification method according to any one of the second aspects when executed by a processor.
According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product including a computer program that realizes the AI chip verification method according to any one of the second aspects when executed by a processor.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a schematic diagram of an AI chip verification system in accordance with an exemplary embodiment;
FIG. 2 is a diagram illustrating a preset use case template in accordance with an illustrative embodiment;
FIG. 3 is a schematic diagram illustrating another AI chip verification system in accordance with an exemplary embodiment;
FIG. 4 is a diagram illustrating another preset use case template, according to an example embodiment;
FIG. 5 is a schematic diagram illustrating yet another AI chip verification system in accordance with an exemplary embodiment;
FIG. 6 is a schematic diagram illustrating yet another AI chip verification system in accordance with an exemplary embodiment;
FIG. 7 is a schematic diagram illustrating a configuration file in accordance with an exemplary embodiment;
FIG. 8 is a schematic diagram illustrating a two-level design under test in accordance with an exemplary embodiment;
FIG. 9 is a schematic diagram illustrating yet another AI chip verification system in accordance with an exemplary embodiment;
fig. 10 is a flowchart illustrating an AI chip verification method according to an example embodiment.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The specific manner described in the following exemplary embodiments does not represent all aspects consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
In order to make the AI chip verification scheme provided by the present disclosure clearer, the following describes in detail the implementation process of the scheme provided by the present disclosure with reference to the accompanying drawings and specific embodiments.
Referring to fig. 1, fig. 1 is an AI chip verification system according to an embodiment of the disclosure. As shown in fig. 1, the AI chip verification system may include: the test system comprises a case generator 101, a to-be-tested design module 102 and a comparison module 103, wherein the case generator 101 is used for generating a verification case, the verification case is input into the to-be-tested design module 102 and processed by the to-be-tested design to obtain a test result, and the test result is input into the comparison module 103 to be processed to obtain a verification result of the to-be-tested design.
In the AI chip verification, it can be verified whether a newly designed AI chip can implement some operations (operators) in the neural network model, such as convolution, pooling in the neural network, etc., and whether the AI chip can successfully perform the operations. In the case of verifying some operators in the neural network model, the operators in the neural network model are used as objects to be verified. Alternatively, it may also be verified whether the newly designed AI chip can successfully execute the complete neural network model. In this case, the entire neural network model is used as the object to be verified. The neural network model may be a neural network for image recognition, such as a convolutional neural network, a neural network for speech recognition, or a neural network for text recognition, which is not limited by the present disclosure.
In the AI chip verification system shown in fig. 1, the use case generator 101 may analyze information of an object to be verified to obtain parameters for generating a verification use case of the object to be verified. The information of the object to be verified may be parameter information included in the fixed-point parameter file and the network structure file, or may be parameter information in a configuration file corresponding to the object to be verified. For example, taking a convolution operation in the neural network model as an object to be verified, the use case generator 101 may analyze parameter information in the fixed-point parameter file and the network structure file corresponding to the convolution operation to obtain parameters of a verification use case used for generating the convolution operation. Or, taking the complete neural network model as an object to be verified, the use case generator 101 may analyze the fixed-point parameter file and the network structure file corresponding to the neural network model to obtain parameters of a verification use case for generating the neural network model.
After obtaining the parameters of the verification use case for generating the object to be verified, the use case generator 101 generates the verification use case of the object to be verified according to the generated parameters and based on the format defined in the preset use case template. The preset case template defines the format of data in the verification case to be generated. For example, the preset use case template may define a format for a particular portion of data of a verification use case to be generated. Specifically, the preset use case template may define formats of instruction stream data and initialization data in the verification use case to be generated, that is, format definition of specific parts (instruction stream data and initialization data) is completed. For data formats other than a specific part in the verification use case to be generated, the preset use case template can be defined according to specific situations. Alternatively, the preset use case template may perform format definition on all data of the verification use case to be generated, which is not limited in this disclosure.
In some optional embodiments, the preset use case template comprises at least one of:
the instruction stream template file comprises an instruction stream template of the verification case;
initializing a data template file, wherein the initialized data template file comprises the initialized data template of the verification case;
and the register configuration template file comprises a register configuration template of the verification use case.
In the above embodiment, the use case generator 101 may generate different data in the verification use case according to each template file in the preset use case template. The use case generator 101 may generate an instruction stream in the instruction stream template definition format according to the instruction stream template in the instruction stream template file; generating initialization data in a format defined by an initialization data template according to the initialization data template in the initialization data template file; and generating register configuration data in a register configuration template definition format according to the register configuration template in the register configuration template file. The use case generator 101 generates a verification use case of the object to be verified according to a format defined by a preset use case template.
Illustratively, FIG. 2 illustrates a preset use case template. The use case generator 101 can generate an instruction stream in a verification use case according to the instruction stream template file; generating initialization data in the verification case according to the 'picture/weight data' template file and the 'Initial list' template file: "picture/weight data" and "Initial list" data; and generating Register configuration data in the verification use case according to the 'Register config' template file.
After the case generator 101 generates the verification case, the to-be-tested Design module 102 may obtain the verification case and execute the verification case by a to-be-tested Design (DUT) to obtain a Test result.
In some optional embodiments, as shown in fig. 3, the system further comprises: the environment driver 301 is configured to drive the verification use case into the to-be-tested design module 102 based on at least one template file included in the preset use case template, so that the to-be-tested design executes the verification use case.
In the above embodiment, the environment driver 301 may parse specific data in the verification case, and perform excitation driving on the parsed data. For example, the environment driver 301 may drive the instruction stream, the initialization data, and the register configuration data in the verification case into the design under test module 102, so that the design under test may execute the verification case.
The verification case generated by the preset case template shown in fig. 2 is taken as an example for explanation. The environment Driver 301 may parse a "picture/weight data" file, "an Initial list" file, "an instruction stream" file, and a Register config file in the Verification use case, and perform excitation driving on four portions of the design module to be tested 102 through a Driver component in a Universal Verification Methodology (UVM).
A first part: and configuring the register of the design to be tested by using the register configuration data in the verification case. Among them, register configuration operation needs to strictly follow interface protocols such as AHB (Advanced High performance Bus) and APB (Advanced Peripheral Bus).
A second part: initializing an external storage model vip model connected with the design to be tested. The initialization of the external storage model vip model needs to utilize a vip background technology to operate on a used memory area.
And a third part: initializing an internal storage model of the design to be tested. The initialization of the DUT internal storage model memory model needs to encapsulate a DUT memory function and perform accurate address write operation on an internal shared storage space.
The fourth part: and performing excitation driving on the instruction stream in the verification case. And transmitting the instruction stream in the verification case instruction stream file to an instruction interface of the DUT according to a specified format by utilizing the time sequence characteristic of the UVM Driver.
The environment driver 301 drives the data of the verification use case into the design under test module 102, and the verification use case is executed by the design under test. The design to be tested included in the module to be tested 102 may be a complete chip of a new design to be verified, or may be a specific module of the new design to be verified, which constitutes the complete chip, and the disclosure is not limited thereto.
After the design to be tested in the design to be tested module 102 executes the verification case, a test result corresponding to the verification case is obtained. The comparison module 103 may obtain a test result after the verification case is executed by the design under test from the design under test module 102.
In some optional embodiments, the preset use case template includes: and the result template file comprises an output result template of the verification case. As shown in fig. 3, the system further comprises: the environment sampler 302 is configured to sample the test result from the to-be-tested design module 102 based on the result template file, and transmit the test result obtained by sampling to the comparison module 103.
Referring to the preset use case template illustrated in fig. 5, the "Dump list" template file may be used as a result template file, and includes an output result template of a verification use case. The environment sampler 302 may sample a test result obtained after the to-be-tested design executes the verification case from the to-be-tested design module 102 according to the output result template in the "Dump list" template file.
For example, the environment sampler 302 performs expansion through a Monitor component in the UVM, customizes a sampling task of the result data dump _ data, and samples the result data in the external storage model into the temporary array according to a timing requirement in the DUT input/output interface. The temporary array is then printed into the file according to the format of the output result template in the result template file, such as according to the format of the "Dump list" template file. The sampled result data comprises a network level operation result of the AI network, and the sampling condition is an msg signal of the network layer end.
The environment driver 301 and the environment sampler 302 in the AI chip verification system shown in fig. 3 are independent from the design module to be tested 102, and in other implementations, the environment driver 301 and the environment sampler 302 may be integrated in the design module to be tested 102, which is not limited in this disclosure.
The reference result used for comparing with the test result in the comparison module 103 may be result data directly obtained from the verification case by the comparison module 103, or may be result data obtained after the verification case executes the verification case through a reference model. The reference model is a model for verifying the AI chip to be verified or each function of the design to be tested and is responsible for simulating the logic behavior of the DUT to be tested. For example, the method is realized by a System C coding model, and is responsible for generating Golden results for comparison, and the Golden results are generated so as to ensure the consistency of the collected data with an environmental sampler.
In some realizable manners, the comparison module 103 is responsible for performing correctness comparison on the test result of the environment sampler 302 and the reference result corresponding to the verification case, and supports multiple forms of result data comparison. For example, comparison of result data stored inside the DUT is supported; and the comparison of network results in an external storage model is supported. For comparison time, data comparison can be performed at the end of each network layer of the network, and the data result of each layer is compared; the data results of all network layers may be compared at the end of the execution of the verification use case. Therefore, the research and development personnel can carry out targeted modification on the currently verified design to be tested according to the verification result.
In the embodiment of the disclosure, the verification use cases are generated according to the preset use case template with the predefined format, so that the verification use cases generated by the objects to be verified corresponding to the neural networks of different types have a uniform format, and therefore, the AI chip verification system provided in the embodiment of the disclosure can adapt to verify the neural networks of various types.
In some alternative embodiments, as shown in fig. 5, another AI chip verification system is provided in the embodiments of the present disclosure, in which the use case generator 101 includes an orientation generator 401. The orientation generator 401 may obtain a parameter file of the object to be verified, analyze object parameters included in the parameter file of the object to be verified, and generate a verification case of the object to be verified. The object parameters in the parameter file are parameters required by the orientation generator 401 when generating the verification use case of the object to be verified. For example, the parameter file may be one or more of a fixed-point parameter file or a network structure file, or may be another parameter file containing object parameters, and the embodiments of the present disclosure are not limited thereto.
Taking the parameter file as an example, including the fixed-point parameter file and the network structure file, the orientation generator 401 may obtain the fixed-point parameter file and the network structure file of the object to be verified, analyze the object parameters included in the two files, and generate the verification case of the object to be verified based on the object parameters.
Specifically, C + + is taken as an example of the programming language. The orientation generator 401 may analyze the localization parameter file and the network structure file of the neural network model, and call a Printf function to generate an instruction stream according to the analyzed object parameters. The orientation generator 401 generates an instruction stream of a verification case according to a format defined by an instruction stream template in a preset case template based on the generated instruction stream; further, other data in the verification use case are generated according to other template files in the preset use case template, and therefore the complete verification use case is obtained. The data of the verification case is driven to the design under test module 102 through the environment driver 301, and the verification case is executed by the design under test DUT. And sampling by the environment sampler 302 to obtain a test result corresponding to the verification case. The related specific processes are similar to those described in the foregoing embodiments, and are not described again here.
In the embodiment of the present disclosure, the verification case generated by the orientation generator 401 may include a reference result of the verification case. Correspondingly, the preset use case template comprises: and the reference result file contains the reference result of the verification case. Taking the preset use case template shown in fig. 4 as an example, the "Layer level data" template file may be used as a reference result file, and contains the reference result of the verification use case generated by the orientation generator 401. In this way, the comparison module 103 may directly obtain the reference result from the verification case and compare the reference result with the test result obtained from the design under test module 102.
In another implementation manner, when generating the verification case, the orientation generator 401 may execute the verification case through the reference model to obtain a reference result, and the reference result is transmitted to the alignment module 103 by the reference model, regardless of whether the reference result of the verification case is generated or not. The reference model is a model for verifying each function of a chip to be verified or a design to be tested and is responsible for simulating the logic behavior of the DUT to be tested.
According to the AI chip verification method, the comparison module can directly obtain the reference result of the verification case from the verification case without executing the verification case by a reference model, so that the execution efficiency of the comparison device is improved, and the verification efficiency of the design to be tested is improved.
In some alternative embodiments, the orientation generator 401 may generate a verification use case in which a certain layer or layers in the neural network model are to be used as objects to be verified. For example, a verification case in which a convolution layer in the convolutional neural network model is used as an object to be verified may be generated. One or more layers in the neural network model are used as objects to be verified and can be called hierarchical verification objects.
Taking the AI chip verification system shown in fig. 5 as an example, when hierarchical verification is supported, the orientation generator 401 may analyze a parameter file corresponding to a target network layer in the neural network model, and generate a verification case in which the target network layer is used as a hierarchical verification object according to a format defined by a preset case template. The target network layer may be one or more network layers in the neural network model, such as a convolutional layer and a pooling layer, or may also be multiple convolutional layers, multiple fully-connected layers, and the like.
Specifically, for example, a convolutional layer in the convolutional neural network model is verified to test whether the design under test can successfully perform the operation of the convolutional layer. In the embodiment of the present disclosure, the orientation generator 401 generates the verification case of the convolutional layer as the hierarchical verification object based on the format defined by the preset case template by analyzing the fixed-point parameter file and the network structure file corresponding to the convolutional layer in the neural network model.
In the embodiment of the present disclosure, the orientation generator 401 may further generate a certain complete neural network model as a verification use case of the object to be verified. For example, a verification case corresponding to the convolutional neural network model may be generated. This model of the entire neural network as the object to be verified may be referred to as a network-level verification object.
Specifically, for example, a convolutional neural network model for image recognition is verified to test whether the design under test can completely perform the operation in the convolutional neural network model. In the embodiment of the present disclosure, the orientation generator 401 generates the verification case of the convolutional neural network model as the network-level verification object by analyzing the fixed-point parameter file and the network structure file corresponding to the convolutional neural network model based on the format defined by the preset case template.
The AI chip verification method provided by the embodiment of the disclosure can verify the hierarchy of the neural network model and verify the whole neural network model, thereby improving the verification depth and flexibility of the design to be tested on the neural network. Therefore, the Bug generated in the chip research and development is more accurately positioned and corrected, and the verification convergence of the chip is accelerated.
In some optional embodiments, as shown in fig. 6, in the embodiment of the present disclosure, a further AI chip verification system is provided, in which the use case generator 101 includes a random generator 501, and a reference model 502 is further included between the verification use case and the comparison apparatus 103. The reference model 502 is a model for verifying the AI chip to be verified or each function of the design to be tested, and is responsible for simulating the logic behavior of the DUT to be tested. For example, the method is realized by a System C coding model, and is responsible for generating Golden results for comparison, and the Golden results are generated so as to ensure the consistency of the collected data with an environmental sampler.
In this embodiment, the random generator 501 may obtain a configuration file of the object to be verified, generate object parameters of the object to be verified by using a randomization technique based on the parameters in the configuration file, and generate a verification case by using the randomly generated object parameters. The configuration file is a file configured by a verifier for a to-be-verified design, such as an Excel file in which the verifier performs parameter configuration by using Excel. An Excel profile is illustrated in figure 7.
The random generator 501 may parse the configuration file, and randomly generate object parameters of the object to be verified by using the parameters configured by the configuration file. For example, using random constraint technology of systemveilog language, a source operand range, a source operand address, a target operand range, a target operand address, and calculation parameters of an operator are generated in a randomized manner. Taking the configuration file illustrated in fig. 7 as an example, the random generator 501 may analyze n parameters in case-1 in an Excel file, and randomly generate a part of the n parameters by using random constraint technology. The "part of the parameters" may be one of the n parameters, or may be several of the n parameters, and the embodiment of the present disclosure is not limited thereto. After randomization, a plurality of sets of parameters different from case-1 are obtained as object parameters of the object to be verified, and the random generator 501 may generate the verification case based on these randomly generated object parameters.
Fig. 7 shows an Excel file configured by a verifier for a neural network model to be verified and a design to be tested, where the configuration file includes parameters for generating a plurality of specific verification use cases. For example, the row with sequence number 1 includes n parameters of case-1: param-1-1. param-1-n, and the corresponding case-2, case-3. case-n, etc., are composed of n parameters. The n configuration parameters in each case correspond to a specific verification case of the case. The random generator 501 may generate a verification case with a wider coverage area based on a specific verification case by using a randomization technique.
In the process of generating the verification use case, the random generator 501 generates the object parameters of the object to be verified by using a random technique, so that the reference result of the generated verification use case is unknown. Therefore, in the embodiment of the present disclosure, the AI chip verification system as shown in fig. 6 includes the reference model 502. The reference model 502 is used for executing the verification case generated by the random generator 501 to obtain a reference result of the verification case. The reference model 502 is a model for verifying each function of a chip to be verified or a design to be tested, and is responsible for simulating the logic behavior of the DUT. For example, the method is realized by a System C coding model, and is responsible for generating Golden results for comparison, and the Golden results are generated so as to ensure the consistency of the collected data with an environmental sampler.
In the above embodiment, the random generator 501 generates different verification use cases by using a random technique based on a specific verification use case according to the configuration file, so that the verification scene coverage is more comprehensive, and the verification completeness is effectively improved.
In the AI chip verification system shown in fig. 6, only the random generator 501 is included in the use case generator 101. In other realizable manners, both the directional generator 401 and the random generator 501 may be included in the use case generator 101. The AI chip verification systems shown in fig. 5 and 6 are shown separately for clarity of description of the embodiments and are not intended to be limiting in any way.
In the AI chip verification process, the design under test to be verified may be divided into a level 1 DUT and a level 2 DUT. Referring to fig. 8, a level 2 DUT adds a scheduling core on a level 1 DUT basis. Wherein the level 1 DUT can execute the instruction stream in the verification use case. The scheduling core in the 2-level DUT can execute the instruction flow of the scheduling core so as to carry, schedule, control and distribute the instruction flow in the verification case, thereby improving the flexibility of the design to be tested.
Take the verification case generated by the preset case template of fig. 4 as an example. The level 1 DUT can execute an instruction stream generated according to the instruction stream template file in the verification case; the scheduling core in the 2-level DUT can execute the scheduling core instruction stream generated according to the scheduling core instruction stream template file in the verification case, and carry, schedule, control and distribute the instruction stream in the verification case. The use case generator (including the random generator and the directional generator) provided in the above embodiment can generate only the verification use case corresponding to the level 1 DUT since the "scheduling core instruction stream" cannot be generated.
As shown in fig. 9, yet another AI chip verification system is provided in the embodiments of the present disclosure. The AI chip verification system also includes a scheduling instruction compiler 601 corresponding to the level 2 DUT. The scheduling instruction compiler 601 may compile an instruction control stream to obtain a scheduling core instruction stream executable by a scheduling core in a level 2 DUT. The instruction control flow is described in a certain programming language and used for controlling the instruction flow in the verification use case. For example, the scheduling instruction compiler 601 compiles an instruction control stream described in the C language code, and outputs a binary assembly file executable by the scheduling core in the 2-level DUT, i.e., obtains the scheduling core instruction stream.
Corresponding to the AI chip verification system shown in fig. 9, the preset case template includes: and scheduling core instruction stream template files, wherein the scheduling core instruction stream templates of the verification cases are contained. The scheduling instruction compiler 601 compiles the instruction control stream, and generates a scheduling core instruction stream in the verification case according to a format defined by the scheduling core instruction stream template. Take the preset use case template shown in fig. 4 as an example. The use case generator 101 may generate data other than the "scheduling core instruction stream" in the 2-level verification use case according to template files other than the "scheduling core instruction stream template file", thereby generating a complete verification use case corresponding to the 2-level DUT.
Compared with the AI chip verification system corresponding to the level-1 DUT, the AI chip verification system in the embodiment of the present disclosure only adds the scheduling instruction compiler 601 and multiplexes other modules of the AI chip verification system for the level-1 DUT. The chip verification workload is reduced, the verification efficiency of the design to be tested is improved, and the verification resources are saved.
In some optional embodiments, as shown in fig. 9, the AI chip verification system further includes a performance statistics engine 602 connected to the design module under test 102. It is understood that this is only one specific configuration of the exemplary AI chip verification system, and that other configurations are possible, such as a system that does not include the schedule instruction compiler 601, but is used to verify a level 1 DUT. In the embodiment of the present disclosure, the performance statistics device 602 may obtain the performance parameters of the to-be-tested design execution verification case from the to-be-tested design module 102, and perform performance statistics based on the obtained performance parameters.
The performance statistics 602 may count the time for the design under test to execute the verification use case. Specifically, the performance statistics unit 602 may count the time of layer operation of each layer of the neural network model, or count the entire execution time of the neural network model, starting from the reset signal being pulled high.
For example, if the verification case executed by the design under test is a verification case of a hierarchical verification object, the performance statistics unit 602 may perform statistics on the execution time of the corresponding network layer in the neural network model in the design under test, for example, statistics on the operation time of the convolution layer in the design under test. For example, if the verification case executed by the design under test is the verification case corresponding to the entire neural network model, the performance statistics unit 602 may perform statistics on the time for the design under test to operate the entire neural network model, for example, perform time statistics on the operation of the entire convolutional neural network model in the design under test.
In some implementations, the performance statistics 602 may also count the actual bandwidth of the design under test. Specifically, the performance statistics device 602 may count the read-write data amount of all the interfaces in the design to be tested, and calculate the actual bandwidth of the design to be tested by combining the counted operation time of the design to be tested corresponding to the read-write data amount. For example, if the design under test executes a level verification case for a hierarchy, the performance statistics engine 602 may calculate the actual bandwidth of the neural network model corresponding to the network layer of the design under test. For example, the design under test implements the actual bandwidth of the verification case for the convolutional layer. For example, if the design under test performs a verification use case for the network level, the performance statistics engine 602 may calculate the actual bandwidth of the entire neural network model of the design under test. For example, the design under test implements the actual bandwidth of the entire convolutional neural network.
In the above embodiment, the AI chip verification system is additionally provided with the performance statistics device, which can perform statistics on performance parameters such as time and bandwidth of the verification case executed by the design to be tested. Moreover, the performance statistics device can be used for counting the operation time and the bandwidth of the hierarchical verification case of the to-be-tested design execution neural network model, and also can be used for counting the operation time and the bandwidth of the network-level verification case of the whole to-be-tested design execution neural network model, so that the performance statistics of the to-be-tested design verification is enriched.
In some alternative embodiments, as shown in fig. 10, an embodiment of the present disclosure shows an AI chip verification method. The AI chip verification method or the AI chip verification system provided by the embodiments of the present disclosure may be applied to a server, a terminal device, or other types of electronic devices that perform AI chip verification. The AI chip verification process shown in fig. 10 includes the following steps:
In this step, the information of the object to be verified may be analyzed to obtain parameters for generating a verification case of the verification object. The object to be verified can be an operator in the neural network model, or can be the whole neural network model. The information of the object to be verified may be parameter information contained in the fixed-point parameter file and the network structure file, or may be parameter information in a configuration file corresponding to the verification object.
And after the parameters of the verification use case for generating the verification object are obtained, generating the verification use case of the object to be verified according to the generated parameters and based on the format defined in the preset use case template. The preset use case template defines a format of data in a verification use case to be generated, and specific description may refer to related description of some embodiments of the verification system.
In some realizable modes, the parameter file of the object to be verified can be obtained, the object parameters contained in the parameter file of the object to be verified are analyzed, and the verification use case of the object to be verified is generated. The related description may refer to the related part description of the embodiment of the verification system. The realization mode can generate a corresponding reference result in the verification case, and the verification case is not required to be executed by a reference model, so that the efficiency of verifying the design to be tested is improved. Moreover, the hierarchy of the neural network model can be verified, the whole neural network model can be verified, and the verification depth and flexibility of the design to be tested on the neural network are improved.
In other realizable manners, a configuration file of the object to be verified can be obtained, the object parameters of the object to be verified are generated by using a randomization technology based on the parameters in the configuration file, and the verification use case is generated by using the randomly generated object parameters. The relevant detailed description may also refer to the description of the relevant parts of the embodiments of the verification system of the present disclosure. According to the implementation mode, different verification cases are generated by using a random technology based on a specific verification case, so that the verification scene coverage is more comprehensive, and the verification completeness is effectively improved.
And 102, executing the verification case through the design to be tested to obtain a test result.
After the verification case of the object to be verified is generated, the verification case can be executed through the design to be tested, and the executed test result is obtained. The design to be tested can be included in the design module to be tested, and the data of the verification case is driven into the design module to be tested through the environment driver to be executed by the design to be tested. After the design to be tested executes the verification case, the environment sampler can sample the test result after the verification case is executed from the design module to be tested. The description is consistent with the description of the embodiments of the verification system in the present disclosure, and is not repeated here.
The reference result compared with the test result of the test case in this step may be result data already existing in the verification case, or may be result data obtained after the verification case is executed through the reference model. The reference model is a model for verifying the AI chip to be verified or each function of the design to be tested and is responsible for simulating the logic behavior of the DUT to be tested.
In the embodiment of the disclosure, the format of the verification case is predefined by using the preset case template, and the verification cases generated by the objects to be verified corresponding to the neural networks of different types have a uniform format, so that the AI chip verification method can be adapted to verify the neural networks of various types.
In some optional embodiments, the instruction control stream may be compiled to obtain a scheduling core instruction stream executable by a scheduling core in the level 2 DUT. The corresponding preset use case template comprises: and scheduling core instruction stream template files, wherein the scheduling core instruction stream templates of the verification cases are contained.
Specifically, the instruction control stream may be compiled, and the scheduling core instruction stream in the verification case is generated according to a format defined by the scheduling core instruction stream template. And then generating other data except the scheduling core instruction stream in the 2-level verification case according to other template files except the scheduling core instruction stream template file, thereby generating a complete verification case corresponding to the 2-level DUT. The method for generating the 2-level DUT verification case can reuse the module for generating the 1-level DUT verification case, reduce the workload of chip verification and improve the verification efficiency.
In some optional embodiments, the performance parameters of the design to be tested execution verification use case may be obtained, and performance statistics may be performed based on the obtained performance parameters. For example, the actual bandwidth of the design to be tested can be calculated by calculating the time for the design to be tested to execute the verification case. The specific statistical process can be referred to the description of the relevant part of the embodiment of the verification system, and is not described here. The performance statistics of the embodiment can be used for counting the operation time and the bandwidth of the hierarchical verification case of the neural network model to be executed by the design to be tested, and also can be used for counting the operation time and the bandwidth of the network-level verification case of the whole neural network model to be executed by the design to be tested, so that the performance statistics of the verification of the design to be tested is enriched.
The disclosure also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor is capable of implementing the AI chip verification method of any embodiment of the disclosure when executing the program.
The present disclosure also provides a computer-readable storage medium on which a computer program is stored, the program being capable of implementing the AI chip verification method of any of the embodiments of the present disclosure when executed by a processor.
The non-transitory computer readable storage medium may be, among others, ROM, Random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like, and the present disclosure is not limited thereto.
The present disclosure also provides a computer program product comprising a computer program that, when executed by a processor, implements the AI chip verification method of any embodiment of the present disclosure.
Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
The above description is only exemplary of the present disclosure and is not intended to limit the present disclosure, so that any modification, equivalent replacement, or improvement made within the spirit and principle of the present disclosure should be included in the scope of the present disclosure.
Claims (21)
1. An AI chip verification system, the system comprising:
the verification case generator is used for generating a verification case of the object to be verified based on information of the object to be verified and a preset case template, wherein the verification case meets a format defined by the preset case template, and the object to be verified comprises a target operator in a neural network model or the neural network model;
the design to be tested module is used for executing the verification case through the design to be tested to obtain a test result;
and the comparison module is used for comparing the test result with the reference result to obtain a verification result of the to-be-verified design on the to-be-verified object.
2. The system of claim 1, wherein the preset use case template comprises at least one of:
the instruction stream template file comprises an instruction stream template of the verification case;
initializing a data template file, wherein the initialized data template file comprises the initialized data template of the verification case;
and the register configuration template file comprises a register configuration template of the verification use case.
3. The system according to any one of claims 1 to 2, further comprising:
and the environment driver is used for driving the verification use case to the to-be-tested design module based on at least one template file included in the preset use case template so as to execute the verification use case by the to-be-tested design.
4. The system according to any one of claims 1 to 3, wherein the preset use case template comprises: a result template file containing an output result template of the verification case;
the system further comprises: and the environment sampler is used for sampling the test result from the design module to be tested based on the result template file and transmitting the test result obtained by sampling to the comparison module.
5. The system according to any one of claims 1 to 4, wherein the use case generator comprises:
and the orientation generator is used for acquiring the parameter file of the object to be verified and generating a verification case of the object to be verified by analyzing the object parameters contained in the parameter file of the object to be verified.
6. The system of claim 5, wherein the parameter file comprises: a fixed-point parameter file and a network structure file.
7. The system according to any one of claims 5 to 6, wherein the preset use case template comprises: a reference result file containing a reference result of the verification case;
the reference result obtained by the comparison module is obtained from the preset case template.
8. The system according to any one of claims 1 to 4, wherein the use case generator comprises:
and the random generator is used for acquiring the configuration file of the object to be verified, generating the object parameter of the object to be verified based on the configuration file, and generating the verification case of the object to be verified based on the generated object parameter.
9. The system according to any one of claims 1 to 8, further comprising:
and the reference model is used for executing the verification use case generated by the use case generator to obtain a corresponding reference result.
10. The system according to any one of claims 1 to 9, wherein the preset use case template comprises: a dispatching core instruction stream template file which contains a dispatching core instruction stream template of the verification case;
the system further comprises:
the scheduling instruction compiler is used for generating the scheduling core instruction stream of the design to be tested based on the scheduling core instruction stream template file;
the use case generator is used for generating the verification use case of the object to be verified based on the scheduling core instruction stream generated by the scheduling instruction compiler.
11. The system according to any one of claims 1 to 10, further comprising a performance statistics engine, coupled to the design under test module, configured to: and acquiring performance parameters of the to-be-tested design for executing the verification case, and acquiring a performance statistical result based on the performance parameters.
12. An AI chip verification method, the method comprising:
generating a verification use case of an object to be verified based on information of the object to be verified and a preset use case template, wherein the verification use case meets a format defined by the preset use case template, and the object to be verified comprises a target operator in a neural network model or the neural network model;
executing the verification case through the design to be tested to obtain a test result;
and comparing the test result with a reference result to obtain a verification result of the design to be tested on the object to be verified.
13. The method of claim 12, wherein the preset use case template comprises at least one of:
the instruction stream template file comprises an instruction stream template of the verification case;
initializing a data template file, wherein the initialized data template file comprises the initialized data template of the verification case;
and the register configuration template file comprises a register configuration template of the verification use case.
14. The method according to any one of claims 12 to 13, wherein after generating the verification use case of the object to be verified based on the information of the object to be verified and a preset use case template, the method further comprises:
and driving the verification case into the to-be-tested design module based on at least one template file included in the preset case template so as to execute the verification case by the to-be-tested design.
15. The method according to any one of claims 12 to 14, wherein the preset use case template comprises: a result template file containing an output result template of the verification case;
after the verification case is executed through the design to be tested to obtain a test result, the method further comprises the following steps:
and sampling the test result from the design module to be tested based on the result template file, and transmitting the test result obtained by sampling to the comparison module.
16. The method according to any one of claims 12 to 15, wherein the generating a verification use case of the object to be verified based on the information of the object to be verified and a preset use case template comprises:
and acquiring a parameter file of the object to be verified, and generating a verification case of the object to be verified by analyzing object parameters contained in the parameter file of the object to be verified.
17. The method of claim 16, wherein the preset use case template comprises: and the reference result file comprises a reference result of the verification case.
18. The method according to any one of claims 12 to 15, wherein the generating a verification use case of the object to be verified based on the information of the object to be verified and a preset use case template comprises:
acquiring a configuration file of the object to be verified, generating object parameters of the object to be verified based on the configuration file, and generating a verification use case of the object to be verified based on the generated object parameters.
19. The method according to any one of claims 12 to 18, wherein the preset use case template comprises: a dispatching core instruction stream template file which contains a dispatching core instruction stream template of the verification case;
the method further comprises the following steps:
generating a dispatching core instruction stream of the design to be tested based on the dispatching core instruction stream template file;
and generating a verification use case of the object to be verified based on the dispatching core instruction stream.
20. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of claims 12-19 when executing the program.
21. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of claims 12-19.
Priority Applications (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010476115.3A CN113742202A (en) | 2020-05-29 | 2020-05-29 | AI chip verification system, method, device and storage medium |
| PCT/CN2020/119406 WO2021238006A1 (en) | 2020-05-29 | 2020-09-30 | Artificial intelligence chip verification |
| KR1020217031299A KR20210149045A (en) | 2020-05-29 | 2020-09-30 | artificial intelligence chip verification |
| JP2021557138A JP7270764B2 (en) | 2020-05-29 | 2020-09-30 | artificial intelligence chip verification |
| TW109145251A TWI778473B (en) | 2020-05-29 | 2020-12-21 | Artificial intelligence chip verification systems and methods, devices and storage media |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010476115.3A CN113742202A (en) | 2020-05-29 | 2020-05-29 | AI chip verification system, method, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113742202A true CN113742202A (en) | 2021-12-03 |
Family
ID=78724695
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010476115.3A Pending CN113742202A (en) | 2020-05-29 | 2020-05-29 | AI chip verification system, method, device and storage medium |
Country Status (5)
| Country | Link |
|---|---|
| JP (1) | JP7270764B2 (en) |
| KR (1) | KR20210149045A (en) |
| CN (1) | CN113742202A (en) |
| TW (1) | TWI778473B (en) |
| WO (1) | WO2021238006A1 (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114912413A (en) * | 2022-04-29 | 2022-08-16 | 上海阵量智能科技有限公司 | Chip verification method and platform |
| CN114996076A (en) * | 2022-07-01 | 2022-09-02 | 北京联盛德微电子有限责任公司 | Traversal type use case verification method and system for chip simulation and electronic equipment |
| CN115495388A (en) * | 2022-11-17 | 2022-12-20 | 瀚博半导体(上海)有限公司 | Chip verification method, device, equipment and medium for AI inference chip |
| CN115792580A (en) * | 2023-01-30 | 2023-03-14 | 北京怀美科技有限公司 | Chip detection system and editable chip detection system |
| CN116431421A (en) * | 2023-06-13 | 2023-07-14 | 成都登临科技有限公司 | Neural network generation method and generator, verification method and system and storage medium |
| CN117493168A (en) * | 2023-09-28 | 2024-02-02 | 中国长江电力股份有限公司 | Water conservancy model calling method capable of conducting offline testing |
| CN117743194A (en) * | 2024-02-19 | 2024-03-22 | 睿思芯科(深圳)技术有限公司 | Automatic generation method, system and related equipment for chip verification use cases |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116302721A (en) * | 2021-12-20 | 2023-06-23 | 瑞昱半导体股份有限公司 | Chip design verification system and method and computer readable recording medium |
| CN114818600A (en) * | 2022-04-29 | 2022-07-29 | 上海阵量智能科技有限公司 | Chip verification method and device, electronic equipment and storage medium |
| CN115099186A (en) * | 2022-07-05 | 2022-09-23 | 爱芯元智半导体(上海)有限公司 | Chip verification method and device, electronic equipment and storage medium |
| CN115047323B (en) * | 2022-08-17 | 2022-10-25 | 中诚华隆计算机技术有限公司 | Intelligent test method and system for chip |
| CN115794503A (en) * | 2022-09-30 | 2023-03-14 | 湖南智存合壹信息科技有限公司 | High-performance testing device and method based on domestic CPU mainboard |
| CN115879409B (en) * | 2022-12-02 | 2023-09-19 | 南京金阵微电子技术有限公司 | Verification method, verification device, medium and electronic equipment |
| KR102662404B1 (en) * | 2023-07-22 | 2024-04-29 | 주식회사 플랜터 | AI-based Multi-Purpose Intelligent Semiconductor System |
| CN117034824B (en) * | 2023-08-25 | 2024-09-20 | 广州市粤港澳大湾区前沿创新技术研究院 | Simulation verification system, method, terminal and medium for multiplexing test cases and verification environments |
| CN117056897B (en) * | 2023-10-13 | 2023-12-26 | 沐曦集成电路(上海)有限公司 | Configuration information processing method for chip verification, electronic device and medium |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100389399C (en) * | 2006-11-20 | 2008-05-21 | 北京中星微电子有限公司 | Method and system of chip checking |
| CN102402628A (en) * | 2010-09-07 | 2012-04-04 | 无锡中星微电子有限公司 | Method and system for generating systems-on-a-chip (SoC) verification platform |
| CN104486169A (en) * | 2015-01-07 | 2015-04-01 | 北京华力创通科技股份有限公司 | Reusable automatic detection and random verification system and method |
| CN104978249A (en) * | 2014-04-09 | 2015-10-14 | 三星电子株式会社 | System On Chip And Verification Method Thereof |
| CN109086199A (en) * | 2018-06-22 | 2018-12-25 | 网宿科技股份有限公司 | A kind of automation generates the method for test script, terminal and can storage medium |
| US20190096659A1 (en) * | 2017-09-26 | 2019-03-28 | Samsung Electronics Co., Ltd. | Systems and methods for wafer map analysis |
| CN109634843A (en) * | 2018-10-31 | 2019-04-16 | 中国科学院软件研究所 | A kind of distributed automatization method for testing software and platform towards AI chip platform |
| CN110489344A (en) * | 2019-08-02 | 2019-11-22 | Oppo广东移动通信有限公司 | Engine test method and Related product |
| US20200042419A1 (en) * | 2018-07-31 | 2020-02-06 | Alibaba Group Holding Limited | System and method for benchmarking ai hardware using synthetic ai model |
| CN111045880A (en) * | 2019-12-17 | 2020-04-21 | 湖南长城银河科技有限公司 | Chip testing method, verification system and storage medium |
| CN111178512A (en) * | 2019-12-31 | 2020-05-19 | 中国科学院自动化研究所南京人工智能芯片创新研究院 | Device operation neural network test method and device |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW200745895A (en) * | 2006-06-07 | 2007-12-16 | Powerchip Semiconductor Corp | Method and system for virtual metrology prediction and prediction model building in semiconductor manufacture |
| US20080115114A1 (en) * | 2006-11-10 | 2008-05-15 | Sashank Palaparthi | Automated software unit testing |
| CN102109572B (en) * | 2009-12-23 | 2014-12-31 | 中兴通讯股份有限公司 | Method for testing and method for testing and controlling transmission chip |
| CN104268078B (en) * | 2014-09-23 | 2017-03-15 | 北京控制工程研究所 | A kind of chip automatic verification method based on parametrization IP test case set |
| US10990300B2 (en) * | 2018-04-16 | 2021-04-27 | Lexmark International, Inc. | Methods for restricting read access to supply chips |
| CN110941519B (en) * | 2019-12-17 | 2022-12-13 | 锐捷网络股份有限公司 | Chip testing method and device of network equipment |
-
2020
- 2020-05-29 CN CN202010476115.3A patent/CN113742202A/en active Pending
- 2020-09-30 KR KR1020217031299A patent/KR20210149045A/en not_active Application Discontinuation
- 2020-09-30 JP JP2021557138A patent/JP7270764B2/en active Active
- 2020-09-30 WO PCT/CN2020/119406 patent/WO2021238006A1/en active Application Filing
- 2020-12-21 TW TW109145251A patent/TWI778473B/en active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100389399C (en) * | 2006-11-20 | 2008-05-21 | 北京中星微电子有限公司 | Method and system of chip checking |
| CN102402628A (en) * | 2010-09-07 | 2012-04-04 | 无锡中星微电子有限公司 | Method and system for generating systems-on-a-chip (SoC) verification platform |
| CN104978249A (en) * | 2014-04-09 | 2015-10-14 | 三星电子株式会社 | System On Chip And Verification Method Thereof |
| CN104486169A (en) * | 2015-01-07 | 2015-04-01 | 北京华力创通科技股份有限公司 | Reusable automatic detection and random verification system and method |
| US20190096659A1 (en) * | 2017-09-26 | 2019-03-28 | Samsung Electronics Co., Ltd. | Systems and methods for wafer map analysis |
| CN109086199A (en) * | 2018-06-22 | 2018-12-25 | 网宿科技股份有限公司 | A kind of automation generates the method for test script, terminal and can storage medium |
| US20200042419A1 (en) * | 2018-07-31 | 2020-02-06 | Alibaba Group Holding Limited | System and method for benchmarking ai hardware using synthetic ai model |
| CN109634843A (en) * | 2018-10-31 | 2019-04-16 | 中国科学院软件研究所 | A kind of distributed automatization method for testing software and platform towards AI chip platform |
| CN110489344A (en) * | 2019-08-02 | 2019-11-22 | Oppo广东移动通信有限公司 | Engine test method and Related product |
| CN111045880A (en) * | 2019-12-17 | 2020-04-21 | 湖南长城银河科技有限公司 | Chip testing method, verification system and storage medium |
| CN111178512A (en) * | 2019-12-31 | 2020-05-19 | 中国科学院自动化研究所南京人工智能芯片创新研究院 | Device operation neural network test method and device |
Non-Patent Citations (2)
| Title |
|---|
| 牛玉坤等: "基于UVM实现SD存储控制器的功能验证", 《工业控制计算机》, no. 08, 25 August 2017 (2017-08-25), pages 23 - 24 * |
| 耿言等: "C编译器自动测试仿真研究", 《计算机仿真》, no. 10, 30 October 2003 (2003-10-30), pages 141 - 144 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114912413A (en) * | 2022-04-29 | 2022-08-16 | 上海阵量智能科技有限公司 | Chip verification method and platform |
| WO2023207965A1 (en) * | 2022-04-29 | 2023-11-02 | 上海商汤智能科技有限公司 | Chip verification method and platform |
| CN114996076A (en) * | 2022-07-01 | 2022-09-02 | 北京联盛德微电子有限责任公司 | Traversal type use case verification method and system for chip simulation and electronic equipment |
| CN114996076B (en) * | 2022-07-01 | 2023-02-07 | 北京联盛德微电子有限责任公司 | Traversal type use case verification method and system for chip simulation and electronic equipment |
| CN115495388A (en) * | 2022-11-17 | 2022-12-20 | 瀚博半导体(上海)有限公司 | Chip verification method, device, equipment and medium for AI inference chip |
| CN115792580A (en) * | 2023-01-30 | 2023-03-14 | 北京怀美科技有限公司 | Chip detection system and editable chip detection system |
| CN116431421A (en) * | 2023-06-13 | 2023-07-14 | 成都登临科技有限公司 | Neural network generation method and generator, verification method and system and storage medium |
| CN116431421B (en) * | 2023-06-13 | 2023-08-29 | 成都登临科技有限公司 | Neural network generation method and generator, verification method and system and storage medium |
| CN117493168A (en) * | 2023-09-28 | 2024-02-02 | 中国长江电力股份有限公司 | Water conservancy model calling method capable of conducting offline testing |
| CN117743194A (en) * | 2024-02-19 | 2024-03-22 | 睿思芯科(深圳)技术有限公司 | Automatic generation method, system and related equipment for chip verification use cases |
| CN117743194B (en) * | 2024-02-19 | 2024-05-07 | 睿思芯科(深圳)技术有限公司 | Automatic generation method, system and related equipment for chip verification use cases |
Also Published As
| Publication number | Publication date |
|---|---|
| TWI778473B (en) | 2022-09-21 |
| WO2021238006A1 (en) | 2021-12-02 |
| JP2022537620A (en) | 2022-08-29 |
| TW202145046A (en) | 2021-12-01 |
| KR20210149045A (en) | 2021-12-08 |
| JP7270764B2 (en) | 2023-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113742202A (en) | AI chip verification system, method, device and storage medium | |
| US6816814B2 (en) | Method and apparatus for decomposing and verifying configurable hardware | |
| CN107066375B (en) | System and method for generating automatic demand-based test case of safety-critical software | |
| US8589521B2 (en) | Method for testing connectivity of software applications hosted on networked computers | |
| GB2587432A (en) | System and method for software architecture redesign | |
| CN111782207A (en) | Method, device and equipment for generating task stream code and storage medium | |
| US11960385B2 (en) | Automatic generation of integrated test procedures using system test procedures | |
| US20050203717A1 (en) | Automated testing system, method and program product using testing map | |
| US6970816B1 (en) | Method and system for efficiently generating parameterized bus transactions | |
| CN112860587A (en) | UI automatic test method and device | |
| US7739671B1 (en) | Systems, methods and apparatus for implementation of formal specifications derived from informal requirements | |
| CN116521217A (en) | Method, system and storage medium for rapidly configuring BSW based on AUTOSAR tool | |
| CN116450431A (en) | Instruction function test system of CPU reference model, method thereof, computer equipment and storage medium | |
| US20120167037A1 (en) | Software static testing apparatus and method | |
| CN116992805A (en) | Chip verification system, method and verification equipment based on generation type constraint model | |
| CN115562931A (en) | Processor debugging module verification method and device, electronic equipment and storage medium | |
| CN113934619A (en) | Sound source testing method, system and storage medium based on Robot Framework testing Framework | |
| EP2782005A1 (en) | Verifying state reachability in a statechart model having computer program code embedded therein | |
| CN113495826A (en) | Generation method of unit test code, unit test method and device | |
| CN117331565B (en) | Software generation method, device, computer equipment and storage medium | |
| CN115409194B (en) | Quantum program generation system and quantum computer operation system | |
| CN117056896B (en) | Intelligent control system form verification method and device | |
| CN110955423B (en) | Model design method, system, storage medium and terminal | |
| CN118035120A (en) | Test case generation method, device, equipment and storage medium | |
| CN116048902A (en) | Back-end testing method, device, storage medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40059121 Country of ref document: HK |