CN115981666B - Neural network information integration method, device, system and storage medium - Google Patents
Neural network information integration method, device, system and storage medium Download PDFInfo
- Publication number
- CN115981666B CN115981666B CN202310275238.4A CN202310275238A CN115981666B CN 115981666 B CN115981666 B CN 115981666B CN 202310275238 A CN202310275238 A CN 202310275238A CN 115981666 B CN115981666 B CN 115981666B
- Authority
- CN
- China
- Prior art keywords
- network
- information
- neural network
- iriff
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 207
- 238000000034 method Methods 0.000 title claims abstract description 54
- 230000010354 integration Effects 0.000 title claims abstract description 44
- 238000003860 storage Methods 0.000 title claims abstract description 20
- 238000004806 packaging method and process Methods 0.000 claims description 24
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000012856 packing Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 2
- 238000012858 packaging process Methods 0.000 claims 2
- 238000012545 processing Methods 0.000 abstract description 13
- 238000010586 diagram Methods 0.000 abstract description 9
- 230000006870 function Effects 0.000 description 14
- 238000004891 communication Methods 0.000 description 5
- 238000001514 detection method Methods 0.000 description 5
- 239000002243 precursor Substances 0.000 description 4
- 210000002569 neuron Anatomy 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000005236 sound signal Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000010897 surface acoustic wave method Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a neural network information integration method, device, system and storage medium. The invention can compile the original data of the preset neural network, output the grading information of the preset neural network, wherein the grading information comprises a characteristic diagram, a network layer, a network segment, a subnet and a network of the preset neural network, extract the neural network characteristic parameters corresponding to the grading information, assemble the neural network characteristic parameters, pack the grading information in sequence, create the header information of the IRIFF file, and write the packed data of the grading information in sequence after the header information to obtain the IRIFF file. According to the embodiment of the application, the characteristic parameters of the neural network can be integrated into the IRIFF file, so that the characteristic parameters can be directly called through the memory in the subsequent use, and the processing efficiency of the neural network is effectively improved.
Description
Technical Field
The invention relates to the field of neural network processing, in particular to a neural network information integration method, device, system and storage medium.
Background
With the development of artificial intelligence technology, neural networks are applied to more and more fields, and improvement is brought to the production and life of people. The neural network is a neural network learning algorithm, and is a hierarchical neural network consisting of an input layer, a middle layer and an output layer, wherein the middle layer can be expanded into multiple layers. The neurons of adjacent layers are fully connected, the neurons of each layer are not connected, the network learns in a teacher teaching mode, and after a pair of learning modes are provided for the network, the neurons obtain the input response of the network to generate a connection Weight (Weight). And then correcting each connection weight layer by layer from the output layer through each intermediate layer according to the direction of reducing the error between the expected output and the actual output, and returning to the input layer. The process is repeatedly and alternately performed until the global error of the network tends to a given minimum value, namely the learning process is completed.
In the prior art, neural networks are of a wide variety, such as BP (Back Propagation) neural networks, radial basis function (RBF-Radial Basis Function) neural networks, sensor neural networks, linear neural networks, ad hoc neural networks, and feedback neural networks, among others. However, the applicant found that each neural network contains various information parameters, such as a network map, auxiliary information, weight data, NPU instruction codes, other relevant information generated by a compiler, etc., which need to be called by a computing platform when the neural network is used, and memory resources in a common computing platform are extremely limited and do not support a file system, so that the parameters can only be called from external memory, resulting in low processing efficiency of the neural network.
Disclosure of Invention
The invention provides a neural network information integration method, a device, a system and a storage medium, which can integrate characteristic parameters of a neural network and directly call the characteristic parameters through a memory when the characteristic parameters are used later, so that the processing efficiency of the neural network is improved.
In order to achieve the beneficial effects, the embodiment of the invention provides the following technical scheme:
in a first aspect, a method for integrating information of a neural network is provided, where the method includes:
Compiling original data of a preset neural network, and outputting grading information of the preset neural network, wherein the grading information comprises a feature map, a network layer, a network segment, a subnet and a network of the preset neural network;
extracting neural network characteristic parameters corresponding to the grading information;
compiling the characteristic parameters of the neural network, and sequentially packaging the grading information;
and creating header information of the IRIFF file, and sequentially writing the packaged data of the grading information after the header information to obtain the IRIFF file.
In a second aspect, please provide a neural network information integration device, comprising:
the compiling unit is used for compiling original data of a preset neural network and outputting grading information of the preset neural network, wherein the grading information comprises a feature map, a network layer, a network segment, a subnet and a network of the preset neural network;
the extraction unit is used for extracting the neural network characteristic parameters corresponding to the grading information;
the packaging unit is used for compiling the characteristic parameters of the neural network and sequentially packaging the grading information;
the creating unit is used for creating the header information of the IRIFF file and writing the packaged data of the grading information after the header information in sequence to obtain the IRIFF file.
In a third aspect, please provide a neural network information integration system, comprising: a computing module and a nonvolatile memory;
the computing module is used for compiling original data of a preset neural network, outputting grading information of the preset neural network, wherein the grading information comprises a feature map, a network layer, a network segment, a subnet and a network of the preset neural network, extracting neural network feature parameters corresponding to the grading information, compiling the neural network feature parameters, sequentially packaging the grading information, creating head information of an IRIFF file, sequentially writing packaged data of the grading information after the head information to obtain the IRIFF file, and storing the IRIFF file into the nonvolatile memory.
In a fourth aspect, a storage medium is provided, where the storage medium stores a plurality of instructions adapted to be loaded by a processor to perform the steps in the neural network information integration method.
According to the embodiment provided by the application, the original data of the preset neural network can be compiled, the grading information of the preset neural network is output, the grading information comprises the characteristic diagram, the network layer, the network segment, the sub-network and the network of the preset neural network, the neural network characteristic parameters corresponding to the grading information are extracted, the neural network characteristic parameters are compiled, the grading information is packaged in sequence, the head information of the IRIFF file is created, and the packaged data of the grading information are written in sequence after the head information, so that the IRIFF file is obtained. According to the embodiment of the application, the characteristic parameters of the neural network can be integrated into the IRIFF file, so that the characteristic parameters can be directly called through the memory in the subsequent use, and the processing efficiency of the neural network is effectively improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a neural network information integration method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a hardware platform according to an embodiment of the present invention;
fig. 3 is another flow chart of a neural network information integration method according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a neural network information integrating device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of another neural network information integrating device according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an intelligent terminal according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
The embodiment of the invention provides a neural network information integration method, and an execution subject of the neural network information integration method can be the neural network information integration device provided by the embodiment of the invention or electronic equipment integrated with the neural network information integration device, wherein the neural network information integration device can be realized in a hardware or software mode.
In this embodiment, description will be made from the perspective of a neural network information integration apparatus, which may be specifically an electronic device, and the electronic device is provided with a storage unit and is capable of running an application program.
A neural network information integration method, the method comprising:
Compiling original data of a preset neural network, and outputting grading information of the preset neural network, wherein the grading information comprises a feature map, a network layer, a network segment, a subnet and a network of the preset neural network;
extracting neural network characteristic parameters corresponding to the grading information;
compiling the characteristic parameters of the neural network, and sequentially packaging the grading information;
and creating header information of the IRIFF file, and sequentially writing the packaged data of the grading information after the header information to obtain the IRIFF file.
Referring to fig. 1, fig. 1 is a flowchart of a neural network information integration method according to an embodiment of the invention. The neural network information integration method comprises the following steps:
and step 101, compiling original data of a preset neural network, and outputting grading information of the preset neural network.
In an embodiment, the neural network information integration method provided in the embodiment of the present application may be applied to a hardware platform as shown in fig. 2, where the hardware platform may include a memory and a plurality of computing modules, where the memory may include a low-speed, nonvolatile memory module, device or equivalent. Such as flash, mechanical hard disk, etc., and may even be a remote file, etc.
In an embodiment, the computing module includes a main control module, at least one computing core, and a Memory, where the Memory may be an SRAM (static random-Access Memory) or a DDR SDRAM (DoubleData Rate Synchronous Dynamic Random Access Memory ), the main control module Host may be a CPU, and the at least one computing core may be a DSP (Digital Signal Processing, digital signal processor), an NPU (Neural-network ProcessingUnit, embedded Neural network processor), a GPU (graphicsprocessing unit, graphics processor), and the like.
It should be noted that, the computing modules may be physically fixed, or may be dynamically combined as required, and the memory of each computing module may be independently addressed, or may be uniformly addressed with the memory of the other computing module or modules. In one embodiment, the computing cores are divided into two types: i.e. automatically and continuously reading the command (which can be an instruction or a configuration parameter) sequence, decoding and executing the command, the command is called an active core; otherwise referred to as a passive core. Wherein each active core within each compute module needs to be numbered, such as CPU, NPU_0, NPU_1, etc. as shown in FIG. 2. Multiple independent computing modules may be running simultaneously, and each computing module may compute one or more neural networks, as this application is not further limited.
In an embodiment, the preset neural network may be finely divided from small to large according to the internal structure, and specifically may include a feature map, a network layer, a network segment, a subnet, and a network. Specifically, the original data of the preset neural network can be compiled, and the grading information of the preset neural network, namely the characteristic map, the network layer, the network segment, the sub-network and the network can be output. Wherein, feature map (abbreviated as fm): i.e. 3-dimensional tensor, 3 latitudes are generally indicated by H, W, C, and the write together is HWC. The data type does not include latitude, and is independently represented by B.
Network layer (layer): is a basic unit, also called layer for short, of the neural network after being preprocessed by the neural network compiler. The difference from what is known as the neural network layer (the original layer for short): which may be a split portion of a given original layer or a fusion of multiple successive original layers. The input is tensor, and the output is usually a feature map. The network layer is further subdivided into two types: one is a layer which can generate a command sequence after compiling through a neural network tool chain, and the command sequence can be executed (calculated) by a certain active core and obtain an output characteristic diagram, which is called a CMD network layer; the rest is called as RAW network layer, namely the original information of the network layer needs to be reserved, and the actual calculation mode and process are determined by a Host during calculation.
Segment (segment): i.e. one or a succession of network layers that fulfill a certain condition. Two categories are also known: can run continuously on a certain active core without switching, and is called CMD network segment; otherwise referred to as a RAW network segment. It should be noted that, during debugging or testing, it is also possible to use each CMD network layer individually as one CMD network segment.
Subnet (subnet): i.e. one or a plurality of consecutive segments with the same calculation frequency in each round of calculation (corresponding to one input feature map (abbreviated as fi)). That is, there is no branching, or looping, in the middle caused by the conditional determination.
Network (net): one or more subnets grouped together by logical relationships. The common various neural networks all contain only one subnet. The calculation results of the layers of the network are called an intermediate feature map. The intermediate feature maps can be divided into two categories: a feature map called static (static) that needs to be saved and participates in a subsequent round of computation; the rest (i.e., not involved in subsequent rounds of computation) is referred to as a local (local) feature map.
In an embodiment, a higher level concept, such as a group (group), may be further defined, and may be formed by a plurality of networks that are logically combined together.
Furthermore, the compiling process can be processed by a neural network compiler, and specifically, various neural network original files or data, such as a neural network overall structure description, details of each original layer, quantization modes of each original layer, weight parameters obtained by training, and the like, can be input. And then outputting the grading information of the network, and outputting the quantized weight parameter data, the reference feature map file, the command sequence, other member information required by the IRIFF and the like.
In the process of compiling, compiling options including compiling scope, compiling time optimization level, and the like may be further set, such as determining whether to compile all original layers or only part of the original layers, or whether to fully compile or compile but ignore reference feature maps, and the like.
And 102, extracting the neural network characteristic parameters corresponding to the grading information.
Further, the neural network characteristic parameters corresponding to the grading information, such as the command sequence, are extracted to be used as the input of the subsequent compiling step, wherein the precursor of the command sequence can comprise the following modes: c code, assembly code, and configuration parameter sequences in text format. The assembly code can be, for example, an NPU assembly in json format, and the text format can be json.
And 103, compiling the characteristic parameters of the neural network, and sequentially packaging the grading information.
In one embodiment, the process of creating the IRIFF file includes inputting a neural network list (such as netlist. Json) and outputting the IRIFF file by a link option, and the software for completing the process is a neural network linker. The neural network characteristic parameters can be compiled by the active core assembler, in particular, the precursors of the command sequences can be compiled into corresponding binary data sequences (i.e. command sequences) corresponding to the precursors of the command sequences, and therefore the following types of the precursors can be also adopted: c code, assembly code, and a sequence of parameters in text format. The C code is assembled by GCC (GNU CompilerCollection, GNU compiler kit) or LLVM (Low Level Virtual Machine, underlying virtual machine), and the assembly code is assembled by an assembler in a general sense, and the parameter sequences in a text format are assembled by a special text-to-binary converter.
In an embodiment, when the hierarchical information is packaged in sequence, the network information of each level may be packaged in sequence according to the sequence of the feature map, the network layer, the network segment, the subnet, and the network. The above feature map, network layer, network segment and subnet all need information from the original data, compiler output and assembler output, and no additional network information is involved. And the network required information may come from raw data, compiler output, assembler output, additional network information, compilation options, and the like. Wherein the additional network information is a logical relationship between sub-networks or a control parameter that the logical relationship needs to use when a network includes a plurality of sub-networks, and the like. Thus when a network contains multiple subnets, the packing process of the network may include: extracting logical relations and control parameters among a plurality of sub-networks to generate additional network information, acquiring original network information of a network, and packaging the original network information and the additional network information.
In an embodiment, when the network information of the feature map, the network layer, the network segment, the subnet and the network is packed, the container header can be created first and the data can be refilled, and specifically, the output from the neural network compiler or the output of the active core assembler can be filled.
And 104, creating header information of the IRIFF file, and sequentially writing packaged data of the grading information after the header information to obtain the IRIFF file.
Specifically, the information such as the bit width of the members such as the base address and the pointer contained in the file container can be defined, the number of networks in the preset neural network can be obtained, then the header of the IRIFF file container is defined, the data part after the header information is filled with fields, namely the packed data of the classification information is filled, and the lengths of the front part of the file container are aligned as required, so that the IRIFF file container is obtained. Wherein, when creating the IRIFF file, the packing can be performed by adopting a page assignment method based on the object.
The related parameter information and data of each neural network may be stored in the IRIFF file, for example, 3 chunks from left to right (corresponding file, i.e. from head to tail direction) in sequence: the 4 th block, namely the reference feature map, can be added during debugging, and the network detail information, the weight parameter, the command sequence and the reference feature map are continuously stored and are connected end to end.
Further, the IRIFF total BNF is as follows:
in an embodiment, the neural network characteristic parameters in the at least one container may include network detail information, weight parameters, command sequences and the like in all the neural networks.
In an embodiment, when the IRIFF file is manufactured, a reference feature map and an array may be further manufactured and added, where the reference feature map array may be added to reference feature maps of all layers, or may be added to only part of layers, and the reference feature maps may be described by taking only part of layers as an example, and may traverse the whole network, count input and output of the RAW layer one by one, add input and output of the whole network, remove repeated data therein, and add the remaining feature maps corresponding to the input and output to the IRIFF.
It can be seen from the foregoing that, the neural network information integration method provided by the embodiment of the present application may compile the original data of the preset neural network, output the classification information of the preset neural network, where the classification information includes a feature map, a network layer, a network segment, a subnet, and a network of the preset neural network, extract the neural network feature parameters corresponding to the classification information, assemble the neural network feature parameters, package the classification information in sequence, create header information of the IRIFF file, and write the packaged data of the classification information in sequence after the header information, so as to obtain the IRIFF file. According to the embodiment of the application, the characteristic parameters of the neural network can be integrated into the IRIFF file, so that the characteristic parameters can be directly called through the memory in the subsequent use, and the processing efficiency of the neural network is effectively improved.
According to the neural network information integration method described in the above embodiment, further details will be described below by way of example.
In this embodiment, the neural network information integrating device is specifically integrated in an intelligent terminal will be described as an example.
Referring to fig. 3, fig. 3 is another flow chart of the neural network information integration method according to the embodiment of the invention. The method flow may include:
step 201, compiling original data of a preset neural network, and outputting grading information of the preset neural network.
The grading information may include a feature map, a network layer, a network segment, a subnet, and a network of the preset neural network.
Step 202, extracting neural network characteristic parameters corresponding to the grading information.
And 203, initializing the allocable memory address and the length information, and compiling the characteristic parameters of the neural network.
Step 204, sequentially packaging the hierarchical information, and updating the memory address and the length information according to the packaged data.
The method comprises the steps of reading, compiling, assembling, packaging and updating, wherein the head address and length information of the memory and the external memory which can be allocated can be updated according to the relation among the networks in the network group, and if the related information of a plurality of networks exists, the steps of reading, compiling, assembling, packaging and updating can be sequentially and repeatedly executed.
Step 205, creating header information of the IRIFF file, and sequentially writing the packaged data of the hierarchical information after the header information to obtain the IRIFF file.
Step 206, storing the IRIFF file in the nonvolatile memory.
Step 207, when the preset neural network is operated, copying the network characteristic parameters in the IRIFF file into the memory of the platform, and calling the network characteristic parameters by calling the computing module of the platform in a memory access mode.
In an embodiment, the step of copying the network feature parameters in the IRIFF file to the memory of the platform may include: and analyzing the header information of the IRIFF file to obtain names corresponding to the characteristic map, the network layer, the network segment, the subnet and the network of the preset neural network respectively, sequentially acquiring the characteristic parameters of the neural network of the characteristic map, the network layer, the network segment, the subnet and the network according to the IRIFF format and the names, and copying the characteristic parameters into the memory of the platform.
In one embodiment, if the memory space is small during the operation of the computing module, and therefore, all network characteristic parameters in the IRIFF file cannot be copied into the memory, a part of network characteristic parameters, that is, target parameters, can be preferentially copied according to the remaining memory values. For example, network detail information of the neural network is copied into a memory, then a flash read weight parameter or a command sequence is required to be accessed into the memory in a specific calculation process of each CMD network segment, and then a calculation core in a calculation module can start calculation.
In another embodiment, if the memory is sufficient, a section of the preset memory area may be divided from the memory of the platform in advance, then the IRIFF file is loaded into the preset memory area, and the computing module of the platform is called to call the network feature parameter by accessing the memory. For example, during debugging, the IRIFF file is directly loaded into a certain area of the memory through the back door. Then the calculation module starts to operate, the process of data copying is omitted, and simulation debugging can be greatly accelerated.
It should be noted that, the nonvolatile memory is, for example, a flash, and the IRIFF file is stored in the nonvolatile memory. After power-on, a certain main control module Host can copy the information of the neural network into the memory of the module 2 according to all the information, wherein the main control module Host is not necessarily the CPU of the module, and is specifically determined according to the design of a hardware platform. And then the computing module starts to operate, and the specific computing process of each layer/each network segment only accesses the memory without accessing the flash.
It can be seen from the foregoing that, the neural network information integration method provided by the embodiment of the present application may compile original data of a preset neural network, output hierarchical information of the preset neural network, extract a neural network feature parameter corresponding to the hierarchical information, initialize an assignable memory address and length information, assemble the neural network feature parameter, sequentially package the hierarchical information, update the memory address and the length information according to the packaged data, create header information of an IRIFF file, sequentially write the header information into the packaged data of the hierarchical information after the header information, so as to obtain the IRIFF file, store the IRIFF file in a nonvolatile memory, copy the network feature parameter in the IRIFF file into a memory of a platform when the preset neural network is operated, and call the network feature parameter by calling a computing module of the platform through accessing the memory. According to the embodiment of the application, the characteristic parameters of the neural network can be integrated into the IRIFF file and copied into the memory of the platform, so that the characteristic parameters can be directly called through the memory in the subsequent use, and the processing efficiency of the neural network is effectively improved.
The application also provides a neural network information integration system, which specifically can comprise a computing module and a nonvolatile memory;
the computing module is used for compiling original data of a preset neural network, outputting grading information of the preset neural network, wherein the grading information comprises a feature map, a network layer, a network segment, a subnet and a network of the preset neural network, extracting neural network feature parameters corresponding to the grading information, compiling the neural network feature parameters, sequentially packaging the grading information, creating head information of an IRIFF file, sequentially writing packaged data of the grading information after the head information to obtain the IRIFF file, and storing the IRIFF file into the nonvolatile memory.
In order to facilitate better implementation of the neural network information integration method provided by the embodiment of the invention, the embodiment of the invention also provides a device based on the neural network information integration method. The meaning of the nouns is the same as that in the neural network information integration method, and specific implementation details can be referred to the description in the method embodiment.
In this embodiment, description will be made in terms of a neural network information integration apparatus, which may be specifically integrated in a system composed of a plurality of intelligent terminals, each of which is an intelligent terminal having a video playing function in a state where a storage unit is provided and a display screen is mounted.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a neural network information integration device 300 according to an embodiment of the invention. The neural network information integration apparatus 300 may include:
the compiling unit 301 is configured to compile original data of a preset neural network, and output classification information of the preset neural network, where the classification information includes a feature map, a network layer, a network segment, a subnet, and a network of the preset neural network;
an extracting unit 302, configured to extract a neural network characteristic parameter corresponding to the classification information;
a packing unit 303, configured to assemble the neural network characteristic parameters, and pack the hierarchical information in sequence;
the creating unit 304 is configured to create header information of the IRIFF file, and sequentially write the packed data of the hierarchical information after the header information, so as to obtain the IRIFF file.
In an embodiment, referring to fig. 5, fig. 5 is a schematic diagram illustrating another structure of a neural network information integration apparatus 300 according to an embodiment of the invention. The neural network information integration apparatus 300 may further include:
an initializing unit 305, configured to initialize an allocatable memory address and length information before the compiling unit 301 compiles the original data of the preset neural network;
And an updating unit 306, configured to update the memory address and the length information according to the packed data after the packing unit 303 packs the hierarchical information in turn.
In an embodiment, when the network includes a plurality of subnets, the packing unit 303 may specifically include:
an extracting subunit 3031, configured to extract the logic relationships and the control parameters between the multiple subnets to generate additional network information;
the wrapping subunit 3032 is configured to obtain the original network information of the network, and package the original network information and the additional network information.
As can be seen from the foregoing, the embodiment of the present invention may compile original data of a preset neural network, output classification information of the preset neural network, where the classification information includes a feature map, a network layer, a network segment, a subnet, and a network of the preset neural network, extract neural network feature parameters corresponding to the classification information, assemble the neural network feature parameters, package the classification information in turn, create header information of an IRIFF file, and write package data of the classification information in turn after the header information to obtain the IRIFF file. According to the embodiment of the application, the characteristic parameters of the neural network can be integrated into the IRIFF file, so that the characteristic parameters can be directly called through the memory in the subsequent use, and the processing efficiency of the neural network is effectively improved.
The embodiment of the present invention further provides an intelligent terminal 600, as shown in fig. 6, where the intelligent terminal 600 may integrate the above neural network information integration device, and may further include a Radio Frequency (RF) circuit 601, a memory 602 including one or more computer readable storage media, an input unit 603, a display unit 604, a sensor 605, an audio circuit 606, a wireless fidelity (WiFi, wireless Fidelity) module 607, a processor 608 including one or more processing cores, and a power supply 609. It will be appreciated by those skilled in the art that the configuration of the intelligent terminal 600 shown in fig. 6 is not limiting of the intelligent terminal 600, and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components. Wherein:
the RF circuit 601 may be used for receiving and transmitting signals during a message or a call, and in particular, after receiving downlink information of a base station, the downlink information is processed by one or more processors 608; in addition, data relating to uplink is transmitted to the base station. Typically, RF circuitry 601 includes, but is not limited to, an antenna, at least one amplifier, a tuner, one or more oscillators, a subscriber identity Module (SIM, subscriberIdentity Module) card, a transceiver, a coupler, a low noise amplifier (LNA, low Noise Amplifier), a duplexer, and the like. In addition, the RF circuitry 601 may also communicate with networks and other devices through wireless communications. The wireless communication may use any communication standard or protocol including, but not limited to, global system for mobile communications (GSM, global Systemof Mobile communication), universal packet Radio Service (GPRS, generalPacket Radio Service), code division multiple access (CDMA, code DivisionMultiple Access), wideband code division multiple access (WCDMA, wideband CodeDivision Multiple Access), long term evolution (LTE, long Term Evolution), email, short message Service (SMS, short Messaging Service), and the like.
The memory 602 may be used to store software programs and modules, and the processor 608 may execute various functional applications and information processing by executing the software programs and modules stored in the memory 602. The memory 602 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playing function, a target data playing function, etc.), and the like; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the intelligent terminal 600, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide access to the memory 602 by the processor 608 and the input unit 603.
The input unit 603 may be used to receive input numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. In particular, in one particular embodiment, the input unit 603 may include a touch-sensitive surface, as well as other input devices. The touch-sensitive surface, also referred to as a touch display screen or a touch pad, may collect touch operations thereon or thereabout by a user (e.g., operations thereon or thereabout by a user using any suitable object or accessory such as a finger, stylus, etc.), and actuate the corresponding connection means according to a predetermined program. Alternatively, the touch-sensitive surface may comprise two parts, a touch detection device and a touch controller. The touch detection device detects the touch azimuth of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 608, and can receive commands from the processor 608 and execute them. In addition, touch sensitive surfaces may be implemented in a variety of types, such as resistive, capacitive, infrared, and surface acoustic waves. The input unit 603 may comprise other input devices in addition to a touch sensitive surface. In particular, other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, mouse, joystick, etc.
The display unit 604 may be used to display information input by a user or information provided to the user and various graphical user interfaces of the intelligent terminal 600, which may be composed of graphics, text, icons, video, and any combination thereof. The display unit 604 may include a display panel, which may be optionally configured in the form of a liquid crystal display (LCD, liquid Crystal Display), an Organic Light-emitting diode (OLED), or the like. Further, the touch-sensitive surface may overlay a display panel, and upon detection of a touch operation thereon or thereabout, the touch-sensitive surface is passed to the processor 608 to determine the type of touch event, and the processor 608 then provides a corresponding visual output on the display panel based on the type of touch event. Although in fig. 6 the touch sensitive surface and the display panel are implemented as two separate components for input and output functions, in some embodiments the touch sensitive surface may be integrated with the display panel to implement the input and output functions.
The intelligent terminal 600 may also include at least one sensor 605, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor that may adjust the brightness of the display panel according to the brightness of ambient light, and a proximity sensor that may turn off the display panel and/or the backlight when the smart terminal 600 moves to the ear. As one of the motion sensors, the gravitational acceleration sensor may detect the acceleration in each direction (generally, three axes), and may detect the gravity and direction when stationary, and may be used in applications for recognizing the gesture of a mobile phone (such as horizontal/vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer, and knocking), and other sensors such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, which may be further configured in the intelligent terminal 600, will not be described herein.
Audio circuitry 606, speakers, and a microphone may provide an audio interface between the user and the intelligent terminal 600. The audio circuit 606 may transmit the received electrical signal after audio data conversion to a speaker, where the electrical signal is converted to a sound signal for output; on the other hand, the microphone converts the collected sound signals into electrical signals, which are received by the audio circuit 606 and converted into audio data, which are processed by the audio data output processor 608 for transmission via the RF circuit 601 to, for example, another intelligent terminal 600, or which are output to the memory 602 for further processing. The audio circuit 606 may also include an ear bud jack to provide communication of the peripheral headphones with the smart terminal 600.
The WiFi belongs to a short-distance wireless transmission technology, and the intelligent terminal 600 can help the user to send and receive e-mail, browse web pages, access streaming media and the like through the WiFi module 607, and provides wireless broadband internet access for the user. Although fig. 6 shows a WiFi module 607, it is understood that it does not belong to the essential constitution of the intelligent terminal 600, and can be omitted entirely as required within the scope of not changing the essence of the invention.
The processor 608 is a control center of the intelligent terminal 600, connects various parts of the entire mobile phone using various interfaces and lines, and performs various functions of the intelligent terminal 600 and processes data by running or executing software programs and/or modules stored in the memory 602 and calling data stored in the memory 602, thereby performing overall monitoring of the mobile phone. Optionally, the processor 608 may include one or more processing cores; preferably, the processor 608 may integrate an application processor that primarily handles operating systems, user interfaces, applications, etc., with a modem processor that primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 608.
The intelligent terminal 600 also includes a power source 609 (e.g., a battery) for powering the various components, which may preferably be logically connected to the processor 608 via a power management system so as to perform functions such as managing charging, discharging, and power consumption via the power management system. The power supply 609 may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power data indicator, and the like.
Although not shown, the intelligent terminal 600 may further include a camera, a bluetooth module, etc., which will not be described herein. In this embodiment, the processor 608 in the intelligent terminal 600 loads executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 608 executes the application programs stored in the memory 602, so as to implement various functions:
compiling original data of a preset neural network, and outputting grading information of the preset neural network, wherein the grading information comprises a feature map, a network layer, a network segment, a subnet and a network of the preset neural network;
extracting neural network characteristic parameters corresponding to the grading information;
Compiling the characteristic parameters of the neural network, and sequentially packaging the grading information;
and creating header information of the IRIFF file, and sequentially writing the packaged data of the grading information after the header information to obtain the IRIFF file.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and the parts of an embodiment that are not described in detail may be referred to the detailed description of the neural network information integration method, which is not described herein.
As can be seen from the foregoing, the intelligent terminal 600 according to the embodiment of the present invention may compile the original data of the preset neural network, output the classification information of the preset neural network, where the classification information includes the feature map, the network layer, the network segment, the subnet and the network of the preset neural network, extract the neural network feature parameters corresponding to the classification information, assemble the neural network feature parameters, package the classification information in sequence, create the header information of the IRIFF file, and write the package data of the classification information in sequence after the header information to obtain the IRIFF file. According to the embodiment of the application, the characteristic parameters of the neural network can be integrated into the IRIFF file, so that the characteristic parameters can be directly called through the memory in the subsequent use, and the processing efficiency of the neural network is effectively improved.
Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.
To this end, the embodiments of the present application further provide a storage medium having stored thereon a plurality of instructions adapted to be loaded by a processor to perform the steps in the above-described neural network information integration method.
The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.
Wherein the storage medium may include: read Only Memory (ROM), random access memory (RAM, random AccessMemory), magnetic or optical disk, and the like.
The instructions stored in the storage medium can execute the steps in any neural network information integration method provided by the embodiment of the present invention, so that the beneficial effects that any neural network information integration method provided by the embodiment of the present invention can be achieved, and detailed descriptions of the previous embodiments are omitted.
The neural network information integration method, device, system and storage medium provided by the embodiments of the present invention are described in detail, and specific examples are applied to illustrate the principles and embodiments of the present invention, and the description of the above embodiments is only used to help understand the method and core idea of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present invention, the present description should not be construed as limiting the present invention.
Claims (6)
1. A method for integrating information of a neural network, the method comprising:
compiling original data of a preset neural network, and outputting grading information of the preset neural network, wherein the grading information comprises a feature map, a network layer, a network segment, a subnet and a network of the preset neural network;
extracting neural network characteristic parameters corresponding to the grading information;
compiling the characteristic parameters of the neural network and sequentially packing the grading information, wherein when the network comprises a plurality of sub-networks, the packing process of the network comprises the following steps: extracting logical relations and control parameters among the plurality of subnets to generate additional network information, acquiring original network information of the network, packaging the original network information and the additional network information, packaging user data information, classifying the user data information according to data value types, arranging character strings of the user data information in classification results, quantifying the value according to arrangement results, and packaging the quantified value and the original key into a custom array container and a custom structure container;
Creating header information of the IRIFF file, and sequentially writing the packaged data of the grading information after the header information to obtain the IRIFF file;
and storing the IRIFF file into a nonvolatile memory, analyzing header information of the IRIFF file when the preset neural network is operated to obtain names respectively corresponding to a feature map, a network layer, a network segment, a subnet and a network of the preset neural network, sequentially acquiring the respective neural network feature parameters of the feature map, the network layer, the network segment, the subnet and the network according to the IRIFF format and the names, copying the neural network feature parameters into a memory of a platform, and calling a calculation module of the platform to call the network feature parameters in a memory access mode.
2. The neural network information integration method according to claim 1, wherein before compiling the original data of the preset neural network, the method further comprises:
initializing an assignable memory address and length information;
and after the hierarchical information is packed in sequence, the memory address and the length information are updated according to packed data.
3. A neural network information integration apparatus, comprising:
The compiling unit is used for compiling original data of a preset neural network and outputting grading information of the preset neural network, wherein the grading information comprises a feature map, a network layer, a network segment, a subnet and a network of the preset neural network;
the extraction unit is used for extracting the neural network characteristic parameters corresponding to the grading information;
the packaging unit is used for compiling the characteristic parameters of the neural network and sequentially packaging the grading information, wherein when the network comprises a plurality of sub-networks, the packaging process of the network comprises the following steps: extracting logical relations and control parameters among the plurality of subnets to generate additional network information, acquiring original network information of the network, packaging the original network information and the additional network information, packaging user data information, classifying the user data information according to data value types, arranging character strings of the user data information in classification results, quantifying the value according to arrangement results, and packaging the quantified value and the original key into a custom array container and a custom structure container;
the creating unit is used for creating head information of the IRIFF file, writing the head information into packed data of the grading information in sequence after the head information to obtain the IRIFF file, storing the IRIFF file into a nonvolatile memory, analyzing the head information of the IRIFF file when the preset neural network is operated to obtain names respectively corresponding to a feature map, a network layer, a network segment, a subnet and a network of the preset neural network, sequentially acquiring the feature map, the network layer, the network segment, the subnet and the respective neural network feature parameters of the network according to the IRIFF format and the names, copying the feature parameters into a memory of a platform, and calling the network feature parameters by calling a computing module of the platform in a memory access mode.
4. The neural network information integration apparatus of claim 3, further comprising:
the initialization unit is used for initializing the allocable memory address and the length information before the compiling unit compiles the original data of the preset neural network;
and the updating unit is used for updating the memory address and the length information according to the packed data after the packing unit packs the grading information in sequence.
5. A neural network information integration system, comprising: a computing module and a nonvolatile memory;
the computing module is used for compiling original data of a preset neural network, outputting grading information of the preset neural network, wherein the grading information comprises a feature map, a network layer, a network segment, a subnet and a network of the preset neural network, extracting neural network feature parameters corresponding to the grading information, compiling the neural network feature parameters and sequentially packaging the grading information, and when the network comprises a plurality of subnets, the packaging process of the network comprises the following steps: extracting logical relations and control parameters among the plurality of subnets to generate additional network information, acquiring original network information of the network, packaging the original network information and the additional network information, packaging user data information, classifying the user data information according to data value types, arranging values in classification results according to character strings of the user data information, quantizing the values according to arrangement results, packaging the quantized values and the original keys into a custom array container and a custom structure container, creating head information of an IRIFF file, sequentially writing packaged data of the hierarchical information after the head information to obtain the IRIFF file, storing the IRIFF file into a nonvolatile memory, analyzing the head information of the IRIFF file when the preset neural network is operated, obtaining a characteristic map, a network layer, a network segment, a subnet and names corresponding to the network respectively of the preset neural network, sequentially acquiring the characteristic map, the network segment and the network name according to the IRIFF format and the names, calling the network segment and calling the network parameters, and accessing the characteristic parameters to a memory platform of the network in a memory platform in a mode of the network.
6. A storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the steps in the neural network information integration method of any one of claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310275238.4A CN115981666B (en) | 2023-03-21 | 2023-03-21 | Neural network information integration method, device, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310275238.4A CN115981666B (en) | 2023-03-21 | 2023-03-21 | Neural network information integration method, device, system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115981666A CN115981666A (en) | 2023-04-18 |
| CN115981666B true CN115981666B (en) | 2023-07-21 |
Family
ID=85976520
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310275238.4A Active CN115981666B (en) | 2023-03-21 | 2023-03-21 | Neural network information integration method, device, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115981666B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111104120A (en) * | 2018-10-29 | 2020-05-05 | 赛灵思公司 | Neural network compiling method and system and corresponding heterogeneous computing platform |
| CN111857723A (en) * | 2020-06-29 | 2020-10-30 | 浪潮电子信息产业股份有限公司 | Parameter compiling method and device and computer readable storage medium |
| CN114399019A (en) * | 2021-12-30 | 2022-04-26 | 南京风兴科技有限公司 | Neural network compiling method, system, computer device and storage medium |
| CN115113814A (en) * | 2022-06-21 | 2022-09-27 | 腾讯科技（深圳）有限公司 | Neural network model online method and related device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107239315B (en) * | 2017-04-11 | 2019-11-15 | 赛灵思公司 | Programming model towards neural network heterogeneous computing platforms |
-
2023
- 2023-03-21 CN CN202310275238.4A patent/CN115981666B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111104120A (en) * | 2018-10-29 | 2020-05-05 | 赛灵思公司 | Neural network compiling method and system and corresponding heterogeneous computing platform |
| CN111857723A (en) * | 2020-06-29 | 2020-10-30 | 浪潮电子信息产业股份有限公司 | Parameter compiling method and device and computer readable storage medium |
| CN114399019A (en) * | 2021-12-30 | 2022-04-26 | 南京风兴科技有限公司 | Neural network compiling method, system, computer device and storage medium |
| CN115113814A (en) * | 2022-06-21 | 2022-09-27 | 腾讯科技（深圳）有限公司 | Neural network model online method and related device |
Non-Patent Citations (1)
| Title |
|---|
| QingLong:一种基于常变量异步拷贝的神经网络编程模型;杜伟健;陈云霁;支天;吴林阳;陈小兵;庄毅敏;;计算机学报(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115981666A (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110515626B (en) | Code compiling method of deep learning calculation framework and related product | |
| CN110152299B (en) | Game resource construction method and device | |
| CN106502703B (en) | Function calling method and device | |
| CN110046497B (en) | Function hook realization method, device and storage medium | |
| CN114185491B (en) | Partition file downloading method and device, storage medium and computer equipment | |
| CN103763112B (en) | A kind of user identity protection method and apparatus | |
| CN106919458B (en) | Method and device for Hook target kernel function | |
| CN109933381A (en) | A kind of loading method and device of kernel | |
| CN105630846A (en) | Head portrait updating method and apparatus | |
| CN106202422A (en) | The treating method and apparatus of Web page icon | |
| CN106293738A (en) | The update method of a kind of facial expression image and device | |
| CN106502833A (en) | Data back up method and device | |
| CN115113814B (en) | Neural network model online method and related device | |
| CN108011687B (en) | Data processing method and device | |
| CN115981798B (en) | File analysis method, device, computer equipment and readable storage medium | |
| CN115982110B (en) | File running method, file running device, computer equipment and readable storage medium | |
| CN104102560A (en) | Method and device for testing system performance | |
| CN115981666B (en) | Neural network information integration method, device, system and storage medium | |
| CN111610977B (en) | Compiling method and related device | |
| CN106775745B (en) | Method and device for merging program codes | |
| CN104111864A (en) | Recording method, playback method, recording device and playback device for application operations | |
| CN112559532B (en) | Data insertion method and device based on red and black trees and electronic equipment | |
| CN116166274A (en) | Neural network information integration method, device, system and storage medium | |
| CN114510417A (en) | Image rendering effect testing method and device, electronic equipment and storage medium | |
| CN116301898A (en) | Neural network information integration method, device, system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Neural network information integration methods, devices, systems, and storage media Granted publication date: 20230721 Pledgee: Jiang Wei Pledgor: BEIJING INTENGINE TECHNOLOGY Co.,Ltd. Registration number: Y2024980019734 |