CN108345938A - A kind of neural network processor and its method including bits switch device - Google Patents

Abstract

The invention provides a neural network processor and a method for performing bit conversion on neural network data by using the neural network processor. The neural network processor includes a bit conversion device, and the bit conversion device includes: an input interface, a control unit, a data conversion unit, and an output interface; wherein the control unit is used to generate a control signal for the data conversion unit ; the input interface is used to receive the original data; the data conversion unit is used to perform bit conversion on the original data according to the control signal, so as to convert the original data into a representation with fewer bits A bit conversion result; the output interface is used to output the bit conversion result to the bit conversion device. The invention can reduce the number of bits used for expressing data, reduce the hardware cost and energy consumption required for calculation, and improve the calculation speed.

Description

A neural network processor including a bit conversion device and its method

technical field

The invention relates to the field of artificial intelligence, in particular to the improvement of the neural network processor.

Background technique

The deep learning technology of artificial intelligence has been developed rapidly in recent years. It has achieved great success in solving advanced abstract cognitive problems, such as image recognition, speech recognition, natural language understanding, weather prediction, gene expression, content recommendation and intelligent robots. It has been widely used and proved to have excellent performance. This makes the development and improvement of artificial intelligence technology a research hotspot in academia and industry.

Deep neural network is one of the perception models with the highest level of development in the field of artificial intelligence. This type of network simulates the neural connection structure of the human brain by building a model, and uses multiple transformation stages to describe the data characteristics layeredly. Large-scale data processing tasks such as audio have brought breakthrough progress. The model of the deep neural network is a kind of computing model, which contains a large number of nodes, and these nodes adopt a mesh interconnection structure, which is called the neuron of the deep neural network. The connection strength between two nodes represents the weighted value of the signal between the two nodes, that is, the weight, corresponding to the memory in the neural network in the biological sense.

A dedicated processor for neural network computing, that is, a neural network processor, has also been developed accordingly. In the actual neural network computing process, it is necessary to repeatedly perform convolution, activation, pooling and other operations on a large amount of data, which consumes a huge amount of computing time and seriously affects the user experience. This makes how to reduce the computation time of neural networks an improvement strategy for neural network processors.

Contents of the invention

Therefore, the object of the present invention is to overcome above-mentioned defective of prior art, provide a kind of neural network processor, comprise bit conversion device in this neural network processor, this bit conversion device comprises:

an input interface, a control unit, a data conversion unit, and an output interface;

in,

The control unit is used to generate a control signal for the data conversion unit;

The input interface is used to receive raw data;

The data conversion unit is configured to perform bit conversion on the original data according to the control signal, so as to convert the original data into a bit conversion result expressed with fewer bits;

The output interface is used to output the bit conversion result to the bit conversion device.

Preferably, according to the neural network processor, wherein the control unit is configured to determine a rule for performing bit conversion according to a set parameter or an input parameter, so as to generate the control signal;

Wherein, the parameter includes information related to the number of bits of the original data and the number of bits of the bit conversion result.

Preferably, according to the neural network processor, wherein the data conversion unit is configured to determine the reserved bits and the truncated bits in the original data according to the control signal, and according to the reserved bits of the original data and the The highest bit among the truncated bits of the original data determines the bit conversion result.

Preferably, according to the neural network processor, wherein the data converting unit is configured to determine reserved bits and truncated bits in the original data according to the control signal, and use the reserved bits in the original data as the The result of the bit conversion described above.

Preferably, according to the neural network processor, wherein the data conversion unit is configured to perform bit conversion on the original data according to the control signal, and convert the original data into a bit conversion expressed by using half of the original number of bits result.

A method for bit-converting the data of the neural network using the neural network processor described in any one of the above, comprising:

1) The control unit generates a control signal for the data conversion unit;

2) The input interface receives raw data that needs to be bit-converted from outside the bit conversion device;

3) The data conversion unit performs bit conversion on the original data according to the control signal, so as to convert the original data into a bit conversion result expressed with fewer bits;

4) The output interface outputs the bit conversion result to the bit conversion device.

Preferably, according to the method, wherein step 1) comprises:

1-1) The control unit determines the rules for performing bit conversion according to set parameters or input parameters;

1-2) The control unit generates a control signal corresponding to the rule;

Wherein, the parameter includes information related to the number of bits of the original data and the number of bits of the bit conversion result.

Preferably, according to the method, wherein step 3) comprises:

The data conversion unit determines the bit conversion result based on the highest bit among the reserved bits of the original data and the truncated bits of the original data according to the control signal.

Preferably, according to the method, wherein step 3) comprises:

The data conversion unit uses the reserved bits in the original data as the bit conversion result according to the control signal.

Preferably, according to the method, when the neural network data has been cached and the convolution operation has not been completed, the cached neural network data is input to the bit conversion device to perform steps 1)-4), or when the neural network data has been When the convolution operation on the data is completed and the activation operation has not been completed, the result of the convolution operation is input to the bit conversion device to perform steps 1)-4).

A computer-readable storage medium, in which a computer program is stored, and the computer program is used to implement the method described in any one of the above when executed.

Compared with the prior art, the present invention has the advantages of:

The invention provides a bit conversion device for a neural network processor, which can be used to adjust the number of bits used to express data in various calculation processes of the neural network. By reducing the number of bits used to express data, it is possible to reduce hardware costs required for calculations, increase calculation speed, reduce the need for data storage space by neural network processors, and reduce energy consumption for performing neural network calculations.

Description of drawings

Embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

Fig. 1 shows a block diagram of a bit conversion device according to an embodiment of the present invention;

Fig. 2 is a connection diagram of each unit in the bit conversion device according to an embodiment of the present invention;

Fig. 3 is the method flow that adopts the bit conversion device as shown in Fig. 1 to carry out bit conversion to neural network data according to an embodiment of the present invention;

Fig. 4a is a hardware structural diagram for performing bit conversion under "rounding mode" in the data conversion unit of the bit conversion device according to an embodiment of the present invention;

Fig. 4b is a hardware structural diagram for performing bit conversion in the "direct truncation mode" in the data conversion unit of the bit conversion device according to an embodiment of the present invention.

Detailed ways

The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

As mentioned earlier, when designing a neural network processor, it is desirable to reduce the computation time of the neural network. In this regard, the inventor believes that by appropriately reducing the number of bits of data involved in neural network calculations, such as using fewer bits to represent data that originally requires more bits to represent, reducing the amount of calculations to reduce the number of neural networks calculation time. This is because, in the research of the prior art, the inventor found that the algorithm of the neural network has a relatively high fault tolerance for the intermediate results of the calculation, even if fewer bits are used to represent the original need for more bits to represent The practice of using data will change the accuracy of the data involved in the calculation and thus affect the accuracy of the intermediate results obtained, but this will not have a major impact on the final output of the neural network.

In the present invention, this way of reducing the bits of the data used in the calculation is referred to as a "cutting operation" on the data. Also, the process of adjusting the number of binary bits required to express a numerical value is called "bit conversion". For example, for the decimal value 0.5, the result represented by Q7 fixed-point data is 01000000 (where Q7 uses the leftmost first bit in the 8 bits as the sign bit, and uses the remaining 7 bits to represent the fractional part, thus to represent a decimal with a precision of 7 between -1 and 1), when performing bit conversion, the result originally expressed by Q7 can be modified to be expressed by Q3, and the result is 0100 (similar to Q7, Q3 also uses The first bit on the left is used as the sign bit, the difference is that it uses 3 bits to represent the fractional part, which can represent a decimal with a precision of 3 between -1 and 1).

Based on the above analysis, the present invention proposes a bit conversion device for a neural network processor. By means of the bit conversion device, a rule for performing bit conversion can be determined according to parameters set or based on user input, so as to perform bit conversion on data. Through such conversion, the neural network processor can process a relatively small amount of data, thereby increasing the processing speed and reducing the energy consumption of the neural network processor. The inventor believes that in a logic combinational circuit, the speed of data operation is inversely proportional to the number of bits expressed in numerical values; the energy consumption of data operations is proportional to the bits expressed in numerical values; therefore, after bit conversion of data, accelerated calculation can be achieved with the effect of reducing power consumption.

FIG. 1 shows a bit conversion device 101 according to an embodiment of the present invention, including: an input bus unit 102 as an input interface, a data conversion unit 103 , an output bus unit 104 as an output interface, and a control unit 105 .

Wherein, the input bus unit 102 is used to obtain the neural network data that needs to be bit-converted, so as to provide it to the data conversion unit 103 . In some embodiments, the input bus unit 102 may receive and/or transmit multiple data to be converted in parallel.

The data conversion unit 103 is configured to perform bit conversion on the neural network data input from the bus unit 102 according to a rule for performing bit conversion that is set or determined based on parameters input by a user, for example.

The output bus unit 104 is configured to output the bit conversion result obtained through the processing of the data conversion unit 103 from the bit conversion device 101 , so as to provide it to the device for performing subsequent processing in the neural network processing.

The control unit 105 is configured to determine a bit conversion rule, and select a corresponding bit conversion mode to control the data conversion unit 103 to perform the bit conversion operation. The control unit 105 can determine the rules for performing bit conversion by analyzing the set parameters or the parameters input by the user, so as to select from various preset conversion modes. The parameters here can include the number of bits of the data to be converted and the number of bits of the converted data, or the binary expression used by the data to be converted and the expected binary expression of the converted data, such as Q7, Q3 and so on. For example, according to the parameters input by the user, it is determined to convert the neural network data represented by Q7 to be represented by Q3. When reducing the bits used for expression, the "rounding" method can be adopted, such as converting 01011000 to 0110, or the "direct truncation" method can be adopted, such as converting 01011000 to 0101. Here, conversion methods such as "rounding" or "direct truncation" can be input by the user, or can be set to be fixed.

In some embodiments, the input bus unit 102 and/or the output bus unit 104 may receive and/or transmit multiple data to be converted in parallel.

Fig. 2 is a connection diagram of various units in the bit conversion device according to an embodiment of the present invention. Wherein, the bit number of the input bus unit is 128 bits, and the bit number of the output bus unit is 64 bits. The control unit receives parameters input by the user from the outside of the bit conversion device, which are used to generate a mode switching signal for the data conversion unit according to the determined bit conversion rule, so that the data conversion unit can know which mode needs to be adopted under the current situation to perform bit conversion. In addition, the control unit can also generate an input control signal for controlling the input bus unit to start or stop receiving data, and an output control signal for controlling the output bus unit to start or stop outputting the bit conversion result.

In the following, an embodiment will be used to introduce the process of using the bit conversion device shown in FIG. 1 to perform bit conversion on neural network data. Referring to Figure 3, the method includes:

Step 1. Based on the set conversion requirement parameters or the parameters input by the user, the control unit 105 in the bit conversion device 101 determines the used bit conversion rule. The set conversion requirement parameters and the parameters input by the user include information related to the number of bits of the neural network data to be converted and the number of bits of the converted data. The set conversion requirement parameters and the parameters input by the user may also include truncation rules when performing bit conversion, such as "rounding up" or "direct truncation" and other rules.

Based on the above rules, the control unit 105 can select from preset bit conversion modes. According to an embodiment of the present invention, the bit conversion mode includes "rounding mode" and "direct truncation mode", and the processing methods for the two different modes will be introduced in subsequent steps.

Step 2. The input bus unit 102 in the bit conversion device 101 provides the obtained neural network data that needs to be bit converted to the data conversion unit 103 .

The input bus unit 102 here may include a plurality of interfaces capable of receiving data in parallel, so as to receive in parallel the neural network data that needs to be bit converted from outside the bit conversion device 101 . Similarly, the input bus unit 102 may also include multiple interfaces capable of outputting data in parallel, so as to provide data to the data conversion unit 103 in parallel for bit conversion processing.

Step 3. The data conversion unit 103 performs bit conversion on the neural network data that needs to be bit converted according to the bit conversion rule determined by the control unit 105 .

In this step, the data conversion unit 103 may receive a control signal from the control unit 105 to perform bit conversion according to the rule.

The inventor found that when reducing the number of bits of the data used for calculation, if the reduced number of bits is greater than or equal to half of the number of bits of the original data, the neural network processor can be improved in terms of hardware cost, processing speed, and There is a compromise between accuracy. Therefore, in the present invention, preferably, the number of bits of the neural network data that needs to perform bit conversion is reduced to half of the original, for example, a fixed hardware structure is used to perform bit conversion to convert 32bit data into 16bit, 16bit Convert the data into 8bit, convert 8bit data into 4bit, convert 4bit data into 2bit, and convert 2bit data into 1bit, etc.

In the process of performing bit conversion, according to the rules, each bit of the neural network data that needs to be bit converted can be divided into a reserved bit and a truncated bit, wherein the reserved bit is the lower bit in each bit of the neural network data. one or more high bits, and the truncation bits are the remaining bits in each bit of the neural network data. For example, for 8-bit data 10101111, if the number of bits is reduced to half of the original, the reserved bits are 1010, and the truncated bits are 1111.

Figure 4a shows a hardware device structure for performing bit conversion in the data conversion unit 103 according to an embodiment of the present invention, wherein 16 8-bit neural network data that need to perform bit conversion are processed in parallel Input into the data conversion unit 103, the bits (for example a ₁ , a ₂ , a ₃ ) and the highest bit in the corresponding truncated bits ( For example, a ₄ ) is respectively used as two inputs of an adder, and the output of the adder and the sign bit in the neural network data are jointly used as a result of bit conversion performed on the 8-bit neural network data.

Referring to Fig. 4a for example, in the "rounding mode", suppose the neural network data input to the conversion unit 103 is 10101111 (inverse code), which means that it represents -0.6328125 in decimal, and its truncated bit is 1111, and the truncated bit The highest bit 1 is added to the 3 bits 010 except the sign bit in the reserved bit, and the result obtained after the bit conversion based on the sign bit in the neural network data and the result of the adder is 1011 (inverse code), indicating Decimal -0.625.

Figure 4b shows a hardware device structure for performing bit conversion in the data conversion unit 103 according to an embodiment of the present invention, wherein 16 8-bit neural network data that need to perform bit conversion are parallelized input into the data conversion unit 103, and the 4-bit reserved bits (such as a ₀ , a ₁ , a ₂ , a ₃ ) in each 8-bit neural network data are directly used as execution bits for the 8-bit neural network data The transformed result.

Referring to FIG. 4b for illustration, in the "direct truncation mode", assuming that the neural network data input to the conversion unit 103 is 10101111 (inverse code), the result after bit conversion is 1010.

Step 4. The output bus unit 104 outputs the result of the bit conversion obtained through the processing of the data conversion unit 103 from the bit conversion device 101 to be provided to the device for performing subsequent processing in the neural network processing.

The bit conversion device provided by the above embodiments of the present invention can be used as a part of the neural network processor in various calculation processes for the neural network.

For example, when the neural network data has been cached and the convolution operation has not been completed, a bit conversion device may be used to perform bit conversion on the cached neural network data. The reason for this is that different network layers of the neural network may have different requirements for the number of bits used in the data. In order to adapt to the required calculation speed and expected energy consumption, the neural network of the cache can be adjusted by the bit conversion device. The network data is bit-converted, and the result obtained through the bit-conversion is provided to a unit for performing a convolution operation to perform the convolution operation.

For another example, when the convolution operation on the data has been completed and the activation operation has not been completed, a bit conversion device may be used to perform bit conversion on the result of the convolution operation. The reason for this is that the accumulation operation of the convolution operation unit will often increase the number of bits of the result of the obtained convolution operation, in order to adapt to the requirements for the number of bits of subsequent operations (for example, for some activations implemented by hardware As far as the arithmetic unit is concerned, the number of bits used is often fixed), and it is necessary to perform bit conversion on the result of the convolution operation.

Based on the above embodiments, the present invention provides a bit conversion device for a neural network processor, which can be used to adjust the number of bits used to express data in various calculation processes of the neural network. By reducing the number of bits used to express data, it is possible to reduce hardware costs required for calculations, increase calculation speed, reduce the need for data storage space by neural network processors, and reduce energy consumption for performing neural network calculations.

It should be noted that not all the steps described in the foregoing embodiments are necessary, and those skilled in the art may make appropriate trade-offs, replacements, modifications, etc. according to actual needs.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than limit them. Although the present invention has been described in detail above with reference to the embodiments, those skilled in the art should understand that modifications or equivalent replacements to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all of them should be covered in Within the scope of the claims of the present invention.

Claims (11)

1. A neural network processor, comprising a bit conversion device in the neural network processor, the bit conversion device comprising: an input interface, a control unit, a data conversion unit, and an output interface; in, The control unit is used to generate a control signal for the data conversion unit; The input interface is used to receive raw data; The data conversion unit is configured to perform bit conversion on the original data according to the control signal, so as to convert the original data into a bit conversion result expressed with fewer bits; The output interface is used to output the bit conversion result to the bit conversion device. 2. The neural network processor according to claim 1, wherein the control unit is used to determine a rule for performing bit conversion according to a set parameter or an input parameter, so as to generate the control signal; Wherein, the parameter includes information related to the number of bits of the original data and the number of bits of the bit conversion result. 3. The neural network processor according to claim 2, wherein the data conversion unit is used to determine the reserved bits and the truncated bits in the original data according to the control signal, and according to the reserved bits of the original data And the highest bit among the truncated bits of the original data determines the bit conversion result. 4. The neural network processor according to claim 2, wherein the data conversion unit is configured to determine the reserved bit and the truncated bit in the original data according to the control signal, and convert the reserved bit in the original data to bit as the bit conversion result. 5. The neural network processor according to claim 1, wherein the data conversion unit is used to perform bit conversion on the original data according to the control signal, and convert the original data into an expression using half of the original number of bits The result of the bit conversion. 6. A method that adopts the neural network processor according to any one of claims 1-5 to carry out bit conversion to the data of the neural network, comprising: 1) The control unit generates a control signal for the data conversion unit; 2) The input interface receives raw data that needs to be bit-converted from outside the bit conversion device; 3) The data conversion unit performs bit conversion on the original data according to the control signal, so as to convert the original data into a bit conversion result expressed with fewer bits; 4) The output interface outputs the bit conversion result to the bit conversion device. 7. The method according to claim 6, wherein step 1) comprises: 1-1) The control unit determines the rules for performing bit conversion according to set parameters or input parameters; 1-2) The control unit generates a control signal corresponding to the rule; Wherein, the parameter includes information related to the number of bits of the original data and the number of bits of the bit conversion result. 8. The method according to claim 7, wherein step 3) comprises: The data conversion unit determines the bit conversion result based on the highest bit among the reserved bits of the original data and the truncated bits of the original data according to the control signal. 9. The method according to claim 7, wherein step 3) comprises: The data conversion unit uses the reserved bits in the original data as the bit conversion result according to the control signal. 10. The method according to any one of claims 6-9, when the buffering of the neural network data has been completed and the convolution operation has not been completed, the buffered neural network data is input to the bit conversion device to perform the steps 1)-4), or when the convolution operation on the data has been completed and the activation operation has not been completed, the result of the convolution operation is input to the bit conversion device to perform steps 1)-4). 11. A computer-readable storage medium, in which a computer program is stored, and when executed, the computer program is used to realize the method according to any one of claims 6-10.