KR20230126639A - Method for lightweighting neural network model and electronic apparatus for performing the same - Google Patents

Abstract

Disclosed is a method for lightweighting a neural network model. The method for lightening the neural network model comprises: a step of inputting an original model including a plurality of layers each containing a plurality of filters, a degree of lightweighting, and a metric; a step of determining the importance of each of the plurality of filters using the metric; a step of normalizing the importance of each of the plurality of filters on a layer-by-layer basis by normalizing the importance of each filter based on the importance of the remaining filters, excluding filters with importance less than the importance of each filter among the multiple filters included in each layer of the original model; and a step of lightening the original model by removing at least one filter among the plurality of filters based on the normalized importance and the degree of lightweighting. Accordingly, the present invention may maximize latency reduction while minimizing reduction in model accuracy.

Description

Neural network model lightweight method and electronic device performing the same

The present disclosure relates to a method for reducing the weight of a neural network model and an electronic device performing the same.

Pruning is one of the lightweight neural network model techniques and is a method of removing relatively unnecessary parameters (eg, weights). Various types of pruning techniques have been developed so far, but there are some problems. For example, in the case of a technique of masking some weights of filters to 0 without removing unnecessary filters, latency is reduced only in a specific device, so there is no advantage in speed improvement through light weight.

As another example, there is also a technique for actually removing unnecessary filters, but there is a problem in that the accuracy of the model is rapidly reduced because the entire specific layer disappears. In addition, even if the entire layer does not disappear, there is a problem in that information contained in the model is distorted by removing the filter without considering the connection relationship between the layers.

As another example, there is a method in which the user directly sets the degree of lightweighting for each layer, but the entry barrier is very high because it requires a very high level of lightweighting knowledge from the user.

Therefore, there is a need for a pruning technique that maximizes latency reduction due to weight reduction while minimizing reduction in model accuracy and improves user convenience.

One technical problem to be solved by the present invention is to provide a lightweight method capable of maximizing latency reduction while minimizing model accuracy reduction.

Another technical problem to be solved by the present invention is to provide a weight reduction method capable of preventing information distortion or a significant decrease in accuracy due to weight reduction of a model.

Another technical problem to be solved by the present invention is to provide a lightweight method with improved user convenience.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

According to an exemplary embodiment of the present disclosure for solving the above-described technical problem, in a method for lightening a neural network model performed by an electronic device, an original model including a plurality of layers each including a plurality of filters, the original model receiving a metric for determining a degree of weight reduction to be applied to a model and an importance of each of the plurality of filters; determining an importance of each of the plurality of filters using the metric; Among the plurality of filters included in each layer of the original model, the importance of each filter is normalized based on the importance of the remaining filters except for filters having an importance smaller than the importance of each filter, so that each of the plurality of filters is layer-by-layer. Normalizing the importance of ; and reducing the weight of the original model by removing at least one filter from among the plurality of filters based on the normalized importance and the degree of weight reduction.

According to an exemplary embodiment of the present disclosure for solving the above-described technical problem, an electronic device for lightening a neural network model, comprising: a memory for storing at least one instruction; and a processor, wherein the processor, by executing the at least one instruction, includes an original model including a plurality of layers each including a plurality of filters, a degree of weight reduction to be applied to the original model, and each of the plurality of filters. Receive a metric for determining the importance of , determine the importance of each of the plurality of filters using the metric, and determine the importance of each filter among the plurality of filters included in each layer of the original model. By normalizing the importance of each filter based on the importance of the remaining filters except for a filter having a small importance, the importance of each of the plurality of filters is normalized in a layer unit, and the plurality of filters are normalized based on the normalized importance and the degree of weight reduction. An electronic device that reduces the weight of the original model by removing at least one of the filters may be provided.

According to an exemplary embodiment of the present disclosure for solving the above-described technical problem, in a method for lightening a neural network model performed by an electronic device,

Receiving an original model including a plurality of layers each including a plurality of filters and a metric for determining importance of each of the plurality of filters; determining an importance of each of the plurality of filters using the metric; Among the plurality of filters included in each layer of the original model, the importance of each filter is normalized based on the importance of the remaining filters except for filters having an importance smaller than the importance of each filter, so that each of the plurality of filters is layer-by-layer. Normalizing the importance of ; and at least one filter among a plurality of filters on the mutually related layer based on the normalized importance and a policy related to a method for removing filters included in mutually related layers among a plurality of layers included in the original model. A method including weight reduction of the original model by removing the; may be provided.

According to an exemplary embodiment of the present disclosure for solving the above-described technical problem, an electronic device for lightening a neural network model, comprising: a memory for storing at least one instruction; and a processor, wherein the processor, by executing the at least one instruction, determines an original model including a plurality of layers each including a plurality of filters and a metric for determining importance of each of the plurality of filters. ) is received, the importance of each of the plurality of filters is determined using the metric, and the remaining filters except for filters having an importance smaller than the importance of each filter among a plurality of filters included in each layer of the original model. By normalizing the importance of each filter based on the importance of , the importance of each of the plurality of filters is normalized in a layer unit, and the normalized importance and the filter included in the layers related to each other among the plurality of layers included in the original model An electronic device for lightening the original model by removing at least one of a plurality of filters on the mutually related layers based on a policy related to a method for removing them together may be provided.

The solutions to the problems of the present disclosure are not limited to the above-described solutions, and solutions not mentioned will be clearly understood by those skilled in the art from this specification and the accompanying drawings. You will be able to.

According to various embodiments of the present disclosure as described above, latency reduction may be maximized while minimizing reduction in model accuracy by using the lightweight method. In addition, information distortion or significant reduction in accuracy due to weight reduction of the model can be prevented. In addition, since the user can conveniently reduce the weight of the original model, satisfaction can be improved.

In addition, effects that can be obtained or predicted due to the embodiments of the present disclosure will be directly or implicitly disclosed in the detailed description of the embodiments of the present disclosure. For example, various effects predicted according to an embodiment of the present disclosure will be disclosed within the detailed description to be described later.

Other aspects, advantages and salient features of the present disclosure will become apparent to those skilled in the art from the following detailed description, which discloses various embodiments of the present invention in conjunction with the accompanying drawings.

Aspects, features, and advantages of specific embodiments of the present disclosure will become more apparent from the following description with reference to the accompanying drawings.
1 is a flowchart illustrating a method for reducing the weight of a neural network model according to an embodiment of the present disclosure.
2 is a flowchart illustrating a method of determining the importance of a filter according to an embodiment of the present disclosure.
3 is a diagram for explaining a method of determining the importance of a filter according to an embodiment of the present disclosure.
4 is a diagram for explaining a method of normalizing importance according to an embodiment of the present disclosure.
5 is a flowchart illustrating a method for reducing the weight of a neural network model according to an embodiment of the present disclosure.
6 is a diagram for explaining a first policy according to an embodiment of the present disclosure.
7 is a diagram for explaining a second policy according to an embodiment of the present disclosure.
8 is a diagram for explaining a third policy according to an embodiment of the present disclosure.
9 is a diagram for explaining a fourth policy according to an embodiment of the present disclosure.
10 is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure.

Terms used in this specification will be briefly described, and the present disclosure will be described in detail.

The terms used in the embodiments of the present disclosure have been selected from general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but they may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technologies, and the like. . In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the disclosure. Therefore, terms used in the present disclosure should be defined based on the meaning of the term and the general content of the present disclosure, not simply the name of the term.

Embodiments of the present disclosure may apply various transformations and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope to specific embodiments, and should be understood to include all transformations, equivalents, and substitutes included in the spirit and scope of technology disclosed. In describing the embodiments, if it is determined that a detailed description of a related known technology may obscure the subject matter, the detailed description will be omitted.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. Terms are only used to distinguish one component from another.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "comprise" or "consist of" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other It should be understood that the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Hereinafter, with reference to the accompanying drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily carry out the present disclosure. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. And in order to clearly describe the present disclosure in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

1 is a flowchart illustrating a method for reducing the weight of a neural network model according to an embodiment of the present disclosure.

Referring to FIG. 1 , the electronic device 100 receives an original model including a plurality of layers each including a plurality of filters, a degree of weight reduction to be applied to the original model, and a metric for determining the importance of each of the plurality of filters. It can (S110).

The original model is a neural network model, and may be, for example, a model learned to classify objects in images. The original model includes a plurality of layers, and at least some of the plurality of layers may include a plurality of filters (or kernels).

The degree of weight reduction is a value representing how much the original model is to be lightened, and may be, for example, a compression ratio or a pruning ratio.

A metric may be a formula for determining the importance of a filter. For example, the metric may include an L2 norm, a geometric median, a nuclear norm, and/or an L1 norm. The importance of a filter may be represented by an importance score.

The electronic device 100 may receive an input of an original model, a degree of weight reduction, and a metric from a user. For example, the user device may receive a user input for selecting an original model, a degree of weight reduction, and a metric. The electronic device 100 may receive an original model, a degree of weight reduction and a metric, or information related thereto from the user device.

The electronic device 100 may determine the importance of each of the plurality of filters using the metric (S120). The electronic device 100 may calculate the importance of each of the plurality of filters using the metric. The importance corresponding to each filter may be a scalar value.

The electronic device 100 may recalculate the importance of each filter based on the redundancy of the importance of each filter. The overlapping degree of importance of a specific filter may indicate how much the importance of the corresponding filter overlaps with the importance of other filters included in the layer including the corresponding filter. For example, as the importance of the first filter approaches the average value of the importance of all filters included in the first layer including the first filter, the degree of overlap of the importance of the first filter may increase. In addition, it can be said that the greater the standard deviation of the importance of all filters included in the first layer, the greater the degree of redundancy of the importance of the first filter.

In the process of recalculating the importance, the importance of each filter may increase or decrease according to the degree of overlap. For example, when the overlapping degree of importance of the first filter is equal to or greater than a certain level, the importance of the first filter may decrease compared to the initial value. When the degree of redundancy of the importance of the first filter is less than a certain level, the importance of the first filter may increase compared to the initial value.

Since the electronic device 100 removes filters with low recalculated importance, it can be said that filters with high redundancy are more likely to be removed. When a filter having a high degree of redundancy is removed, information distortion or reduction in accuracy of the model may be less than when a filter having a low degree of redundancy is removed. Therefore, if the model is lightweight in consideration of redundancy, there is an advantage in that the level of lightweighting can be increased while minimizing the decrease in accuracy due to the lightweighting.

The electronic device 100 normalizes the importance of each filter based on the importance of the remaining filters except for filters having an importance less than the importance of each filter among a plurality of filters included in each layer of the original model, thereby normalizing the importance of each filter in a plurality of layers. The importance of each filter of can be normalized (S130). The electronic device 100 may normalize the importance of each filter by dividing the importance of each filter included in the corresponding layer by the sum of the importances of the filters included in the corresponding layer for each layer. For example, when the first layer includes a first filter, a second filter, and a third filter, the electronic device 100 sums the importance of the first filter, the importance of the second filter, and the importance of the third filter. The importance of the first filter may be normalized by dividing the importance of the first filter by the value.

On the other hand, when normalization is performed by dividing the importance of each filter by the sum of the importance of all filters in the layer, the importance of the filter may fade with the number of filters as the number of filters included in the layer increases. In addition, some layers may be removed as the degree of weight reduction set by the user increases. In this case, information distortion or accuracy reduction of the model may be intensified.

To prevent this, when normalizing the importance of each filter, the electronic device 100 may select only filters having an importance greater than or equal to the importance of the corresponding filter within the layer including the corresponding filter. The electronic device 100 may divide the importance of each filter by the sum of the importance of the selected filters. That is, when the electronic device 100 normalizes the importance of each filter, filters having an importance less than the importance of the corresponding filter may be ignored.

When normalization is completed, the electronic device 100 may sort all filters in order of importance regardless of layer. In addition, the electronic device 100 may remove filters of low importance by a predetermined number based on the degree of weight reduction set by the user. As described above, when the electronic device 100 normalizes the importance of each filter, filters having an importance smaller than the importance of the corresponding filter are ignored, so that each layer after weight reduction has at least one filter (that is, the importance of each layer). The filter with the highest is left. At least one filter is left in each layer, so the layer itself will not be removed. As described above, since the weight reduction method according to the present disclosure does not remove the model layer, there is an advantage in that information distortion or model accuracy reduction is small compared to the existing weight reduction methods.

The electronic device 100 may reduce the weight of the original model by removing at least one filter among a plurality of filters based on the normalized importance and degree of weight reduction (S140). The electronic device 100 may sort all filters based on normalized importance regardless of layers. The electronic device 100 may select at least one normalized filter having a low importance based on the degree of weight reduction. The number of filters selected may be determined according to the degree of weight reduction. For example, as the degree of weight reduction increases, the number of selected filters may also increase.

2 is a flowchart illustrating a method ( S120 ) of determining the importance of a filter according to an embodiment of the present disclosure.

Referring to FIG. 2 , the electronic device 100 may calculate the importance of each of a plurality of filters using a metric (S121).

The electronic device 100 may recalculate the importance of each of the plurality of filters based on the calculated degree of overlap of the importance (S122). For example, the electronic device 100 may recalculate the importance of each filter by standardizing the importance of each filter. That is, the recalculated importance of the filter may be a standard score of the first calculated importance. As the importance of the filter is standardized, the redundancy of the importance may be reflected in the recalculated importance.

The electronic device 100 may adjust the importance of each filter according to the redundancy of the importance of each filter. For example, the electronic device 100 may increase the size of importance with low overlap, and decrease the size of importance with low overlap. When a filter with a low degree of redundancy is removed, it has a large effect on reducing the accuracy of the model, whereas a filter with a high degree of redundancy may have a relatively small effect on the decrease in the accuracy of the model even if it is removed. Therefore, it may be advantageous to preserve the accuracy of the model to remove filters having a relatively high degree of redundancy.

3 is a diagram for explaining a method of determining the importance of a filter according to an embodiment of the present disclosure.

Referring to FIG. 3 , the first layer L1 may include a first filter 31 , a second filter 32 and a third filter 33 . The illustrated circle represents a filter, and the letters W, c, and r displayed in the circle represent the weight, importance, and recalculated importance of the filter, respectively. The subscript notation of each letter indicates (layer index value, filter or channel index value). The weights of the filters 31, 32, and 33 ( , , ) may be a matrix having a predetermined size where each element is a weight. For example, the weight of the first filter 31 ( ) may be a 3x3 matrix. Meanwhile, FIG. 3 describes a method of determining the importance of a filter based on one layer for convenience, but it is noted that the method described later can be applied to other layers included in the original model as it is.

The electronic device 100 calculates the weights of the filters 31, 32, and 33 in the first equation (F1) ( , , ) to determine the importance of the filters 31, 32, and 33 ( , , ) can be calculated. importance ( , , ) may be a scalar value.

In the second equation (F2), the electronic device 100 determines the importance ( , , ) can be applied to recalculate the importance of the filters 31, 32, and 33. Accordingly, the electronic device 100 determines the recalculated importances of the filters 31, 32, and 33 ( , , ) can be obtained. The electronic device 100 may standardize the importance of the filter for each layer by using Equation 2 (F2). In the second equation (F2), represents the standard deviation of the importance of all filters included in each layer, represents the average of the importances of all filters included in each layer. In the case of the first layer (L1), is the importance ( , , ) is the standard deviation of is the importance ( , , ) is the average value of

The second Equation (F2) may reflect the redundancy of the importance of the filter. For example, the smaller the difference between the importance of the first filter included in the first layer and the average of the importance of all filters included in the first layer, the greater the redundancy of the first filter.

4 is a diagram for explaining a method of normalizing importance according to an embodiment of the present disclosure.

Referring to FIG. 4 , the electronic device 100 calculates the recalculated importances of the filters 31, 32, and 33 in the third equation (F3) ( , , ) may be applied to normalize the filters 31, 32, and 33. Accordingly, the electronic device 100 determines the normalized importance levels of the filters 31, 32, and 33 ( , , ) can be obtained.

Referring to Equation 3 (F3), normalization is performed in units of layers. For example, the importance of the first filter 31 ( ), the importance of the filters 31, 32 and 33 included in the first layer L1 ( , , ) is used, but the importance of filters included in other layers may not be used.

Also, as can be seen from the denominator of Equation 3 (F3), when the importance of a specific filter is normalized, the importance of a filter having a lower importance than the corresponding filter may not be considered. For example, when the first filter 31 has the highest importance among the filters 31, 32, and 33, the importance of the remaining filters 32 and 33 when the first filter 31 is normalized. may not be reflected in the denominator of the third equation (F3). Therefore, even if the number of filters included in the first layer L1 increases, the importance of the filters may not fade with the number of filters. In addition, since the importance of the filter having the greatest importance in the first layer (L1) becomes 1 according to normalization, it may not be selected in the process of selecting a filter to be removed afterwards, so that the entire first layer (L1) may not be removed. there is.

On the other hand, in the third equation (F3), the importance of a filter having a lower importance than itself is not reflected in the denominator. Alternatively, in the denominator of Equation 3 (F3), the importance of a filter having a lower importance than itself may be substituted with a very small value close to 0.

The electronic device 100 may normalize the importance of all filters included in the original model. Also, the electronic device 100 may sort all filters based on normalized importance regardless of the layer. The electronic device 100 may select filters of low importance by the determined number based on the degree of weight reduction set by the user. The electronic device 100 may reduce the weight of the original model by removing the selected filter. The electronic device 100 may provide a lightweight model to the user. For example, the electronic device 100 may provide download data corresponding to the lightweight model to the user.

Meanwhile, although the number of filters included in the first layer L1 is 3 as an example in FIGS. 3 and 4 , it should be noted that the number of filters is not limited to 3.

5 is a flowchart illustrating a method for reducing the weight of a neural network model according to an embodiment of the present disclosure.

Referring to FIG. 5 , the electronic device 100 may receive an original model including a plurality of layers each including a plurality of filters and a metric for determining the importance of each of the plurality of filters (S510). . For example, the original model may include a first layer and a second layer. The first layer may include a plurality of first filters. The second layer may include a plurality of second filters.

The electronic device 100 may determine the importance of each of the plurality of filters using the metric (S520). The method for determining the importance of the filter has already been described with reference to FIG. 1, so detailed descriptions thereof will be omitted.

The electronic device 100 normalizes the importance of each filter based on the importance of the remaining filters except for filters having an importance less than the importance of each filter among a plurality of filters included in each layer of the original model, thereby normalizing the importance of each filter in a plurality of layers. The importance of each filter of can be normalized (S530). As this step can be clearly understood through S130 of FIG. 1, a detailed description thereof will be omitted.

The electronic device 100 selects at least one of a plurality of filters on the mutually related layers based on the normalized importance and a policy related to a method for removing filters included in the mutually related layers among the plurality of layers included in the original model. It is possible to reduce the weight of the original model by removing the filter of (S540).

In the present disclosure, a policy related to a method for removing filters included in mutually related layers together may be briefly referred to as a 'filter removal policy', a 'pruning policy', or a 'policy'. The user may input the original model to the user device and set a pruning policy. The electronic device 100 may receive the original model and the pruning policy from the user device.

The pruning policy of the present disclosure may include a first policy, a second policy, a third policy, and a fourth policy. According to the first policy and the second policy, some filters in each layer are determined as removal candidates, and a final removal target may be determined based on the indexes of the filters determined as removal candidates. For example, the electronic device 100 may determine a first removal candidate among a plurality of first filters and a second removal candidate among a plurality of second filters based on the calculated importance and the degree of weight reduction set by the user. .

According to the first policy, among the removal candidates of each layer, a filter having the same index as a removal candidate of another layer may be determined as a removal target in the corresponding layer. For example, among the first removal candidates, a filter having the same index as the second removal candidate may be determined as a removal target in the first layer. The first policy may also be referred to as 'intersection'.

According to the second policy, not only removal candidates of each layer but also filters having the same index as removal candidates of other layers among filters not determined as removal candidates in each layer may be determined as removal targets. For example, the removal target in the first layer may include a first removal candidate and a first filter that is not determined as the first removal candidate in the first layer but has the same index as the second removal candidate in the second layer. there is. The second policy may also be referred to as 'union'.

According to the third policy and the fourth policy, the importance corresponding to each index (or channel) may be calculated through an additional operation between the importance of filters having the same index among filters included in a plurality of layers. In addition, an index having a low importance may be selected based on the degree of weight reduction set by the user and the importance corresponding to each index. A filter having a selected index may be determined as a removal target to be removed from each layer. In the third policy, an additional operation for calculating the importance corresponding to the index may be summation. The third policy may also be referred to as 'sum'. In the fourth policy, an additional operation for calculating the importance corresponding to the index may be averaging. The fourth policy may also be referred to as 'average'.

The electronic device 100 may identify a first layer and a second layer that are related to each other. In the present disclosure, when a plurality of layers are interrelated, it means that the plurality of layers are connected through a predetermined operator. Here, that a plurality of layers are connected through a preset operator means that an operation based on a preset operator is performed between the plurality of layers. For example, there may be a case where the first layer and the second layer are connected through an addition operator so that an addition operation between the first layer and the second layer is performed. Preset operators may include element-wise operators and concatenation operators. Element-wise operators may include arithmetic operators.

The electronic device 100 may calculate the importance of each of the plurality of first filters included in the first layer and the importance of each of the plurality of second filters included in the second layer. For example, the electronic device 100 may calculate the importance of each of the plurality of first filters and the importance of each of the plurality of second filters based on the importance determining method of FIG. 3 . Alternatively, the electronic device 100 may normalize the calculated importance by additionally applying the normalization method of FIG. 4 .

The electronic device 100 may determine filters to be removed from the first layer and the second layer based on the calculated importance and policy. For example, in the case of the first policy or the second policy, the electronic device 100 determines a first removal candidate from among the plurality of first filters based on the calculated importance and the degree of weight reduction set by the user, and the plurality of second policies. Among the filters, a second removal candidate may be determined. The electronic device 100 may select a first removal target from a plurality of first filters and a second removal target from a plurality of second filters based on the policy, the first removal candidate, and the second removal candidate.

In the case of the first policy, the electronic device 100 may select a filter having the same index as the second removal candidate among the first removal candidates as the first removal target. Also, the electronic device 100 may select a filter having the same index as the first removal candidate among the second removal candidates as the second removal target.

In the case of the second policy, the electronic device 100 may determine a filter and a first removal candidate having the same index as the second removal candidate among the plurality of first filters as the first removal target. Also, the electronic device 100 may determine a filter and a second removal candidate having the same index as the first removal candidate among the plurality of second filters as the second removal target.

In the case of the third policy, the electronic device 100 may determine a filter to be removed from each layer based on the sum of importances of filters having the same index among the plurality of first filters and the plurality of second filters. In the case of the fourth policy, the electronic device 100 may determine a filter to be removed from each layer based on an average of importances of filters having the same index among the plurality of first filters and the plurality of second filters.

The aforementioned policies are methods for commonly applying pruning to mutually related layers. If pruning is applied to the original model based on the policy of the present disclosure, information distortion or significant reduction in model accuracy due to unbalanced pruning of interrelated layers can be prevented.

6 is a diagram for explaining a first policy according to an embodiment of the present disclosure. FIG. 6(a) shows a method for determining removal candidates, and FIG. 6(b) shows a method for selecting a removal target.

Referring to FIG. 6 , the first layer L1 may include a first filter 61 , a second filter 62 , a third filter 63 and a fourth filter 64 . The second layer L2 may include a fifth filter 65 , a sixth filter 66 , a seventh filter 67 and an eighth filter 68 . The first filter 61 and the fifth filter 65 have the same index, the second filter 62 and the sixth filter 66 have the same index, and the third filter 63 and the seventh filter 67 ) has the same index, and the fourth filter 64 and the eighth filter 68 have the same index. The number displayed inside the circle representing each filter indicates the importance of that filter. The first layer L1 and the second layer L2 are interrelated layers that are connected through an addition operator.

Referring to (a) of FIG. 6 , the first filter 61 and the second filter 62 having low importance in the first layer L1 may be determined as first removal candidates. In the second layer (L2), the sixth filter 66 and the eighth filter 68 of low importance may be determined as second removal candidates. Meanwhile, in this embodiment, the number of filters determined as removal candidates for each layer is 2, but this is only an example, and the number of filters determined as removal candidates may vary according to the degree of weight reduction set by the user. For example, when the degree of weight reduction set by the user increases, the number of filters determined as removal candidates may increase to three.

Referring to (b) of FIG. 6, in the first removal candidates 61 and 62 and the second removal candidates 66 and 68, the second filter 62 and the sixth filter 66 having the same index are to be removed. can be determined Specifically, in the first removal candidates 61 and 62 , the second filter 62 having the same index as the second removal candidates 66 and 68 may be determined as the first removal target. In the second removal candidates 66 and 68 , a sixth filter 66 having the same index as the first removal candidates 61 and 62 may be determined as a second removal target.

7 is a diagram for explaining a second policy according to an embodiment of the present disclosure. FIG. 7(a) shows a method for determining removal candidates, and FIG. 7(b) shows a method for selecting a removal target.

Referring to FIG. 7 , the first layer L1 may include a first filter 61 , a second filter 62 , a third filter 63 and a fourth filter 64 . The second layer L2 may include a fifth filter 65 , a sixth filter 66 , a seventh filter 67 and an eighth filter 68 . The first filter 61 and the fifth filter 65 have the same index, the second filter 62 and the sixth filter 66 have the same index, and the third filter 63 and the seventh filter 67 ) has the same index, and the fourth filter 64 and the eighth filter 68 have the same index. The number displayed inside the circle representing each filter indicates the importance of that filter. The first layer L1 and the second layer L2 are interrelated layers that are connected through an addition operator.

Referring to (a) of FIG. 7 , the first filter 61 and the second filter 62 having low importance in the first layer L1 may be determined as first removal candidates. In the second layer (L2), the sixth filter 66 and the eighth filter 68 of low importance may be determined as second removal candidates. Meanwhile, in this embodiment, the number of filters determined as removal candidates for each layer is 2, but this is only an example, and the number of filters determined as removal candidates may vary according to the degree of weight reduction set by the user. For example, when the degree of weight reduction set by the user increases, the number of filters determined as removal candidates may increase to three.

Referring to (b) of FIG. 7 , as a first removal target to be removed from the first layer L1, first removal candidates 61 and 62, and filters 61 included in the first layer L1, Among 62, 63, and 64, a fourth filter 64 having the same index as the second removal candidates 66 and 68 may be determined. As the second removal target to be removed from the second layer L2, a first one of the second removal candidates 66 and 68 and the filters 65, 66, 67 and 68 included in the second layer L2. A fifth filter 65 having the same index as the removal candidates 61 and 62 may be determined.

8 is a diagram for explaining a third policy according to an embodiment of the present disclosure. (a) of FIG. 8 shows the sum of importances of filters included in the first layer L1 and the second layer L2. 8(b) shows a method of selecting a removal target.

Referring to FIG. 8 , the first layer L1 may include a first filter 61 , a second filter 62 , a third filter 63 and a fourth filter 64 . The second layer L2 may include a fifth filter 65 , a sixth filter 66 , a seventh filter 67 and an eighth filter 68 . The first filter 61 and the fifth filter 65 have a first index i1, the second filter 62 and the sixth filter 66 have a second index i2, and the third filter ( 63) and the seventh filter 67 may have a third index i3, and the fourth filter 64 and the eighth filter 68 may have a fourth index i4. The number displayed inside the circle representing each filter indicates the importance of that filter. The first layer L1 and the second layer L2 are interrelated layers that are connected through an addition operator.

(a) of FIG. 8 shows the sum of importance for each index. An importance (ie, 32) corresponding to the first index i1 may be calculated based on the importance of the first filter 61 and the fifth filter 65 having the first index i1. An importance (ie, 30) corresponding to the second index i2 may be calculated based on the importance of the second filter 62 and the sixth filter 66 having the second index i2. An importance (ie, 50) corresponding to the third index i3 may be calculated based on the importance of the third filter 63 and the seventh filter 67 having the third index i3. An importance (ie, 40) corresponding to the fourth index i4 may be calculated based on the importance of the fourth filter 64 and the eighth filter 68 having the fourth index i4.

Referring to (b) of FIG. 8 , the first filter 61 and the fifth filter 65 corresponding to the first index i1 having a low importance level may be determined as targets for removal. In addition, the second filter 62 and the sixth filter 66 corresponding to the second index i2 may be determined to be removed. Meanwhile, in this embodiment, the number of filters determined to be removed in each layer is 2, but this is only an example, and the number of filters determined to be removed may vary according to the degree of weight reduction set by the user. For example, when the degree of weight reduction set by the user increases, the number of filters determined to be removed may increase to three. Alternatively, when the degree of weight reduction set by the user decreases, the number of filters determined to be removed may decrease to one.

9 is a diagram for explaining a fourth policy according to an embodiment of the present disclosure. (a) of FIG. 9 shows an average of importances of filters included in the first layer L1 and the second layer L2. 8(b) shows a method of selecting a removal target.

Referring to FIG. 9 , the first layer L1 may include a first filter 61 , a second filter 62 , a third filter 63 and a fourth filter 64 . The second layer L2 may include a fifth filter 65 , a sixth filter 66 , a seventh filter 67 and an eighth filter 68 . The first filter 61 and the fifth filter 65 have a first index i1, the second filter 62 and the sixth filter 66 have a second index i2, and the third filter ( 63) and the seventh filter 67 may have a third index i3, and the fourth filter 64 and the eighth filter 68 may have a fourth index i4. The number displayed inside the circle representing each filter indicates the importance of that filter. The first layer L1 and the second layer L2 are interrelated layers that are connected through an addition operator.

Figure 9 (a) shows the average of importance for each index. An importance (ie, 16) corresponding to the first index i1 may be calculated based on the importance of the first filter 61 and the fifth filter 65 having the first index i1. An importance (ie, 15) corresponding to the second index i2 may be calculated based on the importance of the second filter 62 and the sixth filter 66 having the second index i2. An importance (ie, 25) corresponding to the third index i3 may be calculated based on the importance of the third filter 63 and the seventh filter 67 having the third index i3. An importance (ie, 20) corresponding to the fourth index i4 may be calculated based on the importance of the fourth filter 64 and the eighth filter 68 having the fourth index i4.

Referring to (b) of FIG. 9 , the first filter 61 and the fifth filter 65 corresponding to the first index i1 having a low importance level may be determined as targets for removal. In addition, the second filter 62 and the sixth filter 66 corresponding to the second index i2 may be determined to be removed. Meanwhile, in this embodiment, the number of filters determined to be removed in each layer is 2, but this is only an example, and the number of filters determined to be removed may vary according to the degree of weight reduction set by the user. For example, when the degree of weight reduction set by the user increases, the number of filters determined to be removed may increase to three. Alternatively, when the degree of weight reduction set by the user decreases, the number of filters determined to be removed may decrease to one.

10 is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 10 , the electronic device 100 may include a communication interface 110, a memory 120, and a processor 130. For example, the electronic device 100 may be implemented as a physical server or a cloud server.

The communication interface 110 includes at least one communication circuit and can communicate with various types of external devices. For example, the communication interface 110 may receive information about an original model, a degree of weight reduction, and a metric from an external device. An external device may be a user device. User devices may include personal computers and mobile devices. The user device may receive an original model, a degree of weight reduction, and a metric from the user.

The communication interface 110 is a Wi-Fi communication module, cellular communication module, 3G (3rd generation) mobile communication module, 4G (4th generation) mobile communication module, 4th generation LTE (Long Term Evolution) communication module, 5G (5th generation) mobile communication It may include at least one of a module and wired Ethernet.

The memory 120 may store an Operating System (OS) for controlling overall operations of components of the electronic device 100 and commands or data related to components of the electronic device 100 . In particular, the memory 120 may store instructions for lightening the original model. The memory 120 may be implemented as non-volatile memory (ex: hard disk, solid state drive (SSD), flash memory) or volatile memory.

The processor 130 is electrically connected to the memory 120 to control overall functions and operations of the electronic device 100 . The processor 130 may control the electronic device 100 by executing instructions stored in the memory 120 .

For example, the processor 130 may obtain an original model, a degree of weight reduction to be applied to the original model, and a metric. The processor 130 may receive the original model, the degree of weight reduction, and the metric from the user device through the communication interface 110 . The original model may include a plurality of layers. A layer may include a plurality of filters (or kernels).

The processor 130 may determine the importance of each of the plurality of filters using the metric. The processor 130 may calculate the importance of each of the plurality of filters using the metric, and recalculate the importance of each of the plurality of filters based on the degree of overlap of the calculated importance. The importance of the filter may be adjusted according to the degree of redundancy of the importance. For example, as the redundancy of the calculated importance of the first filter increases, the recalculated importance of the first filter may decrease. The processor 130 may calculate the degree of redundancy of the calculated importance of the first filter based on the calculated importance of the first filter and the average and standard deviation of the calculated importance of the filters included in the first layer.

The processor 130 may normalize the recalculated importance. For example, when normalizing the importance of the first filter included in the first layer, the processor 130 determines the remaining filters except for filters having an importance smaller than the importance of the first filter among the filters included in the first layer. The importance of the first filter may be normalized based on the importance.

The processor 130 may sort the plurality of filters in order of importance. The processor 130 may select at least one filter from a plurality of filters by a predetermined number based on the degree of weight reduction in order of decreasing importance. Alternatively, a threshold importance for selecting a filter to be removed may be determined based on the degree of lightening set by the user. At this time, the processor 130 may select a filter having a lower importance than the critical importance. The processor 130 may reduce the weight of the original model by removing the selected filter. Accordingly, the processor 130 may generate a lightweight model. Each of the plurality of layers included in the lightweight model may include at least one filter.

The processor 130 may obtain an original model and a policy related to a method for removing filters included in mutually related layers among a plurality of layers included in the original model. Also, the processor 130 may acquire a degree of weight reduction for reducing the weight of the original model.

The processor 130 may analyze the original model to identify a first layer and a second layer that are related to each other. For example, when the first layer and the second layer are connected through an element-by-element operator, the processor 130 may determine that the first layer and the second layer are interrelated.

The processor 130 may calculate the importance of each of the plurality of first filters included in the first layer and the importance of each of the plurality of second filters included in the second layer. The processor 130 may determine filters to be removed from the first layer and the second layer based on the calculated importance and policy. Since the method of determining the filter to be removed for each policy has been described above, a detailed description thereof will be omitted.

Various embodiments described above may be implemented in a recording medium readable by a computer or a similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented in a processor itself. When implemented in software, embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

Computer instructions for performing processing operations according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. Computer instructions stored in such a non-transitory computer readable medium may cause a specific device to perform processing operations according to various embodiments described above when executed by a processor.

A non-transitory computer readable medium is a medium that stores data semi-permanently and is readable by a device, not a medium that stores data for a short moment, such as a register, cache, or memory. Specific examples of the non-transitory computer readable media may include CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-temporary storage medium' only means that it is a tangible device and does not contain signals (e.g., electromagnetic waves), and this term refers to the case where data is stored semi-permanently in the storage medium and temporary It does not discriminate if it is saved as . For example, a 'non-temporary storage medium' may include a buffer in which data is temporarily stored.

Methods according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. A computer program product is distributed in the form of a device-readable storage medium (e.g. compact disc read only memory (CD-ROM)), or through an application store (e.g. Play Store™) or on two user devices (e.g. It can be distributed (eg downloaded or uploaded) online, directly between smartphones. In the case of online distribution, at least a part of a computer program product (eg, a downloadable app) is stored on a device-readable storage medium such as a memory of a manufacturer's server, an application store server, or a relay server. It can be temporarily stored or created temporarily.

Although the preferred embodiments of the present disclosure have been shown and described above, the present disclosure is not limited to the specific embodiments described above, and is common in the technical field belonging to the present disclosure without departing from the gist of the present disclosure claimed in the claims. Of course, various modifications are possible by those with knowledge of, and these modifications should not be individually understood from the technical spirit or perspective of the present disclosure.

100: electronic device 110: communication interface
120: memory 130: processor

Claims (20)

In the neural network model lightweight method performed by an electronic device,
receiving a metric for determining an original model including a plurality of layers each including a plurality of filters, a degree of weight reduction to be applied to the original model, and an importance of each of the plurality of filters;
determining an importance of each of the plurality of filters using the metric;
Among the plurality of filters included in each layer of the original model, the importance of each filter is normalized based on the importance of the remaining filters except for filters having an importance smaller than the importance of each filter, so that each of the plurality of filters is layer-by-layer. Normalizing the importance of ; and
Lightening the original model by removing at least one filter from among the plurality of filters based on the normalized importance and the degree of lightening;
method. According to claim 1,
Determining the importance of each of the plurality of filters,
Calculating the importance of each filter using the metric; and
Comprising the step of recalculating the importance of each filter based on the degree of redundancy of the calculated importance of each filter
method. According to claim 2,
Characterized in that the higher the redundancy of the calculated importance of each filter, the smaller the recalculated importance of each filter
method. According to claim 2,
The degree of redundancy of the calculated importance of each filter is,
Calculated based on the calculated importance of each filter and the average and standard deviation of the calculated importance of filters included in each layer including each filter
method. According to claim 1,
At least one filter to be removed,
Arranging the plurality of filters in order of importance, and selecting a predetermined number based on the degree of lightening in order of decreasing importance in the plurality of filters
method. According to claim 1,
In the lightening step,
Based on the normalized importance and a policy related to a method for removing filters included in mutually related layers among a plurality of layers included in the original model, at least one filter among a plurality of filters on the mutually related layer Lightening the original model by removing it
Including, method. In an electronic device for lightening a neural network model,
a memory storing at least one instruction; and
Including; processor;
The processor, by executing the at least one instruction,
An original model including a plurality of layers each including a plurality of filters, a degree of weight reduction to be applied to the original model, and a metric for determining the importance of each of the plurality of filters are input,
determining the importance of each of the plurality of filters using the metric;
Among the plurality of filters included in each layer of the original model, the importance of each filter is normalized based on the importance of the remaining filters except for filters having an importance smaller than the importance of each filter, so that each of the plurality of filters is layer-by-layer. Normalize the importance of
Lightening the original model by removing at least one filter from among the plurality of filters based on the normalized importance and the degree of lightening.
electronic device. According to claim 7,
the processor,
Calculate the importance of each filter using the metric,
Recalculating the importance of each filter based on the degree of redundancy of the calculated importance of each filter
electronic device. According to claim 8,
Characterized in that the higher the redundancy of the calculated importance of each filter, the smaller the recalculated importance of each filter
electronic device. According to claim 8,
the processor,
Calculating the degree of redundancy of the calculated importance of each filter based on the calculated importance of each filter and the average and standard deviation of the calculated importance of filters included in each layer including each filter
electronic device. According to claim 7,
the processor,
Sort the plurality of filters in order of importance,
Selecting at least one filter in order of decreasing importance from the plurality of filters by a number determined based on the degree of lightening
electronic device. According to claim 7,
the processor,
Based on the normalized importance and a policy related to a method for removing filters included in mutually related layers among a plurality of layers included in the original model, at least one filter among a plurality of filters on the mutually related layer To lighten the original model by removing
electronic device. In the neural network model lightweight method performed by an electronic device,
Receiving an original model including a plurality of layers each including a plurality of filters and a metric for determining importance of each of the plurality of filters;
determining an importance of each of the plurality of filters using the metric;
Among the plurality of filters included in each layer of the original model, the importance of each filter is normalized based on the importance of the remaining filters except for filters having an importance smaller than the importance of each filter, so that each of the plurality of filters is layer-by-layer. Normalizing the importance of ; and
Based on the normalized importance and a policy related to a method for removing filters included in mutually related layers among a plurality of layers included in the original model, at least one filter among a plurality of filters on the mutually related layer Lightening the original model by removing; Including,
method. According to claim 13,
The interrelated layers are
A first layer including a plurality of first filters and a second layer including a plurality of second filters,
At least one filter to be removed,
determining a first removal candidate among the plurality of first filters and a second removal candidate among the plurality of second filters based on the normalized importance and the degree of weight reduction set by a user;
Determined by selecting a first removal target from among the plurality of first filters and selecting a second removal target from among the plurality of second filters based on the policy, the first removal candidate, and the second removal candidate
method. According to claim 14,
When the policy is the first policy, the first removal target is selected as a filter having the same index as the second removal candidate among the first removal candidates, and the second removal target is the first removal target among the second removal candidates. A filter with the same index as the removal candidate is selected.
method. According to claim 14,
When the policy is the second policy, the first removal target includes a filter having the same index as the second removal candidate among the plurality of first filters and the first removal candidate, and the second removal target includes the plurality of filters. Among the second filters of, including a filter having the same index as the first removal candidate and the second removal candidate
method. According to claim 13,
At least one filter to be removed,
When the policy is the third policy, determined based on the sum of the importance of filters having the same index among the plurality of filters
method. According to claim 13,
The filter to be removed is
When the policy is the fourth policy, selected based on the average of the importance of filters having the same index among the plurality of filters
method. According to claim 13,
The interrelated layers are
It is connected through a preset operator,
The preset operator is,
with element-wise operators
method. In an electronic device for lightening a neural network model,
a memory storing at least one instruction; and
Including; processor;
The processor, by executing the at least one instruction,
Receiving an original model including a plurality of layers each including a plurality of filters and a metric for determining the importance of each of the plurality of filters;
determining the importance of each of the plurality of filters using the metric;
Among the plurality of filters included in each layer of the original model, the importance of each filter is normalized based on the importance of the remaining filters except for filters having an importance smaller than the importance of each filter, so that each of the plurality of filters is layer-by-layer. Normalize the importance of
Based on the normalized importance and a policy related to a method for removing filters included in mutually related layers among a plurality of layers included in the original model, at least one filter among a plurality of filters on the mutually related layer To lighten the original model by removing it,
electronic device.