KR20230141728A - Method for lightweighting neural network model and electronic apparatus for performing the same - Google Patents

Abstract

Disclosed is a method for lightweighting a neural network model. The method for lightweighting a neural network model comprises the steps of: receiving an original model including a plurality of layers, each including a plurality of filters, a lightening degree, and a metric; determining the importance of each of the plurality of filters using the metric; normalizing the importance of each of the plurality of filters on a layer-by-layer basis by normalizing the importance of each filter based on the importance of the remaining filters, excluding filters with a lower importance than the importance of each filter among the plurality of filters included in each layer of the original model; and lightening the original model by removing at least one filter among the plurality of filters based on the normalized importance and the lightening degree. The present invention can maximize latency reduction while minimizing decrease in model accuracy.

Description

Neural network model lightweight method and electronic device performing the same {METHOD FOR LIGHTWEIGHTING NEURAL NETWORK MODEL AND ELECTRONIC APPARATUS FOR PERFORMING THE SAME}

The present disclosure relates to a method for lightweighting a neural network model and an electronic device that performs the same.

Pruning is one of the neural network model lightweighting techniques and is a method of removing relatively unnecessary parameters (for example, weights). Various types of pruning techniques have been developed so far, but there are several problems. For example, in the case of a technique of masking some weights of the filter to 0 without removing unnecessary filters, latency was reduced only in certain devices, so there was no actual benefit of speed improvement through weight reduction.

As another example, there was a technique to actually remove unnecessary filters, but there was a problem that the accuracy of the model drastically decreased because an entire specific layer disappeared. In addition, even if the entire layer does not disappear, there is a problem that the information contained in the model is distorted by removing the filter without considering the connection between layers.

As another example, there was a method that allowed users to directly set the degree of lightweighting for each layer, but the barrier to entry was very high because it required a very high level of lightweighting knowledge from the user.

Therefore, there is a need for a pruning technique that minimizes the decrease in model accuracy, maximizes latency reduction due to weight reduction, and improves user convenience.

One technical problem that the present invention aims to solve is to provide a lightweight method that can maximize latency reduction while minimizing reduction in model accuracy.

Another technical problem that the present invention aims to solve is to provide a lightweight method that can prevent information distortion or a significant decrease in accuracy due to lightweighting of the model.

Another technical problem to be solved by the present invention is to provide a lightweight method with improved user convenience.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

According to an exemplary embodiment of the present disclosure for solving the above-described technical problem, in a method for lightweighting a neural network model performed by an electronic device, an original model including a plurality of layers each including a plurality of filters, the original model Receiving a metric for determining the degree of lightweighting to be applied to the model and the importance of each of the plurality of filters; determining the importance of each of the plurality of filters using the metric; Among the plurality of filters included in each layer of the original model, the importance of each filter is normalized based on the importance of the remaining filters excluding filters with importance less than the importance of each filter, so that each of the plurality of filters is layer-by-layer. normalizing the importance of; And a step of reducing the weight of the original model by removing at least one filter among the plurality of filters based on the normalized importance and the degree of lightweighting.

According to an exemplary embodiment of the present disclosure for solving the above-described technical problem, an electronic device for lightweighting a neural network model includes: a memory storing at least one instruction; and a processor; by executing the at least one instruction, the processor determines an original model including a plurality of layers each including a plurality of filters, a degree of weight reduction to be applied to the original model, and each of the plurality of filters. Receive a metric for determining the importance of, determine the importance of each of the plurality of filters using the metric, and determine the importance of each filter among the plurality of filters included in each layer of the original model. The importance of each of the plurality of filters is normalized on a layer basis by normalizing the importance of each filter based on the importance of the remaining filters excluding filters with small importance, and the plurality of filters are normalized based on the normalized importance and the degree of lightweighting. An electronic device can be provided that reduces the weight of the original model by removing at least one of the filters.

According to an exemplary embodiment of the present disclosure for solving the above-described technical problem, in a method for reducing the weight of a neural network model performed by an electronic device,

Receiving an original model including a plurality of layers each including a plurality of filters and a metric for determining the importance of each of the plurality of filters; determining the importance of each of the plurality of filters using the metric; Among the plurality of filters included in each layer of the original model, the importance of each filter is normalized based on the importance of the remaining filters excluding filters with importance less than the importance of each filter, so that each of the plurality of filters is layer-by-layer. normalizing the importance of; and at least one filter among a plurality of filters on the interrelated layers based on the normalized importance and a policy related to a method for removing filters included in interrelated layers among the plurality of layers included in the original model. A method including a step of reducing the weight of the original model by removing .

According to an exemplary embodiment of the present disclosure for solving the above-described technical problem, an electronic device for lightweighting a neural network model includes: a memory storing at least one instruction; and a processor, wherein the processor executes the at least one instruction to determine an original model including a plurality of layers each including a plurality of filters and a metric for determining the importance of each of the plurality of filters. ) is input, the importance of each of the plurality of filters is determined using the metric, and the remaining filters excluding the filter with an importance less than the importance of each filter among the plurality of filters included in each layer of the original model The importance of each of the plurality of filters is normalized on a layer-by-layer basis by normalizing the importance of each filter based on the importance of , and the normalized importance and the filters included in interrelated layers among the plurality of layers included in the original model. An electronic device can be provided that reduces the weight of the original model by removing at least one filter among a plurality of filters on the interrelated layer based on a policy related to a method for removing them together.

The means for solving the problem of the present disclosure are not limited to the above-mentioned solution means, and the solution methods not mentioned can be clearly understood by those skilled in the art from this specification and the attached drawings. You will be able to.

According to various embodiments of the present disclosure as described above, latency reduction can be maximized while minimizing reduction in model accuracy by using a lightweight method. Additionally, information distortion or a significant decrease in accuracy due to a lighter model can be prevented. Additionally, users can conveniently reduce the weight of the original model, thereby improving satisfaction.

In addition, effects that can be obtained or expected due to the embodiments of the present disclosure will be disclosed directly or implicitly in the detailed description of the embodiments of the present disclosure. For example, various effects expected according to embodiments of the present disclosure will be disclosed in the detailed description to be described later.

Other aspects, advantages and salient features of the present disclosure will become apparent to those skilled in the art from the following detailed description, taken in conjunction with the accompanying drawings, which disclose various embodiments of the present invention.

Aspects, features and advantages of specific embodiments of the present disclosure will become clearer through the following description with reference to the accompanying drawings.
1 is a flowchart showing a method for lightweighting a neural network model according to an embodiment of the present disclosure.
Figure 2 is a flowchart showing a method of determining the importance of a filter according to an embodiment of the present disclosure.
Figure 3 is a diagram for explaining a method of determining the importance of a filter according to an embodiment of the present disclosure.
Figure 4 is a diagram for explaining a method of normalizing importance according to an embodiment of the present disclosure.
Figure 5 is a flowchart showing a method for lightweighting a neural network model according to an embodiment of the present disclosure.
Figure 6 is a diagram for explaining a first policy according to an embodiment of the present disclosure.
Figure 7 is a diagram for explaining a second policy according to an embodiment of the present disclosure.
Figure 8 is a diagram for explaining a third policy according to an embodiment of the present disclosure.
Figure 9 is a diagram for explaining a fourth policy according to an embodiment of the present disclosure.
Figure 10 is a block diagram showing the configuration of an electronic device according to an embodiment of the present disclosure.

Terms used in this specification will be briefly described, and the present disclosure will be described in detail.

The terms used in the embodiments of the present disclosure have selected general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but this may vary depending on the intention or precedent of a technician working in the art, the emergence of new technology, etc. . In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description part of the relevant disclosure. Therefore, the terms used in this disclosure should be defined based on the meaning of the term and the overall content of this disclosure, rather than simply the name of the term.

Embodiments of the present disclosure may be subject to various changes and may have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope to specific embodiments, and should be understood to include all transformations, equivalents, and substitutes included in the disclosed spirit and technical scope. In describing the embodiments, if it is determined that a detailed description of related known technology may obscure the point, the detailed description will be omitted.

Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. Terms are used only to distinguish one component from another.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “consist of” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are intended to indicate the presence of one or more other It should be understood that this does not exclude in advance the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof.

Below, with reference to the attached drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily practice them. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present disclosure in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

1 is a flowchart showing a method for lightweighting a neural network model according to an embodiment of the present disclosure.

Referring to FIG. 1, the electronic device 100 may receive an original model including a plurality of layers each including a plurality of filters, a metric for determining the degree of lightweighting to be applied to the original model, and the importance of each of the plurality of filters. (S110).

The original model is a neural network model, and for example, may be a model learned to classify objects in images. The original model includes a plurality of layers, and at least some of the plurality of layers may include a plurality of filters (or kernels).

The degree of lightweighting is a value that indicates how much the original model will be lightweighted. For example, it may be a compression ratio or a pruning ratio.

A metric may be a formula for determining the importance of a filter. For example, the metric may include L2 norm, Geometric Median, Nuclear norm, and/or L1 Norm. The importance of a filter can be expressed as an importance score.

The electronic device 100 may receive input from the user regarding the original model, degree of lightweighting, and metrics. For example, the user device may receive user input selecting the original model, degree of lightweighting, and metrics. The electronic device 100 may receive the original model, the degree of lightweighting and metrics, or information related thereto from the user device.

The electronic device 100 may determine the importance of each of the plurality of filters using a metric (S120). The electronic device 100 may calculate the importance of each of the plurality of filters using a metric. The importance corresponding to each filter may be a scalar value.

The electronic device 100 may recalculate the importance of each filter based on the degree of redundancy of the importance of each filter. The degree of overlap in importance of a specific filter may indicate how much the importance of the filter overlaps with the importance of other filters included in the layer including the filter. For example, the closer the importance of the first filter is to the average value of the importance of all filters included in the first layer including the first filter, the greater the degree of overlap in importance of the first filter may be. Additionally, it can be said that the greater the standard deviation of the importance of all filters included in the first layer, the greater the degree of redundancy of the importance of the first filter.

In the process of recalculating the importance, the importance of each filter may increase or decrease depending on the degree of redundancy. For example, if the redundancy of the importance of the first filter is above a certain level, the importance of the first filter may decrease compared to the initial value. If the redundancy of the importance of the first filter is less than a certain level, the importance of the first filter may increase compared to the initial value.

Since the electronic device 100 removes recalculated filters with low importance, it can be said that a filter with a high degree of redundancy and importance is more likely to be removed. When a filter with a high level of redundancy and importance is removed, information distortion or a decrease in model accuracy may be less than when a filter with a low level of redundancy and importance is removed. Therefore, if the model is lightweighted by considering redundancy, there is an advantage in that the level of lightweighting can be increased while minimizing the decrease in accuracy due to lightweighting.

The electronic device 100 normalizes the importance of each filter based on the importance of the remaining filters, excluding filters with an importance less than the importance of each filter, among a plurality of filters included in each layer of the original model, thereby creating a plurality of filters on a layer-by-layer basis. The importance of each filter can be normalized (S130). The electronic device 100 may normalize the importance of each filter by dividing the importance of each filter included in the layer by the sum of the importance of the filters included in the layer. For example, when the first layer includes a first filter, a second filter, and a third filter, the electronic device 100 adds the importance of the first filter, the importance of the second filter, and the third filter. The importance of the first filter can be normalized by dividing the importance of the first filter by the value.

Meanwhile, when normalization is performed by dividing the importance of each filter by the sum of the importance of all filters in the layer, as the number of filters included in the layer increases, the importance of the filter may fade depending on the number of filters. Additionally, as the degree of lightweighting set by the user increases, some layers may be removed. In this case, distortion of model information or reduction in accuracy may intensify.

To prevent this, when normalizing the importance of each filter, the electronic device 100 may select only filters with an importance greater than that of the corresponding filter within a layer including the corresponding filter. The electronic device 100 may divide the importance of each filter by the sum of the importance of the selected filters. That is, when normalizing the importance of each filter, the electronic device 100 may ignore filters with an importance less than that of the corresponding filter.

Once normalization is completed, the electronic device 100 can sort all filters in order of importance without distinction of layers. Additionally, the electronic device 100 may remove a set number of low-importance filters based on the degree of weight reduction set by the user. As described above, when normalizing the importance of each filter, the electronic device 100 ignores filters with an importance less than the importance of the corresponding filter. Therefore, each layer in which lightweighting is completed includes at least one filter (i.e., the importance of each layer is The filter with the highest value) is left behind. Since at least one filter remains in each layer, the layer itself will not be removed. As such, the lightweight method according to the present disclosure has the advantage of reducing information distortion or reducing model accuracy compared to existing lightweight methods because it does not remove layers of the model.

The electronic device 100 may reduce the weight of the original model by removing at least one filter among the plurality of filters based on the normalized importance and degree of lightweighting (S140). The electronic device 100 can sort all filters based on normalized importance without distinction of layers. The electronic device 100 may select at least one filter with low normalized importance based on the degree of weight reduction. The number of filters selected may be determined depending on the degree of lightweighting. For example, as the degree of lightweighting increases, the number of filters selected may also increase.

Figure 2 is a flowchart showing a method (S120) of determining the importance of a filter according to an embodiment of the present disclosure.

Referring to FIG. 2, the electronic device 100 may calculate the importance of each of a plurality of filters using a metric (S121).

The electronic device 100 may recalculate the importance of each of the plurality of filters based on the calculated degree of overlap of importance (S122). For example, the electronic device 100 may recalculate the importance of each filter by converting the importance of each filter into a standard score. In other words, the importance of the recalculated filter may be the standard score of the initially calculated importance. As the importance of the filter is standardized, the degree of importance redundancy can be reflected in the recalculated importance.

The electronic device 100 can adjust the importance of each filter according to the degree of overlap of importance of each filter. For example, the electronic device 100 may increase the size of an importance with low redundancy, and decrease the size of an importance with low redundancy. Filters with low importance with low redundancy may have a large impact on reducing model accuracy when removed, while filters with high importance with high redundancy may have relatively little effect on reducing model accuracy even if they are removed. Therefore, it may be advantageous to preserve the accuracy of the model to remove filters with relatively high redundancy and importance.

Figure 3 is a diagram for explaining a method of determining the importance of a filter according to an embodiment of the present disclosure.

Referring to FIG. 3 , the first layer (L1) may include a first filter 31, a second filter 32, and a third filter 33. The circle shown represents a filter, and the letters (W, c, r) displayed within the circle represent the weight, importance, and recalculated importance of the filter, respectively. The subscript notation of each letter indicates (index value of layer, index value of filter or channel). Weights of filters 31, 32, 33 ( , , ) may be a matrix with a predetermined size where each element is a weight. For example, the weight of the first filter 31 ( ) may be a 3x3 matrix. Meanwhile, in Figure 3, for convenience, a method of determining the importance of a filter based on one layer is explained, but it should be noted that the method described later can be applied to other layers included in the original model.

The electronic device 100 adds the weights of the filters 31, 32, and 33 to the first equation (F1) ( , , ) is applied to determine the importance of the filters (31, 32, 33) ( , , ) can be calculated. Importance ( , , ) may be a scalar value.

The electronic device 100 determines the importance levels in the second equation (F2) ( , , ) can be applied to recalculate the importance of the filters 31, 32, and 33. Accordingly, the electronic device 100 recalculates the importance of the filters 31, 32, and 33 ( , , ) can be obtained. The electronic device 100 can standardize the importance of the filter on a layer-by-layer basis using the second equation (F2). In the second equation (F2), represents the standard deviation of the importance of all filters included in each layer, represents the average of the importance of all filters included in each layer. For the first layer (L1), are the importance levels ( , , ) is the standard deviation of are the importance levels ( , , ) is the average value.

The second equation (F2) may reflect the degree of redundancy of the importance of the filter. For example, it can be said that the smaller the difference between the importance of the first filter included in the first layer and the average of the importance of all filters included in the first layer, the greater the degree of redundancy of the first filter.

Figure 4 is a diagram for explaining a method of normalizing importance according to an embodiment of the present disclosure.

Referring to FIG. 4, the electronic device 100 calculates the recalculated importance levels of the filters 31, 32, and 33 in the third equation (F3). , , ) can be applied to normalize the filters 31, 32, and 33. Accordingly, the electronic device 100 determines the normalized importance of the filters 31, 32, and 33 ( , , ) can be obtained.

Referring to the third equation (F3), normalization is performed on a layer basis. For example, the importance of the first filter 31 ( ), the importance of the filters 31, 32, and 33 included in the first layer (L1) ( , , ) is used, but the importance of filters included in other layers may not be used.

Additionally, as can be seen from the denominator of the third equation (F3), when normalizing the importance of a specific filter, the importance of a filter that is less important than the corresponding filter may not be considered. For example, if the filter with the greatest importance among the filters 31, 32, and 33 is the first filter 31, when normalizing the first filter 31, the importance of the remaining filters 32 and 33 may not be reflected in the denominator of the third equation (F3). Therefore, even if the number of filters included in the first layer (L1) increases, the importance of the filters may not fade depending on the number of filters. In addition, since the importance of the filter with the greatest importance in the first layer (L1) becomes 1 according to normalization, it may not be selected in the process of selecting a filter to be removed later, and the entire first layer (L1) may not be removed. there is.

Meanwhile, in the third equation (F3), the importance of a filter with less importance than the filter is not reflected in the denominator. In another embodiment, in the denominator of the third equation (F3), the importance of a filter with less importance than the filter may be replaced with a very small value close to 0.

The electronic device 100 may perform normalization on the importance of all filters included in the original model. Additionally, the electronic device 100 can sort all filters based on normalized importance regardless of layer. The electronic device 100 may select a number of filters of lower importance determined based on the degree of weight reduction set by the user. The electronic device 100 can reduce the weight of the original model by removing the selected filter. The electronic device 100 may provide a lightweight model to the user. For example, the electronic device 100 may provide download data corresponding to a lightweight model to the user.

Meanwhile, in FIGS. 3 and 4, the number of filters included in the first layer L1 is 3 as an example, but it should be noted that the number of filters is not limited to 3.

Figure 5 is a flowchart showing a method for lightweighting a neural network model according to an embodiment of the present disclosure.

Referring to FIG. 5, the electronic device 100 may receive an original model including a plurality of layers each including a plurality of filters and a metric for determining the importance of each of the plurality of filters (S510). . For example, the original model may include a first layer and a second layer. The first layer may include a plurality of first filters. The second layer may include a plurality of second filters.

The electronic device 100 may determine the importance of each of the plurality of filters using a metric (S520). The method of determining the importance of a filter has already been described in FIG. 1, so detailed description will be omitted.

The electronic device 100 normalizes the importance of each filter based on the importance of the remaining filters, excluding filters with an importance less than the importance of each filter, among a plurality of filters included in each layer of the original model, thereby creating a plurality of filters on a layer-by-layer basis. The importance of each filter can be normalized (S530). Since this step can be clearly understood through S130 in FIG. 1, detailed description will be omitted.

The electronic device 100 selects at least one of the plurality of filters on the interrelated layers based on the normalized importance and a policy related to a method for removing filters included in the interrelated layers among the plurality of layers included in the original model. The original model can be made lighter by removing the filter (S540).

In the present disclosure, a policy related to a method for removing filters included in interrelated layers may be briefly referred to as a 'filter removal policy', 'pruning policy', or 'policy'. Users can input the original model into their device and set pruning policies. The electronic device 100 may receive the original model and pruning policy from the user device.

The pruning policy of the present disclosure may include a first policy, a second policy, a third policy, and a fourth policy. According to the first policy and the second policy, some filters in each layer are determined as candidates for removal, and the final removal target may be determined based on the indices of the filters determined as candidates for removal. For example, the electronic device 100 may determine a first removal candidate among a plurality of first filters and a second removal candidate among a plurality of second filters based on the calculated importance and the degree of weight reduction set by the user. .

According to the first policy, among the removal candidates of each layer, a filter with the same index as the removal candidate of another layer may be determined as the target for removal from that layer. For example, among the first removal candidates, a filter that has the same index as the second removal candidate may be determined as a target for removal in the first layer. The first policy may also be referred to as 'intersection'.

According to the second policy, in addition to the removal candidates of each layer, among filters that are not determined as removal candidates in each layer, filters that have the same index as the removal candidates of other layers may be determined as removal targets. For example, the removal target in the first layer may include a first removal candidate and a first filter that has not been determined as the first removal candidate in the first layer but has the same index as the second removal candidate in the second layer. there is. The second policy may also be referred to as 'union'.

According to the third and fourth policies, the importance corresponding to each index (or channel) can be calculated through an additional calculation between the importance of filters with the same index among filters included in a plurality of layers. Additionally, an index with low importance may be selected based on the degree of lightweighting set by the user and the importance corresponding to each index. A filter with a selected index may be determined as a removal target to be removed from each layer. In the third policy, an additional operation that calculates the importance corresponding to the index may be summation. The third policy may also be referred to as 'sum'. In the fourth policy, an additional operation that calculates the importance corresponding to the index may be averaging. The fourth policy may also be referred to as 'average'.

The electronic device 100 can identify the first layer and the second layer that are related to each other. In the present disclosure, the fact that a plurality of layers are interrelated means that the plurality of layers are connected through a preset operator. Here, the fact that a plurality of layers are connected through a preset operator means that an operation based on the preset operator is performed between the plurality of layers. For example, there may be a case where the first layer and the second layer are connected through an addition operator and an addition operation is performed between the first layer and the second layer. Preset operators may include element-wise operators and concatenation operators. Element-specific operators may include four arithmetic operators.

The electronic device 100 may calculate the importance of each of the plurality of first filters included in the first layer and the importance of each of the plurality of second filters included in the second layer. For example, the electronic device 100 may calculate the importance of each of the plurality of first filters and the importance of each of the plurality of second filters based on the importance determination method of FIG. 3 . Alternatively, the electronic device 100 may normalize the calculated importance by additionally applying the normalization method of FIG. 4.

The electronic device 100 may determine filters to be removed from the first layer and the second layer based on the calculated importance and policy. For example, in the case of the first policy or the second policy, the electronic device 100 determines the first candidate for removal among the plurality of first filters based on the calculated importance and the degree of lightweighting set by the user and selects the plurality of second filters. A second removal candidate can be determined among the filters. The electronic device 100 may select a first removal target from among a plurality of first filters and select a second removal target from a plurality of second filters based on the policy, the first removal candidate, and the second removal candidate.

In the case of the first policy, the electronic device 100 may select a filter that has the same index as the second removal candidate among the first removal candidates as the first removal target. Additionally, the electronic device 100 may select a filter that has the same index as the first removal candidate among the second removal candidates as the second removal target.

In the case of the second policy, the electronic device 100 may determine a filter and a first removal candidate that have the same index as the second removal candidate among the plurality of first filters as the first removal target. Additionally, the electronic device 100 may determine a filter and a second removal candidate that have the same index as the first removal candidate among the plurality of second filters as the second removal target.

In the case of the third policy, the electronic device 100 may determine a filter to be removed from each layer based on the sum of the importance of filters with the same index among the plurality of first filters and the plurality of second filters. In the case of the fourth policy, the electronic device 100 may determine a filter to be removed from each layer based on the average of the importance of filters with the same index among the plurality of first filters and the plurality of second filters.

The above-described policies are methods for commonly applying pruning to interrelated layers. If pruning is applied to the original model based on the policy of the present disclosure, information distortion or significant reduction in model accuracy due to unbalanced pruning of interrelated layers can be prevented.

Figure 6 is a diagram for explaining a first policy according to an embodiment of the present disclosure. Figure 6(a) shows a method for determining a candidate for removal, and Figure 6(b) shows a method for selecting a removal target.

Referring to FIG. 6 , the first layer (L1) may include a first filter 61, a second filter 62, a third filter 63, and a fourth filter 64. The second layer (L2) may include a fifth filter 65, a sixth filter 66, a seventh filter 67, and an eighth filter 68. The first filter 61 and the fifth filter 65 have the same index, the second filter 62 and the sixth filter 66 have the same index, and the third filter 63 and the seventh filter 67 ) has the same index, and the fourth filter 64 and the eighth filter 68 have the same index. The number displayed within the circle representing each filter indicates the importance of that filter. The first layer (L1) and the second layer (L2) are interrelated layers that are connected through an addition operator.

Referring to (a) of FIG. 6, the first filter 61 and the second filter 62 of low importance in the first layer L1 may be determined as first candidates for removal. In the second layer (L2), the sixth filter 66 and the eighth filter 68 of low importance may be determined as second removal candidates. Meanwhile, in this embodiment, the number of filters determined as removal candidates for each layer is 2, but this is only an example and the number of filters determined as removal candidates may vary depending on the degree of lightweighting set by the user. For example, if the degree of lightweighting set by the user increases, the number of filters determined as candidates for removal may increase to 3.

Referring to FIG. 6(b), the second filter 62 and the sixth filter 66 with the same index are selected for removal from the first removal candidates 61 and 62 and the second removal candidates 66 and 68. can be decided. Specifically, in the first removal candidates 61 and 62, the second filter 62 having the same index as the second removal candidates 66 and 68 may be determined as the first removal target. In the second removal candidates 66 and 68, the sixth filter 66 having the same index as the first removal candidates 61 and 62 may be determined as the second removal target.

Figure 7 is a diagram for explaining a second policy according to an embodiment of the present disclosure. Figure 7(a) shows a method for determining a candidate for removal, and Figure 7(b) shows a method for selecting a removal target.

Referring to FIG. 7 , the first layer (L1) may include a first filter 61, a second filter 62, a third filter 63, and a fourth filter 64. The second layer (L2) may include a fifth filter 65, a sixth filter 66, a seventh filter 67, and an eighth filter 68. The first filter 61 and the fifth filter 65 have the same index, the second filter 62 and the sixth filter 66 have the same index, and the third filter 63 and the seventh filter 67 ) has the same index, and the fourth filter 64 and the eighth filter 68 have the same index. The number displayed within the circle representing each filter indicates the importance of that filter. The first layer (L1) and the second layer (L2) are interrelated layers that are connected through an addition operator.

Referring to (a) of FIG. 7, the first filter 61 and the second filter 62 of low importance in the first layer L1 may be determined as first candidates for removal. In the second layer (L2), the sixth filter 66 and the eighth filter 68 of low importance may be determined as second removal candidates. Meanwhile, in this embodiment, the number of filters determined as removal candidates for each layer is 2, but this is only an example and the number of filters determined as removal candidates may vary depending on the degree of lightweighting set by the user. For example, if the degree of lightweighting set by the user increases, the number of filters determined as candidates for removal may increase to 3.

Referring to (b) of FIG. 7, the first removal candidates to be removed from the first layer (L1) are the first removal candidates (61, 62), and the filters (61) included in the first layer (L1). Among 62, 63, and 64, a fourth filter 64 having the same index as the second removal candidates 66 and 68 may be determined. The second removal target to be removed from the second layer (L2) is the second removal candidate (66, 68), and the first of the filters (65, 66, 67, and 68) included in the second layer (L2). A fifth filter 65 having the same index as the removal candidates 61 and 62 may be determined.

Figure 8 is a diagram for explaining a third policy according to an embodiment of the present disclosure. Figure 8(a) shows the sum of the importance of filters included in the first layer (L1) and the second layer (L2). Figure 8(b) shows a method of selecting a removal target.

Referring to FIG. 8 , the first layer (L1) may include a first filter 61, a second filter 62, a third filter 63, and a fourth filter 64. The second layer (L2) may include a fifth filter 65, a sixth filter 66, a seventh filter 67, and an eighth filter 68. The first filter 61 and the fifth filter 65 have a first index (i1), the second filter 62 and the sixth filter 66 have a second index (i2), and the third filter ( 63) and the seventh filter 67 may have a third index (i3), and the fourth filter 64 and the eighth filter 68 may have a fourth index (i4). The number displayed within the circle representing each filter indicates the importance of that filter. The first layer (L1) and the second layer (L2) are interrelated layers that are connected through an addition operator.

Figure 8(a) shows the sum of importance for each index. Based on the importance of the first filter 61 and the fifth filter 65 having the first index (i1), the importance (i.e., 32) corresponding to the first index (i1) may be calculated. Based on the importance of the second filter 62 and the sixth filter 66 having the second index (i2), the importance (i.e., 30) corresponding to the second index (i2) may be calculated. Based on the importance of the third filter 63 and the seventh filter 67 having the third index (i3), the importance (i.e., 50) corresponding to the third index (i3) may be calculated. Based on the importance of the fourth filter 64 and the eighth filter 68 having the fourth index (i4), the importance (i.e., 40) corresponding to the fourth index (i4) may be calculated.

Referring to (b) of FIG. 8, the first filter 61 and the fifth filter 65 corresponding to the first index (i1) of low importance may be determined to be removed. Additionally, the second filter 62 and the sixth filter 66 corresponding to the second index i2 may be determined to be removed. Meanwhile, in this embodiment, the number of filters determined to be removed in each layer is 2, but this is only an example and the number of filters determined to be removed may vary depending on the degree of lightweighting set by the user. For example, if the degree of lightweighting set by the user increases, the number of filters determined to be removed may increase to 3. Alternatively, when the degree of lightweighting set by the user decreases, the number of filters determined to be removed may decrease to 1.

Figure 9 is a diagram for explaining a fourth policy according to an embodiment of the present disclosure. Figure 9(a) shows the average importance of filters included in the first layer (L1) and the second layer (L2). Figure 8(b) shows a method of selecting a removal target.

Referring to FIG. 9 , the first layer (L1) may include a first filter 61, a second filter 62, a third filter 63, and a fourth filter 64. The second layer (L2) may include a fifth filter 65, a sixth filter 66, a seventh filter 67, and an eighth filter 68. The first filter 61 and the fifth filter 65 have a first index (i1), the second filter 62 and the sixth filter 66 have a second index (i2), and the third filter ( 63) and the seventh filter 67 may have a third index (i3), and the fourth filter 64 and the eighth filter 68 may have a fourth index (i4). The number displayed within the circle representing each filter indicates the importance of that filter. The first layer (L1) and the second layer (L2) are interrelated layers that are connected through an addition operator.

Figure 9(a) shows the average importance of each index. Based on the importance of the first filter 61 and the fifth filter 65 having the first index (i1), the importance (i.e., 16) corresponding to the first index (i1) may be calculated. Based on the importance of the second filter 62 and the sixth filter 66 having the second index (i2), the importance (i.e., 15) corresponding to the second index (i2) may be calculated. Based on the importance of the third filter 63 and the seventh filter 67 having the third index (i3), the importance (i.e., 25) corresponding to the third index (i3) may be calculated. Based on the importance of the fourth filter 64 and the eighth filter 68 having the fourth index (i4), the importance (i.e., 20) corresponding to the fourth index (i4) may be calculated.

Referring to FIG. 9(b), the first filter 61 and the fifth filter 65 corresponding to the first index (i1) of low importance may be determined to be removed. Additionally, the second filter 62 and the sixth filter 66 corresponding to the second index i2 may be determined to be removed. Meanwhile, in this embodiment, the number of filters determined to be removed in each layer is 2, but this is only an example and the number of filters determined to be removed may vary depending on the degree of lightweighting set by the user. For example, if the degree of lightweighting set by the user increases, the number of filters determined to be removed may increase to 3. Alternatively, when the degree of lightweighting set by the user decreases, the number of filters determined to be removed may decrease to 1.

Figure 10 is a block diagram showing the configuration of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 10 , the electronic device 100 may include a communication interface 110, a memory 120, and a processor 130. For example, the electronic device 100 may be implemented as a physical server or a cloud server.

The communication interface 110 includes at least one communication circuit and can communicate with various types of external devices. For example, the communication interface 110 may receive information about the original model, degree of lightweighting, and metrics from an external device. The external device may be a user device. User devices may include personal computers and mobile devices. The user device can receive input of the original model, degree of lightweighting, and metrics from the user.

The communication interface 110 is a Wi-Fi communication module, a cellular communication module, a 3G (3rd generation) mobile communication module, a 4G (4th generation) mobile communication module, a 4th generation LTE (Long Term Evolution) communication module, and a 5G (5th generation) mobile communication module. It may include at least one of a module and wired Ethernet.

The memory 120 may store an operating system (OS) for controlling the overall operation of the components of the electronic device 100 and commands or data related to the components of the electronic device 100. In particular, the memory 120 can store instructions for lightweighting the original model. The memory 120 may be implemented as non-volatile memory (ex: hard disk, solid state drive (SSD), flash memory) or volatile memory.

The processor 130 is electrically connected to the memory 120 and can control the overall functions and operations of the electronic device 100. The processor 130 can control the electronic device 100 by executing instructions stored in the memory 120.

For example, the processor 130 may obtain the original model, the degree of lightweighting and metrics to be applied to the original model. The processor 130 may receive the original model, degree of lightweighting, and metrics from the user device through the communication interface 110. The original model may include multiple layers. A layer may include multiple filters (or kernels).

The processor 130 may determine the importance of each of the plurality of filters using a metric. The processor 130 may calculate the importance of each of the plurality of filters using a metric and recalculate the importance of each of the plurality of filters based on the degree of redundancy of the calculated importance. The importance of the filter can be adjusted according to the degree of redundancy. For example, the greater the degree of redundancy of the calculated importance of the first filter, the smaller the recalculated importance of the first filter may be. The processor 130 may calculate the degree of redundancy of the calculated importance of the first filter based on the calculated importance of the first filter and the average and standard deviation of the calculated importance of the filters included in the first layer.

Processor 130 may normalize the recalculated importance. For example, when normalizing the importance of the first filter included in the first layer, the processor 130 excludes the filter with an importance less than the importance of the first filter among the filters included in the first layer. The importance of the first filter can be normalized based on the importance.

The processor 130 may sort the plurality of filters in order of importance. The processor 130 may select at least one filter in order of decreasing importance from a plurality of filters determined based on the degree of lightweighting. Alternatively, a critical importance for selecting a filter to be removed may be determined based on the degree of weight reduction set by the user. At this time, the processor 130 may select a filter whose importance is less than the critical importance. The processor 130 can reduce the weight of the original model by removing the selected filter. Accordingly, the processor 130 can create a lightweight model. Each of the plurality of layers included in the lightweight model may include at least one filter.

The processor 130 may obtain a policy related to a method for removing the original model and filters included in interrelated layers among a plurality of layers included in the original model. Additionally, the processor 130 can obtain a degree of weight reduction to lighten the original model.

The processor 130 may analyze the original model to identify interrelated first and second layers. For example, when the first layer and the second layer are connected through an element-specific operator, the processor 130 may determine that the first layer and the second layer are related to each other.

The processor 130 may calculate the importance of each of the plurality of first filters included in the first layer and the importance of each of the plurality of second filters included in the second layer. The processor 130 may determine filters to be removed from the first layer and the second layer based on the calculated importance and policy. The method for determining the filter to be removed for each policy has been described above, so detailed description thereof will be omitted.

The various embodiments described above may be implemented in a recording medium that can be read by a computer or similar device using software, hardware, or a combination thereof. In some cases, embodiments described herein may be implemented with a processor itself. When implemented as software, embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

Computer instructions for performing processing operations according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. Computer instructions stored in such a non-transitory computer-readable medium, when executed by a processor, can cause a specific device to perform processing operations according to the various embodiments described above.

A non-transitory computer-readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as registers, caches, and memories. Specific examples of non-transitory computer-readable media may include CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, etc.

A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory storage medium' simply means that it is a tangible device and does not contain signals (e.g. electromagnetic waves). This term is used to refer to cases where data is semi-permanently stored in a storage medium and temporary storage media. It does not distinguish between cases where it is stored as . For example, a 'non-transitory storage medium' may include a buffer where data is temporarily stored.

Methods according to various embodiments disclosed in this document may be provided and included in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or through an application store (e.g. Play StoreTM) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smartphones) or online. In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) is stored on a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server. It can be temporarily stored or created temporarily.

In the above, preferred embodiments of the present disclosure have been shown and described, but the present disclosure is not limited to the specific embodiments described above, and may be used in the technical field pertaining to the disclosure without departing from the gist of the disclosure as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical ideas or perspectives of the present disclosure.

100: electronic device 110: communication interface
120: memory 130: processor

Claims (14)

In a method for lightweighting a neural network model performed by an electronic device,
Receiving an original model including a plurality of layers each including a plurality of filters and a metric for determining the importance of each of the plurality of filters;
determining the importance of each of the plurality of filters using the metric; and
A policy related to a method for removing together filters included in interrelated layers among a plurality of layers included in the original model, and removing at least one filter among a plurality of filters on the interrelated layers based on the importance. Including the step of reducing the weight of the original model,
method. According to claim 1,
The interrelated layers are:
A first layer including a plurality of first filters and a second layer including a plurality of second filters,
The at least one filter to be removed is,
Determining a first removal candidate among the plurality of first filters and a second removal candidate among the plurality of second filters based on the normalized importance and the degree of lightweighting set by the user,
Determined by selecting a first removal target from among the plurality of first filters and selecting a second removal target from the plurality of second filters based on the policy, the first removal candidate, and the second removal candidate.
method. According to clause 2,
When the policy is a first policy, the first removal target is selected by a filter that has the same index as the second removal candidate among the first removal candidates, and the second removal target is the first removal target among the second removal candidates. The removal candidate and index are selected using the same filter.
method. According to clause 2,
When the policy is a second policy, the first removal target includes a filter with the same index as the second removal candidate among the plurality of first filters and the first removal candidate, and the second removal target includes the plurality of first removal candidates. Among the second filters, a filter having the same index as the first removal candidate and the second removal candidate are included.
method. According to claim 1,
The at least one filter to be removed is:
When the policy is a third policy, it is determined based on the sum of the importance of filters with the same index among the plurality of filters.
method. According to claim 1,
The filter to be removed is,
When the policy is the fourth policy, it is selected based on the average of the importance of filters with the same index among the plurality of filters.
method. According to claim 1,
The interrelated layers are:
Connected through preset operators,
The preset operators are:
Contains element-wise operators
method. In an electronic device for lightweighting a neural network model,
a memory storing at least one instruction; and
Including a processor;
The processor executes the at least one instruction,
An original model including a plurality of layers each including a plurality of filters and a metric for determining the importance of each of the plurality of filters are input,
Determine the importance of each of the plurality of filters using the metric,
A policy related to a method for removing together filters included in interrelated layers among a plurality of layers included in the original model, and removing at least one filter among a plurality of filters on the interrelated layers based on the importance. Lightening the original model,
Electronic devices. According to clause 8,
The interrelated layers are:
A first layer including a plurality of first filters and a second layer including a plurality of second filters,
The at least one filter to be removed is:
Determining a first removal candidate among the plurality of first filters and a second removal candidate among the plurality of second filters based on the normalized importance and the degree of lightweighting set by the user,
Determined by selecting a first removal target from among the plurality of first filters and selecting a second removal target from the plurality of second filters based on the policy, the first removal candidate, and the second removal candidate.
Electronic devices. According to clause 9,
When the policy is a first policy, the first removal target is selected by a filter that has the same index as the second removal candidate among the first removal candidates, and the second removal target is the first removal target among the second removal candidates. The removal candidate and index are selected using the same filter.
Electronic devices. According to clause 9,
When the policy is a second policy, the first removal target includes a filter with the same index as the second removal candidate among the plurality of first filters and the first removal candidate, and the second removal target includes the plurality of first removal candidates. Among the second filters, a filter having the same index as the first removal candidate and the second removal candidate are included.
Electronic devices. According to clause 8,
The at least one filter to be removed is,
When the policy is a third policy, it is determined based on the sum of the importance of filters with the same index among the plurality of filters.
Electronic devices. According to clause 8,
The filter to be removed is,
When the policy is the fourth policy, it is selected based on the average of the importance of filters with the same index among the plurality of filters.
Electronic devices. According to clause 8,
The interrelated layers are:
Connected through preset operators,
The preset operators are:
Contains element-wise operators
Electronic devices.