JP3368774B2 - neural network - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a large scale logic circuit,
In a neural network applicable to fields such as pattern recognition, associative memory, data conversion, and image processing, by adjusting the parameters related to learning by observing the output error situation of incorrect answer output, the desired output signal can be obtained quickly and stably. It relates to a multi-valued neural network that can obtain

[0002]

2. Description of the Related Art 1 of the prior art of neural networks
One of them is a multilayer neural network, and a back propagation algorithm is widely used as a learning method using a teacher signal of this neural network. In the learning process using this algorithm, after the weighting coefficient is initialized, a teacher signal T (teacher signal element, T ₁ ,
From T ₂ , ..., T _M ), the output unit signal from the output layer with respect to the learning input signal input to the input layer is subtracted through a subtractor to obtain an error signal, and the output unit signal and error signal of each layer are obtained. Based on and, the weighting coefficient between layers is updated so as to minimize the power of the error signal, and learning is performed. The learning including the weighting coefficient adaptive control is executed for all the learning input signals, and is repeated until a converged state in which a correct output is obtained under the required accuracy.

When the error power is minimized in this learning process, the multi-valued output unit signal for the learning input signal coincides with the multi-valued teacher signal and enters a converged state. However, once the error power drops to the local minimum (local minimum), if it is very stable, the error power will not be the minimum even if it is learned after that.
In order to reach the minimum state, there are problems that the number of times of learning increases remarkably and that the convergence characteristic has an initial value dependence of the weighting coefficient.

As a second conventional technique, for example, in order to improve the learning speed of a binary neural network with respect to a binary teacher signal T of 0 and 1, "Parallel Distributed"
Processing "DE Rumelhart, MIT Press. There is a method described here in the multilayer neural network using the learning of the conventional method shown in FIG. Learning is started after setting a new teacher signal T'and performing initial setting by the control signal from the operation mode controller 9. From the teacher signal T'from the terminal 3, learning input from the terminal 2 is started. The output unit signal of the multilayer neural network 1 for the signal is subtracted via the subtracter 4 to obtain an error signal, which is input to the weighting factor controller 5, and the backpropagation algorithm using the learning parameter including the learning coefficient and the inertia coefficient. The weighting factor is updated by the method, and the process of setting again in the multilayer neural network 1 is repeatedly learned. A binary output unit signal is obtained from the output unit signal via the binary threshold circuit 6 for converting the binary signal from the teacher signal T'through the binary threshold circuit 7 for converting the binary signal using the binary discrimination threshold. When the teacher signal T is obtained and the coincidence detector 8 detects a state where these signals completely coincide with each other, it is determined that the multilayer neural network 1 has converged in the binary space, and the learning is completed via the operation mode controller 9.

Thus, the teacher signal T'is set to 0.1 and 0.
It is stated that setting to 9 reduces the number of learning times for convergence as compared with the case of binary teacher signals of 0 and 1. This is the sigmoid function
This is because when the input approaches 0 or 1, the gradient becomes very small and the update rate of the weighting factor becomes small. Therefore, the learning speed for convergence is improved by setting the teacher signals to 0.1 and 0.9 and increasing the gradient. However, it may fall into a stable local minimum in the same way, and once it falls, learning does not progress, and although it converges, the teacher signals are 0.1 and 0.9, and the output unit signal is the final value of these values. However, there is a drawback that the generalization ability is deteriorated because the margin for outputting the correct answer becomes small.

As a third conventional technique, an error perturbation type learning method (Japanese Patent Application No. 7-77168, name: neural network learning method) is taken as an example to clarify problems in the learning process. FIG. 3 shows a configuration example of the error perturbation learning method. The multilayer neural network 1 performs learning using the learning input signal from the terminal 2 and the binary teacher signal T from the terminal 3. In the weighting factor update error signal generator 10, for the output unit in which the correct binary output unit signal is detected by the correctness determination signal of the output unit from the learning state determiner 11, the output unit from the binary teacher signal is detected. When the absolute value of the error signal obtained by subtracting the signal is smaller than the first threshold D1 given in advance, for example, as in the equation (1), the error signal has a polarity opposite to that of the error signal. A weighting coefficient update error signal having an amplitude of D1 (however, ≧ 0) or less, the amplitude of which decreases with distance from the teacher signal T, is generated and output, and when the absolute value of the error signal is equal to or more than the threshold, the same polarity is set. A weighting factor update error signal having an amplitude as small as D1 is generated and output.

On the other hand, in the output unit giving an incorrect output unit signal, for example, as shown in the equation (2), the amplitude is made smaller by the first constant D2 given in advance and outputted as the weighting coefficient update error signal. Let The weighting factor controller 5 uses the weighting factor update error signal to correct the weighting factor by the back propagation algorithm,
The learning is repeated so that the power of the error signal is minimized. In the learning state determiner 11, it is considered that the binary output unit signals for all the learning input signals have converged when they match the binary teacher signal and become correct, and the learning end signal is sent out via the operation mode controller 12. Then, the learning of the multilayer neural network 1 is terminated. Further, in some cases, for all the learning, within the margin of obtaining the correct output calculated from the output unit signal and the teacher signal with respect to the output unit which is sending out the correct binary output unit signal. Sometimes, when the minimum margin for the input signal, that is, the minimum margin, exceeds the preset fourth threshold, the learning is considered to have converged and the learning may be terminated. The operation mode controller 12 controls each initial setting of the multilayer neural network 1, the weighting factor controller 5, and the learning state determiner 11, and further controls the start and end of learning.

Here, an equation showing one method of generating the weighting coefficient update error signal in the weighting coefficient update error signal generator 10 when the binary teacher signal is used is shown below.

When the binary output unit signal of the output unit is correct, Em = Tm-Ym-D1.sgn (Tm-Ym) (1)

When the binary output unit signal of the output unit is incorrect, Em = Tm-Ym-D2.sgn (Tm-Ym) (2)

Here,     sgn (x) = 1: x ≧ 0,                 = -1: x <0, (3)

M is an output unit number (1≤m≤M), E
m is a weight coefficient update error signal of the output unit m, Tm is a multivalued teacher signal (0 or 1 in the case of binary) of the output unit m, Ym is an output unit signal of the output unit m, D1
Is a first threshold and is 0 or a positive constant of 0.5 or less, and D2 is a first constant and is 0 or a positive constant of 0.5 or less.

The first in the weighting factor update error signal generator 10
If the threshold D1 and the first constant D2 of are set to be moderately large, they will hardly fall into a very stable local minimum in the learning process, and even if they fall, they can easily escape from this state and all 2 The value output unit signal is sent to the convergence state and learning is fast. However, especially when the number of learning input signals is very large, and when the number of learning input signals exceeds several thousand, the boundary area surface for correctly classifying the learning input signals according to the binary teacher signal becomes very complicated, the learning coefficient and inertia Depending on the setting of the learning parameter of the coefficient, after the learning progresses and the number of incorrect answer binary output unit signals is considerably reduced, the incorrect answer 2 is gradually increased.
There is a drawback that the number of value output unit signals decreases slowly, and the number of learning times to reach a converged state increases. this is,
This is probably because the margin for giving the correct answer is as narrow as D1.

As described above, in the conventional learning method, it is easy to fall into the state of the local minimum as described above, and since it is difficult to get out of the local minimum state, it is not possible to easily reach a completely converged state with a small number of learnings. There are drawbacks such as. In addition, the conventional error perturbation type learning method improves the convergence property remarkably compared with the previous conventional method, however, when learning a large-scale neural network with a large number of input signals for learning, The boundary area for correctly dividing the input signal becomes very complicated, and it does not fall into the state of the local minimum, but the margin for outputting the correct answer is narrowed by D1 and all the correct binary output unit signals are transmitted. There is a drawback that the number of learning times required to reach such a converged state increases. Although a binary neural network has been described here as an example, these have the same drawbacks even when a multivalued teacher signal is used.

[0015]

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention As described above,
In the conventional learning process of the neural network using the multivalued teacher signal, when the weighting coefficient is updated so that the neural network outputs the desired multivalued output unit signal corresponding to the learning input signal, Has a drawback in that the number of times of learning until reaching a converged state in which a desired multi-level output unit signal is transmitted, that is, the number of times of learning is very large.

An object of the present invention is to solve the above-mentioned problems, to converge with an extremely small number of learning times as compared with a conventional learning method, etc., and to complete learning stably at a high speed. It is an object of the present invention to provide a new learning method for a multi-valued neural network that can easily obtain a desired multi-valued output unit signal for a signal and has a high generalization ability.

[0017]

In order to solve the above problems, in a neural network for learning using a teacher signal, an error signal obtained by subtracting the output unit signal of the correct output unit from the teacher signal If the absolute value is less than or equal to the first threshold D1, a weighting coefficient update error signal having a polarity opposite to that of the error signal and having an amplitude that decreases with distance from the teacher signal is generated.
Is larger than the threshold value D1 of the error signal, at least a weight coefficient update error signal of the same polarity having an amplitude smaller than the error signal is generated, the weight coefficient is updated using the weight coefficient update error signal, and learning is performed to obtain an incorrect solution. When the maximum value of the absolute value of the error signal in the output unit with respect to all the learning input signals is smaller than the second threshold, the first threshold D1 is reduced to perform learning.

Further, in the learning method of the neural network having the above characteristics, the first threshold D1 is set to zero to start learning, and the maximum value is reached at a predetermined number of times of learning. In the case of being equal to or higher than the third threshold, the neural network is characterized in that the first threshold is set to a non-zero value for learning.

In the neural network of the present invention, the operation will be clarified by taking a multivalued neural network to which multivalues are applied as an example. In a learning method of a multi-valued neural network, an output unit in which an error signal obtained by subtracting from the multi-valued teacher signal is equal to or lower than a first threshold D1 and is very close to the multi-valued teacher signal and gives a correct multi-valued output unit signal. , A weighting coefficient update error signal whose amplitude decreases with distance from the teacher signal having a polarity opposite to that of the error signal, and an error signal whose error signal is larger than the first threshold and relatively close to the multivalued teacher signal. In the output unit giving the output unit signal, each weighting factor is updated and learned using at least the error signal and the weighting factor update error signal having the smaller amplitude as D1. In the learning process, the maximum error of the output unit in which the absolute value of the error signal of the incorrect output unit with respect to all the learning input signals is maximum is detected, and when this is less than or equal to the given second threshold, Since it rarely falls into the stable local minimum capture state, thereafter, the error perturbation parameters, that is, the first threshold D1 and the first constant D2 are learned to be smaller or zero. This improves the margin of sending the correct multilevel output unit signal, accelerates the learning speed, and finally converges with a small number of times of learning, so that a higher generalization characteristic can be realized.

As described above, the learning method of the neural network of the present invention does not get caught in a very stable local minimum capture state even when learning a large amount of input signals for learning, and is more effective than the conventional method. Since a desired multi-valued output unit signal can be easily obtained by quickly and surely converging to, it is possible to freely design a large-scale multi-valued neural network. In addition, we have realized a logic system with a learning function that requires re-learning in real time, and a multi-valued logic system using an input signal with an extremely large number of input signal elements and a multilayer neural network with a large number of units. it can. Furthermore, it is difficult to obtain a desired output signal because the conventional method cannot achieve stable, fast, and reliable convergence, and it is also easy to design data conversion, expert systems, etc. It can be realized.

[0021]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of a binary neural network using the learning method of the neural network of the present invention will be given below, and its configuration and operation will be described in detail when a binary teacher signal is used. .

One of the learning processes of the binary neural network using the learning method of the neural network of the present invention.
An example is shown in FIG. The input signal from the terminal 2 is input to the input unit of the input layer, and the output unit signal is output from the output unit of the output layer. Multilayer neural network 1, binary teacher signal, output unit signal, and output Signal, and a weight coefficient update error signal generator 10 that outputs a weight coefficient update error signal, a binary threshold circuit 6 that binarizes the output unit signal to obtain a binary output unit signal, and an input weight coefficient update error A weight coefficient controller 14 for updating and setting the weight coefficient of the multilayer neural network 1 using the signal and the set learning parameter, receives the binary teacher signal and the binary output unit signal, and inputs the binary teacher signal and the binary teacher signal. By comparing with the value output unit signal, the correct and incorrect answers in the output unit are detected and the correctness / wrong judgment signal, the binary teacher signal and the binary output unit are detected. Output accuracy judgment unit sends an output match signal indicating a match between the bets signal 1
3. For inputting the correct / wrong judgment signal, the binary teacher signal and the output unit signal from the output correct / wrong judgment unit 13, for all learning of the absolute value of the difference between the output unit signal and the binary teacher signal in the incorrect output unit. As the absolute value of the maximum value for the input signal, that is, the incorrect output unit maximum error detector 16 that detects the incorrect output unit maximum error, and the difference between the output unit signal and the threshold value of the binary threshold circuit 6 in the correct output unit. The minimum value of the given margin for all learning input signals, that is, the correct output unit minimum margin detector 17 that detects the correct output unit minimum margin, the output coincidence signal from the output correctness determination unit 13, the incorrect output unit maximum error, and Learning state determiner 18 to which the correct output unit minimum margin is input, multilayer neural network 1, weighting factor controller 5, and output positive Decision unit 13, incorrect output unit maximum error detector 16, the correct answer output unit minimum margin detector 1
7 and the learning state determiner 18 are initialized, the learning start control is performed, and the end control of each part is performed based on the learning end signal from the learning state determiner 18.
Composed of and.

Next, the operation in the learning process will be described. The multilayer neural network 1 performs learning using a learning input signal from the terminal 2 and a binary teacher signal from the terminal 3. In the weighting coefficient update error signal generator 10, for the output unit in which the incorrect binary output unit signal is detected by the correctness determination signal from the output correctness determination unit 13,
The error signal obtained by subtracting the output unit signal from the binary teacher signal is used as the first constant D for the amplitude as shown in equation (2).
Reduce by about 2 and output as a weighting factor update error signal,
On the other hand, in the output unit giving the correct output unit signal, the first value given the absolute value of the error signal
If the absolute value of the error signal is greater than the first threshold D1, the weight coefficient update error signal having a reverse polarity and having a smaller value with increasing distance from the teacher signal is generated and output. A weight coefficient update error signal having a small amplitude is generated and output.

Here, if D1 is defined as the convergence region, it can be similarly applied to a neural network that handles a continuous amount of teacher signals. The weighting factor controller 5 uses the weighting factor update error signal to correct the weighting factor by the back propagation algorithm, and repeats learning so that the power of the error signal is minimized. Also, the first constant D2 is D1
When it is captured by a local minimum very rarely under 0, it works to escape quickly, but it rarely occurs.
Since D1 also works to escape, D2 = 0 may be set.

In the learning state determiner 18, when the maximum error output unit maximum error from the incorrect output unit maximum error detector 16 falls below the second threshold, the learning progresses and it is difficult to be captured by a stable local minimum. Therefore, the error coefficient perturbation parameter change request output is sent to the weight coefficient update error signal generator 10 and the error perturbation parameter in the weight coefficient update error signal generator 10 is set to zero (D1 = 0). , D2 = 0), and the learning is continued with the error perturbation stopped. After that, it is usually not captured in a stable local minimum. However, when a large D1 is set, if this is suddenly set to zero, the state of the weighting factor may be completely initialized, so that there is a possibility of being captured by the local minimum thereafter. Therefore, in this case, a small minimum value D1 is once set and learned, and when the incorrect output unit maximum error becomes equal to or less than the second threshold again, D1 = 0 is set and learning is performed. It can be converged without being captured by.

In either case, after the error perturbation parameters D1 and D2 are finally learned as zero, the output correctness / error determination unit 13 eliminates the error and the binary teacher signal and the binary output unit signal match. When the converged state is detected, the coincidence signal is sent to the learning state determiner 18. Learning state determiner 1
In 8, the weighting factor controller 5 continues until the correct output unit minimum margin, which is the output signal of the correct output unit minimum margin detector 17 in the above-mentioned convergent state, exceeds the predetermined fourth threshold. When the updating of the weighting factor is continued and once the threshold is exceeded, the learning state determiner 18 sends a learning end signal through the operation mode controller 15 to end the learning of the multilayer neural network 1. This avoids unnecessary learning and
Since D1 = 0, a very high generalization capability can be obtained without deterioration of the margin for transmitting the correct binary output unit signal.

The above-mentioned learning method can obviously and stably converge at high speed as compared with the conventional method. However, depending on the initial value of the weighting factor, there is a case where the convergence does not occur in a stable local minimum without introducing an error perturbation and the convergence is fast with a very small number of learnings. Therefore, in the above-mentioned neural network learning method, the error perturbation parameter is set to zero, that is, the first threshold D1 and the first constant D2 are set to zero, and learning is started. If the maximum error of the correct output unit is equal to or higher than the third threshold, it is very likely to be captured by the local minimum. Therefore, at least the first threshold D1 is set to a non-zero value, and the error as described above is set. Introduce perturbations to continue learning and adjust error perturbation parameters to converge. On the other hand, if it is smaller than the third threshold, it is rarely captured by the local minimum. Therefore, even if learning is continued with D1 and D2 set to zero without introducing an error perturbation, fast convergence is achieved. Good.

In the learning state determiner 18, the influence of the initial value of the weighting factor on the local minimum capture is detected and determined by using the maximum error of the incorrect output unit as described above, so that the convergence state is reached even faster. It can be done. According to previous simulations,
After starting the learning by setting the threshold D1 and the first constant D2 of 0 to zero, the incorrect output unit maximum error becomes 1 at the time when the learning count is 5, for example, due to the influence of the initial value of the weighting coefficient. If it has a very close value and exceeds the third threshold, for example, 0.98, even if the learning is continued, it is possible that the maximum value of the incorrect output unit is very close to 1. It has been confirmed by simulation that, when the number of incorrect answers continues to decrease, it is very likely that a stable local minimum will be caught suddenly. Further, when the value has a value apart from 1, all the binary output unit signals smoothly and quickly become correct and converge at high speed as the learning thereafter progresses. Therefore, in the learning state determiner 18, after starting learning,
Whether the incorrect output unit maximum error from the incorrect output unit maximum error detector 16 is compared with the third threshold at a predetermined number of times of learning, and whether an error perturbation is introduced in the subsequent learning, that is, , It is possible to reach the convergent state at a higher speed by determining whether or not at least D1 ≠ 0 and performing the subsequent learning.

Although a series of learning methods for the learning input signal have been described above, there is another iterative learning method called a learning input signal addition type iterative learning method. That is, for a given learning input signal, once a convergence state is reached in which correct outputs are obtained for all learning input signals, and when the learning is completed by exceeding the fourth threshold, the generalization capability Input the test input signal for evaluation,
The correct / incorrect answer of the binary output unit signal to this is judged, and if there is an error, only the test input signal in which the error has occurred is transferred as the learning input signal, and the learning input signals up to that point are transferred. In addition, iterative learning is performed again for the new learning input signal, and if there is no error in the test input signal, the iterative learning ends. In this iterative learning method, the initial value of the weighting coefficient is set every time iterative learning is performed, but the influence of the initial value of the above weighting coefficient on the local minimum capture is detected using the incorrect output unit maximum error. Therefore, controlling the error perturbation parameter by making a decision is very effective in speeding up the iterative learning.

Although not described in detail in this embodiment, an incorrect output unit minimum error is obtained in addition to the incorrect output unit maximum error, and the learning state determiner 18 determines the incorrect output unit minimum error or converges. If the minimum margin of the correct output unit in the absence state exceeds any of the newly prepared thresholds, it is considered that the stable local minimum capture state is captured, and the first threshold D1 and Even if the first constant D2 is once increased, that is, the error perturbation parameters D1 and D2 are increased, and the local minimum capture state is escaped, D1 and D2 are decreased again and the error perturbation parameter is decreased to converge. Good. Also, the error perturbation parameters D1 and D2
May be changed and learning parameters such as a learning coefficient and an inertia coefficient may be changed.

The learning method of the present invention has been described by taking a binary example as a multi-valued neural network. However, in a neural network which handles a continuous amount of teacher signals in addition to this, it converges to a teacher signal and a correct answer is obtained. Applying the same method as above, by providing a pre-given area to consider, if the output unit signal is in that area, correct answer, if not, incorrect answer It can be widely applied to learning of neural networks using general teacher signals.

In the above embodiments, the explanation has been made on the premise of the multilayer neural network, but a neural network other than the above may be used as long as it is a neural network which performs learning by using a teacher signal.

[0033]

As described above, in the learning method of the neural network of the present invention, the error between the multilevel teacher signal and the multilevel output unit signal is corrected in the learning process by adjusting the error perturbation parameter in time. It is not a very stable local minimum or local minimum that does not change, but it does not fall into a gradual state,
It is possible to quickly and surely converge without reducing the margin for obtaining the correct output due to the error perturbation, and it is possible to realize a very high generalization ability. Therefore, the number of times of learning can be reduced by about 10 times as compared with the conventional error perturbation method. Further, since it does not fall into the local minimum, it is possible to stably converge to the optimum state by using the minimum required number of intermediate units or intermediate layers, and it is possible to reduce the hardware scale and the amount of calculation.

The multi-valued neural network using the learning method of the present invention uses a smaller number of intermediate layer units or intermediate layers than the conventional method, has no initial dependence, and has a low calculation accuracy. Can converge at high speed and stably, and a desired multilevel output signal can be transmitted.
From this, it is necessary to freely design and realize a large-scale multivalued logic circuit or the like, which is difficult to be realized by the conventional technology, in a short time by the multivalued neural network using the present invention, and rapid learning so far. In addition, it has a very wide range of effects such as wide application to artificial intelligence systems and search systems, pattern recognition, data conversion, data compression, multi-valued image processing, and communication systems that require complete convergence. There is.

[Brief description of drawings]

FIG. 1 is a configuration example of a learning process of a multilayer neural network using a learning method of a neural network of the present invention in an embodiment.

FIG. 2 is a configuration example of a learning process in a multilayer neural network according to a conventional learning method.

FIG. 3 is a configuration example of a learning process in a multilayer neural network according to a conventional error perturbation learning method.

[Explanation of symbols]

1 Multilayer neural network 2 Input signal input terminal 2 ₁ Input signal unit input terminal 2 ₂ Input signal unit input terminal 2 _N Input signal unit input terminal 3 Teacher signal input terminal 3 ₁ Teacher signal unit input terminal 3 ₂ Teacher signal unit input terminal 3 _M teacher signal unit input terminal 4 subtracter _{4 1} subtractor _{4 second} subtracter _{4 M} subtracter 5 weight coefficient controller 6 binary threshold circuit ₆₁ binary threshold circuit _{6 2} binary threshold circuit _{6 M} binary threshold circuit 7 Binary Threshold Circuit 7 ₁ Binary Threshold Circuit 7 ₂ Binary Threshold Circuit 7 _M Binary Threshold Circuit 8 Match Detector 9 Operation Mode Controller 10 Weighting Factor Update Error Signal Generator 10 ₁ Weighting Factor Update Error Signal Generator 10 ₂ weight coefficient updating error signal generator 10 _M weighting coefficient updating error signal generator 11 learning state Joki 12 operation mode controller 13 outputs accuracy determination unit 14 weighting coefficient controller 15 operation mode controller 16 Incorrect output unit maximum error detector 17 correct output units minimum margin detector 18 learning state determiner

Continuation of the front page (56) Reference JP-A-4-15860 (JP, A) JP-A-4-251392 (JP, A) JP-A-3-218559 (JP, A) JP-A-8-249304 (JP , A) JP-A-2-277180 (JP, A) JP-A-4-30280 (JP, A) Yotaro Yatsuka, “Fast convergence of binary three-layer neural network by error perturbation”, IEICE 1995 General Conference Proceedings, Japan, The Institute of Electronics, Information and Communication Engineers, March 10, 1995, Volume 1, p. 24 Yotaro Yatsuka, “Detection and generalization of local minimum in error perturbation type binary three-layer neural network”, Proc. Of Information and Systems Society Conference of the Institute of Electronics, Information and Communication Engineers 1995, Japan, Japan Society The Institute of Electronics, Information and Communication Engineers, August 15, 1995, pp. 14 Enomoto Univ., Yotarou Yatsuka, "Proposal of learning data addition type sequential learning using generalization ability of binary three-layer error perturbation switching neural network" , Proceedings of the 1996 Information and Systems Society Conference of the Institute of Electronics, Information and Communication Engineers, Japan, The Institute of Electronics, Information and Communication Engineers, August 30, 1996, pp. 30 Dai Enomoto, Yotaro Yatsuka, "Improvement of generalization characteristics of binary 3-layer neural network by error perturbation switching method", Proc. Of IEICE 1996 General Conference, Japan, The Institute of Electronics, Information and Communication Engineers, March 11, 1996, Volume 1, pp. 21 Yatsuzuki, Y .; , An Error Perforation for Learning and Deletion of Local Local Minima in Binary 3-Layered Neural Networks, Proc. or 1995 IEEE Int. Conf. on Neural Networks, December 1995, Vol. 1, pp. 63-68, (IS BN: 0-7803-2768-3) (58) Fields investigated (Int.Cl. ⁷ , DB name) G06N 1/00-7/08 G06G 7/60 JST file (JOIS) CSDB (Japan Patent Office)

Claims (5)

(57) [Claims] 1. In learning of a neural network for learning using a teacher signal, when the absolute value of an error signal obtained by subtracting the output unit signal of the correct output unit from the teacher signal is less than or equal to a first threshold, A weighting coefficient update error signal having a polarity opposite to that of the error signal and having an amplitude that decreases as the distance from the teacher signal increases, and if the absolute value of the error signal is greater than the first threshold, the weight signal update error signal has an amplitude smaller than the error signal. Generate at least a polarity weighting factor update error signal, update the weighting factor using the weighting factor update error signal, and perform learning to determine the maximum of the absolute value of the error signal in the incorrect output unit for all learning input signals. A neural network characterized in that when the value is smaller than the second threshold, the first threshold is reduced for learning. . 2. The neural network according to claim 1, wherein a weight coefficient update error signal having an amplitude smaller than the error signal by a first constant is generated for an incorrect output unit, and the weight coefficient update error is generated. A neural network characterized in that a weighting coefficient is updated using a signal to perform learning, and when the maximum value is smaller than the second threshold, the first constant is decreased to continue learning. 3. The neural network according to claim 1, wherein the first threshold is set to zero to start learning, and the maximum value is equal to or higher than the third threshold at a predetermined number of learning times. In this case, at least the first threshold is set to a non-zero value for learning. 4. The neural network according to claim 2, wherein each of the first threshold and the first constant is set to zero to start learning, and the maximum value is set at a point of a predetermined number of times of learning. A neural network characterized in that at least the first threshold is set to a non-zero value for learning when the threshold is equal to or higher than the third threshold. 5. The neural network according to claim 1, 2, 3 and 4, wherein after all output unit signals for all learning input signals match the teacher signal within a given range, A neural network characterized by obtaining a margin for obtaining a correct output unit signal from the teacher signal and terminating learning when the minimum margin for all learning input signals is larger than a fourth threshold prepared in advance.