CN109389207A - A kind of adaptive neural network learning method and nerve network system - Google Patents

Abstract

The invention discloses an adaptive neural network learning method and the neural network system. The neural network system includes an input layer, a hidden layer, an output layer and a knowledge base. The method includes: (1) creating and initializing the knowledge base, Adjust the weights of the hidden layer and the output layer so that the similarity of the output result satisfies the convergence condition; (2) Online adaptive learning, for a certain input data, the set of connection weights is used as the initial value, and the learning algorithm is used to obtain the output Result; (3) Determine whether the similarity between the output result and the corresponding expected output meets the requirements: when the requirements are met, output the result; otherwise, repeat step (2), traverse all the knowledge in the knowledge base, but still not found, Then regard the data as a sample of new knowledge; (4) adjust the connection weight according to step (1) and add it to the knowledge base. The method of the invention can process and discriminate internal and external data, new and old data, and realize online self-adaptive learning and identification functions.

Description

An adaptive neural network learning method and neural network system

technical field

The invention belongs to the technical field of artificial intelligence, and in particular relates to an adaptive neural network learning method and a neural network system.

Background technique

Within the discipline of artificial intelligence, the research results of neural networks have been successfully transplanted into quite a few fields, such as decision support, face recognition, knowledge base systems, expert systems and emotional robots. In representational traditional research, most models generally work in the domain that has been explained: that is, for the context of explanation, system designers usually give some implicit prior conventions, under which it is difficult to follow The context, goal, or representation is transformed as the problem solving process progresses.

At present, the main method of image thinking simulation is to simulate and replicate the image thinking related "Xiangzhi"-artificial neural network connection-based connection mechanism. In terms of computing and processing methods, the connection mechanism method has opened up a new way, that is, the method of parallel processing and distributed expression. Specifically, this method uses "a network of several nodes that can be connected between every two nodes" to represent information. In the past, the semantic network used to represent knowledge is that a node corresponds to a concept, while the artificial neural network corresponds to a concept based on a distribution pattern of nodes and the size of the weight, so that even the information attributes on each node correspond to a concept. Distortions and distortions have occurred, but will not cause significant changes in the properties of the concepts expressed by the network. In addition, the information on some common units can also be used to express similar concepts, but using various methods including the above-mentioned neural network connection method to simulate image thinking is like the symbolic representation of logical thinking. Totally successful.

The neural network system can recognize the input, and it will represent the input related knowledge and grammar rules with the neural network structure and neuron connection weights, and it even has a certain degree of fault tolerance. Neural network systems are data-driven, and it is unclear whether these data sources are internal or external, new or old. After the neural network is trained, it is assumed that the input into the system is data that is not included in the training set, that is, a new data sample. The neural network cannot determine whether the input is new information relative to its knowledge, nor can it actively learn this new data. data, but use the knowledge gained from the training process to make misjudgments on this new data sample. Or, in the online training process, the neural network does not distinguish between those that are its own existing knowledge states and those that are external input data, which are always processed indiscriminately.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide an adaptive neural network learning method and neural network system, which solves the problem that the existing neural network system cannot distinguish between new and old data and learning, can process and distinguish internal and external data, new and old data, and realize online automatic Adaptive learning and recognition functions.

In order to achieve the above object, the present invention provides an adaptive neural network learning method, the neural network system includes: an input layer, a hidden layer, an output layer and a knowledge base, the knowledge base K=(S ₁ , S ₂ ...... S _m ), S _b is the connection weight of the neural network, S _b = (V|W|Y), b=1, 2... m, Y is the expected output, V and W are the connection weight of the hidden layer of the network respectively and the connection weights of the output layer.

The method contains:

(1) Create and initialize the knowledge base: obtain the knowledge of the training set. Each knowledge corresponds to a set of connection weights V of the hidden layer and connection weights W of the output layer. The weights are adjusted, and the similarity E of the output result satisfies the convergence condition, and the connection weight of the final output is determined;

(2) Online adaptive learning, search for any knowledge in the knowledge base, and obtain the connection weight V of the corresponding hidden layer and the connection weight W of the output layer. For a certain input data, the group of connection weights is obtained. value as the initial value, use the adjustment of the weight value in step 1 to adjust the connection weight value, continuously operate the specified number of learning N, and obtain the output result of the actual operation;

(3) Judging whether the similarity between the output result and the corresponding expected output meets the requirements: when the similarity between the output result and the corresponding expected output meets the requirements, the result is output; when the similarity between the output result and the corresponding expected output does not meet the requirements , then sequentially search and select a new knowledge from the knowledge base, obtain a new set of hidden layer, output layer and expected output, repeat step (2), if all knowledge in the knowledge base is traversed, When the corresponding knowledge has not been found, the data is regarded as a sample of new knowledge;

(4) Adjust the connection weights according to the adjustment of the weights in step (1) until the similarity E of the output result satisfies the convergence condition, and then adjust the connection weights of the corresponding output layer, the connection weights of the hidden layer and the expected output Merged into a new knowledge, added to the knowledge base K.

Preferably, the connection weights of the hidden layer of the network and the connection weights of the output layer are both matrices; wherein,

Preferably, in step (1), the adjustment algorithm of the weight includes:

δ _j =(y _j -o _j )f'(net _j ) (3).

In formulas (1)-(3), w _ij and v _ij are the connection weights of the output layer and the hidden layer in the matrix (i, j), respectively, α is the scale coefficient, and δ _j is the learning of the j column in the matrix. rate, f'(net _j ) is the derivative of neuron excitation function, y _j and o _j are the expected output and actual output of column j in the matrix, respectively, and o _i is the actual output of row i in the matrix.

Preferably, in step (1), the equation of the convergence condition of the output result similarity E is:

E=ΣE _p (5).

In equations (4) and (5), E _p represents the result similarity of the ppth output neuron, y _Pj and o _Pj are the expected output and actual output of (p, j) in the matrix, respectively, and E represents the result similarity , which is used to judge whether the network meets the convergence requirements.

Preferably, in step (1), the similarity requirement is satisfied when the result similarity E is lower than 20% of the expected output result; in step (4), when the result similarity E is lower than 5% of the expected output result to meet the similarity requirement.

The present invention also provides an adaptive neural network system, the neural network system includes: an input layer, a hidden layer, an output layer and a knowledge base, the knowledge base K=(S ₁ , S ₂ ...... S _m ), S _b is the connection weight of the neural network, S _b = (V|W|Y), b=1, 2...m, Y is the expected output, V and W are the connection weight of the hidden layer of the network and the output layer, respectively Connection weight; the neural network system is a forward neural network system, the input layer, the hidden layer and the output layer are connected in turn for transmission, and the knowledge base is connected with the hidden layer.

Preferably, the connection weights of the hidden layer of the network and the connection weights of the output layer are both matrices; wherein,

The adaptive neural network learning method and neural network system of the present invention solve the problem that the existing neural network system cannot distinguish between old and new data and learning, and has the following advantages:

The adaptive neural network learning method and the neural network system of the present invention store the neural network connection weights in the knowledge base in the form of knowledge, so as to realize processing such as searching and adding knowledge, so that the neural network system has the ability to process and discriminate internal and external data, New and old data functions, so as to realize online adaptive learning and recognition functions. Furthermore, the method of the present invention has the flexibility to transform contexts, goals or representations as the problem solving process progresses with respect to the context of the interpretation.

Description of drawings

FIG. 1 is a schematic diagram of the neural network system of the present invention.

FIG. 2 is a flowchart of the adaptive neural network learning method of the present invention.

Detailed ways

The technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.

An adaptive neural network learning method, as shown in FIG. ₁ , _{is a schematic} diagram of the neural network system of the present invention. Containing layer L _B =(b ₁ ......b _i ...... b _p ), output layer L _C =(c ₁ ...... c _j ...... c _q ) and knowledge base K=(S ₁ ,S ₂ ...... S _m ), S _b is the neural network connection weight, S _b = (V|W|Y), b=1,2...m (natural number), Y is the expected output, V and W are the connections of the hidden layer of the network, respectively weights and connection weights of the output layer, where,

As shown in Figure 2, it is a flowchart of the adaptive neural network learning method of the present invention, and the method includes:

(1) Create and initialize the knowledge base: For each data set sent to the input layer, that is, the training set, a piece of knowledge can be obtained, that is, the connection weight V of the corresponding set of hidden layers and the connection of the output layer The weight W, the weight adjustment algorithm of the hidden layer and the output layer in the learning process is:

δ _j =(y _j -o _j )f'(net _j ) (3);

In formulas (1)-(3), w _ij and v _ij are the connection weights of the output layer and the hidden layer in the matrix (i, j), respectively, α is the scale coefficient, and δ _j is the learning of the j column in the matrix. rate, f'(net _j ) is the derivative of neuron excitation function, y _j and o _j are the expected output and actual output of column j in the matrix, respectively, and o _i is the actual output of row i in the matrix.

According to whether the similarity E of the output result satisfies the convergence condition, the connection weight of the final output is determined, that is, a knowledge in the knowledge base is determined:

E=∑E _p (5);

In equations (4) and (5), E _p represents the result similarity of the ppth output neuron, y _Pj and o _Pj are the expected output and actual output of (p, j) in the matrix, respectively, and E represents the result similarity , which is used to judge whether the network meets the convergence requirements.

(2) Online adaptive learning, set the number of learning N, set the similarity of the output result, that is, the error E value, search for any knowledge in the knowledge base, and separate the connection weight V and output of the corresponding hidden layer. The connection weight W of the layer, for a certain input data, apply this group of connection weights as the initial value, use the adjustment of the weights in step (1) to adjust the connection weights, and continuously calculate the specified number of learning times N to obtain the actual operation. output result.

(3) Determine whether the similarity between the output result and the corresponding expected output meets the requirements: if the similarity between the output result and the corresponding expected output meets the requirements, output the result, when the result similarity E is lower than 20% of the expected output result In order to meet the similarity requirements; if the similarity between the output result and the corresponding expected output does not meet the requirements, search and select a new knowledge from the knowledge base in order, that is, select a new set of hidden layer, output layer and expected output. , repeat step (2), if the corresponding knowledge is not found after traversing all the knowledge in the knowledge base, even if the similarity between the output result and the corresponding expected output meets the requirements, the data is regarded as a sample of new knowledge;

(4) Adjust the connection weights according to the formulas (1)-(3) in step (1) until the similarity E of the output result satisfies the convergence condition, and when the error is less than 5% of the expected output result, the similarity is satisfied It is required to combine the connection weight of the corresponding output layer, the connection weight of the hidden layer and the expected output into a new knowledge and add it to the knowledge base K.

An adaptive neural network system, as shown in FIG. 1 , is a schematic diagram of the neural network system of the present invention. The neural network system is a forward neural network system, including: an input layer L _A =(a ₁ … a _h ... a _n ), hidden layer L _B =(b ₁ ...b _i ... b _p ), output layer L _C =(c ₁ ... c _j ... b _q ) and knowledge base K = (S ₁ , S ₂ ……S _m ), S _b is the neural network connection weight, S _b = (V|W|Y), b=1, 2……m, Y is the expected output, V and W are the network hidden The connection weight of the containing layer and the connection weight of the output layer, the input layer, the hidden layer and the output layer are connected in turn for transmission, and the knowledge base and the hidden layer are connected, among which,

To sum up, the adaptive neural network learning method and neural network system of the present invention store the neural network connection weights in the knowledge base in the form of knowledge, so as to realize processing such as searching and adding knowledge, so that the neural network system has the ability to process knowledge. And the function of distinguishing internal and external data, old and new data, so as to realize online adaptive learning and recognition functions.

While the content of the present invention has been described in detail by way of the above preferred embodiments, it should be appreciated that the above description should not be construed as limiting the present invention. Various modifications and alternatives to the present invention will be apparent to those skilled in the art upon reading the foregoing. Accordingly, the scope of protection of the present invention should be defined by the appended claims.

Claims (7)

1. An adaptive neural network learning method, characterized in that the neural network system comprises: an input layer, a hidden layer, an output layer and a knowledge base, the knowledge base K=(S ₁ , S ₂ ...... S _m ), S _b is the connection weight of the neural network, S _b =(V|W|Y), b=1,2...m, Y is the expected output, V and W are the connection weight of the hidden layer and the output layer of the network, respectively The connection weight of ; The method contains: (1) Create and initialize the knowledge base: obtain the knowledge of the training set. Each knowledge corresponds to a set of connection weights V of the hidden layer and connection weights W of the output layer. The weights are adjusted, and the similarity E of the output result satisfies the convergence condition, and the connection weight of the final output is determined; (2) Online adaptive learning, search for any knowledge in the knowledge base, and obtain the connection weight V of the corresponding hidden layer and the connection weight W of the output layer. For a certain input data, the group of connection weights is obtained. value as the initial value, use the adjustment of the weight value in step 1 to adjust the connection weight value, continuously operate the specified number of learning N, and obtain the output result of the actual operation; (3) Judging whether the similarity between the output result and the corresponding expected output meets the requirements: when the similarity between the output result and the corresponding expected output meets the requirements, the result is output; when the similarity between the output result and the corresponding expected output does not meet the requirements , then sequentially search and select a new knowledge from the knowledge base, obtain a new set of hidden layer, output layer and expected output, repeat step (2), if all knowledge in the knowledge base is traversed, When the corresponding knowledge has not been found, the data is regarded as a sample of new knowledge; (4) Adjust the connection weights according to the adjustment of the weights in step (1) until the similarity E of the output result satisfies the convergence condition, and then adjust the connection weights of the corresponding output layer, the connection weights of the hidden layer and the expected output Merged into a new knowledge, added to the knowledge base K. 2. The adaptive neural network learning method according to claim 1, wherein the connection weight of the hidden layer of the network and the connection weight of the output layer are both matrices; wherein, 3. The adaptive neural network learning method according to claim 2, wherein in step (1), the adjustment algorithm of the weight comprises: δ _j =(y _j -o _j )f'(net _j ) (3); In formulas (1)-(3), w _ij and v _ij are the connection weights of the output layer and the hidden layer in the matrix (i, j), respectively, α is the scale coefficient, and δ _j is the learning of the j column in the matrix. rate, f'(net _j ) is the derivative of neuron excitation function, y _j and o _j are the expected output and actual output of column j in the matrix, respectively, and o _i is the actual output of row i in the matrix. 4. The adaptive neural network learning method according to any one of claims 1-3, wherein in step (1), the equation of the convergence condition of the output result similarity E is: E=∑E _p (5); In equations (4) and (5), E _p represents the result similarity of the pth output neuron, y _Pj and o _Pj are the expected output and actual output of (p, j) in the matrix, respectively, and E represents the result similarity , which is used to judge whether the network meets the convergence requirements. 5. adaptive neural network learning method according to claim 1, is characterized in that, in step (1), when result similarity E is lower than 20% of expected output result, is to satisfy similarity requirement; In step ( 4), when the result similarity E is lower than 5% of the expected output result, the similarity requirement is met. 6. An adaptive neural network system, characterized in that the neural network system comprises: an input layer, a hidden layer, an output layer and a knowledge base, the knowledge base K=(S ₁ , S ₂ ...... S _m ), S _b is the connection weight of the neural network, S _b =(V|W|Y), b=1,2...m, Y is the expected output, V and W are the connection weight of the hidden layer and the output layer of the network, respectively The neural network system is a forward neural network system, the input layer, the hidden layer and the output layer are connected in turn for transmission, and the knowledge base is connected with the hidden layer. 7. The adaptive neural network system according to claim 6, wherein the connection weights of the hidden layer of the network and the connection weights of the output layer are both matrices; wherein,