CN109766418A - Method and apparatus for output information - Google Patents

Abstract

The embodiment of the present application discloses the method and apparatus for output information.One specific embodiment of this method includes: to obtain inquiry text and target text；It will inquire that text and target text are input to answer extracting model trained in advance, obtain answer text corresponding with the inquiry text and without answer probability, wherein, answer extracting model is for characterizing inquiry text, text and answering text, without the corresponding relationship between answer probability, and no answer probability is for characterizing the probability that can not be extracted from target text with the answer of the inquiry text matches；Output answers text and without answer probability.The embodiment realizes output answer text corresponding with text is inquired and output, and for characterizing, there is no the probability with the answer of the inquiry text matches in the target text.

Description

Method and apparatus for output information

Technical field

The invention relates to field of computer technology, and in particular to the method and apparatus for output information.

Background technique

With the rapid development of artificial intelligence technology, general answer extracting technology, which reads understanding field in machine, also to seem It is more and more important.For particular problem, the technology for the answer for excavating correspondence problem in given text is the weight of question answering system Want one of component part.

Relevant mode is usually to guarantee input problem defeated as far as possible by the fuzzy matching to input problem and text Entering has answer in text.Furthermore, it is possible to carry out volume to the matching of output answer and former problem by network or authoritative data Outer confidence level verification, to judge whether answer and former problem are corresponding.

Summary of the invention

The embodiment of the present application proposes the method and apparatus for output information.

In a first aspect, the embodiment of the present application provides a kind of method for output information, ask this method comprises: obtaining Ask text and target text；It will inquire that text and target text are input to answer extracting model trained in advance, obtain and inquire The corresponding answer text of text and without answer probability, wherein answer extracting model is for characterizing inquiry text, text and answer Text, without the corresponding relationship between answer probability, no answer probability can not be extracted and be ask from target text for characterizing Ask the probability of the answer of text matches；Output answers text and without answer probability.

In some embodiments, above-mentioned answer extracting model includes the first coding layer, is based on attention mechanism (attention) the first alternation of bed, Recognition with Recurrent Neural Network, neural network output layer, the first hidden layer and the output of the first hidden layer Layer；And it is above-mentioned will inquire that text and target text are input to answer extracting model trained in advance, obtain and inquiry text pair The answer text answered and without answer probability, comprising: inquiry text and target text are input to the first coding layer, obtain first Inquire text vector and first object text vector；First inquiry text vector and first object text vector are input to base In the first alternation of bed of attention mechanism, the first output matrix is obtained；First output matrix is input to Recognition with Recurrent Neural Network, Obtain output vector；Output vector is input to neural network output layer, obtains position of the answer in target text；By One output matrix and output vector are input to the first hidden layer, obtain the first probability vector；First probability vector is input to One hidden layer output layer obtains no answer probability；According to position of the answer in target text, generates and answer text.

In some embodiments, this method further include: inquiry text, target text and text input will be answered to preparatory Trained answer credibility model, obtains answer fiducial probability, wherein answer credibility model is for characterizing inquiry text, text Originally, the corresponding relationship between text and answer fiducial probability is answered, answer fiducial probability is for characterizing answer text, inquiring text Matching degree between sheet and target text；Export answer fiducial probability.

In some embodiments, above-mentioned answer credibility model includes the second coding layer, second based on attention mechanism Alternation of bed, the second hidden layer and the second hidden layer output layer；And it is above-mentioned will inquiry text, target text and answer text input extremely Trained answer credibility model in advance, obtains answer fiducial probability, comprising: by inquiry text, target text and answers text It is input to the second coding layer, obtains the second inquiry text vector, the second target text vector sum answers text vector；By second Inquiry text vector, the second target text vector sum answer text vector and are input to the second alternation of bed based on attention mechanism, Obtain the second output matrix；Second output matrix is input to the second hidden layer, obtains the second probability vector；By the second probability to Amount is input to the second hidden layer output layer, obtains answer fiducial probability.

In some embodiments, this method further include: based on no answer probability and answer fiducial probability, determine answer just True probability, wherein answer correct probability is used to characterize the order of accuarcy for answering answer of the text as inquiry text；Output is answered Case correct probability.

Second aspect, the embodiment of the present application provide a kind of device for output information, which includes: to obtain list Member is configured to obtain inquiry text and target text；First determination unit is configured to inquire text and target text It is input to answer extracting model trained in advance, obtains answer text corresponding with inquiry text and without answer probability, wherein Answer extracting model is for characterizing inquiry text, text and answering text, without the corresponding relationship between answer probability, no answer Probability is used to characterize the probability that can not be extracted from target text with the answer of inquiry text matches；First output unit, quilt It is configured to output and answers text and without answer probability.

In some embodiments, above-mentioned answer extracting model includes the first coding layer, the first friendship based on attention mechanism Alternating layers, Recognition with Recurrent Neural Network, neural network output layer, the first hidden layer and the first hidden layer output layer；And above-mentioned first determination list Member is further configured to: will inquiry text and target text be input to the first coding layer, obtain the first inquiry text vector and First object text vector；First inquiry text vector and first object text vector are input to based on attention mechanism First alternation of bed obtains the first output matrix；First output matrix is input to Recognition with Recurrent Neural Network, obtains output vector；It will Output vector is input to neural network output layer, obtains position of the answer in target text；By the first output matrix and output Vector is input to the first hidden layer, obtains the first probability vector；First probability vector is input to the first hidden layer output layer, is obtained Without answer probability；According to position of the answer in target text, generates and answer text.

In some embodiments, device further include: the second determination unit is configured to inquire text, target text With answer text input to answer credibility model trained in advance, answer fiducial probability is obtained, wherein answer confidence level mould Type is used to characterize the corresponding relationship between inquiry text, text, answer text and answer fiducial probability, and answer fiducial probability is used The matching degree between text, inquiry text and target text is answered in characterization；Second output unit is configured to export and answer Case fiducial probability.

In some embodiments, above-mentioned answer credibility model includes the second coding layer, second based on attention mechanism Alternation of bed, the second hidden layer and the second hidden layer output layer；And above-mentioned second determination unit is further configured to: by inquiry text Originally, target text and answer text input obtain the second inquiry text vector, the second target text vector to the second coding layer With answer text vector；Inquire that text vector, the second target text vector sum answer text vector are input to and are based on for second Second alternation of bed of attention mechanism, obtains the second output matrix；Second output matrix is input to the second hidden layer, obtains Two probability vectors；Second probability vector is input to the second hidden layer output layer, obtains answer fiducial probability.

In some embodiments, device further include: third determination unit is configured to based on no answer probability and answers Case fiducial probability determines answer correct probability, wherein answer correct probability, which is used to characterize, answers text as inquiry text The order of accuarcy of answer；Third output unit is configured to export answer correct probability.

The third aspect, the embodiment of the present application provide a kind of electronic equipment, which includes: one or more places Manage device；Storage device is stored thereon with one or more programs；When one or more programs are held by one or more processors Row, so that one or more processors realize the method as described in implementation any in first aspect.

Fourth aspect, the embodiment of the present application provide a kind of computer-readable medium, are stored thereon with computer program, The method as described in implementation any in first aspect is realized when the program is executed by processor.

Method and apparatus provided by the embodiments of the present application for output information obtain inquiry text and target text first This；Then, it will inquire that text and target text are input to answer extracting model trained in advance, obtain and the inquiry text pair The answer text answered and without answer probability, wherein answer extracting model is for characterizing inquiry text, text and answering text, nothing Corresponding relationship between answer probability, no answer probability can not extract and the inquiry text for characterizing from target text The probability of matched answer；Finally, output answers text and without answer probability.To realize according to inquiry text and target Text, output answer text corresponding with inquiry text and output are not present and the inquiry for characterizing in the target text The probability of the answer of text matches.

Detailed description of the invention

By reading a detailed description of non-restrictive embodiments in the light of the attached drawings below, the application its Its feature, objects and advantages will become more apparent upon:

Fig. 1 is that one embodiment of the application can be applied to exemplary system architecture figure therein；

Fig. 2 is the flow chart according to one embodiment of the method for output information of the application；

Fig. 3 is the schematic diagram according to an application scenarios of the method for output information of the embodiment of the present application；

Fig. 4 is the flow chart according to another embodiment of the method for output information of the application；

Fig. 5 is the structural schematic diagram according to one embodiment of the device for output information of the application；

Fig. 6 is adapted for the structural schematic diagram for the computer system for realizing the electronic equipment of the embodiment of the present application.

Specific embodiment

The application is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched The specific embodiment stated is used only for explaining related invention, rather than the restriction to the invention.It also should be noted that being Convenient for description, part relevant to related invention is illustrated only in attached drawing.

It should be noted that in the absence of conflict, the features in the embodiments and the embodiments of the present application can phase Mutually combination.The application is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

Fig. 1, which is shown, can apply the method for output information of the application or showing for the device for output information Example property framework 100.

As shown in Figure 1, system architecture 100 may include terminal device 101,102,103, network 104 and server 105. Network 104 between terminal device 101,102,103 and server 105 to provide the medium of communication link.Network 104 can To include various connection types, such as wired, wireless communication link or fiber optic cables etc..

Terminal device 101,102,103 is interacted by network 104 with server 105, to receive or send message etc..Eventually Various telecommunication customer end applications can be installed, such as web browser applications, searching class are answered in end equipment 101,102,103 With, instant messaging tools, mailbox client, social platform software, text editing class application, read class application etc..

Terminal device 101,102,103 can be hardware, be also possible to software.When terminal device 101,102,103 is hard When part, the various electronic equipments of text-processing are can be with display screen and supported, including but not limited to smart phone, flat Plate computer, E-book reader, pocket computer on knee and desktop computer etc..When terminal device 101,102,103 is When software, it may be mounted in above-mentioned cited electronic equipment.Its may be implemented into multiple softwares or software module (such as For providing Distributed Services), single software or software module also may be implemented into.It is not specifically limited herein.

Server 105 can be to provide the server of various services, for example, export on terminal device 101,102,103 Text of answering corresponding with inquiry text provides the background server supported.Background server can be to the inquiry text of acquisition Carry out analysis and the processing such as answer extracting with target text, and by processing result (answer text such as corresponding with text is inquired and No answer probability) feed back to terminal device.

It should be noted that above-mentioned inquiry text and target text can also be stored directly in the local of server 105, Server 105 can directly extract the local inquiry text stored and target text and be handled, at this point it is possible to not deposit In terminal device 101,102,103 and network 104.

It should be noted that server can be hardware, it is also possible to software.When server is hardware, may be implemented At the distributed server cluster that multiple servers form, individual server also may be implemented into.When server is software, Multiple softwares or software module (such as providing Distributed Services) may be implemented into, also may be implemented into single software or Software module.It is not specifically limited herein.

It should be noted that the method provided by the embodiment of the present application for output information can be held by server 105 Row, can also be executed by terminal device 101,102,103.Correspondingly, it can be set for the device of output information in service In device 105, also it can be set in terminal device 101,102,103.

It should be understood that the number of terminal device, network and server in Fig. 1 is only schematical.According to realization need It wants, can have any number of terminal device, network and server.

With continued reference to Fig. 2, the process of one embodiment of the method for output information according to the application is shown 200.This for output information method the following steps are included:

Step 201, inquiry text and target text are obtained.

In the present embodiment, above-mentioned inquiry text can be with the text for puing question to property.Inquiry text, which can be, includes The text of preset characters.Above-mentioned preset characters can include but is not limited at least one of following: " who ", " what ", " where ", " why ", "? ", " how many ", " may I ask ", " suddenly asking ".As an example, above-mentioned inquiry text for example can be " whom the world richest is? " Be also possible to " what if having caught a cold? " it can also be " suddenly seeking the download address of XX e-book ".Above-mentioned target text can be root According to actual application demand, preassigned any text in preset corpus；It can also be the text depending on rule This, such as text relevant to the content of above-mentioned inquiry text.As an example, above-mentioned target text can be and above-mentioned inquiry The semantic relevancy of text is more than the text of preset threshold.As another example, if including according to above-mentioned inquiry in text Ask the keyword that text is determined, then the text can be determined that target text.Wherein, above-mentioned keyword can be according to TF- IDF (term frequency-inverse document frequency, word frequency-inverse file frequency) algorithm determines.

It in the present embodiment, can be with for the executing subject of the method for output information (server 105 as shown in Figure 1) Inquiry text and target text are obtained by wired connection mode or radio connection.As an example, above-mentioned executing subject Inquiry text and target text can be obtained from the data server of communication connection.As another example, above-mentioned executing subject is also The voice messaging that user can be obtained from voice-input device, by speech recognition conversion at inquiry text.Then, above-mentioned to hold Row main body can also grab target text from internet.

Step 202, it will inquire that text and target text are input to answer extracting model trained in advance, obtain and inquire The corresponding answer text of text and without answer probability.

In the present embodiment, answer extracting model can be used for characterizing inquiry text, text and answer text, without answer Corresponding relationship between probability.Above-mentioned no answer probability can be used for characterizing can not extract and inquiry text from target text The probability of this matched answer.In practice, above-mentioned answer extracting model can be the various answer extractings applied to question answering system Method.As an example, firstly, above-mentioned executing subject can according to preset character (such as ", " ".""！" "? " Deng) will be above-mentioned Target text splits into sentence set of segments.Wherein, sentence segment can be an individual word, be also possible to word, can be with It is phrase or short sentence.Then, for the sentence segment in above-mentioned sentence set of segments, above-mentioned executing subject can calculate the language Similarity between sentence segment and inquiry text.Next, above-mentioned executing subject can be by obtained similarity maximum value Corresponding sentence segment is determined as candidate answer text.Then, it is answered using semantic analysis model trained in advance from candidate It is extracted in text and answers text.Finally, number 1 and the difference of above-mentioned maximum similarity can be determined as by above-mentioned executing subject Without answer probability.Wherein, the method for above-mentioned calculating similarity can be first with word2vec and GloVe scheduling algorithm by text Vector form is converted to, the cosine similarity between vector is then calculated or is calculated using deep learning model trained in advance Semantic similarity.It should be noted that the method and semantic analysis model of the similarity calculation between above-mentioned text are wide at present The well-known technique of general research and application.Details are not described herein.

In some optional implementations of the present embodiment, above-mentioned answer extracting model may include the first coding layer, The first alternation of bed, Recognition with Recurrent Neural Network, neural network output layer, the first hidden layer and the first hidden layer based on attention mechanism are defeated Layer out.To which above-mentioned executing subject can will inquire that text and target text are input to answering for training in advance in accordance with the following steps Case extraction model obtains answer text corresponding with inquiry text and without answer probability:

Inquiry text and target text are input to the first coding layer, obtain the first inquiry text vector and the by the first step One target text vector.

In these implementations, above-mentioned first coding layer can be used for characterizing the correspondence between text and text vector Relationship.Above-mentioned first coding layer can be the various methods for generating term vector.As an example, above-mentioned first coding layer can To be LSA (Latent semantic analysis, implicit semantic analysis) matrix decomposition model.As another example, above-mentioned First coding layer can also be Word2Vector model.Above-mentioned executing subject can be by the text of the inquiry as acquired in step 201 This and target text are input to above-mentioned first coding layer, obtain the first inquiry text vector corresponding with inquiry text and and mesh Mark the corresponding first object text vector of text.

First inquiry text vector and first object text vector are input to the based on attention mechanism by second step One alternation of bed obtains the first output matrix.

In these implementations, above-mentioned the first alternation of bed based on attention mechanism can be used for characterizing text vector Corresponding relationship between output matrix.Above-mentioned the first alternation of bed based on attention mechanism can be various ANN (Artificial Neural Network, artificial neural network), such as CNN (Convolutional Neural Network, convolutional neural networks), RNN (Recurrent Neural Network, Recognition with Recurrent Neural Network).Introduce attention After mechanism, above-mentioned first alternation of bed can be determined in the first inquiry text vector and first object text vector of input not With the weight of element.

Optionally, above-mentioned the first alternation of bed based on attention mechanism may include the first text alternation of bed and pay attention to certainly Power layer.Above-mentioned first text alternation of bed can be the first inquiry text vector, first object text vector and intermediate output matrix Alignment model.The above-mentioned corresponding pass that can be used for characterizing from attention layer between intermediate output matrix and the first output matrix System.Using above-mentioned from attention layer, the element in intermediate output matrix can be made directly to contact by a calculating step Come, to be easier to capture the complementary feature of text middle and long distance.

Above-mentioned executing subject can will inquire text vector and first object text by the above-mentioned first step obtained first Vector is input to above-mentioned the first alternation of bed based on attention mechanism, obtains and the first inquiry text vector and first object text Corresponding first output matrix of this vector.

First output matrix is input to Recognition with Recurrent Neural Network, obtains output vector by third step.

In these implementations, above-mentioned RNN can be used for characterizing the corresponding pass between output matrix and output vector System.Optionally, above-mentioned RNN can be LSTM (Long-Short Term Memory, long short-term memory) network.Above-mentioned execution master Obtained first output matrix of above-mentioned second step can be input to above-mentioned RNN by body, be obtained corresponding with the first output matrix Output vector.

Output vector is input to neural network output layer by the 4th step, obtains position of the answer in target text.

In these implementations, above-mentioned neural network output layer can be used for characterizing output vector and answer in target The corresponding relationship between position in text.Above-mentioned neural network output layer can be various defeated for more Classification Neurals Activation primitive out, such as Softmax function.Above-mentioned executing subject can be defeated by the obtained output vector of above-mentioned third step Enter to above-mentioned neural network output layer, obtains position of the answer in target text.Wherein, above-mentioned answer is in target text Position may include initial position and answer end position in target text of the answer in target text.Upper rheme Various representations can be had by setting.For example, which word in sentence or which section in target text.

First output matrix and output vector are input to the first hidden layer, obtain the first probability vector by the 5th step.

In these implementations, above-mentioned first hidden layer can be used for characterizing output matrix, output vector and probability vector Between corresponding relationship.Above-mentioned executing subject can be by obtained first output matrix of above-mentioned second step and above-mentioned third step Obtained output vector is input to above-mentioned first hidden layer, obtains corresponding with above-mentioned first output matrix and output vector One probability vector.

First probability vector is input to the first hidden layer output layer, obtains no answer probability by the 6th step.

In these implementations, above-mentioned first hidden layer output layer can be used for characterizing probability vector and without answer probability Between corresponding relationship.Above-mentioned first hidden layer output layer can be the various activation primitives for hidden neuron output, such as Sigmoid function.It is hidden that above-mentioned obtained first probability vector of 5th step can be input to above-mentioned first by above-mentioned executing subject Layer output layer, obtains no answer probability.

7th step generates according to position of the answer in target text and answers text.

In these implementations, above-mentioned executing subject can the answer according to determined by above-mentioned 4th step target text Position in this extracts the corresponding text in position, generates and answer text.As an example, inquiry text is " XX in 2008 It can be held where ".Target text is " Beijing2008 XX meeting opening ceremony ".Above-mentioned executing subject can be determined according to abovementioned steps Initial position of the answer in target text is first word, and end position is also first word.So above-mentioned execution master Body according to position of the above-mentioned answer in target text, can extract first word " Beijing " of target text, literary as answering This.

It should be noted that above-mentioned 4th step can be basically executed in parallel with the five to six step；It can also first carry out The five to six step is stated, then executes above-mentioned 4th step, is not limited thereto.

In these implementations, above-mentioned answer extracting model can be obtained by following steps training:

S1, initial answer extracting model is obtained.

In these implementations, initial answer extracting model can be various ANN.As an example, initial answer extracting Model can include but is not limited to RNN, CNN and combinations thereof.

S2, training sample set is obtained.

In these implementations, each training sample in training sample set may include sample queries text, sample This target text and sample corresponding with sample queries text, sample object text answer text and sample without answer probability.Its In, sample can be used for characterizing without answer probability to be extracted from sample object text and sample queries text matches The probability of answer.

In practice, training sample can obtain in several ways.As an example, user can be inputted search engine The problem of as sample queries text.It can be by search engine inputted aiming at the problem that and included in the webpage that returns Word segment is as sample object text.It is then possible to which extracting answer from sample object text as sample answers text. Later, it is verified according to preset knowledge base, for the sample of correct option of inquiry text can be used as to answer text, It can be relatively fractional value between 0~1 by the corresponding sample of text is answered without answer determine the probability with the sample, such as 0；It is right Text is answered in the sample for the correct option that can not be used as inquiry text, can will answer text corresponding sample with the sample This is without the bigger numerical that answer determine the probability is between 0~1, such as 1.Finally, by sample queries text, sample object text, Sample answers text and sample corresponding with sample answer text and is associated storage without answer probability, finally obtains training Sample.A large amount of training sample is formed by a large amount of data, and then forms training sample set.

S3, the method using machine learning, by the sample queries text and sample of the training sample in training sample set Input of the target text as initial answer extracting model, will be corresponding with the sample queries text of input and sample object text Sample answer text and sample and be used as desired output without answer probability, train and obtain above-mentioned answer extracting model.

Specifically, the executing subject of above-mentioned training step can ask the sample of the training sample in training sample set Ask that text and sample object text input to initial answer extracting model, obtain the answer text of the training sample and without answer Probability.It is then possible to answer text using the sample that preset loss function calculates obtained answer text and the training sample Difference degree between this；And the sample of obtained no answer probability and the training sample is calculated without between answer probability Difference degree.Next, can use the complexity of regularization term computation model.Later, based on the resulting difference journey of calculating The complexity of degree and model, adjusts the structural parameters of initial answer extracting model, and meeting preset trained termination condition In the case of, terminate training.Finally, the initial answer extracting model that training obtains is determined as answer extracting model.

It should be noted that above-mentioned loss function can use logarithm loss function, above-mentioned regularization term can use L2 Norm or Dropout technology.Above-mentioned preset trained termination condition can include but is not limited at least one of following: when training Between be more than preset duration；Frequency of training is more than preset times；Resulting difference degree is calculated less than preset discrepancy threshold；It surveys Accuracy rate on examination collection reaches preset accuracy rate threshold value；Coverage rate on test set reaches preset coverage rate threshold value.

It should also be noted that, answer text based on obtained training sample, without answer probability and the training sample Sample answer text, sample without the difference degree between answer probability, can adopt and adjust initial answer extracting in various manners The structural parameters of model.For example, BP (Back Propagation, backpropagation) algorithm or SGD can be used (Stochastic Gradient Descent, stochastic gradient descent) algorithm is joined to adjust the network of initial answer extracting model Number.

It is worth noting that, the executing subject of above-mentioned training step can be with the execution master of the method for output information Body is same or different.If identical, the executing subject of above-mentioned training step can be after training obtains answer extracting model The structural information of trained answer extracting model and parameter value are stored in local.If it is different, then above-mentioned training step Executing subject can be after training obtains answer extracting model by the structural information and parameter of trained answer extracting model Value is sent to the executing subject of the method for output information.

It should be pointed out that the first coding layer in above-mentioned answer extracting model, the first interaction based on attention mechanism Layer, Recognition with Recurrent Neural Network and the first hidden layer can be separated and be trained, and can also be used as an entirety while training, the present embodiment pair This is without limiting.

Step 203, output answers text and without answer probability.

In the present embodiment, above-mentioned executing subject from step 202 obtain answer text and without answer probability after, can be defeated Text is answered out and without answer probability.Wherein, output can be there are many form.As an example, above-mentioned executing subject can will return Answer text and the display equipment without answer probability output to communication connection, such as display.It is thus possible to will be according to above-mentioned step It rapid obtained answer text and is showed without answer probability.As another example, above-mentioned executing subject can also will return Answer text and the storage medium without answer probability output to communication connection, such as hard disk.It is thus possible to will be according to above-mentioned steps It obtained answer text and is stored without answer probability, for subsequent use.

With continued reference to one that Fig. 3, Fig. 3 are according to the application scenarios of the method for output information of the embodiment of the present application A schematic diagram.In the application scenarios of Fig. 3, user's using terminal equipment 301 input text " where XX can hold within 2008? " 304.Server 302 obtains above-mentioned text 304 as inquiry text from terminal device.Then, server 302 is from communicating It is obtained in the database server 303 of connection comprising " 2008 Beijing XX meeting, when evening 8 on the 8th of August in 2008 are whole in China Hold in people's republic, Beijing, capital " text 305 as target text.Later, server 302 will inquire text 304 and mesh Mark text 305 is input to answer extracting model 306 trained in advance.Target text 305 can be split Chinese idiom by server 302 Sentence set of segments " 2008 Beijing XX meeting ", " whole when the evening 8 on the 8th of August in 2008 ", " in beijing Beijing It holds ".Server 302 can determine the similarity between inquiry text 304 and above-mentioned each sentence segment.Then, server 302 can be determined as the sentence segment " 2008 Beijing XX meeting " of similarity highest (such as 0.9) candidate answer text.Again It is answered using semantic analysis model from candidate and extracts " Beijing " in text as answer text 307.Next, server 302 It can be determined as no answer probability 308 for 0.1.Finally, server 302 can be by identified answer text 307 and without answer Probability 308 carries out output and shows.Optionally, server 302 can also be by identified answer text 307 and without answer probability 308 are integrated into the information 309 of " Beijing " " 0.1 ", and above- mentioned information 309 are sent to terminal device 301.

The method provided by the above embodiment of the application obtains inquiry text and target text first；Then, it will inquire Text and target text are input in advance trained answer extracting model, obtain answer text corresponding with the inquiry text and Without answer probability, wherein answer extracting model is for characterizing inquiry text, text and answering text, without between answer probability Corresponding relationship, no answer probability for characterize can not be extracted from target text it is general with the answer of the inquiry text matches Rate；Finally, output answers text and without answer probability.It realizes according to inquiry text and target text, output and inquiry text There is no general with the answer of the inquiry text matches in the target text for characterizing for corresponding answer text and output Rate.So as to learn the confidence level of the answer text while obtaining and answering text.

With further reference to Fig. 4, it illustrates the processes 400 of another embodiment of the method for output information.The use In the process 400 of the method for output information, comprising the following steps:

Step 401, inquiry text and target text are obtained.

Step 402, it will inquire that text and target text are input to answer extracting model trained in advance, obtain and inquire The corresponding answer text of text and without answer probability.

Step 403, output answers text and without answer probability.

Above-mentioned steps 401, step 402, step 403 respectively with step 201, step 202, the step in previous embodiment 203 is consistent, and the description above with respect to step 201, step 202 and step 203 is also applied for step 401, step 402 and step 403, details are not described herein again.

Step 404, text, target text will be inquired and answer text input to answer credibility model trained in advance, Obtain answer fiducial probability.

In the present embodiment, above-mentioned answer credibility model can be used for characterizing inquiry text, text, answer text with Corresponding relationship between answer fiducial probability.Above-mentioned answer fiducial probability, which can be used for characterizing, answers text, inquiry text and mesh Mark the matching degree between text.In practice, above-mentioned answer credibility model can be various for definite response text, inquiry The method for asking the matching degree between text and target text.As an example, above-mentioned executing subject can be in accordance with the following steps Obtain answer fiducial probability:

The first step, will answer text and inquiry text combines, and generates answer and verifies text.Above-mentioned generation answer verifying The mode of text can be various modes.As an example, the interrogative in text replacement inquiry text can will be answered.For example, Inquire that text is " whom the wife of Zhang San is? " answering text is " Li Si ".So, answer verifying text generated can be " wife of Zhang San is Li Si ".

Above-mentioned answer is verified text input to language identification model trained in advance, is estimated according to maximum likelihood by second step Meter obtains sentence probability.As an example, above-mentioned language identification model can be N-gram (N metagrammar) language model.It needs Bright, above-mentioned language identification model is the well-known technique studied and applied extensively at present.Details are not described herein.

Third step calculates the similarity between the sentence that answer verifying text and target text are formed after splitting.For The sentence in sentence set that target text is formed after splitting, above-mentioned executing subject can calculate answer verifying text and the language Similarity between sentence, obtains similarity set.

4th step calculates the average value of the maximum value and obtained sentence probability in obtained similarity set, will Above-mentioned average value is determined as answer fiducial probability.

In some optional implementations of the present embodiment, above-mentioned answer credibility model may include the second coding Layer, the second alternation of bed based on attention mechanism, the second hidden layer and the second hidden layer output layer.To which, above-mentioned executing subject can be with Text, target text will be inquired in accordance with the following steps and answers text input to answer credibility model trained in advance, are obtained Answer fiducial probability:

The first step by inquiry text, target text and answers text input to the second coding layer, obtains the second inquiry text This vector, the second target text vector sum answer text vector.

In these implementations, above-mentioned second coding layer can be used for characterizing the correspondence between text and text vector Relationship.Above-mentioned second coding layer can be the various methods for generating term vector.As an example, above-mentioned first coding layer can To be LSA matrix decomposition model.As another example, above-mentioned second coding layer can also be Word2Vector model.It is above-mentioned Executing subject can be defeated by inquiry text, target text as acquired in step 401 and the obtained answer text of step 402 Enter to above-mentioned second coding layer, obtains and inquire that text corresponding second inquires text vector, corresponding with target text second Target text vector sum answer text vector corresponding with text is answered.

Second step inquires that text vector, the second target text vector sum are answered text vector and be input to based on note for second Second alternation of bed of power mechanism of anticipating, obtains the second output matrix.

In these implementations, above-mentioned the second alternation of bed based on attention mechanism can be used for characterizing text vector Corresponding relationship between output matrix.Above-mentioned the second alternation of bed based on attention mechanism can be various ANN, such as CNN,RNN.After introducing attention mechanism, above-mentioned second alternation of bed can determine the second inquiry text vector, the second mesh of input It marks text vector and answers the weight of the different elements in text vector.

Above-mentioned executing subject can will inquire text vector, the second target text by the above-mentioned first step obtained second Vector sum answer text vector be input to above-mentioned the second alternation of bed based on attention mechanism, obtain with second inquiry text to Amount, the second target text vector sum answer text vector corresponding second output matrix.

Second output matrix is input to the second hidden layer, obtains the second probability vector by third step.

In these implementations, above-mentioned second hidden layer can be used for characterizing pair between output matrix and probability vector It should be related to.Obtained second output matrix of above-mentioned second step can be input to above-mentioned second hidden layer by above-mentioned executing subject, be obtained To the second probability vector corresponding with above-mentioned second output matrix.

Second probability vector is input to the second hidden layer output layer, obtains answer fiducial probability by the 4th step.

In these implementations, above-mentioned second hidden layer output layer can be used for characterizing probability vector and answer is credible general Corresponding relationship between rate.Above-mentioned second hidden layer output layer can be the various activation primitives for hidden neuron output, example Such as Sigmoid function.Obtained second probability vector of above-mentioned third step can be input to above-mentioned second by above-mentioned executing subject Hidden layer output layer obtains answer fiducial probability.

In these implementations, above-mentioned answer credibility model is obtained by following steps training:

S1, initial answer credibility model is obtained.

In these implementations, initial answer credibility model can be various ANN.As an example, initial answer can Credit model can include but is not limited to RNN, CNN and combinations thereof.

S2, training sample set is obtained.

In these implementations, each training sample in training sample set may include sample queries text, sample This target text, sample answer text and sample corresponding with sample queries text, sample object text, sample answer text Answer fiducial probability.Wherein, sample answer fiducial probability can be used for characterizing sample answer text, sample queries text and sample Matching degree between this target text.

In practice, training sample can obtain in several ways.As an example, user can be inputted search engine The problem of as sample queries text.It can be by search engine inputted aiming at the problem that and included in the webpage that returns Word segment is as sample object text.It is then possible to which extracting answer from sample object text as sample answers text. Later, sample can be answered between text, sample queries text and sample object text according to preset matching rule Matching degree is labeled.As an example, answering text, sample queries text for the sample that matching degree is greater than preset threshold Originally with sample object text, corresponding sample answer fiducial probability can be determined as the bigger numerical between 0~1, example Such as 1；The sample for being less than or equal to preset threshold for matching degree answers text, sample queries text and sample object text, Corresponding sample answer fiducial probability can be determined as to the relatively fractional value between 0~1, such as 0.Finally, by sample It answers text, sample queries text, sample object text and corresponding sample answer fiducial probability and is associated storage, Finally obtain training sample.A large amount of training sample is formed by a large amount of data, and then forms training sample set.

S3, the method using machine learning, by sample queries text, the sample of the training sample in training sample set Target text and sample answer input of the text as initial answer credibility model, by sample queries text, the sample with input For this target text sample answer fiducial probability corresponding with sample answer text as desired output, training obtains above-mentioned answer Credibility model.

Specifically, the executing subject of above-mentioned training step can ask the sample of the training sample in training sample set It asks that text, sample object text and sample answer text input to initial answer credibility model, obtains answering for the training sample Case fiducial probability.It is then possible to credible using the obtained answer fiducial probability of preset loss function calculating and sample answer Difference degree between probability.Next, can use the complexity of regularization term computation model.Later, based on calculating institute The complexity of the difference degree and model that obtain, adjusts the structural parameters of initial answer credibility model, and meeting preset instruction In the case where practicing termination condition, terminate training.It can finally, the initial answer credibility model that training obtains is determined as answer Credit model.

It should be noted that above-mentioned loss function can use logarithm loss function, above-mentioned regularization term can use L2 Norm or Dropout technology.Above-mentioned preset trained termination condition can include but is not limited at least one of following: when training Between be more than preset duration；Frequency of training is more than preset times；Resulting difference degree is calculated less than preset discrepancy threshold；It surveys Accuracy rate on examination collection reaches preset accuracy rate threshold value；Coverage rate on test set reaches preset coverage rate threshold value.

It should also be noted that, the sample of answer fiducial probability and the training sample based on obtained training sample Difference degree between answer fiducial probability can adopt the structural parameters for adjusting initial answer credibility model in various manners. For example, BP (Back Propagation, backpropagation) algorithm or SGD (Stochastic Gradient can be used Descent, stochastic gradient descent) algorithm adjusts the network parameter of initial answer credibility model.

It is worth noting that, the executing subject of above-mentioned training step can be with the execution master of the method for output information Body is same or different.If identical, the executing subject of above-mentioned training step can obtain answer credibility model in training The structural information of trained answer credibility model and parameter value are stored in local afterwards.If it is different, then above-mentioned training step Rapid executing subject can believe the structure of trained answer credibility model after training obtains answer credibility model Breath and parameter value are sent to the executing subject of the method for output information.It is appreciated that the executing subject of above-mentioned training step It can also be same or different with the executing subject of the training step of step 202 in previous embodiment.It is not limited thereto.

It should be pointed out that the second coding layer in above-mentioned answer credibility model, the second friendship based on attention mechanism Alternating layers and the second hidden layer, which can separate, trains, and can also be used as an entirety while training, the present embodiment is to this without limit It is fixed.

Step 405, based on no answer probability and answer fiducial probability, answer correct probability is determined.

In the present embodiment, above-mentioned answer correct probability, which can be used for characterizing, answers text as the correct of inquiry text The probability of answer.

Above-mentioned executing subject is based on the obtained no answer probability of step 402 and the obtained answer of step 404 is credible general Rate can use the various evaluation methods that can comprehensively utilize above-mentioned no answer probability and answer fiducial probability to determine answer Correct probability.As an example, above-mentioned executing subject can calculate number 1 and first without the difference between answer probability, then calculate The average value of above-mentioned difference and above-mentioned answer fiducial probability obtains above-mentioned answer correct probability.As another example, above-mentioned to hold Row main body can be determined first without whether answer probability is less than preset threshold value (such as 0.1)；It is less than in response to determination, it is above-mentioned to hold Answer fiducial probability directly can be determined as above-mentioned answer correct probability by row main body.In general, setting one for above-mentioned threshold value Lesser numerical value.It is appreciated that when no answer probability is less than a lesser threshold value, it is meant that identified answer Has higher confidence level in terms of matching degree of the text between inquiry text and target text.Therefore, it can will be used for It is correctly general that the answer fiducial probability of matching degree between characterization inquiry text, target text and answer text is determined as answer Rate.

Step 406, answer correct probability is exported.

In the present embodiment, after above-mentioned executing subject obtains answer correct probability from step 405, answer can exported just True probability.Wherein, output can be there are many form.As an example, above-mentioned executing subject answer correct probability can be exported to The display equipment of communication connection, such as display.It is thus possible to will be according to the obtained answer correct probability of above-mentioned steps Reveal and.As another example, above-mentioned executing subject can also export answer correct probability to storage Jie of communication connection Matter, such as hard disk.It is thus possible to will be stored according to the obtained answer correct probability of above-mentioned steps, make for subsequent With.

Figure 4, it is seen that the method for output information compared with the corresponding embodiment of Fig. 2, in the present embodiment Process 400 embody and will inquire text, target text and answer text input to answer credibility model trained in advance, The step of obtaining answer fiducial probability, and based on no answer probability and answer fiducial probability, determines and to export answer correctly general The step of rate.The scheme of the present embodiment description can be by obtained answer text and inquiry text and target text as a result, Between matching degree comprehensively considered, thus realize to answer text confidence level objectively evaluate.

With further reference to Fig. 5, as the realization to method shown in above-mentioned each figure, this application provides be used for output information Device one embodiment, the Installation practice is corresponding with embodiment of the method shown in Fig. 2, which can specifically answer For in various electronic equipments.

As shown in figure 5, the device 500 provided in this embodiment for output information includes that acquiring unit 501, first is true Order member 502 and the first output unit 503.Wherein, acquiring unit 501 are configured to obtain inquiry text and target text； First determination unit 502 is configured to inquire that text and target text are input to answer extracting model trained in advance, obtains To answer text corresponding with inquiry text and without answer probability, wherein answer extracting model is for characterizing inquiry text, text Originally and text is answered, without the corresponding relationship between answer probability, no answer probability can not be extracted for characterizing from target text Out with inquiry text matches answer probability；First output unit 503 is configured to export and answers text and general without answer Rate.

In the present embodiment, in the device of output information 500: acquiring unit 501, the first determination unit 502 and The specific processing of one output unit 503 and its brought technical effect can be respectively with reference to the steps in Fig. 2 corresponding embodiment 201, the related description of step 202 and step 203, details are not described herein.

In some optional implementations of the present embodiment, above-mentioned answer extracting model may include the first coding layer, The first alternation of bed, Recognition with Recurrent Neural Network, neural network output layer, the first hidden layer and the first hidden layer based on attention mechanism are defeated Layer out；And above-mentioned first determination unit 501 can be further configured to: inquiry text and target text are input to the One coding layer obtains the first inquiry text vector and first object text vector；By the first inquiry text vector and the first mesh Mark text vector is input to the first alternation of bed based on attention mechanism, obtains the first output matrix；First output matrix is defeated Enter to Recognition with Recurrent Neural Network, obtains output vector；Output vector is input to neural network output layer, obtains answer in target Position in text；First output matrix and output vector are input to the first hidden layer, obtain the first probability vector；By first Probability vector is input to the first hidden layer output layer, obtains no answer probability；According to position of the answer in target text, generate Answer text.

In some optional implementations of the present embodiment, the above-mentioned device 500 for output information can also include Second determination unit (not shown) and the second output unit (not shown).Wherein, above-mentioned second determination unit, can To be configured to inquire text, target text and answer text input to answer credibility model trained in advance, answered Case fiducial probability, wherein it is credible that answer credibility model can be used for characterizing inquiry text, text, answer text and answer Corresponding relationship between probability, answer fiducial probability, which can be used for characterizing, answers between text, inquiry text and target text Matching degree；Above-mentioned second output unit may be configured to output answer fiducial probability.

In some optional implementations of the present embodiment, above-mentioned answer credibility model may include the second coding Layer, the second alternation of bed based on attention mechanism, the second hidden layer and the second hidden layer output layer；And above-mentioned second determination unit It can be further configured to: by inquiry text, target text and answer text input to the second coding layer, obtain the second inquiry Ask that text vector, the second target text vector sum answer text vector；By second inquire text vector, the second target text to Amount and answer text vector are input to the second alternation of bed based on attention mechanism, obtain the second output matrix；By the second output Input matrix obtains the second probability vector to the second hidden layer；Second probability vector is input to the second hidden layer output layer, is obtained Answer fiducial probability.

In some optional implementations of the present embodiment, the above-mentioned device 500 for output information can also include Third determination unit (not shown) and third output unit (not shown).Wherein, above-mentioned third determination unit, can To be configured to determine answer correct probability, wherein answer correct probability can based on no answer probability and answer fiducial probability For characterizing the order of accuarcy for answering the answer of text as inquiry text；Above-mentioned third output unit, can be configured At output answer correct probability.

The device provided by the above embodiment of the application obtains inquiry text and target text by acquiring unit 501 first This；Then, the first determination unit 502 will inquire that text and target text are input to answer extracting model trained in advance, obtain Answer text corresponding with the inquiry text and without answer probability, wherein answer extracting model is for characterizing inquiry text, text Originally and text is answered, without the corresponding relationship between answer probability, no answer probability can not be taken out for characterizing from target text Take out the probability with the answer of the inquiry text matches；Finally, text and general without answer is answered in the output of the first output unit 503 Rate.To realize according to inquiry text and target text, output and the inquiry corresponding answer text of text and the answer are literary This reliability.

Below with reference to Fig. 6, it illustrates the computer systems for the electronic equipment for being suitable for being used to realize the embodiment of the present application 600 structural schematic diagram.Electronic equipment shown in Fig. 6 is only an example, function to the embodiment of the present application and should not be made With range band come any restrictions.

As shown in fig. 6, computer system 600 includes central processing unit (CPU) 601, it can be read-only according to being stored in Program in memory (ROM) 602 is loaded into the program in random access storage device (RAM) 603 from storage section 608 And execute various movements appropriate and processing.In RAM 603, also it is stored with system 600 and operates required various program sum numbers According to.CPU 601, ROM 602 and RAM 603 are connected with each other by bus 604.Input/output (I/O) interface 605 also connects To bus 604.

I/O interface 605 is connected to lower component: the importation 606 including keyboard, mouse etc.；Including such as liquid crystal Show the output par, c 607 of device (LCD) etc.；Storage section 608 including hard disk etc.；And including such as LAN card, modulation /demodulation The communications portion 609 of the network interface card of device etc..Communications portion 609 executes communication process via the network of such as internet.It drives Dynamic device 610 is also connected to I/O interface 605 as needed.Detachable media 611, such as disk, CD, magneto-optic disk, semiconductor Memory etc. is mounted on as needed on driver 610, in order to from the computer program read thereon quilt as needed It installs into storage section 608.

Particularly, in accordance with an embodiment of the present disclosure, it may be implemented as computer above with reference to the process of flow chart description Software program.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer-readable Jie Computer program in matter, the computer program include the program code for method shown in execution flow chart.Such In embodiment, which can be downloaded and installed from network by communications portion 609, and/or from detachable Medium 611 is mounted.When the computer program is executed by central processing unit (CPU) 601, execute in the present processes The above-mentioned function of limiting.

It should be noted that the computer-readable medium of the application can be computer-readable signal media or calculating Machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have one or more conducting wires electrical connection, Portable computer diskette, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only deposit Reservoir (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory Part or above-mentioned any appropriate combination.In this application, computer readable storage medium, which can be, any include or deposits The tangible medium of program is stored up, which can be commanded execution system, device or device use or in connection. And in this application, computer-readable signal media may include in a base band or as carrier wave a part propagate number It is believed that number, wherein carrying computer-readable program code.The data-signal of this propagation can take various forms, packet Include but be not limited to electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be Any computer-readable medium other than computer readable storage medium, which can send, propagate or Transmission is for by the use of instruction execution system, device or device or program in connection.Computer-readable medium On include program code can transmit with any suitable medium, including but not limited to: wireless, electric wire, optical cable, RF etc., Or above-mentioned any appropriate combination.

The calculating of the operation for executing the application can be write with one or more programming languages or combinations thereof Machine program code, described program design language include object-oriented programming language-such as Java, Smalltalk, C+ +, it further include conventional procedural programming language-such as " C " language or similar programming language.Program code can Fully to execute, partly execute on the user computer on the user computer, be held as an independent software package Part executes on the remote computer or holds on a remote computer or server completely on the user computer for row, part Row.In situations involving remote computers, remote computer can pass through the network of any kind --- including local area network (LAN) or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as using because of spy Service provider is netted to connect by internet).

Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the application, method and computer journey The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can be with A part of a module, program segment or code is represented, a part of the module, program segment or code includes one or more A executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, box Middle marked function can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated Can actually be basically executed in parallel, they can also be executed in the opposite order sometimes, this according to related function and It is fixed.It is also noted that the group of each box in block diagram and or flow chart and the box in block diagram and or flow chart It closes, can be realized with the dedicated hardware based system for executing defined functions or operations, or specialized hardware can be used Combination with computer instruction is realized.

Being described in unit involved in the embodiment of the present application can be realized by way of software, can also be passed through The mode of hardware is realized.Described unit also can be set in the processor, for example, can be described as: a kind of processing Device, including acquiring unit, the first determination unit and the first output unit.Wherein, the title of these units is under certain conditions simultaneously The restriction to the unit itself is not constituted, for example, acquiring unit is also described as " obtaining inquiry text and target text Unit ".

As on the other hand, present invention also provides a kind of computer-readable medium, which be can be Included in electronic equipment described in above-described embodiment；It is also possible to individualism, and without the supplying electronic equipment In.Above-mentioned computer-readable medium carries one or more program, when said one or multiple programs are held by the device When row, so that the electronic equipment: obtaining inquiry text and target text；Inquiry text and target text are input to preparatory instruction Experienced answer extracting model obtains answer text corresponding with the inquiry text and without answer probability, wherein answer extracting mould For type for characterizing inquiry text, text and answering text, without the corresponding relationship between answer probability, no answer probability is used for table Sign can not extract the probability with the answer of the inquiry text matches from target text；Text and general without answer is answered in output Rate.

Above description is only the preferred embodiment of the application and the explanation to institute's application technology principle.Art technology Personnel should be appreciated that invention scope involved in the application, however it is not limited to skill made of the specific combination of above-mentioned technical characteristic Art scheme, while should also cover in the case where not departing from foregoing invention design, by above-mentioned technical characteristic or its equivalent feature into Row any combination and the other technical solutions formed.Such as features described above and (but being not limited to) disclosed herein have class Technical characteristic like function is replaced mutually and the technical solution that is formed.

Claims (12)

1. a kind of method for output information, comprising:

Obtain inquiry text and target text；

The inquiry text and the target text are input to answer extracting model trained in advance, obtained and the inquiry text This corresponding answers text and without answer probability, wherein the answer extracting model inquires text, text and answer for characterizing Text, without the corresponding relationship between answer probability, the no answer probability can not be extracted for characterizing from the target text Out with it is described inquiry text matches answer probability；

Export the answer text and the no answer probability.

2. according to the method described in claim 1, wherein, the answer extracting model includes the first coding layer, is based on attention The first alternation of bed, Recognition with Recurrent Neural Network, neural network output layer, the first hidden layer and the first hidden layer output layer of mechanism；And

It is described that the inquiry text and the target text are input to answer extracting model trained in advance, it obtains and the inquiry Ask the corresponding answer text of text and without answer probability, comprising:

The inquiry text and the target text are input to first coding layer, obtain the first inquiry text vector and One target text vector；

The first inquiry text vector and the first object text vector are input to the based on attention mechanism One alternation of bed obtains the first output matrix；

First output matrix is input to the Recognition with Recurrent Neural Network, obtains output vector；

The output vector is input to the neural network output layer, obtains position of the answer in the target text；

First output matrix and the output vector are input to first hidden layer, obtain the first probability vector；

First probability vector is input to the first hidden layer output layer, obtains no answer probability；

According to position of the answer in the target text, generates and answer text.

3. according to the method described in claim 1, wherein, the method also includes:

By the inquiry text, the target text and the text input of answering to answer credibility model trained in advance, Obtain answer fiducial probability, wherein the answer credibility model is for characterizing inquiry text, text, answering text and answer Corresponding relationship between fiducial probability, the answer fiducial probability is for characterizing the answer text, the inquiry text and institute State the matching degree between target text；

Export the answer fiducial probability.

4. according to the method described in claim 3, wherein, the answer credibility model includes the second coding layer, based on attention The second alternation of bed, the second hidden layer and the second hidden layer output layer of power mechanism；And

It is described to inquire text, the target text and the text input of answering to answer confidence level mould trained in advance for described Type obtains answer fiducial probability, comprising:

By the inquiry text, the target text and the answer text input to second coding layer, the second inquiry is obtained Ask that text vector, the second target text vector sum answer text vector；

Answer text vector described in the second inquiry text vector, the second target text vector sum is input to the base In the second alternation of bed of attention mechanism, the second output matrix is obtained；

Second output matrix is input to second hidden layer, obtains the second probability vector；

Second probability vector is input to the second hidden layer output layer, obtains answer fiducial probability.

5. the method according to claim 3 or 4, wherein the method also includes:

Based on the no answer probability and the answer fiducial probability, answer correct probability is determined, wherein the answer is correctly general Rate is used to characterize the order of accuarcy of answer of the answer text as the inquiry text；

Export the answer correct probability.

6. a kind of device for output information, comprising:

Acquiring unit is configured to obtain inquiry text and target text；

First determination unit is configured to for the inquiry text and the target text to be input to answer extracting trained in advance Model obtains answer text corresponding with the inquiry text and without answer probability, wherein the answer extracting model is used for table It consults and asks text, text and answer text, without the corresponding relationship between answer probability, the no answer probability can not for characterizing The probability with the answer of the inquiry text matches is extracted from the target text；

First output unit is configured to export the answer text and the no answer probability.

7. device according to claim 6, wherein the answer extracting model includes the first coding layer, is based on attention The first alternation of bed, Recognition with Recurrent Neural Network, neural network output layer, the first hidden layer and the first hidden layer output layer of mechanism；

First determination unit is further configured to:

The inquiry text and the target text are input to first coding layer, obtain the first inquiry text vector and One target text vector；

The first inquiry text vector and the first object text vector are input to the based on attention mechanism One alternation of bed obtains the first output matrix；

First output matrix is input to the Recognition with Recurrent Neural Network, obtains output vector；

The output vector is input to the neural network output layer, obtains position of the answer in the target text；

First output matrix and the output vector are input to first hidden layer, obtain the first probability vector；

First probability vector is input to the first hidden layer output layer, obtains no answer probability；

According to position of the answer in the target text, generates and answer text.

8. device according to claim 6, wherein described device further include:

Second determination unit is configured to the inquiry text, the target text and the answer text input to preparatory Trained answer credibility model, obtains answer fiducial probability, wherein the answer credibility model is for characterizing inquiry text Corresponding relationship between sheet, text, answer text and answer fiducial probability, the answer fiducial probability is for characterizing the answer Matching degree between text, the inquiry text and the target text；

Second output unit is configured to export the answer fiducial probability.

9. device according to claim 8, wherein the answer credibility model includes the second coding layer, based on attention The second alternation of bed, the second hidden layer and the second hidden layer output layer of power mechanism；

Second determination unit is further configured to:

By the inquiry text, the target text and the answer text input to second coding layer, the second inquiry is obtained Ask that text vector, the second target text vector sum answer text vector；

Answer text vector described in the second inquiry text vector, the second target text vector sum is input to the base In the second alternation of bed of attention mechanism, the second output matrix is obtained；

Second output matrix is input to second hidden layer, obtains the second probability vector；

Second probability vector is input to the second hidden layer output layer, obtains answer fiducial probability.

10. device according to claim 8 or claim 9, wherein described device further include:

Third determination unit is configured to determine that answer is correctly general based on the no answer probability and the answer fiducial probability Rate, wherein the answer correct probability is used to characterize the order of accuarcy of answer of the answer text as the inquiry text；

Third output unit is configured to export the answer correct probability.

11. a kind of electronic equipment, comprising:

One or more processors；

Storage device is stored thereon with one or more programs；

When one or more of programs are executed by one or more of processors, so that one or more of processors are real Now such as method as claimed in any one of claims 1 to 5.

12. a kind of computer-readable medium, is stored thereon with computer program, wherein the realization when program is executed by processor Such as method as claimed in any one of claims 1 to 5.