CN107679082A - Question and answer searching method, device and electronic equipment - Google Patents
Question and answer searching method, device and electronic equipment Download PDFInfo
- Publication number
- CN107679082A CN107679082A CN201710769081.5A CN201710769081A CN107679082A CN 107679082 A CN107679082 A CN 107679082A CN 201710769081 A CN201710769081 A CN 201710769081A CN 107679082 A CN107679082 A CN 107679082A
- Authority
- CN
- China
- Prior art keywords
- user
- question
- information
- answer
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 238000012549 training Methods 0.000 claims abstract description 49
- 230000006399 behavior Effects 0.000 claims description 103
- 239000013598 vector Substances 0.000 claims description 64
- 238000012163 sequencing technique Methods 0.000 claims description 24
- 238000004422 calculation algorithm Methods 0.000 claims description 20
- 238000012216 screening Methods 0.000 claims description 13
- 230000011218 segmentation Effects 0.000 claims description 8
- 238000003066 decision tree Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 6
- 230000009471 action Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 20
- 230000008569 process Effects 0.000 description 15
- 230000006870 function Effects 0.000 description 13
- 230000006872 improvement Effects 0.000 description 9
- 238000004590 computer program Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000013507 mapping Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 229920001296 polysiloxane Polymers 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This specification embodiment discloses question and answer searching method, device and electronic equipment, and the question and answer searching method includes：According to user's question sentence and the action trail information of the user, the DSSM obtained using training, answer corresponding to user's question sentence is obtained.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method and an apparatus for question and answer search and an electronic device.
Background
With the development of artificial intelligence technology, more and more human-computer interaction scenes exist. In some of these scenarios, the user may directly perform human-computer interaction through natural language. For example, the user can make ticket booking consultation, weather forecast inquiry and medical consultation through the voice assistant, so that convenience is brought to the daily life of the user.
In the prior art, when performing human-computer interaction based on natural language, an intelligent device or software generally performs character string matching in preset standard question sentences according to user question sentences provided by a user, and takes standard answers corresponding to the standard question sentences with the highest matching degree as answers corresponding to the user question sentences.
Based on the prior art, a scheme capable of more accurately obtaining answers corresponding to user question sentences is needed.
Disclosure of Invention
The embodiment of the specification provides a question and answer searching method, a question and answer searching device and electronic equipment, and is used for solving the following technical problems: there is a need for a solution that can more accurately obtain answers corresponding to user question sentences.
In order to solve the above technical problem, the embodiments of the present specification are implemented as follows:
the question and answer searching method provided by the embodiment of the specification comprises the following steps:
receiving a question of a user;
acquiring user behavior track information and a plurality of candidate answers;
according to the user behavior track information, calculating scores corresponding to the candidate answers respectively by utilizing a Deep Structured Semantic Model (DSSM) trained on the basis of the user question-answer information and the user behavior track information;
and sequencing the candidate answers according to the scores and the text characteristics of the candidate answers, and obtaining answers corresponding to the question of the user according to the sequencing result.
The question answering search device provided by the embodiment of the specification comprises:
the receiving module is used for receiving a question of a user;
the information acquisition module is used for acquiring user behavior track information and a plurality of candidate answers;
the scoring module is used for calculating scores corresponding to the candidate answers according to the user behavior track information by utilizing DSSM obtained by training based on the user question-answer information and the user behavior track information;
and the reordering module is used for ordering the candidate answers according to the scores and the text characteristics of the candidate answers and obtaining answers corresponding to the question of the user according to the ordering result.
An electronic device provided in an embodiment of the present specification includes:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
to receive user question;
acquiring user behavior track information and a plurality of candidate answers;
calculating scores corresponding to the candidate answers respectively by utilizing DSSM obtained based on the user question-answer information and the user behavior track information training according to the user behavior track information;
and sequencing the candidate answers according to the scores and the text characteristics of the candidate answers, and obtaining answers corresponding to the question of the user according to the sequencing result.
The embodiment of the specification adopts at least one technical scheme which can achieve the following beneficial effects: based on the user behavior track information and the text characteristics, the DSSM is used for answer search, so that answers corresponding to the user question can be obtained more accurately.
Drawings
In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present specification, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort.
Fig. 1 is a schematic diagram of an overall architecture involved in a practical application scenario of the solution of the present specification;
fig. 2 is a schematic flow chart of a question and answer searching method provided in an embodiment of the present specification;
fig. 3 is a schematic flowchart of a DSSM training method provided in an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a DSSM provided in an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of a reordering model provided in an embodiment of the present disclosure;
FIG. 6 is a schematic diagram of an operation of a question-and-answer search system provided in an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of a question and answer searching device provided in an embodiment of the present specification;
fig. 8a and 8b are schematic diagrams illustrating the effect of the question and answer search scheme provided in the embodiment of the present specification in an actual application scenario.
Detailed Description
The embodiment of the specification provides a question and answer searching method and device and electronic equipment.
In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any inventive step based on the embodiments of the present disclosure, shall fall within the scope of protection of the present application.
Fig. 1 is a schematic diagram of an overall architecture related to the solution of the present specification in a practical application scenario. The work flow of the whole framework mainly comprises the following steps: according to the user question and the corresponding user behavior track information thereof, the DSSM device is used for grading a plurality of candidate answers to obtain scores corresponding to the candidate answers respectively, and the scores are sent to the reordering model device through the network to be reordered so as to obtain the answers corresponding to the user question.
Based on this overall architecture, the following describes the scheme of the present specification in detail.
An embodiment of the present specification provides a question and answer searching method, and fig. 2 is a schematic flow chart of the question and answer searching method, where the flow chart may include the following steps:
s202: and receiving a question of the user.
S204: and acquiring user behavior track information and a plurality of candidate answers.
The user behavior trace information may refer to an operation record left by the user when performing the business operation. Preferably, the user behavior trace information may be related to a user question corresponding to the user behavior trace information, so that an answer corresponding to the user question is obtained according to the user behavior trace information.
For example, after inquiring about the debt of the user, the user wants to pay, but does not know how to perform the payment operation, and then the user sends a user question asking how to pay; in this case, the user behavior trace information may be a record corresponding to the arrears inquiry action.
S206: and calculating scores corresponding to the candidate answers respectively by utilizing DSSM obtained by training based on the user question-answer information and the user behavior track information according to the user behavior track information.
The DSSM is a deep neural network model based on semantic similarity for modeling.
In the embodiment of the present specification, the DSSM is obtained by training using the user question-answer information and the user behavior trajectory information.
Further, when searching for an answer, according to the currently received user question and the acquired corresponding user behavior track information, scores of a plurality of candidate answers are calculated by using the DSSM, and the candidate answers are ranked according to the scores, so that the candidate answers which are more likely to be accurate answers are screened out.
In the embodiment of the description, the user behavior trajectory information can be used as auxiliary information which is helpful for understanding the question of the user, so that richer basis is provided for the DSSM to score the candidate answers, and the method is beneficial for obtaining relatively accurate answers under the condition that the question of the user is not clear enough.
S208: and sequencing the candidate answers according to the scores and the text characteristics of the candidate answers, and obtaining answers corresponding to the question of the user according to the sequencing result.
As mentioned above, DSSM is semantic-based, and in practical applications, text features other than semantic may be further utilized to help more accurately obtain answers corresponding to user question sentences.
In this embodiment of the present specification, for step S202, in addition to receiving the user question, a user identifier (user ID) and/or a session identifier (session ID) corresponding to the user question may be obtained, so as to query and obtain corresponding user behavior track information.
The session identifier corresponding to the user question may be a session identifier between the user and the execution main body of the flow in fig. 2, or may be a session identifier between the user and one or more other service systems.
Further, for step S204, the acquiring the user behavior trace information may specifically include: and acquiring user behavior track information according to the user identification and/or the session identification corresponding to the user question.
For example, the user behavior trace information may include user behavior trace points for a period of time, preferably, the period of time may be a recent period of time. For example, 30 minutes before the time when the user question is received.
The user behavior trace points may refer to: and recording the business operation of the user at a certain moment by the executive body or the other business system or systems.
For example, it may be a Remote Procedure Call Protocol (RPC) record and/or a Uniform Resource Locator (URL) record, and the like. Generally, for a scene that a user accesses a corresponding server through an APP, a corresponding RPC record can be used as a user behavior track point, and for a scene that the user accesses a corresponding website through a PC, a corresponding URL record can be used as a user behavior track point.
It should be noted that, acquiring the user behavior trace information according to the user identifier and/or the session identifier is only an optional implementation manner, and in practical application, other implementation manners may also be adopted to acquire the user behavior trace information.
For example, a hot question related to the question may be obtained according to a query of the user question, and behavior trajectory information of the user who has proposed the hot question is obtained as user behavior trajectory information corresponding to the user question.
In this embodiment of the present specification, for step S204, the obtaining a plurality of candidate answers specifically includes: and screening a plurality of candidate answers in the specified answer set according to the text features of the question of the user and the text features of the answers contained in the specified answer set.
The specified answer set may be selected from a larger answer set or may be predefined and categorized.
Further, the screening of the specified answer set according to the text features of the user question and the text features of the answers included in the specified answer set to obtain a plurality of candidate answers may specifically include: extracting the designated answer set corresponding to the user question using a predetermined search engine; sorting the answers in the specified answer set by using a LamdamAT algorithm to obtain a first sorting result; and screening to obtain a plurality of candidate answers according to the first sequencing result.
Through the search engine, preliminary search can be carried out based on the question of the user to obtain a corresponding specified answer set. For example, assuming that the user question is "how to pay", the corresponding specified answer set may include a plurality of answers related to payment, such as "credit card payment method", "house loan payment method", and the like.
The ranking scheme corresponding to the first ranking result may be ranked according to the degree of correlation between the answers and the question of the user, and the answer with the larger degree of correlation is ranked earlier. As explained in the above example, assuming that the user issues the question "how to pay" through the APP specific to a certain credit card, the correlation between the "credit card payment method" and the question is higher than the correlation between the "house loan payment method" and the question according to general experience, so that the "credit card payment method" can be ranked earlier than the "house loan payment method" in the corresponding first ranking result.
In this embodiment of the present specification, for step S206, the question-answer information of the user includes question information and answer information corresponding to the question information (where the answer information here may be a correct answer as a positive sample, or an incorrect answer as a negative sample); the DSSM can be obtained through training based on the user question and answer information and the user behavior track information.
For convenience of understanding, fig. 3 is a schematic flow chart of the DSSM training method, and the flow chart may include the following steps:
s302: generating a first input vector according to the problem information and the user behavior track information; and the number of the first and second groups,
s304: generating a second input vector according to answer information corresponding to the question information;
s306: and training by using the first input vector and the second input vector to obtain the DSSM.
In the training process, combining the problem information with corresponding user behavior track information, and using the combined information as a first input vector for training the DSSM.
The embodiment of the present specification further provides a schematic structural diagram of a DSSM, which is specifically shown in fig. 4.
The DSSM in fig. 4 is a neural network model including three hidden layers, where the number of nodes included in the three hidden layers is: 300. 300, 128, the DSSM of figure 4 may omit the Word hashing (Word hashing) layer as compared to a standard DSSM. The training data of the DSSM may be obtained by pre-filtering and screening, and the training data may be user question information and user behavior trajectory information corresponding thereto, or identification information (e.g., token ID) of these information.
Taking token ID as an example, combining (for example, directly performing splicing or performing arithmetic operation, etc.) the token ID corresponding to the user behavior trajectory information and the user question information according to a certain rule to obtain a first input vector, and using the first input vector as the input of the query (query) end of the DSSM;
further, a second input vector may be generated according to each answer information corresponding to the current question information, and the second input vector may be used as an input of the DSSM document (document) end.
For example, two second input vectors are shown in fig. 4, one of which corresponds to positive samples and the other of which corresponds to negative samples.
In this embodiment of this specification, for step S302, the generating a first input vector according to the question information and the user behavior trajectory information may specifically include: acquiring a problem information identifier appointed for the problem information and a track information identifier appointed for user behavior track information corresponding to the problem information; and generating a first input vector according to the problem information identifier and the track information identifier.
In practical applications, the problem information may be obtained by normalizing some raw data in order to reduce interference. The normalization process may include, for example, filtering, case conversion, punctuation addition and deletion, and the like.
Taking filtering as an example, the raw data may be question sentences in some actual corpora. These questions may contain some words of relatively lesser importance, which may be removed by a filtering process. For example, assume the raw data is: "do you i need to consult a movie ticket buying process? "can filter out the words of" you' y "," i need to consult ", etc., will leave" movie ticket buying process? "as question information.
The problem information identifier and the track information identifier may be directly specified manually, or a word vector corresponding to the problem information or the user behavior track information may be extracted as the identifier. Assuming that the identifier is manually and directly specified, a mapping relationship between the problem information and the problem information identifier and a mapping relationship between the user behavior track information and the track information identifier can be respectively established in advance, and the identifier is specified through the establishment of the mapping relationship.
In this embodiment of the present specification, for step S304, the generating a second input vector according to answer information corresponding to the question information may specifically include: extracting knowledge point title word segmentation of answer information corresponding to the question information; and generating a second input vector according to the knowledge point title word segmentation.
The knowledge point title participles can reflect the main semantics of the corresponding answer information, and can be generally understood as subject words in the answer information.
In this embodiment of the present specification, for step S306, the training to obtain DSSM by using the first input vector and the second input vector may specifically include: inputting the first input vector into a DSSM prototype to obtain a first output result; inputting the second input vector into the DSSM prototype to obtain a second output result; and calculating the correlation between the first output result and the second output result, and training the DSSM prototype according to the correlation to obtain the DSSM.
A DSSM prototype may refer to a predefined DSSM that has not begun training or that has begun training but has not yet completed training.
The correlation may be measured in various ways, for example, by cosine calculation, or by space vector distance measurement.
For example, training of a particular DSSM may be performed according to the following formula:
li＝Wix；
li＝f(Wili-l+bi),i＝2,…，N-1；
y＝f(WNlN-1+bN)；
wherein liW as representing the i-th hidden layeriTo represent the parameter matrix of the i-th hidden layer, biFor the bias vector of the i-th hidden layer, f (x) usually takes the tanh function, y as output. And measuring the correlation by using cosine calculation values of the output of the query end and the output of the document end.
In an embodiment of the present specification, the ranking the multiple candidate answers according to the scores and the text features of the multiple candidate answers, and obtaining the answer corresponding to the question of the user according to the ranking result may specifically include:
according to the scores and the text features of the candidate answers, ranking the candidate answers by using a Lamdamard algorithm to obtain a second ranking result; according to the first sorting result and the second sorting result, the candidate answers are reordered to obtain a reordering result; and obtaining an answer corresponding to the question of the user according to the reordering result.
The text features may be, for example: the type of the subject term, the part of speech of the subject term, the placeholder corresponding to the subject term, etc.
Further, the reordering the plurality of candidate answers according to the first ordering result and the second ordering result to obtain a reordering result may specifically include:
according to the first sequencing result and the second sequencing result, the multiple candidate answers are reordered by utilizing a reordering model obtained by training, and a reordering result is obtained; wherein training the reordered model comprises: acquiring a user click log; and training to obtain the reordering model by utilizing a LamdamAT algorithm according to the user click log and the score.
The user click log contains historical click data for the current user and/or other users for the answer. The historical click data may specifically include: and statistics data such as the number of times each answer corresponding to each question is clicked by each user. Click data may reflect the user's propensity for an answer that helps to determine whether the answer accurately hits the user question.
For example, after a user issues a user question, assuming that the executing entity finally gives one or more corresponding answers through searching and returns the answers to the user, the user tends to click the correct answer rather than the wrong answer, and thus the number of clicks of the correct answer is generally greater than that of the wrong answer.
In practical applications, the accuracy of the answer determined by reordering based on the first ranking result and the second ranking result is often higher than that determined by determining the answer based on only the first ranking result or only the second ranking result, which is described below with reference to specific embodiments.
Based on the same user click log, Normalized broken Cumulative Gain (NDCG) @1 of the three models is compared in an actual application scene. Generally, the larger the NDCG @1 value, the better the corresponding model effect), as shown in table 1:
| description of the model | NDCG@1 |
| Dssm feature-reranker | 0.46 |
| Base feature-reranker | 0.41 |
| Base+Dssm-reranker | 0.48 |
TABLE 1 NDCG @1 comparison of three sets of models
Here, the DSSM feature-reranker may be a model for obtaining the second ranking result, the baseefficiency-reranker may be a model for obtaining the first ranking result, and the Base + DSSM-reranker may be a model for obtaining the reordering result.
As can be seen from Table 1, the value of NDCG @1 for Base + Dssm-reranger is greater than the value of NDCG @1 for Dssm feature-reranger and greater than the value of NDCG @1 for Base feature-reranger, and therefore, it can be seen that this scheme of obtaining answers based on the reordering results is relatively superior.
In order to better illustrate the above reordering principle, the embodiment of the present specification further provides a schematic diagram of a reordering model, as shown in fig. 5, and the principle may specifically include:
input data to the reorder model includes: a current DSSM feature (specifically, the score) and an answer text feature (specifically, the answer information). And according to the sorting result (specifically, the second sorting result) corresponding to the DSSM feature and another sorting result (specifically, the first sorting result) corresponding to the answer text feature, performing reordering by using a LamdaMART algorithm to obtain a reordering result.
And further, combining the reordering result and the first ordering result, and performing decision operation by using a strategy model to obtain a corresponding answer.
The reordering model may be obtained by training in advance according to the DSSM feature, the answer text feature, and the click log, which are history as training data.
In this embodiment of this specification, for step S208, after obtaining the answer corresponding to the question of the user, the following may be further performed: and according to the reordering result and the first ordering result, making a decision by using a decision tree algorithm to obtain an answer corresponding to the question of the user.
The decision tree algorithm is responsible for deciding between the re-ranking result and the first ranking result and determining the answer.
An embodiment of the present specification further provides a schematic diagram of a specific working scheme of a question and answer search system in an actual application scenario, as shown in fig. 6, the steps involved in the scheme may include:
after receiving a user question sent by a user, carrying out primary screening by a search engine to obtain the answer of the front 3000;
further, the answers of the top 3000 are roughly ranked to obtain top 400 answers, and the top 400 answers can be used as the specified answer set; the rough sorting may be text matching sorting according to words, for example, a user issues a user question "how to open online payment? When rough sorting is performed, answers including the four words of "online", "payment", "how" and "opening" are arranged at the front position, and other answers are sorted in sequence according to the number of the four words.
Further, performing fine sorting according to the text features of the first 400 answers to obtain a plurality of candidate answers and a first sorting result corresponding to the candidate answers; according to the multiple candidate answers and the user behavior track information corresponding to the user question, inputting the multiple candidate answers into the DSSM to score and sort the multiple candidate answers, and obtaining the second sorting result and the score corresponding to the second sorting result; further, the first 3 answers are obtained by performing reordering according to the scores and based on the first ordering result, and the first 3 answers may be used as answers corresponding to the above-mentioned question of the user, or of course, a decision may be further made with respect to the first 3 answers and the previous ordering result to determine the answer corresponding to the question of the user.
It should be noted that the number of answers screened in each step in fig. 6 is exemplary and not limiting.
Based on the same idea, an embodiment of this specification further provides a question and answer search device, and fig. 7 is a schematic structural diagram of the question and answer search device, where the device includes:
a receiving module 701, which receives a question from a user;
an information obtaining module 702, configured to obtain user behavior trajectory information and a plurality of candidate answers;
the scoring module 703 is configured to calculate, according to the user behavior trajectory information, scores corresponding to the plurality of candidate answers respectively by using a deep-structured semantic model DSSM trained based on the user question-answer information and the user behavior trajectory information;
and the reordering module 704 is configured to order the candidate answers according to the scores and the text features of the candidate answers, and obtain answers corresponding to the user question according to the ordering result.
Based on the user behavior track information and the text characteristics, the DSSM is used for answer search, so that answers corresponding to the user question can be obtained more accurately.
In this embodiment of the present specification, the receiving module 701 receives a question from a user, and further includes: and receiving the user identification and/or the session identification corresponding to the user question through a receiving module.
The session identifier corresponding to the user question may be a session identifier between the user and the execution main body of the flow in fig. 2, or may be a session identifier between the user and one or more other service systems.
In this embodiment of this specification, the information obtaining module 702 obtains the user behavior trace information, which may specifically include: and acquiring user behavior track information according to the user identification and/or the session identification corresponding to the user question.
In an embodiment of the present specification, the user behavior trace information includes user behavior trace points within a period of time, where the user behavior trace points are remote procedure call protocol (RPC) records and/or Uniform Resource Locator (URL) records.
The user behavior trace point refers to an operation record left by a user during service operation. For example, after the user accesses the system through the APP, the user may leave an accessed RPC (Remote Procedure Call Protocol) record; after the user accesses the system through the PC, the user leaves a record of the accessed URL (Uniform Resource Locator).
In the actual use process, user behavior track points within a period of time when a user proposes a question need to be collected, and preferably, data at the moment when the user proposes the question and within 30 minutes before the moment are collected as the user behavior track points.
In an embodiment of this specification, the information obtaining module 702 obtains a plurality of candidate answers, which may specifically include: and screening a plurality of candidate answers in the specified answer set according to the text features of the question of the user and the text features of the answers contained in the specified answer set.
The specified answer set can be obtained by screening or can be obtained by predefining and classifying; based on the appointed answer set, the candidate answers are further obtained, the problem of low efficiency when the candidate answers are selected from a plurality of answers is avoided, and the efficiency of obtaining the candidate answers is improved.
In an embodiment of the present specification, the screening, according to the text features of the user question and the text features of the answers included in the specified answer set, to obtain a plurality of candidate answers in the specified answer set may specifically include: extracting the designated answer set corresponding to the user question using a predetermined search engine; sorting the answers in the specified answer set by using a LamdamAT algorithm to obtain a first sorting result; and screening to obtain a plurality of candidate answers according to the first sequencing result.
Through the search engine, preliminary search can be carried out based on the question of the user to obtain a corresponding specified answer set.
In the embodiment of the present specification, the question and answer information of the user includes question information and answer information corresponding to the question information; training to obtain the DSSM based on the user question and answer information and the user behavior track information comprises the following steps: generating a first input vector according to the problem information and the user behavior track information; generating a second input vector according to answer information corresponding to the question information; and training by using the first input vector and the second input vector to obtain the DSSM.
Combining the question information with corresponding user behavior track information, using the user behavior track information as a supplement of the question information, using the whole as a first input vector, and further training the DSSM, wherein the DSSM is favorable for more accurately acquiring answers corresponding to the question of the user when performing answer search according to the user questions.
In an embodiment of the present specification, the generating a first input vector according to the question information and the user behavior trajectory information; acquiring a problem information identifier appointed for the problem information and a track information identifier appointed for user behavior track information corresponding to the problem information; and generating a first input vector according to the problem information identifier and the track information identifier.
The problem information identifier and the track information identifier may be represented by extracting word vectors, or may be artificially defined identifiers. When the artificially defined identifier is adopted, a mapping relation between the question information and the question information identifier and a mapping relation between the user behavior track information and the track information identifier need to be established in advance. Further, according to the two identifications, a first input vector is generated.
In an embodiment of this specification, the generating a second input vector according to answer information corresponding to the question information may specifically include: extracting knowledge point title word segmentation of answer information corresponding to the question information; and generating a second input vector according to the knowledge point title word segmentation.
And training the DSSM by adopting a mode of generating a second input vector through knowledge point title participles corresponding to answer information corresponding to the question information, wherein the knowledge point title participles are used for extracting main information as a title of the answer according to the content of the answer.
In an embodiment of this specification, the training to obtain the DSSM by using the first input vector and the second input vector may specifically include: inputting the first input vector into a DSSM prototype to obtain a first output result; inputting the second input vector into the DSSM prototype to obtain a second output result; and calculating the correlation between the first output result and the second output result, and training the DSSM prototype according to the correlation to obtain the DSSM.
In the training process, combining the problem information with corresponding user behavior track information, and using the combined information as a first input vector for training the DSSM.
In an embodiment of the present specification, the ranking the multiple candidate answers according to the scores and the text features of the multiple candidate answers, and obtaining the answer corresponding to the question of the user according to the ranking result may specifically include: sorting by using a Lamdamard algorithm according to the scores and the text characteristics of the candidate answers to obtain a second sorting result; reordering according to the first ordering result and the second ordering result to obtain a reordering result; and obtaining an answer corresponding to the question of the user according to the reordering result.
And obtaining a second ranking result of the multiple candidate answers by using a Lamdamard algorithm according to the matrix or vector corresponding to the score obtained through the DSSM and the matrix or vector corresponding to the text features of the multiple candidate answers, and then selecting the first answers of the second ranking result as correct answers to send to the user, or sending all the answers to the user according to the ranking.
In an embodiment of the present specification, the reordering the plurality of candidate answers according to the first ordering result and the second ordering result to obtain a reordered result may specifically include: according to the first sequencing result and the second sequencing result, the multiple candidate answers are reordered by utilizing a reordering model obtained by training, and a reordering result is obtained; wherein training the reordered model comprises: acquiring a user click log; and training to obtain the reordering model by utilizing a LamdamAT algorithm according to the user click log and the score.
In an embodiment of this specification, the obtaining an answer corresponding to the question of the user according to the reordering result may specifically include: and according to the reordering result and the first ordering result, making a decision by using a decision tree algorithm to obtain an answer corresponding to the question of the user. By using the decision tree algorithm, a choice is made between the reordering result and the first ordering result, and the accuracy of obtaining the answer under the condition of insufficient question information can be effectively improved.
The following also provides examples of comparison of the effects of answers based on different schemes:
the scheme of the specification helps the execution main body to better analyze the question of the user by using the behavior track information left by the user before asking the question, thereby being beneficial to improving the accuracy of the corresponding answer obtained aiming at the user question which is difficult to accurately understand. See the examples in table 2:
the first column is user behavior track information and comprises IDs corresponding to all behavior track points;
the second column is a user question, the third column is an answer obtained by a scheme other than the scheme of the present specification, and the fourth column is an answer obtained by the scheme of the present specification.
TABLE 2 comparison of question and answer effects based on DSSM
In the above examples in table 2, either the user simply asks for a sentence and lacks necessary information; or describe a specific scene with too much interference information in a very complicated way. If the answer is searched by simply adopting a mode such as text matching, the answer really needed by the user is difficult to obtain, but if the scheme of the specification is adopted, the deep semantic association between the user behavior track information and the question and the corresponding answer of the user can be explored through the DSSM, so that the answer can be obtained more accurately.
For another example, when the question of the user is "how to pay", fig. 8a and 8b are schematic diagrams of the effect of the question and answer search scheme using the present specification in an actual application scenario, and the following are specifically compared:
the question of the user is only "how to pay", which is fuzzy because there may be a plurality of functional units having a payment function, which makes it impossible to determine which functional unit the user wants to pay based on, and furthermore, it is difficult to accurately give a corresponding answer (for example, a payment instruction) to "how to pay".
In fig. 8a, a user sends a user question "how to repay" to a question-and-answer robot, and the recent user behavior track information found by the question-and-answer robot by the user is: visit credit borrowing function unit → check bill → question and answer robot; the answer given by the question-answering robot may be: an operation description of "how to repay the credit loan".
In fig. 8b, the user also sends "how to repay" to the question-and-answer robot, and the track information of the user behavior found by the question-and-answer robot by the second user is: accessing a loan function unit → viewing bills → a question and answer robot; the answer given by the question-answering robot may be: an operation description of "how to repay the loan".
It can be seen that in fig. 8a and 8b, the user issues the same user question, but the services related to the user behavior trajectory information before the two questions are obviously different, and based on the scheme of the present specification, the answer is searched based on the user behavior trajectory information, so that the respective real intentions of the two questions of the user can be more accurately understood, and thus, a more accurate answer is given.
Based on the same idea, an embodiment of this specification further provides an electronic device, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
to receive user question;
acquiring user behavior track information and a plurality of candidate answers;
according to the user behavior track information, calculating scores corresponding to the candidate answers respectively by using a deep-structured semantic model DSSM obtained based on the user question-answer information and the user behavior track information;
and sequencing the candidate answers according to the scores and the text characteristics of the candidate answers, and obtaining answers corresponding to the question of the user according to the sequencing result.
Based on the user behavior track information and the text characteristics, the DSSM is used for answer search, so that answers corresponding to the user question can be obtained more accurately.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the embodiments of the apparatus, the electronic device, and the nonvolatile computer storage medium, since they are substantially similar to the embodiments of the method, the description is simple, and the relevant points can be referred to the partial description of the embodiments of the method.
The apparatus, the electronic device, the nonvolatile computer storage medium and the method provided in the embodiments of the present description correspond to each other, and therefore, the apparatus, the electronic device, and the nonvolatile computer storage medium also have similar advantageous technical effects to the corresponding method.
In the 90 s of the 20 th century, improvements in a technology could clearly distinguish between improvements in hardware (e.g., improvements in circuit structures such as diodes, transistors, switches, etc.) and improvements in software (improvements in process flow). However, as technology advances, many of today's process flow improvements have been seen as direct improvements in hardware circuit architecture. Designers almost always obtain the corresponding hardware circuit structure by programming an improved method flow into the hardware circuit. Thus, it cannot be said that an improvement in the process flow cannot be realized by hardware physical modules. For example, a Programmable Logic Device (PLD), such as a Field Programmable Gate Array (FPGA), is an integrated circuit whose Logic functions are determined by programming the Device by a user. A digital system is "integrated" on a PLD by the designer's own programming without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Furthermore, nowadays, instead of manually making an integrated Circuit chip, such Programming is often implemented by "logic compiler" software, which is similar to a software compiler used in program development and writing, but the original code before compiling is also written by a specific Programming Language, which is called Hardware Description Language (HDL), and HDL is not only one but many, such as abel (advanced Boolean Expression Language), ahdl (alternate Language Description Language), traffic, pl (core unified Programming Language), HDCal, JHDL (Java Hardware Description Language), langue, Lola, HDL, laspam, hardsradware (Hardware Description Language), vhjhd (Hardware Description Language), and vhigh-Language, which are currently used in most common. It will also be apparent to those skilled in the art that hardware circuitry that implements the logical method flows can be readily obtained by merely slightly programming the method flows into an integrated circuit using the hardware description languages described above.
The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, an Application Specific Integrated Circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic for the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may thus be considered a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the various elements may be implemented in the same one or more software and/or hardware implementations in implementing one or more embodiments of the present description.
As will be appreciated by one skilled in the art, the present specification embodiments may be provided as a method, system, or computer program product. Accordingly, embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
The description has been presented with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the description. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
This description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only an example of the present specification, and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (27)
1. A question-answer searching method comprises the following steps:
receiving a question of a user;
acquiring user behavior track information and a plurality of candidate answers;
according to the user behavior track information, calculating scores corresponding to the candidate answers respectively by using a deep-structured semantic model DSSM obtained based on the user question-answer information and the user behavior track information;
and sequencing the candidate answers according to the scores and the text characteristics of the candidate answers, and obtaining answers corresponding to the question of the user according to the sequencing result.
2. The method of claim 1, the receiving a user question, further comprising: and acquiring a user identifier and/or a session identifier corresponding to the user question.
3. The method according to claim 2, wherein the acquiring of the user behavior trace information specifically includes:
and acquiring user behavior track information according to the user identification and/or the session identification corresponding to the user question.
4. The method of claim 3, wherein the user behavior trace information comprises user behavior trace points over a period of time, the user behavior trace points being remote procedure call protocol (RPC) records and/or Uniform Resource Locator (URL) records.
5. The method of claim 1, wherein the obtaining a plurality of candidate answers specifically comprises:
and screening a plurality of candidate answers in the specified answer set according to the text features of the question of the user and the text features of the answers contained in the specified answer set.
6. The method according to claim 5, wherein the step of obtaining a plurality of candidate answers by filtering in the specified answer set according to the text features of the user question and the text features of the answers included in the specified answer set specifically comprises:
extracting the designated answer set corresponding to the user question using a predetermined search engine;
sorting the answers in the specified answer set by using a LamdamAT algorithm to obtain a first sorting result;
and screening to obtain a plurality of candidate answers according to the first sequencing result.
7. The method of claim 1, wherein the user question-answer information comprises question information and corresponding answer information thereof; training to obtain the DSSM based on the user question and answer information and the user behavior track information comprises the following steps:
generating a first input vector according to the problem information and the user behavior track information; and the number of the first and second groups,
generating a second input vector according to answer information corresponding to the question information;
and training by using the first input vector and the second input vector to obtain the DSSM.
8. The method of claim 7, generating a first input vector based on the question information and user behavior trajectory information;
acquiring a problem information identifier appointed for the problem information and a track information identifier appointed for user behavior track information corresponding to the problem information;
and generating a first input vector according to the problem information identifier and the track information identifier.
9. The method according to claim 7, wherein the generating a second input vector according to the answer information corresponding to the question information specifically includes:
extracting knowledge point title word segmentation of answer information corresponding to the question information;
and generating a second input vector according to the knowledge point title word segmentation.
10. The method of claim 7, wherein training with the first input vector and the second input vector to obtain DSSM specifically comprises:
inputting the first input vector into a DSSM prototype to obtain a first output result; and the number of the first and second groups,
inputting the second input vector into the DSSM prototype to obtain a second output result;
and calculating the correlation between the first output result and the second output result, and training the DSSM prototype according to the correlation to obtain the DSSM.
11. The method according to claim 6, wherein the ranking the candidate answers according to the scores and the text features of the candidate answers, and obtaining the answer corresponding to the question from the user according to the ranking result, specifically comprises:
according to the scores and the text features of the candidate answers, ranking the candidate answers by using a Lamdamard algorithm to obtain a second ranking result;
according to the first sorting result and the second sorting result, the candidate answers are reordered to obtain a reordering result;
and obtaining an answer corresponding to the question of the user according to the reordering result.
12. The method of claim 11, wherein the reordering the plurality of candidate answers according to the first and second ranking results to obtain a reordered result comprises:
according to the first sequencing result and the second sequencing result, the multiple candidate answers are reordered by utilizing a reordering model obtained by training, and a reordering result is obtained;
wherein training the reordered model comprises:
acquiring a user click log;
and training to obtain the reordering model by utilizing a LamdamAT algorithm according to the user click log and the score.
13. The method according to claim 11, wherein the obtaining the answer corresponding to the question from the user according to the reordering result specifically comprises:
and according to the reordering result and the first ordering result, making a decision by using a decision tree algorithm to obtain an answer corresponding to the question of the user.
14. A question-answer search apparatus comprising:
the receiving module is used for receiving a question of a user;
the information acquisition module is used for acquiring user behavior track information and a plurality of candidate answers;
the scoring module is used for calculating scores corresponding to the candidate answers according to the user behavior track information by utilizing a deep structured semantic model DSSM obtained based on the user question-answer information and the user behavior track information;
and the reordering module is used for ordering the candidate answers according to the scores and the text characteristics of the candidate answers and obtaining answers corresponding to the question of the user according to the ordering result.
15. The apparatus of claim 14, the receiving module to receive a user question, further comprising: and acquiring a user identifier and/or a session identifier corresponding to the user question through the receiving module.
16. The apparatus according to claim 15, wherein the information obtaining module obtains the user behavior trace information, and specifically includes:
and acquiring user behavior track information according to the user identification and/or the session identification corresponding to the user question.
17. The apparatus of claim 16, the user behavior trace information comprising user behavior trace points over a period of time, the user behavior trace points being remote procedure call protocol (RPC) records and/or Uniform Resource Locator (URL) records.
18. The apparatus of claim 14, wherein the information obtaining module obtains a plurality of candidate answers, and specifically comprises:
and screening a plurality of candidate answers in the specified answer set according to the text features of the question of the user and the text features of the answers contained in the specified answer set.
19. The apparatus according to claim 18, wherein the step of filtering out a plurality of candidate answers from the specified answer set according to the text features of the user question and the text features of the answers included in the specified answer set includes:
extracting the designated answer set corresponding to the user question using a predetermined search engine;
sorting the answers in the specified answer set by using a LamdamAT algorithm to obtain a first sorting result;
and screening to obtain a plurality of candidate answers according to the first sequencing result.
20. The apparatus of claim 14, wherein the user question-answer information comprises question information and corresponding answer information thereof; training to obtain the DSSM based on the user question and answer information and the user behavior track information comprises the following steps:
generating a first input vector according to the problem information and the user behavior track information; and the number of the first and second groups,
generating a second input vector according to answer information corresponding to the question information;
and training by using the first input vector and the second input vector to obtain the DSSM.
21. The apparatus of claim 20, said generating a first input vector based on said question information and user behavior trajectory information;
acquiring a problem information identifier appointed for the problem information and a track information identifier appointed for user behavior track information corresponding to the problem information;
and generating a first input vector according to the problem information identifier and the track information identifier.
22. The apparatus according to claim 20, wherein the generating a second input vector according to the answer information corresponding to the question information specifically includes:
extracting knowledge point title word segmentation of answer information corresponding to the question information;
and generating a second input vector according to the knowledge point title word segmentation.
23. The apparatus of claim 20, wherein the training with the first input vector and the second input vector to obtain DSSM comprises:
inputting the first input vector into a DSSM prototype to obtain a first output result; and the number of the first and second groups,
inputting the second input vector into the DSSM prototype to obtain a second output result;
and calculating the correlation between the first output result and the second output result, and training the DSSM prototype according to the correlation to obtain the DSSM.
24. The apparatus according to claim 20, wherein the ranking the candidate answers according to the scores and the text features of the candidate answers, and obtaining the answer corresponding to the question from the user according to the ranking result, specifically comprises:
according to the scores and the text features of the candidate answers, ranking the candidate answers by using a Lamdamard algorithm to obtain a second ranking result;
according to the first sorting result and the second sorting result, the candidate answers are reordered to obtain a reordering result;
and obtaining an answer corresponding to the question of the user according to the reordering result.
25. The apparatus of claim 24, wherein the reordering the plurality of candidate answers according to the first and second ranking results to obtain a reordered result comprises:
according to the first sequencing result and the second sequencing result, the multiple candidate answers are reordered by utilizing a reordering model obtained by training, and a reordering result is obtained;
wherein training the reordered model comprises:
acquiring a user click log;
and training to obtain the reordering model by utilizing a LamdamAT algorithm according to the user click log and the score.
26. The apparatus according to claim 24, wherein the obtaining of the answer corresponding to the question from the user according to the re-ranking result specifically comprises:
and according to the reordering result and the first ordering result, making a decision by using a decision tree algorithm to obtain an answer corresponding to the question of the user.
27. An electronic device, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
to receive user question;
acquiring user behavior track information and a plurality of candidate answers;
according to the user behavior track information, calculating scores corresponding to the candidate answers respectively by using a deep-structured semantic model DSSM obtained based on the user question-answer information and the user behavior track information;
and sequencing the candidate answers according to the scores and the text characteristics of the candidate answers, and obtaining answers corresponding to the question of the user according to the sequencing result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710769081.5A CN107679082A (en) | 2017-08-31 | 2017-08-31 | Question and answer searching method, device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710769081.5A CN107679082A (en) | 2017-08-31 | 2017-08-31 | Question and answer searching method, device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107679082A true CN107679082A (en) | 2018-02-09 |
Family
ID=61135568
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710769081.5A Pending CN107679082A (en) | 2017-08-31 | 2017-08-31 | Question and answer searching method, device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107679082A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108962228A (en) * | 2018-07-16 | 2018-12-07 | 北京百度网讯科技有限公司 | model training method and device |
| CN108984658A (en) * | 2018-06-28 | 2018-12-11 | 阿里巴巴集团控股有限公司 | A kind of intelligent answer data processing method and device |
| CN108984475A (en) * | 2018-07-06 | 2018-12-11 | 北京慧闻科技发展有限公司 | Answer selection method, device and electronic equipment based on holographic neural network |
| CN108984666A (en) * | 2018-06-29 | 2018-12-11 | 阿里巴巴集团控股有限公司 | Data processing method, data processing equipment and server |
| CN109065154A (en) * | 2018-07-27 | 2018-12-21 | 清华大学 | A kind of result of decision determines method, apparatus, equipment and readable storage medium storing program for executing |
| CN109086417A (en) * | 2018-08-06 | 2018-12-25 | 清华大学 | Search for evaluation method and device |
| CN109376298A (en) * | 2018-09-14 | 2019-02-22 | 广州神马移动信息科技有限公司 | Data processing method, device, terminal device and computer storage medium |
| CN110008320A (en) * | 2019-03-05 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Method and apparatus for answering customer problem |
| CN110008322A (en) * | 2019-03-25 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Art recommended method and device under more wheel session operational scenarios |
| CN110059178A (en) * | 2019-02-12 | 2019-07-26 | 阿里巴巴集团控股有限公司 | Problem distributing method and device |
| CN110909145A (en) * | 2019-11-29 | 2020-03-24 | 支付宝(杭州)信息技术有限公司 | Training method and device for multi-task model |
| CN111309882A (en) * | 2020-02-13 | 2020-06-19 | 支付宝(杭州)信息技术有限公司 | Method and device for realizing intelligent customer service question answering |
| WO2020143346A1 (en) * | 2019-01-10 | 2020-07-16 | 北京三快在线科技有限公司 | Sorting |
| CN111428125A (en) * | 2019-01-10 | 2020-07-17 | 北京三快在线科技有限公司 | Sorting method and device, electronic equipment and readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012162822A1 (en) * | 2011-05-27 | 2012-12-06 | International Business Machines Corporation | Automated self-service user support based on ontology analysis |
| CN105308591A (en) * | 2014-03-03 | 2016-02-03 | 微软技术许可有限责任公司 | Dynamics of tie strength from social interaction |
| CN106202159A (en) * | 2016-06-23 | 2016-12-07 | 深圳追科技有限公司 | A kind of man-machine interaction method of customer service system |
| CN106415535A (en) * | 2014-04-14 | 2017-02-15 | 微软技术许可有限责任公司 | Context-sensitive search using a deep learning model |
| CN106649786A (en) * | 2016-12-28 | 2017-05-10 | 北京百度网讯科技有限公司 | Deep question answer-based answer retrieval method and device |
| CN106663124A (en) * | 2014-08-11 | 2017-05-10 | 微软技术许可有限责任公司 | Generating and using a knowledge-enhanced model |
-
2017
- 2017-08-31 CN CN201710769081.5A patent/CN107679082A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012162822A1 (en) * | 2011-05-27 | 2012-12-06 | International Business Machines Corporation | Automated self-service user support based on ontology analysis |
| CN105308591A (en) * | 2014-03-03 | 2016-02-03 | 微软技术许可有限责任公司 | Dynamics of tie strength from social interaction |
| CN106415535A (en) * | 2014-04-14 | 2017-02-15 | 微软技术许可有限责任公司 | Context-sensitive search using a deep learning model |
| CN106663124A (en) * | 2014-08-11 | 2017-05-10 | 微软技术许可有限责任公司 | Generating and using a knowledge-enhanced model |
| CN106202159A (en) * | 2016-06-23 | 2016-12-07 | 深圳追科技有限公司 | A kind of man-machine interaction method of customer service system |
| CN106649786A (en) * | 2016-12-28 | 2017-05-10 | 北京百度网讯科技有限公司 | Deep question answer-based answer retrieval method and device |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108984658A (en) * | 2018-06-28 | 2018-12-11 | 阿里巴巴集团控股有限公司 | A kind of intelligent answer data processing method and device |
| CN108984666A (en) * | 2018-06-29 | 2018-12-11 | 阿里巴巴集团控股有限公司 | Data processing method, data processing equipment and server |
| CN108984475A (en) * | 2018-07-06 | 2018-12-11 | 北京慧闻科技发展有限公司 | Answer selection method, device and electronic equipment based on holographic neural network |
| CN108962228B (en) * | 2018-07-16 | 2022-03-15 | 北京百度网讯科技有限公司 | Model training method and device |
| CN108962228A (en) * | 2018-07-16 | 2018-12-07 | 北京百度网讯科技有限公司 | model training method and device |
| CN109065154A (en) * | 2018-07-27 | 2018-12-21 | 清华大学 | A kind of result of decision determines method, apparatus, equipment and readable storage medium storing program for executing |
| CN109086417A (en) * | 2018-08-06 | 2018-12-25 | 清华大学 | Search for evaluation method and device |
| CN109376298B (en) * | 2018-09-14 | 2022-01-25 | 阿里巴巴（中国）有限公司 | Data processing method and device, terminal equipment and computer storage medium |
| CN109376298A (en) * | 2018-09-14 | 2019-02-22 | 广州神马移动信息科技有限公司 | Data processing method, device, terminal device and computer storage medium |
| WO2020143346A1 (en) * | 2019-01-10 | 2020-07-16 | 北京三快在线科技有限公司 | Sorting |
| CN111428125A (en) * | 2019-01-10 | 2020-07-17 | 北京三快在线科技有限公司 | Sorting method and device, electronic equipment and readable storage medium |
| CN111428125B (en) * | 2019-01-10 | 2023-05-30 | 北京三快在线科技有限公司 | Ordering method, ordering device, electronic equipment and readable storage medium |
| CN110059178A (en) * | 2019-02-12 | 2019-07-26 | 阿里巴巴集团控股有限公司 | Problem distributing method and device |
| CN110008320A (en) * | 2019-03-05 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Method and apparatus for answering customer problem |
| CN110008320B (en) * | 2019-03-05 | 2023-10-27 | 创新先进技术有限公司 | Method and device for answering user questions |
| CN110008322B (en) * | 2019-03-25 | 2023-04-07 | 创新先进技术有限公司 | Method and device for recommending dialogues in multi-turn conversation scene |
| CN110008322A (en) * | 2019-03-25 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Art recommended method and device under more wheel session operational scenarios |
| CN110909145A (en) * | 2019-11-29 | 2020-03-24 | 支付宝(杭州)信息技术有限公司 | Training method and device for multi-task model |
| CN110909145B (en) * | 2019-11-29 | 2022-08-09 | 支付宝(杭州)信息技术有限公司 | Training method and device for multi-task model |
| CN111309882A (en) * | 2020-02-13 | 2020-06-19 | 支付宝(杭州)信息技术有限公司 | Method and device for realizing intelligent customer service question answering |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107679082A (en) | Question and answer searching method, device and electronic equipment | |
| CN110427463B (en) | Search statement response method and device, server and storage medium | |
| CN111177569A (en) | Recommendation processing method, device and equipment based on artificial intelligence | |
| CN109299245B (en) | Method and device for recalling knowledge points | |
| CN112799747A (en) | Intelligent assistant evaluation and recommendation method, system, terminal and readable storage medium | |
| CN117609444B (en) | Searching question-answering method based on large model | |
| CN110765247A (en) | Input prompting method and device for question-answering robot | |
| CN110033382B (en) | Insurance service processing method, device and equipment | |
| CN110795544B (en) | Content searching method, device, equipment and storage medium | |
| CN116796857A (en) | LLM model training method, device, equipment and storage medium thereof | |
| CN117520503A (en) | Financial customer service dialogue generation method, device, equipment and medium based on LLM model | |
| Yen et al. | Unanswerable question correction in question answering over personal knowledge base | |
| CN110717008B (en) | Search result ordering method and related device based on semantic recognition | |
| CN111259975B (en) | Method and device for generating classifier and method and device for classifying text | |
| CN108681490A (en) | For the vector processing method, device and equipment of RPC information | |
| CN110019556A (en) | A kind of topic news acquisition methods, device and its equipment | |
| CN112131884A (en) | Method and device for entity classification and method and device for entity presentation | |
| CN108509571B (en) | General method for webpage information data mining | |
| CN110008352A (en) | Entity finds method and device | |
| CN116028626A (en) | Text matching method and device, storage medium and electronic equipment | |
| CN115827847A (en) | Knowledge graph construction method and device, electronic equipment and storage medium | |
| Qiu et al. | Deep active learning with crowdsourcing data for privacy policy classification | |
| CN111126066A (en) | Method and device for determining Chinese retrieval method based on neural network | |
| CN115658935B (en) | Personalized comment generation method and device | |
| CN118070925B (en) | Model training method, device, electronic equipment, storage medium and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1248867 Country of ref document: HK |
|
| TA01 | Transfer of patent application right |
Effective date of registration: 20200927 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20200927 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180209 |
|
| RJ01 | Rejection of invention patent application after publication |