CN110096702B - Subjective question scoring method and device - Google Patents
Subjective question scoring method and device Download PDFInfo
- Publication number
- CN110096702B CN110096702B CN201910323564.1A CN201910323564A CN110096702B CN 110096702 B CN110096702 B CN 110096702B CN 201910323564 A CN201910323564 A CN 201910323564A CN 110096702 B CN110096702 B CN 110096702B
- Authority
- CN
- China
- Prior art keywords
- data
- score
- answer data
- semantic similarity
- test paper
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Electrically Operated Instructional Devices (AREA)
- Machine Translation (AREA)
Abstract
The embodiment of the application provides a subjective question evaluation method and device, wherein the method comprises the following steps: receiving score associated data of subjective questions; calculating the semantic similarity value of the standard answer data and the test paper answer data; calculating a basic score coefficient of the subjective question by using a basic score equation corresponding to the stem data and the semantic similarity value; calculating an additional score coefficient corresponding to the answer data of the test paper and a preset additional score dimension; and calculating a final score coefficient corresponding to the ontology vector by using a final score model, and obtaining a final score by calculating the product of the final score coefficient and the total score of the subjective questions. The final score obtained by the method comprises the judgment of the subjective questions by a plurality of dimensions, so that the score is closer to the real demand, the accuracy is higher, the workload of manual rechecking is greatly reduced, the scoring efficiency is improved, and the problems of low accuracy and low efficiency when the subjective questions are scored by the automatic scoring mode of the existing system can be effectively solved.
Description
Technical Field
The present disclosure relates to the field of natural language processing, and in particular, to a subjective question scoring method and apparatus.
Background
With the development of computer networks, network examination has become one of the main examination forms. After the computer for the examination is connected to the appointed network system by the main examination party, the examination staff answers on the computer and uploads the answers of the examination papers to the network system so as to score the examination papers by the examination staff. Generally, the scoring mode of the test paper answers mainly comprises two modes of manual paper examination and automatic system paper examination and manual rechecking.
Because the manual paper marking mode has high cost and low efficiency, the manual paper marking mode is gradually replaced by the automatic paper marking mode of the system. At present, the automatic scoring mode of the system is mainly applied to scoring objective questions. With the continuous development of natural language processing technology, the automatic scoring mode of the system is gradually applied to the scoring of subjective questions. The main method of utilizing the automatic examination paper mode of the system is that keywords are extracted from examination paper answers and standard answers of examination workers respectively, the co-occurrence degree of the keywords of the examination paper answers and the standard answers is calculated, and corresponding scores are correspondingly obtained according to the calculated co-occurrence degree; or calculating the similarity of the test paper answers and the standard answers of the testee by using a simple semantic similarity calculation model, and correspondingly obtaining corresponding scores according to the calculated similarity.
However, because the answers corresponding to the subjective questions have subjectivity, and the language expression forms of different examinees are different, namely, the answers of the test paper of different examinees are different in terms of the keywords expressing the same semantic, the difficulty and the accuracy of scoring by calculating the co-occurrence degree of the keywords are increased; or, the score is judged only by the parameter of semantic similarity, so that the requirements of a real data environment, such as smoothness, integrity and the like of sentences, are easily difficult to meet. Meanwhile, in order to solve the problem of low accuracy in scoring subjective questions in an automatic scoring mode of the system, the system can go on to go through manual rechecking or spot check, so that the cost is still increased, and the scoring efficiency is reduced.
Disclosure of Invention
The application provides a subjective question scoring method and device, which are used for solving the problems of low accuracy and low efficiency when subjective questions are scored in an automatic scoring mode of an existing system.
In a first aspect, an embodiment of the present application provides a subjective question scoring method, including:
receiving score-related data of subjective questions, the score-related data comprising: question stem data, standard answer data and test paper answer data;
calculating the semantic similarity value of the standard answer data and the test paper answer data;
Calculating a basic score coefficient of the subjective question by using a basic score equation corresponding to the stem data and the semantic similarity value;
calculating an additional score coefficient corresponding to the answer data of the test paper and a preset additional score dimension;
calculating a final score coefficient corresponding to an ontology vector by using a final score model, wherein the ontology vector is generated by the basic score coefficient and each additional score coefficient;
a final score is calculated, the final score being the product of the final score coefficient and the total score of the subjective questions.
In a second aspect, the present application provides a subjective question scoring apparatus, including:
the data receiving module is used for receiving the grading association data of the subjective questions, and the grading association data comprises: question stem data, standard answer data and test paper answer data;
the similarity calculation module is used for calculating the semantic similarity value of the standard answer data and the test paper answer data;
the basic score coefficient calculation module is used for calculating the basic score coefficient of the subjective question by utilizing a basic score equation corresponding to the stem data and the semantic similarity value;
the additional score coefficient calculation module is used for calculating an additional score coefficient corresponding to the answer data of the test paper and a preset additional score dimension;
The final score coefficient calculation module is used for calculating a final score coefficient corresponding to an ontology vector by utilizing a final score model, wherein the ontology vector is generated by the basic score coefficient and each additional score coefficient;
and the final score calculation module is used for calculating a final score, wherein the final score is the product of the final score coefficient and the total score of the subjective questions.
According to the subjective question scoring method and device, the semantic similarity value of standard answer data and test paper answer data in the score associated data is calculated by receiving the score associated data of the subjective questions; and simultaneously, calculating a basic score coefficient of the subjective question by using a basic score equation corresponding to the stem data in the score associated data and the calculated semantic similarity value. In order to make the scoring accuracy of the subjective questions higher, the scoring dimension of the subjective questions is increased, namely, an additional scoring coefficient corresponding to the preset additional scoring dimension of the answer data of the calculated test paper is calculated. And deeply calculating a basic score coefficient and an additional score coefficient of the subjective questions by using a final score model to obtain a final score coefficient, and obtaining a final score by calculating the product of the final score coefficient and the total score of the subjective questions. The final score comprises the judgment of subjective questions in multiple dimensions, so that the score is closer to the real demand, the accuracy is higher, the workload of manual review is greatly reduced, and the scoring efficiency is further improved. Therefore, the subjective question scoring method provided by the application can effectively solve the problems of low accuracy and low efficiency when the subjective questions are scored in an automatic paper marking mode of the existing system.
Drawings
In order to more clearly illustrate the technical solutions of the present application, the drawings that are needed in the embodiments will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
Fig. 1 is a flowchart of a subjective question scoring method according to an embodiment of the present application;
fig. 2 is a flowchart of a subjective question scoring method step S2 provided in the embodiment of the present application;
FIG. 3 is a flowchart of a method for determining the type of a subjective question according to an embodiment of the present application;
FIG. 4 is a flowchart of a method for calculating a noun interpretation base score coefficient according to an embodiment of the disclosure;
FIG. 5 is a flowchart of a method for calculating a translation base score coefficient according to an embodiment of the present application;
FIG. 6 is a flowchart of a method for calculating a basic scoring coefficient for a simple answer according to an embodiment of the present application;
FIG. 7 is a flowchart of a method for calculating semantic similarity of score points according to an embodiment of the present application;
FIG. 8 is a flowchart of another method for calculating a base scoring coefficient for a simple question according to an embodiment of the present application;
FIG. 9 is a flowchart of a method for training a final scoring model provided in an embodiment of the present application;
fig. 10 is a schematic diagram of a subjective question scoring apparatus according to an embodiment of the present application.
Detailed Description
In order to better understand the technical solutions in the present application, the following description will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
With the development of computer networks, network examination has become one of the main examination forms. After the computer for the examination is connected to the appointed network system by the main examination party, the examination staff answers on the computer and uploads the answers of the examination papers to the network system so as to score the examination papers by the examination staff. Generally, the scoring mode of the test paper answers mainly comprises two modes of manual paper examination and automatic system paper examination and manual rechecking.
Because the manual paper marking mode has high cost and low efficiency, the manual paper marking mode is gradually replaced by the automatic paper marking mode of the system. At present, the automatic scoring mode of the system is mainly applied to scoring objective questions. With the continuous development of natural language processing technology, the automatic scoring mode of the system is gradually applied to the scoring of subjective questions. The main method of utilizing the automatic examination paper mode of the system is that the key words are extracted from the examination paper answer and the standard answer of the examination paper respectively, and the co-occurrence degree of the key words of the examination paper answer and the standard answer is calculated, wherein the co-occurrence degree is the ratio of the intersection of the key words of the examination paper answer and the standard answer to the union of the key words of the examination paper answer and the standard answer. Corresponding to the calculated co-occurrence degree, obtaining corresponding scores; or calculating the similarity of the test paper answers and the standard answers of the testee by using a simple semantic similarity calculation model, and correspondingly obtaining corresponding scores according to the calculated similarity.
However, because the answers corresponding to the subjective questions have subjectivity, and the language expression forms of different examinees are different, namely, the answers of the test paper of different examinees are different in terms of the keywords expressing the same semantic, the difficulty and the accuracy of scoring by calculating the co-occurrence degree of the keywords are increased; or, the score is judged only by the parameter of semantic similarity, so that the requirements of a real data environment, such as smoothness, integrity and the like of sentences, are easily difficult to meet. Meanwhile, in order to solve the problem of low accuracy in scoring subjective questions in an automatic scoring mode of the system, the system can go on to go through manual rechecking or spot check, so that the cost is still increased, and the scoring efficiency is reduced.
Therefore, the subjective questions are reviewed by adopting the automatic examination method of the existing system, and although a certain manual burden can be reduced, the problems of low scoring accuracy and non-compliance with real requirements still exist, and in order to solve the problems, manual rechecking is still needed, so that the examination efficiency cannot be truly improved.
In order to solve the above problems, the embodiments of the present application provide a subjective question scoring method and apparatus.
The following are method embodiments of the present application.
Fig. 1 is a flowchart of a subjective question scoring method according to an embodiment of the present application. The method can be applied to various equipment capable of scoring, such as a server, a PC (personal computer), a tablet personal computer, a mobile phone and the like.
Referring to fig. 1, the method includes the steps of:
s1, receiving score associated data of subjective questions, wherein the score associated data comprises: stem data, standard answer data, and test paper answer data.
And uploading the grading related data to the subjective question grading device through an interface of the subjective question grading device, wherein the transmission mode can be wired network transmission, wireless network transmission or real-time network transmission. The grading association data at least comprises the stem data of the subjective questions, the answer data of test papers which are answered by the examinee aiming at the subjective questions and the standard answer data corresponding to the subjective questions, and meanwhile, the grading association data can also comprise the environment data (whether network data except an examination network is used) of the examinee when answering, the discipline data (whether the illicit data of the invigilation record exists) of the examinee when answering, the answering time (aiming at the subjective questions which have answer time requirements) and the like.
Because the subjective question scoring device needs to improve the scoring accuracy and make the scoring closer to the real data, the scoring association data should be greatly expanded to make the scoring association data contain more data except answer data, such as environment data, network data, discipline data and the like, which can influence the answer of the test paper.
S2, calculating the semantic similarity value of the standard answer data and the test paper answer data.
Specifically, a TNet network model can be used to calculate the semantic similarity value of standard answer data and test paper answer data, namely, calculate the similarity degree of two sections of text answers. Specifically, firstly, the semanteme of the standard answer data and the test paper answer data is analyzed, the semanteme vector of the standard answer data and the test paper answer data is determined, then the distance between the semanteme vector of the standard answer data and the test paper answer data is calculated through a TNet network model, and finally, the semanteme similarity value of the semanteme standard answer data and the test paper answer data is obtained through training. The calculated semantic similarity value is in [0,1], and the higher the semantic similarity value is, the higher the similarity between the answer data of the test paper and the standard answer data is, and especially when the semantic similarity value is equal to 1, the answer data of the test paper is completely the same as the standard answer data, and special attention is still required in the situation.
Illustratively, english translation questions:
translation 1: a lever;
standard answer data 1: loving;
test paper answer data 1: loving people.
Translation 2: no man or woman is worth your tears, and the one who is, won't make you cry.
Standard answer data 2: no one is worth tearing and no one is worth doing so would not cry.
Test paper answer data 2: no one is worth tearing and no one is worth doing so would not cry.
Obviously, in the example, the answer data of the test paper is completely consistent with the standard answer data. However, in fact, the answers about two translation questions are not only one answer, for example, a lover, but also translated into a lover, and the semantics of the lover can still be classified as the answers similar to the standard answer data because the lover and lover are close words. For long sentence translation, because of the difference of personal expression forms and the existence of the paraphrasing in Chinese, the condition that the answer data of the test paper is completely consistent with the standard answer data is more difficult to appear, and therefore, once the condition appears, the answer data of the test paper needs to be further judged.
For example, for the answer data of the test paper with the marked semantic similarity value equal to 1, additional scores of other scoring dimensions corresponding to the answer data of the test paper are required to be further calculated, or the number of subjective questions with the semantic similarity value equal to 1 between the answer data of the test paper and the standard answer data is counted, and the ratio of the number of the subjective questions to the total number of the subjective questions is calculated, if the ratio is higher than a preset ratio threshold, the fact that the cheating suspicion exists in the test paper is indicated; if the ratio is lower than a preset ratio threshold, the examinee is suspected; or the ratio is also used as one item of the additional score, and the answer of the test paper is further judged, so that the problems of plagiarism, answer leakage and the like are avoided.
Therefore, the semantic similarity between the answer data of the test paper and the standard answer data is not simply judged, so that the real situation of answering the subjective questions is reflected, and when the answer data of the test paper is abnormal, for example, the answer data of the test paper is completely consistent with the standard answer data, the basis of further scoring is needed instead of the termination of scoring. The subjective question scoring method provided by the application can continuously judge the answer data of the test paper from the dimensionalities except the semantic similarity value, so that the degree of fit between the scoring of the answer data of the test paper and the real situation is effectively improved.
And S3, calculating a basic score coefficient of the subjective question by using a basic score equation corresponding to the stem data and the semantic similarity value.
The question type of the subjective question can be determined through the question stem data of the subjective question, and different question types are required to correspond to different basic scoring equations in order to ensure the scoring accuracy because answer modes, answer key points and the like corresponding to different question types are different. For example: the answer key points of the translation question type are the semantic similarity of test paper answer data and standard answer data, keyword translation, part-of-speech translation and the like, so that a basic scoring equation set is formed by the answer key points and the keyword co-occurrence degree, the part-of-speech similarity and the like on the basis of the calculated semantic similarity value, and the basic scoring coefficient of the translation question type subjective questions is calculated; the answer key points of the simple answer type are the semantic similarity of each score point, the reproduction of the keywords and the like, so that the semantic similarity of each score point and the reproduction of the keywords in each score point are specially distinguished on the basis of the calculated semantic similarity value to form a basic scoring equation set together, and the basic scoring coefficient of the subjective questions of the simple answer type is calculated.
Therefore, the subjective question scoring method provided by the application can calculate the basic scoring coefficient of the subjective questions by adopting different rule logics (basic scoring equation) aiming at different subjective question types, so that the accuracy of evaluating the basic scoring coefficient is improved.
And S4, calculating an additional score coefficient corresponding to the answer data of the test paper and a preset additional score dimension.
Although the basic score coefficient of each subjective question has been calculated through the above-mentioned process, the basic score coefficient only mainly reflects the semantic similarity value between the answer data of the test paper and the standard answer data, and although the basic score coefficient has a certain pertinence to the calculation of the question type, it is still insufficient to truly reflect the true score condition of the answer data of the test paper, because, as will be known by referring to the above-mentioned partial description, the scoring of the subjective questions cannot rely solely on the semantic similarity value between the answer data of the test paper and the standard answer data, but the answer data of the test paper should be further evaluated from more dimensions, such as the length, the smoothness, the wrongly written word ratio, etc.
Illustratively, english translation questions:
translation: i love my mom.
Standard answer data: i love my mother.
Answer data of test paper: i love, my mother.
It can be seen that, in the above example, in answer data of test paper, not only the smoothness of sentences is damaged due to the wrong use of punctuation marks, but also the sentence structure and the semantics of sentences are greatly changed due to wrongly written words. Therefore, even if the basic score coefficient of the answer data of the test paper is high, the additional score coefficient of the score dimension is added, and the final score coefficient is reduced.
Further, the preset additional scoring dimensions may include the smoothness of the answer data of the test paper, the length scale, the wrongly written word duty ratio, the score of the translation evaluation criterion (for the translation question type), and the like.
Specifically, the smoothness of answer data of the test paper can be calculated through a deep learning model, for example, keywords in the answer data of the test paper are found through semantic analysis, the probability of occurrence of a third keyword is calculated by using the deep learning model through the first two keywords according to the occurrence sequence of the keywords in the answer data of the test paper, the probability of occurrence of a fourth keyword is calculated through the probability of occurrence of the first three keywords, the probability of occurrence of all keywords is calculated in a recursive mode sequentially, and finally the smoothness of the answer data of the test paper is calculated according to the probability of occurrence of all keywords.
Specifically, the present invention relates to a method for manufacturing a semiconductor device. The length ratio can be obtained by calculating the ratio of the character length of the answer data of the test paper to the character length of the standard answer data. Generally, the larger the character length of the answer data of the test paper is, the more information the answer data of the test paper contains is indicated, and the answer is relatively more complete. The length ratio mentioned in the application can be the ratio obtained by taking the longer character length party as the denominator and the shorter character length party as the numerator in the answer data of the test paper and the standard answer data, and the ratio is constant in [0,1 ]; the ratio of the character length of the answer data of the test paper to the character length of the standard answer data can be larger than 0, and if the ratio is larger than 1 and larger than a preset ratio threshold value, the character length of the answer data of the test paper is far larger than the character length of the standard answer data, and the relative integrity of the answer data of the test paper is higher.
Specifically, the number of wrongly written words in the answer data of the test paper can be determined by dividing the word group of the answer data of the test paper and comparing the semantics of the answer data of the test paper and the standard answer data, and the ratio of the number to the total vocabulary number of the answer data of the test paper or the total number of the characters of the answer data of the test paper is calculated to obtain the ratio of wrongly written words.
Therefore, the subjective question scoring method provided by the application can effectively obtain scores of other additional scoring dimensions except semantic similarity of the subjective questions, and further enhance the accuracy of final scores of the subjective questions. Further, in order to further improve the accuracy of final scoring of the subjective questions, the types and the number of preset additional scoring dimensions can be increased, so that the subjective questions can be evaluated more comprehensively and are closer to the actual demands.
S5, calculating a final score coefficient corresponding to an ontology vector by using a final score model, wherein the ontology vector is generated by the basic score coefficient and each additional score coefficient.
Generating an ontology vector by the basic score coefficient and each additional score coefficient, and calculating the ontology vector by utilizing a final score model, which can be a deep learning model or a linear regression model, so as to finally obtain a final score coefficient with the value range of [0,1 ].
Before calculating the ontology vector by using the final scoring model, the ontology vector needs to be normalized by the features to obtain a normalized vector conforming to the final scoring model format.
And S6, calculating a final score, wherein the final score is the product of the final score coefficient and the total score of the subjective questions.
In order to match the real scoring criteria of the test paper, the final scoring coefficient is multiplied by the total score of the corresponding subjective questions to obtain the final score. Illustratively, the subjective question total score is 10, and when the final score coefficient is 0.85, the final score is 10×0.85=8.5.
According to the technical scheme, the subjective question scoring method is provided, and semantic similarity values of standard answer data and test paper answer data in the score associated data are calculated by receiving the score associated data of the subjective questions; and simultaneously, calculating a basic score coefficient of the subjective question by using a basic score equation corresponding to the stem data in the score associated data and the calculated semantic similarity value. In order to make the scoring accuracy of the subjective questions higher, the scoring dimension of the subjective questions is increased, namely, an additional scoring coefficient corresponding to the preset additional scoring dimension of the answer data of the calculated test paper is calculated. And deeply calculating a basic score coefficient and an additional score coefficient of the subjective questions by using a final score model to obtain a final score coefficient, and obtaining a final score by calculating the product of the final score coefficient and the total score of the subjective questions. The final score comprises the judgment of subjective questions in multiple dimensions, so that the score is closer to the real demand, the accuracy is higher, the workload of manual review is greatly reduced, and the scoring efficiency is further improved. Therefore, the subjective question scoring method provided by the application can effectively solve the problems of low accuracy and low efficiency when the subjective questions are scored in an automatic paper marking mode of the existing system.
Fig. 2 is a flowchart of a subjective question scoring method step S2 according to an embodiment of the present application.
In one embodiment, as shown in fig. 2, step S2 may include the steps of:
and S201, unifying synonymous structures in the standard answer data and the test paper answer data to respectively obtain the unified standard answer data and the unified test paper answer data, wherein the synonymous structures are different vocabularies corresponding to the same semantic meaning or different expression languages of the same number.
Because the language structure or expression mode of the standard answer data and the test paper answer data are different, the comparison process of the standard answer data and the test paper answer data is influenced, and therefore the synonymous structures in the standard answer data and the test paper answer data need to be unified.
Illustratively:
standard answer data: 1. opening the heart;
answer data of test paper: 1. a pleasure &;
it can be seen that "1" and "one" are different expression languages of the same number, and in this embodiment, the unified "1" is adopted; the "happy" and "happy" are different words corresponding to the same semantic meaning, and in this embodiment, the unification is "happy", so the unification should be:
Unifying standard answer data: 1. is happy;
unifying answer data of test paper: 1. happy and &.
S202, removing special symbols in the unified standard answer data and the unified test paper answer data to respectively obtain the standardized standard answer data and the standardized test paper answer data.
In the above example, the unified test paper answer data includes a special symbol "≡", which needs to be removed from the test paper answer data, so as to obtain:
standardized test paper answer data: 1. is happy.
S203, calculating the semantic similarity value of the standardized standard answer data and the standardized test paper answer data.
Therefore, by the standardized standard answer data and the standardized test paper answer data provided by the embodiment of the application, comparison time can be effectively saved, ambiguity and interference caused by different special symbols and language structures can be avoided, and therefore accuracy and efficiency of calculating semantic similarity are improved.
As can be seen from the above description, the subjective questions comprise multiple types, each type corresponds to a different basic scoring equation, and in order to improve the calculation accuracy of the basic scoring coefficient of the subjective questions, it is necessary to accurately determine which type of question the currently evaluated subjective questions belong to.
Fig. 3 is a flowchart of a method for determining the type of the subjective questions according to an embodiment of the present application.
S301, determining the length of the stem according to the stem data;
s302, if the length of the question stem is smaller than the preset length of the question stem, determining that the question type is noun interpretation.
In general, the stem is a structure with short character length such as a phrase or a phrase for a noun interpretation question, and a structure with long character length such as a long sentence or a short text for a translation and a brief answer. Therefore, the length of the characters of the stem is obtained by analyzing the stem data, the length of the stem can be further determined, and the subjective questions belonging to noun interpretation can be directly determined by the length of the stem.
S303, if the length of the stem is greater than or equal to the preset stem length, extracting the topic type characteristic information from the stem data, wherein the topic type characteristic information comprises topic type labels and/or stem keywords;
s304, if the question stem data comprises a question type label, determining that the question type is translation; and if the stem keywords in the stem data are matched with keywords in a preset simple answer keyword library, determining that the questions are simple answer questions.
If the length of the question stem is greater than or equal to the length of the preset question stem, the subjective question is interpreted or answered briefly. Typically, the translated stem data will have a question label.
Illustratively: translation: i love my mom.
The "translation" is the question type label in the question stem data. Other question type labels, such as transfer, T, etc., and even special symbols, such as @, #, etc., may be used as long as the question type label is capable of distinguishing the question type of the subject question from other question types.
In general, stem data of a brief answer is usually a sentence or a paragraph, and the key points of the inquiry are included in the stem data. Usually, the key points of the inquiry are corresponding to the stem keywords of the simple answer questions, the stem keywords are matched with keywords in a preset simple answer keyword library, and if the matching degree reaches above a preset value, the subjective questions can be determined to be the simple answer questions. If the matching degree is smaller than the preset numerical value, the subjective question is not a simple answer, the subjective question may be a novel subjective question, or keywords in the subjective question are keywords which do not exist in a preset simple answer keyword library, at this time, through manual judgment, if the subjective question is finally judged to be a simple answer, the stem keywords corresponding to the subjective question can be filled into the preset simple answer keyword library.
Illustratively: please answer a special breakfast of what time and place the reddish is done?
Through semantic analysis, the stem keywords are determined to be 'reddish', 'time', 'place', 'special breakfast', the preset simple answer keyword library contains keywords of 'time', 'place' and 'person', at the moment, the 'time', 'place' in the preset simple answer keyword library is matched with the stem keywords, but through semantic analysis, the 'person' can be determined to be matched with the stem keywords of 'reddish', at the moment, the matching rate reaches 75%, and is greater than a preset numerical value of 50%, and the subjective questions can be determined to be simple answer questions.
The subjective question type determining method provided by the invention can accurately determine the question type of the subjective questions, thereby facilitating the follow-up accurate correspondence to the basic scoring equation and improving the accuracy of the basic scoring coefficient of the subjective questions.
Aiming at three different question types of subjective questions, the embodiment provides corresponding basic scoring equations respectively.
Fig. 4 is a flow chart of a method of calculating noun interpretation base score coefficients.
In one embodiment, as shown in fig. 4, if the subjective question type is determined to be a noun interpretation by S302, a base score coefficient of the noun interpretation is determined according to the following steps.
S311, if the question type is noun interpretation, determining the semantic similarity value as a basic score coefficient.
The answering key point of the noun interpretation is the semantic similarity of the test paper answer data and the standard answer data, so that the semantic similarity value of the test paper answer data and the standard answer in the noun interpretation questions can be directly used as a basic scoring coefficient.
Fig. 5 is a flow chart of a method of calculating a translation base score coefficient.
In one embodiment, as shown in fig. 5, if the subjective question type is determined to be a translation by S304, a base score coefficient of the translation is calculated according to the following steps.
S321, if the question type is translation, determining a preset scoring interval corresponding to the semantic similarity value, wherein the preset scoring interval comprises a first interval, a second interval and a third interval which are continuously divided between 0 and 1;
s322, if the semantic similarity value corresponds to the first interval, determining the semantic similarity value as a basic score coefficient, wherein the first interval is an interval in which the semantic similarity value is greater than or equal to a preset upper limit threshold;
s323, if the semantic similarity value corresponds to the second interval, calculating the co-occurrence degree of the keyword of the standard answer data and the keyword of the test paper answer data, and determining the product of the semantic similarity value and the co-occurrence degree as a basic score coefficient, wherein the second interval is an interval of the semantic similarity value between a preset upper limit threshold value and a preset lower limit threshold value;
S324, if the semantic similarity value corresponds to the third interval, calculating the part-of-speech similarity of the standard answer data and the test paper answer data; and determining a basic score coefficient according to the semantic similarity value and the part-of-speech similarity, wherein the third interval is an interval in which the semantic similarity value is smaller than a preset lower limit threshold value.
For example, taking 0.5 as the preset lower threshold and 0.8 as the preset upper threshold, the first interval is [0.8,1], the second interval is [0.5,0.8 ], the third interval is [0, 0.5), the first interval, the second interval and the third interval form a complete and non-repeated [0,1] interval, and it should be noted that the first, second and third intervals in this embodiment are merely descriptive language and are not limited to the intervals. If the semantic similarity value corresponds to the first interval, directly determining the semantic similarity value as a basic score coefficient.
If the semantic similarity corresponds to the second interval, the basic score coefficient needs to be calculated together with the co-occurrence degree and the semantic similarity value of the keyword.
For example, the semantic similarity value is 0.7, and the standard answer data is "my love my mom" and the test paper answer data is "my love my mom" corresponding to the second interval. The keywords of the standard answer data are "me", "love", "mom", and the keywords of the test paper answer data are "me", "love", "mom", and it is seen that the co-occurrence degree of the keywords is 0.67, and the basic score coefficient should be 0.7x0.67=0.469.
If the semantic similarity corresponds to the third interval, the basic score coefficient needs to be calculated together with the semantic similarity value by combining the part-of-speech similarity. Specifically, standard answer data and test paper answer data of the word segmentation are obtained, the part of speech of each segmented word is obtained after the standard answer data and the test paper answer data are segmented, and the segmented words are reassembled into sentences according to the part of speech, and then the similarity is calculated.
For example, the semantic similarity value is 0.3, and the standard answer data is "my love my mom" and the test paper answer data is "my mom, my love" corresponding to the third interval. After word segmentation, standard answer data are I, love and mom, and the parts of speech corresponding to the standard answer data are nouns, verbs and nouns respectively; after word segmentation, answer data of the test paper are "mom", "me" and "love", and the parts of speech corresponding to the answer data are nouns, nouns and verbs respectively. According to the part of speech of each word, after being reassembled into sentences, the standard answer data is "my love mom", the test paper answer data is "mommy love me", and the part of speech similarity at the moment is 0.33. Setting a part-of-speech similarity threshold value to be 0.85, when the part-of-speech similarity is smaller than 0.85, adding 1 to the part-of-speech similarity, and multiplying the value obtained by the semantic similarity value to serve as a basic score coefficient, wherein the basic score of the embodiment is (0.33+1) multiplied by 0.3=0.129; when the part-of-speech similarity is greater than or equal to 0.85, calculating a difference between 1 and the part-of-speech similarity, and multiplying a value obtained by the semantic similarity value to obtain a basic score coefficient.
The process is equivalent to the calculation of the basic scoring equation corresponding to the basic scoring coefficient of the translation question type, and the result shows that the different conditions of the semantic similarity value between the answer data of the test paper in the translation question type and the standard answer data respectively correspond to the respective basic scoring formulas, so that the calculation of the basic scoring coefficient is closer to the real data and is more accurate.
Fig. 6 is a flow chart of a method of calculating a base score for a simple answer.
In one embodiment, as shown in fig. 6, if the subjective question type is determined to be a simple answer by S304, a basic score coefficient of the simple answer is calculated according to the following steps.
S331, if the question type is a simple answer, determining the query type of the subjective question according to the question stem keyword, wherein the query type comprises an entity type and a non-entity type.
Specifically, the entity type generally includes a time type, a person type, a place type, or an organization type, etc.
Illustratively: what time the reddish was a special breakfast?
As can be seen from semantic analysis of the stem data, the stem keywords are "reddish", "time", "do", "special breakfast", and obviously "time" is the key point of the subjective questions, so the type of the subjective questions is time type.
Non-entity types typically do not appear as distinct objective keywords, but rather are interrogated by a sentence or a paragraph.
Illustratively: what is the special breakfast made with reddish?
According to semantic analysis of the stem data, the stem keywords are "reddish", "special breakfast", "generating", "influencing", and it can be seen that the query is focused by analyzing the whole semantic, and the query is focused on the non-entity focus of "influencing", so that the query type of the subjective questions should be the non-entity type.
And S332, if the answer is of an entity type, calculating the Jacquard similarity of the related phrase of the standard answer data and the related phrase in the test paper answer data, wherein the related phrase is a word set related to the stem keyword in the standard answer data and the test paper answer data, and determining the Jacquard similarity as a basic scoring coefficient.
Specifically, firstly, determining the phrase related to the question stem keyword in the standard answer data and the test paper answer data, then determining the intersection number and the union number of the phrase, and finally calculating the ratio of the intersection number to the union number, wherein the ratio is Jaccard (Jaccard) similarity.
Illustratively: what time does the reddish a special breakfast? How long is it time consuming? How long is it eaten?
Standard answer data: 4.11.2019, 6:00 am; 1 hour; 30 minutes.
Answer data of test paper: 6 a.m.; 1h; half an hour.
Intersection is "6:00 am", "1 hour" and "30 minutes", number 3, and intersection is "2019, 4 months, 11 days", "6:00 am", "1 hour", "30 minutes", number 4, then the jaccard similarity is 3/4=0.75.
At this time, since the query type of the above example belongs to the entity type, the 0.75-based score coefficient can be directly determined.
S333, if the answer is of a non-entity type, determining score point data in the standard answer data according to the semantics of the standard answer data, calculating the semantic similarity of the answer data of the test paper and the score point data, and determining the sum of the semantic similarity and the product of the preset score weight as a basic score coefficient.
Specifically, referring to FIG. 7, a flow chart of a method of computing semantic similarity of point data is provided.
In this embodiment, the following steps are adopted to calculate the semantic similarity between the answer data and the score data of the test paper.
S3331, dividing the answer point data in the answer data of the test paper according to the punctuation marks in the answer data of the test paper;
s3332, matching the association degree of the semantics of the reply point data and the semantics of the first sentence data in the score point data, and determining the score point data corresponding to the reply point data, wherein the first sentence data is the data of the first sentence in the score point data;
s3333, calculating sub-semantic similarity of a reply point group and corresponding score point data, wherein the reply point group is all reply point data corresponding to the same score point data;
s3334, calculating the semantic similarity of the answer data of the test paper and the score data, wherein the semantic similarity is the product of the sub-semantic similarity.
And S333, determining score point data in the standard answer data through semantic analysis, positioning punctuation marks in the answer data of the test paper, and dividing answer point data in the answer data of the test paper according to the punctuation marks.
Illustratively: what do the reddish? What does it?
Standard answer data: the red food is in canteen and eat special breakfast. At 6:00 am.
Answer data of test paper: the reddish is eating breakfast, 6 am in the restaurant.
Punctuation marks disclosed in this embodiment include commas, stop signs, periods, semicolons, and the like. Thus, the answer data of the test paper can be divided into two answer point data of "breakfast being eaten by the reddish in the canteen" and "6 o' clock in the morning" according to punctuation marks.
Through semantic analysis, the score point data in the standard answer data can be determined as 'little red in canteen, eat special breakfast' and '6:00 am'. It can be seen that the first score data consists of two clauses, where "xiaohong at canteen" shall be the first sentence data. It should be noted that if the score point data has only one clause, the clause is the first sentence.
The first sentence data of each reply point data and each score point data are matched in sequence, namely, the small red is matched with the small red in breakfast, the small red in canteen and the small red in the morning of 6:00, and the small red in the morning of 6:00. By comparing the degree of association of the semantics of the reply point data with the semantics of the score point data, it is possible to divide each reply point data to the corresponding reply point group. Specifically, the association degree is obtained by semantically analyzing the answer point data and the score point data, and then obtaining keywords such as environment, character, event, action, time and the like contained in the sentence. For example: the keywords such as "canteen", "restaurant", "breakfast", "eat", "chef", "tableware" are highly related, while the keywords such as "canteen", "6-point", "drive", "book", "doctor", "screw" are low in related.
Obviously, the reply point data of "small red is eating breakfast" and "in restaurant" has a higher degree of association with the first sentence data of "small red in canteen" of the score point data, and therefore, the "small red is eating breakfast" and "in restaurant" are the same score point data "small red in canteen". Special breakfast is eaten. "reply dot group; the relevance of the reply point data of "6 am" and the score point data of "6:00 am" is high, so the reply point of "6 am" is the score point data of "6:00 am". The number of the reply point data contained in the reply point group may be 1 or more than 1.
As can be obtained by semantic similarity calculation, in this example, the sub-semantic similarity of the reply point group "small red is eating breakfast" and "in restaurant" and the score point data "small red is in canteen" and eating special breakfast "is 0.5; the sub-semantic similarity of the reply point "6 a.m." and the score point data "6:00 a.m." is 1. The semantic similarity of the answer data of the test paper and the score data is 0.5×1=0.5.
In general, the questions may have a point of interest, and the point of interest may be reflected to the score, that is, the ratio of the points of interest in the standard answer data may be different, or the weights may be different.
Illustratively: the weight of "xiaohong at canteen, eat special breakfast" is 0.4, and the weight of "at 6:00 am" is 0.6.
At this time, the semantic similarity between the answer data of the test paper and the score data is (0.5×0.4) + (1×0.6) =0.62.
It should be noted that if the question person does not have the review emphasis, the weight does not need to be set, or the weights of the score points are set to be equal.
Further, as shown in fig. 8, if score point data cannot be accurately determined in standard answer data by semantic analysis, a basic score coefficient of a simple answer is calculated with reference to the following steps.
S3335, if the score point data in the standard answer data cannot be determined, calculating the overall semantic similarity value of the test paper answer data and the standard answer data;
s3336, calculating the length ratio of the standard answer data to the test paper answer data, wherein the length ratio is the ratio calculated by the numerator by taking the longer character length in the standard answer data and the test paper answer data as the denominator and the shorter character length;
s3337, determining a product of the overall semantic similarity value and the length proportion as a basic score coefficient.
Illustratively: standard answer data: the reddish has made special breakfast in the kitchen of the own home.
Answer data of test paper: the reddish has made special breakfast in the dormitory kitchen.
As can be seen, the overall semantic similarity between the answer data of the test paper and the standard answer data is 0.67, wherein the character length of the standard answer data is 16, and the character length of the answer data of the test paper is 15, so the length ratio should be the character length of the answer data of the test paper/the character length of the standard answer data=15/16=0.94. Finally, according to the overall semantic similarity and the length ratio, a basic score coefficient of 0.67×0.94=0.63 is calculated.
Fig. 9 is a flowchart of a method for training a final scoring model according to an embodiment of the present application.
In one embodiment, as shown in fig. 9, before calculating the final score corresponding to the ontology vector, it includes:
s501, training the final scoring model by taking the ontology vector as data of a training model and standard answer data as labels of the training model on the basis of preset training data.
Specifically, the preset training data may be related historical online test data, where the historical online test data at least includes corresponding stem data, standard answer data, test paper answer data, and final score coefficient data. The larger the preset training data is, the more preset additional scoring dimensions are contained in the ontology vector, and the higher the scoring accuracy of the final scoring model is.
After calculating the final score coefficient of the current subjective questions by using the final score model, the final score model can be trained again by using the score associated data of the current subjective questions and the final score coefficient, so that the score accuracy of the final score model is further improved, and the new final score model obtained by training is applied to the scoring process of the next subjective questions.
Therefore, by the subjective question scoring method provided by the embodiment, the scoring associated data of the current subjective questions can be combined with the final scoring model, so that the accuracy of scoring the subjective questions and the fitting degree with real data are improved; and the final scoring model can be trained by further utilizing the scoring association data and the final score of the subjective questions of each judgment, so that the scoring accuracy of the final scoring model is improved.
Fig. 10 is a schematic diagram of a subjective question scoring apparatus according to an embodiment of the present application. The device can be applied to various equipment capable of scoring, such as a server, a PC (personal computer), a tablet personal computer, a mobile phone and the like.
As shown in fig. 10, the apparatus may include:
the data receiving module 1 is configured to receive score association data of subjective questions, where the score association data includes: question stem data, standard answer data and test paper answer data;
The similarity calculation module 2 is used for calculating the semantic similarity value of the standard answer data and the test paper answer data;
the basic score coefficient calculation module 3 is used for calculating the basic score coefficient of the subjective question by utilizing a basic score equation corresponding to the stem data and the semantic similarity value;
the additional score coefficient calculation module 4 is used for calculating an additional score coefficient corresponding to the answer data of the test paper and a preset additional score dimension;
a final score coefficient calculation module 5, configured to calculate a final score coefficient corresponding to an ontology vector, where the ontology vector is generated by the base score coefficient and each additional score coefficient;
a final score calculating module 6, configured to calculate a final score, where the final score is a product of the final score coefficient and the total score of the subjective questions.
According to the subjective question scoring method and device, the semantic similarity value of standard answer data and test paper answer data in the score associated data is calculated by receiving the score associated data of the subjective questions; and simultaneously, calculating a basic score coefficient of the subjective question by using a basic score equation corresponding to the stem data in the score associated data and the calculated semantic similarity value. In order to make the scoring accuracy of the subjective questions higher, the scoring dimension of the subjective questions is increased, namely, an additional scoring coefficient corresponding to the preset additional scoring dimension of the answer data of the calculated test paper is calculated. And deeply calculating a basic score coefficient and an additional score coefficient of the subjective questions by using a final score model to obtain a final score coefficient, and obtaining a final score by calculating the product of the final score coefficient and the total score of the subjective questions. The final score comprises the judgment of subjective questions in multiple dimensions, so that the score is closer to the real demand, the accuracy is higher, the workload of manual review is greatly reduced, and the scoring efficiency is further improved. Therefore, the subjective question scoring method provided by the application can effectively solve the problems of low accuracy and low efficiency when the subjective questions are scored in an automatic paper marking mode of the existing system.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It is to be understood that the present application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Claims (8)
1. A subjective question scoring method, comprising:
receiving score-related data of subjective questions, the score-related data comprising: question stem data, standard answer data and test paper answer data;
calculating the semantic similarity value of the standard answer data and the test paper answer data;
determining the length of the stem according to the stem data;
if the length of the question stem is smaller than the length of the preset question stem, determining the question type as noun interpretation;
If the length of the question stem is greater than or equal to the preset question stem length, extracting question type characteristic information from the question stem data, wherein the question type characteristic information comprises question type labels and/or question stem keywords;
if the question stem data comprises a question type label, determining the question type as translation; if the stem keywords in the stem data are matched with keywords in a preset simple answer keyword library, determining that the questions are simple answer questions;
calculating a basic score coefficient of the subjective question by using a basic score equation corresponding to the stem data and the semantic similarity value;
calculating an additional score coefficient corresponding to the answer data of the test paper and a preset additional score dimension;
calculating a final score coefficient corresponding to an ontology vector by using a final score model, wherein the ontology vector is generated by the basic score coefficient and each additional score coefficient;
calculating a final score, wherein the final score is the product of the final score coefficient and the total score of the subjective questions;
the step of calculating the basic score coefficient of the subjective question by using the basic score equation corresponding to the stem data and the semantic similarity value comprises the following steps:
If the question type is translation, determining a preset scoring interval corresponding to the semantic similarity value, wherein the preset scoring interval comprises a first interval, a second interval and a third interval which are continuously divided between 0 and 1;
if the semantic similarity value corresponds to the first interval, determining the semantic similarity value as a basic score coefficient, wherein the first interval is an interval in which the semantic similarity value is greater than or equal to a preset upper limit threshold;
if the semantic similarity value corresponds to the second interval, calculating the co-occurrence degree of the keyword of the standard answer data and the keyword of the test paper answer data, and determining the product of the semantic similarity value and the co-occurrence degree as a basic score coefficient, wherein the second interval is an interval between a preset upper limit threshold value and a preset lower limit threshold value of the semantic similarity value;
if the semantic similarity value corresponds to the third interval, calculating the part-of-speech similarity of the standard answer data and the test paper answer data; and determining a basic score coefficient according to the semantic similarity value and the part-of-speech similarity, wherein the third interval is an interval in which the semantic similarity value is smaller than a preset lower limit threshold value.
2. The method of claim 1, wherein calculating the semantic similarity value for the standard answer data and the test paper answer data comprises:
unifying synonymous structures in the standard answer data and the test paper answer data to respectively obtain unified standard answer data and unified test paper answer data, wherein the synonymous structures are different vocabularies corresponding to the same semantic or different expression languages of the same number;
removing special symbols in the unified standard answer data and the unified test paper answer data to respectively obtain the standardized standard answer data and the standardized test paper answer data;
and calculating the semantic similarity value of the standardized standard answer data and the standardized test paper answer data.
3. The method of claim 1, wherein calculating the base score coefficient of the subjective question using the base score equation and the semantic similarity value corresponding to the stem data further comprises:
and if the question type is noun interpretation, determining the semantic similarity value as a basic score coefficient.
4. The method of claim 1, wherein calculating the base score coefficient of the subjective question using the base score equation and the semantic similarity value corresponding to the stem data further comprises:
If the question type is a simple answer, determining the query type of the subjective question according to the question stem keyword, wherein the query type comprises an entity type and a non-entity type;
if the answer is of an entity type, calculating the Jacquard similarity of the related phrase of the standard answer data and the related phrase in the test paper answer data, wherein the related phrase is a word set related to the stem keywords in the standard answer data and the test paper answer data, and determining the Jacquard similarity as a basic scoring coefficient;
if the answer is of a non-entity type, determining score point data in the standard answer data according to the semantics of the standard answer data, calculating the semantic similarity of the answer data of the test paper and the score point data, and determining the sum of the semantic similarity and the product of the preset score weight as a basic score coefficient.
5. The method of claim 4, wherein calculating semantic similarity of answer data to test paper and score data comprises:
dividing the answer point data in the answer data of the test paper according to the punctuation marks in the answer data of the test paper;
matching the association degree of the semantics of the reply point data and the semantics of first sentence data in each score point data, and determining the score point data corresponding to the reply point data, wherein the first sentence data is the data of the first sentence in the score point data;
Calculating sub-semantic similarity of a reply point group and corresponding score point data, wherein the reply point group is all reply point data corresponding to the same score point data;
and calculating the semantic similarity of the answer data of the test paper and the score data, wherein the semantic similarity is the product of all the sub-semantic similarities.
6. The method of claim 4, wherein determining the base score if the base score is of a non-entity type further comprises:
if the score point data in the standard answer data cannot be determined, calculating the overall semantic similarity value of the test paper answer data and the standard answer data;
calculating the length ratio of the standard answer data to the test paper answer data, wherein the length ratio is the ratio calculated by taking the longer character length in the standard answer data and the test paper answer data as the denominator and the shorter character length as the numerator;
and determining the product of the overall semantic similarity value and the length proportion as a basic score coefficient.
7. The method of claim 1, wherein before calculating the final score corresponding to the ontology vector using the final scoring model, the method further comprises:
and training the final scoring model by taking the ontology vector as data of a training model and standard answer data as a label of the training model on the basis of preset training data.
8. A subjective question scoring device, comprising:
the data receiving module is used for receiving the grading association data of the subjective questions, and the grading association data comprises: question stem data, standard answer data and test paper answer data;
the similarity calculation module is used for calculating the semantic similarity value of the standard answer data and the test paper answer data;
the basic score coefficient calculation module is used for calculating the basic score coefficient of the subjective question by utilizing a basic score equation corresponding to the stem data and the semantic similarity value;
the calculating the basic score coefficient of the subjective question by using the basic score equation corresponding to the stem data and the semantic similarity value comprises the following steps:
if the question type is translation, determining a preset scoring interval corresponding to the semantic similarity value, wherein the preset scoring interval comprises a first interval, a second interval and a third interval which are continuously divided between 0 and 1;
if the semantic similarity value corresponds to the first interval, determining the semantic similarity value as a basic score coefficient, wherein the first interval is an interval in which the semantic similarity value is greater than or equal to a preset upper limit threshold;
if the semantic similarity value corresponds to the second interval, calculating the co-occurrence degree of the keyword of the standard answer data and the keyword of the test paper answer data, and determining the product of the semantic similarity value and the co-occurrence degree as a basic score coefficient, wherein the second interval is an interval between a preset upper limit threshold value and a preset lower limit threshold value of the semantic similarity value;
If the semantic similarity value corresponds to the third interval, calculating the part-of-speech similarity of the standard answer data and the test paper answer data; determining a basic score coefficient according to the semantic similarity value and the part-of-speech similarity, wherein the third interval is an interval in which the semantic similarity value is smaller than a preset lower limit threshold value;
the additional score coefficient calculation module is used for calculating an additional score coefficient corresponding to the answer data of the test paper and a preset additional score dimension;
the final score coefficient calculation module is used for calculating a final score coefficient corresponding to an ontology vector by utilizing a final score model, wherein the ontology vector is generated by the basic score coefficient and each additional score coefficient;
and the final score calculation module is used for calculating a final score, wherein the final score is the product of the final score coefficient and the total score of the subjective questions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910323564.1A CN110096702B (en) | 2019-04-22 | 2019-04-22 | Subjective question scoring method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910323564.1A CN110096702B (en) | 2019-04-22 | 2019-04-22 | Subjective question scoring method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110096702A CN110096702A (en) | 2019-08-06 |
| CN110096702B true CN110096702B (en) | 2023-07-25 |
Family
ID=67445405
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910323564.1A Active CN110096702B (en) | 2019-04-22 | 2019-04-22 | Subjective question scoring method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110096702B (en) |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110471936B (en) * | 2019-08-19 | 2022-06-07 | 福建工程学院 | Hybrid SQL automatic scoring method |
| CN110609953A (en) * | 2019-08-28 | 2019-12-24 | 苏州承儒信息科技有限公司 | Reading and amending method and system for internet education |
| CN111324692B (en) * | 2020-01-16 | 2020-12-22 | 深圳市芥菜种科技有限公司 | Automatic subjective question scoring method and device based on artificial intelligence |
| CN111414765B (en) * | 2020-03-20 | 2023-07-25 | 北京百度网讯科技有限公司 | Sentence consistency determination method and device, electronic equipment and readable storage medium |
| CN111768126B (en) * | 2020-07-20 | 2023-06-09 | 陈亮 | Evaluation scoring method for non-standard evaluation questions |
| CN112085985B (en) * | 2020-08-20 | 2022-05-10 | 安徽七天网络科技有限公司 | Student answer automatic scoring method for English examination translation questions |
| CN114333461B (en) * | 2020-09-30 | 2024-03-12 | 董黎明 | Automatic subjective question scoring method and system |
| CN112184032A (en) * | 2020-09-30 | 2021-01-05 | 广州思酷信息科技有限公司 | Method and system for intelligently scoring subjective questions |
| CN112232681B (en) * | 2020-10-19 | 2023-01-31 | 山东山大鸥玛软件股份有限公司 | Intelligent examination paper marking method for computational analysis type non-choice questions |
| CN113343972A (en) * | 2021-06-29 | 2021-09-03 | 北京市商汤科技开发有限公司 | Paper marking method and device, electronic equipment and storage medium |
| CN113822040B (en) * | 2021-08-06 | 2024-07-02 | 深圳市卓帆技术有限公司 | Subjective question scoring method, subjective question scoring device, computer equipment and storage medium |
| CN113987296B (en) * | 2021-11-22 | 2024-06-11 | 腾讯科技（深圳）有限公司 | Solution detection method and device for application questions |
| CN116957368A (en) * | 2022-03-31 | 2023-10-27 | 华为技术有限公司 | Scoring method and related device |
| CN114743421B (en) * | 2022-04-27 | 2023-05-16 | 广东亚外国际文化产业有限公司 | Comprehensive assessment system and method for foreign language learning intelligent teaching |
| CN114969260A (en) * | 2022-05-30 | 2022-08-30 | 桂林旅游学院 | Automatic subjective question evaluation and reading deep learning method combining test question classification and evaluation learning |
| CN115080690A (en) * | 2022-06-17 | 2022-09-20 | 瀚云瑞科技(北京)有限公司 | NLP-based automatic correction method and system for test paper text |
| CN115630613B (en) * | 2022-12-19 | 2023-04-07 | 长沙冉星信息科技有限公司 | Automatic coding system and method for evaluation problems in questionnaire survey |
| CN117808367B (en) * | 2024-01-02 | 2024-07-19 | 汉考国际教育科技(北京)有限公司 | Intelligent examination paper evaluation system for Chinese examination |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103955874A (en) * | 2014-03-31 | 2014-07-30 | 西南林业大学 | Automatic subjective-question scoring system and method based on semantic similarity interval |
| WO2018161917A1 (en) * | 2017-03-07 | 2018-09-13 | 百度在线网络技术（北京）有限公司 | Intelligent scoring method and apparatus, computer device, and computer-readable medium |
| CN108877839A (en) * | 2018-08-02 | 2018-11-23 | 南京华苏科技有限公司 | The method and system of perceptual evaluation of speech quality based on voice semantics recognition technology |
-
2019
- 2019-04-22 CN CN201910323564.1A patent/CN110096702B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103955874A (en) * | 2014-03-31 | 2014-07-30 | 西南林业大学 | Automatic subjective-question scoring system and method based on semantic similarity interval |
| WO2018161917A1 (en) * | 2017-03-07 | 2018-09-13 | 百度在线网络技术（北京）有限公司 | Intelligent scoring method and apparatus, computer device, and computer-readable medium |
| CN108877839A (en) * | 2018-08-02 | 2018-11-23 | 南京华苏科技有限公司 | The method and system of perceptual evaluation of speech quality based on voice semantics recognition technology |
Non-Patent Citations (1)
| Title |
|---|
| 一种基于短文本相似度计算的主观题自动阅卷方法;张均胜等;《图书情报工作》;20141005(第19期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110096702A (en) | 2019-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110096702B (en) | Subjective question scoring method and device | |
| Lu et al. | Matching phrase-frames to rhetorical moves in social science research article introductions | |
| Heilman | Automatic factual question generation from text | |
| US9959776B1 (en) | System and method for automated scoring of texual responses to picture-based items | |
| US9020805B2 (en) | Context-based disambiguation of acronyms and abbreviations | |
| Sung et al. | Factors influencing the learning of Chinese characters | |
| CN106796594B (en) | Generating high-level questions from sentences | |
| CN111310463B (en) | Test question difficulty estimation method and device, electronic equipment and storage medium | |
| CN111833853A (en) | Voice processing method and device, electronic equipment and computer readable storage medium | |
| WO2020199600A1 (en) | Sentiment polarity analysis method and related device | |
| Kunilovskaya et al. | Learner vs. professional translations into Russian: Lexical profiles | |
| Xu et al. | Assessing L2 English speaking using automated scoring technology: examining automarker reliability | |
| CN114462389A (en) | Automatic test paper subjective question scoring method | |
| Goulart | Register variation in L1 and L2 student writing: A multidimensional analysis | |
| Wijeratne et al. | Natural language processing for government: Problems and potential | |
| Maicher et al. | Artificial intelligence in virtual standardized patients: combining natural language understanding and rule based dialogue management to improve conversational fidelity | |
| Kapelner et al. | Predicting contextual informativeness for vocabulary learning | |
| Sukkarieh et al. | Auto-marking 2: An update on the UCLES-Oxford University research into using computational linguistics to score short, free text responses | |
| Lee et al. | Traditional readability formulas compared for english | |
| Alobed et al. | A comparative analysis of Euclidean, Jaccard and Cosine similarity measure and arabic wordnet for automated arabic essay scoring | |
| McDonald et al. | Text analytic tools to illuminate student learning | |
| Cushing | Corpus linguistics and language testing | |
| Schneider et al. | Statistics for Linguists: A patient, slow-paced introduction to statistics and to the programming language R | |
| Khamkhien | Textual organisation and linguistic features in applied linguistics research articles: Moving from Introduction to Methods | |
| LaFlair et al. | Connecting corpus linguistics and assessment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |