FR3030809A1 - METHOD FOR AUTOMATICALLY ANALYZING THE LITERARY QUALITY OF A TEXT - Google Patents

Abstract

The invention relates to a method for analyzing the literary quality of a text, implemented by a computer program.

Description

FIELD OF THE INVENTION The present invention relates to a method for automatically analyzing the literary quality of a text. By "literary quality of a text" is meant, in the context of the invention, the literary quality of a text that is intrinsic to it. This intrinsic quality can be materialized by a discrete score of 1 to 10, a continuous score in [0, 1], as a real score [-00, +00] or as "very good", "good", "Medium", etc.

By "classification" is meant within the scope of the invention, the usual meaning of the technique, namely the prediction of the class (discrete value), of a set of data from a pre-training basis. -étiquetée. By "regression" is meant within the scope of the invention, the usual meaning of the technique, namely the prediction of numerical values and continuous associated with a set of data from a learning base. STATE OF THE ART In general, the purpose of automatic text categorization is to teach a computer machine to classify a text in the correct category based on its content.

Categorization algorithms can solve various problems of categorization of texts. Regarding the analysis of the quality of a literary or scientific text, different approaches have already been made and different categorization algorithms implemented.

There are thus several works that concern the quality of a literary text but most are not relevant because they define the notion of quality in a sense that is their own and therefore not really independent of the factors that are chosen. One can quote the patent US7200606 in which the notion of quality is considered in the sense of relevance vis-à-vis a user request.

Thus, one of the relevant approaches is the so-called intrinsic approach according to which it is a question of using categorization algorithms to classify documents according to textual characteristics (indicators) which are intrinsic to the text: composition, elements of style, precision vocabulary in relation to a subject, construction of reasoning, spelling, etc. The characteristics of sorting depend on very variable orthographic, lexical and stylistic approaches, among which the length of the words, regularity of the vocabulary, analysis of the cooccurrences, use of the punctuation, detection of grammatical and orthographical errors, facility of reading, lexical links with a theme or genre, etc. These characteristics related to the text can be usefully supplemented by semantic methods made around the relationship between quality and respect for orthographic and typographic rules, grammar (measurement of quality on long n-grams), capitalization, density of the text. text (relationship between letters and spaces) or its entropy (at the level of words, or even at the level of characters). Lexicometry, a method of quantitative analysis of texts, can prove to be a useful tool for measuring quality or non-quality, paradoxically. Whatever methods and categorization algorithms are used, the main difficulty lies in the choice of indicators and the algorithm, and in their combination to evaluate the literary quality of a text. There is little literature that focuses on the literary quality of a text by intrinsic approach. We can first of all cite the publications [1] and [2] which describe an extraction of intrinsic indicators from a raw literary text and then a regression or a classification to reach the target value that one seeks to determine. . The choice of indicators remains relatively brief, which makes it difficult to refine the quality analysis with a very good precision. The publication [3] discloses a prediction of quality from a small number of newspaper articles ("Wall Street Journal"). The analysis according to this publication remains basic, since only a correlation is established between each indicator and a target value carried out on about thirty reference articles. There is therefore a need to improve the analysis of the literary quality of a text, especially with a view to ensuring better accuracy.

The object of the invention is to respond at least in part to this need.

DISCLOSURE OF THE INVENTION To this end, the subject of the invention is a method for analyzing the literary quality of a text, implemented by a computer program, comprising the following steps: a / receiving a plain text to analyze; b / generate several vector sub-representations of the received text to obtain indicators, called low-level indicators, the sub-representations consisting of: - a representation by word bag according to which the distributions of each word are analyzed and the distributions are analyzed certain unigrams, bi-grams, 3-gram, 4-gram, 5-gram and 6-gram on the scale of the word and characters, - a so-called morphosyntactic structure representation, according to which one computes the parameters of the distributions of the grammatical words in the text and analyzes the distributions of each syntactic function in the text, the paragraphs, the sentences and the propositions, - a representation of the writing errors according to which one calculates the number of times each rule of each category of writing mistakes is not respected, - a representation of stylometry according to which one calculates the length of the text, the length of the paragraphs, the length of s sentences, the length of the propositions, the length of the words in characters, the number of each punctuation mark, and finally the parameters of the distribution of dialogs in the text; c / generate: - a meta-description according to which the vocabulary of the text is analyzed by the different levels of scarcity of words, the lexical fields used, the words adapted to youth, and we compute aggregations and ratios of low-level indicators obtained in b /; a representation of the lexical fields present in the text from the word bag representation performed in b /, principal component analysis (PCA) and / or a latent semantic analysis (LSA). , acronym for "Latent Semantic Analysis") and / or a factorization in non-negative matrixes (NMF, acronym for "Non-negative Matrix Factorization"). d / concatenation of vector under-representation generated in b / and c / in a final representation of the text; e / subjecting the final vector of the text to a classifier whose classification algorithm (s) is (are) driven from a set of reference texts whose literary quality has been measured numerically by a population of individuals preferably a population of experts, then weighted. With regard to the digital measurement of the reference texts, which constitutes the learning phase of the method according to the invention, the following procedure can be carried out.

Each individual of a particular population, preferably an expert member of an editorial board, gives series of notes to a given literary text, which are indicated in a database. The series of notes are given in several categories of text, among which are the short stories, the poems, the comic strips.

The series of notes for each individual and in each of the categories of text are then weighted by being centered then reduced according to the equation: x '= (x - m) / s where: x is the score given between 1 and 10 by a individual M for a work of a given category C, m is the average of the marks given by M in the category C, s is the standard deviation of the marks given by M in the category C, x 'is the new note corrected . Thus, in the learning phase, the vector representation of all the texts of the database is carried out by implementing steps a / d and the classifier is trained to reproduce the target value that is to be obtained. , that is to say the literary quality. After a thorough state-of-the-art research, the inventor analyzed that there was no real work on the literary analysis of a text.

Starting from the observation that the known methods of analysis of literary quality were not sufficiently precise, the inventor was also able to analyze that this was mainly due to the simple indicators that were extracted from each text.

The inventor then thought to combine different initial representations to obtain more complex indicators, then to concatenate these different initial representations in order to have a final representation more complete than those according to the state of the art.

In other words, by the choice of complex indicators, the invention makes it possible to be more exhaustive in the measurement related to the intrinsic literary quality of a text. The literary quality analysis obtained by virtue of the invention is all the more precise in that the inventor has been able to construct a reliable and important database with a large number of texts evaluated by individuals and series of scores attributed. The invention has the major advantage of being able to treat all literary texts with the same requirements in terms of quality of texts and to do the analysis with the same delays whatever the text. Preferably, the analysis of the distributions of each word performed in b / is made after the NLT (Automatic Natural Language Processing) treatments. According to an advantageous embodiment, the calculation of the parameters of the distribution of the grammatical words and the dialogues in the text is made from criteria chosen from the mean, the variance, the standard deviation, the entropy, the distance between the distributions or a combination thereof.

The method advantageously comprises the implementation of one or more dimensionality reduction algorithms on the lexical fields obtained in c /. According to a first variant embodiment, the classification algorithm (s) is (are) adapted to enable a binary classification of the text according to the quality.

According to a second variant embodiment, the classification algorithm (s) is (are) adapted to enable a multi-class classification of the text according to the quality. The invention also relates to a computer program for implementing the method described above.

DETAILED DESCRIPTION Other advantages and features of the invention will become more apparent upon reading the detailed description of exemplary embodiments of the invention, given by way of non-limiting illustration with reference to the single appended figure, which is a flowchart of steps of the method according to the invention implemented by computer program. Subsequently, the terms "algorithm" and "computer program" which is the computer-readable coding of the algorithm are used interchangeably.

Thus, an algorithm is an execution plan for a computer. The computer takes incoming data, applies the processing described by the algorithm and returns a result to the user. In the context of the invention, the algorithm implemented for the predictive analysis is a machine learning algorithm. In this type of algorithm, his decision rules are not fixed to the design, because it is designed so that he can modify his decision rules, according to the data he sees. The method according to the invention comprises two successive phases, the first being a learning phase and the next being the prediction phase.

Thus, we first realize the learning phase. A population of experts grouped into an editorial committee gives a series of notes for each literary text of a set of which will serve as a set of reference text. These notes measure the literary quality of each text and are weighted by being centered and then reduced according to the equation: x '= (x - m) / s where: x is the score given between 1 and 10 by an individual M for a given of a given category C, m is the average of the notes given by M in the category C, s is the standard deviation of the notes given by M in the category C, x 'is the new corrected note. We then build the vector representation of each previously evaluated literary text. After having trained several algorithms on the data, that is to say after having made the learning of the parameters by several algorithms, one chooses the one which is the most efficient.

The hyper-parameters of the algorithm are then estimated by applying a cross-validation method. Once the hyper-parameters have been defined, the algorithm is again dragged onto the set of vector representations of all the previously evaluated texts.

For the actual prediction phase of a new literary text, its vector representation is first constructed. Then, the vector thus constructed is submitted to the trained algorithm which will predict a value for the literary quality of the new text as well as an estimation of the error. In other words, we submit the vector of the text to a classifier.

The construction of the vector representation of each text is now described in more detail. The algorithm performs the following steps, from a raw text to be analyzed (step SO). It generates several vector sub-representations (1) of the received text to obtain low-level indicators. The first under-representation consists of a word bag representation (step Si) in which the distributions of each word are analyzed and the distributions of certain unigrams, bi-grams, 3-grams, 4-grams, 5-grams are analyzed. and 6-grams at the scale of the word and characters. Thus, in this step Si, the text is transformed into a sequence of tokens according to regular expressions of division. Word-bag representation does not take into account text formatting, word order, meaning, or relationships structured by linking words. The second underrepresentation represents the morphosyntactic structure (step S2), in which the parameters of the distributions of the grammatical words in the text are calculated and the distributions of each syntactic function are analyzed in the text, paragraphs, sentences and propositions. Grammatical words are articles, prepositions, non-qualifying adjectives. The calculation of the parameters of the distribution of the grammatical words is made from criteria chosen from the mean, the variance, the standard deviation, the entropy, the distance between the distributions or a combination of these. A syntactic function is a verb, a noun, an adjective, an adverb, a determinant, a preposition. Thus, this step S2 makes it possible to extract structural elements of the text in so far as to go up to the pragmatic level of the general comprehension of the text. The third underrepresentation represents writing errors (step S3) according to which the number of times each rule of each of the categories of writing faults is not respected is calculated. The errors of writing are the errors of spelling, grammar, conjugation, anglicism, syntax, expression, and use. Thus, this step S3 consists of automatically analyzing the different types of faults appearing in the text. The fourth underrepresentation represents the stylometry (step S4) according to which the length of the text, the length of the paragraphs, the length of the sentences, the length of the propositions, the length of the words in characters, the number of each punctuation mark are calculated. , and finally the parameters of the dialog distribution in the text. The length of the text is calculated from the number of paragraphs, sentences, propositions, words, characters. The length of a paragraph is calculated from the number of sentences, propositions, words, characters. The length of sentences is calculated from the number of propositions, words, characters. The length of the proposals is calculated from the number of words, characters. The calculation of the dialogue distribution parameters in the text is made from criteria chosen from the mean, the variance, the standard deviation, the entropy, the distance between the distributions or a combination of these. Thus, this step S4 makes it possible to identify the style of the text. From all the preceding sub-representations, the algorithm generates a fifth subrepresentation which is a meta-description (step S5) according to which the vocabulary of the text is analyzed by the different levels of rarity of the words, the lexical fields used. , the words adapted to youth, and one calculates aggregations (sums) and ratios (divisions) of the low-level indicators obtained previously. An example of aggregation calculated from low-level indicators is given below: - NIN = number of verbs in the infinitive - NPR = number of verbs in the present - NFU = number of verbs in the future - NPA = number of verbs in the past.

Calculated aggregation yields an intermediate level indicator NV which is the total number of verbs, NV = NIN + NPR + NFU + NPA. An example of a ratio calculated from low-level indicators is given below: - NP = number of sentences - NV = number of verbs. The calculated ratio gives an intermediate level indicator NM which is the average number of verbs per sentence, ie NM = NV / NP. Thus, this step S5 makes it possible to obtain meta-descriptions such as readability, range of vocabulary or lexical cohesion. From the bag-of-words sub-representation, the algorithm generates a sixth under-representation which represents lexical fields (step S6) present in the text, by a principal component analysis (PCA, acronym for Principal Components Analysis ") and / or latent semantic analysis (LSA) and / or non-negative Matrix Factorization (NMF). This is therefore a dimensionality reduction step to obtain lexical fields. When too many lexical fields are obtained by these three analyzes, the agorithm generates an additional step of reducing the dimensionality (step S7). This step S7 therefore consists in putting all the lexical fields together and keeping only a small number so that those kept are unique and relevant fields. In other words, in case of redundancy in the components of the vector generated according to step S6, this step S7 makes it possible to select the non-redundant components of the vector. Once all the vector sub-representations have been generated, the algorithm realizes their concatenation (step S8) into a final representation of the text. Many variants and improvements can be envisaged without departing from the scope of the invention. Thus, one can consider different variants of classification according to the desired use.

One can thus classify the works in order of literary quality rather than to attribute a precise note of the quality for each work. It is thus a question of positioning the texts of a set with respect to each other.

Ranking can be reduced to a regression or classification. The ranking may be derived from the regression problem simply by classifying the texts according to the score assigned by the program implemented. Another approach for ranking is to derive it from the classification problem. In the same way as for regression, we can classify the texts by probability of belonging to a class. There are algorithms that directly optimize the classification of objects. These algorithms can be used from the vector representation of texts according to the invention.

Alternatively, one can consider a binary classification of texts according to the quality. One can also consider a multi-class classification which is an extension of the binary classification. This time, it is no longer a question of dividing the texts into two subsets but into n subsets (n> 2). One can, for example, try to make four equal subsets of quality: the very good, the good, the mediocre and the bad. An advantage of this classification is that we can calculate the probabilities of belonging to each class, which gives us a distribution over the space of the notes, if the notes are discrete and go from 1 to 10 by example. A distribution on the notes gives more information than a simple regression.

Thus, learning the distribution for a particular work allows to answer to finer questions on the text like: - to be able to differentiate a constant distribution (flat) of a distribution with two modes (in camel's back) which divides the people and created the controversy, - appreciate the probability that the text is very or very bad, - find the values discovered with the regression: the mean and the standard deviation of the distribution. According to yet another variant, one may not seek to predict universal literary quality (or distribution a note space), but rather seek to personalize the prediction. Thus, it is conceivable to group the annotators into large types of identified profiles, possibly by clustering ("data clustering" in English), or by hand, and to drive the algorithm according to the invention for each identified reader group. The major advantage of this variant is to reduce the noise in the target value, the literary quality of a text, which one seeks to predict. If we group the annotators by profiles, we will create groups of users who generally arrive at consensus on the texts. Creating a template by group of annotators and clearly identifying their profiles then makes it possible to make different predictions according to the use that one wishes to make of the prediction. If the prediction is for the use of an identified person, we can then initially identify the group to which it belongs and then make the prediction with the literary quality analysis of this group.10 REFERENCE S CITEES [ 1]: "DEFT2014, automatic analysis of literary and scientific texts in French", Lecluze et al., 21st Automatic Processing of Natural Languages, Marseille, 2014; [2]: "Fine Semantic Categorization of Expression of Opinion for Consensus Detection", Benamara et al., 21st Automatic Language Processing, Marseille, 2014; [3]: "Revisiting Readability: A Unified Framework for Predicting Text Quality", Pitler et al. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP '08). Association for Computational Linguistics, Stroudsburg, PA, USA, 186-195.15

Claims (7)

REVENDICATIONS1. A method for analyzing the literary quality of a text, implemented by a (micro) computer processor, comprising the steps of: a / receiving a raw text to be analyzed (step SO); b / generating several vector sub-representations (1) of the received text to obtain indicators, called low-level indicators, the sub-representations consisting of: - a representation by word bag (step Si) according to which the distributions of each word and analyzes the distributions of certain unigrams, bi-grams, 3-grams, 4-grams, 5-grams and 6-grams at the scale of the word and characters, a representation called morphosyntactic structure (step S2 ), according to which the parameters of the distributions of the grammatical words in the text are calculated and the distributions of each syntactic function in the text, the paragraphs, the sentences and the propositions are analyzed, - a representation of the writing errors (step S3) according to which one calculates the number of times each rule of each of the categories of writing faults is not respected, - a stylometry representation (step S4) according to which the length of d the text, the length of the paragraphs, the length of the sentences, the length of the sentences, the length of the words in characters, the number of each punctuation sign, and finally the parameters of the distribution of the dialogues in the text; c / generate (2): a meta-description (step S5) according to which the vocabulary of the text is analyzed by the different levels of rarity of the words, the lexical fields used, the words adapted to the youth, and computations are calculated and ratios of low-level indicators 25 obtained in b /; a representation of the lexical fields (step S6) present in the text from the word bag representation performed in b /, by principal component analysis (PCA) and / or an analysis Latent semantics (LSA) and / or non-negative matrix factorization (NMF) .d / concatenation (step S8) of vector-generated under-representation. in b / and c / in a final representation of the text; e / subjecting the final vector of the text to a classifier whose classification algorithm (s) is (are) driven from a set of reference texts whose literary quality has been measured numerically by a population of individuals, preferably a population of experts, and then weighted. 2. Analysis method according to claim 1, wherein the analysis of the distributions of each word performed in b / is made after the NLT (Automatic Natural Language Processing) treatments. 3. Analysis method according to claim 1 or 2, wherein the calculation of the parameters of the distribution of grammatical words and dialogues in the text is made from criteria selected from the mean, the variance, the standard deviation, entropy, distance between distributions, or a combination of these. 4. Analysis method according to one of claims 1 to 3, comprising the implementation of one or more dimensionality reduction algorithms (step S7) on the lexical fields obtained in c /. 5. Analysis method according to one of the preceding claims, the (the) classification algorithm (s) being adapted (s) to allow achieve a binary classification of the text according to the quality. 6. Analysis method according to one of claims 1 to 4, the (the) classification algorithm (s) being adapted (s) to enable a multi-class classification of text according to the quality. A computer program comprising program code instructions for performing the steps of the method according to one of the preceding claims when said program is run on a computer.