CN107291688A - Judgement document's similarity analysis method based on topic model - Google Patents

Abstract

The invention discloses a kind of judgement document's similarity analysis method based on topic model.This method uses LDA (Latent Dirichlet Allocation) topic model in machine learning, for judgement document, proposes a kind of based on semantic, semi-automatic, general similarity analysis method.This method mainly includes choosing language material, sets up the steps such as similarity mark, Text Pretreatment, input selection, parameter setting, repetitive exercise, generation model and application model.This method takes into full account the characteristics of specialized vocabulary is abundant, semanteme is complicated in judgement document's content, using the semi-structured characteristic of judgement document, so as to improve the accuracy and applicability of judgement document's similarity analysis on the basis of general similarity analysis method.

Description

Similarity Analysis Method of Judgment Documents Based on Topic Model

technical field

The invention is a text similarity classification method, aimed at the court's internal judgment documents, and belongs to the technical fields of machine learning and text mining.

Background technique

The China Judgment Documents Network started construction in 2013. As of May 14, 2017, it has accumulated more than 29 million documents and has gradually grown into the world's largest website for sharing judicial documents. Based on these data, a series of judicial big data research and analysis work has been carried out one after another. While achieving remarkable results, there are still many problems and challenges. Some of the problems focus on the inadequacy of court data mining and analysis capabilities and related research.

Judgment documents, as an important part of the court's work, record the process and results of the people's court's trial. It is not only the carrier of the results of the court's litigation activities, but also the only evidence for the people's court to determine and distribute the substantive rights and obligations of the parties. Judgment documents gathered during the informatization process of Chinese courts have become valuable data resources in the trial field. Through big data mining research on judgment documents, more intelligent information technology methods can be proposed to assist judges in handling cases. For example: excavate the judgment documents of similar cases from the existing judgment document database, and provide similar case recommendations for judges; the court can evaluate the workload of a judge for a period of time according to the similarity of the judgment documents handled by a judge; judges, litigation participants, Legal workers, etc. can enter the case to view the relevant legal provisions that may be involved in a case. In response to these application scenarios and requirements, this patent proposes a text similarity analysis method for referee documents.

Due to the various requirements of the court trial work, the various characteristics of the judgment documents themselves, and the limitations and difficulties of the existing similarity analysis methods, it is urgent to study a similarity analysis method tailored to the characteristics of the judgment documents. Among them, the semi-structured nature of Chinese court judgment documents provides the possibility to improve the results of text similarity analysis and provides a basis for the evaluation of similarity results. The relatively fixed classification and identification information such as cause of action and legal provisions in Chinese court judgment documents provide inspiration for us to use the topic model method. The characteristics of the text content of the judgment documents, which focus on logic and reasoning, put forward corresponding requirements for the semantic understanding ability of the similarity method. Based on the above reasons, this paper proposes a semantic-based, semi-automatic, and general-purpose similarity analysis method for referee documents using LDA (Latent Dirichlet Allocation) topic model.

As an important research direction in natural language processing, the text similarity analysis method is used to measure the similarity between target objects, and has been applied in many fields such as information extraction, text classification, text clustering, topic exploration, and topic tracking. There are generally two key points in the similarity method: one is the representation of features, and the other is the calculation of the similarity relationship between features. Existing text similarity analysis methods, from purely character-based analysis methods to analysis methods based on corpus and knowledge system mining text semantics, have experienced long-term development. On the basis of this, the Chinese similarity method further explores the characteristics of the Chinese language. Due to the diversity of the similarity problem itself, when targeting different target objects (words, short texts, long texts) or in different application scenarios, it is often necessary to use different similarity methods to achieve better analysis results.

The topic model series methods, especially LSA (Latent Semantic Analysis) and LDA related methods, are an important direction for text similarity research. In short, the topic model is based on the assumption that each word belongs to certain topics with a certain probability, and each text expresses several topics. When the topic model is applied in the similarity analysis method, the topic attribution of a text can be inferred for the trained topic model, so that the similarity between different texts can be further calculated according to the topic situation. From another perspective, the topic model is equivalent to mapping a vocabulary-based high-dimensional vector to a semantic space to reduce its dimension. The topic model is developed from Latent Semantic Indexing (LSI), and Probabilistic Latent Semantic Analysis (pLSA) is the first influential topic probability model. Blei introduced Dirichlet distribution on the basis of pLSA, proposed LDA, and further generalized the topic model method. In the subsequent applied research, in order to address different problems and improve the efficiency of LDA (parallelization), a series of other improved methods related to LDA have been published.

The full name of LDA is Latent Dirichlet Allocation, which was proposed by Blei in 2003. It is an unsupervised topic model method that can be used for semantic understanding and hidden topic identification on large-scale document sets or corpora. Compared with the original topic model method, the LDA method introduces the Dirichlet distribution and adds a priori probability assumption. This makes it easier for the model to be applied to texts other than the training corpus, reduces the possibility of model overfitting, and has better expressiveness for corpora with a small amount of data. At present, the LDA method has been widely tried and applied in the fields of text information extraction, text classification, automatic text summarization, and image processing.

The topic model approach is built on the assumption of the following formula:

Among them, P(t _l |D _i ) represents the probability of word t _l appearing in document D _i , p(t _l |T _j ) represents the probability of word t _l appearing in topic T _j , P(T _j |D _i ) represents the probability that topic T _j appears in document D _i . The model assumes that the probability of a word appearing in a document is equal to the sum of the probabilities of all topics to which the document may belong multiplied by the probability of the word appearing in each topic.

The training of the LDA model mainly includes two different methods: Gibbs Sampling and variational EM. Among them, GibbsSampling simulates the sampling process on the basis of the Markov chain, in order to obtain a stable probability distribution in the process of probability transfer. The variational EM method is built on the basis of Bayesian calculation, which is mainly completed by two steps: finding the optimal solution of variational parameters (E step) and estimating the original model parameters and maximizing the lower bound of the model (M step). Its model diagram is shown in Figure 4.

The figure shows the content as follows: the topic distribution θ _i of the document is generated from the document obtained according to the hyperparameter α, and the Dirichlet prior distribution among topics; the topic z _{i, j} of the document is generated from the multinomial distribution θ _i of the topic; Generate the word distribution of topic z _{i, j} in the Dirichlet prior distribution between topics and words obtained by hyperparameter β from the multinomial distribution of words In the end, the word w _{i, j} is generated. There is a certain difference in the specific expression of the model between Gibbs Sampling and variational EM. In comparison, the variational EM method has a faster training speed than the Gibbs Sampling method, but the result obtained by the variational EM method is a local optimum, not necessarily equal to the global optimum. At the same time, although the program logic of Gibbs Sampling is relatively simple, it cannot support distributed computing like the variational EM method.

Contents of the invention

The technical problem to be solved by the present invention is: how to propose a general, semi-automatic similarity analysis method for judgment documents, so as to be applied to similarity-based document classification, similar document recommendation, and judge work based on similarity of judgment documents. Quantitative evaluation, prediction of legal provisions for cases, etc. This method uses the TF-IDF method and LDA method in text mining technology to establish a similarity analysis model through a series of processing work and iterative training methods, and obtain the similarity relationship between documents according to the analysis model. This method has good similarity analysis results and has fast similarity calculation capabilities, thus providing a better foundation for similarity-related applications based on referee documents.

Technical scheme of the present invention is:

1. The similarity analysis method of referee documents based on topic model is characterized in that the text mining method based on topic model is used to analyze text similarity for referee documents and their characteristics. The brief process steps of this method are shown in Figure 1, in which the text preprocessing and parameter selection parts have multiple sub-steps, and the iterative training part can be further expanded. The detailed process steps are shown in Figure 2, specifically as follows:

(1) Use the structured classification information of the judgment document (including the cause of action, case type, etc.) as the target corpus;

(2) The target corpus is divided into training corpus and test corpus, and the test corpus is marked with similarity;

(3) Perform preprocessing operations on the document text as the training corpus, including document segmentation, document screening, Chinese word segmentation, word acquisition and filtering operations before and after word segmentation;

(4) Select the high-confidence part of the target corpus as the input content;

(5) Set various parameters, including stop words, LDA model training parameters, TF-IDF input and evaluation standard settings;

(6) Use training corpus, carry out model training according to LDA;

(7) Use the test corpus to evaluate the training model (referring to the degree of conformity with the test corpus similarity label);

(8) Adjust the parameters, iteratively execute step (6), until the traversal is completed for all required parameters;

(9) According to the accuracy under different parameters, select appropriate parameters to generate a training model;

(10) Apply the training model.

2. In step (2), its specific content is shown in FIG. 3 . First, the target corpus obtained in step (1) should be divided into two parts: training corpus and test corpus. Afterwards, similarity labeling is required for the test corpus.

Similarity labeling refers to labeling the expected output results for a certain amount of target documents. For example, annotate the results of similarity measures for each document relative to other documents, or classification, ranking, and similar results based on similarity. This process is determined by two dimensions: one dimension is the labeling method, which indicates the implementation method of labeling; the other dimension is labeling granularity, which shows the degree of detail of labeling.

There are two types of labeling methods: one is automatic labeling, which needs to formulate and implement corresponding similarity judgment strategies; the other is manual labeling, which is completed by relevant court experts.

There are two types of labeling granularity: the first is digital labeling, which refers to the use of digital forms to mark the similarity of each document compared with the target document; the second is non-digital labeling, which is aimed at situations where it is not possible to mark each document in digital form , you can use similar labeling methods such as classification and sorting.

3. Step (3) is aimed at simplifying input and removing interference, including five specific preprocessing sub-steps:

(3.1) Segment the judgment document;

(3.2) Remove irregularly written judgment documents;

(3.3) Delete stop words that are harmful to word segmentation in the judgment documents;

(3.4) Carry out Chinese word segmentation for the judgment document;

(3.5) Generate exclusive stop words for referee documents.

4. Since there are differences in the importance and credibility of each part of the referee document in the similarity analysis, step (4) needs to select the highly credible part of the target corpus as the input content.

5. Step (5) is aimed at constructing model training parameters and completing pre-training preparations, including the following four sub-steps:

(5.1) Set stop words;

(5.2) Set training parameters;

(5.3) generate TF-IDF vector for training corpus;

(5.4) The evaluation standard setting is used to determine the actual effect of the training model.

6. In step (10), the model obtained by training can be used to calculate the topic-based similarity relationship between any two documents, so that the similarity between any two documents can be quickly obtained, and a series of Applications based on similarity include classification of similarity of judgment documents, recommendation of similar judgment documents, workload evaluation of judges based on similarity of judgment documents, recommendation of legal provisions based on case conditions, etc.

According to the content of the present invention, we have developed a similarity analysis tool for judgment documents based on python language. This tool can support model training work, and at the same time, it can also be directly used to recommend similar judgment documents and predict legal provisions based on case conditions. . In addition, based on the judgment document similarity analysis tool, more abundant similarity analysis and applications can be expanded.

Based on the general similarity analysis method, this method fully considers the characteristics of rich professional vocabulary and complex semantics in the content of the judgment documents, and utilizes the semi-structured characteristics of the judgment documents to improve the accuracy and applicability of the similarity analysis of the judgment documents. In addition, because the similarity analysis method adopts the topic model method, it can improve the real-time response speed of similarity analysis through offline processing, thereby improving the use efficiency of related applications.

Description of drawings

Figure 1. Brief flow chart of similarity analysis method for referee documents based on topic model

Fig. 2 Detailed flow chart of similarity analysis method of referee documents based on topic model

Figure 3 Classification and labeling methods of target referee documents

Fig.4 Schematic diagram of LDA model

Figure 5 Example of similarity labeling

Figure 6 Example of the Basic Situation of Judgment Document Cases

Figure 7 The core structure of the referee document

Figure 8 Example steps of training model evaluation

Figure 9 Example of a line chart of the number of topics and model evaluation results

Figure 10 similarity recommendation application flow chart

detailed description

In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be described in detail below in conjunction with the accompanying drawings and specific examples.

The invention aims at analyzing the similarity of the judgment documents. The analysis results can be applied to scenarios such as classification of judgment documents based on similarity, recommendation of similar judgment documents, evaluation of judge workload based on similarity of judgment documents, and prediction of legal provisions of cases. This method uses the TF-IDF method and the LDA method, and at the same time performs special processing and measurement for the characteristics of the referee document. The specific steps are as follows:

(1) In the set of judgment documents, a certain attribute (such as the cause of action, case type, etc.) is used as a screening condition to extract a subset of target documents as the target corpus;

(2) The target corpus is divided into training corpus and test corpus, and the test corpus is marked with similarity;

(3) Perform preprocessing operations on the document text as the training corpus, including document segmentation, document screening, Chinese word segmentation, word acquisition and filtering operations before and after word segmentation;

(4) Select the high-confidence part of the target corpus as the input content;

(5) Set various parameters, including setting stop words, LDA topic model training parameters, TF-IDF input and evaluation criteria;

(6) Using the training corpus, apply the LDA topic model to carry out model training;

(7) Use the test corpus to evaluate the training model (referring to the degree of conformity with the test corpus similarity label);

(8) Adjust the parameters, iteratively execute step (6), until the traversal is completed for all required parameters;

(9) According to the accuracy under different parameters, select appropriate parameters to generate a training model for similarity analysis of referee documents;

(10) Apply the training model generated in step (9) to analyze the similarity of referee documents.

The following will be combined with the example of similarity analysis of civil first-instance case judgment documents to give a specific explanation. This application example aims to predict legal provisions that may be related to it based on the content of the basic situation of the case through the similarity of documents. This function can help judges in the adjudication process, and at the same time, help parties complete automated legal consultation:

(1) This step is intended to obtain the target document corpus, and this part of the document will be used as the object for training and test verification in the future process. Since the judgment documents adopt a semi-structured format, the case type and cause of action information in the documents can help us further classify the documents. At the same time, under different classifications, case conditions and corresponding legal provisions also correspond to varying degrees. Therefore, in order to improve the accuracy of subsequent model training and reduce complexity, this method requires further classification operations on documents, and separate processing for different types of content. The dimensions of classification include case type and cause of action. Among them, in terms of classification granularity, case type > cause of action.

In this example, only case types are used for classification, and the case type is selected as civil first-instance cases, with a total of 53,000 articles. Due to the timeliness of the law itself, this part of the documents are selected after 2014 in the filing year.

(2) This step is intended to divide the document into two parts, the training set and the test set. The former is used for model training, and the latter needs to be marked with similarity to show the expected output results for testing and evaluating the model results, so as to obtain an ideal model that can be used for similarity analysis in an iterative manner.

As mentioned in the content of the invention above, the similarity labeling process is determined by two dimensions, one is the labeling method, which indicates the implementation of the labeling. Another dimension is labeling granularity, which indicates the level of detail of labeling.

There are two marking methods. One is automatic labeling, which requires formulating and implementing corresponding similarity judgment strategies. The other is manual labeling, which is done by relevant court experts.

There are two types of labeling granularity. The first one is digital labeling, which refers to marking the similarity of each document with other documents in the target document in digital form. For example, document 1 is labeled with document 2, and the similarities of document 3 are 80% and 60%, respectively. The second is non-digital labeling. For situations where it is not possible to label articles in digital form, labeling methods such as classification and sorting can be used. For example, if it is expected to use this method to recommend similar documents, relevant court experts can manually classify the documents in the test corpus, and use the classification as the label content.

The relationship and advantages and disadvantages of the two are shown in Table 1. Since digital annotation is more accurate than non-digital annotation and is conducive to better results, digital annotation should be used under the same conditions. For manual labeling, since digital labeling is often difficult to achieve, more non-digital labeling is used.

Table 1 The labeling method has shortcomings and the relationship with the labeling granularity

In this example, 50,000 documents are selected as the training set. 3000 documents will be used as the test set. For the test set, the automatic and non-digital labeling methods are selected for labeling. Since the goal of this example is to predict the legal provisions based on the case, for each document in the test corpus, the main legal provisions cited by it are selected as the annotations of this document. Due to the relatively fixed nature of the legal text and its own writing, the labeling process can be completed in a certain automated way. The specific form of expression is shown in Figure 5, where each document is related to several legal provisions, and the numbers in square brackets represent the serial numbers of the corresponding documents and legal provisions. Due to the randomness of the writing of the legal provisions themselves, it is necessary to deal with and correspond to the legal provisions when labeling. In this example, only specific legal entries are considered, and the cited sums are not further recorded to simplify the calculation. Correspondingly, in step (7), the method compares the legal clauses predicted by the model with the legal clauses actually marked after inputting the case, so as to complete the test and evaluation work.

(3) This step is aimed at preprocessing the training set documents. Its main objectives are as follows: 1. Obtain the required passages in documents and training; 2. Eliminate noise interference. The steps are described in detail below:

(3.1) Judgment documents are semi-structured. The courts of our country have formulated the paragraph structure specification of the judgment document. Based on the paragraph structure specification and the commonly used characteristic vocabulary of each paragraph, the text of each paragraph of the judgment document can be obtained, which will be beneficial to our subsequent training and analysis work.

(3.2) Due to the phenomenon that individual judgment documents do not follow the paragraph structure specification of the judgment document, some documents are written too casually. Regarding these judgment documents that do not follow the paragraph structure specification, we will remove them from the training set to reduce interference.

(3.4) Word segmentation is often the basis of Chinese language processing. In this example, jieba word segmentation is used to carry out specific word segmentation work.

(3.3), (3.5) In the judgment documents, there will be a large number of place names (such as XX city, XX county), proper nouns (such as plaintiff, defendant) and low-frequency words. These words are not only meaningless to compare the similarity of documents, but may interfere with the training results. For example, words such as "plaintiff", "defendant" and "this court" in the judgment documents. Therefore, in most cases, this part of vocabulary needs to be removed. Since some words may cause additional interference during word segmentation, we choose to remove some words in step (3.3). At the same time, some court high-frequency words need to be obtained by statistics in the judgment documents, so in step (3.3) In (3.5), count the high-frequency non-specific vocabulary as the thesaurus of subsequent stop words.

In this example, the basic information of the case in the training corpus is extracted as the input paragraph, and the main part of the paragraph can be further disassembled into the plaintiff’s statement, the defendant’s defense, the fact-finding paragraph, and the evidence paragraph. The production specifications of relevant judgment documents (for details, please refer to the content of relevant court document production specifications: http://www.cibsn.com/Article/Detailed/43618), and has an obvious segmentation principle, which can be automatically disassembled. The basic situation of a document and its subsections are shown in Figure 6.

Among them, if the above paragraph does not exist in the training corpus document, or the corresponding paragraph cannot be disassembled, then the document is excluded. At the same time, in (3.3), part of the unique vocabulary of the judgment document is removed, the place names at all levels are removed, and the name and name pronoun in the document are removed, similar to Wang, Wang, Wang, etc. In (3.5), the stop words after the addition of the high-frequency non-special meaning specific vocabulary of the judgment document in this part of the document are counted.

(4) For judgment documents, in the process of similarity analysis, the importance of different paragraphs is different. At its root, this is related to the structure of the judgment document itself. For a judgment document, its core content consists of evidence, facts, legal provisions, and judgments. The evidence is confirmed by the evidence, the facts are derived from the evidence (or facts), the facts are related to the legal provisions, and the judgment results are obtained from this, as shown in Figure 7. Among them, the judgment is the result, the legal provisions are clear terms, while the evidence and facts are full of uncertainties. For example, some evidence has been confirmed by the court, while some evidence is not admissible; the facts described in the plaintiff's statement are not as credible as the facts in the ascertaining facts paragraph. Therefore, if the credibility of the listed facts and evidence can be obtained, the different importance of different words can be reflected more effectively. However, in actual operation, due to the randomness of the natural language of the document, it is difficult to obtain the credibility of the evidence and facts one by one, so we generally select the highly credible part from the corpus as the input for subsequent training.

In this example, the main part of the basic information section of the case can be decomposed into the plaintiff's statement, the defendant's defense, the fact-finding section, and the evidence section. Among them, the fact-finding segment and the evidence segment can be used as high-confidence parts, which can be used as input for our subsequent training.

(5) In addition to setting the training parameters for the LDA model, this step also needs to complete the preparatory work before training.

(5.1) Set stop words, including vocabulary in (3.5) and common stop words. The specific content can be adjusted according to actual requirements.

(5.2) Set training parameters to provide boundaries for iterative training. This part mainly includes the topic number range and the topic interval number, and the topic number can be set by referring to the number of legal articles under the corresponding type.

(5.3) Generate TF-IDF vectors for the corpus. Compared with using bag-of-words or word-set models alone as input, the input method of TF-IDF vector is more expressive.

(5.4) Set evaluation criteria. This step determines the relevant parameters when using the test corpus to calculate the accuracy of the model in step (7).

In this example, the commonly used stop words are words commonly used in Chinese without special meaning, including words such as "的" and "了". The topic range is 300-900, and the topic interval is 50.

(6) According to the settings, use the LDA algorithm to train on the processed training set. Due to the slow convergence speed of the LDA model, when the training scale is large, the resources and time required for training are longer.

In this example, the gensim program is used as the underlying LDA algorithm library to complete the experiment. The training starts when the number of topics is 300, and then the number of topics increases by 50 each time until the number of topics increases to 800. During this period, each result is passed to the next Evaluate the trained model in one step.

(7) Use the test corpus to calculate the accuracy of this training model. The operation process of this part is related to the similarity labeling method in step 2, and different accuracy calculation schemes will be obtained according to different similarity labeling schemes. If digital labeling is used, it is recommended to use indicators such as precision rate and recall rate for calculation, for example, to evaluate by the number of predicted legal provisions that are successfully hit within a certain number. However, if a non-digital labeling method is used, the design of this part should be carried out according to the actual design requirements and display needs. For example, if the similarity labeling method is to select n other documents with the highest similarity for any document, there is only sorting but no specific value, then according to the importance of different sorting, different weights can be assigned to each position of the sorting, and then Calculation.

In this example, an automated, non-digital labeling method is used. The specific evaluation method is obtained according to the prediction accuracy of the legal provisions. The specific steps are shown in Figure 8, that is, the model is used to predict the legal provisions, and the prediction accuracy is taken as the result of the model evaluation.

(8) In this step, steps (6) and (7) need to be executed iteratively. Repeat steps (6) and (7) within the set parameter iteration range and parameter iteration rules. And record the accuracy exhibited by the model obtained under different parameters in the evaluation. Intuitively, this step will draw us a line graph with the iteration parameters on the horizontal axis and the accuracy of the training model on the vertical axis. We can make subsequent decisions based on the line graph.

In this example, we use the number of topics of the LDA model as the iteration parameter, increasing the number of topics from 300 to 800 will generate 11 different training models. An example of the line chart of the number of topics and model evaluation results is shown in Figure 9. The evaluation results has a local maximum.

(9) According to the previously obtained line chart about the number of topics and model evaluation results, decide what number of topics needs to be selected for model training.

In this example, the accuracy of the trained model is highest when the number of topics is around 450.

However, the results in this example are a one-off presentation after many experiments. In fact, in actual operation, it may be difficult to determine the appropriate number of topics at one time, and even if the statistical curve shows a clear peak like the figure below, it is impossible to be sure that the corresponding value is the global optimum rather than the local optimum . Therefore, in this step, if conditions permit, please increase the number of iterations as much as possible, and at the same time, determine how to select the number of topics together with the expected accuracy requirements. Afterwards, training can be carried out according to the determined appropriate parameters, and the results after training at this time will be used as the underlying model for future applications.

(10) In the previous steps, we obtained the basic model that can be used for similarity analysis. However, in practical applications, we also need to add an application layer to it to improve the visibility of the results. As mentioned above, this model can support applications such as classification of similarity of judgment documents, recommendation of similar judgment documents, workload evaluation based on similarity of judgment documents, and prediction of legal provisions based on case conditions.

In this example, the prediction application of legal clauses based on the case is implemented on the basis of the underlying model. The effect is that when inputting the basic situation of a civil first-instance case or the fact-finding paragraphs and evidence paragraphs, the system can predict the legal provisions that may be relevant to the case according to the model. The implementation process is shown in Figure 10, that is, apply the similarity model obtained by this method, first search for similar documents according to the input of the case, and then obtain the predicted legal provisions according to the statistics of the legal provisions cited by similar documents.

Claims (4)

1. The similarity analysis method of referee documents based on topic model is characterized in that for referee documents and their characteristics, the text mining method based on topic model is used to carry out text similarity analysis, and its steps are as follows: (1) In the set of judgment documents, a certain attribute (such as cause of action, case type, etc.) is used as a screening condition to extract a subset of target documents as the target corpus; (2) The target corpus is divided into training corpus and test corpus, and the test corpus is marked with similarity; (3) Perform preprocessing operations on the document text as the training corpus, including document segmentation, document screening, Chinese word segmentation, word acquisition and filtering operations before and after word segmentation; (4) Select the high-confidence part of the target corpus as the input content; (5) Set various parameters, including setting stop words, LDA topic model training parameters, TF-IDF input and evaluation criteria; (6) Using the training corpus, apply the LDA topic model to carry out model training; (7) Use the test corpus to evaluate the training model (referring to the degree of conformity with the test corpus similarity label); (8) Adjust the parameters, iteratively execute step (6), until the traversal is completed for all required parameters; (9) According to the accuracy under different parameters, select appropriate parameters to generate a training model for similarity analysis of referee documents; (10) Apply the training model generated in step (9) to analyze the similarity of referee documents. 2. The method for similarity analysis of referee documents based on topic model according to claim 1, characterized in that: step (3) includes five specific preprocessing methods for the purpose of simplifying input and removing interference through a preprocessing process Substeps: (3.1) Segment the judgment document; (3.2) Remove irregularly written judgment documents; (3.3) Delete stop words that are harmful to word segmentation in the judgment documents; (3.4) Carry out Chinese word segmentation for the judgment document; (3.5) Generate exclusive stop words for referee documents. 3. the method for similarity analysis of referee documents based on topic model according to claim 1, characterized in that: step (5) is aimed at building model training parameters and completing preparatory work before training, including the following four sub-steps: (5.1) Set stop words; (5.2) Set training parameters; (5.3) generate TF-IDF vector for training corpus; (5.4) The evaluation standard setting is used to determine the actual effect of the training model. 4. The method for analyzing similarity of referee documents based on topic model according to claim 1, characterized in that: in step (10), the model obtained by training can be used to calculate the topic-based similarity between any two documents , so that the similarity between any two documents can be quickly obtained, and a series of applications based on similarity can be developed, including similarity classification of judgment documents, recommendation of similar judgment documents, judge workload evaluation based on similarity of judgment documents, Recommendations on legal provisions based on the facts of the case, etc.