CN115391522A - Text topic modeling method and system based on social platform metadata - Google Patents

Abstract

The present invention proposes a text topic modeling method and system based on social platform metadata, including keywords based on text data, constructing a bag of words representation of text data; based on metadata categories of text data, training attribute value prediction of corresponding categories The task is to fine-tune the pre-trained semantic extraction model to obtain the target semantic extraction model, and use the target semantic extraction model to extract the text semantic representation of the text data; construct a semantic constraint target based on the text semantic representation, guided by the semantic constraint target, and use the bag of words representation as the Input and reconstruct targets, train a neural topic model based on a variational autoencoder, obtain a topic extraction model, and derive topic-keyword distribution and topic embedding representations from the model. The method and system can perform topic modeling on short text messages widely existing in mobile applications, extract topic keywords and learn topic embedding representations.

Description

A text topic modeling method and system based on social platform metadata

technical field

The invention is applicable to the field of mobile application big data analysis, relates to a mobile application metadata-oriented topic modeling method and system, and in particular relates to a social application platform metadata-oriented topic classification method and system.

Background technique

The topic modeling task aims to perform probabilistic modeling on the corpus and discover a set of potential topics. The obtained topics can be used in fields such as user portraits, public opinion analysis and tracking, and human-computer dialogue. Each topic can be used to describe an interpretable semantic concept, corresponding to a probability distribution over the vocabulary. At the same time, given a document, the topic model can infer its topic distribution. As a powerful unsupervised text analysis technique, topic modeling can extract the topics discussed in massive texts, and cluster or classify the texts according to the topic distribution.

Latent Dirichlet Allocation (LDA) is a Bayesian probabilistic topic model proposed in 2003, which infers the topic distribution of documents by modeling the generation process of documents. As shown in Figure 5, in the LDA topic model, there are M document-topic Dirichlet prior distributions, corresponding to M document-topic multinomial posterior distributions, so that α→θ _d →z _d constitutes the Dirichlet -Multi conjugation, using the Gibbs sampling method to obtain the document-topic posterior distribution based on the Dirichlet distribution.

Although the LDA model has achieved great success in topic modeling tasks, its inference efficiency is too low to be applied in large-scale analysis scenarios. To address the aforementioned efficiency issues, the variational autoencoder-based neural topic model employs a deep neural network-based decoder and encoder to fit the process of document generation and topic inference. The decoder and encoder of the model are trained by maximizing the Evidence Lower Bound (ELBO) of the marginal likelihood function of the input data and used for subsequent topic inference tasks:

Among them, p represents the posterior probability distribution, q represents the prior probability distribution, x represents the word distribution of the document, and z represents the topic distribution of the document.

The word distribution of short text messages in social platforms is sparse and the content forms are diverse. There are still some shortcomings and deficiencies in the existing topic modeling methods.

First of all, in terms of computational efficiency, the traditional topic modeling method represented by the LDA model needs to infer the topic of the document to be predicted by sampling, which is inefficient and cannot be adapted to large-scale data analysis scenarios, especially social media streams. data scenarios. The emergence of neural topic modeling methods has alleviated this deficiency to a certain extent.

Second, the existing neural topic models mainly use the bag-of-words model of documents as input. However, in short text scenarios, the word distribution of a single document is very sparse, and it is difficult to accurately infer the topic of the document using only discrete word distribution information. To solve this problem, some existing models consider adding constraints to the topic distribution of short texts, such as constraining all words in short texts to belong to the same topic. Although this type of method can alleviate the problems caused by the sparsity of the corpus to a certain extent, it also greatly reduces the modeling ability of the model.

In addition, the existing neural topic models still mainly use text content as the main basis for topic modeling. However, in mobile applications, especially social applications, short texts often contain rich attribute information, such as Hashtag, URL, etc., which can provide important clues for topic modeling. Although the existing models introduce covariates and label information that are closely related to corpus documents, they require the attachment information of each document to be complete, which greatly limits the scope of use of topic models. In fact, there is often a large amount of missing attribute information in short texts, and direct use of existing models will introduce a lot of noise, resulting in failure of topic modeling.

To sum up, there are two main problems in existing topic modeling methods: (1) the bag-of-words model used cannot fully express the semantics and topics of sparse short texts; (2) existing methods cannot effectively utilize the richness of short texts. attribute information.

Contents of the invention

In order to solve the problem that the existing topic model cannot fully express the semantics of short texts using the bag-of-words model, and cannot effectively utilize the multi-attribute information of short texts, the present invention proposes a topic modeling method and system for mobile application metadata . The method and system can perform topic modeling on short text messages widely existing in mobile applications, extract topic keywords and learn topic embedding representations, and can also effectively integrate sparse multi-attribute information of short texts to further improve topic construction. The effect of the model, and supports the analysis of the attribute value distribution of the topic on each attribute information.

Aiming at the deficiencies of the prior art, the present invention proposes a text topic modeling method based on social platform metadata, which includes:

Step 1. Obtain the text data to be modeled and the metadata of the text data from the social platform;

Step 2, constructing a bag-of-words representation of the text data based on the keywords of the text data;

Step 3. Based on the category of the metadata, train the attribute value prediction task of the corresponding category to fine-tune the pre-trained semantic extraction model to obtain the target semantic extraction model, and use the target semantic extraction model to extract the text semantic representation of the text data;

Step 4. Construct a semantic constraint target based on the semantic representation of the text. Guided by the semantic constraint target, the bag-of-words representation is used as the input and reconstruction target to train a neural topic model based on a variational autoencoder to obtain a topic extraction model, and Deriving topic-keyword distributions and topic embedding representations from the model;

Step 5, input the topic embedding representation into the attribute value prediction task, obtain the attribute value distribution of the topic on the corresponding attribute, and merge the same topics according to the attribute value distribution, the topic-keyword distribution and the topic embedding representation, And the combined result is used as the topic model of the text data.

The described text topic modeling method based on social platform metadata, wherein this step 3 comprises:

Classify the attributes of metadata into discrete attributes, continuous attributes and text attributes;

For discrete attributes, based on the counting of attribute values that have appeared in the corpus, according to the process of constructing the vocabulary, the attribute values whose occurrence times exceed the preset threshold are taken to form an attribute value set, and a classification of predicted attribute values is constructed based on the attribute value set task, using cross-entropy as the loss function of the classification task;

For a continuous attribute, convert its attribute value to a distribution with a mean value of 0 and a variance of 1; construct a regression task to predict the converted attribute value based on the continuous attribute, and use MSE as the loss function of the regression task;

For text-type attributes, the text data is spliced with it to obtain the spliced text, which is input into the pre-trained semantic extraction model to generate a text semantic vector;

Build an adversarial classification task to determine the attribute category of the text semantic vector, using cross-entropy as the loss function.

The text topic modeling method based on social platform metadata, wherein the step 5 includes: according to the attribute value distribution, constructing an attribute value list of each attribute of the topic; according to the topic-keyword distribution, constructing a keyword list;

When merging topics, use the Jaccard coefficient distribution to measure the similarity between the keyword lists and attribute value lists of the topics, get the first similarity and the second similarity, and use the cosine similarity to measure the embedding representation of the topics The similarity between the subjects is obtained to obtain the third similarity; the weighted average of the first similarity, the second similarity and the third similarity is obtained to obtain the final similarity between the topics, and the topics whose final similarity is greater than the preset value are merged .

The text topic modeling method based on social platform metadata, wherein the metadata includes: posting time, posting user ID, posting user's profile, @User, #Tag and URL.

The present invention also proposes a text topic modeling system based on social platform metadata, which includes:

The initial module is used to obtain the text data to be modeled and the metadata of the text data from the social platform; and based on the keywords of the text data, construct the word bag representation of the text data;

The fine-tuning module is used to train the attribute value prediction task of the corresponding category according to the category of the metadata, so as to fine-tune the pre-trained semantic extraction model to obtain the target semantic extraction model, and use the target semantic extraction model to extract the text semantic representation of the text data;

The extraction module is used to construct a semantically constrained target based on the semantic representation of the text, guided by the semantically constrained target, taking the bag-of-words representation as input and reconstructing the target, training a neural topic model based on a variational autoencoder, and obtaining a topic extraction model , and derive topic-keyword distribution and topic embedding representations from the model;

The merging module is used to input the topic embedding representation into the attribute value prediction task, obtain the attribute value distribution of the topic on the corresponding attribute, and conduct the same topic according to the attribute value distribution, the topic-keyword distribution and the topic embedding representation Merge and use the merged result as the topic model of the text data.

The text topic modeling system based on social platform metadata, wherein the fine-tuning module is specifically used for:

Classify the attributes of metadata into discrete attributes, continuous attributes and text attributes;

For discrete attributes, based on the counting of attribute values that have appeared in the corpus, according to the process of constructing the vocabulary, the attribute values whose occurrence times exceed the preset threshold are taken to form an attribute value set, and a classification of predicted attribute values is constructed based on the attribute value set task, using cross-entropy as the loss function of the classification task;

For a continuous attribute, convert its attribute value to a distribution with a mean value of 0 and a variance of 1; construct a regression task to predict the converted attribute value based on the continuous attribute, and use MSE as the loss function of the regression task;

For text-type attributes, the text data is spliced with it to obtain the spliced text, which is input into the pre-trained semantic extraction model to generate a text semantic vector;

Build an adversarial classification task to determine the attribute category of the text semantic vector, using cross-entropy as the loss function.

The text topic modeling system based on social platform metadata, wherein the merging module is used to: according to the attribute value distribution, construct the attribute value list of each attribute of the topic; according to the topic-keyword distribution, construct the keyword list;

When merging topics, use the Jaccard coefficient distribution to measure the similarity between the keyword lists and attribute value lists of the topics, get the first similarity and the second similarity, and use the cosine similarity to measure the embedding representation of the topics The similarity between the subjects is obtained to obtain the third similarity; the weighted average of the first similarity, the second similarity and the third similarity is obtained to obtain the final similarity between the topics, and the topics whose final similarity is greater than the preset value are merged .

The text topic modeling system based on social platform metadata, wherein the metadata includes: publishing time, publishing user ID, publishing user's profile, @User, #Tag and URL.

The present invention also provides a storage medium for storing a program for executing any one of the social platform metadata-based text topic modeling methods.

The present invention also proposes a client for any of the text topic modeling systems based on social platform metadata.

As can be seen from the above scheme, the present invention has the advantages of:

1. Aiming at the problem that the neural topic model cannot fully express the semantics of short texts using the bag of words model, this invention proposes a constraint method based on the text semantic representation of the pre-trained model, and introduces external knowledge to guide topic modeling learning, which can effectively improve the neural network. The modeling effect of the topic model, and synchronously learn the semantic representation of each topic, and project the topic to the pre-training semantic representation space.

2. Aiming at the problem that the neural topic model cannot effectively utilize short text multi-attribute information, this invention proposes a set of preprocessing schemes for different types of attribute data, and integrates multi-attribute information into text semantic representation through multi-task learning and confrontation auxiliary tasks , introducing its own metadata to guide topic modeling learning can effectively improve the modeling effect of the neural topic model, and model and analyze the attribute value distribution of each topic on each attribute.

3. The present invention constructs a text theme analysis system oriented to mobile application metadata. Realized the preprocessing of short text content in mobile applications (word segmentation, part-of-speech recognition, entity recognition, vocabulary construction, word bag model representation conversion), metadata preprocessing (conversion processing of discrete, continuous and text attributes) ), the fine-tuning of the semantic representation of the pre-training model, the extraction of topic keywords (subjects (persons, institutions, organizational entities, etc.), actions (verbs, gerunds, etc.), time, place, etc.), the export of topic embedding representations and topics A series of processes such as the analysis of attribute value distribution on each attribute can comprehensively and efficiently mine and analyze keywords, embedded representations and attribute value distribution of short text topics in mobile applications.

Description of drawings

Figure 1 is a schematic diagram of a topic modeling method based on semantic constraints;

Fig. 2 is a schematic diagram of a topic modeling method for fusing metadata information;

Fig. 3 is the basic flowchart of the subject modeling method for mobile application metadata;

Fig. 4 is a schematic diagram of business logic of a theme modeling system oriented to mobile application metadata;

Fig. 5 is a system block diagram of the prior art LDA model.

Detailed ways

In order to achieve the above technical effects, the present invention includes the following key technical points:

Key point 1: Aiming at the problem that the existing neural topic model cannot fully express the semantics of short texts using the bag-of-words model, a neural topic modeling method based on pre-trained model text semantic constraints is proposed. Significantly improve the topic modeling effect of the neural topic model, and learn the embedding representation of the topic;

Key point 2: Aiming at the problem that the existing neural topic models cannot effectively utilize short text multi-attribute information, a multi-task learning-based multi-attribute information conditional topic modeling scheme is proposed. Overcome the negative impact of missing attribute values on topic modeling, significantly improve the topic modeling effect of neural topic models on attribute-rich short texts, and support the analysis of the attribute value distribution of topic questions on each attribute information.

Key point 3: A set of text topic analysis system for mobile application metadata is constructed. Realized the preprocessing of short text content in mobile applications (word segmentation, part-of-speech recognition, entity recognition, vocabulary construction, word bag model representation conversion), metadata preprocessing (conversion processing of discrete, continuous and text attributes) ), the fine-tuning of the semantic representation of the pre-training model, the extraction of topic keywords (subjects (persons, institutions, organizational entities, etc.), actions (verbs, gerunds, etc.), time, place, etc., the export of topic embedding representations, and the topic in each A series of processes such as the analysis of attribute value distribution on attributes can comprehensively and efficiently mine and analyze the keywords, embedded representations and attribute value distribution of the topics of online short texts. The keywords of the topics are used to construct user portrait tag data, Public opinion event title generation, etc., and the embedding representation of topics can be used for similarity calculation of user interest portraits, so as to better serve business applications such as recommendation, search, and intelligent dialogue.

In order to make the above-mentioned features and effects of the present invention more clear and understandable, the following specific examples are given together with the accompanying drawings for detailed description as follows.

Aiming at the problem that the bag-of-words model cannot fully express the semantics of short texts, this invention proposes to regard the fixed-dimensional text semantic vector generated by the pre-training model as the semantic representation of the document, and integrate it into the encoding and decoding process of the neural topic model. In the scene where the distribution of words in the document of the short text corpus is relatively sparse, the neural topic model overcomes the limitation of the bag of words model through the constraints of dense semantic representation including word order and context information, and makes more accurate short text topics. Ground inference and modeling, the principle is shown in Figure 1.

Aiming at the problem that the neural topic model cannot effectively use the multi-attribute information of short texts, the inventors observed that: in the real scene of online short texts, short texts discussing the same topic can be compared with publisher portrait, release time, Hashtag, URL address, user Mention, etc. The distribution of multi-attribute values is localized, that is, similar publishers often discuss the same topic in short texts with similar attribute information sent within a short period of time. Therefore, the present invention proposes to construct multi-tasks for predicting attribute values of short texts based on sparse multi-attribute information of short texts, use multi-task learning to fine-tune the pre-training model, and make the pre-training model generate text semantic vectors that are more relevant to topic information as semantic Then, through semantic constraints, the purpose of indirect fusion of short text multi-attribute information is achieved, thereby avoiding the noise interference introduced by a large number of missing attribute values. The schematic diagram is shown in Figure 2.

A topic modeling method for mobile application metadata, including the following processing flow:

1. Clean the text content, identify entity words and predicate words in the text, build a vocabulary, and convert the text content into a bag of words model representation.

2. Clean and transform the metadata of the text according to the attribute type, construct the corresponding attribute value prediction task, fine-tune the semantic representation of the pre-trained model, and integrate multi-attribute information through multi-task learning and auxiliary confrontation tasks. Using the fine-tuned pre-trained model, the concatenated text of text content and text-type attribute values is converted into a text semantic representation.

3. Construct a semantic constraint target based on the text semantic representation, guide the learning of the neural topic model, and export the modeled topic-word distribution and topic embedding representation after the training is completed. The topic-word distribution is different from the bag-of-words model representation of the above document-words. The input of the topic modeling task is a document collection, which contains multiple documents. The task of topic modeling is to find out which topics are contained in it. Topic-word distribution is to use several keywords to describe the topic. Topic embedding representation is a commonly used representation method in deep learning, which uses a real-valued vector to represent a topic.

4. Embed the topic into the attribute value prediction network corresponding to the representation input, and obtain the distribution of attribute values of the topic on each attribute. A topic is a semantic concept, which can be described by a set of keywords, by participating users, or by hashtags used by users when discussing the topic. Further analysis of the attribute distribution of the topic is to describe the topic from more dimensions, so that users can better understand the text keywords related to the topic, the main participating users, mainstream hashtags, and the time range covered by the topic, etc.

The specific process is shown in Figure 3:

1. Data acquisition: through data collection/data query and other means, collect text data and corresponding metadata published by a specific social platform within a period of time, including but not limited to: release time, release user ID, release user profile, @ Metadata such as User, #Tag, URL, etc.

2. Text preprocessing: clean the text content, identify entity words (person, institution, organization, time, place, other entities, etc.) and predicate words (event trigger words) in the text, build a vocabulary, and convert the text content into its The corresponding bag-of-words representation.

3. Metadata preprocessing: Divide metadata into three categories: discrete attributes, continuous attributes and text attributes, which are processed separately.

1) Discrete attributes: such as posting user ID, #Tag, URL, etc. For discrete attributes, based on the counts of attribute values that have appeared in the corpus, according to the process of constructing the vocabulary, the attribute values whose occurrence times exceed the set threshold are taken to form an attribute value set. In particular, for the attribute value of the URL address, the Host field in the URL address is used instead of the original URL address as the attribute value of the URL address attribute, representing the information source of the online short text.

2) Continuous attributes: such as release time, evaluation score, etc. For continuous attributes, the attribute values are standardized and transformed into distributions with a mean of 0 and a variance of 1. In particular, for time class attribute values, first convert to timestamp and then standardize.

3) Text-type attributes: such as publishing a user profile, etc. For text-type attributes, the text content is concatenated with it as the input for obtaining the text semantic representation of the pre-trained model.

4. Metadata fusion: use the preprocessed metadata attribute values as labels, construct multiple attribute value prediction tasks, and fine-tune the pre-training model through multi-task learning to fuse metadata information. Each discrete attribute corresponds to a classification task of predicting the attribute value ID, and cross-entropy is used as the loss function of the classification task. Each continuous attribute corresponds to a regression task that predicts the transformed attribute value, and MSE is used as the loss function of the regression task. In order to avoid the high coupling between the text semantic vector generated by the fine-tuned pre-training model and a certain attribute, make the shared parameters of the pre-training model more inclined to capture the co-occurrence rules between multiple attributes, and learn semantic representations that are more relevant to topic information. The invention also adds an auxiliary adversarial classification task: to determine which attribute the text semantic vector generated by the pre-training model corresponds to, and use cross-entropy as the loss function.

5. Topic modeling: The text semantic representation fused with metadata information is used as a semantic constraint to guide the training of the neural topic model, and derive the modeled topic-keyword distribution and topic embedding representation.

6. Post-processing: Embed the topic into the attribute value prediction task network in 4, get the distribution of the topic's attribute values on the corresponding attributes, and take Top K as the attribute value list of each attribute of the topic. Divide the topic-keyword distribution of the topic according to keyword categories (entity words (persons, institutions, organizations, time, places, other entities, etc.) and predicate words (event trigger words)), and take Top K as each category of the topic list of keywords. Similar topics are merged based on keyword lists, attribute value lists, and topic embedding representations.

The significance of the above merging is that there may be multiple documents describing the same topic. The purpose of topic modeling is to compress a text collection and find out the main events it describes. Therefore, the goal of topic modeling is to get a description about each topic, such as the topic-key distribution here.

A common form of topic-keyword distribution is to use the K most important keywords to describe a topic. For example, the keyword distribution under a certain geo-conflict theme can be "XX country, ZZ country, leader of XX country, head of ZZ country, third-party participating organizations, military operations, sanctions".

When merging two topics, the Jaccard coefficient is used to measure the similarity between the keyword list and the attribute value list of the topics, and the cosine similarity is used to measure the similarity between the embedding representations of the topics. After obtaining three similarity scores, a weighted average is used to obtain the final similarity between two topics. Merge topics whose similarity exceeds a specified threshold.

When inferring the topic distribution of a given text, the bag-of-words representation and semantic representation of the text can be directly input into the topic model to obtain its topic distribution, and the topic corresponding to the dimension with the largest dimension value in the topic distribution can be used as the topic of the text theme.

The following are system embodiments corresponding to the foregoing method embodiments, and this implementation manner may be implemented in cooperation with the foregoing implementation manners. The relevant technical details mentioned in the foregoing implementation manners are still valid in this implementation manner, and will not be repeated here in order to reduce repetition. Correspondingly, the relevant technical details mentioned in this implementation manner may also be applied in the foregoing implementation manners.

A text theme analysis system for mobile application metadata, its business logic is shown in Figure 4, at least including the following modules:

1. Preprocessing module: mainly responsible for reading the text data to be analyzed from the file/database/data warehouse, and performing the following preprocessing operations: (1) cleaning the text content; (2) identifying entity words and predicate words in the text , build a vocabulary; (3) extract the attribute values of various attributes in the text, and perform cleaning and conversion according to attribute types.

2. Topic modeling learning module: It is mainly responsible for training the neural topic model, and outputting the keywords, embedding representation and attribute value distribution of the topic it has modeled. Including bag-of-words model representation for topic modeling, conversion and loading of pre-trained semantic representations, training neural topic models, exporting keywords of modeled topics, embedding representations and their attribute value distributions, and inferring topic distributions of texts.

3. Post-processing module: It is mainly responsible for processing and converting the keywords, embedded representations and attribute value distribution of the exported topics. It includes merging similar topics, extracting topic titles and description information, assigning cluster labels for each text, etc.

4. Data storage module: mainly responsible for saving the output results in the database. Including creating topics, keywords, and text objects, and handling the foreign key relationship between them, connecting to the database and writing the data into the corresponding data table.

5. Metadata fusion module: It is mainly responsible for fine-tuning the pre-training model so that it can generate semantic representations that are more relevant to topic information. Including conversion and loading of text data and multi-attribute data for fine-tuning tasks, fine-tuning pre-training models, exporting and saving fine-tuned pre-training model parameters.

The present invention also proposes a text topic modeling system based on social platform metadata, which includes:

The initial module is used to obtain the text data to be modeled and the metadata of the text data from the social platform; and based on the keywords of the text data, construct the word bag representation of the text data;

The fine-tuning module is used to train the attribute value prediction task of the corresponding category according to the category of the metadata, so as to fine-tune the pre-trained semantic extraction model to obtain the target semantic extraction model, and use the target semantic extraction model to extract the text semantic representation of the text data;

The extraction module is used to construct a semantically constrained target based on the semantic representation of the text, guided by the semantically constrained target, taking the bag-of-words representation as input and reconstructing the target, training a neural topic model based on a variational autoencoder, and obtaining a topic extraction model , and derive topic-keyword distribution and topic embedding representations from the model;

The merging module is used to input the topic embedding representation into the attribute value prediction task, obtain the attribute value distribution of the topic on the corresponding attribute, and conduct the same topic according to the attribute value distribution, the topic-keyword distribution and the topic embedding representation Merge and use the merged result as the topic model of the text data.

The text topic modeling system based on social platform metadata, wherein the fine-tuning module is specifically used for:

Classify the attributes of metadata into discrete attributes, continuous attributes and text attributes;

For discrete attributes, based on the counting of attribute values that have appeared in the corpus, according to the process of constructing the vocabulary, the attribute values whose occurrence times exceed the preset threshold are taken to form an attribute value set, and a classification of predicted attribute values is constructed based on the attribute value set task, using cross-entropy as the loss function of the classification task;

For a continuous attribute, convert its attribute value to a distribution with a mean value of 0 and a variance of 1; construct a regression task to predict the converted attribute value based on the continuous attribute, and use MSE as the loss function of the regression task;

For text-type attributes, the text data is spliced with it to obtain the spliced text, which is input into the pre-trained semantic extraction model to generate a text semantic vector;

Build an adversarial classification task to determine the attribute category of the text semantic vector, using cross-entropy as the loss function.

The text topic modeling system based on social platform metadata, wherein the merging module is used to: according to the attribute value distribution, construct the attribute value list of each attribute of the topic; according to the topic-keyword distribution, construct the keyword list;

When merging topics, use the Jaccard coefficient distribution to measure the similarity between the keyword lists and attribute value lists of the topics, get the first similarity and the second similarity, and use the cosine similarity to measure the embedding representation of the topics The similarity between the subjects is obtained to obtain the third similarity; the weighted average of the first similarity, the second similarity and the third similarity is obtained to obtain the final similarity between the topics, and the topics whose final similarity is greater than the preset value are merged .

The text topic modeling system based on social platform metadata, wherein the metadata includes: publishing time, publishing user ID, publishing user's profile, @User, #Tag and URL.

The present invention also provides a storage medium for storing a program for executing any one of the social platform metadata-based text topic modeling methods.

The present invention also proposes a client for any of the text topic modeling systems based on social platform metadata.

Claims (10)

1. A text topic modeling method based on social platform metadata is characterized by comprising the following steps:

step 1, obtaining text data to be subject modeled and metadata of the text data from a social platform;

step 2, constructing a bag-of-words representation of the text data based on the keywords of the text data;

step 3, training attribute value prediction tasks of corresponding categories based on the categories of the metadata to finely adjust a pre-training semantic extraction model to obtain a target semantic extraction model, and extracting text semantic representation of the text data by using the target semantic extraction model;

step 4, constructing a semantic constraint target based on the text semantic representation, training a neural topic model based on a variational self-encoder by taking the semantic constraint target as a guide and taking bag-of-words representation as an input and reconstruction target to obtain a topic extraction model, and deriving topic-keyword distribution and topic embedding representation from the model;

and 5, inputting the theme embedded representation into the attribute value prediction task to obtain attribute value distribution of the theme on the corresponding attribute, merging the same theme according to the attribute value distribution, the theme-keyword distribution and the theme embedded representation, and taking a merging result as a theme model of the text data.

2. The method of claim 1, wherein the step 3 comprises:

classifying attributes of the metadata into a discrete type attribute, a continuous type attribute and a text type attribute;

counting discrete attributes respectively based on attribute values appearing in a corpus set, taking out attribute values with the current number exceeding a preset threshold value to form an attribute value set according to the process of constructing a word list, constructing a classification task for predicting the attribute values based on the attribute value set, and adopting cross entropy as a loss function of the classification task;

for the continuous attribute, converting the attribute value into the distribution with the mean value of 0 and the variance of 1; constructing a regression task for predicting the converted attribute values based on the continuous attributes, and adopting MSE as a loss function of the regression task;

splicing the text data with the text type attribute to obtain a spliced text, inputting the pre-training semantic extraction model, and generating a text semantic vector;

and constructing a confrontation classification task for judging the attribute category of the text semantic vector, and taking the cross entropy as a loss function.

3. The method of claim 1, wherein the step 5 comprises: constructing an attribute value list of each attribute of the theme according to the attribute value distribution; constructing a keyword list according to the topic-keyword distribution;

when the topics are combined, the Jacard coefficient distribution is used for measuring the similarity between the keyword lists and the attribute value lists of the topics to obtain a first similarity and a second similarity, and the cosine similarity is used for measuring the similarity between the embedded representations of the topics to obtain a third similarity; and weighting and averaging the first similarity, the second similarity and the third similarity to obtain the final similarity among the topics, and combining the topics with the final similarity larger than a preset value.

4. The social platform metadata based text topic modeling method of claim 1, wherein the metadata comprises: time of publication, user ID of publication, user profile of publication, @ User, # Tag, and URL.

5. A text topic modeling system based on social platform metadata, comprising:

the system comprises an initial module, a topic modeling module and a topic modeling module, wherein the initial module is used for acquiring text data to be subject modeled and metadata of the text data from a social platform; constructing a bag-of-words representation of the text data based on the keywords of the text data;

the fine tuning module is used for training attribute value prediction tasks of corresponding categories according to the categories of the metadata so as to fine tune a pre-training semantic extraction model to obtain a target semantic extraction model, and extracting text semantic representation of the text data by using the target semantic extraction model;

the extraction module is used for constructing a semantic constraint target according to the text semantic representation, taking the semantic constraint target as guidance, taking bag-of-words representation as an input and reconstruction target, training a neural topic model based on a variational self-encoder to obtain a topic extraction model, and deriving topic-keyword distribution and topic embedded representation from the model;

and the merging module is used for inputting the theme embedded representation into the attribute value prediction task to obtain the attribute value distribution of the theme on the corresponding attribute, merging the same theme according to the attribute value distribution, the theme-keyword distribution and the theme embedded representation, and taking a merging result as a theme model of the text data.

6. The social platform metadata based text topic modeling system of claim 5 wherein the hinting module is specifically configured to:

classifying attributes of the metadata into a discrete type attribute, a continuous type attribute and a text type attribute;

counting discrete attributes respectively based on attribute values appearing in a corpus set, taking out attribute values with the current number exceeding a preset threshold value to form an attribute value set according to the process of constructing a word list, constructing a classification task for predicting the attribute values based on the attribute value set, and adopting cross entropy as a loss function of the classification task;

for the continuous attribute, converting the attribute value into the distribution with the mean value of 0 and the variance of 1; constructing a regression task for predicting the converted attribute values based on the continuous attributes, and adopting MSE as a loss function of the regression task;

splicing the text data with the text type attribute to obtain a spliced text, inputting the pre-training semantic extraction model, and generating a text semantic vector;

and constructing a confrontation classification task for judging the attribute category of the text semantic vector, and taking the cross entropy as a loss function.

7. The social platform metadata based text topic modeling system of claim 5 wherein the merging module is to: constructing an attribute value list of each attribute of the theme according to the attribute value distribution; constructing a keyword list according to the topic-keyword distribution;

when the topics are combined, the Jacard coefficient distribution is used for measuring the similarity between the keyword lists and the attribute value lists of the topics to obtain a first similarity and a second similarity, and the cosine similarity is used for measuring the similarity between the embedded representations of the topics to obtain a third similarity; and weighting and averaging the first similarity, the second similarity and the third similarity to obtain the final similarity among the topics, and combining the topics with the final similarity larger than a preset value.

8. The social platform metadata based text topic modeling system of claim 5 wherein the metadata comprises: time of publication, user ID of publication, user profile of publication, @ User, # Tag, and URL.

9. A storage medium storing a program for executing the method for modeling a text topic based on metadata of a social platform according to any one of claims 1 to 4.

10. A client for use in the social platform metadata based text topic modeling system of any one of claims 5 to 8.

Images