CN104516947A - Chinese microblog emotion analysis method fused with dominant and recessive characters - Google Patents

Abstract

A Chinese microblog sentiment analysis method that combines dominant and recessive features, comprising the following steps: 1) microblog dominant feature processing, 1.1) emoticon processing; 1.2) emotional word processing; 2) microblog recessive feature processing : Create initial sentiment clusters based on frequent itemsets. Each initial sentiment cluster text contains frequent itemsets. Using HowNet’s Chinese semantic similarity model, each initial sentiment cluster is separated according to the principle of maximum semantic membership; finally, by defining the inter-cluster The semantic similarity matrix completes the cohesive hierarchical clustering of microblog sentiment clusters, and optimizes the final sentiment clusters to realize microblog sentiment analysis. The invention provides a Chinese microblog emotion analysis method with high flexibility and good reliability which integrates dominant and recessive features.

Description

A Sentiment Analysis Method of Chinese Microblogs Combining Dominant and Implicit Features

technical field

The invention relates to the technical field of Internet public opinion content analysis, in particular to a text microblog sentiment analysis method.

Background technique

Sentiment analysis (Sentiment analysis) is the process of analyzing, processing, summarizing and inferring subjective text with emotional color, the purpose is to extract the user's point of view from the text information with subjective emotional color published by the user, and judge its emotion polarity.

Due to the complexity of human emotions, there is no uniform standard for the division of emotion categories. Common methods, for example, divide the emotion classification task into two types: binary classification of subjective and objective information, and emotional classification of subjective information, including the most common binary classification of praise and criticism and more detailed multivariate classification. For multivariate classification, some studies have proposed four types of emotions: anger, disgusting, happy, sad, or seven types of emotions: anger, disgust, fear, happiness, like, sadness, surprise, etc. .

For emotion monitoring methods, foreign methods have proposed a distance supervised learning method for emotional classification of messages in Twitter, that is, given a search term, messages are automatically classified into positive or negative information, and messages containing emoticons in Twitter are extracted as training sets , and finally use algorithms such as naive Bayesian, maximum entropy, and support vector machines to classify; for Chinese microblogs, a multi-strategy method based on hierarchical structure is proposed to carry out emotion monitoring research on Sina microblog data, and when feature extraction Using topic-related features, the experimental results show that the highest accuracy rate obtained after using topic-related features is increased from 66.467% to 67.283%, but the analysis process of this method is cumbersome.

Weibo has the characteristics of originality and unpredictability. The number of words in a single Weibo is less than 140. It combines the dominant features such as Internet language and emoticons and the recessive features of Weibo semantic emotion. This brings new insights into Weibo sentiment analysis. challenge. There are widely homophonic words and abbreviations in Weibo, such as "porridge" means "like", "cupware" means "tragedy", etc., and these words change with time, and new words appear constantly, it is necessary to establish a specific A dictionary of online terms; microblog emoticons can usually directly express emotions, but there are various emoticons, and a specific emoticon emotion classification needs to be established; in addition, a microblog may contain multiple different emotions, and sentiment analysis is generally based on the blogger’s main Emotion prevails. In the prior art, it is impossible to analyze the emotion of Chinese Weibo.

Contents of the invention

In order to overcome the disadvantage of being unable to analyze Chinese microblog sentiment in the prior art, the present invention provides a Chinese microblog sentiment analysis method with high flexibility and good reliability which integrates dominant and recessive features.

The technical solution adopted by the present invention to solve its technical problems is:

A Chinese microblog sentiment analysis method combining dominant and recessive features, said Chinese microblog sentiment analysis method comprising the following steps:

1) Microblog dominant feature processing, specifically including the following processes:

1.1) Emoticon processing: construct an emotional symbol library based on the emoticons that come with Weibo, and divide emotions into seven categories: happiness, liking, anger, sadness, fear, disgust, and surprise according to the seven types of emotion classification methods, and rank them by frequency For the first 150 emoticons, perform unified processing, that is, create an emoticon table first, put 150 emoticons into the emoticon table, and judge whether the emoticon belongs to the emoticon table by looking up the table, and if so, extract the emoticon, Write into the emotional feature table after converting into an emotional category;

1.2) Emotional word processing: first create an emotional word list based on the emotional dictionary, put the emotional words in Weibo into the word list, and judge whether it is an emotional word after text segmentation by looking up the table, and if so, extract the emotional word , and write into the emotional feature table;

Then build an emotional vocabulary based on online vocabulary, put the online vocabulary in Weibo into the vocabulary, and determine the emotional category of some Weibo content by looking up the table;

2) Microblog recessive feature processing: create initial sentiment clusters based on frequent itemsets, each initial sentiment cluster text contains frequent itemsets, use HowNet’s Chinese semantic similarity model, and separate each initial sentiment according to the principle of maximum semantic membership Clusters; finally, by defining the semantic similarity matrix between clusters, the agglomerative hierarchical clustering of microblog sentiment clusters is completed, and the final sentiment clusters are optimized to realize microblog sentiment analysis.

Further, the step 2) includes the following processes:

2.1) Use the frequent set mining algorithm Apriori to calculate and mine frequent word sets

Use the frequent itemset division to construct the initial sentiment cluster, divide the microblog containing the frequent trend word set into a cluster, and obtain the initial sentiment cluster based on the frequent itemset, and at the same time, use the frequent itemset describing the initial sentiment cluster as the temporary identifier of the corresponding sentiment cluster, By extracting the frequent itemsets of each initial emotional cluster to represent the emotional semantics of this initial emotional cluster;

2.2) Initial cluster overlap reduction of microblog semantic membership degree

Assign each microblog to an emotional cluster, calculate the emotional semantic membership degree of the overlapping part between the clusters to the initial emotional cluster, and finally assign clusters according to the principle of maximum semantic membership degree; then delete those empty clusters whose size is 0 after the initial cluster separation , the initial cluster after overlap reduction is called the candidate sentiment cluster;

2.3) Cohesive sentiment clustering based on semantic similarity: perform cohesive hierarchical clustering on candidate sentiment clusters, and merge sentiment clusters.

Further, in the step 2.1),

Definition 1: For an item set X in database E, if the number of occurrences of item set X in database E is greater than the preset ratio, X is said to be a frequent itemset of database E, and this preset ratio is called the minimum support;

If the text is regarded as a transaction, and the vocabulary of the text corresponds to the items in the transaction, then the text d can be expressed as: d=<t ₁ , t ₂ ,...,t _n >, where n represents the characteristic vocabulary contained in the text d quantity;

Definition 2: For a word set W of text set D, if the support degree of W in D is s(W)≥min_s, then the potential set W is called the frequent word set of text set D, and min_s is the global minimum support degree;

Scan the text set D, use the word frequency trend to count the number of occurrences of candidate item sets, collect itemsets that meet the minimum support min_s setting, and record them as frequent itemsets; use the generated frequent k-itemsets to construct strong association rules, use frequent The k-itemset constructs the candidate (k+1)-itemset, and iterates repeatedly until the candidate (k+1)-itemset is empty.

Further, in the step 2.2),

Definition 3: If the microblog doc _j is assigned to the initial emotional cluster C _i , it is said that the microblog doc _j supports the cluster C _i ;

Definition 4: Note that D _i and D _j are microblog sets that support clusters C _i and C _j , and D _i ∩ _{D j} ≠ 0, then there is inter-cluster overlap between clusters C _i and C _j ;

Definition 5: Microblog emotional semantic membership degree, the present invention defines the emotional semantic membership function of microblog doc _j to cluster C _i as follows: $\mathrm{Score}(C_i\&LeftArrow;{\mathrm{doc}}_j)=\frac{{\mathrm{\&Sigma;}}_{l=1}^{\mathrm{no}}\max k=\mathrm{1,2},...,m\left\{\mathrm{sim}(f_{\mathrm{ik}},t_{\mathrm{jl}})\right\}}{\mathrm{no}};$ Among them, the cluster frequent 1-itemset {f _i1 , f _i2 ,..., fi _im } represents the emotional feature item of the initial cluster C _i , {t _j1 , t _j2 ,..., t _jn } represents the initial cluster C The feature item of the microblog text doc _j in _i ; sim(fi _ik , t _jl ) is the semantic similarity between the cluster feature item f _jk and the text feature item t _jl defined in HowNet, and n is the microblog text doc _j The number of feature items, m is the number of cluster feature items.

Still further, in the step 2.3),

Definition 6: Cluster eigenvector. For the candidate emotional cluster CT _i , the cluster frequent 1-itemset of CT _i is mined, that is, the cluster eigenvector that constitutes the cluster, denoted as

Definition 7: Cluster similarity matrix, remember the cluster feature vectors of two different candidate emotional clusters CT _i and CT _j are: and Where n and m sub-tables represent the number of feature words, then the cluster semantic similarity matrix composed of feature items of CT _i and CT _j is defined in the manner of Table 1;

Table 1

Definition 8: Semantic similarity of emotional clusters, select k feature item pairs with the largest semantic similarity in the similarity matrix to calculate the similarity between candidate emotions, denoted as {sim(t _i t _j ) ₁ , sim(t _i t _j ) ₂ ,..., sim(t _i t _j ) _k }, the semantic similarity of candidate emotion clusters is defined as:

$\mathrm{<span\; class="notranslate">\; sim</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; CT</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; CT</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}\mathrm{<span\; class="notranslate">\; sim</span>}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span>}_{\mathrm{<span\; class="notranslate">\; l</span>}}}{\mathrm{<span\; class="notranslate">\; k</span>}}$

The process of agglomerative sentiment clustering based on semantic similarity is as follows:

Step 1: Extract the feature vectors of each candidate emotional cluster, and calculate the semantic similarity of the candidate emotional clusters;

Step 2: Construct the semantic similarity matrix of candidate emotional clusters, which can be known from the definition of cluster similarity

sim(CT _i , CT _j )=sim(CT _j , CT _i ), that is, the similarity matrix is a symmetric matrix;

Step 3: Select the largest inter-cluster similarity from the similarity matrix, denoted as

max{sim(CT _i , CT _j )},

If max{sim(CT _i , CT _j )}≤λ, execute Step 6; otherwise, execute Step 4;

Step 4: If max{sim(CT _i , CT _j )}>λ, the similarity between CT _i and CT _j is relatively large, so the two clusters CT _i and CT _j are merged to form a new cluster CT _i ′, delete the original CT _i , and recalculate the cluster feature vector, and update the semantic similarity matrix;

Step 5: If the number of rows or columns of the inter-cluster semantic similarity matrix is less than or equal to the preset minimum number of clusters μ, execute Step 6; otherwise, the clustering has not yet ended, and return to Step 3;

Step 6: The agglomerative hierarchical clustering ends, and the emotional cluster CT′ is obtained.

In the step 1.2), collect the set of negative words, analyze whether there are negative words before the emotional vocabulary, and if so, write the negative words and emotional words into the emotional feature table.

The technical concept of the present invention is: the present invention will be based on emoticons, combined with the Chinese ontology resources marked by the Information Retrieval Research Office of Dalian University of Technology and the sentiment analysis vocabulary set provided by HowNet (both are public resource libraries) to construct Emoji library, emotional word dictionary and Internet term dictionary; extract explicit emotional features from them, and integrate implicit semantic features, and use clustering based on the similarity of similar emotional microblog texts and the small similarity of different emotional microblog texts Thoughts for sentiment analysis. Clustering does not require a training process and manual labeling of documents in advance, and is directly based on frequent itemsets and semantic clustering algorithms, which has good flexibility and automatic processing capabilities.

The beneficial effects of the invention are mainly manifested in: higher flexibility and better reliability.

Description of drawings

Figure 1 is a graph of the proportion of sampled microblogs containing different numbers of emoticons.

Figure 2 is a flowchart of a microblog sentiment classification method combining frequent itemsets and semantic clustering.

Figure 3 is a schematic diagram of the emotional change trend of the "Malaysia Airlines" incident.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings.

Referring to Figures 1 to 3, a Chinese microblog sentiment analysis method that integrates explicit and implicit features, microblog emoticons are an intuitive and explicit emotional feature, while content semantics are implicit, and have Sentiment judgment plays a decisive role, so the present invention proposes a microblog sentiment analysis method that combines two characteristic factors. Firstly, the sentiment analysis dictionary, the network term dictionary and the emoticon library are constructed, and the frequent feature word set of Weibo is defined. According to the frequent feature word set, the initial emotion cluster of Weibo is obtained by using the maximum frequent itemset; in view of the text overlap between the initial clusters, a proposed Based on the inter-cluster overlap reduction algorithm of extended semantic membership of short text, completely separated initial clusters are obtained; according to the cluster semantic similarity matrix, an agglomerative emotional clustering method is given.

Chinese microblog emotion analysis method of the present invention comprises following three steps:

1), Weibo dominant feature processing

1.1) Emoji processing

The emoticons on English Weibo are usually input by users themselves, such as ":)"; the emoticons provided by the Sina Weibo platform are expressed by text enclosed in square brackets, such as expression The corresponding text is "[呵呵]". Emoticons are widely used in microblogs. For example, 5000 Sina Weibo posts were randomly selected, and the number of microblogs containing emoticons was 1071, accounting for 21.24%. A single microblog may contain multiple emoticons. Figure 1 shows the sampling statistics of the number of microblogs containing different numbers of emoticons. The results show that the proportion of Sina Weibo users who use one emoticon is about 62%, and those who use 2- The proportion of 5 emoji is about 30%, indicating that Weibo users are more willing to use a single emoji.

The present invention adopts the emoticons that come with Sina Weibo to construct an emotional symbol library, and divides emotions into seven categories of happiness, liking, anger, sadness, fear, disgust, and surprise according to the seven types of emotion classification methods. The top 150 emoticons with the highest frequency of occurrence are unified, that is, the emotional symbol table is established first, and 150 emoticons are put into the emotional symbol table, as shown in Table 2. Determine whether the emotional symbol belongs to the emotional symbol table by means of table lookup, if so, extract the emotional symbol, convert it into an emotional category, and write it into the emotional feature table. Experiments show that the unified processing of emoji is beneficial to produce better clustering effect, so as to achieve more accurate sentiment analysis.

Table 2 Emotion categories and typical emoticons for each category

1.2) Emotional word processing

Emotional words can best reflect the text emotion of Weibo, so the construction of emotional dictionary and network vocabulary dictionary is the basic work of determining the emotional orientation of Weibo.

Chinese emotional lexicon classification: Emotional vocabulary is complex and has many parts of speech, including adjectives, nouns, adverbs, etc. It is not scientific to choose emotional words only considering the part of speech. Most nouns do not have emotional color, and the selection will reduce the classification performance. The present invention adopts the Chinese ontology resource provided by the Information Retrieval Laboratory of Dalian University of Technology, which contains 27467 Chinese emotional words. As shown in Table 3, first establish an emotional word list of an emotional dictionary, and put these emotional words into the word list. Check the table to determine whether the text is an emotional word after word segmentation, and if so, extract the emotional word and write it into the emotional feature table.

Table 3 The Chinese Ontology Resource Sentiment Classification Table selected by the present invention

In addition, the negative word sets in Weibo such as "no", "no", "impossible" and "difficult" are also collected, and whether there are negative words before the emotional words are analyzed, and if there are, the negative words are combined with the emotional words Write it into the emotion feature table together.

Internet vocabulary dictionary construction: Microblog emotions are often original. With the development of the Internet, new words appear continuously, including homophonic words, abbreviated words, Internet languages, etc. Therefore, the present invention constructs an online vocabulary dictionary for the emotional tendency of microblog emotions determination. Through the collection and arrangement of social networks, a total of 141 Internet words were used, which were respectively labeled with emotions and unified. That is, firstly, an emotional word list of Internet words was established, and these Internet words were put into the word list. Many online words have ambiguous emotional tendencies in the context of no context, and the text only retains online words with obvious emotions. Some online words and their emotional tendencies are marked in Table 4. Similarly, based on the network vocabulary dictionary, the emotional category of some microblog content can be directly determined by looking up the table.

Table 4 Examples of emotion labeling of some network words and their tendencies

2), Weibo recessive feature processing

FIHC (Frequent Itemset-based Hierarchical Clustering) is a text clustering algorithm widely used in the industry at present. The algorithm is centered on clustering clusters, and directly uses frequent itemsets to measure the degree of aggregation between clusters, and believes that: documents belonging to the same relationship share more frequent itemsets, and documents belonging to different relationships share less frequent itemsets , use the concept of frequent itemsets to divide the text.

Both the speech and semantics of Weibo content can be regarded as the implicit emotional characteristics of Weibo. The present invention adopts the idea of "creating clusters first, then deduplication and then agglomeration" of the FIHC algorithm, and proposes a new method combining frequent itemsets and semantic clustering. The main process of clustering is shown in Figure 2.

The main process of sentiment classification is as follows: First, create initial sentiment clusters based on frequent itemsets, and each initial sentiment cluster text contains frequent itemsets, which leads to overlapping texts between initial sentiment clusters; in order to more accurately eliminate text overlap between initial sentiment clusters , using the Chinese semantic similarity model of HowNet, and separating each initial emotional cluster according to the principle of maximum semantic membership; finally, by defining the semantic similarity matrix between clusters, complete the cohesive hierarchical clustering of microblog emotional clusters, and optimize to obtain the final Sentiment clusters to realize microblog sentiment analysis.

2.1) Obtain frequent itemset method

Definition 1: For an itemset X in the database E, if the number of occurrences of the itemset X in the database E is greater than the preset ratio, then X is said to be a frequent itemset of the database E, and this preset ratio is called the minimum support.

If the text is regarded as a transaction, and the vocabulary of the text corresponds to the items in the transaction, then the text d can be expressed as: d=<t ₁ , t ₂ ,...,t _n >

Definition 2: For a word set W of text set D, if the support degree of W in D is s(W)≥min_s, then the potential set W is called a frequent word set of text set D, and min_s is the global minimum support degree.

The invention adopts frequent set mining algorithm Apriori to calculate and mine frequent word sets.

Algorithm: Apriori Algorithm

Input: Weibo data, minimum cluster support min_s

Output: frequent itemsets in Weibo data

method:

The first step is to scan the text set D, use the word frequency trend to count the number of occurrences of candidate item sets, collect itemsets that meet the minimum support min_s setting, and record them as frequent itemsets;

The second step is to use the generated frequent k-itemsets to construct strong association rules, use frequent k-itemsets to construct candidate (k+1)-itemsets, and iterate repeatedly until the candidate (k+1)-itemsets are empty. the

Frequent itemsets describe sentiment information in microblogs. The present invention constructs initial emotion clusters by dividing frequent itemsets, divides microblogs containing frequent trend word sets into one cluster, and obtains initial emotion clusters based on frequent itemsets. At the same time, the frequent itemsets describing the initial emotion clusters are used as temporary Identification, by extracting the frequent itemsets of each initial emotional cluster to represent the emotional semantics of this initial emotional cluster.

2.2) The initial cluster overlap and reduction of the semantic membership degree of microblog: the text expression of microblog is concise and random, the same emotional microblog has different expressions, and a microblog may contain multiple different emotions, resulting in a large number of initial emotional clusters. The text overlaps, and the sentiment analysis should be based on the main sentiment of the blogger, and each microblog needs to be assigned to an sentiment cluster.

Starting from the semantic level, the present invention introduces the extended semantic information of the "HowNet" semantic database, calculates the emotional semantic membership of the overlapping parts between the clusters to the initial emotional clusters, and finally allocates the clusters according to the principle of the maximum semantic membership.

Definition 3: If the microblog doc _j is assigned to the initial sentiment cluster C _i , then the microblog doc _j is said to support the cluster C _i .

Definition 4: Note that D _i and D _j are microblog collections supporting clusters C _i and C _j , and D _i ∩ _{D j} ≠ 0, then there is inter-cluster overlap between clusters C _i and C _j .

Definition 5: Microblog emotional semantic membership degree, the present invention defines the emotional semantic membership function of microblog doc _j to cluster C _i as follows: $\mathrm{Score}(C_i\&LeftArrow;{\mathrm{doc}}_j)=\frac{{\mathrm{\&Sigma;}}_{l=1}^{\mathrm{no}}\max k=\mathrm{1,2},...,m\left\{\mathrm{sim}(f_{\mathrm{ik}},t_{\mathrm{jl}})\right\}}{\mathrm{no}}.$

Among them, the cluster frequent 1-itemset {f _i1 , f _i2 ,..., fi _im } represents the emotional feature item of the initial cluster C _i , {t _j1 , t _j2 ,..., t _jn } represents the initial cluster C The feature item of the microblog text doc _j in _i ; sim(fi _ik , t _jl ) is the semantic similarity between the cluster feature item f _jk and the text feature item t _jl defined in HowNet, and n is the microblog text doc _j The number of feature items, m is the number of cluster feature items.

Algorithm: Microblog Semantic Membership Initial Cluster Overlap Subtraction Algorithm

Input: Initial clusters C ₁ , C ₂ , ..., C _n with overlaps

Output: initial clusters C′ ₁ , C′ ₂ , ..., C′ _n after overlap reduction

method:

After doc _j performs initial cluster overlap reduction, delete those empty clusters whose size is 0 after the initial cluster separation, and finally get the final candidate emotional clusters.

2.3) Agglomerative sentiment clustering based on semantic similarity

Candidate emotion clusters for microblog clustering emotion detection can be obtained by overlapping subtraction among initial emotion clusters, but these emotion clusters can all belong to a certain large emotion, so it is necessary to perform agglomerative hierarchical clustering on candidate emotion clusters and merge emotion cluster.

Definition 6: Cluster eigenvectors. For the candidate emotion cluster CT _i , the cluster frequent 1-itemset of CT _i is mined, that is, the cluster feature vector that constitutes the cluster, denoted as

Definition 7: Cluster similarity matrix. Note that the cluster feature vectors of two different candidate emotion clusters CT _i and CT _j are: and Then the cluster semantic similarity matrix formed by the feature items of CT _i and CT _j is defined in the manner of Table 5.

Table 5 Cluster Semantic Similarity Matrix Definition Table

Definition 8: Semantic similarity of emotion clusters. In order to avoid the noise of too many non-key feature word pairs in the semantic similarity between clusters, only the k feature item pairs with the largest semantic similarity in the similarity matrix are selected to calculate the similarity between candidate emotions, which is denoted as {sim(t _i t _j ) ₁ , sim(t _i t _j ) ₂ ,..., sim(t _i t _j ) _k }, the semantic similarity of candidate emotion clusters is defined as:

$\mathrm{<span\; class="notranslate">\; sim</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; CT</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; CT</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}\mathrm{<span\; class="notranslate">\; sim</span>}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span>}_{\mathrm{<span\; class="notranslate">\; l</span>}}}{\mathrm{<span\; class="notranslate">\; k</span>}}$

Algorithm: Hierarchical Clustering of Candidate Sentiment Clusters

Input: Candidate emotion cluster CT{CT ₁ , CT ₂ ..CT _i },

λ (minimum threshold for merging two clusters), μ (minimum number of clusters)

Output: emotion cluster CT′

Step 1: Extract the feature vectors of each candidate emotional cluster, and calculate the semantic similarity of the candidate emotional clusters.

Step 2: Construct the semantic similarity matrix of candidate emotional clusters, which can be known from the definition of cluster similarity

sim(CT _i , CT _j )=sim(CT _j , CT _i ), that is, the similarity matrix is a symmetric matrix.

Step 3: Select the largest inter-cluster similarity from the similarity matrix, denoted as

max{sim(CT _i , CT _j )},

If max{sim(CT _i , CT _j )}≤λ, execute Step 6; otherwise, execute Step 4.

Step 4: If max{sim(CT _i , CT _j )}>λ, the similarity between CT _i and CT _j is relatively large, so the two clusters CT _i and CT _j are merged to form a new cluster CT _i ′, delete the original CT _i , and recalculate the cluster feature vector, and update the semantic similarity matrix.

Step 5: If the number of rows or columns of the inter-cluster semantic similarity matrix is less than or equal to the preset minimum number of clusters μ, execute Step 6; otherwise, the clustering has not yet ended, and return to Step 3.

Step 6: The agglomerative hierarchical clustering ends, and the emotional cluster CT′ is obtained.

In the present embodiment, the microblog emoticon set and emotional vocabulary set etc. are subjected to unified feature processing, so that the emotional word set obtained by selecting can not only effectively reduce the text feature dimension, but also retain the explicit emotional information in the original microblog collection.

Using the maximum frequent itemset clustering to obtain the initial cluster of explicit emotion, and then calculate the semantic similarity of microblog after expanding the semantic information implied by the short text through the "HowNet" semantic database, and propose an initial cluster overlap based on the division of semantic membership reduction method.

By defining the semantic similarity between the initial clusters, an agglomerative hierarchical clustering method for microblog sentiment is given. The clustering parameters can be adjusted to obtain the best microblog sentiment classification, and finally achieve accurate sentiment based on the sentiment classification results. analyze.

All the algorithms and implementation steps involved in the microblog emotion analysis method disclosed by the present invention have sufficient theoretical basis, detailed implementation steps, and accurate analysis results, and can be widely used in public opinion monitoring on social networks and the like.

Example: In order to confirm the detection effect of the method proposed in the present invention on microblog for the detection effect of a certain event sentiment analysis, the present invention obtains from March 8, 2014 to May 12, 2014 by keyword search on Sina Weibo Square. There are 44,524 microblog data about the "Malaysia Airlines Incident", and the emotional changes of the "Malaysia Airlines" incident are shown in Figure 3 below.

Combining the emotional change trend of the "Malaysia Airlines" incident in Figure 3 and the actual development of the "Malaysia Airlines" incident, the following is an analysis of several key time points:

On March 8, the official website of Malaysia Airlines released the first statement: confirming that flight MH370 lost contact with the control tower at 2:40 Beijing time on the 8th. The emotions on Weibo are "sadness", "surprise" and "fear", expressing the public's worry about the passengers in distress, shock and fear of the aviation safety, while the emotions of "happiness" and "like" are at a relatively low level.

On March 9, the Malay Minister of Transport confirmed that the ticket numbers of the two false passport holders were connected. Since there has been no news of the missing plane for more than 40 hours, people's "sadness" has risen significantly, and there have been incidents of false passports, and "fear" and "disgust" have also risen.

On March 10, Malaysian officials admitted that the missing flight might have been hijacked. People's "sadness" continues, and people's "fear" continues to rise due to "hijacking" and suspected terrorist attacks.

On March 12, Malaysia was questioned whether it deliberately concealed information or delayed the search and rescue process. Therefore, the emotion of "anger" rose sharply, and accounted for 58% of the microblog volume that day.

On March 24, Prime Minister Ma held a press conference. The Malaysia Airlines MH370, which had been missing for many days, crashed into the southern Indian Ocean. No one on board survived. The emotion of "sorrow" reached the highest level, and the people expressed their deep sorrow for the bad news.

With the passage of time, the entire Malaysia Airlines incident has entered the later stage of reflection and processing, and the focus of public attention has gradually shifted.

Claims (6)

1. merge a Chinese microblog emotional analytical approach that is dominant and recessive character, it is characterized in that: described Chinese microblog emotional analytical approach comprises the following steps:

1) microblogging dominant character process, specifically comprises following process:

1.1) emoticon process: build emotional symbol storehouse according to the expression that microblogging carries, according to 7 class sensibility classification methods, emotion is divided into happiness, hobby, indignation, sadness, fear, detest, surprised seven classifications, the frequency of occurrences is come the emoticon of front 150, do unitized process, namely emotional symbol table is first set up, 150 emoticons are put into emotional symbol table, judge whether this emotional symbol belongs to emotional symbol table by lookup table mode, if then extract emotional symbol, after converting emotion classification to, write affective characteristics table;

1.2) these emotion word are put into vocabulary by emotion word process: the emotion vocabulary setting up a sentiment dictionary, judge, by whether being emotion word after text participle, if then extract emotion word, and to write affective characteristics table by the mode of tabling look-up;

First set up the emotion vocabulary of a network words, these network words are put into vocabulary, by the emotion classification of lookup table mode judging section content of microblog;

2) microblogging recessive character process: create initial emotion bunch based on frequent item set, each initial emotion bunch text, containing frequent item set, adopts the Chinese semantic similarity model knowing net, is separated each initial emotion bunch according to maximum simple semantic degree principle; Finally, by semantic similarity matrix between definition bunch, complete the Agglomerative hierarchical clustering of microblog emotional bunch, and optimize and obtain final emotion bunch, realize microblog emotional analysis.

2. a kind of dominant and Chinese microblog emotional analytical approach that is recessive character of merging as claimed in claim 1, is characterized in that: described step 2) comprise following process:

2.1) Frequent Itemsets Mining Association Rules Algorithm Apriori is adopted to calculate Mining Frequent word set

Frequent item set is utilized to divide the initial emotion bunch of structure, frequent tendency word set microblogging will be comprised and be divided into one bunch, obtain based on the initial emotion bunch of frequent item set, simultaneously, to the frequent item set of initial emotion bunch be described as corresponding emotion bunch temporary mark, represent this initial emotion bunch emotional semantic by the frequent item set extracting each initial emotion bunch;

2.2) microblogging simple semantic degree initial cluster overlap abatement

Every bar microblogging is belonged to an emotion bunch, and between compute cluster, lap is to the emotional semantic degree of membership of initial emotion bunch, is finally undertaken bunch distributing by maximum simple semantic degree principle; Deleting the rear size of those initial cluster separation is again the sky bunch of 0, and the initial cluster after overlapping abatement is called candidate's emotion bunch;

2.3) based on the coagulation type emotion cluster of semantic similarity: carry out Agglomerative hierarchical clustering to candidate's emotion bunch, emotion bunch is merged.

3. a kind of dominant and Chinese microblog emotional analytical approach that is recessive character of merging as claimed in claim 2, is characterized in that: described step 2.1) in,

Definition 1: to certain collection X in database E, if the number of times that item collection X occurs in database E is greater than preset ratio, then title X is the frequent item set of database E, and this preset ratio is called minimum support;

If text to be regarded as affairs, the project in the corresponding affairs of text vocabulary, then can be expressed as text d: d=<t ₁, t ₂..., t _n>, wherein n represents the feature vocabulary quantity that text d comprises;

Definition 2: to certain word set W of text set D, if the support s of W in D (W) >=min_s, then title power set W is the frequent term set of text set D, and min_s is global minima support;

Scan text collection D, utilizes word frequency Trend Degree to add up the number of times of candidate appearance, collects the item collection meeting minimum support min_s and set, is designated as frequent item set; Utilize the frequent k-item collection structure Strong association rule produced, utilize frequent k-item collection to construct candidate (k+1)-item collection, iterate until candidate (k+1)-Xiang Jiwei is empty.

4. a kind of dominant and Chinese microblog emotional analytical approach that is recessive character of merging as claimed in claim 2, is characterized in that: described step 2.2) in,

Definition 3: if microblogging doc _jbe assigned to initial emotion bunch C _iin, then claim microblogging doc _jsupport bunch C _i;

Definition 4: note D _iand D _jsupport bunch C _iand C _jmicroblogging set, and D _i∩ D _j≠ 0, then claim bunch C _iwith a bunch C _joverlapping between existing bunch;

Definition 5: microblog emotional simple semantic degree, the present invention is by microblogging doc _jto a bunch C _iemotional semantic membership function be defined as follows: $\mathrm{Score}(C_i\&LeftArrow;{\mathrm{doc}}_j)=\frac{{\mathrm{\&Sigma;}}_{l=1}^n\max k=\mathrm{1,2},...,m\left\{\mathrm{sim}(f_{\mathrm{ik}},t_{\mathrm{jl}})\right\}}{n};$

Wherein, bunch frequent 1-item collection { f _i1, f _i2..., f _imrepresent initial cluster C _iaffective characteristics item, { t _j1, t _j2..., t _jnrepresent initial cluster C _imiddle microblogging text doc _jcharacteristic item; Sim (f _ik, t _jl) be a bunch characteristic item f _ikwith text feature item t _jlthe semantic similarity of definition in " knowing net ", n is microblogging text doc _jcharacteristic item number, m is a bunch characteristic item number.

5. a kind of dominant and Chinese microblog emotional analytical approach that is recessive character of merging as claimed in claim 2, is characterized in that: described step 2.3) in,

Definition 6: bunch proper vector, for candidate's emotion bunch CT _i, excavate CT _ibunch frequent 1-item collection, namely form bunch proper vector of this bunch, be designated as

Definition 7: bunch similarity matrix, remembers two different candidate's emotion bunch CT _iand CT _ja bunch proper vector be respectively: with wherein n and m submeter representation feature vocabulary quantity, then CT _iand CT _jcharacteristic item form a bunch semantic similarity matrix define by the mode of table 1;

Table 1

Definition 8: emotion bunch semantic similarity, choosing the maximum k stack features of semantic similarity in similarity matrix item to carrying out Similarity Measure between candidate's emotion, being designated as { simt _it _j1, simt _it _j2..., simt _it _jk, the semantic similarity of candidate's emotion bunch is defined as:

$\mathrm{sim}({\mathrm{CT}}_i,{\mathrm{CT}}_j)=\frac{{\mathrm{\&Sigma;}}_{l=1}^k\mathrm{sim}{\left(t_i{t}_j\right)}_l}{k}$

Coagulation type emotion cluster process based on semantic similarity is as follows:

Step 1: the proper vector extracting each candidate's emotion bunch, the semantic similarity of calculated candidate emotion bunch;

Step 2: the semantic similarity matrix building candidate emotion bunch, from the definition of bunch similarity

SimCT _i, CT _j=simCT _j, CT _i, namely this similarity matrix is a symmetric matrix;

Step 3: select similarity between maximum bunch from similarity matrix, be designated as max{simCT _i, CT _j,

If max{simCT _i, CT _j}≤C, performs Step 6; Otherwise, perform Step 4;

Step 4: if max{simCT _i, CT _j> λ, CT _iand CT _jbetween similarity comparatively large, therefore by CT _iand CT _jtwo bunches of merging, form a new bunch CT _i', delete former CT _i, and compute cluster proper vector again, update semantics similarity matrix;

Step 5: if bunch between the line number of semantic similarity matrix or columns be less than or equal to default minimum number of clusters order μ, perform Step 6; Otherwise cluster not yet terminates, come back to Step 3;

Step 6: Agglomerative hierarchical clustering terminates, obtains emotion clustering cluster CT '.

6. a kind of as described in one of Claims 1 to 5 merges dominant and Chinese microblog emotional analytical approach that is recessive character, it is characterized in that: described step 1.2) in, collect negative word set, whether with negative word before parsing emotion vocabulary, if having, negative word and emotion word are write affective characteristics table in the lump.