[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Attention based hierarchical LSTM network for context-aware microblog sentiment classification

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Analyzing sentiment polarities of microblog has become a hot research topic for both academic and industrial communities. Most of the existing algorithms regard each microblog as an independent training instance. However, the sentiments embedded in short tweets are usually ambiguous and context-aware. Even a non-sentiment word might convey a clear emotional tendency in certain microblog conversations. In this paper, we regard the microblog conversation as sequence, and develop a Context Attention based Long Short-Term Memory (CA-LSTM) network to incorporate preceding tweets for context-aware sentiment classification. The CA-LSTM network has a hierarchial structure for modeling microblog sequence and allocates the words and tweets with different weights using attention mechanism. Our proposed method can not only alleviate the sparsity problem in feature space, but also capture long distance sentiment context dependency in microblog conversations. Experimental evaluations on a public available dataset show that the proposed CA-LSTM network with context information can outperform other strong baselines by a large margin.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9

Similar content being viewed by others

Notes

  1. http://www.twitter.com/.

  2. http://www.weibo.com/.

  3. http://www.ccir2015.com/.

  4. http://github.com/leotywy/coae2015.

  5. http://deeplearning.net/software/theano.

  6. http://scikit-learn.org/stable/.

  7. http://pystruct.github.io/.

References

  1. Chen, H., Sun, M., Tu, C., Lin, Y., Liu, Z.: Neural sentiment classification with user and product attention. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1650–1659 (2016)

  2. Cheng, J., Zhang, X., Li, P., Zhang, S., Ding, Z., Wang, H.: Exploring sentiment parsing of microblogging texts for opinion polling on Chinese public figures. Appl. Intell. 45(2), 429–442 (2016)

    Article  Google Scholar 

  3. Chung, J., Kastner, K., Dinh, L., Goel, K., Courville, A. C., Bengio, Y.: A recurrent latent variable model for sequential data. In: Proceedings of Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems (NIPS), pp. 2980–2988 (2015)

  4. Fernȧndez, S., Graves, A., Schmidhuber, J.: Sequence labelling in structured domains with hierarchical recurrent neural networks. In: Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI), pp. 774–779 (2007)

  5. Frege, G.: On sense and reference. Philos. Rev. 57(3), 1296–1323 (2010)

    Google Scholar 

  6. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. J. Mach. Learn. Res. 9, 249–256 (2010)

    Google Scholar 

  7. Goldberg, Y.: A primer on neural network models for natural language processing. arXiv:http://arXiv.org/abs/1510.00726 (2015)

  8. Greff, K., Srivastava, R. K., Koutník, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: A search space odyssey. arXiv:http://arXiv.org/abs/1503.04069 (2015)

  9. Hihi, S. E., Bengio, Y.: Hierarchical recurrent neural networks for long-term dependencies. In: Proceedings of Advances in Neural Information Processing Systems 8 (NIPS), pp. 493–499 (1995)

  10. Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Improving neural networks by preventing co-adaptation of feature detectors. arXiv:http://arXiv.org/abs/1207.0580 (2012)

  11. Huang, M., Cao, Y., Dong, C.: Modeling rich contexts for sentiment classification with LSTM,. arXiv:http://arXiv.org/abs/1605.01478 (2016)

  12. Katz, G., Ofek, N., Shapira, B.: Consent: Context-based sentiment analysis. Knowl.-Based Syst. 84, 162–178 (2015)

    Article  Google Scholar 

  13. Kwak, H., Lee, C., Park, H., Moon, S. B.: What is twitter, a social network or a news media? In: Proceedings of the 19th International Conference on World Wide Web (WWW), pp. 591–600 (2010)

  14. Lafferty, J. D., McCallum, A., Pereira, F. C. N.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning (ICML), pp. 282–289 (2001)

  15. Li, D., Shuai, X., Sun, G., Tang, J., Ding, Y., Luo, Z.: Mining topic-level opinion influence in microblog. In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management (CIKM), pp. 1562–1566 (2012)

  16. Li, J., Luong, T., Jurafsky, D., Hovy, E. H.: When are tree structures necessary for deep learning of representations? In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2304–2314 (2015)

  17. Lu, Y., Castellanos, M., Dayal, U., Zhai, C.: Automatic construction of a context-aware sentiment lexicon: an optimization approach. In: Proceedings of the 20th International Conference on World Wide Web (WWW), pp. 347–356 (2011)

  18. McDonald, R. T., Hannan, K., Neylon, T., Wells, M., Reynar, J. C.: Structured models for fine-to-coarse sentiment analysis. In: Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 432—-439 (2007)

  19. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv:http://arXiv.org/abs/1301.3781 (2013)

  20. Mnih, A., Hinton, G. E.: A scalable hierarchical distributed language model. In: Proceedings of Advances in Neural Information Processing Systems 21: Annual Conference on Neural Information Processing Systems (NIPS), pp. 1081–1088 (2008)

  21. Muhammad, A., Wiratunga, N., Lothian, R.: Contextual sentiment analysis for social media genres. Knowl.-Based Syst. 108, 92–101 (2016)

    Article  Google Scholar 

  22. Mukherjee, S., Bhattacharyya, P.: Sentiment analysis in twitter with lightweight discourse analysis. In: Proceedings of the Conference 24th International Conference on Computational Linguistics (COLING), pp. 1847–1864 (2012)

  23. Ravi, K., Ravi, V.: A survey on opinion mining and sentiment analysis: Tasks, approaches and applications. Knowl.-Based Syst. 89, 14–46 (2015)

    Article  Google Scholar 

  24. Ren, Y., Zhang, Y., Zhang, M., Ji, D.: Context-sensitive twitter sentiment classification using neural network. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI), pp. 215–221 (2016)

  25. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  26. Sutskever, I., Vinyals, O., Le, Q. V.: Sequence to sequence learning with neural networks. In: Proceedings of Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems (NIPS), pp. 3104–3112 (2014)

  27. Tai, K. S., Socher, R., Manning, C. D.: Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing (ACL-IJCNLP), pp. 1556–1566 (2015)

  28. Tan, C., Lee, L., Tang, J., Jiang, L., Zhou, M., Li, P.: User-level sentiment analysis incorporating social networks. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pp. 1397–1405 (2011)

  29. Tang, D., Qin, B., Liu, T.: Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1422–1432 (2015)

  30. Turney, P. D.: Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 417–424 (2002)

  31. Vanzo, A., Croce, D., Basili, R.: A context-based model for sentiment analysis in twitter. In: Proceedings of the 25th International Conference on Computational Linguistics (COLING), pp. 2345–2354 (2014)

  32. Wang, X., Liu, Y., Sun, C., Wang, B., Wang, X.: Predicting polarities of tweets by composing word embeddings with long short-term memory. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing (ACL-IJCNLP), pp. 1343–1353 (2015)

  33. Wang, Y., Feng, S., Wang, D., Zhang, Y., Yu, G.: Context-aware chinese microblog sentiment classification with bidirectional LSTM. In: Proceedings of 18th Asia-Pacific Web Conference (APWeb), pp. 594–606 (2016)

  34. Wang, Y., Huang, M., Zhu, X., Zhao, L.: Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 606–615 (2016)

  35. Wu, F., Huang, Y., Song, Y.: Structured microblog sentiment classification via social context regularization. Neurocomputing 175, 599–609 (2016)

    Article  Google Scholar 

  36. Wu, F., Song, Y., Huang, Y.: Microblog sentiment classification with contextual knowledge regularization. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI), pp. 2332–2338 (2015)

  37. Yang, B., Cardie, C.: Context-aware learning for sentence-level sentiment analysis with posterior regularization. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL), pp. 325–335 (2014)

  38. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), pp. 1480–1489 (2016)

  39. Zeiler, M. D.: Adadelta: An adaptive learning rate method. arXiv:http://arXiv.org/abs/1212.5701 (2012)

  40. Zhao, Z., Lu, H., Cai, D., He, X., Zhuang, Y.: Microblog sentiment classification via recurrent random walk network learning. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI), pp. 3532–3538 (2017)

  41. Zhou, X., Wan, X., Xiao, J.: Attention-based LSTM network for cross-lingual sentiment classification. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 247–256 (2016)

  42. Zhu, X., Sobhani, P., Guo, H.: Long short-term memory over recursive structures. In: Proceedings of the 32nd International Conference on Machine Learning (ICML), pp. 1604–1612 (2015)

Download references

Acknowledgments

The work was supported by National Natural Science Foundation of China (61370074, 61402091), the Fundamental Research Funds for the Central Universities of China under Grant N140404012.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Northeastern University, No.195 Chuangxin Road, Hunnan District, Shenyang, China

    Shi Feng, Yang Wang, Liran Liu, Daling Wang & Ge Yu

Authors
  1. Shi Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liran Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daling Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ge Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shi Feng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Feng, S., Wang, Y., Liu, L. et al. Attention based hierarchical LSTM network for context-aware microblog sentiment classification. World Wide Web 22, 59–81 (2019). https://doi.org/10.1007/s11280-018-0529-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-018-0529-6

Keywords

Search

Navigation