Unraveling the Tangle of Disinformation: A Multimodal Approach for Fake News Identification on Social Media
Pages 1849 - 1853
Abstract
The growth of interactive and multimedia content on the Internet has made it an essential news source for people worldwide. Social media is a platform for sharing information and facilitates the spread of fake news. The dissemination of disinformation on social media has a significant impact on society. Conventional methods used in the identification of fake news often struggle to analyze textual, visual, and combined aspects of news shared on social media. Therefore, we propose the Multimodal Approach for Fake News Identification (MuAFaNI), which uses a combined representation of text and images to assess news authenticity as fake or real. MuAFaNI uses the RoBERTa language model for text analysis and ResNet-50 for image analysis. Experiments on two prominent social media datasets, Twitter and Weibo, showed that MuAFaNI performed better than state-of-the-art fake news techniques in terms of accuracy, precision, recall and F1 score.
References
[1]
Firoj Alam, Stefano Cresci, Tanmoy Chakraborty, Fabrizio Silvestri, Dimiter Dimitrov, Giovanni Da San Martino, Shaden Shaar, Hamed Firooz, Preslav Nakov, et al. 2022. A Survey on Multimodal Disinformation Detection. In Proceedings of the 29th International Conference on Computational Linguistics. International Committee on Computational Linguistics, 6625--6643.
[2]
Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision. 2425--2433.
[3]
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).
[4]
Christina Boididou, Katerina Andreadou, Symeon Papadopoulos, Duc-Tien Dang- Nguyen, Giulia Boato, Michael Riegler, Yiannis Kompatsiaris, et al. 2015. Verifying multimedia use at mediaeval 2015. MediaEval 3, 3 (2015), 7.
[5]
Carlos Castillo, Marcelo Mendoza, and Barbara Poblete. 2011. Information credibility on twitter. In Proceedings of the 20th international conference on World wide web. 675--684.
[6]
Tong Chen, Xue Li, Hongzhi Yin, and Jun Zhang. 2018. Call attention to rumors: Deep attention based recurrent neural networks for early rumor detection. In Trends and Applications in Knowledge Discovery and Data Mining: PAKDD 2018 Workshops, BDASC, BDM, ML4Cyber, PAISI, DaMEMO, Melbourne, VIC, Australia, June 3, 2018, Revised Selected Papers 22. Springer, 40--52.
[7]
Micha? Chora?, Konstantinos Demestichas, Agata Gie?czyk, Álvaro Herrero, Pawe? Ksieniewicz, Konstantina Remoundou, Daniel Urda, and Micha? Wo?niak. 2021. Advanced Machine Learning techniques for fake news (online disinformation) detection: A systematic mapping study. Applied Soft Computing 101 (2021), 107050.
[8]
Souvick Ghosh and Chirag Shah. 2018. Towards automatic fake news classification. Proceedings of the Association for Information Science and Technology 55, 1 (2018), 805--807.
[9]
Genevieve Gorrell, Elena Kochkina, Maria Liakata, Ahmet Aker, Arkaitz Zubiaga, Kalina Bontcheva, and Leon Derczynski. 2019. SemEval-2019 Task 7: RumourEval 2019: Determining Rumour Veracity and Support for Rumours. In Proceedings of the 13th International Workshop on Semantic Evaluation: NAACL HLT 2019. Association for Computational Linguistics, 845--854.
[10]
Ying Guo, Hong Ge, and Jinhong Li. 2023. A two-branch multimodal fake news detection model based on multimodal bilinear pooling and attention mechanism. Frontiers in Computer Science 5 (2023), 1159063.
[11]
Ammara Habib, Muhammad Zubair Asghar, Adil Khan, Anam Habib, and Aurangzeb Khan. 2019. False information detection in online content and its role in decision making: a systematic literature review. Social Network Analysis and Mining 9 (2019), 1--20.
[12]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.
[13]
Cherilyn Ireton and Julie Posetti. 2018. Journalism, fake news & disinformation: handbook for journalism education and training. Unesco Publishing.
[14]
Ramji Jaiswal, Upendra Pratap Singh, and Krishna Pratap Singh. 2021. Fake news detection using bert-vgg19 multimodal variational autoencoder. In 2021 IEEE 8th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON). IEEE, 1--5.
[15]
Zhiwei Jin, Juan Cao, Han Guo, Yongdong Zhang, and Jiebo Luo. 2017. Multimodal fusion with recurrent neural networks for rumor detection on microblogs. In Proceedings of the 25th ACM international conference on Multimedia. 795--816.
[16]
Zhiwei Jin, Juan Cao, Yongdong Zhang, Jianshe Zhou, and Qi Tian. 2016. Novel visual and statistical image features for microblogs news verification. IEEE transactions on multimedia 19, 3 (2016), 598--608.
[17]
Dhruv Khattar, Jaipal Singh Goud, Manish Gupta, and Vasudeva Varma. 2019. Mvae: Multimodal variational autoencoder for fake news detection. In The world wide web conference. 2915--2921.
[18]
Yash Khurana, Swamita Gupta, R Sathyaraj, and SP Raja. 2022. RobinNet: A Multimodal Speech Emotion Recognition System With Speaker Recognition for Social Interactions. IEEE Transactions on Computational Social Systems (2022).
[19]
Taehyeon Kim, Jaehoon Oh, NakYil Kim, Sangwook Cho, and Se-Young Yun. 2021. Comparing kullback-leibler divergence and mean squared error loss in knowledge distillation. arXiv preprint arXiv:2105.08919 (2021).
[20]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[21]
Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).
[22]
Rina Kumari and Asif Ekbal. 2021. Amfb: Attention based multimodal factorized bilinear pooling for multimodal fake news detection. Expert Systems with Applications 184 (2021), 115412.
[23]
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019).
[24]
Yang Liu and Yi-Fang Wu. 2018. Early detection of fake news on social media through propagation path classification with recurrent and convolutional networks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32.
[25]
Anqi Mao, Mehryar Mohri, and Yutao Zhong. 2023. Cross-entropy loss functions: Theoretical analysis and applications. In International Conference on Machine Learning. PMLR, 23803--23828.
[26]
Kashyap Popat, Subhabrata Mukherjee, Jannik Strötgen, and Gerhard Weikum. 2016. Credibility assessment of textual claims on the web. In Proceedings of the 25th ACM international on conference on information and knowledge management. 2173--2178.
[27]
Martin Potthast, Johannes Kiesel, Kevin Reinartz, Janek Bevendorff, and Benno Stein. 2017. A stylometric inquiry into hyperpartisan and fake news. arXiv preprint arXiv:1702.05638 (2017).
[28]
Natali Ruchansky, Sungyong Seo, and Yan Liu. 2017. Csi: A hybrid deep model for fake news detection. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 797--806.
[29]
Gulshan Shrivastava, Prabhat Kumar, Rudra Pratap Ojha, Pramod Kumar Srivastava, Senthilkumar Mohan, and Gautam Srivastava. 2020. Defensive modeling of fake news through online social networks. IEEE Transactions on Computational Social Systems 7, 5 (2020), 1159--1167.
[30]
Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dongwon Lee, and Huan Liu. 2020. Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media. Big data 8, 3 (2020), 171--188.
[31]
Kai Shu, Xinyi Zhou, Suhang Wang, Reza Zafarani, and Huan Liu. 2019. The role of user profiles for fake news detection. In Proceedings of the 2019 IEEE/ACM international conference on advances in social networks analysis and mining. 436-- 439.
[32]
Shivangi Singhal, Rajiv Ratn Shah, Tanmoy Chakraborty, Ponnurangam Kumaraguru, and Shin'ichi Satoh. 2019. Spotfake: A multi-modal framework for fake news detection. In 2019 IEEE fifth international conference on multimedia big data (BigMM). IEEE, 39--47.
[33]
Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3156--3164.
[34]
Soroush Vosoughi, Mostafa ?Neo' Mohsenvand, and Deb Roy. 2017. Rumor gauge: Predicting the veracity of rumors on Twitter. ACM transactions on knowledge discovery from data (TKDD) 11, 4 (2017), 1--36.
[35]
Yaqing Wang, Fenglong Ma, Zhiwei Jin, Ye Yuan, Guangxu Xun, Kishlay Jha, Lu Su, and Jing Gao. 2018. Eann: Event adversarial neural networks for multi-modal fake news detection. In Proceedings of the 24th acm sigkdd international conference on knowledge discovery & data mining. 849--857.
[36]
Liang Wu and Huan Liu. 2018. Tracing fake-news footprints: Characterizing social media messages by how they propagate. In Proceedings of the eleventh ACM international conference on Web Search and Data Mining. 637--645.
[37]
YangWu, Pengwei Zhan, Yunjian Zhang, LimingWang, and Zhen Xu. 2021. Multimodal fusion with co-attention networks for fake news detection. In Findings of the association for computational linguistics: ACL-IJCNLP 2021. 2560--2569.
[38]
Kuai Xu, FengWang, HaiyanWang, and Bo Yang. 2019. Detecting fake news over online social media via domain reputations and content understanding. Tsinghua Science and Technology 25, 1 (2019), 20--27.
Index Terms
- Unraveling the Tangle of Disinformation: A Multimodal Approach for Fake News Identification on Social Media
Recommendations
Science Disinformation: On the Problem of Fake News
AbstractThis article is devoted to an important socio-cultural phenomenon that undermines public confidence in science, that is, fake science news. The term fake news is analyzed and data on the dissemination of fake news on social networks is provided. ...
Fake News, Disinformation, Propaganda, and Media Bias
CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge ManagementThe rise of Internet and social media changed not only how we consume information, but it also democratized the process of content creation and dissemination, thus making it easily available to anybody. Despite the hugely positive impact, this situation ...
Consuming Fake News: A Matter of Age? The Perception of Political Fake News Stories in Facebook Ads
Human Aspects of IT for the Aged Population. Technology and SocietyAbstractSocial media are increasingly being used by young and old as a source of information. Fake news is also on the rise. The role played by age in the consumption of fake news on social media, however, is unclear. This paper explores the generational ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
May 2024
1928 pages
ISBN:9798400701726
DOI:10.1145/3589335
- General Chairs:
- Tat-Seng Chua,
- Chong-Wah Ngo,
- Program Chairs:
- Ravi Kumar,
- Hady W. Lauw,
- Roy Ka-Wei Lee
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 May 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT)
Conference
WWW '24
Sponsor:
Acceptance Rates
Overall Acceptance Rate 1,899 of 8,196 submissions, 23%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 165Total Downloads
- Downloads (Last 12 months)165
- Downloads (Last 6 weeks)11
Reflects downloads up to 09 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in