Just Accepted
Misinformation, Disinformation, and Generative AI: Implications for Perception and Policy
Abstract
The emergence of generative artificial intelligence (GenAI) has exacerbated the challenges of Misinformation, Disinformation, and Mal-information (MDM) within digital ecosystems. These multifaceted challenges demand a re-evaluation of the digital information lifecycle and a deep understanding of its social impact. An interdisciplinary strategy integrating insights from technology, social sciences, and policy analysis is crucial to address these issues effectively. This paper introduces a three-tiered framework to scrutinize the lifecycle of GenAI-driven content from creation to consumption, emphasizing the consumer perspective. We examine the dynamics of consumer behavior that drive interactions with MDM, pinpoints vulnerabilities in the information dissemination process, and advocates for adaptive, evidence-based policies. Our interdisciplinary methodology aims to bolster information integrity and fortify public trust, equipping digital societies to manage the complexities of GenAI and proactively address the evolving challenges of digital misinformation. We conclude by discussing how GenAI can be leveraged to combat MDM, thereby creating a reflective cycle of technological advancement and mitigation.
References
[1]
Rachel Adams, Ayantola Alayande, Zameer Brey, Brantley Browning, Michael Gastrow, Jerry John Kponyo, Dona Mathew, Moremi Nkosi, Henry Nunoo-Mensah, Diana Nyakundi, et al. 2023. A new research agenda for African generative AI. Nature Human Behaviour 7, 11 (2023), 1839–1841.
[2]
Saifuddin Ahmed. 2023. Examining public perception and cognitive biases in the presumed influence of deepfakes threat: empirical evidence of third person perception from three studies. Asian Journal of Communication 33, 3 (2023), 308–331.
[3]
Saifuddin Ahmed and Han Wei Tan. 2022. Personality and perspicacity: Role of personality traits and cognitive ability in political misinformation discernment and sharing behavior. Personality and Individual Differences 196 (2022), 111747.
[4]
Hunt Allcott, Luca Braghieri, Sarah Eichmeyer, and Matthew Gentzkow. 2020. The welfare effects of social media. American economic review 110, 3 (2020), 629–676.
[5]
Antonio A Arechar, Jennifer Allen, Adam J Berinsky, Rocky Cole, Ziv Epstein, Kiran Garimella, Andrew Gully, Jackson G Lu, Robert M Ross, Michael N Stagnaro, et al. 2023. Understanding and combatting misinformation across 16 countries on six continents. Nature Human Behaviour 7, 9 (2023), 1502–1513.
[6]
Robert Baldwin, Martin Cave, and Martin Lodge. 2011. UNDERSTANDING REGULATION 2E P: Theory, Strategy, and Practice. Oxford university press.
[7]
Gordon M Becker, Morris H DeGroot, and Jacob Marschak. 1964. Measuring utility by a single-response sequential method. Behavioral science 9, 3 (1964), 226–232.
[8]
Claudi L Bockting, Eva AM van Dis, Robert van Rooij, Willem Zuidema, and Johan Bollen. 2023. Living guidelines for generative AI—why scientists must oversee its use. Nature 622, 7984 (2023), 693–696.
[9]
Lee C Bollinger and Geoffrey R Stone. 2022. Social media, freedom of speech, and the future of our democracy. Oxford University Press.
[10]
Antonio Carnevale, Claudia Falchi Delgado, and Piercosma Bisconti. 2023. Hybrid Ethics for Generative AI: Some Philosophical Inquiries on GANs. HUMANA. MENTE Journal of Philosophical Studies 16, 44 (2023), 33–56.
[11]
Sanjay Chawla, Preslav Nakov, Ahmed Ali, Wendy Hall, Issa Khalil, Xiaosong Ma, Husrev Taha Sencar, Ingmar Weber, Michael Wooldridge, and Ting Yu. 2023. Ten years after ImageNet: a 360° perspective on artificial intelligence. Royal Society Open Science 10, 3 (2023), 221414.
[12]
Zhe Chen, Aixin Sun, and Xiaokui Xiao. 2021. Incremental community detection on large complex attributed network. ACM Transactions on Knowledge Discovery from Data (TKDD) 15, 6(2021), 1–20.
[13]
Simon Chesterman. 2024. Good models borrow, great models steal: intellectual property rights and generative AI. Policy and Society (2024), puae006.
[14]
Kevin Chung, Keehyung Kim, and Noah Lim. 2020. Social structures and reputation in expert review systems. Management Science 66, 7 (2020), 3249–3276.
[15]
European Commission. [n. d.]. The Digital Services Act package. European Commission([n. d.]). https://digital-strategy.ec.europa.eu/en/policies/digital-services-act-package
[16]
Gregory Currie. 2007. Both sides of the story: explaining events in a narrative. Philosophical Studies 135 (2007), 49–63.
[17]
Philipe de Freitas Melo, Carolina Coimbra Vieira, Kiran Garimella, Pedro OS Vaz de Melo, and Fabrício Benevenuto. 2020. Can WhatsApp counter misinformation by limiting message forwarding?. In Complex Networks and Their Applications VIII: Volume 1 Proceedings of the Eighth International Conference on Complex Networks and Their Applications COMPLEX NETWORKS 2019 8. Springer, 372–384.
[18]
Cristina Godoy B de Oliveira, Fabio G Cozman, and João Paulo C Veiga. 2023. This hot AI summer will impact Brazil’s democracy. Nature Human Behaviour(2023), 1–3.
[19]
Bruce J Dickson. 2011. No ”Jasmine” for China. Current History (2011). https://www.jstor.org/stable/45319730
[20]
Benjamin Enke and Florian Zimmermann. 2019. Correlation neglect in belief formation. The Review of Economic Studies 86, 1 (2019), 313–332.
[21]
Erik Eyster and Georg Weizsacker. 2010. Correlation neglect in financial decision-making. (2010).
[22]
Leon Festinger. 1962. A Theory of Cognitive Dissonance (Evanston: Row Peterson, 1957).’Cognitive Dissonance’. Scientific American 207(1962).
[23]
Sean Fischer, Kokil Jaidka, and Yphtach Lelkes. 2020. Auditing local news presence on Google News. Nature Human Behaviour 4, 12 (2020), 1236–1244.
[24]
Serena Giusti and Elisa Piras. 2020. Democracy and fake news: information manipulation and post-truth politics. Routledge.
[25]
Lucas Gover. 2023. Political bias in large language models. The Commons: Puget Sound Journal of Politics 4, 1 (2023), 2.
[26]
James Griffiths. 2021. The great firewall of China: How to build and control an alternative version of the internet. Bloomsbury Publishing.
[27]
The White House. 2023. FACT SHEET: President Biden Issues Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence. The White House (2023). https://www.whitehouse.gov/briefing-room/statements-releases/2023/10/30/fact-sheet-president-biden-issues-executive-order-on-safe-secure-and-trustworthy-artificial-intelligence/
[28]
Matthew Hutson. 2023. Rules to keep AI in check: nations carve different paths for tech regulation.Nature 620, 7973 (2023), 260–263.
[29]
Radosveta Ivanova-Stenzel and Gyula Seres. 2021. Are strategies anchored?European Economic Review 135 (2021), 103725.
[30]
Kokil Jaidka, Sean Fischer, Yphtach Lelkes, and Yifei Wang. 2023. News nationalization in a digital age: An examination of how local protests are covered and curated online. The ANNALS of the American Academy of Political and Social Science 707, 1(2023), 189–207.
[31]
Kokil Jaidka, Subhayan Mukerjee, and Yphtach Lelkes. 2023. Silenced on social media: the gatekeeping functions of shadowbans in the American Twitterverse. Journal of Communication 73, 2 (2023), 163–178.
[32]
Shashi Jayakumar, Benjamin Ang, and Nur Diyanah Anwar. 2021. Fake News and disinformation: Singapore perspectives. Disinformation and Fake News(2021), 137–158.
[33]
Nan Jiang, Bangjie Sun, Terence Sim, and Jun Han. 2024. Can I Hear Your Face? Pervasive Attack on Voice Authentication Systems with a Single Face Image. In Proceedings of the 33rd Usenix Security Symposium.
[34]
Rohit Kumar Kaliyar, Anurag Goswami, and Pratik Narang. 2021. FakeBERT: Fake news detection in social media with a BERT-based deep learning approach. Multimedia Tools and Applications 80 (2021), 11765 – 11788. https://api.semanticscholar.org/CorpusID:230800534
[35]
Shaleen Khanal, Hongzhou Zhang, and Araz Taeihagh. 2024. Why and how is the power of Big Tech increasing in the policy process? The case of generative AI. Policy and Society (2024), puae012.
[36]
Jan Kietzmann, Linda W. Lee, Ian P. McCarthy, and Tim C. Kietzmann. 2020. Deepfakes: Trick or treat?Business Horizons 63, 2 (2020), 135–146. https://doi.org/10.1016/j.bushor.2019.11.006 ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING.
[37]
Samuli Laato, AKM Najmul Islam, Muhammad Nazrul Islam, and Eoin Whelan. 2020. What drives unverified information sharing and cyberchondria during the COVID-19 pandemic?European journal of information systems 29, 3 (2020), 288–305.
[38]
Laks V. S. Lakshmanan, Michael Simpson, and Saravanan Thirumuruganathan. 2019. Combating Fake News: A Data Management and Mining Perspective. Proc. VLDB Endow. 12(2019), 1990–1993. https://api.semanticscholar.org/CorpusID:201653243
[39]
David MJ Lazer, Matthew A Baum, Yochai Benkler, Adam J Berinsky, Kelly M Greenhill, Filippo Menczer, Miriam J Metzger, Brendan Nyhan, Gordon Pennycook, David Rothschild, et al. 2018. The science of fake news. Science 359, 6380 (2018), 1094–1096.
[40]
Legislation.gov.uk. 2023. Online Safety Act 2023. (2023). https://www.legislation.gov.uk/ukpga/2023/50/contents/enacted
[41]
Jia Li, Noah Lim, and Hua Chen. 2020. Examining salesperson effort allocation in teams: A randomized field experiment. Marketing Science 39, 6 (2020), 1122–1141.
[42]
Yan Liu, Yu Liu, Xiaokang Chen, Pin-Yu Chen, Daoguang Zan, Min-Yen Kan, and Tsung-Yi Ho. 2024. The devil is in the neurons: Interpreting and mitigating social biases in language models. In The Twelfth International Conference on Learning Representations.
[43]
Dani Madrid-Morales, Herman Wasserman, Gregory Gondwe, Khulekani Ndlovu, Etse Sikanku, Melissa Tully, Emeka Umejei, and Chikezie Uzuegbunam. 2021. Motivations for sharing misinformation: A comparative study in six Sub-Saharan African countries. International Journal of Communication 15, 2021 (2021), 1200–1219.
[44]
Aqdas Malik, Amandeep Dhir, Puneet Kaur, and Aditya Johri. 2020. Correlates of social media fatigue and academic performance decrement: A large cross-sectional study. Information Technology & People 34, 2 (2020), 557–580.
[45]
Helen Margetts and Cosmina Dorobantu. 2019. Rethink government with AI. Nature 568, 7751 (2019), 163–165.
[46]
Marissa Mock, Suzanne Edavettal, Christopher Langmead, and Alan Russell. 2023. AI can help to speed up drug discovery—but only if we give it the right data. Nature 621, 7979 (2023), 467–470.
[47]
Subhayan Mukerjee, Kokil Jaidka, and Yphtach Lelkes. 2022. The political landscape of the US Twitterverse. Political Communication 39, 5 (2022), 565–588.
[48]
Subhayan Mukerjee and Tian Yang. 2021. Choosing to avoid? A conjoint experimental study to understand selective exposure and avoidance on social media. Political Communication 38, 3 (2021), 222–240.
[49]
Van-Hoang Nguyen, Kazunari Sugiyama, Preslav Nakov, and Min-Yen Kan. 2020. FANG: Leveraging Social Context for Fake News Detection Using Graph Representation. Proceedings of the 29th ACM International Conference on Information & Knowledge Management (2020). https://api.semanticscholar.org/CorpusID:221150991
[50]
Linda Nordling. 2019. A fairer way forward for AI in health care. Nature 573, 7775 (2019), S103–S103.
[51]
Charles N Noussair and Gyula Seres. 2020. The effect of collusion on efficiency in experimental auctions. Games and Economic Behavior 119 (2020), 267–287.
[52]
Parliament of the Commonwealth of Australia. 2023. Communications Legislation Amendment (Combatting Misinformation and Disinformation) Bill 2023. (2023). https://www.infrastructure.gov.au/sites/default/files/documents/communications-legislation-amendment-combatting-misinformation-and-disinformation-bill2023-june2023.pdf
[53]
Ofcom. 2023. Online safety - what is Ofcom’s role, and what does it mean for you?(2023). https://www.ofcom.org.uk/news-centre/2023/online-safety-ofcom-role-and-what-it-means-for-you
[54]
Jeff Z. Pan, Siyana Pavlova, Chenxi Li, Ningxi Li, Yangmei Li, and Jinshuo Liu. 2018. Content Based Fake News Detection Using Knowledge Graphs. In International Workshop on the Semantic Web. https://api.semanticscholar.org/CorpusID:52900831
[55]
Liangming Pan, Xinyuan Lu, Min-Yen Kan, and Preslav Nakov. 2023. QACheck: A Demonstration System for Question-Guided Multi-Hop Fact-Checking. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Yansong Feng and Els Lefever (Eds.). Association for Computational Linguistics, Singapore, 264–273. https://doi.org/10.18653/v1/2023.emnlp-demo.23
[56]
Liangming Pan, Xiaobao Wu, Xinyuan Lu, Anh Tuan Luu, William Yang Wang, Min-Yen Kan, and Preslav Nakov. 2023. Fact-Checking Complex Claims with Program-Guided Reasoning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). Association for Computational Linguistics, Toronto, Canada, 6981–7004. https://doi.org/10.18653/v1/2023.acl-long.386
[57]
Sankalp Phartiyal and Krishna V Kurup. 2018. WhatsApp curbs message forwarding in bid to deter India lynch mobs. Reuters (2018). https://www.reuters.com/article/idUSKBN1KB026/
[58]
Francesco Pierri, Carlo Piccardi, and Stefano Ceri. 2019. Topology comparison of Twitter diffusion networks effectively reveals misleading information. Scientific Reports 10(2019). https://api.semanticscholar.org/CorpusID:147703897
[59]
Peng Qi, Zehong Yan, Wynne Hsu, and Mong Li Lee. 2024. SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection. In Proceedings of the 2024 Conference on Computer Vision and Pattern Recognition.
[60]
Alex Rees-Jones and Dmitry Taubinsky. 2020. Measuring “schmeduling”. The Review of Economic Studies 87, 5 (2020), 2399–2438.
[61]
Amy Remeikis. 2023. Why is Labor’s bill on combatting disinformation so controversial?The Guardian (2023). https://www.theguardian.com/australia-news/2023/oct/01/why-is-labors-bill-on-combatting-disinformation-so-controversial
[62]
Ronald E Robertson, Shan Jiang, Kenneth Joseph, Lisa Friedland, David Lazer, and Christo Wilson. 2018. Auditing partisan audience bias within google search. Proceedings of the ACM on Human-Computer Interaction 2, CSCW(2018), 1–22.
[63]
David Rozado. 2023. The political biases of chatgpt. Social Sciences 12, 3 (2023), 148.
[64]
Sanjay Saha, Rashindrie Perera, Sachith Seneviratne, Tamasha Malepathirana, Sanka Rasnayaka, Deshani Geethika, Terence Sim, and Saman Halgamuge. 2023. Undercover Deepfakes: Detecting Fake Segments in Videos. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 415–425.
[65]
Francesco Salvi, Manoel Horta Ribeiro, Riccardo Gallotti, and Robert West. 2024. On the conversational persuasiveness of large language models: A randomized controlled trial. arXiv preprint arXiv:2403.14380(2024).
[66]
Sabrina Siddiqui and Deepa Seetharaman. 2023. White House Says Amazon, Google, Meta, Microsoft Agree to AI Safeguards. The Wall Street Journal(2023). https://www.wsj.com/articles/white-house-says-amazon-google-meta-microsoft-agree-to-ai-safeguards-eabe3680
[67]
Kazunari Sugiyama and Min-Yen Kan. 2015. ”Towards higher relevance and serendipity in scholarly paper recommendation” by Kazunari Sugiyama and Min-Yen Kan with Martin Vesely as coordinator. SIGWEB Newsl. 2015, Winter, Article 4 (feb 2015), 16 pages. https://doi.org/10.1145/2719943.2719947
[68]
Josh Taylor. 2023. Labor to overhaul misinformation bill after objections over freedom of speech. The Guardian (2023). https://www.theguardian.com/australia-news/2023/nov/13/labor-misinformation-bill-objections-freedom-of-speech-religious-freedom
[69]
Jefferson Thomas. 1787. Jefferson’s preference for ”newspapers without government” over ”government without newspapers”. Online Library of Liberty(1787). https://oll.libertyfund.org/quote/jefferson-s-preference-for-newspapers-without-government-over-government-without-newspapers-1787
[70]
The Straits Times. 2017. ’Oh, bother’: Chinese censors can’t bear Winnie the Pooh. The Straits Times (2017). https://www.straitstimes.com/asia/east-asia/oh-bother-chinese-censors-cant-bear-winnie-the-pooh
[71]
Augustin Toma, Senthujan Senkaiahliyan, Patrick R Lawler, Barry Rubin, and Bo Wang. 2023. Generative AI could revolutionize health care—but not if control is ceded to big tech. Nature 624, 7990 (2023), 36–38.
[72]
Chris Tyler, KL Akerlof, Alessandro Allegra, Zachary Arnold, Henriette Canino, Marius A Doornenbal, Josh A Goldstein, David Budtz Pedersen, and William J Sutherland. 2023. AI tools as science policy advisers? The potential and the pitfalls. Nature 622, 7981 (2023), 27–30.
[73]
Francielle Vargas, Kokil Jaidka, Thiago AS Pardo, and Fabrício Benevenuto. 2023. Predicting sentence-level factuality of news and bias of media outlets. arXiv preprint arXiv:2301.11850(2023).
[74]
Preetika Verma, Hansin Ahuja, and Kokil Jaidka. 2023. Quantify the political bias in news edits: experiments with few-shot learners (student abstract). In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 16354–16355.
[75]
Christian von der Weth, Ashraf Abdul, Shaojing Fan, and Mohan Kankanhalli. 2020. Helping Users Tackle Algorithmic Threats on Social Media: A Multimedia Research Agenda. In Proceedings of the 28th ACM International Conference on Multimedia (Seattle, WA, USA) (MM ’20). Association for Computing Machinery, New York, NY, USA, 4425–4434. https://doi.org/10.1145/3394171.3414692
[76]
Human Rights Watch. 2021. Singapore: ’Fake News’ Law Curtails Speech. Human Rights Watch (2021). https://www.hrw.org/news/2021/01/13/singapore-fake-news-law-curtails-speech
[77]
Nick Wingfield, Mike Isaac, and Katie Benner. 2016. Google and Facebook take aim at fake news sites. The New York Times (2016). https://www.nytimes.com/2016/11/15/technology/google-will-ban-websites-that-host-fake- news-from-using-its-ad-service.html
[78]
Konarski Xawery. 2023. The Digital Services Act (DSA) and combating disinformation - 10 key takeaways. (2023). https://www.traple.pl/en/the-digital-services-act-dsa-and-combating-disinformation-10-key-takeaways/
[79]
Yueji Yang, Yuchen Li, Panagiotis Karras, and Anthony KH Tung. 2021. Context-aware outstanding fact mining from knowledge graphs. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2006–2016.
[80]
Yueji Yang, Yuchen Li, and Anthony KH Tung. 2021. Newslink: Empowering intuitive news search with knowledge graphs. In 2021 IEEE 37th International Conference on Data Engineering (ICDE). IEEE, 876–887.
[81]
Jing Zeng and Chung-hong Chan. 2021. A cross-national diagnosis of infodemics: Comparing the topical and temporal features of misinformation around COVID-19 in China, India, the US, Germany and France. Online Information Review 45, 4 (2021), 709–728.
[82]
Bowen Zhang and Terence Sim. 2022. Localizing fake segments in speech. In 2022 26th International Conference on Pattern Recognition (ICPR). IEEE, 3224–3230.
[83]
Shiqi Zhang, Jiachen Sun, Wenqing Lin, Xiaokui Xiao, Yiqian Huang, and Bo Tang. 2024. Information Diffusion Meets Invitation Mechanism. (2024).
Index Terms
- Misinformation, Disinformation, and Generative AI: Implications for Perception and Policy
Recommendations
Interactions with Potential Mis/Disinformation URLs Among U.S. Users on Facebook, 2017-2019
FOCI '21: Proceedings of the ACM SIGCOMM 2021 Workshop on Free and Open Communications on the InternetMisinformation and disinformation online---and on social media in particular--- have become a topic of widespread concern. Recently, Facebook and Social Science One released a large, unique, privacy-preserving dataset to researchers that contains data on ...
On the Misinformation Beat: Understanding the Work of Investigative Journalists Reporting on Problematic Information Online
CSCWJournalists are increasingly investigating and reporting on problematic online content such as misinformation, disinformation, and conspiracy theories, leading to the creation of a new misinformation beat. The process of collecting, analyzing, and ...
Science Disinformation: On the Problem of Fake News
AbstractThis article is devoted to an important socio-cultural phenomenon that undermines public confidence in science, that is, fake science news. The term fake news is analyzed and data on the dissemination of fake news on social networks is provided. ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
EISSN:2639-0175
Table of ContentsCopyright © 2024 Copyright held by the owner/author(s).
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Online AM: 23 August 2024
Accepted: 07 August 2024
Revised: 15 June 2024
Received: 02 February 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 1,443Total Downloads
- Downloads (Last 12 months)1,443
- Downloads (Last 6 weeks)295
Reflects downloads up to 15 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in