[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3501247.3531574acmconferencesArticle/Chapter ViewAbstractPublication PageswebsciConference Proceedingsconference-collections
research-article

Should we tweet this? Generative response modeling for predicting reception of public health messaging on Twitter

Published: 26 June 2022 Publication History

Abstract

The way people respond to messaging from public health organizations on social media can provide insight into public perceptions on critical health issues, especially during a global crisis such as COVID-19. It could be valuable for high-impact organizations such as the US Centers for Disease Control and Prevention (CDC) or the World Health Organization (WHO) to understand how these perceptions impact reception of messaging on health policy recommendations. We collect two datasets of public health messages and their responses from Twitter relating to COVID-19 and Vaccines, and introduce a predictive method which can be used to explore the potential reception of such messages. Specifically, we harness a generative model (GPT-2) to directly predict probable future responses and demonstrate how it can be used to optimize expected reception of important health guidance. Finally, we introduce a novel evaluation scheme with extensive statistical testing which allows us to conclude that our models capture the semantics and sentiment found in actual public health responses.

Supplementary Material

MP4 File (WS22_S6_106.mp4)
Presentation video

References

[1]
S Bao, H He, F Wang, H Wu, and H Wang. 2020. PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable. In Proc of the 58th Annual Meeting of the Assoc. for Computational Linguistics. Association for Computational Linguistics, Online, 85–96. https://doi.org/10.18653/v1/2020.acl-main.9
[2]
S Bao, H He, F Wang, H Wu, H Wang, W Wu, Z Guo, Z Liu, and X Xu. 2021. PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning. In Findings of the Assoc. for Computational Linguistics: ACL-IJCNLP 2021. Association for Computational Linguistics, Online, 2513–2525. https://doi.org/10.18653/v1/2021.findings-acl.222
[3]
F Barbieri, J Camacho-Collados, L Espinosa Anke, and L Neves. 2020. TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification. In Findings of the Assoc for Computational Linguistics: EMNLP 2020. Association for Computational Linguistics, Online, 1644–1650. https://doi.org/10.18653/v1/2020.findings-emnlp.148
[4]
J Bi and K P Bennett. 2003. Regression error characteristic curves. In Proceedings of the 20th Intl conf on machine learning (ICML-03). 43–50.
[5]
T Brown, B Mann, N Ryder, M Subbiah, J D Kaplan, P Dhariwal, A Neelakantan, P Shyam, G Sastry, A Askell, 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
[6]
Y M Çetinkaya, İ Hı Toroslu, and H Davulcu. 2020. Developing a Twitter bot that can join a discussion using state-of-the-art architectures. Social network analysis and mining 10, 1 (2020), 1–21.
[7]
L-A Cotfas, C Delcea, I Roxin, C Ioanăş, D S Gherai, and F Tajariol. 2021. The Longest Month: Analyzing COVID-19 Vaccination Opinions Dynamics From Tweets in the Month Following the First Vaccine Announcement. IEEE Access 9(2021), 33203–33223. https://doi.org/10.1109/ACCESS.2021.3059821
[8]
M Galley, C Brockett, A Sordoni, Y Ji, M Auli, C Quirk, M Mitchell, J Gao, and B Dolan. 2015. deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets. In Proc of the 53rd Annual Meeting of the Assoc for Computational Linguistics and the 7th Intl Joint Conf on Natural Language Processing (Volume 2: Short Papers). Association for Computational Linguistics, Beijing, China, 445–450. https://doi.org/10.3115/v1/P15-2073
[9]
M D. Garvey, J Samuel, and A Pelaez. 2021. Would you please like my tweet?! An artificially intelligent, generative probabilistic, and econometric based system design for popularity-driven tweet content generation. Decision Support Systems 144 (2021), 113497. https://doi.org/10.1016/j.dss.2021.113497
[10]
P Gupta, S Mehri, T Zhao, A Pavel, M Eskenazi, and J Bigham. 2019. Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple References. In Proc of the 20th Annual SIGdial Meeting on Discourse and Dialogue. Association for Computational Linguistics, Stockholm, Sweden, 379–391. https://doi.org/10.18653/v1/W19-5944
[11]
P Lewis, E Perez, A Piktus, F Petroni, V Karpukhin, N Goyal, H Küttler, M Lewis, W-t Yih, T Rocktäschel, 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems 33 (2020), 9459–9474.
[12]
T Liu, Y Zhang, C Brockett, Y Mao, Z Sui, W Chen, and B Dolan. 2021. A token-level reference-free hallucination detection benchmark for free-form text generation. arXiv preprint arXiv:2104.08704(2021).
[13]
Y Liu, M Ott, N Goyal, J Du, M Joshi, D Chen, O Levy, M Lewis, L Zettlemoyer, and V Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. CoRR abs/1907.11692(2019). arXiv:1907.11692http://arxiv.org/abs/1907.11692
[14]
C Lofi and R Krestel. 2012. iParticipate: Automatic Tweet Generation from Local Government Data. In Database Systems for Advanced Applications, S-G Lee, Z Peng, X Zhou, Y-S Moon, R Unland, and J Yoo (Eds.). Springer, Berlin, Heidelberg, 295–298.
[15]
I Loshchilov and F Hutter. 2019. Decoupled Weight Decay Regularization. In Intl Conf on Learning Representations. https://openreview.net/forum?id=Bkg6RiCqY7
[16]
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
[17]
S Roller, E Dinan, N Goyal, D Ju, M Williamson, Y Liu, J Xu, M Ott, EM Smith, Y-Lan Boureau, and J Weston. 2021. Recipes for Building an Open-Domain Chatbot. In Proc of the 16th Conference of the European Chapter of the Assoc for Computational Linguistics: Main Volume. Association for Computational Linguistics, Online, 300–325. https://doi.org/10.18653/v1/2021.eacl-main.24
[18]
S Rosenthal, N Farra, and P Nakov. 2017. SemEval-2017 Task 4: Sentiment Analysis in Twitter. In Proc of the 11th Intl Workshop on Semantic Evaluation (SemEval-2017). Association for Computational Linguistics, Vancouver, Canada, 502–518. https://doi.org/10.18653/v1/S17-2088
[19]
B Roy, P Dutta, and S Choudhury. 2022. Probabilistic Traffic Tweet Generation Model. In Advances in Distributed Computing and Machine Learning, J P Sahoo, A K Tripathy, M Mohanty, K-C Li, and A K Nayak (Eds.). Springer, Singapore, 250–260.
[20]
A C Sanders, R C White, L S Severson, R Ma, R McQueen, H C A Paulo, Y Zhang, J S Erickson, and K P Bennett. 2021. Unmasking the conversation on masks: Natural language processing for topical sentiment analysis of COVID-19 Twitter discourse. In AMIA Annual Symposium Proc, Vol. 2021. American Medical Informatics Association, 555.
[21]
K Tran and F Popowich. 2016. Automatic Tweet Generation From Traffic Incident Data. In Proc of the 2nd Intl Workshop on Natural Language Generation and the Semantic Web (WebNLG 2016). Association for Computational Linguistics, Edinburgh, Scotland, 59–66. https://aclanthology.org/W16-3512
[22]
W Wang, F Wei, L Dong, H Bao, N Yang, and M Zhou. 2020. MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers. arxiv:2002.10957 [cs.CL]
[23]
Y Wang, H Hao, and L Sundahl Platt. 2021. Examining risk and crisis communications of government agencies and stakeholders during early-stages of COVID-19 on Twitter. Computers in Human Behavior 114 (2021), 106568. https://doi.org/10.1016/j.chb.2020.106568
[24]
Bo Wei, S Lu, L Mou, H Zhou, P Poupart, G Li, and Z Jin. 2019. Why Do Neural Dialog Systems Generate Short and Meaningless Replies? a Comparison between Dialog and Translation. In ICASSP 2019 - 2019 IEEE Intl Conf on Acoustics, Speech and Signal Processing (ICASSP). 7290–7294. https://doi.org/10.1109/ICASSP.2019.8682634
[25]
Y Zhang, S Sun, M Galley, Y-C Chen, C Brockett, X Gao, J Gao, J Liu, and B Dolan. 2020. DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation. In Proc of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Association for Computational Linguistics, Online, 270–278. https://doi.org/10.18653/v1/2020.acl-demos.30

Cited By

View all
  • (2023)Public health agencies’ use of social media for communication during pandemics: a scoping review of the literatureOsong Public Health and Research Perspectives10.24171/j.phrp.2023.009514:4(235-251)Online publication date: 31-Aug-2023
  • (2023)Perceiving the Ukraine-Russia Conflict: Topic Modeling and Clustering on Twitter Data2023 IEEE Ninth International Conference on Big Data Computing Service and Applications (BigDataService)10.1109/BigDataService58306.2023.00028(147-148)Online publication date: Jul-2023

Index Terms

  1. Should we tweet this? Generative response modeling for predicting reception of public health messaging on Twitter
        Index terms have been assigned to the content through auto-classification.

        Recommendations

        Comments

        Please enable JavaScript to view thecomments powered by Disqus.

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        WebSci '22: Proceedings of the 14th ACM Web Science Conference 2022
        June 2022
        479 pages
        ISBN:9781450391917
        DOI:10.1145/3501247
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 26 June 2022

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. COVID-19
        2. public health
        3. sentiment analysis
        4. tweet response generation
        5. vaccines

        Qualifiers

        • Research-article
        • Research
        • Refereed limited

        Funding Sources

        Conference

        WebSci '22
        Sponsor:
        WebSci '22: 14th ACM Web Science Conference 2022
        June 26 - 29, 2022
        Barcelona, Spain

        Acceptance Rates

        Overall Acceptance Rate 245 of 933 submissions, 26%

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)23
        • Downloads (Last 6 weeks)1
        Reflects downloads up to 13 Dec 2024

        Other Metrics

        Citations

        Cited By

        View all
        • (2023)Public health agencies’ use of social media for communication during pandemics: a scoping review of the literatureOsong Public Health and Research Perspectives10.24171/j.phrp.2023.009514:4(235-251)Online publication date: 31-Aug-2023
        • (2023)Perceiving the Ukraine-Russia Conflict: Topic Modeling and Clustering on Twitter Data2023 IEEE Ninth International Conference on Big Data Computing Service and Applications (BigDataService)10.1109/BigDataService58306.2023.00028(147-148)Online publication date: Jul-2023

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format.

        HTML Format

        Media

        Figures

        Other

        Tables

        Share

        Share

        Share this Publication link

        Share on social media