More Web Proxy on the site http://driver.im/

research-article

Open access

Just Accepted

Comparative Analysis of ChatGPT and Bard in Digital Governance: Accuracy, Adaptability, and Readability Insights

Authors:

Ramana AcharyuluAuthors Info & Claims

Digital Government: Research and Practice

Accepted on 26 May 2024

https://doi.org/10.1145/3674846

Online AM: 24 June 2024 Publication History

Abstract

In a comprehensive assessment of ChatGPT and Bard's performance across three key indices—Government AI Readiness, Digital Economy and Society, and UN E-Government Survey, the study delves into nuanced insights regarding their accuracy, adaptability, and readability within the context of Digital Governance. ChatGPT demonstrated a superior accuracy rate of 93.55%, surpassing Bard's performance at 88.57%. Notably, both models exhibited variations in individual and mutual error correction capabilities, particularly evident when faced with confirmation queries. Bard showcased an adjustment post-confirmation, suggesting potential error correction, whereas ChatGPT displayed limited adaptability in similar scenarios. While there was a notable congruence in their responses to Digital Governance content, challenges arose in deciphering complex information, especially concerning sustainability initiatives. Bard generally produced more accessible content, evident in readability metrics, in contrast to ChatGPT's inclination towards using complex language. Both models demonstrated promising alignment in addressing intricate topics within the realm of Digital Governance. The findings emphasize the need for policymakers to critically evaluate the adaptability and accuracy of language models like ChatGPT and Bard when considering their integration into digital governance practices. Awareness of their diverse performance and error correction capabilities is crucial for responsible implementation, ensuring the maximal benefits of AI in public decision-making.

References

[1]

Androutsopoulou, A., Karacapilidis, N., Loukis, E., & Charalabidis, Y. 2019. Transforming the communication between citizens and government through AI-guided chatbots. Government Information Quarterly, 36(2), 358-367.

[2]

Barber, B. 1983. The logic and limits of trust.

[3]

Bernd W. Wirtz, Jan C. Weyerer & Carolin Geyer. 2019. Artificial Intelligence and the Public Sector—Applications and Challenges, International Journal of Public Administration, 42(7), 596-615.

[4]

Bruce, D., Fadia, A., Isherwood, T., Marcati, C., Mitchell, A., Münstermann, B., Shenai, G., Vuppala, H. and Weber, T. 2023. Unlocking the potential of generative AI: Three key questions for government agencies. McKinsey's Public Sector Practice.

[5]

Carrasco, M., Habib, C., Felden, F., Sargeant, R., Mills, S., Shenton, S., Ingram, J., and Dando, G. 2023. Generative AI for the Public Sector: From Opportunities to Value. Boston Consulting Group.

[6]

Cascella, M., Montomoli, J., Bellini, V., & Bignami, E. 2023. Evaluating the feasibility of ChatGPT in healthcare: An analysis of multiple clinical and research scenarios. Journal of Medical Systems, 47(1), 1–5.

[7]

Dadkhah, M., Oermann, M. H., Mihály, H., Raman, R., & Dávid, L. D. (2024). Diagnosis Unreliability of ChatGPT for Journal Evaluation.

[8]

Dasborough, M. T. 2023. Awe-inspiring advancements in AI: The impact of ChatGPT on the field of organizational behavior. Journal of Organizational Behavior, 44(2), 177–179.

[9]

Devis Bianchini, Carlo Bono, Alessandro Campi, Cinzia Cappiello, Stefano Ceri, Francesca De Luzi, Massimo Mecella, Barbara Pernici, and Pierluigi Plebani. 2023. Challenges in AI-supported process analysis in the Italian judicial system: what after digitalization? Commentary paper. Digit. Gov.: Res. Pract.

[10]

Dhoni PS, Kumar, R. 2023. Synergizing Generative Artificial Intelligence (AI) and Cybersecurity: Roles of Generative AI Entities, Companies, Agencies, and Government in Enhancing Cybersecurity. J Glob Res Comput Sci, 14(4).

[11]

Fiona Fui-Hoon Nah, Ruilin Zheng, Jingyuan Cai, Keng Siau & Langtao Chen. 2023. Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration, Journal of Information Technology Case and Application Research, 25:3, 277-304.

[12]

G. Misuraca, G., Medaglia, R., & Aquaro, V. 2021, October. Re-designing the UN e-Government Survey in light of the 2030 Agenda for Sustainable Development: Towards a post-COVID digital society. In Proceedings of the 14th International Conference on Theory and Practice of Electronic Governance (pp. 198-202)

Digital Library

[13]

Geertsema, P., Bifet, A., & Green, R. 2023. ChatGPT and Large Language Models: What are the Implications for Policy Makers?. Available at SSRN 4424048.

[14]

Hyder, Z., Siau, K., & Nah, F. F. H. 2019. Artificial intelligence, machine learning, and autonomous technologies in mining industry. Journal of Database Management, 30(2), 67–79.

[15]

Ismailova, R., Inal, Y. 2017. Web site accessibility and quality in use: a comparative study of government Web sites in Kyrgyzstan, Azerbaijan, Kazakhstan and Turkey. Univ. Access Inf. Soc. 16(4), 987–996

Digital Library

[16]

Jain, S. K. 2022, December. Comparative Study of United Nations E-Government Indicators Between World Leaders and India (Measuring Digital India). In 2022 5th International Conference on Advances in Science and Technology (ICAST) (pp. 662-669). IEEE.

[17]

Jefferies, Duncan. 2016. The automated city: Do we still need humans to run public services? Edited by The Guardian.

[18]

Jindal, P., & MacDermid, J. C. (2017). Assessing reading levels of health information: uses and limitations of flesch formula. Education for Health, 30(1), 84-88.

[19]

Kaplan, A., & Haenlein, M. 2019. Siri, Siri, in my hand: Who's the fairest in the land? On the interpretations, illustrations, and implications of artificial intelligence. Business Horizons, 62 (1), 15–25.

[20]

Kapoor, N. J., Maurya, A. P., Raman, R., Govind, K., & Nedungadi, P. 2021. Digital India eGovernance Initiative for Tribal Empowerment: Performance Dashboard of the Ministry of Tribal Affairs. In Information and Communication Technology for Competitive Strategies (ICTCS 2020) Intelligent Strategies for ICT (pp. 1017-1027). Springer Singapore.

[21]

Kasabwala, K., Agarwal, N., Hansberry, D. R., Baredes, S., & Eloy, J. A. 2012. Readability assessment of patient education materials from the American Academy of Otolaryngology—Head and Neck Surgery Foundation. Otolaryngology–Head and Neck Surgery, 147(3), 466-471.

[22]

Lee, J., Moray, N. 1992. Trust, control strategies, and allocation of function in human-machine systems. Ergonomics, 35(10), 1243-70.

[23]

Liu, T. C. 2022. Digital policy in European countries from the perspective of the Digital Economy and Society Index. Policy & Internet, 14(1), 202-218.

[24]

Longoni, C., Fradkin, A., Cian, L., Pennycook, G. 2022. News from Generative Artificial Intelligence Is Believed Less. FAccT '22: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, June 2022, Pages 97–106

Digital Library

[25]

Loukis, E., Saxena, S., Rizun, N., Maratsi, M.I., Ali, M., Alexopoulos, C. 2023. ChatGPT Application vis-a-vis Open Government Data (OGD): Capabilities, Public Values, Issues and a Research Agenda. In: Lindgren, I., et al. Electronic Government. EGOV 2023. Lecture Notes in Computer Science, vol 14130. Springer, Cham.

[26]

Nedungadi, P.P., Menon, R., Gutjahr, G., Erickson, L. and Raman, R. 2018. Towards an inclusive digital literacy framework for digital India, Education + Training, Vol. 60 No. 6, pp. 516-528.

[27]

Oxford Insights. 2023. Government AI Readiness Index. https://oxfordinsights.com/ai-readiness/ai-readiness-index/

[28]

Patnaik, S. S., & Hoffmann, U. 2023. Comparison of ChatGPT vs. Bard to Anesthesia-related Queries. medRxiv, 2023-06.

[29]

Paul, S. 2023. Accessibility analysis using WCAG 2.1: evidence from Indian e-government websites. Univ Access Inf Soc. 22, 663–669.

Digital Library

[30]

Raman, R. (2023). Transparency in research: an analysis of ChatGPT usage acknowledgment by authors across disciplines and geographies. Accountability in Research, 1-22.

[31]

Raman, R., Calyam, P., & Achuthan, K. (2024b). ChatGPT or Bard: Who is a better Certified Ethical Hacker?. Computers & Security, 103804.

[32]

Raman, R., Venugopalan, M., & Kamal, A (2024a). Evaluating human resources management literacy: A performance analysis of ChatGPT and bard. Heliyon.

[33]

Rempel, J. K., Holmes, J. G., & Zanna, M. P. 1985. Trust in close relationships. Journal of Personality and Social Psychology, 49(1), 95–112.Romberg, J. and Escher, T. 2023. Making Sense of Citizens' Input through Artificial Intelligence: A Review of Methods for Computational Text Analysis to Support the Evaluation of Contributions in Public Participation. Digital Government: Research and Practice.

[34]

Safaei, M., & Longo, J. 2021. The End of the Policy Analyst? Testing the Capability of Artificial Intelligence to Generate Plausible, Persuasive, and Useful Policy Analysis. Digital Government: Research and Practice.

[35]

Salah, M., Abdelfattah, F., & Al Halbusi, H. 2023. Generative Artificial Intelligence (ChatGPT & Bard) in Public Administration Research: A Double-Edged Sword for Street-Level Bureaucracy Studies. International Journal of Public Administration, 1-7.

[36]

Singh, S. K., Kumar, S., & Mehra, P. S. 2023, June. Chat GPT & Google Bard AI: A Review. In 2023 International Conference on IoT, Communication and Automation Technology (ICICAT) (pp. 1-6). IEEE.

[37]

Titus, L. M. 2024. Does ChatGPT have semantic understanding? A problem with the statistics-of-occurrence strategy. Cognitive Systems Research, 83, 101174.

Digital Library

[38]

Woolf, M. 2022, December 8. ChatGPT passes the turing test. Metaverse Post.

Cited By

Adel AAhsan ADavison C(2024)ChatGPT Promises and Challenges in Education: Computational and Ethical PerspectivesEducation Sciences10.3390/educsci1408081414:8(814)Online publication date: 25-Jul-2024
https://doi.org/10.3390/educsci14080814
Raman RSreenivasan ASuresh MGunasekaran ANedungadi P(2024)AI-driven education: a comparative study on ChatGPT and Bard in supply chain management contextsCogent Business & Management10.1080/23311975.2024.241274211:1Online publication date: 18-Oct-2024
https://doi.org/10.1080/23311975.2024.2412742

Recommendations

Promoting social participation through digital governance: identifying barriers in the brazilian public administration
dg.o '18: Proceedings of the 19th Annual International Conference on Digital Government Research: Governance in the Data Age

Public organizations are expanding their use of the democratic potential of information and communication technologies to promote the engagement of civil society in their organizational processes. Therefore, it is possible to perceive the transformation ...
Digital Governance and Individuals' Empowerment: a Conceptual Framework and Some Preliminary Results
ICEGOV '17: Proceedings of the 10th International Conference on Theory and Practice of Electronic Governance

Adoption of information and communication technologies (ICT) in governance contexts is considered empowering for governments and the citizens. Digital governance aims at transforming internal processes, structures and working practices leading to ...
Individual Empowerment through Digital Governance: Proposing an Assessment Framework for India
ICEGOV '17: Proceedings of the Special Collection on eGovernment Innovations in India

Adoption of information and communication technologies (ICT) in governance contexts is considered empowering for governments and citizens. Digital Governance aims at transforming internal processes, structures and working practices leading to greater ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Digital Government: Research and Practice

Digital Government: Research and Practice Just Accepted

EISSN:2639-0175

Table of Contents

Copyright © 2024 Copyright held by the owner/author(s).

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Online AM: 24 June 2024

Accepted: 26 May 2024

Revision received: 16 March 2024

Received: 04 January 2024

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
347
Total Downloads

Downloads (Last 12 months)347
Downloads (Last 6 weeks)65

Reflects downloads up to 12 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Adel AAhsan ADavison C(2024)ChatGPT Promises and Challenges in Education: Computational and Ethical PerspectivesEducation Sciences10.3390/educsci1408081414:8(814)Online publication date: 25-Jul-2024
https://doi.org/10.3390/educsci14080814
Raman RSreenivasan ASuresh MGunasekaran ANedungadi P(2024)AI-driven education: a comparative study on ChatGPT and Bard in supply chain management contextsCogent Business & Management10.1080/23311975.2024.241274211:1Online publication date: 18-Oct-2024
https://doi.org/10.1080/23311975.2024.2412742

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables