[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
survey

Frameworks for Collective Intelligence: A Systematic Literature Review

Published: 06 February 2020 Publication History

Abstract

Over the last few years, Collective Intelligence (CI) platforms have become a vital resource for learning, problem solving, decision-making, and predictions. This rising interest in the topic has to led to the development of several models and frameworks available in published literature. Unfortunately, most of these models are built around domain-specific requirements, i.e., they are often based on the intuitions of their domain experts and developers. This has created a gap in our knowledge in the theoretical foundations of CI systems and models, in general. In this article, we attempt to fill this gap by conducting a systematic review of CI models and frameworks, identified from a collection of 9,418 scholarly articles published since 2000. Eventually, we contribute by aggregating the available knowledge from 12 CI models into one novel framework and present a generic model that describes CI systems irrespective of their domains. We add to the previously available CI models by providing a more granular view of how different components of CI systems interact. We evaluate the proposed model by examining it with respect to six popular, ongoing CI initiatives available on the Web.

Supplementary Material

a14-suran-suppl.pdf (suran.zip)
Supplemental movie, appendix, image and software files for, Frameworks for Collective Intelligence: A Systematic Literature Review

References

[1]
Lisa R. Anderson and Charles A. Holt. 1997. Information cascades in the laboratory. The American Economic Review 87, 5 (1997), 847--862. http://www.jstor.org/stable/2951328.
[2]
Kirsimarja Blomqvist. 1997. The many faces of trust. Scandinavian Journal of Management 13, 3 (1997), 271--286.
[3]
Andre Boder. 2006. Collective intelligence: A keystone in knowledge management. Journal of Knowledge Management 10, 1 (2006), 81--93.
[4]
Eric Bonabeau. 2009. Decisions 2.0: The power of collective intelligence. MIT Sloan Management Review 50, 2 (Winter 2009), 45--52.
[5]
E. Bonabeau and C. Meyer. 2001. Swarm intelligence—A whole new way to think about business. Harvard Business Review 79, 5 (May 2001), 106+.
[6]
T. Bosse, C. M. Jonker, M. C. Schut, and J. Treur. 2006. Collective representational content for shared extended mind. Cognitive Systems Research 7, 2--3 (2006), 151--174.
[7]
Daren C. Brabham. 2008. Crowdsourcing as a model for problem solving: An introduction and cases. Convergence 14, 1 (2008), 75--90.
[8]
Daren C. Brabham. 2010. Moving the crowd at Threadless. Information, Communication 8 Society 13, 8 (2010), 1122--1145.
[9]
David Budgen and Pearl Brereton. 2006. Performing systematic literature reviews in software engineering. In Proceedings of the 28th International Conference on Software Engineering (ICSE’06). ACM, New York, NY, 1051--1052.
[10]
Romina Cachia, Ramón Compañó, and Olivier Da Costa. 2007. Grasping the potential of online social networks for foresight. Technological Forecasting and Social Change 74, 8 (2007), 1179--1203.
[11]
Kyung Jin Cha, Yang Sok Kim, Byeonghwa Park, and Choong Kwon Lee. 2015. Knowledge management technologies for collaborative intelligence: A study of case company in Korea. International Journal of Distributed Sensor Networks 11, 9 (2015), 368273.
[12]
Cary Cherniss. 2010. Emotional intelligence: Toward clarification of a concept. Industrial and Organizational Psychology 3, 2 (2010), 110--126.
[13]
Prerna Chikersal, Maria Tomprou, Young Ji Kim, Anita Williams Woolley, and Laura Dabbish. 2017. Deep structures of collaboration: Physiological correlates of collective intelligence and group satisfaction. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW’17). ACM, New York, NY, 873--888.
[14]
R. I. Damper. 2000. Editorial for the special issue on ‘Emergent Properties of Complex Systems’: Emergence and levels of abstraction. International Journal of Systems Science 31, 7 (2000), 811--818.
[15]
Miguel de Castro Neto and Ana Espírtio Santo. 2012. Emerging collective intelligence business models. In MCIS 2012 Proceedings. Mediterranean Conference on Information Systems. https://aisel.aisnet.org/mcis2012/14.
[16]
Tony Diggle. 2013. Water: How collective intelligence initiatives can address this challenge. Foresight 15, 5 (2013), 342--353.
[17]
Anhai Doan, Raghu Ramakrishnan, and Alon Y. Halevy. 2011. Crowdsourcing systems on the World-Wide Web. Communications of the ACM 54, 4 (2011), 86.
[18]
Ketry Gorete Farias dos Passos and Edna Lúcia da Silva. 2012. Effect of collective intelligence in organizations. Transinformação 24, 2 (May 2012), 127--136.
[19]
Vasco Furtado, Leonardo Ayres, Marcos de Oliveira, Eurico Vasconcelos, Carlos Caminha, Johnatas D’Orleans, and Mairon Belchior. 2010. Collective intelligence in law enforcement—The WikiCrimes system. Information Sciences 180, 1 (2010), 4--17.
[20]
Jing Gao, Qi Li, Bo Zhao, Wei Fan, and Jiawei Han. 2015. Truth discovery and crowdsourcing aggregation: A unified perspective. PVLDB 8, 12 (2015), 2048--2049.
[21]
Sandro Georgi and Reinhard Jung. 2012. Collective intelligence model: How to describe collective intelligence. In Advances in Intelligent and Soft Computing. Vol. 113. Springer, 53--64.
[22]
Jim Giles. 2005. Internet encyclopaedias go head to head. Nature 438, 7070 (Dec. 2005), 900--901.
[23]
Jerome C. Glenn. 2013. Collective intelligence and an application by the millennium project. World Futures Review 5, 3 (2013), 235--243.
[24]
Vijay Govindarajan and Chris Trimble. 2011. The CEO’s role in business model reinvention. Harvard Business Review 89, 1--2 (2011), 108--14.
[25]
Antonietta Grasso and Gregorio Convertino. 2012. Collective intelligence in organizations: Tools and studies. Computer Supported Cooperative Work (CSCW) 21, 4 (01 Oct 2012), 357--369.
[26]
Dawn Gregg. 2009. Developing a collective intelligence application for special education. Decision Support Systems 47, 4 (2009), 455--465.
[27]
Dawn G. Gregg. 2010. Designing for collective intelligence. Communications of the ACM 53, 4 (April 2010), 134--138.
[28]
Tom Gruber. 2008. Collective knowledge systems: Where the social web meets the semantic web. Web Semantics: Science, Services and Agents on the World Wide Web 6, 1 (2008), 4--13.
[29]
Friedrich August Hayek. 1945. The use of knowledge in society. The American Economic Review 35, 4 (1945), 519--530.
[30]
Sylvia Ann Hewlett, Melinda Marshall, and Laura Sherbin. 2013. How diversity can drive innovation. Harvard Business Review 91, 12 (2013), 30--30.
[31]
Lu Hong and Scott E. Page. 2004. Groups of diverse problem solvers can outperform groups of high-ability problem solvers. Proceedings of the National Academy of Sciences 101, 46 (2004), 16385--16389.
[32]
Luca Iandoli. 2009. Leveraging the power of collective intelligence through IT-enabled global collaboration. Journal of Global Information Technology Management 12, 3 (2009), 1--6.
[33]
Luca Iandoli, Mark Klein, and Giuseppe Zollo. 2009. Enabling on-line deliberation and collective decision-making through large-scale argumentation. International Journal of Decision Support System Technology 1, 1 (Jan. 2009), 69--92.
[34]
Joshua Introne, Robert Laubacher, Gary Olson, and Thomas Malone. 2011. The climate CoLab: Large scale model-based collaborative planning. In Proceedings of the 2011 International Conference on Collaboration Technologies and Systems (CTS’11). 40--47.
[35]
Caroline Just. 2011. A review of literature on the general factor of personality. Personality and Individual Differences 50, 6 (2011), 765--771.
[36]
Stuart A. Kauffman. 1993. The Origins of Order: Self-organization and Selection in Evolution. Oxford University Press.
[37]
Barbara A. Kitchenham and S. Charters. 2007. Guidelines for Performing Systematic Literature Reviews in Software Engineering. Technical Report EBSE-2007-01. Keele University.
[38]
Gary A Klein. 2004. The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work. Crown Business.
[39]
A. Kornrumpf and U. Baumöl. 2014. A design science approach to collective intelligence systems. In 2014 47th Hawaii International Conference on System Sciences. 361--370.
[40]
Evangelos Kosmidis et al. 2018. hackAIR: Towards raising awareness about air quality in Europe by developing a collective online platform. ISPRS International Journal of Geo-Information 7, 5 (2018).
[41]
Hélène Landemore and Jon Elster. 2012. Collective Wisdom: Principles and Mechanisms. Cambridge University Press.
[42]
Vita Lanfranchi and Neil Ireson. 2009. User requirements for a collective intelligence emergency response system. In Proceedings of the 23rd British HCI Group Annual Conference on People and Computers: Celebrating People and Technology (BCS-HCI’09). British Computer Society, Swinton, UK, 198--203. http://dl.acm.org/citation.cfm?id=1671011.1671035.
[43]
Pierre Levy. 1997. Collective Intelligence: Mankind’s Emerging World in Cyberspace. Perseus Books, Cambridge, MA.
[44]
Pierre Lévy. 2010. From social computing to reflexive collective intelligence: The IEML research program. Information Sciences 180, 1 (2010), 71--94.
[45]
Randall M. Livingstone. 2016. Models for understanding collective intelligence on wikipedia. Social Science Computer Review 34, 4 (Aug. 2016), 497--508.
[46]
Jan Lorenz, Heiko Rauhut, Frank Schweitzer, and Dirk Helbing. 2011. How social influence can undermine the wisdom of crowd effect. Proceedings of the National Academy of Sciences 108, 22 (2011), 9020--9025.
[47]
Shuangling Luo, Haoxiang Xia, Taketoshi Yoshida, and Zhongtuo Wang. 2009. Toward collective intelligence of online communities: A primitive conceptual model. Journal of Systems Science and Systems Engineering 18, 2 (01 June 2009), 203--221.
[48]
Ioanna Lykourentzou, Dimitrios J. Vergados, Epaminondas Kapetanios, and Vassili Loumos. 2011. Collective intelligence systems: Classification and modeling. Journal of Emerging Technologies in Web Intelligence 3, 3 (Aug. 2011).
[49]
Ioanna Lykourentzou, Dimitrios J. Vergados, and Vassili Loumos. 2009. Collective intelligence system engineering. In Proceedings of the International Conference on Management of Emergent Digital EcoSystems (MEDES’09). ACM, New York, NY, Article 20, 7 pages.
[50]
T. W. Malone and M. S. Bernstein. 2015. Handbook of Collective Intelligence. MIT Press. 2015029234 https://books.google.ee/books?id=Px3iCgAAQBAJ.
[51]
Thomas W. Malone and Mark Klein. 2007. Harnessing collective intelligence to address global climate change. Innovations: Technology, Governance, Globalization 2, 3 (2007), 15--26.
[52]
Thomas W. Malone, Robert Laubacher, and Chrysanthos N. Dellarocas. 2009. Harnessing crowds: Mapping the genome of collective intelligence. SSRN Electronic Journal (2009).
[53]
Kurt Matzler, Andreas Strobl, and Franz Bailom. 2016. Leadership and the wisdom of crowds: How to tap into the collective intelligence of an organization. Strategy 8 Leadership 44, 1 (2016), 30--35. arXiv:https://doi.org/10.1108/SL-06-2015-0049
[54]
Gavin McCrory, Carina Veeckman, and Laurence Claeys. 2017. Citizen science is in the air—Engagement mechanisms from technology-mediated citizen science projects addressing air pollution. In Internet Science. Springer International Publishing, Cham, 28--38.
[55]
Lev Muchnik, Sinan Aral, and Sean J. Taylor. 2013. Social influence bias: A randomized experiment. Science 341, 6146 (2013), 647--651.
[56]
J. Musil, A. Musil, D. Weyns, and S. Biffl. 2015. An architecture framework for collective intelligence systems. In 2015 12th Working IEEE/IFIP Conference on Software Architecture (WICSA’15). 21--30.
[57]
Kimberly Nehls. 2015. Crowdsourcing. Learning, Media and Technology 40, 1 (2015), 123--126.
[58]
Van Du Nguyen and Ngoc Thanh Nguyen. 2018. Intelligent collectives: Theory, applications, and research challenges. Cybernetics and Systems 49, 5-6 (2018), 261--279.
[59]
Ikujiro Nonaka and Takeuchi Hirotaka. 1995. The Knowledge-Creating Company: How Japanese Companies Create the Dynamics of Innovation. Oxford University Press. 94040408 https://books.google.ee/books?id=B-qxrPaU1-MC.
[60]
Julio M. Ottino. 2004. Engineering complex systems. Nature 427, 6973 (2004), 399.
[61]
S. E. Page and Princeton University Press. 2007. The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies. Princeton University Press. 2006044678 https://books.google.ee/books?id=FAFVHnJ7uK0C.
[62]
Chloe Parker. 2015. The IWA WaterWiki: An open-access publishing platform providing free content on all aspects of water, wastewater and the environment. Environment and Urbanization 27, 1 (2015), 137--138.
[63]
Michael A. Peters and Richard Heraud. 2015. Toward a political theory of social innovation: Collective intelligence and the co-creation of social goods. 3, 3 (2015), 7--23. https://researchcommons.waikato.ac.nz/handle/10289/9569.
[64]
Rolf Pfeifer, Jan Henrik Sieg, Thierry Bücheler, and Rudolf Marcel Füchslin. 2010. Crowdsourcing, open innovation and collective intelligence in the scientific method: A research agenda and operational framework. (2010).
[65]
P. Clint Rogers, Stephen W. Liddle, Peter Chan, Aaron Doxey, and Brady Isom. 2007. WEB 2.0 learning platform: Harnessing collective intelligence. Turkish Online Journal of Distance Education 8, 3 (2007), 16--33. http://tojde.anadolu.edu.tr/makale_goster.php?id=348.
[66]
Frantz Rowe. 2014. What literature review is not: Diversity, boundaries and recommendations. European Journal of Information Systems 23, 3 (01 May 2014), 241--255.
[67]
Juho Salminen. 2015. The Role of Collective Intelligence in Crowdsourcing Innovation. PhD dissertation. Lappeenranta University of Technology.
[68]
H. Santos, L. Ayres, C. Caminha, and V. Furtado. 2012. Open government and citizen participation in law enforcement via crowd mapping. IEEE Intelligent Systems 27 (2012), 63--69.
[69]
Detlef Schoder, Peter A. Gloor, and Panagiotis Takis Metaxas. 2013. Social media and collective intelligence—Ongoing and future research streams. KI - Künstliche Intelligenz 27, 1 (1 Feb. 2013), 9--15.
[70]
Martijn C. Schut. 2010. On model design for simulation of collective intelligence. Information Sciences 180, 1 (2010), 132--155. Special Issue on Collective Intelligence.
[71]
Toby Segaran. 2007. Programming Collective Intelligence : Building Smart Web 2.0 Applications. O’Reilly, Beijing/Sebastapol, CA.
[72]
V. Singh, G. Singh, and S. Pande. 2013. Emergence, self-organization and collective intelligence—Modeling the dynamics of complex collectives in social and organizational settings. In 2013 UKSim 15th International Conference on Computer Modelling and Simulation. 182--189.
[73]
Aelita Skarzauskiene and Monika Maciuliene. 2015. Modelling the index of collective intelligence in online community projects. In International Conference on Cyber Warfare and Security. Academic Conferences International Limited, 313.
[74]
Daniel Stieger, Kurt Matzler, Sayan Chatterjee, and Florian Ladstaetter-Fussenegger. 2012. Democratizing strategy: How crowdsourcing can be used for strategy dialogues. California Management Review 54, 4 (2012), 44--68.
[75]
Tim Sullivan. 2014. Embracing Complexity. Retrieved on March 8, 2019 from https://hbr.org/2011/09/embracing-complexity.
[76]
Shweta Suran, V. Pattanaik, Sadok B. Yahia, and Dirk Draheim. 2019. Exploratory analysis of collective intelligence projects developed within the EU-Horizon 2020 framework. In Computational Collective Intelligence. Springer International Publishing, 285--296.
[77]
James Surowiecki. 2005. The Wisdom of Crowds. Anchor.
[78]
Guy Theraulaz and Eric Bonbeau. 1999. A brief history of stigmergy. Artificial Life 5, 2 (April 1999), 97--116.
[79]
O’Reilly Tim. 2007. What Is Web 2.0: Design Patterns and Business Models for the Next Generation of Software. MPRA Paper. University Library of Munich, Germany. https://EconPapers.repec.org/RePEc:pra:mprapa:4578.
[80]
John P. van Gigch. 1991. System Design Modeling and Metamodeling. Springer US.
[81]
Dimitrios J. Vergados, Ioanna Lykourentzou, and Epaminondas Kapetanios. 2010. A resource allocation framework for collective intelligence system engineering. In Proceedings of the International Conference on Management of Emergent Digital EcoSystems (MEDES’10). ACM, New York, NY, 182--188.
[82]
Jennifer H. Watkins. 2007. Prediction markets as an aggregation mechanism for collective intelligence. Retrieved from https://escholarship.org/uc/item/8mg0p0zc.
[83]
Sean Wise, Robert A. Paton, and Thomas Gegenhuber. 2012. Value co-creation through collective intelligence in the public sector: A review of US and European initiatives. VINE 42, 2 (2012), 251--276.
[84]
Michael A. Woodley and Edward Bell. 2011. Is collective intelligence (mostly) the General Factor of Personality? A comment on Woolley, Chabris, Pentland, Hashmi and Malone (2010). Intelligence 39, 2 (2011), 79--81.
[85]
Anita Williams Woolley, Christopher F. Chabris, Alex Pentland, Nada Hashmi, and Thomas W. Malone. 2010. Evidence for a collective intelligence factor in the performance of human groups. Science 330, 6004 (2010), 686--688.
[86]
Koji Zettsu and Yasushi Kiyoki. 2006. Towards knowledge management based on harnessing collective intelligence on the web. In Managing Knowledge in a World of Networks. Springer, Berlin, 350--357.

Cited By

View all
  • (2025)Federated Learning in Agents Based Cyber-Physical SystemsArtificial Intelligence Techniques for Analysing Sensitive Data in Medical Cyber-Physical Systems10.1007/978-3-031-70775-9_5(73-94)Online publication date: 23-Jan-2025
  • (2024)Collaborative Motivation Framework Leveraging Similar Interest Behavior in Semantic Web ApplicationsInternational Journal on Semantic Web & Information Systems10.4018/IJSWIS.36591221:1(1-33)Online publication date: 10-Dec-2024
  • (2024)Collective Intelligence for Democracy: Empowering Minorities and Everyone in Participatory BudgetingSSRN Electronic Journal10.2139/ssrn.4832498Online publication date: 2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Computing Surveys
ACM Computing Surveys  Volume 53, Issue 1
January 2021
781 pages
ISSN:0360-0300
EISSN:1557-7341
DOI:10.1145/3382040
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 February 2020
Accepted: 01 October 2019
Revised: 01 October 2019
Received: 01 March 2019
Published in CSUR Volume 53, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Collective intelligence
  2. Web 2.0
  3. crowdsourcing
  4. human computer interaction
  5. systematic literature review
  6. wisdom of crowds

Qualifiers

  • Survey
  • Survey
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)476
  • Downloads (Last 6 weeks)39
Reflects downloads up to 28 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Federated Learning in Agents Based Cyber-Physical SystemsArtificial Intelligence Techniques for Analysing Sensitive Data in Medical Cyber-Physical Systems10.1007/978-3-031-70775-9_5(73-94)Online publication date: 23-Jan-2025
  • (2024)Collaborative Motivation Framework Leveraging Similar Interest Behavior in Semantic Web ApplicationsInternational Journal on Semantic Web & Information Systems10.4018/IJSWIS.36591221:1(1-33)Online publication date: 10-Dec-2024
  • (2024)Collective Intelligence for Democracy: Empowering Minorities and Everyone in Participatory BudgetingSSRN Electronic Journal10.2139/ssrn.4832498Online publication date: 2024
  • (2024)Conversational AI as a Facilitator Improves Participant Engagement and Problem-Solving in Online Discussion: Sharing Evidence from Five Cities in AfghanistanIEICE Transactions on Information and Systems10.1587/transinf.2023IHP0014E107.D:4(434-442)Online publication date: 1-Apr-2024
  • (2024)The utility of indigenous knowledge in the pursuit of a UN sustainable development goalCollective Intelligence10.1177/263391372412412953:2Online publication date: 6-May-2024
  • (2024)A Sustainable Collaborative Talent Management Through Collaborative Intelligence Mindset Theory: A Systematic ReviewSage Open10.1177/2158244024126185114:2Online publication date: 17-Jun-2024
  • (2024)Tripartite Intelligence: Synergizing Deep Neural Network, Large Language Model, and Human Intelligence for Public Health Misinformation Detection (Archival Full Paper)Proceedings of the ACM Collective Intelligence Conference10.1145/3643562.3672613(63-75)Online publication date: 27-Jun-2024
  • (2024)Empowering minorities and everyone in participatory budgeting: an agent-based modelling perspectivePhilosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences10.1098/rsta.2024.0090382:2285Online publication date: 13-Nov-2024
  • (2024)Confronting Fake News Through Collective Intelligence in AdolescentsInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2405567(1-14)Online publication date: 2-Oct-2024
  • (2024)Combining Multiple Large Language Models Improves Diagnostic AccuracyNEJM AI10.1056/AIcs24005021:11Online publication date: 24-Oct-2024
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media