[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
research-article

Knowledge Graphs in Practice: Characterizing their Users, Challenges, and Visualization Opportunities

Published: 01 January 2024 Publication History

Abstract

This study presents insights from interviews with nineteen Knowledge Graph (KG) practitioners who work in both enterprise and academic settings on a wide variety of use cases. Through this study, we identify critical challenges experienced by KG practitioners when creating, exploring, and analyzing KGs that could be alleviated through visualization design. Our findings reveal three major personas among KG practitioners – KG Builders, Analysts, and Consumers – each of whom have their own distinct expertise and needs. We discover that KG Builders would benefit from schema enforcers, while KG Analysts need customizable query builders that provide interim query results. For KG Consumers, we identify a lack of efficacy for node-link diagrams, and the need for tailored domain-specific visualizations to promote KG adoption and comprehension. Lastly, we find that implementing KGs effectively in practice requires both technical and social solutions that are not addressed with current tools, technologies, and collaborative workflows. From the analysis of our interviews, we distill several visualization research directions to improve KG usability, including knowledge cards that balance digestibility and discoverability, timeline views to track temporal changes, interfaces that support organic discovery, and semantic explanations for AI and machine learning predictions.

References

[1]
B. Abu-Salih. Domain-specific knowledge graphs: A survey. J. Netw. Comput. Appl., 185: pp. 103076, 2021. 1, 8.
[2]
S. Ahmad, D. Sessler, and J. Kohlhammer. Towards a comprehensive cohort visualization of patients with inflammatory bowel disease. In Proc. VAHC, pp. 25–29. IEEE Computer Society, Los Alamitos, 2021. 2.
[3]
B. AlKhamissi, M. Li, A. Celikyilmaz, M. Diab, and M. Ghazvininejad. A review on language models as knowledge bases. arXiv preprint arXiv:, 2022. 2.
[4]
S. Alspaugh, N. Zokaei, A. Liu, C. Jin, and M. A. Hearst. Futzing and moseying: Interviews with professional data analysts on exploration practices. IEEE Trans. Vis. Comput. Graph., 25(1): pp. 22–31, 2019. 2.
[5]
M. Angelini, G. Santucci, H. Schumann, and H.-J. Schulz. A review and characterization of progressive visual analytics. Informatics, 5(3), 2018. 6.
[6]
R. Angles, A. Bonifati, S. Dumbrava, G. Fletcher, A. Green, J. Hidders, B. Li, L. Libkin, V. Marsault, W. Martens, F. Murlak, S. Plantikow, O. Savkovic, M. Schmidt, J. Sequeda, S. Staworko, D. Tomaszuk, H. Voigt, D. Vrgoc, M. Wu, and D. Zivkovic. Pg-schema: Schemas for property graphs. in Proc. ACM Manag. Data, 1(2), 2023. 9.
[7]
S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, and Z. Ives. “Dbpedia: A nucleus for a web of open data”. In K. Aberer, K.-S. Choi, N. Noy, D. Allemang, K.-I. Lee, L. Nixon, J. Golbeck, P. Mika, D. Maynard, R. Mizoguchi, G. Schreiber, and P. Cudré-Mauroux, eds., Semant. Web, pp. 722–735. Berlin, Heidelberg: Springer Berlin Heidelberg, 2007. 2.
[8]
J. Aurisano, A. Kumar, A. Gonzales, J. Leigh, B. DiEugenio, and A. Johnson. Articulate 2: Toward a conversational interface for visual data exploration. In Proc. VIS, 2016. 8.
[9]
M. Bastian, S. Heymann, and M. Jacomy. Gephi: An open source software for exploring and manipulating networks. in Proc. AAAI, 3(1): pp. 361–362, 2009. 4, 6.
[10]
T. Berners-Lee, J. Hendler, and O. Lassila. The semantic web. Sci. Am., 284(5): pp. 34–43, 2001. 2.
[11]
P. A. Bonatti, S. Decker, A. Polleres, and V. Presutti. Knowledge Graphs: New Directions for Knowledge Representation on the Semantic Web (Dagstuhl Seminar 18371). Dagstuhl Reports, 8(9): pp. 29–111, 2019. 2.
[12]
M. Bostock, V. Ogievetsky, and J. Heer. D3 data-driven documents. IEEE Trans. Vis. Comput. Graph., 17(12): pp. 2301–2309, 2011. 4.
[13]
V. Braun and V. Clarke. Using thematic analysis in psychology. Qual. Res. Psychol., 3(2): pp. 77–101, 2006. 3.
[14]
M. Brehmer, B. Lee, B. Bach, N. H. Riche, and T. Munzner. Timelines revisited: A design space and considerations for expressive storytelling. IEEE Trans. Vis. Comput. Graph., 23(9): pp. 2151–2164, 2017. 8.
[15]
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei. “Language models are few-shot learners”. In Adv. Neural Inf., vol. 33, pp. 1877–1901. Curran Associates, Inc., 2020. 2.
[16]
D. Cashman, S. Xu, S. Das, F. Heimerl, C. Liu, S. R. Humayoun, M. Gleicher, A. Endert, and R. Chang. Cava: A visual analytics system for exploratory columnar data augmentation using knowledge graphs. IEEE Trans. Vis. Comput. Graph., 27(2): pp. 1731–1741, 2021. 2, 8, 9.
[17]
B. Chan, L. Wu, P. Hanrahan, J. Talbot, and M. Cammarano. Vispedia: Interactive visual exploration of wikipedia data via search-based integration. IEEE Trans. Vis. Comput. Graph., 14(06): pp. 1213–1220, 2008. 2.
[18]
X. Chen, S. Jia, and Y. Xiang. A review: Knowledge reasoning over knowledge graph. Expert Syst. Appl., 141: pp. 112948, 2020. 9.
[19]
P. Cimiano and H. Paulheim. Knowledge graph refinement: A survey of approaches and evaluation methods. Semant. Web, 8(3): pp. 489–508, 2017. 5.
[20]
J. T. DeCuir-Gunby, P. L. Marshall, and A. W. McCulloch. Developing and using a codebook for the analysis of interview data: An example from a professional development research project. Field Methods, 23(2): pp. 136–155, 2011. 3.
[21]
L. Ehrlinger and W. Wöß. Towards a definition of knowledge graphs. in Proc. ESWC Posters and Demos Track, 48(1–4): pp. 2, 2016. 2.
[22]
B. Ell, A. Harth, and E. Simperl. Sparql query verbalization for explaining semantic search engine queries. In V. Presutti, C. d'Amato, F. Gandon, M. d'Aquin, S. Staab, and A. Tordai, eds., Proc. ESWC, pp. 426–441. Springer International Publishing, Cham, 2014. 2.
[23]
S. Ferré. Sparklis: An expressive query builder for sparql endpoints with guidance in natural language. Semant. Web, 8(3): pp. 405–418, 2017. 8.
[24]
N. Francis, A. Green, P. Guagliardo, L. Libkin, T. Lindaaker, V. Marsault, S. Plantikow, M. Rydberg, P. Selmer, and A. Taylor. Cypher: An evolving query language for property graphs. In Proc. SIGMOD, pp. 1433–1445. ACM, New York, 2018. 2.
[25]
A. Gal. Uncertain entity resolution: Re-evaluating entity resolution in the big data era: Tutorial. in Proc. VLDB Endow., 7(13): pp. 1711–1712, 2014. 5.
[26]
Y. Gan, X. Chen, J. Xie, M. Purver, J. R. Woodward, J. Drake, and Q. Zhang. Natural SQL: Making SQL easier to infer from natural language specifications. In Proc. EMNLP, pp. 2030–2042. ACL, Punta Cana, Dominican Republic, 2021. 8.
[27]
T. Gao, M. Dontcheva, E. Adar, Z. Liu, and K. G. Karahalios. Datatone: Managing ambiguity in natural language interfaces for data visualization. In Proc. UIST, pp. 489–500. ACM, New York, 2015. 2.
[28]
H. Gibson, J. Faith, and P. Vickers. A survey of two-dimensional graph layout techniques for information visualisation. in Proc. InfoVis, 12(3–4): pp. 324–357, 2013. 6, 8.
[29]
J. Gómez-Romero, M. Molina-Solana, A. Oehmichen, and Y. Guo. Visualizing large knowledge graphs: A performance analysis. Future Gener. Comput. Syst., 89: pp. 224–238, 2018. 6.
[30]
S. Gottschalk and E. Demidova. Eventkg: A multilingual event-centric temporal knowledge graph. In Proc. ESWC, pp. 272–287. Springer, 2018. 2, 3, 8, 9.
[31]
P. Grafkin, M. Mironov, M. Fellmann, B. Lantow, K. Sandkuhl, and A. V. Smirnov. Sparql query builders: Overview and comparison. In BIR Workshops, pp. 255–274, 2016. 8.
[32]
Q. Guo, F. Zhuang, C. Qin, H. Zhu, X. Xie, H. Xiong, and Q. He. A survey on knowledge graph-based recommender systems. IEEE Trans. Knowl. Data Eng., 34(08): pp. 3549–3568, 2022. 1.
[33]
A. Hagberg, P. Swart, and D. S Chult. Exploring network structure, dynamics, and function using networkx. Technical report, Los Alamos National Lab. (LANL), Los Alamos, NM (United States), 2008. 4.
[34]
J. Han, H. E, G. Le, and J. Du. Survey on NoSQL database. In Proc. ICPCA, pp. 363–366, 2011. 8.
[35]
S. A. Helmers. Microsoft Visio 2016 Step By Step: MS Visio 2016 Ste by Ste_p1. Microsoft Press, 2015. 9.
[36]
E. Hemberg, J. Kelly, M. Shlapentokh-Rothman, B. Reinstadler, K. Xu, N. Rutar, and U.-M. O'Reilly. Linking threat tactics, techniques, and patterns with defensive weaknesses, vulnerabilities and affected platform configurations for cyber hunting. arXiv preprint arXiv:, 2020. 4.
[37]
I. Herman, G. Melancon, and M. Marshall. Graph visualization and navigation in information visualization: A survey. IEEE Trans. Vis. Comput. Graph., 6(1): pp. 24–43, 2000. 6, 9.
[38]
A. Hogan, E. Blomqvist, M. Cochez, C. D'amato, G. D. Melo, C. Gutierrez, S. Kirrane, J. E. L. Gayo, R. Navigli, S. Neumaier, A.-C. N. Ngomo, A. Polleres, S. M. Rashid, A. Rula, L. Schmelzeisen, J. Sequeda, S. Staab, and A. Zimmermann. Knowledge graphs. ACM Comput. Surv., 54(4), 2021. 1, 2, 5.
[39]
S. R. Hong, J. Hullman, and E. Bertini. Human factors in model interpretability: Industry practices, challenges, and needs. in Proc. CHI, 4(CSCW1), 2020. 2.
[40]
J. Huang, Y. Xi, J. Hu, and J. Tao. Flownl: Asking the flow data in natural languages. IEEE Trans. Vis. Comput. Graph., 29(1): pp. 1200–1210, 2023. 8.
[41]
X. Huang, J. Zhang, D. Li, and P. Li. Knowledge graph embedding based question answering. In Proc. WSDM, pp. 105–113. ACM, New York, 2019. 1.
[42]
F. Husain, R. Romero-Gomez, E. Kuang, D. Segura, A. Carolli, L. Liu, M. Cheung, and Y. Paris. A multi-scale visual analytics approach for exploring biomedical knowledge. In Proc. VAHC, pp. 30–35. IEEE Computer Society, Los Alamitos, 2021. 2.
[43]
S. Kandel, A. Paepcke, J. M. Hellerstein, and J. Heer. Enterprise data analysis and visualization: An interview study. IEEE Trans. Vis. Comput. Graph., 18(12): pp. 2917–2926, 2012. 2.
[44]
M. Kivelä, F. McGee, G. Melançon, N. H. Riche, and T. von Landesberger. Visual Analytics of Multilayer Networks Across Disciplines (Dagstuhl Seminar 19061). Dagstuhl Reports, 9(2): pp. 1–26, 2019. 2.
[45]
K. Klein, J. F. Sequeda, H.-Y. Wu, and D. Yan. Bringing Graph Databases and Network Visualization Together (Dagstuhl Seminar 22031). Dagstuhl Reports, 12(1): pp. 67–82, 2022. 2, 6.
[46]
S. Latif, S. Agarwal, S. Gottschalk, C. Chrosch, F. Feit, J. Jahn, T. Braun, Y. Tchenko, E. Demidova, and F. Beck. Visually connecting historical figures through event knowledge graphs. In Proc. VIS, pp. 156–160. IEEE Computer Society, Los Alamitos, 2021. 2, 3, 9.
[47]
F. Lecue. On the role of knowledge graphs in explainable AI. Semant. Web, 11(1): pp. 41–51, 2020. 1, 8.
[48]
B. Lee, C. Plaisant, C. S. Parr, J.-D. Fekete, and N. Henry. Task taxonomy for graph visualization. In Proc. BELIV, pp. 1–5. ACM, New York, 2006. 6, 9.
[49]
H. Li, Y. Wang, S. Zhang, Y. Song, and H. Qu. Kg4vis: A knowledge graph-based approach for visualization recommendation. IEEE Trans. Vis. Comput. Graph., 28(01): pp. 195–205, 2022. 2, 9.
[50]
M. Lissandrini, D. Mottin, K. Hose, and T. B. Pedersen. Knowledge graph exploration systems: are we lost? In CIDR, vol. 22, pp. 10–13, 2022. 2, 5, 9.
[51]
Z. Liu and J. Heer. The effects of interactive latency on exploratory visual analysis. IEEE Trans. Vis. Comput. Graph., 20(12): pp. 2122–2131, 2014. 6.
[52]
K. M. MacQueen, E. McLellan, K. Kay, and B. Milstein. Codebook development for team-based qualitative analysis. CAM j., 10(2): pp. 31–36, 1998. 3.
[53]
J. J. Miller. Graph database applications and concepts with Neo4J. in Proc. SAIS, 2324(36), 2013. 2.
[54]
R. Mitra, A. Narechania, A. Endert, and J. Stasko. Facilitating conversational interaction in natural language interfaces for visualization. In Proc. VIS, pp. 6–10, 2022. 8.
[55]
A. Mosca, S. Robinson, M. Clarke, R. Redelmeier, S. Coates, D. Cashman, and R. Chang. Defining an Analysis: A Study of Client-Facing Data Scientists. In J. Johansson, F. Sadlo, and G. E. Marai, eds., EuroVis 2019 - Short Papers. The Eurographics Association, 2019. 2.
[56]
A. Narechania, A. Srinivasan, and J. Stasko. NL4DV: A Toolkit for generating Analytic Specifications for Data Visualization from Natural Language queries. IEEE Trans. Vis. Comput. Graph., 2020. 8.
[57]
Neo4J. Neo4j Bloom. [Online]. Available: https://neo4j.com/product/bloom/. Accessed: 2023-03-24. 3, 4, 9.
[58]
C. Nobre, M. Streit, and A. Lex. Juniper: A tree+table approach to multi-variate graph visualization. IEEE Trans. Vis. Comput. Graph., 25(1): pp. 544–554, 2018. 6.
[59]
C. Partl, S. Gratzl, M. Streit, A. M. Wassermann, H. Pfister, D. Schmalstieg, and A. Lex. Pathfinder: Visual analysis of paths in graphs. Comput. Graph. Forum, 35(3): pp. 71–80, 2016. 2, 6.
[60]
S. Passi and S. J. Jackson. Trust in data science: Collaboration, translation, and accountability in corporate data science projects. in Proc. CHI, 2(CSCW), 2018. 2, 6.
[61]
F. Petroni, T. Rocktäschel, S. Riedel, P. Lewis, A. Bakhtin, Y. Wu, and A. Miller. Language models as knowledge bases? In Proc. EMNLP/IJCNLP, pp. 2463–2473. ACL, Hong Kong, 2019. 2.
[62]
A. Rossi, D. Barbosa, D. Firmani, A. Matinata, and P. Merialdo. Knowledge graph embedding for link prediction: A comparative analysis. ACM Trans. Knowl. Discov. Data, 15(2), 2021. 9.
[63]
C. Rudin and J. Radin. Why Are We Using Black Box Models in AI When We Don't Need To? A Lesson From an Explainable AI Competition. Harv. Bus. Rev., 2019. 8.
[64]
N. Sambasivan, S. Kapania, H. Highfill, D. Akrong, P. Paritosh, and L. M. Aroyo. “Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI. In Proc. CHI. ACM, New York, 2021. 2, 5, 9.
[65]
M. Sedlmair, M. Meyer, and T. Munzner. Design study methodology: Reflections from the trenches and the stacks. IEEE Trans. Vis. Comput. Graph., 18(12): pp. 2431–2440, 2012. 9.
[66]
P. Shannon, A. Markiel, O. Ozier, N. S. Baliga, J. T. Wang, D. Ramage, N. Amin, B. Schwikowski, and T. Ideker. Cytoscape: a software environment for integrated models of biomolecular interaction networks. Genome research, 13(11): pp. 2498–2504, 2003. 4.
[67]
B. Shneiderman. Response time and display rate in human performance with computers. ACM Comput. Surv., 16(3): pp. 265–285, 1984. 6.
[68]
K. Simonyan, A. Vedaldi, and A. Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:, 2014. 8.
[69]
Stardog. Stardog the enterprise knowledge graph platform. [Online]. Available: https://www.stardog.com/. Accessed: 2023-03-24. 2.
[70]
M.-A. D. Storey, D. Čubranić, and D. M. German. On the use of visualization to support awareness of human activities in software development: a survey and a framework. In Proc. SoftVis, pp. 193–202, 2005. 8.
[71]
A. Suh, M. Hajij, B. Wang, C. Scheidegger, and P. Rosen. Persistent homology guided force-directed graph layouts. IEEE Trans. Vis. Comput. Graph., 26(1): pp. 697–707, 2020. 6.
[72]
H. Suresh, S. R. Gomez, K. K. Nam, and A. Satyanarayan. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. in Proc. CHI, 2021. 2, 4, 9.
[73]
I. Tiddi, F. Lécué, and P. Hitzler, eds. “Knowledge Graphs for eXplainable Artificial Intelligence: Foundations, Applications and Challenges”, vol. 47 of Studies on the Semantic Web. IOS Press, 2020. 8.
[74]
E. Toussaint, P. Guagliardo, L. Libkin, and J. Sequeda. Troubles with nulls, views from the users. in Proc. VLDB Endow., 15(11): pp. 2613–2625, 2022. 5.
[75]
T. Von Landesberger, A. Kuijper, T. Schreck, J. Kohlhammer, J. J. van Wijk, J.-D. Fekete, and D. W. Fellner. Visual analysis of large graphs: state-of-the-art and future research challenges. Comput. Graph. Forum., 30(6): pp. 1719–1749, 2011. 6.
[76]
A. Waagmeester, G. Stupp, S. Burgstaller-Muehlbacher, B. M. Good, M. Griffith, O. L. Griffith, K. Hanspers, H. Hermjakob, T. S. Hudson, K. Hybiske et al., Wikidata as a knowledge graph for the life sciences. Elife, 9: pp. e52614, 2020. 8.
[77]
Wikipedia. Wikidata Statistics. [Online]. Available: https://www.wikidata.org/wiki/ Wikidata: Statistics. Accessed: 2023-03-24. 5.
[78]
K. Wongsuphasawat, Y. Liu, and J. Heer. Goals, process, and challenges of exploratory data analysis: an interview study. arXiv preprint arXiv:, 2019. 2.
[79]
K. Wongsuphasawat, Z. Qu, D. Moritz, R. Chang, F. Ouk, A. Anand, J. Mackinlay, B. Howe, and J. Heer. Voyager 2: Augmenting visual analysis with partial view specifications. In Proc. CHI, pp. 2648–2659. ACM, New York, 2017. 8.
[80]
G. Xiao, L. Ding, B. Cogrel, and D. Calvanese. Virtual knowledge graphs: An overview of systems and use cases. Data In tell., 1(3): pp. 201–223, 2019. 1.
[81]
Y. Yoon, B. A. Myers, and S. Koo. Visualization of fine-grained code change history. In IEEE VL/HCC, pp. 119–126, 2013. 8.

Cited By

View all
  • (2024)Learner-centered Ontology for Explainable Educational RecommendationAdjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3631700.3665226(567-575)Online publication date: 27-Jun-2024
  • (2024)KGScope: Interactive Visual Exploration of Knowledge Graphs With Embedding-Based GuidanceIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.336069030:12(7702-7716)Online publication date: 1-Dec-2024

Index Terms

  1. Knowledge Graphs in Practice: Characterizing their Users, Challenges, and Visualization Opportunities
      Index terms have been assigned to the content through auto-classification.

      Recommendations

      Comments

      Please enable JavaScript to view thecomments powered by Disqus.

      Information & Contributors

      Information

      Published In

      cover image IEEE Transactions on Visualization and Computer Graphics
      IEEE Transactions on Visualization and Computer Graphics  Volume 30, Issue 1
      Jan. 2024
      1456 pages

      Publisher

      IEEE Educational Activities Department

      United States

      Publication History

      Published: 01 January 2024

      Qualifiers

      • Research-article

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)0
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 29 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Learner-centered Ontology for Explainable Educational RecommendationAdjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3631700.3665226(567-575)Online publication date: 27-Jun-2024
      • (2024)KGScope: Interactive Visual Exploration of Knowledge Graphs With Embedding-Based GuidanceIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.336069030:12(7702-7716)Online publication date: 1-Dec-2024

      View Options

      View options

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media