[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content

Advertisement

Springer Nature Link
Log in

Graph Interpretation, Summarization and Visualization Techniques: A Review and Open Research Issues

  • 1209 - Recent Advances on Social Media Analytics and Multimedia Systems: Issues and Challenges
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Graphs has been a ubiquitous way of representing heterogeneous data. There are many studies focused on graph learning highlighting the approaches for graph data extraction, interpretation and graph summarization. Graph data summarization is achieving more expansion due to the broader length of sizeable applications and interpretation of proper understanding about the hidden details of the data using deep learning-based graph representation. Graph interpretation and summarization have come up as an interdisciplinary room that has vividly broader influence over multiple parallel areas and real-world applications. In other words, extraction of relevant data from massive and complex graph structure, enables the data to be used by many application area. However, it is found that recognizing the discriminatory and hidden properties from massive heterogeneous data is not easy in case of both nodal graph and graph image (also known as chart image). Hence, deep learning based approaches eventuated as a satisfactory solution. This paper presents an outline of the quantitative and statistical approaches used for learning and understanding different integrant of nodal graph and information graph, such as data extraction and processing, interpretation, summarization and visualization, by using graph-based learning methods. These integrant are broadly considered under (or as) SIV Model in this paper. Paper also discusses the influence of summarization techniques on the visualization of large data graphs and upcoming research areas of summarization. Lastly, paper provides with brief overview of challenges, application area, benefits of graph interpretation, summarization, and visualization, while providing existing tools and datasets available for graph processing and learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Zuo Y, Fang Q, Qian S, Zhang X, Xu C. Representation learning of knowledge graphs with entity attributes and multimedia descriptions. In2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM) 2018;13(1-5). IEEE

  2. Rudinac S, Chua TS, Diaz-Ferreyra N, Friedland G, Gornostaja T, Huet B, Kaptein R, Lindn K, Moens MF, Peltonen J, Redi M. Rethinking summarization and storytelling for modern social multimedia. In International Conference on Multimedia Modeling 2018;5 (632–644). Springer, Cham

    Google Scholar 

  3. Cao S, Lu W, Xu Q. Deep neural networks for learning graph representations. In Thirtieth AAAI conference on artificial intelligence 2016;21.

  4. Battaglia PW, Hamrick JB, Bapst V, Sanchez-Gonzalez A, Zambaldi V, Malinowski M, Tacchetti A, Raposo D, Santoro A, Faulkner R, Gulcehre C. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261. 2018;4.

  5. Wang H. Time-variant graph learning and classification (Doctoral dissertation)

  6. Bronstein MM, Bruna J, LeCun Y, Szlam A, Vandergheynst P. Geometric deep learning: going beyond Euclidean data. IEEE Signal Processing Magazine. 2017;34(4):18–42.

    Article  Google Scholar 

  7. Latouche P, Rossi F. Graphs in machine learning: an introduction. In European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), Proceedings of the 23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2015) 2015;22 (207-218).

  8. Aditya Bharadwaj, Divit P Singh, Anna Ritz, Allison N Tegge, Christopher L Poirel, Pavel Kraikivski, Neil Adames, Kurt Luther, Shiv D Kale, Jean Peccoud, John J Tyson, T M Murali, GraphSpace: stimulating interdisciplinary collaborations in network biology, Bioinformatics, 2017;33:19(3134–3136).

    Article  Google Scholar 

  9. Khan A, Bhowmick SS, Bonchi F. Summarizing static and dynamic big graphs. Proceedings of the VLDB Endowment. 2017;1:10(12):1981-4

  10. Fan W, Li J, Wang X, Wu Y. Query preserving graph compression. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data 2012;20(157-168).

  11. Liu Y, Safavi T, Dighe A, Koutra D. Graph summarization methods and applications: A survey. ACM Computing Surveys (CSUR). 2018;22:51(3):62

  12. Dong X, Thanou D, Rabbat M, Frossard P. Learning graphs from data: A signal representation perspective. IEEE Signal Processing Magazine. 2019 6;36(3):44-63

  13. Ehrig H, Taentzer G. Graphical representation and graph transformation. ACM Computing Surveys. 1999;31:(1–3)1.

    Google Scholar 

  14. Jiang W, Wang G, Bhuiyan MZ, Wu J. Understanding graph-based trust evaluation in online social networks: Methodologies and challenges. ACM Computing Surveys (CSUR). 2016;28:49(1):10.

  15. Riondato M, Garcia-Soriano D, Bonchi F. Graph summarization with quality guarantees. Data mining and knowledge discovery. 2017;31:(314–49)2.

    Article  MathSciNet  MATH  Google Scholar 

  16. Ebiri, Goasdou F, Kondylakis H, Kotzinos D, Manolescu I, Troullinou G, Zneika M. Summarizing semantic graphs a survey. The VLDB J. 2019;28(3):295-327.

  17. Ebiri, Goasdou F, Guzewicz P, Manolescu I. Compact Summaries of Rich Heterogeneous Graphs. 2018.

  18. Shin K, Ghoting A, Kim M, Raghavan H. Sweg: Lossless and lossy summarization of web-scale graphs. In The World Wide Web Conference 2019;13(1679-1690).

  19. Navlakha S, Rastogi R, Shrivastava N. Graph summarization with bounded error. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data 2008;9(419-432).

  20. Pouyanfar S, Sadiq S, Yan Y, Tian H, Tao Y, Reyes MP, Shyu ML, Chen SC, Iyengar SS. A survey on deep learning: Algorithms, techniques, and applications. ACM Computing Surveys (CSUR). 2018;18:51(5):92.

  21. Bronstein MM, Bruna J, LeCun Y, Szlam A, Vandergheynst P. Geometric deep learning: going beyond Euclidean data. IEEE Signal Processing Magazine. 2017;34(4):18–42,

    Article  Google Scholar 

  22. Cui P, Wang X, Pei J, Zhu W. A survey on network embedding. IEEE Transactions on Knowledge and Data Engineering. 2018;22.

  23. Shen X, Pan S, Liu W, Ong YS, Sun QS. Discrete network embedding. In Proceedings of the 27th International Joint Conference on Artificial Intelligence 2018;13(3549-3555). AAAI Press.

  24. Yang H, Pan S, Zhang P, Chen L, Lian D, Zhang C. Binarized attributed network embedding. In2018 IEEE International Conference on Data Mining (ICDM) 2018;17(1476-1481). IEEE

  25. Perozzi B, Al-Rfou R, Skiena S. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining 2014;24(701-710). ACM

  26. Published online:https://towardsdatascience.com/graph-embeddings-the-summary-cc6075aba007

  27. Rossi RA, Zhou R, Ahmed NK. Deep feature learning for graphs. arXiv preprint arXiv:1704.08829. 2017;28.

  28. Lee JB, Rossi RA, Kim S, Ahmed NK, Koh E. Attention models in graphs: A survey. arXiv preprint arXiv:1807.07984. 2018;20.

  29. Zhang Z, Cui P, Zhu W. Deep learning on graphs: A survey. arXiv preprint arXiv:1812.04202. 2018;11.

  30. Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS. A comprehensive survey on graph neural networks. arXiv preprint arXiv:1901.00596. 2019;3.

  31. Tixier AJ, Nikolentzos G, Meladianos P, Vazirgiannis M. Graph Classification with 2D Convolutional Neural Networks. arXiv preprint arXiv:1708.02218. 2017;29.

  32. Bresson X, Laurent T. An Experimental Study of Neural Networks for Variable Graphs

  33. Li Y, Tarlow D, Brockschmidt M, Zemel R. Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493. 2015;17.

  34. Sukhbaatar S, Fergus R. Learning multiagent communication with backpropagation. In Advances in Neural Information Processing Systems 2016;(2244-2252).

  35. Marcheggiani D, Titov I. Encoding sentences with graph convolutional networks for semantic role labeling. arXiv preprint arXiv:1703.04826. 2017;14.

  36. Akoglu L, Tong H, Koutra D. Graph based anomaly detection and description: a survey. Data mining and knowledge discovery. 2015;29:3(626-88)1.

  37. Koutra D, Kang U, Vreeken J, Faloutsos C. Summarizing and understanding large graphs. Statistical Analysis and Data Mining: The ASA Data Sci J. 2015;8(3):183–202.

    Article  MathSciNet  MATH  Google Scholar 

  38. Wu Y, Zhong Z, Xiong W, Jing N. Graph summarization for attributed graphs. In 2014 International Conference on Information Science, Electronics and Electrical Engineering 2014;26:1(503-507). IEEE.

  39. Zhang N, Tian Y, Patel JM. Discovery-driven graph summarization. In2010 IEEE 26th International Conference on Data Engineering (ICDE 2010) 2010;1(880-891). IEEE

  40. Ahn KJ, Guha S, McGregor A. Graph sketches: sparsification, spanners, and subgraphs. In Proceedings of the 31st ACM SIGMOD-SIGACT-SIGAI symposium on Principles of Database Systems 2012;21(5-14). ACM

  41. LeFevre K, Terzi E. GraSS: Graph structure summarization. In Proceedings of the 2010 SIAM International Conference on Data Mining. Society for Industrial and Appl Math 2010;29(454-465). 

  42. Bojchevski A, Shchur O, Zgner D, Gnnemann S. Netgan: Generating graphs via random walks. arXiv preprint arXiv:1803.00816. 2018;2.

  43. Shi L, Tong H, Tang J, Lin C. Vegas: Visual influence graph summarization on citation networks. IEEE Transactions on Knowledge and Data Engineering. 2015;1:27(12):3417-31.

  44. Fukushima K. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 1980;36(4)193–202.

    Article  MATH  Google Scholar 

  45. Atlas, Les E., Homma T, Marks R. An artificial neural network for spatiotemporal bipolar patterns: Application to phoneme classification. In Anderson, D.Z. (ed.), Neural Inform Process Sys, 1988;31-40.

  46. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015;521:436–444.

    Article  Google Scholar 

  47. Wang H, Wang J, Wang J, Zhao M, Zhang W, Zhang F, Xie X, Guo M. Graphgan: Graph representation learning with generative adversarial nets. In Thirty-Second AAAI Conference on Artificial Intelligence 2018;26.

  48. Koutra D, Kang U, Vreeken J, Faloutsos C. Vog: Summarizing and understanding large graphs. In Proceedings of the 2014 SIAM international conference on data mining. Society for Industrial and Applied Mathematics 2014;28(91-99).

  49. Wu F, Zhang T, Souza Jr AH, Fifty C, Yu T, Weinberger KQ. Simplifying Graph Convolutional Networks. arXiv preprint arXiv:1902.07153. 2019;19.

  50. Goonetilleke O, Koutra D, Sellis T, Liao K. Edge Labeling Schemes for Graph Data. In SSDBM 2017;27(12-1).

  51. Chakrabarti D, Faloutsos C. Graph mining: Laws, generators, and algorithms. ACM computing surveys (CSUR). 2006;29:38(1):2.

  52. Kipf TN, Welling M. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907. 2016;9.

  53. Zhang M, Cui Z, Neumann M, Chen Y. An end-to-end deep learning architecture for graph classification. In Thirty-Second AAAI Conference on Artificial Intelligence 2018;29.

  54. Ying Z, You J, Morris C, Ren X, Hamilton W, Leskovec J. Hierarchical graph representation learning with differentiable pooling. In Advances in Neural Information Processing Systems 2018;(4800-4810).

  55. Pan S, Wu J, Zhu X, Zhang C, Philip SY. Joint structure feature exploration and regularization for multi-task graph classification. IEEE Transactions on Knowledge and Data Engineering. 2016;1:28(715-28)3.

  56. Pan S, Wu J, Zhu X, Long G, Zhang C. Task sensitive feature exploration and learning for multitask graph classification. IEEE transactions on cybernetics. 2017;47(3):744–58.

    Article  Google Scholar 

  57. Pan S, Hu R, Long G, Jiang J, Yao L, Zhang C. Adversarially Regularized Graph Autoencoder for Graph Embedding. IJCAI International Joint Conference on Artificial Intelligence (IJCAI), 2018;2609-2615.

  58. Yan S, Xu D, Zhang B, Zhang HJ, Yang Q, Lin S. Graph embedding and extensions: A general framework for dimensionality reduction. IEEE Transactions on Pattern Analysis & Machine Intelligence. 2007;1(40–51)1.

    Article  Google Scholar 

  59. Goyal P, Ferrara E. Graph embedding techniques, applications, and performance: A survey. Knowledge-Based Systems. 2018;151(78–94)1.

    Article  Google Scholar 

  60. Cai H, Zheng VW, Chang KC. A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Transactions on Knowledge and Data Engineering. 2018;30(9):1616–37.

    Article  Google Scholar 

  61. Wang Q, Mao Z, Wang B, Guo L. Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering. 2017;1:29(12):2724-43.

  62. Nie F, Zhu W, Li X. Unsupervised large graph embedding. In Thirty-first AAAI conference on artificial intelligence 2017;13.

  63. Mao Q, Wang L, Tsang IW, Sun Y. Principal graph and structure learning based on reversed graph embedding. IEEE transactions on pattern analysis and machine intelligence. 2017;39(11):2227-41.

  64. Ding W, Lin C, Ishwar P. Node embedding via word embedding for network community discovery. IEEE Transactions on Signal and Information Processing over Networks. 2017;3(3):539–52.

    Article  MathSciNet  Google Scholar 

  65. Xu H, Luo D, Zha H, Carin L. Gromov-Wasserstein Learning for Graph Matching and Node Embedding. arXiv preprint arXiv:1901.06003. 2019;17.

  66. Cavallari S, Zheng VW, Cai H, Chang KC, Cambria E. Learning community embedding with community detection and node embedding on graphs. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management 2017;6(377-386). ACM

  67. Faerman E, Borutta F, Fountoulakis K, Mahoney MW. Lasagne: Locality and structure aware graph node embedding. In2018 IEEE/WIC/ACM International Conference on Web Intelligence (WI) 2018;3(246-253). IEEE

  68. Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G. The graph neural network model. IEEE Transactions on Neural Networks. 2009; 20(1):61–80.

    Article  Google Scholar 

  69. Monti F, Bronstein M, Bresson X. Geometric matrix completion with recurrent multi-graph neural networks. InAdvances in Neural Information Processing Systems 2017;( 3697-3707)

  70. Seo Y, Defferrard M, Vandergheynst P, Bresson X. Structured sequence modeling with graph convolutional recurrent networks. In International Conference on Neural Information Processing 2018;13(362–373). Springer, Cham

    Google Scholar 

  71. Niepert M, Ahmed M, Kutzkov K. Learning convolutional neural networks for graphs. In International conference on machine learning 2016;11(2014-2023).

  72. Henaff M, Bruna J, LeCun Y. Deep convolutional networks on graph-structured data. arXiv preprint arXiv:1506.05163. 2015;16.

  73. Hamilton W, Ying Z, Leskovec J. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems 2017;(1024-1034).

  74. Wang C, Pan S, Long G, Zhu X, Jiang J. Mgae: Marginalized graph autoencoder for graph clustering. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management 2017;6(889-898). ACM

  75. Kipf TN, Welling M. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308. 2016;21

  76. Simonovsky M, Komodakis N. Graphvae: Towards generation of small graphs using variational autoencoders. In International Conference on Artificial Neural Networks 2018;4(412–422). Springer, Cham

    Google Scholar 

  77. Tran PV. Learning to make predictions on graphs with autoencoders. In 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) 2018;1(237-245). IEEE

  78. Wang D, Cui P, Zhu W. Structural deep network embedding. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining 2016;13(1225-1234). ACM

  79. Li C, Wang S, Yang D, Li Z, Yang Y, Zhang X, and Zhou J. Ppne: Property preserving network embedding. In International Conference on Database Systems for Advanced Applications, 2017;163-179. Springer

  80. Wang J. Yu L, Zhang W, Gong Y, Xu Y, Wang B, Zhang P, Zhang D. Irgan: A minimax game for unifying generative and discriminative information retrieval models. In SIGIR. ACM 2017a.

    Book  Google Scholar 

  81. Tang J, Qu M, Wang M, Zhang M, Yan J, Mei Q. Line: Large-scale information network embedding. In WWW, 1067-1077. Int World Wide Web Conferences Steering Committee 2015.

  82. Denton EL, Chintala S, Fergus R et al. Deep generative image models using a Laplacian pyramid of adversarial networks. In NIPS, 2015;1486–1494.

  83. Yu L, Zhang W, Wang J, Yu Y. Seqgan: Sequence generative adversarial nets with policy gradient, In AAAI, 2017;2852–2858.

  84. Bojchevski A, Shchur O, Zgner D, Gnnemann S. GraphGAN: Generating Graphs via Random Walks 2018.

  85. Tarawaneh RA, Keller P, Ebert A. A general introduction to graph visualization techniques. In Visualization of Large and Unstructured Data Sets: Applications in Geospatial Planning, Modeling and Engineering-Proceedings of IRTG 1131 Workshop 2011 2012. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik 2012.

  86. Schulz HJ, Schumann H. Visualizing graphs-a generalized view. In Tenth International Conference on Information Visualisation (IV’06) 2006;5(166-173). IEEE

  87. Hu Y, Shi L. Visualizing large graphs. Wiley Interdisciplinary Reviews: Computational Statistics. 2015;7(2):115–36.

    Article  MathSciNet  Google Scholar 

  88. Liu X, Tian Y, He Q, Lee WC, McPherson J. Distributed graph summarization. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management 2014 3 (pp. 799-808). ACM

  89. Wan X, Wang H, Li J. LKAQ: Large-scale knowledge graph approximate query algorithm. Information Sciences. 2019;1:505:306–24.

    Article  Google Scholar 

  90. Arleo A, Didimo W, Liotta G, Montecchiani F. Large graph visualizations using a distributed computing platform. Information Sciences, 2017;381:124–141.

    Article  Google Scholar 

  91. Zheng W, Zou L, Lian X, Zhang H, Wang W, Zhao D. SQBC: An efficient subgraph matching method over large and dense graphs. Information Sciences, 2014;261(116–131).

    Article  MathSciNet  MATH  Google Scholar 

  92. Livadas PE, Johnson T. An optimal algorithm for the construction of the system dependence graph. Information Sciences, 2000;125(1–4):99–131.

    Article  MATH  Google Scholar 

  93. Baralis E, Cagliero L, Mahoto N, Fiori A. GRAPHSUM: Discovering correlations among multiple terms for graph-based summarization. Information Sciences, 2013;249(96–109).

    Article  MathSciNet  Google Scholar 

  94. Vanetik N, Litvak M, Churkin E, Last M. An unsupervised constrained optimization approach to compressive summarization. Information Sciences, 2020;509(22–35).

    Article  Google Scholar 

  95. Zhou F, Qu Q, Toivonen H. Summarisation of weighted networks. Journal of Experimental & Theoretical Artificial Intelligence, 2017;29(5),1023–1052.

    Article  Google Scholar 

  96. Xie Y, Gong M, Qin AK, Tang Z, Fan X. TPNE: Topology preserving network embedding. Information Sci, 2019;504(20–31).

    Article  MathSciNet  MATH  Google Scholar 

  97. Constantin MG, Redi M, Zen G, Ionescu B. Computational understanding of visual interestingness beyond semantics: literature survey and analysis of covariates. ACM Computing Surveys (CSUR). 2019;27,52(2):25.

  98. Gambhir M, Vishal G. Recent automatic text summarization techniques: a survey. Artificial Intelligence Review, 2017;1:(1–66)47.

    Article  Google Scholar 

  99. Kanapala A, Sukomal P, Rajendra P. Text summarization from legal documents: a survey. Artificial Intelligence Review, 2019;3(371–402)51.

    Article  Google Scholar 

  100. Cytoscape. online: https://cytoscape.org

  101. Gephi. Online: https://gephi.org

  102. Linkurious. Online: https://linkurio.us

  103. SNAP. Online: http://snap.stanford.edu/snap

  104. Pajek. Online: http://mrvar.fdv.uni-lj.si/pajek

  105. IBM i2 Analyst’s Workstation. Online: https://www.ibm.com/us-en/marketplace/analysts-notebook

  106. Polinode. Online: https://www.polinode.com

  107. GUESS. Online: http://graphexploration.cond.org

  108. Dong, Y, Chawla NV, Ananthram S. metapath2vec: Scalable representation learning for heterogeneous networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, 2017;135-144.

  109. Chen H, Perozzi B, Hu Y, Skiena S. Harp: Hierarchical representation learning for networks. arXiv preprint arXiv:1706.07845, 2017.

  110. Bhagat S, Graham C, Muthukrishnan S. Node classification in social networks. In Social network data analytics. Springer, Boston MA, 2011;115-148.

  111. Liben Nowell D, Kleinberg J. The link prediction problem for social networks. Journal of the American society for information science and technology, 2007;7(1019-1031)58.

  112. Vishwanathan S, Vichy N, Nicol N. Schraudolph, Risi K, Karsten M. Borgwardt. Graph kernels. The Journal of Machine Learning Research 2010;10(1201-1242).

  113. Herman I, Guy M, Scott M. Graph visualization and navigation in information visualization: A survey. IEEE Transactions on visualization and computer graphics, 2000;1(24-43)6.

  114. Bach B, Dragicevic P, Archambault D, Hurter C, Carpendale S. A descriptive framework for temporal data visualizations based on generalized space-time cubes. Computer Graphics Forum, 2017;36(36–61)6. 10.1111/cgf.12804.

    Article  Google Scholar 

  115. Beck F, Burch M, Diehl S, Weiskopf D. A Taxonomy and Survey of Dynamic Graph Visualization. Computer Graphics Forum, 2017;36:1(133–159). 10.1111/cgf.12791.

    Article  Google Scholar 

  116. Vehlow C, Fabian B, Daniel W. Visualizing group structures in graphs: A survey. In Computer Graphics Forum, 2017;36(201–225)6.

    Article  Google Scholar 

  117. Burch M, Vehlow C, Beck F, Diehl S, Weiskopf D. Parallel edge splatting for scalable dynamic graph visualization. IEEE Transactions on Visualization and Computer Graphics, 2011;17:(2344–2353)12.

    Article  Google Scholar 

  118. Bezerianos A, Chevalier F, Dragicevic P, Elmqvist N, Fekete J.D. GraphDice: A system for exploring multivariate social networks. In Computer Graphics Forum, 2010;29(863-872). https://doi.org/10.1111/j1467-8659.2009.01687.x

  119. Hadlak S, Schulz HJ, Schumann H. In situ exploration of large dynamic networks. IEEE Transactions on Visualization and Computer Graphics, 2011;17(12)2334–2343. https://doi.org/10.1109/tvcg.2011.213

    Article  Google Scholar 

  120. Bach B, Kerracher N, Hall KW, Carpendale S, Kennedy J, Henry Riche N. Telling stories about dynamic networks with graph comics. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI 16), 2016;3670-3682. https://doi.org/10.1145/2858036.2858387

  121. van den Elzen S, Danny H, Jorik B, Jarke J. van Wijk. Dynamic network visualization withextended massive sequence views. IEEE transactions on visualization and computer graphics, 2013;20(8)1087-1099.

  122. Lee A, Daniel A, Miguel A. Nacenta. The Effectiveness of Interactive Visualization Techniques for Time Navigation of Dynamic Graphs on Large Displays. arXiv preprint arXiv:2008.12747 2020.

  123. Cspedes-Hernandez D, Juan Manuel G. C, Josefina G. G, Liliana R. V. Gathering, Classifying and Visualizing Results from Social Surveys using OCR and Machine Learning Techniques. Res Comput Sci 2017;145:(81-95).

  124. Rodrigues J. F, Traina A. J. M, Faloutsos C, Traina C. SuperGraph visualization. In Proceedings of the IEEE International Symposium on Multimedia (ISM06),Washington, DC, USA, 2006.

  125. MA K.L. Image graphs - A novel approach to visual data exploration. In Proceedings of IEEE Visualization Conference 1999;(81-88).

  126. JANKUN-KELLY T. J., MA K. L.: Visualization exploration and encapsulation via a spreadsheet-like interface. IEEE Transactions on Visualization and Computer Graphics 2001;7(3)275-287.

  127. JANKUN-KELLY T., MA K.-L., GERTZ M.: A model for the visualization exploration process. In Proceedings of IEEE Visualization Conference 2002;323-330.

  128. Woodring J, Shen H.W.: Multi-variate, time-varying, and comparative visualization with contextual cues. IEEE Transactions on Visualization and Computer Graphics 2006;12(5)909-916.

  129. Wohlfart M, Hauser H. Story telling for presentation in volume visualization. In Proceedings of Joint Eurographics - IEEE VGTC Symposium on Visualization 2007;91-98.

  130. Akiba H, Wang C, Ma K. L. AniViz: A template based animation tool for volume visualization. IEEE Computer Graphics and Applications 2010;30(5)61-71.

  131. Bruckner S, M’OLLER T. Result-driven exploration of simulation parameter spaces for visual effects design. IEEE Transactions on Visualization and Computer Graphics 2010;16(6)1468-1476.

  132. Purchase HC. Twelve years of diagrams research. Elsevier Journal of Visual Languages & Computing. 2014;1:2(57-75)25.

  133. Praczyk PA, Nogueras-Iso J. Automatic extraction of figures from scientific publications in high-energy physics. Information Technology and Libraries. 2013;22:32(4):25-52.

  134. Liu Y, Xiaoqing L, Yeyang Q, Zhi T, Jianbo X. Review of chart recognition in document images. In Visualization and Data Analysis, International Society for Optics and Photonics, 2013;8654(865410).

  135. Jung D, Kim W, Song H, Hwang JI, Lee B, Kim B, Seo J. Chartsense: Interactive data extraction from chart images. In Proceedings of the 2017 chi conference on human factors in computing systems 2017;2(6706-6717).

  136. Choi J, Jung S, Park DG, Choo J, Elmqvist N. Visualizing for the Non-Visual: Enabling the Visually Impaired to Use Visualization. In Computer Graphics Forum 2019;38(249-260)3.

  137. Poco J, Heer J. Reverse-engineering visualizations: Recovering visual encodings from chart images. In Computer Graphics Forum 2017;36:3(353-363).

  138. Huang W, Tan CL. A system for understanding imaged infographics and its applications. In Proceedings of the 2007 ACM symposium on Document engineering 2007;28(9-18).

  139. Blue Leaf Software - Dagra. [Online] Link: https://blueleafsoftware.com

  140. Plot Digitizer. [Online] Link: http://plotdigitizer.sourceforge.net

  141. Engauge Digitizer. [Online] Link: http://markummitchell.github.io/engauge-digitizer/

  142. Web Plot Digitizer. [Online] Link: https://automeris.io/WebPlotDigitizer/

  143. Graph Reader. [Online] Link: http://www.graphreader.com

  144. Data Thief. [Online] Link: https://datathief.org

  145. Zhou F, Zhao Y, Chen W, Tan Y, Xu Y, Chen Y, Liu C, Zhao Y. Reverse-engineering bar charts using neural networks. Journal of Visualization. 2021;24(2):419–35

    Article  Google Scholar 

  146. Liu X, Klabjan D, NBless P. Data extraction from charts via single deep neural network. arXiv preprint arXiv:1906.11906. 2019;6.

  147. Balaji A, Ramanathan T, Sonathi V. Chart-text: A fully automated chart image descriptor. arXiv preprint arXiv:1812.10636. 2018;27.

  148. Savva M, Kong N, Chhajta A, Fei-Fei L, Agrawala M, Heer J. Revision: Automated classification, analysis and redesign of chart images. In Proceedings of the 24th annual ACM symposium on User interface software and technology 2011;16(393-402).

  149. Zhou YP, Tan CL. Learning-based scientific chart recognition. In 4th IAPR International Workshop on Graphics Recognition, GREC 2001;7(482-492).

  150. Svendsen JP. Chart detection and recognition in graphics intensive business documents (Doctoral dissertation).

  151. Cliche M, Rosenberg D, Madeka D, Yee C. Scatteract: Automated extraction of data from scatter plots. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, Cham, 2017;18(135–150).

  152. Dai W, Wang M, Niu Z, Zhang J. Chart decoder: Generating textual and numeric information from chart images automatically. Journal of Visual Languages & Computing. 2018;1:48(101–9).

  153. Al-Zaidy RA, Giles CL. A machine learning approach for semantic structuring of scientific charts in scholarly documents. In Twenty-Ninth IAAI Conference 2017;8.

  154. Molla MK, Talukder KH, Hossain MA. Line chart recognition and data extraction technique. InInternational Conference on Intelligent Data Engineering and Automated Learning. Springer, Berlin, Heidelberg, 2003;21(865–870).

    Google Scholar 

  155. Reddy VK, Kaushik CM. Image processing based data extraction from graphical representation. In 2015 IEEE International Conference on Computer Graphics, Vision and Information Security (CGVIS) 2015;2(190-194). IEEE

  156. Chester D, Elzer S. Getting computers to see information graphics so users do not have to. In International Symposium on Methodologies for Intelligent Systems. Springer, Berlin, Heidelberg 2005;25(660–668).

    Google Scholar 

  157. Mishchenko A, Vassilieva N. Chart image understanding and numerical data extraction. In2011 Sixth International Conference on Digital Information Management 2011;26(115-120). IEEE

  158. Browuer W, Kataria S, Das S, Mitra P, Giles CL. Segregating and extracting overlapping data points in two-dimensional plots. In Proceedings of the 8th ACM/IEEE-CS joint conference on Digital libraries 2008;16(276-279).

  159. Ray CS, Wang S, Giles CL. Curve separation for line graphs in scholarly documents. In Proceedings of the 16th ACM/IEEE-CS on Joint Conference on Digital Libraries 2016;19(277-278).

  160. De P. Automatic data extraction from 2D and 3D pie chart images. In 2018 IEEE 8th International Advance Computing Conference (IACC) 2018;14(20-25). IEEE

  161. Ma Y, Tung AK, Wang W, Gao X, Pan Z, Chen W. Scatternet: A deep subjective similarity model for visual analysis of scatterplots. IEEE transactions on visualization and computer graphics. 2018;12:26(3):1562-76.

  162. Tang B, Liu X, Lei J, Song M, Tao D, Sun S, Dong F. Deepchart: Combining deep convolutional networks and deep belief networks in chart classification. Signal Processing. 2016;124:(156–61)1.

    Article  Google Scholar 

  163. Karthikeyani V, Nagarajan S. Machine learning classification algorithms to recognize chart types in portable document format (pdf) files. International Journal of Computer Applications. 2012;39(2):1–5.

    Article  Google Scholar 

  164. Demir S, Carberry S, McCoy KF. Summarizing information graphics textually. Computational Linguistics. 2012;1:38(3):527-74.

  165. Chen C, Zhang R, Koh E, Kim S, Cohen S, Rossi R. Figure Captioning with Relation Maps for Reasoning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 2020;(1537-1545).

  166. Bhatia S, Mitra P. Summarizing figures, tables, and algorithms in scientific publications to augment search results. ACM Transactions on Information Systems (TOIS). 2012 6;30(1):1-24.

  167. Demir S, Carberry S, Elzer S. Effectively realizing the inferred message of an information graphic. InProceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP) 2007;27(150-156).

  168. Choudhury SR, Wang S, Giles CL. Scalable algorithms for scholarly figure mining and semantics. In Proceedings of the International Workshop on Semantic Big Data 2016;26(1-6).

  169. Mahmood A, Bajwa IS, Qazi K. An automated approach for interpretation of statistical graphics. In2014 Sixth International Conference on Intelligent Human-Machine Systems and Cybernetics 2014;26:2(376-379). IEEE

  170. Kallimani JS, Srinivasa KG, Eswara RB. Extraction and interpretation of charts in technical documents. In2013 International Conference on Advances in Computing, Communications and Informatics (ICACCI) 2013;22(382-387). IEEE

  171. Demir S, Schwartz SE, Burns R, Carberry S. What is being Measured in an Information Graphic?. In International Conference on Intelligent Text Processing and Computational Linguistics 2013;24(501–512). Springer, Berlin, Heidelberg

    Google Scholar 

  172. Elzer S, Carberry S, Demir S. Communicative signals as the key to automated understanding of simple bar charts. InInternational Conference on Theory and Application of Diagrams 2006;28(25–39). Springer, Berlin, Heidelberg

    Google Scholar 

  173. Carberry S, Elzer S, Demir S. Information graphics: an untapped resource for digital libraries. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval 2006;6(581-588)

  174. Burns R, Carberry S, Elzer Schwartz S. An automated approach for the recognition of intended messages in grouped bar charts. Computational Intelligence. 2019; 35(4):955–1002

    Article  MathSciNet  Google Scholar 

  175. Greenbacker C, Wu P, Carberry S, McCoy KF, Elzer S. Abstractive summarization of line graphs from popular media. In Proceedings of the Workshop on Automatic Summarization for Different Genres, Media, and Languages 2011;(41-48).

  176. Burns R, Balawejder E, Domanowska W, Schwartz SE, Carberry S. Exploring the Types of Messages that Pie Charts Convey in Popular Media. In International Conference on Theory and Application of Diagrams 2016;7(265–271). Springer, Cham

    Google Scholar 

  177. Nair RR, Sankaran N, Nwogu I, Govindaraju V. Understanding line plots using Bayesian Network. In 2016 12th IAPR Workshop on Document Analysis Systems (DAS) 2016;11(108-113). IEEE

  178. Al-Zaidy RA, Choudhury SR, Giles CL. Automatic summary generation for scientific data charts. In Workshops at the thirtieth aaai conference on artificial intelligence 2016;29.

  179. Di Sorbo A, Panichella S, Alexandru CV, Visaggio CA, Canfora G. SURF: summarizer of user reviews feedback. In 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C) 2017;20(55-58). IEEE

  180. Xu B, Xing Z, Xia X, Lo D. AnswerBot: Automated generation of answer summary to developers’ technical questions. In 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE) 2017:1(706-716). IEEE

  181. Woodsend K, Lapata M. Automatic generation of story highlights. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics 2010;(565-574).

  182. Leppanen L, Munezero M, Granroth-Wilding M, Toivonen H. Data-driven news generation for automated journalism. In Proceedings of the 10th International Conference on Natural Language Generation 2017;(188-197).

  183. Demir S, Oliver D, Schwartz E, Elzer S, Carberry S, Mccoy KF, Chester D. Interactive SIGHT: textual access to simple bar charts. New Review of Hypermedia and Multimedia. 2010;1:16(3):245-79.

  184. Demir S, Carberry S, McCoy KF. Generating textual summaries of bar charts. InProceedings of the Fifth International Natural Language Generation Conference 2008;(7-15).

  185. Moraes P, Sina G, McCoy K, Carberry S. Generating summaries of line graphs. In Proceedings of the 8th International Natural Language Generation Conference (INLG) 2014;(95-98).

  186. Elzer S, Carberry S, Zukerman I. The automated understanding of simple bar charts. Artificial Intelligence. 2011;1:175(2):526-55.

  187. Nair RR, Sankaran N, Nwogu I, Govindaraju V. Automated analysis of line plots in documents. In 2015 13th international conference on document analysis and recognition (icdar) 2015;23(796-800). IEEE

  188. Methani N, Ganguly P, Khapra MM, Kumar P. Plotqa: Reasoning over scientific plots. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 2020;(1527-1536).

  189. Kafle K, Price B, Cohen S, Kanan C. Dvqa: Understanding data visualizations via question answering. InProceedings of the IEEE conference on computer vision and pattern recognition 2018;(5648-5656).

  190. Amara J, Kaur P, Owonibi M, Bouaziz B. Convolutional neural network based chart image classification 2017.

  191. Gao J, Carrillo RE, Barner KE. Image categorization for improving accessibility to information graphics. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility 2010;25(265-266).

  192. Siddiqui SA, Malik MI, Agne S, Dengel A, Ahmed S. Decnt: Deep deformable cnn for table detection. IEEE Access. 2018;6:(74151–61)20.

    Article  Google Scholar 

  193. Schreiber S, Agne S, Wolf I, Dengel A, Ahmed S. Deepdesrt: Deep learning for detection and structure recognition of tables in document images. In 2017 14th IAPR international conference on document analysis and recognition (ICDAR) 2017;9(1)1162-1167. IEEE

  194. Vo ND, Nguyen K, Nguyen TV, Nguyen K. Ensemble of deep object detectors for page object detection. In Proceedings of the 12th International Conference on Ubiquitous Information Management and Communication 2018;5(1-6).

  195. Saha R, Mondal A, Jawahar CV. Graphical object detection in document images. In 2019 International Conference on Document Analysis and Recognition (ICDAR) 2019;20 (51-58). IEEE

  196. Gilani A, Qasim SR, Malik I, Shafait F. Table detection using deep learning. In2017 14th IAPR international conference on document analysis and recognition (ICDAR) 2017;9 :1(771-776). IEEE

  197. Younas J, Siddiqui SA, Munir M, Malik MI, Shafait F, Lukowicz P, Ahmed S. Fi-Fo Detector: Figure and Formula Detection Using Deformable Networks. Applied Sciences. 2020;10(18):6460.

    Article  Google Scholar 

  198. Agarwal M, Mondal A, Jawahar CV. Cdec-net: Composite deformable cascade network for table detection in document images. In 2020 25th International Conference on Pattern Recognition (ICPR) 2021;10(9491-9498). IEEE

  199. Mei H, Ma Y, Wei Y, Chen W. The design space of construction tools for information visualization: A survey. Journal of Visual Languages & Computing.  2018;1(44):120–32.

    Article  Google Scholar 

  200. Mishra P, Kumar S, Chaube MK. Dissimilarity Based Regularized Deep Learning Model for Information Charts. In 2020 Joint 9th International Conference on Informatics, Electronics & Vision (ICIEV) and 2020 4th International Conference on Imaging, Vision & Pattern Recognition (icIVPR) 2020;26(1-6). IEEE

  201. Harper J, Agrawala M. Converting basic D3 charts into reusable style templates. IEEE transactions on visualization and computer graphics. 2017;7:24(3):1274-86.

  202. Kong N, Agrawala M. Graphical overlays: Using layered elements to aid chart reading. IEEE transactions on visualization and computer graphics. 2012;8:18(12):2631-8.

  203. Mendez GG, Nacenta MA, Vandenheste S. iVoLVER: Interactive visual language for visualization extraction and reconstruction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems 2016;7(4073-4085).

  204. Harper J, Agrawala M. Deconstructing and restyling D3 visualizations. In Proceedings of the 27th annual ACM symposium on User interface software and technology 2014;5(253-262).

  205. Poco J, Mayhua A, Heer J. Extracting and retargeting color mappings from bitmap images of visualizations. IEEE transactions on visualization and computer graphics. 2017;29:24(1):637-46.

  206. Kim Y, Wongsuphasawat K, Hullman J, Heer J. Graphscape: A model for automated reasoning about visualization similarity and sequencing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems 2017;2(2628-2638).

  207. Burns R, Schwartz SE, Carberry S. Towards Adapting Information Graphics to Individual Users to Support Recognizing Intended Messages. In UMAP Workshops 2013.

  208. Chen Z, Wang Y, Wang Q, Wang Y, Qu H. Towards automated infographic design: Deep learning-based auto-extraction of extensible timeline. IEEE transactions on visualization and computer graphics. 2019;20:26(1):917-26.

  209. Srinivasan A, Drucker SM, Endert A, Stasko J. Augmenting visualizations with interactive data facts to facilitate interpretation and communication. IEEE transactions on visualization and computer graphics. 2018;20:25(1):672-81.

  210. Kong N, Hearst MA, Agrawala M. Extracting references between text and charts via crowdsourcing. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems 2014;26(31-40).

  211. Kahou SE, Michalski V, Atkinson A, Kdr, Trischler A, Bengio Y. Figureqa: An annotated figure dataset for visual reasoning. arXiv preprint arXiv:1710.07300. 2017;19.

  212. Kafle K, Shrestha R, Cohen S, Price B, Kanan C. Answering questions about data visualizations using efficient bimodal fusion. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 2020;(1498-1507).

  213. Li Z, Carberry S, Fang H, McCoy KF, Peterson K, Stagitis M. A novel methodology for retrieving infographics utilizing structure and message content. Data & Knowledge Engineering. 2015;1:100(191–210).

  214. Hoque E, Agrawala M. Searching the visual style and structure of d3 visualizations. IEEE transactions on visualization and computer graphics. 2019;22:26(1):1236-45.

  215. Lee PS, West JD, Howe B. Viziometrics: Analyzing visual information in the scientific literature. IEEE Transactions on Big Data. 2017;2:4(1):117-29.

  216. Chen Z, Cafarella M, Adar E. Diagramflyer: A search engine for data-driven diagrams. In Proceedings of the 24th International Conference on World Wide Web 2015;18(183-186).

  217. Bylinskii Z, Kim NW, O’Donovan P, Alsheikh S, Madan S, Pfister H, Durand F, Russell B, Hertzmann A. Learning visual importance for graphic designs and data visualizations. In Proceedings of the 30th Annual ACM symposium on user interface software and technology 2017;20(57-69).

  218. Lee DJ, Dev H, Hu H, Elmeleegy H, Parameswaran A. Avoiding drill-down fallacies with vispilot: Assisted exploration of data subsets. In Proceedings of the 24th International Conference on Intelligent User Interfaces 2019;17(186-196).

  219. Chaudhry R, Shekhar S, Gupta U, Maneriker P, Bansal P, Joshi A. Leaf-qa: Locate, encode & attend for figure question answering. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 2020;(3512-3521).

  220. Siegel N, Horvitz Z, Levin R, Divvala S, Farhadi A. Figureseer: Parsing result-figures in research papers. In European Conference on Computer Vision 2016;8(664–680). Springer, Cham

  221. Davila K, Kota BU, Setlur S, Govindaraju V, Tensmeyer C, Shekhar S, Chaudhry R. ICDAR 2019 Competition on Harvesting Raw Tables from Infographics (CHART-Infographics). In 2019 International Conference on Document Analysis and Recognition (ICDAR) 2019;20(1594-1599). IEEE

  222. Gbel M, Hassan T, Oro E, Orsi G. ICDAR 2013 table competition. In 2013 12th International Conference on Document Analysis and Recognition 2013;25(1449-1453). IEEE

  223. Li M, Xu Y, Cui L, Huang S, Wei F, Li Z, Zhou M. DocBank: A benchmark dataset for document layout analysis. arXiv preprint arXiv:2006.01038. 2020;1.

  224. Zhong X, Tang J, Yepes AJ. Publaynet: largest dataset ever for document layout analysis. In 2019 International Conference on Document Analysis and Recognition (ICDAR) 2019;20(1015-1022). IEEE

  225. Gao L, Yi X, Jiang Z, Hao L, Tang Z. ICDAR2017 competition on page object detection. In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) 2017;1(1417-1422). IEEE

Download references

Author information

Authors and Affiliations

  1. Department of CSE, DSPM-IIITNR, Raipur, India

    Prerna Mishra & Santosh Kumar

  2. Department of Mathematics, DSPM-IIITNR, Raipur, India

    Mithilesh Kumar Chaube

Authors
  1. Prerna Mishra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Santosh Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mithilesh Kumar Chaube
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Prerna Mishra.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mishra, P., Kumar, S. & Chaube, M.K. Graph Interpretation, Summarization and Visualization Techniques: A Review and Open Research Issues. Multimed Tools Appl 82, 8729–8771 (2023). https://doi.org/10.1007/s11042-021-11582-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11582-9

Keywords

Search

Navigation