More Web Proxy on the site http://driver.im/

research-article

Graph Interpretation, Summarization and Visualization Techniques: A Review and Open Research Issues

Authors:

Mithilesh Kumar ChaubeAuthors Info & Claims

Multimedia Tools and Applications, Volume 82, Issue 6

Pages 8729 - 8771

https://doi.org/10.1007/s11042-021-11582-9

Published: 08 January 2022 Publication History

Abstract

Graphs has been a ubiquitous way of representing heterogeneous data. There are many studies focused on graph learning highlighting the approaches for graph data extraction, interpretation and graph summarization. Graph data summarization is achieving more expansion due to the broader length of sizeable applications and interpretation of proper understanding about the hidden details of the data using deep learning-based graph representation. Graph interpretation and summarization have come up as an interdisciplinary room that has vividly broader influence over multiple parallel areas and real-world applications. In other words, extraction of relevant data from massive and complex graph structure, enables the data to be used by many application area. However, it is found that recognizing the discriminatory and hidden properties from massive heterogeneous data is not easy in case of both nodal graph and graph image (also known as chart image). Hence, deep learning based approaches eventuated as a satisfactory solution. This paper presents an outline of the quantitative and statistical approaches used for learning and understanding different integrant of nodal graph and information graph, such as data extraction and processing, interpretation, summarization and visualization, by using graph-based learning methods. These integrant are broadly considered under (or as) SIV Model in this paper. Paper also discusses the influence of summarization techniques on the visualization of large data graphs and upcoming research areas of summarization. Lastly, paper provides with brief overview of challenges, application area, benefits of graph interpretation, summarization, and visualization, while providing existing tools and datasets available for graph processing and learning.

References

[1]

Zuo Y, Fang Q, Qian S, Zhang X, Xu C. Representation learning of knowledge graphs with entity attributes and multimedia descriptions. In2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM) 2018;13(1-5). IEEE

[2]

Rudinac S, Chua TS, Diaz-Ferreyra N, Friedland G, Gornostaja T, Huet B, Kaptein R, Lindn K, Moens MF, Peltonen J, Redi M. Rethinking summarization and storytelling for modern social multimedia. In International Conference on Multimedia Modeling 5 2018 Cham Springer 632-644

[3]

Cao S, Lu W, Xu Q. Deep neural networks for learning graph representations. In Thirtieth AAAI conference on artificial intelligence 2016;21.

[4]

Battaglia PW, Hamrick JB, Bapst V, Sanchez-Gonzalez A, Zambaldi V, Malinowski M, Tacchetti A, Raposo D, Santoro A, Faulkner R, Gulcehre C. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261. 2018;4.

[5]

Wang H. Time-variant graph learning and classification (Doctoral dissertation)

[6]

Bronstein MM, Bruna J, LeCun Y, Szlam A, and Vandergheynst P Geometric deep learning: going beyond Euclidean data IEEE Signal Processing Magazine. 2017 34 4 18-42

[7]

Latouche P, Rossi F. Graphs in machine learning: an introduction. In European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), Proceedings of the 23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2015) 2015;22 (207-218).

[8]

Bharadwaj Aditya Divit P Singh, Anna Ritz, Allison N Tegge, Christopher L Poirel, Pavel Kraikivski, Neil Adames, Kurt Luther, Shiv D Kale, Jean Peccoud, John J Tyson, T M Murali, GraphSpace: stimulating interdisciplinary collaborations in network biology Bioinformatics 2017 33 19 3134-3136

[9]

Khan A, Bhowmick SS, Bonchi F. Summarizing static and dynamic big graphs. Proceedings of the VLDB Endowment. 2017;1:10(12):1981-4

[10]

Fan W, Li J, Wang X, Wu Y. Query preserving graph compression. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data 2012;20(157-168).

[11]

Liu Y, Safavi T, Dighe A, Koutra D. Graph summarization methods and applications: A survey. ACM Computing Surveys (CSUR). 2018;22:51(3):62

[12]

Dong X, Thanou D, Rabbat M, Frossard P. Learning graphs from data: A signal representation perspective. IEEE Signal Processing Magazine. 2019 6;36(3):44-63

[13]

Ehrig H and Taentzer G Graphical representation and graph transformation ACM Computing Surveys. 1999 1 31 G1-3

[14]

Jiang W, Wang G, Bhuiyan MZ, Wu J. Understanding graph-based trust evaluation in online social networks: Methodologies and challenges. ACM Computing Surveys (CSUR). 2016;28:49(1):10.

[15]

Riondato M, Garcia-Soriano D, and Bonchi F Graph summarization with quality guarantees Data mining and knowledge discovery. 2017 31 2 314-49

[16]

Ebiri, Goasdou F, Kondylakis H, Kotzinos D, Manolescu I, Troullinou G, Zneika M. Summarizing semantic graphs a survey. The VLDB J. 2019;28(3):295-327.

[17]

Ebiri, Goasdou F, Guzewicz P, Manolescu I. Compact Summaries of Rich Heterogeneous Graphs. 2018.

[18]

Shin K, Ghoting A, Kim M, Raghavan H. Sweg: Lossless and lossy summarization of web-scale graphs. In The World Wide Web Conference 2019;13(1679-1690).

[19]

Navlakha S, Rastogi R, Shrivastava N. Graph summarization with bounded error. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data 2008;9(419-432).

[20]

Pouyanfar S, Sadiq S, Yan Y, Tian H, Tao Y, Reyes MP, Shyu ML, Chen SC, Iyengar SS. A survey on deep learning: Algorithms, techniques, and applications. ACM Computing Surveys (CSUR). 2018;18:51(5):92.

[21]

Bronstein MM, Bruna J, LeCun Y, Szlam A, and Vandergheynst P Geometric deep learning: going beyond Euclidean data IEEE Signal Processing Magazine. 2017 34 4 18-42

[22]

Cui P, Wang X, Pei J, Zhu W. A survey on network embedding. IEEE Transactions on Knowledge and Data Engineering. 2018;22.

[23]

Shen X, Pan S, Liu W, Ong YS, Sun QS. Discrete network embedding. In Proceedings of the 27th International Joint Conference on Artificial Intelligence 2018;13(3549-3555). AAAI Press.

[24]

Yang H, Pan S, Zhang P, Chen L, Lian D, Zhang C. Binarized attributed network embedding. In2018 IEEE International Conference on Data Mining (ICDM) 2018;17(1476-1481). IEEE

[25]

Perozzi B, Al-Rfou R, Skiena S. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining 2014;24(701-710). ACM

[26]

Published online:https://towardsdatascience.com/graph-embeddings-the-summary-cc6075aba007

[27]

Rossi RA, Zhou R, Ahmed NK. Deep feature learning for graphs. arXiv preprint arXiv:1704.08829. 2017;28.

[28]

Lee JB, Rossi RA, Kim S, Ahmed NK, Koh E. Attention models in graphs: A survey. arXiv preprint arXiv:1807.07984. 2018;20.

[29]

Zhang Z, Cui P, Zhu W. Deep learning on graphs: A survey. arXiv preprint arXiv:1812.04202. 2018;11.

[30]

Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS. A comprehensive survey on graph neural networks. arXiv preprint arXiv:1901.00596. 2019;3.

[31]

Tixier AJ, Nikolentzos G, Meladianos P, Vazirgiannis M. Graph Classification with 2D Convolutional Neural Networks. arXiv preprint arXiv:1708.02218. 2017;29.

[32]

Bresson X, Laurent T. An Experimental Study of Neural Networks for Variable Graphs

[33]

Li Y, Tarlow D, Brockschmidt M, Zemel R. Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493. 2015;17.

[34]

Sukhbaatar S, Fergus R. Learning multiagent communication with backpropagation. In Advances in Neural Information Processing Systems 2016;(2244-2252).

[35]

Marcheggiani D, Titov I. Encoding sentences with graph convolutional networks for semantic role labeling. arXiv preprint arXiv:1703.04826. 2017;14.

[36]

Akoglu L, Tong H, Koutra D. Graph based anomaly detection and description: a survey. Data mining and knowledge discovery. 2015;29:3(626-88)1.

[37]

Koutra D, Kang U, Vreeken J, and Faloutsos C Summarizing and understanding large graphs Statistical Analysis and Data Mining: The ASA Data Science Journal. 2015 8 3 183-202

[38]

Wu Y, Zhong Z, Xiong W, Jing N. Graph summarization for attributed graphs. In 2014 International Conference on Information Science, Electronics and Electrical Engineering 2014;26:1(503-507). IEEE.

[39]

Zhang N, Tian Y, Patel JM. Discovery-driven graph summarization. In2010 IEEE 26th International Conference on Data Engineering (ICDE 2010) 2010;1(880-891). IEEE

[40]

Ahn KJ, Guha S, McGregor A. Graph sketches: sparsification, spanners, and subgraphs. In Proceedings of the 31st ACM SIGMOD-SIGACT-SIGAI symposium on Principles of Database Systems 2012;21(5-14). ACM

[41]

LeFevre K, Terzi E. GraSS: Graph structure summarization. In Proceedings of the 2010 SIAM International Conference on Data Mining. Society for Industrial and Appl Math 2010;29(454-465).

[42]

Bojchevski A, Shchur O, Zgner D, Gnnemann S. Netgan: Generating graphs via random walks. arXiv preprint arXiv:1803.00816. 2018;2.

[43]

Shi L, Tong H, Tang J, Lin C. Vegas: Visual influence graph summarization on citation networks. IEEE Transactions on Knowledge and Data Engineering. 2015;1:27(12):3417-31.

[44]

Fukushima Kunihiko Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position Biological Cybernetics 1980 36 4 193-202

[45]

Atlas, Les E., Homma T, Marks R. An artificial neural network for spatiotemporal bipolar patterns: Application to phoneme classification. In Anderson, D.Z. (ed.), Neural Inform Process Sys, 1988;31-40.

[46]

LeCun Yann, Bengio Yoshua, and Hinton Geoffrey Deep learning Nature 2015 521 436-444

[47]

Wang H, Wang J, Wang J, Zhao M, Zhang W, Zhang F, Xie X, Guo M. Graphgan: Graph representation learning with generative adversarial nets. In Thirty-Second AAAI Conference on Artificial Intelligence 2018;26.

[48]

Koutra D, Kang U, Vreeken J, Faloutsos C. Vog: Summarizing and understanding large graphs. In Proceedings of the 2014 SIAM international conference on data mining. Society for Industrial and Applied Mathematics 2014;28(91-99).

[49]

Wu F, Zhang T, Souza Jr AH, Fifty C, Yu T, Weinberger KQ. Simplifying Graph Convolutional Networks. arXiv preprint arXiv:1902.07153. 2019;19.

[50]

Goonetilleke O, Koutra D, Sellis T, Liao K. Edge Labeling Schemes for Graph Data. In SSDBM 2017;27(12-1).

[51]

Chakrabarti D, Faloutsos C. Graph mining: Laws, generators, and algorithms. ACM computing surveys (CSUR). 2006;29:38(1):2.

[52]

Kipf TN, Welling M. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907. 2016;9.

[53]

Zhang M, Cui Z, Neumann M, Chen Y. An end-to-end deep learning architecture for graph classification. In Thirty-Second AAAI Conference on Artificial Intelligence 2018;29.

[54]

Ying Z, You J, Morris C, Ren X, Hamilton W, Leskovec J. Hierarchical graph representation learning with differentiable pooling. In Advances in Neural Information Processing Systems 2018;(4800-4810).

[55]

Pan S, Wu J, Zhu X, Zhang C, Philip SY. Joint structure feature exploration and regularization for multi-task graph classification. IEEE Transactions on Knowledge and Data Engineering. 2016;1:28(715-28)3.

[56]

Pan S, Wu J, Zhu X, Long G, and Zhang C Task sensitive feature exploration and learning for multitask graph classification IEEE transactions on cybernetics. 2017 47 3 744-58

[57]

Pan S, Hu R, Long G, Jiang J, Yao L, Zhang C. Adversarially Regularized Graph Autoencoder for Graph Embedding. IJCAI International Joint Conference on Artificial Intelligence (IJCAI), 2018;2609-2615.

[58]

Yan S, Xu D, Zhang B, Zhang HJ, Yang Q, and Lin S Graph embedding and extensions: A general framework for dimensionality reduction IEEE Transactions on Pattern Analysis & Machine Intelligence. 2007 1 1 40-51

[59]

Goyal P and Ferrara E Graph embedding techniques, applications, and performance: A survey Knowledge-Based Systems. 2018 1 151 78-94

[60]

Cai H, Zheng VW, and Chang KC A comprehensive survey of graph embedding: Problems, techniques, and applications IEEE Transactions on Knowledge and Data Engineering. 2018 30 9 1616-37

[61]

Wang Q, Mao Z, Wang B, Guo L. Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering. 2017;1:29(12):2724-43.

[62]

Nie F, Zhu W, Li X. Unsupervised large graph embedding. In Thirty-first AAAI conference on artificial intelligence 2017;13.

[63]

Mao Q, Wang L, Tsang IW, Sun Y. Principal graph and structure learning based on reversed graph embedding. IEEE transactions on pattern analysis and machine intelligence. 2017;39(11):2227-41.

[64]

Ding W, Lin C, and Ishwar P Node embedding via word embedding for network community discovery IEEE Transactions on Signal and Information Processing over Networks. 2017 3 3 539-52

[65]

Xu H, Luo D, Zha H, Carin L. Gromov-Wasserstein Learning for Graph Matching and Node Embedding. arXiv preprint arXiv:1901.06003. 2019;17.

[66]

Cavallari S, Zheng VW, Cai H, Chang KC, Cambria E. Learning community embedding with community detection and node embedding on graphs. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management 2017;6(377-386). ACM

[67]

Faerman E, Borutta F, Fountoulakis K, Mahoney MW. Lasagne: Locality and structure aware graph node embedding. In2018 IEEE/WIC/ACM International Conference on Web Intelligence (WI) 2018;3(246-253). IEEE

[68]

Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, and Monfardini G The graph neural network model IEEE Transactions on Neural Networks. 2009 20 1 61-80

[69]

Monti F, Bronstein M, Bresson X. Geometric matrix completion with recurrent multi-graph neural networks. InAdvances in Neural Information Processing Systems 2017;( 3697-3707)

[70]

Seo Y, Defferrard M, Vandergheynst P, Bresson X. Structured sequence modeling with graph convolutional recurrent networks. In International Conference on Neural Information Processing 13 2018 Cham Springer 362-373

[71]

Niepert M, Ahmed M, Kutzkov K. Learning convolutional neural networks for graphs. In International conference on machine learning 2016;11(2014-2023).

[72]

Henaff M, Bruna J, LeCun Y. Deep convolutional networks on graph-structured data. arXiv preprint arXiv:1506.05163. 2015;16.

[73]

Hamilton W, Ying Z, Leskovec J. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems 2017;(1024-1034).

[74]

Wang C, Pan S, Long G, Zhu X, Jiang J. Mgae: Marginalized graph autoencoder for graph clustering. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management 2017;6(889-898). ACM

[75]

Kipf TN, Welling M. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308. 2016;21

[76]

Simonovsky M, Komodakis N. Graphvae: Towards generation of small graphs using variational autoencoders. In International Conference on Artificial Neural Networks 4 2018 Cham Springer 412-422

[77]

Tran PV. Learning to make predictions on graphs with autoencoders. In 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) 2018;1(237-245). IEEE

[78]

Wang D, Cui P, Zhu W. Structural deep network embedding. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining 2016;13(1225-1234). ACM

[79]

Li C, Wang S, Yang D, Li Z, Yang Y, Zhang X, and Zhou J. Ppne: Property preserving network embedding. In International Conference on Database Systems for Advanced Applications, 2017;163-179. Springer

[80]

Wang J, Yu L, Zhang W, Gong Y, Xu Y, Wang B, Zhang P, and Zhang D Irgan: A minimax game for unifying generative and discriminative information retrieval models 2017 In SIGIR ACM

[81]

Tang J, Qu M, Wang M, Zhang M, Yan J, Mei Q. Line: Large-scale information network embedding. In WWW, 1067-1077. Int World Wide Web Conferences Steering Committee 2015.

[82]

Denton EL, Chintala S, Fergus R et al. Deep generative image models using a Laplacian pyramid of adversarial networks. In NIPS, 2015;1486–1494.

[83]

Yu L, Zhang W, Wang J, Yu Y. Seqgan: Sequence generative adversarial nets with policy gradient, In AAAI, 2017;2852–2858.

[84]

Bojchevski A, Shchur O, Zgner D, Gnnemann S. GraphGAN: Generating Graphs via Random Walks 2018.

[85]

Tarawaneh RA, Keller P, Ebert A. A general introduction to graph visualization techniques. In Visualization of Large and Unstructured Data Sets: Applications in Geospatial Planning, Modeling and Engineering-Proceedings of IRTG 1131 Workshop 2011 2012. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik 2012.

[86]

Schulz HJ, Schumann H. Visualizing graphs-a generalized view. In Tenth International Conference on Information Visualisation (IV’06) 2006;5(166-173). IEEE

[87]

Hu Y and Shi L Visualizing large graphs Wiley Interdisciplinary Reviews: Computational Statistics. 2015 7 2 115-36

[88]

Liu X, Tian Y, He Q, Lee WC, McPherson J. Distributed graph summarization. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management 2014 3 (pp. 799-808). ACM

[89]

Wan X, Wang H, and Li J LKAQ: Large-scale knowledge graph approximate query algorithm Information Sciences. 2019 1 505 306-24

[90]

Arleo A, Didimo W, Liotta G, and Montecchiani F Large graph visualizations using a distributed computing platform Information Sciences 2017 381 124-141

[91]

Zheng W, Zou L, Lian X, Zhang H, Wang W, and Zhao D SQBC: An efficient subgraph matching method over large and dense graphs Information Sciences 2014 261 116-131

[92]

Livadas PE and Johnson T An optimal algorithm for the construction of the system dependence graph Information Sciences 2000 125 1–4 99-131

[93]

Baralis E, Cagliero L, Mahoto N, and Fiori A GRAPHSUM: Discovering correlations among multiple terms for graph-based summarization Information Sciences 2013 249 96-109

[94]

Vanetik N, Litvak M, Churkin E, and Last M An unsupervised constrained optimization approach to compressive summarization Information Sciences 2020 509 22-35

[95]

Zhou F, Qu Q, and Toivonen H Summarisation of weighted networks Journal of Experimental & Theoretical Artificial Intelligence 2017 29 5 1023-1052

[96]

Xie Y, Gong M, Qin AK, Tang Z, and Fan X TPNE: Topology preserving network embedding Information Sciences 2019 504 20-31

[97]

Constantin MG, Redi M, Zen G, Ionescu B. Computational understanding of visual interestingness beyond semantics: literature survey and analysis of covariates. ACM Computing Surveys (CSUR). 2019;27,52(2):25.

[98]

Gambhir Mahak and Gupta Vishal Recent automatic text summarization techniques: a survey Artificial Intelligence Review 2017 47 1 1-66

[99]

Kanapala Ambedkar, Pal Sukomal, and Pamula Rajendra Text summarization from legal documents: a survey Artificial Intelligence Review 2019 51 3 371-402

[100]

Cytoscape. online: https://cytoscape.org

[101]

Gephi. Online: https://gephi.org

[102]

Linkurious. Online: https://linkurio.us

[103]

SNAP. Online: http://snap.stanford.edu/snap

[104]

Pajek. Online: http://mrvar.fdv.uni-lj.si/pajek

[105]

IBM i2 Analyst’s Workstation. Online: https://www.ibm.com/us-en/marketplace/analysts-notebook

[106]

Polinode. Online: https://www.polinode.com

[107]

GUESS. Online: http://graphexploration.cond.org

[108]

Dong, Y, Chawla NV, Ananthram S. metapath2vec: Scalable representation learning for heterogeneous networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, 2017;135-144.

[109]

Chen H, Perozzi B, Hu Y, Skiena S. Harp: Hierarchical representation learning for networks. arXiv preprint arXiv:1706.07845, 2017.

[110]

Bhagat S, Graham C, Muthukrishnan S. Node classification in social networks. In Social network data analytics. Springer, Boston MA, 2011;115-148.

[111]

Liben Nowell D, Kleinberg J. The link prediction problem for social networks. Journal of the American society for information science and technology, 2007;7(1019-1031)58.

[112]

Vishwanathan S, Vichy N, Nicol N. Schraudolph, Risi K, Karsten M. Borgwardt. Graph kernels. The Journal of Machine Learning Research 2010;10(1201-1242).

[113]

Herman I, Guy M, Scott M. Graph visualization and navigation in information visualization: A survey. IEEE Transactions on visualization and computer graphics, 2000;1(24-43)6.

[114]

Bach B, Dragicevic P, Archambault D, Hurter C, and Carpendale S A descriptive framework for temporal data visualizations based on generalized space-time cubes Computer Graphics Forum 2017 36 6 36-61

[115]

Beck F, Burch M, Diehl S, and Weiskopf D A Taxonomy and Survey of Dynamic Graph Visualization Computer Graphics Forum 2017 36 1 133-159

[116]

Vehlow Corinna, Beck Fabian, and Weiskopf Daniel Visualizing group structures in graphs: A survey In Computer Graphics Forum 2017 36 6 201-225

[117]

Burch M, Vehlow C, Beck F, Diehl S, and Weiskopf D Parallel edge splatting for scalable dynamic graph visualization IEEE Transactions on Visualization and Computer Graphics 2011 17 12 2344-2353

[118]

Bezerianos A, Chevalier F, Dragicevic P, Elmqvist N, Fekete J.D. GraphDice: A system for exploring multivariate social networks. In Computer Graphics Forum, 2010;29(863-872). https://doi.org/10.1111/j1467-8659.2009.01687.x

[119]

Hadlak S, Schulz H-J, and Schumann H In situ exploration of large dynamic networks IEEE Transactions on Visualization and Computer Graphics 2011 17 12 2334-2343

[120]

Bach B, Kerracher N, Hall KW, Carpendale S, Kennedy J, Henry Riche N. Telling stories about dynamic networks with graph comics. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI 16), 2016;3670-3682. https://doi.org/10.1145/2858036.2858387

[121]

van den Elzen S, Danny H, Jorik B, Jarke J. van Wijk. Dynamic network visualization withextended massive sequence views. IEEE transactions on visualization and computer graphics, 2013;20(8)1087-1099.

[122]

Lee A, Daniel A, Miguel A. Nacenta. The Effectiveness of Interactive Visualization Techniques for Time Navigation of Dynamic Graphs on Large Displays. arXiv preprint arXiv:2008.12747 2020.

[123]

Cspedes-Hernandez D, Juan Manuel G. C, Josefina G. G, Liliana R. V. Gathering, Classifying and Visualizing Results from Social Surveys using OCR and Machine Learning Techniques. Res Comput Sci 2017;145:(81-95).

[124]

Rodrigues J. F, Traina A. J. M, Faloutsos C, Traina C. SuperGraph visualization. In Proceedings of the IEEE International Symposium on Multimedia (ISM06),Washington, DC, USA, 2006.

[125]

MA K.L. Image graphs - A novel approach to visual data exploration. In Proceedings of IEEE Visualization Conference 1999;(81-88).

[126]

JANKUN-KELLY T. J., MA K. L.: Visualization exploration and encapsulation via a spreadsheet-like interface. IEEE Transactions on Visualization and Computer Graphics 2001;7(3)275-287.

[127]

JANKUN-KELLY T., MA K.-L., GERTZ M.: A model for the visualization exploration process. In Proceedings of IEEE Visualization Conference 2002;323-330.

[128]

Woodring J, Shen H.W.: Multi-variate, time-varying, and comparative visualization with contextual cues. IEEE Transactions on Visualization and Computer Graphics 2006;12(5)909-916.

[129]

Wohlfart M, Hauser H. Story telling for presentation in volume visualization. In Proceedings of Joint Eurographics - IEEE VGTC Symposium on Visualization 2007;91-98.

[130]

Akiba H, Wang C, Ma K. L. AniViz: A template based animation tool for volume visualization. IEEE Computer Graphics and Applications 2010;30(5)61-71.

[131]

Bruckner S, M’OLLER T. Result-driven exploration of simulation parameter spaces for visual effects design. IEEE Transactions on Visualization and Computer Graphics 2010;16(6)1468-1476.

[132]

Purchase HC. Twelve years of diagrams research. Elsevier Journal of Visual Languages & Computing. 2014;1:2(57-75)25.

[133]

Praczyk PA, Nogueras-Iso J. Automatic extraction of figures from scientific publications in high-energy physics. Information Technology and Libraries. 2013;22:32(4):25-52.

[134]

Liu Y, Xiaoqing L, Yeyang Q, Zhi T, Jianbo X. Review of chart recognition in document images. In Visualization and Data Analysis, International Society for Optics and Photonics, 2013;8654(865410).

[135]

Jung D, Kim W, Song H, Hwang JI, Lee B, Kim B, Seo J. Chartsense: Interactive data extraction from chart images. In Proceedings of the 2017 chi conference on human factors in computing systems 2017;2(6706-6717).

[136]

Choi J, Jung S, Park DG, Choo J, Elmqvist N. Visualizing for the Non-Visual: Enabling the Visually Impaired to Use Visualization. In Computer Graphics Forum 2019;38(249-260)3.

[137]

Poco J, Heer J. Reverse-engineering visualizations: Recovering visual encodings from chart images. In Computer Graphics Forum 2017;36:3(353-363).

[138]

Huang W, Tan CL. A system for understanding imaged infographics and its applications. In Proceedings of the 2007 ACM symposium on Document engineering 2007;28(9-18).

[139]

Blue Leaf Software - Dagra. [Online] Link: https://blueleafsoftware.com

[140]

Plot Digitizer. [Online] Link: http://plotdigitizer.sourceforge.net

[141]

Engauge Digitizer. [Online] Link: http://markummitchell.github.io/engauge-digitizer/

[142]

Web Plot Digitizer. [Online] Link: https://automeris.io/WebPlotDigitizer/

[143]

Graph Reader. [Online] Link: http://www.graphreader.com

[144]

Data Thief. [Online] Link: https://datathief.org

[145]

Zhou F, Zhao Y, Chen W, Tan Y, Xu Y, Chen Y, Liu C, and Zhao Y Reverse-engineering bar charts using neural networks Journal of Visualization. 2021 24 2 419-35

[146]

Liu X, Klabjan D, NBless P. Data extraction from charts via single deep neural network. arXiv preprint arXiv:1906.11906. 2019;6.

[147]

Balaji A, Ramanathan T, Sonathi V. Chart-text: A fully automated chart image descriptor. arXiv preprint arXiv:1812.10636. 2018;27.

[148]

Savva M, Kong N, Chhajta A, Fei-Fei L, Agrawala M, Heer J. Revision: Automated classification, analysis and redesign of chart images. In Proceedings of the 24th annual ACM symposium on User interface software and technology 2011;16(393-402).

[149]

Zhou YP, Tan CL. Learning-based scientific chart recognition. In 4th IAPR International Workshop on Graphics Recognition, GREC 2001;7(482-492).

[150]

Svendsen JP. Chart detection and recognition in graphics intensive business documents (Doctoral dissertation).

[151]

Cliche M, Rosenberg D, Madeka D, Yee C. Scatteract: Automated extraction of data from scatter plots. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, Cham, 2017;18(135–150).

[152]

Dai W, Wang M, Niu Z, Zhang J. Chart decoder: Generating textual and numeric information from chart images automatically. Journal of Visual Languages & Computing. 2018;1:48(101–9).

[153]

Al-Zaidy RA, Giles CL. A machine learning approach for semantic structuring of scientific charts in scholarly documents. In Twenty-Ninth IAAI Conference 2017;8.

[154]

Molla MK, Talukder KH, Hossain MA. Line chart recognition and data extraction technique. InInternational Conference on Intelligent Data Engineering and Automated Learning 21 2003 Berlin, Heidelberg Springer 865-870

[155]

Reddy VK, Kaushik CM. Image processing based data extraction from graphical representation. In 2015 IEEE International Conference on Computer Graphics, Vision and Information Security (CGVIS) 2015;2(190-194). IEEE

[156]

Chester D, Elzer S. Getting computers to see information graphics so users do not have to. In International Symposium on Methodologies for Intelligent Systems 25 2005 Berlin, Heidelberg Springer 660-668

[157]

Mishchenko A, Vassilieva N. Chart image understanding and numerical data extraction. In2011 Sixth International Conference on Digital Information Management 2011;26(115-120). IEEE

[158]

Browuer W, Kataria S, Das S, Mitra P, Giles CL. Segregating and extracting overlapping data points in two-dimensional plots. In Proceedings of the 8th ACM/IEEE-CS joint conference on Digital libraries 2008;16(276-279).

[159]

Ray CS, Wang S, Giles CL. Curve separation for line graphs in scholarly documents. In Proceedings of the 16th ACM/IEEE-CS on Joint Conference on Digital Libraries 2016;19(277-278).

[160]

De P. Automatic data extraction from 2D and 3D pie chart images. In 2018 IEEE 8th International Advance Computing Conference (IACC) 2018;14(20-25). IEEE

[161]

Ma Y, Tung AK, Wang W, Gao X, Pan Z, Chen W. Scatternet: A deep subjective similarity model for visual analysis of scatterplots. IEEE transactions on visualization and computer graphics. 2018;12:26(3):1562-76.

[162]

Tang B, Liu X, Lei J, Song M, Tao D, Sun S, and Dong F Deepchart: Combining deep convolutional networks and deep belief networks in chart classification Signal Processing. 2016 1 124 156-61

[163]

Karthikeyani V and Nagarajan S Machine learning classification algorithms to recognize chart types in portable document format (pdf) files International Journal of Computer Applications. 2012 39 2 1-5

[164]

Demir S, Carberry S, McCoy KF. Summarizing information graphics textually. Computational Linguistics. 2012;1:38(3):527-74.

[165]

Chen C, Zhang R, Koh E, Kim S, Cohen S, Rossi R. Figure Captioning with Relation Maps for Reasoning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 2020;(1537-1545).

[166]

Bhatia S, Mitra P. Summarizing figures, tables, and algorithms in scientific publications to augment search results. ACM Transactions on Information Systems (TOIS). 2012 6;30(1):1-24.

[167]

Demir S, Carberry S, Elzer S. Effectively realizing the inferred message of an information graphic. InProceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP) 2007;27(150-156).

[168]

Choudhury SR, Wang S, Giles CL. Scalable algorithms for scholarly figure mining and semantics. In Proceedings of the International Workshop on Semantic Big Data 2016;26(1-6).

[169]

Mahmood A, Bajwa IS, Qazi K. An automated approach for interpretation of statistical graphics. In2014 Sixth International Conference on Intelligent Human-Machine Systems and Cybernetics 2014;26:2(376-379). IEEE

[170]

Kallimani JS, Srinivasa KG, Eswara RB. Extraction and interpretation of charts in technical documents. In2013 International Conference on Advances in Computing, Communications and Informatics (ICACCI) 2013;22(382-387). IEEE

[171]

Demir S, Schwartz SE, Burns R, Carberry S. What is being Measured in an Information Graphic?. In International Conference on Intelligent Text Processing and Computational Linguistics 24 2013 Berlin, Heidelberg Springer 501-512

[172]

Elzer S, Carberry S, Demir S. Communicative signals as the key to automated understanding of simple bar charts. InInternational Conference on Theory and Application of Diagrams 28 2006 Berlin, Heidelberg Springer 25-39

[173]

Carberry S, Elzer S, Demir S. Information graphics: an untapped resource for digital libraries. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval 2006;6(581-588)

[174]

Burns R, Carberry S, and Elzer Schwartz S An automated approach for the recognition of intended messages in grouped bar charts Computational Intelligence. 2019 35 4 955-1002

[175]

Greenbacker C, Wu P, Carberry S, McCoy KF, Elzer S. Abstractive summarization of line graphs from popular media. In Proceedings of the Workshop on Automatic Summarization for Different Genres, Media, and Languages 2011;(41-48).

[176]

Burns R, Balawejder E, Domanowska W, Schwartz SE, Carberry S. Exploring the Types of Messages that Pie Charts Convey in Popular Media. In International Conference on Theory and Application of Diagrams 7 2016 Cham Springer 265-271

[177]

Nair RR, Sankaran N, Nwogu I, Govindaraju V. Understanding line plots using Bayesian Network. In 2016 12th IAPR Workshop on Document Analysis Systems (DAS) 2016;11(108-113). IEEE

[178]

Al-Zaidy RA, Choudhury SR, Giles CL. Automatic summary generation for scientific data charts. In Workshops at the thirtieth aaai conference on artificial intelligence 2016;29.

[179]

Di Sorbo A, Panichella S, Alexandru CV, Visaggio CA, Canfora G. SURF: summarizer of user reviews feedback. In 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C) 2017;20(55-58). IEEE

[180]

Xu B, Xing Z, Xia X, Lo D. AnswerBot: Automated generation of answer summary to developers’ technical questions. In 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE) 2017:1(706-716). IEEE

[181]

Woodsend K, Lapata M. Automatic generation of story highlights. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics 2010;(565-574).

[182]

Leppanen L, Munezero M, Granroth-Wilding M, Toivonen H. Data-driven news generation for automated journalism. In Proceedings of the 10th International Conference on Natural Language Generation 2017;(188-197).

[183]

Demir S, Oliver D, Schwartz E, Elzer S, Carberry S, Mccoy KF, Chester D. Interactive SIGHT: textual access to simple bar charts. New Review of Hypermedia and Multimedia. 2010;1:16(3):245-79.

[184]

Demir S, Carberry S, McCoy KF. Generating textual summaries of bar charts. InProceedings of the Fifth International Natural Language Generation Conference 2008;(7-15).

[185]

Moraes P, Sina G, McCoy K, Carberry S. Generating summaries of line graphs. In Proceedings of the 8th International Natural Language Generation Conference (INLG) 2014;(95-98).

[186]

Elzer S, Carberry S, Zukerman I. The automated understanding of simple bar charts. Artificial Intelligence. 2011;1:175(2):526-55.

[187]

Nair RR, Sankaran N, Nwogu I, Govindaraju V. Automated analysis of line plots in documents. In 2015 13th international conference on document analysis and recognition (icdar) 2015;23(796-800). IEEE

[188]

Methani N, Ganguly P, Khapra MM, Kumar P. Plotqa: Reasoning over scientific plots. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 2020;(1527-1536).

[189]

Kafle K, Price B, Cohen S, Kanan C. Dvqa: Understanding data visualizations via question answering. InProceedings of the IEEE conference on computer vision and pattern recognition 2018;(5648-5656).

[190]

Amara J, Kaur P, Owonibi M, Bouaziz B. Convolutional neural network based chart image classification 2017.

[191]

Gao J, Carrillo RE, Barner KE. Image categorization for improving accessibility to information graphics. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility 2010;25(265-266).

[192]

Siddiqui SA, Malik MI, Agne S, Dengel A, and Ahmed S Decnt: Deep deformable cnn for table detection IEEE Access. 2018 20 6 74151-61

[193]

Schreiber S, Agne S, Wolf I, Dengel A, Ahmed S. Deepdesrt: Deep learning for detection and structure recognition of tables in document images. In 2017 14th IAPR international conference on document analysis and recognition (ICDAR) 2017;9(1)1162-1167. IEEE

[194]

Vo ND, Nguyen K, Nguyen TV, Nguyen K. Ensemble of deep object detectors for page object detection. In Proceedings of the 12th International Conference on Ubiquitous Information Management and Communication 2018;5(1-6).

[195]

Saha R, Mondal A, Jawahar CV. Graphical object detection in document images. In 2019 International Conference on Document Analysis and Recognition (ICDAR) 2019;20 (51-58). IEEE

[196]

Gilani A, Qasim SR, Malik I, Shafait F. Table detection using deep learning. In2017 14th IAPR international conference on document analysis and recognition (ICDAR) 2017;9 :1(771-776). IEEE

[197]

Younas J, Siddiqui SA, Munir M, Malik MI, Shafait F, Lukowicz P, and Ahmed S Fi-Fo Detector: Figure and Formula Detection Using Deformable Networks Applied Sciences. 2020 10 18 6460

[198]

Agarwal M, Mondal A, Jawahar CV. Cdec-net: Composite deformable cascade network for table detection in document images. In 2020 25th International Conference on Pattern Recognition (ICPR) 2021;10(9491-9498). IEEE

[199]

Mei H, Ma Y, Wei Y, and Chen W The design space of construction tools for information visualization: A survey Journal of Visual Languages & Computing. 2018 1 44 120-32

[200]

Mishra P, Kumar S, Chaube MK. Dissimilarity Based Regularized Deep Learning Model for Information Charts. In 2020 Joint 9th International Conference on Informatics, Electronics & Vision (ICIEV) and 2020 4th International Conference on Imaging, Vision & Pattern Recognition (icIVPR) 2020;26(1-6). IEEE

[201]

Harper J, Agrawala M. Converting basic D3 charts into reusable style templates. IEEE transactions on visualization and computer graphics. 2017;7:24(3):1274-86.

[202]

Kong N, Agrawala M. Graphical overlays: Using layered elements to aid chart reading. IEEE transactions on visualization and computer graphics. 2012;8:18(12):2631-8.

[203]

Mendez GG, Nacenta MA, Vandenheste S. iVoLVER: Interactive visual language for visualization extraction and reconstruction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems 2016;7(4073-4085).

[204]

Harper J, Agrawala M. Deconstructing and restyling D3 visualizations. In Proceedings of the 27th annual ACM symposium on User interface software and technology 2014;5(253-262).

[205]

Poco J, Mayhua A, Heer J. Extracting and retargeting color mappings from bitmap images of visualizations. IEEE transactions on visualization and computer graphics. 2017;29:24(1):637-46.

[206]

Kim Y, Wongsuphasawat K, Hullman J, Heer J. Graphscape: A model for automated reasoning about visualization similarity and sequencing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems 2017;2(2628-2638).

[207]

Burns R, Schwartz SE, Carberry S. Towards Adapting Information Graphics to Individual Users to Support Recognizing Intended Messages. In UMAP Workshops 2013.

[208]

Chen Z, Wang Y, Wang Q, Wang Y, Qu H. Towards automated infographic design: Deep learning-based auto-extraction of extensible timeline. IEEE transactions on visualization and computer graphics. 2019;20:26(1):917-26.

[209]

Srinivasan A, Drucker SM, Endert A, Stasko J. Augmenting visualizations with interactive data facts to facilitate interpretation and communication. IEEE transactions on visualization and computer graphics. 2018;20:25(1):672-81.

[210]

Kong N, Hearst MA, Agrawala M. Extracting references between text and charts via crowdsourcing. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems 2014;26(31-40).

[211]

Kahou SE, Michalski V, Atkinson A, Kdr, Trischler A, Bengio Y. Figureqa: An annotated figure dataset for visual reasoning. arXiv preprint arXiv:1710.07300. 2017;19.

[212]

Kafle K, Shrestha R, Cohen S, Price B, Kanan C. Answering questions about data visualizations using efficient bimodal fusion. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 2020;(1498-1507).

[213]

Li Z, Carberry S, Fang H, McCoy KF, Peterson K, Stagitis M. A novel methodology for retrieving infographics utilizing structure and message content. Data & Knowledge Engineering. 2015;1:100(191–210).

[214]

Hoque E, Agrawala M. Searching the visual style and structure of d3 visualizations. IEEE transactions on visualization and computer graphics. 2019;22:26(1):1236-45.

[215]

Lee PS, West JD, Howe B. Viziometrics: Analyzing visual information in the scientific literature. IEEE Transactions on Big Data. 2017;2:4(1):117-29.

[216]

Chen Z, Cafarella M, Adar E. Diagramflyer: A search engine for data-driven diagrams. In Proceedings of the 24th International Conference on World Wide Web 2015;18(183-186).

[217]

Bylinskii Z, Kim NW, O’Donovan P, Alsheikh S, Madan S, Pfister H, Durand F, Russell B, Hertzmann A. Learning visual importance for graphic designs and data visualizations. In Proceedings of the 30th Annual ACM symposium on user interface software and technology 2017;20(57-69).

[218]

Lee DJ, Dev H, Hu H, Elmeleegy H, Parameswaran A. Avoiding drill-down fallacies with vispilot: Assisted exploration of data subsets. In Proceedings of the 24th International Conference on Intelligent User Interfaces 2019;17(186-196).

[219]

Chaudhry R, Shekhar S, Gupta U, Maneriker P, Bansal P, Joshi A. Leaf-qa: Locate, encode & attend for figure question answering. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 2020;(3512-3521).

[220]

Siegel N, Horvitz Z, Levin R, Divvala S, Farhadi A. Figureseer: Parsing result-figures in research papers. In European Conference on Computer Vision 2016;8(664–680). Springer, Cham

[221]

Davila K, Kota BU, Setlur S, Govindaraju V, Tensmeyer C, Shekhar S, Chaudhry R. ICDAR 2019 Competition on Harvesting Raw Tables from Infographics (CHART-Infographics). In 2019 International Conference on Document Analysis and Recognition (ICDAR) 2019;20(1594-1599). IEEE

[222]

Gbel M, Hassan T, Oro E, Orsi G. ICDAR 2013 table competition. In 2013 12th International Conference on Document Analysis and Recognition 2013;25(1449-1453). IEEE

[223]

Li M, Xu Y, Cui L, Huang S, Wei F, Li Z, Zhou M. DocBank: A benchmark dataset for document layout analysis. arXiv preprint arXiv:2006.01038. 2020;1.

[224]

Zhong X, Tang J, Yepes AJ. Publaynet: largest dataset ever for document layout analysis. In 2019 International Conference on Document Analysis and Recognition (ICDAR) 2019;20(1015-1022). IEEE

[225]

Gao L, Yi X, Jiang Z, Hao L, Tang Z. ICDAR2017 competition on page object detection. In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) 2017;1(1417-1422). IEEE

Cited By

Robinson Moore WKalal MTennison JGiudice NGorlewicz J(2024)Spatial Audio-Enhanced Multimodal Graph Rendering for Efficient Data Trend Learning on Touchscreen DevicesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641959(1-12)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641959

Index Terms

Graph Interpretation, Summarization and Visualization Techniques: A Review and Open Research Issues

Index terms have been assigned to the content through auto-classification.

Recommendations

Incremental Lossless Graph Summarization
KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Given a fully dynamic graph, represented as a stream of edge insertions and deletions, how can we obtain and incrementally update a lossless summary of its current snapshot? As large-scale graphs are prevalent, concisely representing them is inevitable ...
Graph Summarization Methods and Applications: A Survey

While advances in computing resources have made processing enormous amounts of data possible, human ability to identify patterns in such data has not scaled accordingly. Efficient computational methods for condensing and simplifying data are thus ...
Graph Summarization for Entity Relatedness Visualization
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

In modern search engines, Knowledge Graphs have become a key component for knowledge discovery. When a user searches for an entity, the existing systems usually provide a list of related entities, but they do not necessarily give explanations of how ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Multimedia Tools and Applications

Multimedia Tools and Applications Volume 82, Issue 6

Mar 2023

1544 pages

ISSN:1380-7501

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2021.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 08 January 2022

Accepted: 20 September 2021

Revision received: 02 August 2021

Received: 18 November 2020

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Robinson Moore WKalal MTennison JGiudice NGorlewicz J(2024)Spatial Audio-Enhanced Multimodal Graph Rendering for Efficient Data Trend Learning on Touchscreen DevicesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641959(1-12)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641959

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents