Abstract
In Greek art, the phase from 900 to 700 BCE is referred to as the Geometric period due to the characteristically simple geometry-like ornamentations appearing on painted pottery surfaces during this era. Distinctive geometric patterns are typical for specific periods, regions, workshops as well as painters and are an important cue for archaeological tasks, such as dating and attribution. To date, these analyses are mostly conducted with the support of information technology. The primitives of an artefact’s ornamentation can be generally classified into a set of distinguishable pattern classes, which also appear in a similar fashion on other objects. Although a taxonomy of known pattern classes is given in subject-specific publications, the automatic detection and classification of surface patterns from object depictions poses a non-trivial challenge. Our long-term goal is to provide this classification functionality using a specifically designed and trained neural network. This, however, requires a large amount of labelled training data, which at this point does not exist for this domain context. In this work, we propose an effective annotation system, which allows a domain expert to interactively segment and label parts of digitized vessel surfaces. These user inputs are constantly fed back to a Convolutional Neural Network (CNN), enabling the prediction of pattern classes for a given surface area with ever increasing precision. Our work paves the way for a fully automatic classification and analysis of large surface pattern collections, which, with the help of suitable visual analysis techniques, can answer research questions like pattern variability or change over time. While the capability of our proposed annotation pipeline is demonstrated at the example of two characteristic Greek pottery artefacts from the Geometric period, the proposed methods can be readily adopted for the patternation in any other chronological periods as well as for stamped motifs.
Funding source: Austrian Science Fund
Funding source: Agencia Nacional de Investigación y Desarrollo
Award Identifier / Grant number: ICN17_002
Funding statement: This article was produced within the interdisciplinary project “Crossmodal Search and Visual Exploration of 3D Cultural Heritage Objects” (P31317-NBL) co-funded by the Austrian Science Fund FWF and the State of Styria, Austria. Benjamin Bustos was funded by ANID – Millennium Science Initiative Program – Code ICN17_002.
About the authors
Dipl.-Ing. Stefan Lengauer is doing his PhD in the field of crossmodal search and visual exploration of cultural heritage objects. Since 2018 he is a member at Institute of Computer Graphics and Knowledge Visualization and has authored several publications on the topic of shape-based and motif-based retrieval, as well as interactive visualization and reconstruction of ancient pottery.
Dipl.-Ing. Peter Houska is a PhD student at the Institute of Computer Graphics and Knowledge Visualization at Graz University of Technology. His research interests include interactive visualization, geometry processing, and digital archaeology.
Dr. techn. Reinhold Preiner is a senior researcher in computer science, specialized in 3D computer graphics and visualization. He received his PhD from TU Wien in 2017 and is now with the Institute of Computer Graphics and Knowledge Visualization at Graz University of Technology. His main fields of research include applied geometry processing, rendering, and interactive visualization, involving beside others applications in digital archaeology.
Dr. phil. Elisabeth Trinkl is a staff member of the Institute of Classics at Graz University. She received her PhD in Classical Archaeology at Vienna University in 1998. Her research interests focus on Greek archaeology in general, the understanding of ancient textiles and the use of computer-assisted methods in archaeological research. She is author of several articles on these topics, a. o. of a volume of the international publication series Corpus Vasorum Antiquorum (CVA), and editor of collections of essays.
Dr. phil. Stephan Karl is a scientific member of the Institute of Classics at Graz University. He received his PhD in Classical Archaeology in 2013. His research interests include the study of Early Greek pottery, Roman provincial stone monuments and Roman marble quarries, with a special focus on the application of 3D technologies for visualising and analysing purposes in archaeological research. He is a. o. author of two volumes of the international publication series Corpus Vasorum Antiquorum (CVA).
Prof. Dr. Ivan Sipiran is an Assistant Professor at the Department of Computer Science, University of Chile. He received the PhD in computer science from the University of Chile in 2014. He works in the areas of geometry processing, shape analysis and the application of computer graphics in cultural heritage.
Prof. Dr. Benjamin Bustos is a Full Professor at the Department of Computer Science, University of Chile, and Associate Researcher at the Millennium Institute Foundational Research on Data. He received the Doctoral degree in natural sciences from the University of Konstanz, Germany, in 2006. He leads research projects in the domain of content-based multimedia information retrieval. His research interests include similarity search, 3D object retrieval, multimedia mining, metric/nonmetric indexing, and pattern recognition.
Prof. Dr. Tobias Schreck is a Professor and head of the Institute of Computer Graphics and Knowledge Visualization at Graz University of Technology. He previously was Assistant Professor with University of Konstanz, Germany, and Postdoc Fellow with Technical University of Darmstadt. He obtained a PhD in Computer Science in 2006 from the University of Konstanz. Tobias Schreck works in the areas of Visual Analytics, Information Visualization, and Applied 3D Object Retrieval.
Acknowledgment
We are very grateful to the editors and the reviewers for their valuable comments and suggestions that helped to improve the manuscript.
References
1. N. Kunisch. Ornamente geometrischer Vasen. Ein Kompendium. Cologne-Weimar, Böhlau Verlag, 1998.Search in Google Scholar
2. W. Liu, Z. Wang, X. Liu, N. Zeng, Y. Liu and F.E. Alsaadi. A survey of deep neural network architectures and their applications. Neurocomputing 234, 2017, 11–26.10.1016/j.neucom.2016.12.038Search in Google Scholar
3. E. Beauxis-Aussalet, M. Behrisch, R. Borgo, D. Horng Chau, C. Collins, D. Ebert, M. El-Assady, A. Endert, D. Keim, J. Kohlhammer, D. Oelke, J. Peltonen, M. Riveiro, T. Schreck, H. Strobelt and J. van Wijk. The Role of Interactive Visualization in Fostering Trust in AI. IEEE Computer Graphics and Applications 41(6), 2021, 7–12.10.1109/MCG.2021.3107875Search in Google Scholar PubMed
4. A. Endert, W. Ribarsky, C. Turkay, W. Wong, I. Nabney, I. Díaz Blanco and F. Rossi. The State of the Art in Integrating Machine Learning into Visual Analytics. Comput. Graph. Forum 36(8), 2017, 458–486.10.1111/cgf.13092Search in Google Scholar
5. M. Chegini, J. Bernard, J. Cui, F. Chegini, A. Sourin, K. Andrews and T. Schreck. Interactive visual labelling versus active learning: an experimental comparison. Frontiers of Information Technology & Electronic Engineering 21(4), 2020, 524–535.10.1631/FITEE.1900549Search in Google Scholar
6. J. Bernard, M.Zeppelzauer, M. Sedlmair and W. Aigner. A unified process for visual-interactive labeling. In: Proceedings of the EuroVis Workshop on Visual Analytics, 2017, 73–77.Search in Google Scholar
7. S. Lengauer, A. Komar, S. Karl, E. Trinkl, I. Sipiran, T. Schreck and R. Preiner. Semi-automated Annotation of Repetitive Ornaments on 3D Painted Pottery Surfaces. In: Eurographics Workshop on Graphics and Cultural Heritage. Eurographics Assoc., 2020.Search in Google Scholar
8. J. Zhou, K. Smith, G. Wilsbacher, P. Sagona, D. Reddy and B. Torkian. Building Science Gateways for Humanities. In: Practice and Experience in Advanced Research Computing, 2020, 327–332.10.1145/3311790.3396628Search in Google Scholar
9. E.J. Crowley and A. Zisserman. Of Gods and Goats: Weakly Supervised Learning of Figurative Art. In: T. Burghardt, D. Damen, W. Mayol-Cuevas and M. Mirmehdi (eds.), Proceedings of the British Machine Vision Conference (Bristol, UK; September 9–13, 2013), 2013, art. 39, 1–11.Search in Google Scholar
10. S. Biasotti, E. Moscoso Thompson, M. Aono, H. Ben, B. Bustos, S. Dong, B. Du, A. Fehri, H. Amin, H. Li, F.A. Limberger, M. Masoumi, M. Rezaei, I. Sipiran, L. Sun, A. Tatsuma, S. Velasco Forero, R.C. Wilson, Y. Wu, J. Zhang, T. Zhao, F. Fornasa and A. Giachetti. SHREC’17 track: Retrieval of surfaces with similar relief patterns. In: I. Pratikakis, F. Dupont and M. Ovsjanikov (eds.), 10th Eurographics Workshop on 3D Object Retrieval (Lyon, France; April 23–24, 2017), The Eurographics Association, 2017, 95–103.Search in Google Scholar
11. S. Biasotti, E. Moscoso Thompson, L. Bathe, S. Berretti, A. Giachetti, T. Lejemble, N. Mellado, K. Moustakas, I. Manolas, D. Dimou, C. Tortorici, S. Velasco-Forero, N. Werghi, M. Polig, G. Sorrentino and S. Hermon. SHREC’18 track: Recognition of geometric patterns over 3D models. In: A.C. Telea, T. Theoharis and R.C. Veltkamp (eds.), 11th Eurographics Workshop on 3D Object Retrieval (Delft, Netherlands; April 16, 2018), The Eurographics Association, 2018, 71–77.Search in Google Scholar
12. E. Moscoso Thompson, C. Tortorici, N. Werghi, S. Berretti, S. Velasco-Forero and S. Biasotti. SHREC’18 track: Retrieval of gray patterns depicted on 3D models. In: A.C. Telea, T. Theoharis and R.C. Veltkamp (eds.), 11th Eurographics Workshop on 3D Object Retrieval (Delft, Netherlands; April 16, 2018), The Eurographics Association, 2018, 63–69.Search in Google Scholar
13. E. Moscoso Thompson, S. Biasotti, A. Giachetti, C. Tortorici, N. Werghi, A. Shaker Obeid, S. Berretti, H.-P. Nguyen-Dinh, M.-Q. Le, H.-D. Nguyen, M.-T. Tran, L. Gigli, S. Velasco-Forero, B. Marcotegui, I. Sipiran, B. Bustos, I. Romanelis, V. Fotis, G. Arvanitis, K. Moustakas, E. Otu, R. Zwiggelaar, D. Hunter, Y. Liu, Y. Arteaga and R. Luxman. SHREC 2020: Retrieval of digital surfaces with similar geometric reliefs. Computers & Graphics 91, 2020, 199–218.10.1016/j.cag.2020.07.011Search in Google Scholar
14. S. Lengauer, I. Sipiran, R. Preiner, T. Schreck and B. Bustos. A Benchmark Dataset for Repetitive Pattern Recognition on Textured 3D Surfaces. In: K. Crane and J. Digne (eds.), Eurographics Symposium on Geometry Processing 2021 (Toronto, Canada; July 12–14, 2021), Computer Graphics Forum 40, 2021, 1–8.Search in Google Scholar
15. C. Steckner. Begriffliche und empirische Objektordnung. In: C. Steckner (eds.), Archäologie und Neue Technologien, Schriften des Deutschen Archäologen-Verbandes 10, Deutscher Archäologen-Verband, Freiburg, 1988, 102–139.Search in Google Scholar
16. R. Unnikrishnan and R. Smith. Combined script and page orientation estimation using the tesseract ocr engine. In: Proceedings of the international workshop on multilingual OCR, 2009, 1–7.10.1145/1577802.1577809Search in Google Scholar
17. B. Rieck, H. Mara and S. Krömker. Unwrapping highly-detailed 3d meshes of rotationally symmetric man-made objects. In: ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2013, 259–264.10.5194/isprsannals-II-5-W1-259-2013Search in Google Scholar
18. Y. Liu, B. Zhang and L. Wan. Automatically unrolling decorations painted on 3D pottery. In: Proceedings of the Computer Graphics International Conference, 2017, 1–6.10.1145/3095140.3095177Search in Google Scholar
19. R. Preiner, S. Karl, P. Bayer and T. Schreck. Elastic Flattening of Painted Pottery Surfaces. In: R. Sablatnig and M. Wimmer (eds.), 16th Eurographics Workshop on Graphics and Cultural Heritage (Vienna, Austria; November 12–15, 2018), The Eurographics Association, 2018, 165–168.Search in Google Scholar
20. P. Houska, S. Lengauer, S. Karl and R. Preiner. Direct Elastic Unrollings of Painted Pottery Surfaces from Sparse Image Sets. In: A. Chalmers and V. Hulusic (eds.), 19th Eurographics Workshop on Graphics and Cultural Heritage (virtual, November 4–6, 2021), The Eurographics Association, 2021, 131–140.Search in Google Scholar
21. R.C. Gonzalez and R.E. Woods. Digital Image Processing (3rd Edition), Prentice-Hall, Inc., 2006.Search in Google Scholar
22. Y.J. Lee, C.L. Zitnick and M.F. Cohen. Shadowdraw: real-time user guidance for freehand drawing. ACM Transactions on Graphics (TOG) 30(4), 2011, 1–10.10.1145/1964921.1964922Search in Google Scholar
23. N. Otsu. A Threshold Selection Method from Gray-Level Histograms. IEEE Transactions on Systems, Man, and Cybernetics 9(1), 1979, 62–66.10.1109/TSMC.1979.4310076Search in Google Scholar
24. J. Gama, I. Zliobaite, A. Bifet, M. Pechenizkiy and A. Bouchachia. A Survey on Concept Drift Adaptation. ACM Comput. Surv. 46, 2014.10.1145/2523813Search in Google Scholar
25. A. Bhattacharyya. On a measure of divergence between two statistical populations defined by their probability distributions. Bull. Calcutta Math. Soc. 35, 1943, 99–109.Search in Google Scholar
26. K. Kübler. Ergebnisse der Ausgrabungen V 1: Die Nekropole des 10. bis 8. Jahrhunderts. Walter de Gruyter & Co., Berlin, 1954.Search in Google Scholar
27. J. Bobadilla, F. Ortega, A. Hernando and J. Bernal. A collaborative filtering approach to mitigate the new user cold start problem. Knowledge-based systems 26, 2012, 225–238.10.1016/j.knosys.2011.07.021Search in Google Scholar
28. S. Minaee, YY. Boykov, F. Porikli, AJ. Plaza, N. Kehtarnavaz and D. Terzopoulos. Image segmentation using deep learning: A survey. IEEE transactions on pattern analysis and machine intelligence, 2021.10.1109/TPAMI.2021.3059968Search in Google Scholar PubMed
29. K.U. Barthel. Improved image retrieval using automatic image sorting and semi-automatic generation of image semantics. In: 2008 Ninth International Workshop on Image Analysis for Multimedia Interactive Services. IEEE, 2008.10.1109/WIAMIS.2008.56Search in Google Scholar
30. F. Windhager, S. Salisu, R.A. Leite, V.A. Filipov, S. Miksch, G. Schreder and E. Mayr. Many Views Are Not Enough: Designing for Synoptic Insights in Cultural Collections. IEEE computer graphics and applications 40(3), 2020, 58–71.10.1109/MCG.2020.2985368Search in Google Scholar PubMed
31. S. Lengauer, A. Komar, S. Karl, E. Trinkl, R. Preiner and T. Schreck. Visual Exploration of Cultural Heritage Collections with Linked Spatiotemporal, Shape and Metadata Views. In: J. Krüger, M. Niessner and J. Stückler (eds.), 25th International Symposium on Vision, Modeling, and Visualization (Tübingen, Germany, virtual; September 28–October 1, 2020), The Eurographics Association, 2020, 137–144.Search in Google Scholar
© 2022 Walter de Gruyter GmbH, Berlin/Boston