Abstract
A software system generates extensive log data, reflecting its workload and potential failures during operation. Log anomaly detection algorithms use this data to identify deviations in system behavior, especially when errors occur. Workload patterns can vary with time, depending on factors like the time of day or day of the week, affecting log entry volumes. Thus, it’s essential for log anomaly detection to consider temporal information that captures workload variations. This paper introduces a novel log anomaly detection method that incorporates such time information and demonstrates how smaller models enhance anomaly detection precision. We evaluate this method on a high-throughput production workload of a software system, showcasing its superior performance over conventional log anomaly detection methods.
This research was supported by the Student Summer Research Program 2021 of FIT CTU in Prague and the Grant Agency of the Czech Technical University in Prague, grant No. SGS20/209/OHK3/3T/18.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alahakoon, D., Halgamuge, S., Bala, S.: Dynamic self-organizing maps with controlled growth for knowledge discovery. IEEE Trans. Neural Netw. 11(3), 601–614 (2000). https://doi.org/10.1109/72.846732
He, S., Zhu, J., He, P., Lyu, M.R.: Experience report: System log analysis for anomaly detection. In: 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE), pp. 207–218 (2016). https://doi.org/10.1109/ISSRE.2016.21
Le, Q., Mikolov, T.: Distributed representations of sentences and documents. In: Xing, E.P., Jebara, T. (eds.) Proceedings of the 31st International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 32, pp. 1188–1196. PMLR, Beijing (2014). https://proceedings.mlr.press/v32/le14.html
Liang, Y., Zhang, Y., Xiong, H., Sahoo, R.: Failure prediction in IBM BlueGene/L event logs, pp. 583–588 (2007). https://doi.org/10.1109/ICDM.2007.46
Meng, W., et al.: LogAnomaly: unsupervised detection of sequential and quantitative anomalies in unstructured logs (2019)
Mikolov, T., Yih, W.T., Zweig, G.: Linguistic regularities in continuous space word representations. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 746–751. Association for Computational Linguistics, Atlanta (2013). https://aclanthology.org/N13-1090
Vinayakumar, R., Soman, K.P., Poornachandran, P.: Long short-term memory based operation log anomaly detection. In: 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 236–242 (2017). https://doi.org/10.1109/ICACCI.2017.8125846
Wurzenberger, M., Skopik, F., Landauer, M., Greitbauer, P., Fiedler, R., Kstner, W.: Incremental clustering for semi-supervised anomaly detection applied on log data (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fedotov, D., Kuchar, J., Vitvar, T. (2023). Time-Aware Log Anomaly Detection Based on Growing Self-organizing Map. In: Monti, F., Rinderle-Ma, S., Ruiz Cortés, A., Zheng, Z., Mecella, M. (eds) Service-Oriented Computing. ICSOC 2023. Lecture Notes in Computer Science, vol 14419. Springer, Cham. https://doi.org/10.1007/978-3-031-48421-6_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-48421-6_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-48420-9
Online ISBN: 978-3-031-48421-6
eBook Packages: Computer ScienceComputer Science (R0)