More Web Proxy on the site http://driver.im/

research-article

Open access

Informer: irregular traffic detection for containerized microservices RPC in the real world

Authors:

Hao ChenAuthors Info & Claims

SEC '19: Proceedings of the 4th ACM/IEEE Symposium on Edge Computing

Pages 389 - 394

https://doi.org/10.1145/3318216.3363375

Published: 07 November 2019 Publication History

Abstract

Containerized microservices have been widely deployed in industry. Meanwhile, security issues also arise. Many security enhancement mechanisms for containerized microservices require predefined rules and policies. However, it is challenging when it comes to thousands of microservices and a massive amount of real-time unstructured data. Hence, automatic policy generation becomes indispensable. In this paper, we focus on the automatic solution for the security problem: irregular traffic detection for RPCs.

We propose Informer, which is a two-phase machine learning framework to track the traffic of each RPC and report anomalous points automatically. Firstly, we identify RPC chain patterns by density-based clustering techniques and build a graph for each critical pattern. Next, we solve the irregular RPC traffic detection problem as a prediction problem for time-series of attributed graphs by leveraging spatial-temporal graph convolution networks. Since the framework builds multiple models and makes individual predictions for each RPC chain pattern, it can be efficiently updated upon legitimate changes in any of the graphs.

In evaluations, we applied Informer to a dataset containing more than 7 billion lines of raw RPC logs sampled from an large Kubernetes system for two weeks. We provide two case studies of detected real-world threats. As a result, our framework found fine-grained RPC chain patterns and accurately captured the anomalies in a dynamic and complicated microservice production scenario, which demonstrates the effectiveness of Informer.

References

[1]

James Atwood and Don Towsley. 2015. Search-Convolutional Neural Networks. CoRR abs/1511.02136 (2015). arXiv:1511.02136 http://arxiv.org/abs/1511.02136

[2]

Shaosheng Cao, Wei Lu, and Qiongkai Xu. 2016. Deep neural networks for learning graph representations. In Thirtieth AAAI Conference on Artificial Intelligence.

Digital Library

[3]

Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014).

[4]

Docker Inc. [n.d.]. Docker: Enterprise Container Platform. https://www.docker.com/

[5]

Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al. [n.d.]. A density-based algorithm for discovering clusters in large spatial databases with noise.

[6]

Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. 2019. Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 922--929.

Digital Library

[7]

Mikael Henaff, Joan Bruna, and Yann LeCun. 2015. Deep convolutional networks on graph-structured data. arXiv preprint arXiv:1506.05163 (2015).

[8]

Kubernetes contributors. [n.d.]. Kubernetes: Production-Grade Container Orchestration. https://kubernetes.io/

[9]

Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2017. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint arXiv:1707.01926 (2017).

[10]

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. CoRR abs/1409.3215 (2014). arXiv:1409.3215 http://arxiv.org/abs/1409.3215

[11]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).

[12]

MK Vijaymeena and K Kavitha. 2016. A survey on similarity measures in text mining. (2016).

[13]

Daixin Wang, Peng Cui, and Wenwu Zhu. 2016. Structural deep network embedding. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 1225--1234.

Digital Library

[14]

Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2017. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint arXiv:1709.04875 (2017).

[15]

Jiani Zhang, Xingjian Shi, Junyuan Xie, Hao Ma, Irwin King, and Dit-Yan Yeung. 2018. Gaan: Gated attention networks for learning on large and spatiotemporal graphs. arXiv preprint arXiv.1803.07294 (2018).

Cited By

Xu SZhou QHuang HJia XDu HChen YXie Y(2023)Log2Policy: An Approach to Generate Fine-Grained Access Control Rules for Microservices from ScratchProceedings of the 39th Annual Computer Security Applications Conference10.1145/3627106.3627137(229-240)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1145/3627106.3627137
Minna FMassacci F(2023)SoKComputers and Security10.1016/j.cose.2023.103119127:COnline publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1016/j.cose.2023.103119
Hannousse AYahiouche S(2021)Securing microservices and microservice architecturesComputer Science Review10.1016/j.cosrev.2021.10041541:COnline publication date: 1-Aug-2021
https://dl.acm.org/doi/10.1016/j.cosrev.2021.100415
Show More Cited By

Recommendations

Anomaly Detection and Failure Root Cause Analysis in (Micro) Service-Based Cloud Applications: A Survey
The proliferation of services and service interactions within microservices and cloud-native applications, makes it harder to detect failures and to identify their possible root causes, which is, on the other hand crucial to promptly recover and fix ...
BSDG: Anomaly Detection of Microservice Trace Based on Dual Graph Convolutional Neural Network
Service-Oriented Computing
Abstract
Microservice architecture has been widely used by more and more developers in recent years. Accurate anomaly detection is crucial for system maintenance. Trace data can reflect the microservice dependency relationship and response time, which has ...
Network Quality of Service in Docker Containers
CLUSTER '15: Proceedings of the 2015 IEEE International Conference on Cluster Computing

This poster presents an extension to the currently limited Docker's networks. Specifically, to guarantee quality of service (QoS) on the network, our extension allows users to assign priorities to Docker's containers and configures the network to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SEC '19: Proceedings of the 4th ACM/IEEE Symposium on Edge Computing

November 2019

455 pages

ISBN:9781450367332

DOI:10.1145/3318216

General Chairs:
Songqing Chen
George Mason University
,
Ryokichi Onishi
Toyota
,
Program Chairs:
Ganesh Ananthanarayanan
Microsoft Research
,
Qun Li
College of William & Mary

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

In-Cooperation

IEEE-CS\DATC: IEEE Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

SEC '19

Sponsor:

SIGMOBILE

SEC '19: The Fourth ACM/IEEE Symposium on Edge Computing

November 7 - 9, 2019

Virginia, Arlington

Acceptance Rates

SEC '19 Paper Acceptance Rate 20 of 59 submissions, 34%;

Overall Acceptance Rate 40 of 100 submissions, 40%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
709
Total Downloads

Downloads (Last 12 months)154
Downloads (Last 6 weeks)22

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xu SZhou QHuang HJia XDu HChen YXie Y(2023)Log2Policy: An Approach to Generate Fine-Grained Access Control Rules for Microservices from ScratchProceedings of the 39th Annual Computer Security Applications Conference10.1145/3627106.3627137(229-240)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1145/3627106.3627137
Minna FMassacci F(2023)SoKComputers and Security10.1016/j.cose.2023.103119127:COnline publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1016/j.cose.2023.103119
Hannousse AYahiouche S(2021)Securing microservices and microservice architecturesComputer Science Review10.1016/j.cosrev.2021.10041541:COnline publication date: 1-Aug-2021
https://dl.acm.org/doi/10.1016/j.cosrev.2021.100415
Sauwens MHeydari Beni EJannes KLagaisse BJoosen W(2021)ThunQ: A Distributed and Deep Authorization Middleware for Early and Lazy Policy Enforcement in Microservice ApplicationsService-Oriented Computing10.1007/978-3-030-91431-8_13(204-220)Online publication date: 18-Nov-2021
https://doi.org/10.1007/978-3-030-91431-8_13

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents