[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3514221.3526136acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
research-article

DLACEP: A Deep-Learning Based Framework for Approximate Complex Event Processing

Published: 11 June 2022 Publication History

Abstract

Complex event processing (CEP) is employed to detect user-specified patterns of events in data streams. CEP mechanisms operate by maintaining all sets of events that can potentially be composed into a pattern match. This approach can be wasteful when many of the sets do not participate in an actual match and are therefore discarded.
We present DLACEP, a novel framework that fuses deep learning with CEP to efficiently extract complex pattern matches from streams. To the best of our knowledge, this is the first time deep learning is employed to detect events constituting a pattern match in the realm of CEP. To assess our approach, we performed extensive empirical testing on various scenarios with both real-world and synthetic data. We showcase examples in which our method achieves an increase in throughput of up to three orders of magnitude compared to solely employing CEP, while only suffering a minor loss in the number of detected matches.

References

[1]
http://www.eoddata.com/.
[2]
Overview of the Event Processing Language (EPL), Oct 2021.
[3]
J. Agrawal, Y. Diao, D. Gyllstrom, and N. Immerman. Efficient pattern matching over event streams. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, SIGMOD '08, page 147--160, New York, NY, USA, 2008. Association for Computing Machinery.
[4]
Z. Allen-Zhu, Y. Li, and Y. Liang. Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers. arXiv, Nov 2018.
[5]
A. Arasu, B. Babcock, S. Babu, J. Cieslewicz, M. Datar, K. Ito, R. Motwani, U. Srivastava, and J. Widom. STREAM: The Stanford Data Stream Management System. In Data Stream Management, pages 317--336. Springer, Berlin, Germany, Jul 2016.
[6]
R. S. Barga, J. Goldstein, M. Ali, and M. Hong. Consistent streaming through time: A vision for event stream processing. In In CIDR, pages 363--374, 2007.
[7]
L. Baumgärtner, C. Strack, B. Hoßbach, M. Seidemann, B. Seeger, and B. Freisleben. Complex event processing for reactive security monitoring in virtualized computer systems. DEBS '15, page 22--33, New York, NY, USA, 2015. Association for Computing Machinery.
[8]
Y. Bengio. Learning Deep Architectures for AI. Foundations and Trends in Machine Learning, 2(1):1--55, Jan 2009.
[9]
M. Blount, M. Ebling, J. Eklund, A. James, C. Mcgregor, N. Percival, K. Smith, and D. Sow. Real-Time Analysis for Intensive Care: Development and Deployment of the Artemis Analytic System. IEEE engineering in medicine and biology magazine : the quarterly magazine of the Engineering in Medicine & Biology Society, 29(2):110--8, May 2010.
[10]
L. Cai, S. Zhou, X. Yan, and R. Yuan. A Stacked BiLSTM Neural Network Based on Coattention Mechanism for Question Answering. Comput. Intell. Neurosci., 2019, Aug 2019.
[11]
K. Chapnik, I. Kolchinsky, and A. Schuster. Darling: Data-aware load shedding in complex event processing systems. Sydney, Australia, September 2022. 48th International Conference on Very Large Data Bases (PVLDB).
[12]
T. Chen, R. Xu, Y. He, and X. Wang. Improving sentiment analysis via sentence type classification using bilstm-crf and cnn. Expert Syst. Appl., 72(C):221--230, Apr. 2017.
[13]
F. Chollet. keras. https://github.com/fchollet/keras, 2015.
[14]
M. Christ, J. Krumeich, and A. W. Kempa-Liehr. Integrating predictive analytics into complex event processing by using conditional density estimations. In 2016 IEEE 20th International Enterprise Distributed Object Computing Workshop (EDOCW), pages 1--8, 2016.
[15]
G. Cugola and A. Margara. Processing flows of information: from data stream to complex event processing. ACM COMPUTING SURVEYS, 2011.
[16]
G. Cugola and A. Margara. Complex event processing with t-rex. J. Syst. Softw., 85(8):1709--1728, Aug. 2012.
[17]
M. Dayarathna and S. Perera. Recent advancements in event processing. ACM Comput. Surv., 51(2), Feb. 2018.
[18]
A. Demers, J. Gehrke, M. Hong, M. Riedewald, andW. White. Towards Expressive Publish/Subscribe Systems. In Advances in Database Technology - EDBT 2006, pages 627--644. Springer, Berlin, Germany, Mar 2006.
[19]
L. Ding, S. Chen, E. A. Rundensteiner, J. Tatemura, W. Hsiung, and K. S. Candan. Runtime semantic query optimization for event stream processing. In 2008 IEEE 24th International Conference on Data Engineering, pages 676--685, 2008.
[20]
O. Etzion and P. Niblett. Event Processing in Action. Manning Publications Co., USA, 1st edition, 2010.
[21]
I. Flouris, N. Giatrakos, A. Deligiannakis, M. Garofalakis, M. Kamp, and M. Mock. Issues in complex event processing: status and prospects in the big data era. Journal of Systems and Software, 127:217--236, May 2017.
[22]
L. J. Fülöp, A. Beszédes, G. Tóth, H. Demeter, L. Vidács, and L. Farkas. Predictive complex event processing: A conceptual framework for combining complex event processing and predictive analytics. In Proceedings of the Fifth Balkan Conference in Informatics, BCI '12, page 26--31, New York, NY, USA, 2012. Association for Computing Machinery.
[23]
X. Gao, J. Zhang, and Z. Wei. Deep learning for sequence pattern recognition. In 2018 IEEE 15th International Conference on Networking, Sensing and Control (ICNSC), pages 1--6, 2018.
[24]
L. Gong, X. Zhang, T. Chen, and L. Zhang. Recognition of Disease Genetic Information from Unstructured Text Data Based on BiLSTM-CRF for Molecular Mechanisms. Secur. Commun. Netw., 2021, Feb 2021.
[25]
C. Goutte and E. Gaussier. A Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation. In Advances in Information Retrieval, pages 345--359. Springer, Berlin, Germany, Mar 2005.
[26]
A. Graves, A. Mohamed, and G. Hinton. Speech recognition with deep recurrent neural networks. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 6645--6649, 2013.
[27]
D. Gyllstrom, E. Wu, H.-J. Chae, Y. Diao, P. Stahlberg, and G. Anderson. Sase: Complex event processing over streams. ArXiv, abs/cs/0612128, 2006.
[28]
S. Hallé and S. Varvaressos. A formalization of complex event stream processing. In 2014 IEEE 18th International Enterprise Distributed Object Computing Conference, pages 2--11, 2014.
[29]
Y. He, S. Barman, and J. F. Naughton. On load shedding in complex event processing, 2013.
[30]
M. Hirzel, R. Soulé, S. Schneider, B. Gedik, and R. Grimm. A catalog of stream processing optimizations. ACM Comput. Surv., 46(4), Mar. 2014.
[31]
S. Hochreiter and J. Schmidhuber. Long short-term memory. 9(8):1735--1780, Nov. 1997.
[32]
S. C. H. Hoi, D. Sahoo, J. Lu, and P. Zhao. Online learning: A comprehensive survey. Neurocomputing, 459:249--289, 2021.
[33]
M. Z. Hossain, F. Sohel, M. F. Shiratuddin, and H. Laga. A comprehensive survey of deep learning for image captioning. ACM Comput. Surv., 51(6), Feb. 2019.
[34]
Z. Huang, W. Xu, and K. Yu. Bidirectional lstm-crf models for sequence tagging, 2015.
[35]
Y. Jiang, B. Neyshabur, H. Mobahi, D. Krishnan, and S. Bengio. Fantastic generalization measures and where to find them. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26--30, 2020. OpenReview.net, 2020.
[36]
I. Kolchinsky. Opencep. https://github.com/ilya-kolchinsky/OpenCEP, 2020.
[37]
I. Kolchinsky and A. Schuster. Efficient adaptive detection of complex event patterns. Proc. VLDB Endow., 11(11):1346--1359, July 2018.
[38]
I. Kolchinsky and A. Schuster. Join query optimization techniques for complex event processing applications. Proc. VLDB Endow., 11:1332--1345, 2018.
[39]
I. Kolchinsky and A. Schuster. Join query optimization techniques for complex event processing applications. 11(11), 2018.
[40]
I. Kolchinsky and A. Schuster. Real-time multi-pattern detection over event streams. In Proceedings of the 2019 International Conference on Management of Data, SIGMOD '19, page 589--606, New York, NY, USA, 2019. Association for Computing Machinery.
[41]
I. Kolchinsky, I. Sharfman, and A. Schuster. Lazy evaluation methods for detecting complex events. In Proceedings of the 9th ACM International Conference on Distributed Event-Based Systems, DEBS '15, page 34--45, New York, NY, USA, 2015. Association for Computing Machinery.
[42]
E. Kosman, I. Kolchinsky, and A. Schuster. Mining logical arithmetic expressions from proper representations. Alexandria, Virginia, USA, April 2022. SIAM International Conference on Data Mining (SDM).
[43]
J. D. Lafferty, A. McCallum, and F. C. N. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning, ICML '01, page 282--289, San Francisco, CA, USA, 2001. Morgan Kaufmann Publishers Inc.
[44]
G. Lample, M. Ballesteros, S. Subramanian, K. Kawakami, and C. Dyer. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 260--270, San Diego, California, June 2016. Association for Computational Linguistics.
[45]
C. Lea, R. Vidal, A. Reiter, and G. D. Hager. Temporal Convolutional Networks: A Unified Approach to Action Segmentation. arXiv, Aug 2016.
[46]
J. Li, K. Tufte, V. Shkapenyuk, V. Papadimos, T. Johnson, and D. Maier. Out-of-order processing: A new architecture for high-performance stream systems. Proc. VLDB Endow., 1(1):274--288, Aug. 2008.
[47]
Q. Li, H. Peng, J. Li, C. Xia, R. Yang, L. Sun, P. Yu, and L. He. A Text Classification Survey: From Shallow to Deep Learning. ResearchGate, Aug 2020.
[48]
Z. Li and T. Ge. History is a mirror to the future: Best-effort approximate complex event matching with insufficient resources. Proc. VLDB Endow., 10(4):397--408, Nov. 2016.
[49]
M. Liu, M. Li, D. Golovnya, E. A. Rundensteiner, and K. Claypool. Sequence pattern query processing over out-of-order event streams. In 2009 IEEE 25th International Conference on Data Engineering, pages 784--795, 2009.
[50]
Z. Liu, M. Yang, X.Wang, Q. Chen, B. Tang, Z.Wang, and H. Xu. Entity recognition from clinical texts via recurrent neural network. BMC Med. Inf. Decis. Making, 17(S2), Jul 2017.
[51]
X. Ma and E. Hovy. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1064--1074, Berlin, Germany, Aug. 2016. Association for Computational Linguistics.
[52]
A. Margara, G. Cugola, and G. Tamburrelli. Towards automated rule learning for complex event processing. Technical report, 2013.
[53]
N. Mehdiyev, J. Krumeich, D. Enke, D. Werth, and P. Loos. Determination of Rule Patterns in Complex Event Processing Using Machine Learning Techniques. Procedia Comput. Sci., 61:395--401, Jan 2015.
[54]
Y. Mei and S. Madden. Zstream: A cost-based query processor for adaptively detecting composite events. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data, SIGMOD '09, page 193--206, New York, NY, USA, 2009. Association for Computing Machinery.
[55]
S. Minaee, N. Kalchbrenner, E. Cambria, N. Nikzad, M. Chenaghlu, and J. Gao. Deep learning based text classification: A comprehensive review, 2021.
[56]
openspecs sql. [MS-CEPM]: Glossary, Jan 2020.
[57]
A. Ozbayoglu, M. U. Gudelek, and O. B. Sezer. Deep learning for financial applications : A survey. Appl. Soft Comput., 93:106384, 2020.
[58]
R. Panchendrarajan and A. Amaresan. Bidirectional LSTM-CRF for Named Entity Recognition. ResearchGate, May 2019.
[59]
R. Pascanu, T. Mikolov, and Y. Bengio. On the difficulty of training recurrent neural networks. In Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28, ICML'13, page III--1310--III--1318. JMLR.org, 2013.
[60]
A. Pogiatzis and G. Samakovitis. Using BiLSTM Networks for Context-Aware Deep Sensitivity Labelling on Conversational Data. Appl. Sci., 10(24):8924, Dec 2020.
[61]
O. Poppe, C. Lei, S. Ahmed, and E. A. Rundensteiner. Complete event trend detection in high-rate event streams. In Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD '17, page 109--124, New York, NY, USA, 2017. Association for Computing Machinery.
[62]
A. Qayyum, J. Qadir, M. Bilal, and A. Al-Fuqaha. Secure and robust machine learning for healthcare: A survey. IEEE Reviews in Biomedical Engineering, 14:156--180, 2021.
[63]
Y. Qian, T. Tan, and D. Yu. Neural network based multi-factor aware joint training for robust speech recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 24(12):2231--2240, 2016.
[64]
E. Rabinovich, O. Etzion, and A. Gal. Pattern rewriting framework for event processing optimization. In Proceedings of the 5th ACM International Conference on Distributed Event-Based System, DEBS '11, page 101--112, New York, NY, USA, 2011. Association for Computing Machinery.
[65]
T. Rabl, J. Traub, A. Katsifodimos, and V. Markl. Apache flink in current research. it - Information Technology, 58, 01 2016.
[66]
G. Ramena, D. Nagaraju, S. Moharana, D. Prasanna Mohanty, and N. Purre. An efficient architecture for predicting the case of characters using sequence models. In 2020 IEEE 14th International Conference on Semantic Computing (ICSC), pages 174--177, 2020.
[67]
N. Reimers and I. Gurevych. Optimal hyperparameters for deep lstm-networks for sequence labeling tasks. ArXiv, abs/1707.06799, 2017.
[68]
D. Sahoo, Q. Pham, J. Lu, and S. C. H. Hoi. Online deep learning: Learning deep neural networks on the fly. In IJCAI, 2018.
[69]
H. Sak, A. W. Senior, and F. Beaufays. Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition. CoRR, abs/1402.1128, 2014.
[70]
G. Shapira, I. Kolchinsky, and A. Schuster. Semi-supervised frequent pattern mining for cep. Manuscript paper, 2022.
[71]
G. Sidi, I. Kolchinsky, and A. Schuster. Delete: Using deep learning to minimize latency in cep systems. Manuscript paper, 2022.
[72]
M. U. Simsek, F. Y. Okay, and S. Ozdemir. A deep learning-based CEP rule extraction framework for IoT data. J. Supercomput., pages 1--30, Jan 2021.
[73]
H. Sivan, M. Gabel, and A. Schuster. Incremental sensitivity analysis for kernelized models. ECML-PKDD, 2020.
[74]
H. Sivan, M. Gabel, and A. Schuster. Automon: Automatic distributed monitoring for arbitrary multivariate functions. Philadelphia, PA, USA, 2022. SIGMOD.
[75]
A. Slo, S. Bhowmik, A. Flaig, and K. Rothermel. pspice: Partial match shedding for complex event processing. 2019 IEEE International Conference on Big Data (Big Data), pages 372--382, 2019.
[76]
A. Slo, S. Bhowmik, and K. Rothermel. State-aware load shedding from input event streams in complex event processing. IEEE Transactions on Big Data, pages 1--1, 2020.
[77]
U. Srivastava and J. Widom. Flexible time management in data stream systems. In Proceedings of the Twenty-Third ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS '04, page 263--274, New York, NY, USA, 2004. Association for Computing Machinery.
[78]
R. Sun. Optimization for deep learning: theory and algorithms. ArXiv, abs/1912.08957, 2019.
[79]
C. Sutton and A. McCallum. An introduction to conditional random fields. Found. Trends Mach. Learn., 4(4):267--373, Apr. 2012.
[80]
C. Tan, F. Sun, T. Kong,W. Zhang, C. Yang, and C. Liu. A survey on deep transfer learning. ArXiv, abs/1808.01974, 2018.
[81]
K. Tawsif, J. Hossen, J. E. Raja, M. Z. H. Jesmeen, and E. M. H. Arif. A review on complex event processing systems for big data. In 2018 Fourth International Conference on Information Retrieval and Knowledge Management (CAMP), pages 1--6, 2018.
[82]
E. Tutubalina and S. Nikolenko. Combination of Deep Recurrent Neural Networks and Conditional Random Fields for Extracting Adverse Drug Reactions from User Reviews. J. Healthcare Eng., 2017, Sep 2017.
[83]
G. Uziel. Deep online learning with stochastic constraints. CoRR, abs/1905.10817, 2019.
[84]
C.Wang, H. Yang, and C. Meinel. Image captioning with deep bidirectional lstms and multi-task learning. ACM Trans. Multimedia Comput. Commun. Appl., 14(2s), Apr. 2018.
[85]
Y. Wang, H. Gao, and G. Chen. Predictive complex event processing based on evolving Bayesian networks. Pattern Recognit. Lett., 105:207--216, Apr 2018.
[86]
K. R.Weiss, T. Khoshgoftaar, and D.Wang. A survey of transfer learning. Journal of Big Data, 3:1--40, 2016.
[87]
E. Wu, Y. Diao, and S. Rizvi. High-performance complex event processing over streams. In Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data, SIGMOD '06, page 407--418, New York, NY, USA, 2006. Association for Computing Machinery.
[88]
Y. Wu, E. Dobriban, and S. B. Davidson. Deltagrad: Rapid retraining of machine learning models. In ICML, 2020.
[89]
T. Xing, M. Roig Vilamala, L. Garcia, F. Cerutti, L. Kaplan, A. Preece, and M. Srivastava. Deepcep: Deep complex event processing using distributed multimodal information. In 2019 IEEE International Conference on Smart Computing (SMARTCOMP), pages 87--92, 2019.
[90]
M. Yankovitch, I. Kolchinsky, and A. Schuster. Hypersonic: A hybrid parallelization approach for scalable complex event processing. Philadelphia, PA, USA, 2022. SIGMOD 2022.
[91]
K. Yun, A. Huyen, and T. Lu. Deep neural networks for pattern recognition, 2018.
[92]
Z. Zainuddin and Ong. Function approximation using artificial neural networks. WSEAS Transactions on Mathematics, 7(6), Jun 2008.
[93]
H. Zhang, Y. Diao, and N. Immerman. On complexity and optimization of expensive queries in complex event processing. In Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, SIGMOD '14, page 217--228, New York, NY, USA, 2014. Association for Computing Machinery.
[94]
H. Zhang, Y. Diao, and N. Immerman. On complexity and optimization of expensive queries in complex event processing. In Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, SIGMOD '14, page 217--228, New York, NY, USA, 2014. Association for Computing Machinery.
[95]
B. Zhao. Complex event processing under constrained resources by state-based load shedding. 2018 IEEE 34th International Conference on Data Engineering (ICDE), pages 1699--1703, 2018.
[96]
Q. Zhou, Y. Simmhan, and V. Prasanna. Incorporating semantic knowledge into dynamic data processing for smart power grids. In Proceedings of the 11th International Conference on The Semantic Web - Volume Part II, ISWC'12, page 257--273, Berlin, Heidelberg, 2012. Springer-Verlag.

Cited By

View all
  • (2024)An Efficient Algorithm for Continuous Complex Event Matching Using Bit-Parallelism2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00037(396-408)Online publication date: 13-May-2024
  • (2023)Ontology Development for Detecting Complex Events in Stream Processing: Use Case of Air Quality MonitoringComputers10.3390/computers1211023812:11(238)Online publication date: 16-Nov-2023

Index Terms

  1. DLACEP: A Deep-Learning Based Framework for Approximate Complex Event Processing

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGMOD '22: Proceedings of the 2022 International Conference on Management of Data
    June 2022
    2597 pages
    ISBN:9781450392495
    DOI:10.1145/3514221
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 11 June 2022

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. complex event processing
    2. deep learning
    3. neural networks

    Qualifiers

    • Research-article

    Conference

    SIGMOD/PODS '22
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 785 of 4,003 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)57
    • Downloads (Last 6 weeks)9
    Reflects downloads up to 25 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)An Efficient Algorithm for Continuous Complex Event Matching Using Bit-Parallelism2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00037(396-408)Online publication date: 13-May-2024
    • (2023)Ontology Development for Detecting Complex Events in Stream Processing: Use Case of Air Quality MonitoringComputers10.3390/computers1211023812:11(238)Online publication date: 16-Nov-2023

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media