More Web Proxy on the site http://driver.im/

tutorial

An Outlook to Declarative Languages for Big Steaming Data

Authors:

Riccardo Tommasini,

Marco Balduini,

Emanuele Della ValleAuthors Info & Claims

DEBS '19: Proceedings of the 13th ACM International Conference on Distributed and Event-based Systems

Pages 199 - 202

https://doi.org/10.1145/3328905.3332462

Published: 24 June 2019 Publication History

Abstract

In the Big Data context, data streaming systems have been introduced to tame velocity and enable reactive decision making. However, approaching such systems is still too complex due to the paradigm shift they require, i.e., moving from scalable batch processing to continuous analysis and detection. Initially, modern big stream processing systems (e.g., Flink, Spark, Storm) have been lacking the support of declarative languages to express the streaming-based data processing tasks and have been mainly relying on providing low-level APIs for the end-users to implement their tasks. However, recently, this fact has been changing and most of them started to provide SQL-like languages for their end-users.

In general, declarative Languages are playing a crucial role in fostering the adoption of Stream Processing. This tutorial focuses on introducing various approaches for declarative querying of the state-of-the-art big data streaming frameworks. In addition, we provide guidelines and practical examples on developing and deploying Stream Processing applications using a variety of SQL-like languages, such as Flink-SQL, KSQL and Spark Streaming SQL.

References

[1]

Arvind Arasu et al. The CQL continuous query language: semantic foundations and query execution. The VLDB Journal, 15(2), 2006.

Digital Library

[2]

Arvind Arasu et al. STREAM: the stanford data stream management system. In Data Stream Management - Processing High-Speed Data Streams. 2016.

[3]

Michael Armbrust et al. Structured streaming: A declarative api for real-time applications in apache spark. In SIGMOD, 2018.

Digital Library

[4]

Edmon Begoli et al. Apache calcite: A foundational framework for optimized query processing over heterogeneous data sources. In SIGMOD, 2018.

[5]

Paris Carbone et al. Apache flink: Stream and batch processing in a single engine. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 36(4), 2015.

[6]

Donald D Chamberlin. Early history of sql. IEEE Annals of the History of Computing, 34(4):78--82, 2012.

Digital Library

[7]

Edgar F Codd. Relational completeness of data base sublanguages. Citeseer, 1972.

[8]

Jeffrey Dean and Sanjay Ghemawat. Mapreduce: simplified data processing on large clusters. Communications of the ACM, 51(1), 2008.

Digital Library

[9]

Martin Hirzel, Guillaume Baudart, Angela Bonifati, Emanuele Della Valle, Sherif Sakr, and Akrivi Vlachou. Stream processing languages in the big data era. SIGMOD Record, 47(2), 2018.

[10]

Volker Markl. Breaking the chains: On declarative data analysis and data independence in the big data era. Proceedings of the VLDB Endowment, 7(13), 2014.

Digital Library

[11]

Shanmugavelayutham Muthukrishnan et al. Data streams: Algorithms and applications. Foundations and Trends® in Theoretical Computer Science, 1(2), 2005.

Digital Library

[12]

Sherif Sakr. Big Data 2.0 Processing Systems - A Survey. Springer Briefs in Computer Science. Springer, 2016.

Digital Library

[13]

Matthias J. Sax et al. Streams and tables: Two sides of the same coin. In BIRTE Workshop, 2018.

[14]

Michael Stonebraker and Ugur Çetintemel. Stream processing. In Encyclopedia of Database Systems, Second Edition. 2018.

[15]

Michael Stonebraker, Uğur Çetintemel, and Stan Zdonik. The 8 requirements of real-time stream processing. ACM Sigmod Record, 34(4), 2005.

Cited By

Bonifati ATommasini RBarcelo PSanchez-Pi NMeliou ASudarshan S(2024)An Overview of Continuous Querying in (Modern) Data SystemsCompanion of the 2024 International Conference on Management of Data10.1145/3626246.3654679(605-612)Online publication date: 9-Jun-2024
https://dl.acm.org/doi/10.1145/3626246.3654679
Korkut ESurer E(2023)Visualization in virtual reality: a systematic reviewVirtual Reality10.1007/s10055-023-00753-827:2(1447-1480)Online publication date: 17-Jan-2023
https://doi.org/10.1007/s10055-023-00753-8
van Noort M(2023)Formalizing Stream Reasoning for a Decentralized Semantic WebThe Semantic Web: ESWC 2023 Satellite Events10.1007/978-3-031-43458-7_46(277-287)Online publication date: 21-Oct-2023
https://doi.org/10.1007/978-3-031-43458-7_46
Show More Cited By

Index Terms

An Outlook to Declarative Languages for Big Steaming Data
1. Information systems
  1. Data management systems

Recommendations

Stream Processing Languages in the Big Data Era

This paper is a survey of recent stream processing languages, which are programming languages for writing applications that analyze data streams. Data streams, or continuous data flows, have been around for decades. But with the advent of the big-data ...
Taming velocity and variety simultaneously in big data with stream reasoning: tutorial
DEBS '16: Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems

Many "big data" applications must tame velocity (processing data in-motion) and variety (processing many different types of data) simultaneously.

The research on knowledge representation and reasoning has focused on the variety of data, devising data ...
Characterization of Big Data Stream Processing Pipeline: A Case Study using Flink and Kafka
BDCAT '17: Proceedings of the Fourth IEEE/ACM International Conference on Big Data Computing, Applications and Technologies

In recent years there has been a surge in applications focusing on streaming data to generate insights in real-time. Both academia, as well as industry, have tried to address this use case by developing a variety of Stream Processing Engines (SPEs) with ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

DEBS '19: Proceedings of the 13th ACM International Conference on Distributed and Event-based Systems

June 2019

291 pages

ISBN:9781450367943

DOI:10.1145/3328905

Copyright © 2019 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 June 2019

Check for updates

Author Tags

Qualifiers

Tutorial
Research
Refereed limited

Conference

DEBS '19

Sponsor:

DEBS '19: The 13th ACM International Conference on Distributed and Event-based Systems

June 24 - 28, 2019

Darmstadt, Germany

Acceptance Rates

DEBS '19 Paper Acceptance Rate 13 of 47 submissions, 28%;

Overall Acceptance Rate 145 of 583 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
185
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)1

Reflects downloads up to 17 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bonifati ATommasini RBarcelo PSanchez-Pi NMeliou ASudarshan S(2024)An Overview of Continuous Querying in (Modern) Data SystemsCompanion of the 2024 International Conference on Management of Data10.1145/3626246.3654679(605-612)Online publication date: 9-Jun-2024
https://dl.acm.org/doi/10.1145/3626246.3654679
Korkut ESurer E(2023)Visualization in virtual reality: a systematic reviewVirtual Reality10.1007/s10055-023-00753-827:2(1447-1480)Online publication date: 17-Jan-2023
https://doi.org/10.1007/s10055-023-00753-8
van Noort M(2023)Formalizing Stream Reasoning for a Decentralized Semantic WebThe Semantic Web: ESWC 2023 Satellite Events10.1007/978-3-031-43458-7_46(277-287)Online publication date: 21-Oct-2023
https://doi.org/10.1007/978-3-031-43458-7_46
Giallorenzo SMontesi FSafina LZingaro S(2022)Ephemeral data handling in microservices with TqueryPeerJ Computer Science10.7717/peerj-cs.10378(e1037)Online publication date: 22-Jul-2022
https://doi.org/10.7717/peerj-cs.1037
Tommasini RBonte PSpiga FDella Valle ETommasini RBonte PSpiga FDella Valle E(2022)PreliminariesStreaming Linked Data10.1007/978-3-031-15371-6_2(17-40)Online publication date: 22-Aug-2022
https://doi.org/10.1007/978-3-031-15371-6_2
Costa RMoreira JPintor Pdos Santos VLifschitz S(2021)A Survey on Data-driven Performance Tuning for Big Data Analytics PlatformsBig Data Research10.1016/j.bdr.2021.10020625:COnline publication date: 29-Dec-2021
https://dl.acm.org/doi/10.1016/j.bdr.2021.100206

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents