More Web Proxy on the site http://driver.im/

survey

Resource Management and Scheduling in Distributed Stream Processing Systems: A Taxonomy, Review, and Future Directions

Authors:

Rajkumar BuyyaAuthors Info & Claims

ACM Computing Surveys (CSUR), Volume 53, Issue 3

Article No.: 50, Pages 1 - 41

https://doi.org/10.1145/3355399

Published: 28 May 2020 Publication History

Abstract

Stream processing is an emerging paradigm to handle data streams upon arrival, powering latency-critical application such as fraud detection, algorithmic trading, and health surveillance. Though there are a variety of Distributed Stream Processing Systems (DSPSs) that facilitate the development of streaming applications, resource management and task scheduling is not automatically handled by the DSPS middleware and requires a laborious process to tune toward specific deployment targets. As the advent of cloud computing has supported renting resources on-demand, it is of great interest to review the research progress of hosting streaming systems in clouds under certain Service Level Agreements (SLA) and cost constraints. In this article, we introduce the hierarchical structure of streaming systems, define the scope of the resource management problem, and present a comprehensive taxonomy in this context covering critical research topics such as resource provisioning, operator parallelisation, and task scheduling. The literature is then reviewed following the taxonomy structure, facilitating a deeper understanding of the research landscape through classification and comparison of existing works. Finally, we discuss the open issues and future research directions toward realising an automatic, SLA-aware resource management framework.

Supplementary Material

a50-liu-suppl.pdf (liu.zip)

Supplemental movie, appendix, image and software files for, Resource Management and Scheduling in Distributed Stream Processing Systems: A Taxonomy, Review, and Future Directions

Download
213.22 KB

References

[1]

Mohammad Sadoghi, Martin Labrecque, Harsh Singh, Warren Shum, and Hans-Arno Jacobsen. 2010. Efficient event processing through reconfigurable hardware for algorithmic trading. Proc. VLDB Endow. 3, 1--x2 (2010), 1525--1528. https://dl.acm.org/doi/10.14778/1920841.1921029

Digital Library

[2]

Leonardo Neumeyer, Bruce Robbins, Anish Nair, and Anand Kesari. 2010. S4: Distributed stream computing platform. In Proceedings of the IEEE International Conference on Data Mining Workshops. IEEE, 170--177. https://www.cs.cmu.edu/∼pavlo/courses/fall2013/static/papers/S4PaperV2.pdf.

Digital Library

[3]

Martin Hirzel, Robert Soule, Scott Schneider, Bugra Gedik, and Robert Grimm. 2014. A catalog of stream processing optimizations. Comput. Surveys 46, 4 (2014), 1--34. https://dl.acm.org/doi/10.1145/2528412

Digital Library

[4]

Leonardo Aniello, Roberto Baldoni, and Leonardo Querzoni. 2013. Adaptive online scheduling in Storm. In Proceedings of the 7th ACM International Conference on Distributed Event-based Systems. ACM Press, 207--218.

Digital Library

[5]

Joshua Auerbach, David F. Bacon, Perry Cheng, and Rodric Rabbah. 2010. Lime: A Java-compatible and synthesizable language for heterogeneous architectures. ACM SIGPLAN Not. 45, 10 (2010), 89--108.

Digital Library

[6]

Nathan Backman, Rodrigo Fonseca, and Ugur Çetintemel. 2012. Managing parallelism for stream processing in the cloud. In Proceedings of the 1st International Workshop on Hot Topics in Cloud Data Processing (HotCDP’12). ACM Press, 1--5.

Digital Library

[7]

Cagri Balkesen, Nesime Tatbul, and M. Tamer Özsu. 2013. Adaptive input admission and management for parallel stream processing. In Proceedings of the 7th ACM International Conference on Distributed Event-based Systems. ACM Press, 15--24.

Digital Library

[8]

Anne Benoit, Henri Casanova, Veronika Rehn-Sonigo, and Yves Robert. 2009. Resource allocation strategies for constructive in-network stream processing. In Proceedings of the IEEE International Symposium on Parallel and Distributed Processing. IEEE, 1--8.

Digital Library

[9]

Anne Benoit, Henri Casanova, Veronika Rehn-Sonigo, and Yves Robert. 2011. Resource allocation for multiple concurrent in-network stream-processing applications. Parallel Comput. 37, 8 (2011), 331--348.

Digital Library

[10]

Muhammad Bilal and Marco Canini. 2017. Towards automatic parameter tuning of stream processing systems. In Proceedings of the ACM Symposium on Cloud Computing. ACM Press, 189--200.

Digital Library

[11]

Ioannis Boutsis and Vana Kalogeraki. 2012. RADAR: Adaptive rate allocation in distributed stream processing systems under bursty workloads. In Proceedings of the 31st IEEE Symposium on Reliable Distributed Systems. IEEE, 285--290.

Digital Library

[12]

Antonio Brogi, Gabriele Mencagli, Davide Neri, Jacopo Soldani, and Massimo Torquati. 2018. Container-based support for autonomic data stream processing through the fog. In Proceedings of the 23rd European Conference on Parallel Processing, Vol. 8374. Springer, 17--28.

[13]

Thilina Buddhika, Ryan Stern, Kira Lindburg, Kathleen Ericson, and Shrideep Pallickara. 2017. Online scheduling and interference alleviation for low-latency, high-throughput processing of data streams. IEEE Trans. Parallel Distrib. Syst. 28, 12 (2017), 3553--3569.

Digital Library

[14]

Michael Cammert, Christoph Heinz, Jurgen Kramer, Bernhard Seeger, Sonny Vaupel, and Udo Wolske. 2007. Flexible multi-threaded scheduling for continuous queries over data streams. In Proceedings of the 23rd IEEE International Conference on Data Engineering Workshop. IEEE, 624--633.

Digital Library

[15]

Michael Cammert, J. Kramer, B. Seeger, and S. Vaupel. 2008. A cost-based approach to adaptive resource management in data stream systems. IEEE Trans. Knowl. Data Eng. 20, 2 (2008), 230--245.

Digital Library

[16]

Paris Carbone, Stephan Ewen, Seif Haridi, Asterios Katsifodimos, Volker Markl, and Kostas Tzoumas. 2015. Apache Flink: Unified stream and batch processing in a single engine. Bull. IEEE Comput. Soc. Tech. Committee Data Eng. 36, 1 (2015), 28--38.

[17]

Valeria Cardellini, Vincenzo Grassi, Francesco Lo Presti, and Matteo Nardelli. 2015. Distributed QoS-aware scheduling in Storm. In Proceedings of the 9th ACM International Conference on Distributed Event-based Systems (DEBS’15). ACM Press, 344--347.

Digital Library

[18]

Valeria Cardellini, Vincenzo Grassi, Francesco Lo Presti, and Matteo Nardelli. 2016. Optimal operator placement for distributed stream processing applications. In Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems. ACM Press, 69--80.

Digital Library

[19]

Valeria Cardellini, Vincenzo Grassi, Francesco Lo Presti, and Matteo Nardelli. 2017. Optimal operator replication and placement for distributed stream processing systems. ACM SIGMETRICS Perform. Eval. Rev. 44, 4 (2017), 11--22.

Digital Library

[20]

Valeria Cardellini, Vincenzo Grassi, Francesco Lo Presti, and Matteo Nardelli. 2015. On QoS-aware scheduling of data stream applications over fog computing infrastructures. In Proceedings of the IEEE Symposium on Computers and Communication. IEEE, 271--276.

Digital Library

[21]

Valeria Cardellini, Francesco Lo Presti, Matteo Nardelli, and Gabriele Russo Russo. 2017. Optimal operator deployment and replication for elastic distributed data stream processing. Concurr. Comput.: Pract. Exper. 43, 34 (2017), 4334--4353.

[22]

Valeria Cardellini, Matteo Nardelli, and Dario Luzi. 2016. Elastic stateful stream processing in Storm. In Proceedings of the International Conference on High-performance Computing 8 Simulation. IEEE, 583--590.

[23]

Raul Castro Fernandez, Matteo Migliavacca, Evangelia Kalyvianaki, and Peter Pietzuch. 2013. Integrating scale out and fault tolerance in stream processing using operator state management. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD’13). ACM Press, 725--736.

[24]

Javier Cervino, Evangelia Kalyvianaki, Joaquin Salvachua, and Peter Pietzuch. 2012. Adaptive provisioning of stream processing systems in the cloud. In Proceedings of the 28th IEEE International Conference on Data Engineering Workshops. IEEE, 295--301.

Digital Library

[25]

Badrish Chandramouli, Jonathan Goldstein, Roger Barga, Mirek Riedewald, and Ivo Santos. 2011. Accurate latency estimation in a distributed event processing system. In Proceedings of the 27th IEEE International Conference on Data Engineering (ICDE’11). IEEE, 255--266.

Digital Library

[26]

Shilpa Chaturvedi, Sahil Tyagi, and Yogesh Simmhan. 2017. Collaborative reuse of streaming dataflows in IoT applications. In Proceedings of the 13th IEEE International Conference on e-Science. IEEE, 403--412.

[27]

Andreas Chatzistergiou and Stratis D. Viglas. 2014. Fast heuristics for near-optimal task allocation in data stream processing over clusters. In Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM’14). ACM Press, 1579--1588.

[28]

Wuhui Chen, Incheon Paik, and Zhenni Li. 2016. Cost-aware streaming workflow allocation on geo-distributed data centers. IEEE Trans. Comput. 1 (2016), 1--14.

[29]

Zhenhua Chen, Jielong Xu, Jian Tang, Kevin Kwiat, Charles Kamhoua, and Chonggang Wang. 2016. GPU-accelerated high-throughput online stream data processing. IEEE Trans. Big Data 3, 99 (2016), 1--12.

[30]

Sanket Chintapalli, Derek Dagit, Bobby Evans, Reza Farivar, Thomas Graves, Mark Holderbaugh, Zhuo Liu, Kyle Nusbaum, Kishorkumar Patil, Boyang J. Peng, and Paul Poulosky. 2016. Benchmarking streaming computation engines: Storm, Flink and Spark streaming. In Proceedings of the IEEE International Parallel and Distributed Processing Symposium Workshops. IEEE, 1789--1792.

[31]

Gianpaolo Cugola and Alessandro Margara. 2012. Processing flows of information: From data stream to complex event processing. Comput. Surveys 44, 3 (2012), 1--62.

Digital Library

[32]

Frank Dabek, Russ Cox, Frans Kaashoek, and Robert Morris. 2004. Vivaldi: A decentralized network coordinate system. ACM SIGCOMM Comput. Commun. Rev. 34, 4 (2004), 15--26.

Digital Library

[33]

Wenyun Dai, Longfei Qiu, Ana Wu, and Meikang Qiu. 2016. Cloud infrastructure resource allocation for big data applications. IEEE Trans. Big Data 3, 99 (2016), 1--11.

[34]

Miyuru Dayarathna and Srinath Perera. 2018. Recent advancements in event processing. Comput. Surveys 51, 2 (2018), 1--36.

Digital Library

[35]

Tiziano De Matteis and Gabriele Mencagli. 2016. Keep calm and react with foresight: Strategies for low-latency and energy-efficient elastic data stream processing. In Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. ACM Press, 1--12.

Digital Library

[36]

Tiziano De Matteis and Gabriele Mencagli. 2017. Elastic scaling for distributed latency-sensitive data stream operators. In Proceedings of the 25th Euromicro International Conference on Parallel, Distributed and Network-based Processing. IEEE, 61--68.

[37]

Tiziano De Matteis and Gabriele Mencagli. 2017. Proactive elasticity and energy awareness in data stream processing. J. Syst. Softw. 127, C (2017), 302--319.

[38]

Marcos Dias de Assunção, Alexandre da Silva Veith, and Rajkumar Buyya. 2018. Distributed data stream processing and edge computing: A survey on resource elasticity and future directions. J. Netw. Comput. Appl. 103, 1 (2018), 1--17.

Digital Library

[39]

Michael Duller, Jan S. Rellermeyer, Gustavo Alonso, and Nesime Tatbul. 2011. Virtualizing stream processing. In Proceedings of the 12th International on Middleware Conference. Springer, 269--288.

Digital Library

[40]

Raphael Eidenbenz and Thomas Locher. 2016. Task allocation for distributed stream processing. In Proceedings of the 35th Annual IEEE International Conference on Computer Communications. IEEE, 1--9.

Digital Library

[41]

Leila Eskandari, Zhiyi Huang, and David Eyers. 2016. P-scheduler: Adaptive hierarchical scheduling in Apache Storm. In Proceedings of the Australasian Computer Science Week Multiconference (ACSW’16). ACM Press, 1--10.

Digital Library

[42]

Havard Espeland, Paul B. Beskow, Hakon K. Stensland, Preben N. Olsen, Stale Kristoffersen, Carsten Griwodz, and Pal Halvorsen. 2011. P2G: A framework for distributed real-time processing of multimedia data. In Proceedings of the 40th International Conference on Parallel Processing Workshops. IEEE, 416--426.

Digital Library

[43]

Lorenz Fischer and Abraham Bernstein. 2015. Workload scheduling in distributed stream processors using graph partitioning. In Proceedings of the IEEE International Conference on Big Data. IEEE, 124--133.

Digital Library

[44]

Lorenz Fischer, Thomas Scharrenbach, and Abraham Bernstein. 2013. Scalable linked data stream processing via network-aware workload scheduling. In Proceedings of the 9th International Conference on Scalable Semantic Web Knowledge Base Systems. Springer, 81--96.

[45]

Avrilia Floratou, Ashvin Agrawal, Bill Graham, Sriram Rao, and Karthik Ramasamy. 2017. Dhalion: Self-regulating stream processing in Heron. Proceedings of the VLDB Endowment 10, 12 (2017), 1825--1836.

Digital Library

[46]

Tom Z. J. Fu, Jianbing Ding, Richard T. B. Ma, Marianne Winslett, Yin Yang, and Zhenjie Zhang. 2015. DRS: Dynamic resource scheduling for real-time analytics over fast streams. In Proceedings of the 35th IEEE International Conference on Distributed Computing Systems. IEEE, 411--420.

[47]

Bugra Gedik. 2014. Partitioning functions for stateful data parallelism in stream processing. VLDB J. 23, 4 (2014), 517--539.

Digital Library

[48]

Bugra Gedik, Henrique Andrade, Kun-Lung Wu, Philip S. Yu, and Myungcheol Doo. 2008. SPADE: The system S declarative stream processing engine. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD’08). ACM Press, 1123--1132.

Digital Library

[49]

Bugra Gedik, Scott Schneider, Martin Hirzel, and Kun-Lung Wu. 2014. Elastic scaling for data stream processing. IEEE Trans. Parallel Distrib. Syst. 25, 6 (2014), 1447--1463.

Digital Library

[50]

Javad Ghaderi, Sanjay Shakkottai, and R. Srikant. 2016. Scheduling storms and streams in the cloud. ACM Trans. Model. Perform. Eval. Comput. Syst. 1, 4 (2016), 1--28.

Digital Library

[51]

P. Taylor Goetz and Brian O’Neill. 2014. Storm Blueprints: Patterns for Distributed Real-time Computation. Packt Pub. 1--426 pages. Retrieved from https://www.oreilly.com/library/view/storm-blueprints-patterns/9781782168294/.

[52]

Vincenzo Gulisano, Ricardo Jimenez-Peris, Marta Patino-Martinez, Claudio Soriente, and Patrick Valduriez. 2012. StreamCloud: An elastic and scalable data streaming system. IEEE Trans. Parallel Distrib. Syst. 23, 12 (2012), 2351--2365.

Digital Library

[53]

Jiong He, Yao Chen, Tom Z. J. Fu, Xin Long, Marianne Winslett, Liang You, and Zhenjie Zhang. 2018. HaaS: Cloud-based real-time data analytics with heterogeneity-aware scheduling. In Proceedings of the 38th IEEE International Conference on Distributed Computing Systems, Vol. 1. IEEE, 1017--1028.

[54]

Thomas Heinze. 2011. Elastic complex event processing. In Proceedings of the 8th Doctoral Symposium on Middleware (MDS’11). ACM Press, 1--6.

Digital Library

[55]

Thomas Heinze, Leonardo Aniello, Leonardo Querzoni, and Zbigniew Jerzak. 2014. Cloud-based data stream processing. In Proceedings of the 8th ACM International Conference on Distributed Event-based Systems (DEBS’14). ACM Press, 238--245.

Digital Library

[56]

Thomas Heinze, Zbigniew Jerzak, Gregor Hackenbroich, and Christof Fetzer. 2014. Latency-aware elastic scaling for distributed data stream processing systems. In Proceedings of the 8th ACM International Conference on Distributed Event-based Systems (DEBS’14). ACM Press, 13--22.

Digital Library

[57]

Thomas Heinze, Yuanzhen Ji, Yinying Pan, Franz Josef Grueneberger, Zbigniew Jerzak, and Christof Fetzer. 2013. Elastic complex event processing under varying query load. In Proceedings of the 1st International Workshop on Big Dynamic Distributed Data. Springer, 25--30.

[58]

Thomas Heinze, Valerio Pappalardo, Zbigniew Jerzak, and Christof Fetzer. 2014. Auto-scaling techniques for elastic data stream processing. In Proceedings of the 30th IEEE International Conference on Data Engineering Workshops. IEEE, 296--302.

[59]

Thomas Heinze, Lars Roediger, Andreas Meister, Yuanzhen Ji, Zbigniew Jerzak, and Christof Fetzer. 2015. Online parameter optimization for elastic data stream processing. In Proceedings of the 6th ACM Symposium on Cloud Computing (SoCC’15). ACM Press, 276--287.

Digital Library

[60]

Nicolas Hidalgo, Daniel Wladdimiro, and Erika Rosas. 2017. Self-adaptive processing graph with operator fission for elastic stream processing. J. Syst. Softw. 127, 1 (2017), 205--216.

Digital Library

[61]

Christoph Hochreiner, Stefan Schulte, Schahram Dustdar, and Freddy Lecue. 2015. Elastic stream processing for distributed environments. IEEE Internet Comput. 19, 6 (2015), 54--59.

Digital Library

[62]

Christoph Hochreiner, Michael Vogler, Stefan Schulte, and Schahram Dustdar. 2016. Elastic stream processing for the Internet of Things. In Proceedings of the 9th IEEE International Conference on Cloud Computing. IEEE, 100--107.

[63]

M. Reza Hoseiny Farahabady, Hamid R. Dehghani Samani, Yidan Wang, Albert Y. Zomaya, and Zahir Tari. 2016. A QoS-aware controller for Apache Storm. In Proceedings of the 15th IEEE International Symposium on Network Computing and Applications. IEEE, 334--342.

[64]

Mohammad Reza Hoseiny Farahabady, Albert Y. Zomaya, and Zahir Tari. 2017. QoS- and contention- aware resource provisioning in a stream processing engine. In Proceedings of the IEEE International Conference on Cluster Computing. IEEE, 137--146.

[65]

Mahammad Humayoo, Yanlong Zhai, Yan He, Bingqing Xu, and Chen Wang. 2014. Operator scale out using time utility function in big data stream processing. In Proceedings of the International Conference on Wireless Algorithms, Systems, and Applications. Springer, 54--65.

Digital Library

[66]

Waldemar Hummer, Christian Inzinger, Philipp Leitner, Benjamin Satzger, and Schahram Dustdar. 2012. Deriving a unified fault taxonomy for event-based systems. In Proceedings of the 6th ACM International Conference on Distributed Event-based Systems (DEBS’12). ACM Press, 167--178.

Digital Library

[67]

Waldemar Hummer, Benjamin Satzger, and Schahram Dustdar. 2013. Elastic stream processing in the cloud. Wiley Interdisc. Rev.: Data Min. Knowl. Discov. 3, 5 (2013), 333--345.

Digital Library

[68]

Shigeru Imai, Thomas Chestna, and Carlos A. Varela. 2012. Elastic scalable cloud computing using application-level migration. In Proceedings of the 5th IEEE International Conference on Utility and Cloud Computing. IEEE, 91--98.

[69]

Shigeru Imai, Stacy Patterson, and Carlos A. Varela. 2017. Maximum sustainable throughput prediction for data stream processing over public clouds. In Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. IEEE, 1--10.

[70]

Atsushi Ishii and Toyotaro Suzumura. 2011. Elastic stream computing with clouds. In Proceedings of the 4th IEEE International Conference on Cloud Computing. IEEE, 195--202.

Digital Library

[71]

Gabriela Jacques-Silva, Bugra Gedik, Rohit Wagle, Kun-Lung Wu, and Vibhore Kumar. 2012. Building user-defined runtime adaptation routines for stream processing applications. Proc. VLDB Endow. 5, 12 (2012), 1826--1837.

Digital Library

[72]

Jiahua Fan, Haopeng Chen, and Fei Hu. 2015. Adaptive task scheduling in Storm. In Proceedings of the 4th International Conference on Computer Science and Network Technology. IEEE, 309--314.

[73]

Jiawei Jiang, Zhipeng Zhang, Bin Cui, Yunhai Tong, and Ning Xu. 2017. StroMAX: Partitioning-based scheduler for real-time stream processing system. In Proceedings of the International Conference on Database Systems for Advanced Applications, Vol. 3882. Springer, 269--288.

[74]

Yuxuan Jiang, Zhe Huang, and Danny H. K. Tsang. 2017. Towards max-min fair resource allocation for stream big data analytics in shared clouds. IEEE Trans. Big Data 4, 1 (2017), 130--137.

[75]

Supun Kamburugamuve, Leif Christiansen, and Geoffrey Fox. 2015. A framework for real time processing of sensor data in the cloud. J. Sensors 2015, 1 (2015), 1--11.

[76]

Supun Kamburugamuve and Geoffrey Fox. 2013. Survey of distributed stream processing for large stream sources. Grids UCS Indiana Edu. 2 (2013), 1--16.

[77]

Supun Kamburugamuve, Karthik Ramasamy, Martin Swany, and Geoffrey Fox. 2017. Low latency stream processing: Apache Heron with Infiniband 8 Intel Omni-Path. In Proceedings of the 10th International Conference on Utility and Cloud Computing. ACM Press, 101--110.

Digital Library

[78]

Jeyhun Karimov, Tilmann Rabl, Asterios Katsifodimos, RomanSamarev, Henri Heiskanen, and Volker Markl. 2018. Benchmarking distributed stream data processing systems. In Proceedings of the IEEE 34th International Conference on Data Engineering. IEEE, 1507--1518.

[79]

J. O. Kephart and D. M. Chess. 2003. The vision of autonomic computing. Computer 36, 1 (2003), 41--50.

Digital Library

[80]

Danish Khan, Kshiteej Mahajan, Rahul Godha, and Yuvraj Patel. 2015. Empirical Study of Stragglers in Spark SQL and Spark Streaming. Technical Report. 1--12. Retrieved from http://pages.cs.wisc.edu/.

[81]

Rohit Khandekar, Kirsten Hildrum, Sujay Parekh, Deepak Rajan, Joel Wolf, Kun-Lung Wu, Henrique Andrade, and Bugra Gedik. 2009. COLA: Optimizing stream processing applications via graph partitioning. In Proceedings of the 10th ACM/IFIP/USENIX International Conference on Middleware. Springer, 308--327.

[82]

Alireza Khoshkbarforoushha, Rajiv Ranjan, Raj Gaire, Prem P. Jayaraman, John Hosking, and Ehsan Abbasnejad. 2015. Resource usage estimation of data stream processing workloads in datacenter clouds. arxiv:1501.07020.

[83]

Alireza Khoshkbarforoushha, Rajiv Ranjan, and Peter Strazdins. 2016. Resource distribution estimation for data-intensive workloads: Give me my share and no one gets hurt! In Communications in Computer and Information Science. Vol. 393. Springer, 228--237.

[84]

Wilhelm Kleiminger, Evangelia Kalyvianaki, and Peter Pietzuch. 2011. Balancing load in stream processing with the cloud. In Proceedings of the 27th IEEE International Conference on Data Engineering Workshops. IEEE, 16--21.

Digital Library

[85]

Boris Koldehofe, Ruben Mayer, Umakishore Ramachandran, Kurt Rothermel, and Marco Völz. 2013. Rollback-recovery without checkpoints in distributed event processing systems. In Proceedings of the 7th ACM International Conference on Distributed Event-based Systems (DEBS’13). ACM, 27--38.

Digital Library

[86]

Roland Kotto Kombi, Nicolas Lumineau, and Philippe Lamarre. 2017. A preventive auto-parallelization approach for elastic stream processing. In Proceedings of the 37th IEEE International Conference on Distributed Computing Systems. IEEE, 1532--1542.

[87]

Sanjeev Kulkarni, Nikunj Bhagat, Masong Fu, Vikas Kedigehalli, Christopher Kellogg, Sailesh Mittal, Jignesh M. Patel, Karthik Ramasamy, and Siddarth Taneja. 2015. Twitter Heron: Stream processing at scale. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD’15). ACM Press, 239--250.

Digital Library

[88]

Alok Gautam Kumbhare, Yogesh Simmhan, and Viktor K. Prasanna. 2014. PLAStiCC: Predictive look-ahead scheduling for continuous dataflows on clouds. In Proceedings of the 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. IEEE, 344--353.

[89]

Geetika T. Lakshmanan, Ying Li, and Rob Strom. 2008. Placement strategies for Internet-scale data stream systems. IEEE Internet Computing 12, 6 (2008), 50--60.

Digital Library

[90]

Myungcheol Lee, Miyoung Lee, Sung Jin Hur, and Ikkyun Kim. 2015. Load adaptive distributed stream processing system for explosive stream data. In Proceedings of the 17th International Conference on Advanced Communication Technology, Vol. 5. IEEE, 753--757.

[91]

Boduo Li, Yanlei Diao, and Prashant Shenoy. 2015. Supporting scalable analytics with latency constraints. Proc. VLDB Endow. 8, 11 (2015), 1166--1177.

Digital Library

[92]

Chunlin Li, Jing Zhang, and Youlong Luo. 2017. Real-time scheduling based on optimized topology and communication traffic in distributed real-time computation platform of Storm. J. Netw. Comput. Appl. 87, 10 (2017), 100--115.

Digital Library

[93]

Teng Li, Jian Tang, and Jielong Xu. 2016. Performance modeling and predictive scheduling for distributed stream data processing. IEEE Trans. Big Data 7790, 99 (2016), 1--12.

[94]

Harold Lim and Shivnath Babu. 2013. Execution and optimization of continuous queries with cyclops. In Proceedings of the International Conference on Management of Data (SIGMOD’13). ACM Press, 1069--1072.

Digital Library

[95]

Qian Lin, Beng Chin Ooi, Zhengkui Wang, and Cui Yu. 2015. Scalable distributed stream join processing. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD’15). ACM Press, 811--825.

Digital Library

[96]

Ming Liu, Liang Luo, Jacob Nelson, Luis Ceze, Arvind Krishnamurthy, and Kishore Atreya. 2017. IncBricks: Toward in-network computation with an in-network cache. ACM SIGARCH Comput. Architect. News 45, 1 (2017), 795--809.

Digital Library

[97]

Ning Liu, Zhe Li, Jielong Xu, Zhiyuan Xu, Sheng Lin, Qinru Qiu, Jian Tang, and Yanzhi Wang. 2017. A hierarchical framework of cloud resource allocation and power management using deep reinforcement learning. In Proceedings of the 37th IEEE International Conference on Distributed Computing Systems. IEEE, 372--382.

[98]

Xunyun Liu and Rajkumar Buyya. 2017. D-Storm: Dynamic resource-efficient scheduling of stream processing applications. In Proceedings of the 23rd International Conference on Parallel and Distributed Systems. IEEE, 1--8.

[99]

Xunyun Liu and Rajkumar Buyya. 2017. Performance-oriented deployment of streaming applications on cloud. IEEE Trans. Big Data 14, 8 (2017), 1--14.

[100]

Xunyun Liu, Amir Vahid Dastjerdi, Rodrigo N. Calheiros, Chenhao Qu, and Rajkumar Buyya. 2017. A stepwise auto-profiling method for performance optimization of streaming applications. ACM Trans. Auton. Adapt. Syst. 12, 4 (2017), 1--33.

Digital Library

[101]

Xunyun Liu, Aaron Harwood, Shanika Karunasekera, Benjamin Rubinstein, and Rajkumar Buyya. 2017. E-Storm: Replication-based state management in distributed stream processing systems. In Proceedings of the 46th International Conference on Parallel Processing. IEEE, 571--580.

[102]

Yuan Liu, Xuanhua Shi, and Hai Jin. 2016. Runtime-aware adaptive scheduling in stream processing. Concurr. Comput.: Pract. Exper. 28, 14 (2016), 3830--3843.

Digital Library

[103]

Giorgia Lodi, Leonardo Aniello, Giuseppe A. Di Luna, and Roberto Baldoni. 2014. An event-based platform for collaborative threats detection and monitoring. Info. Syst. 39 (2014), 175--195.

Digital Library

[104]

Bjorn Lohrmann, Peter Janacik, and Odej Kao. 2015. Elastic stream processing with latency guarantees. In Proceedings of the 35th IEEE International Conference on Distributed Computing Systems. IEEE, 399--410.

[105]

Björn Lohrmann, Daniel Warneke, and Odej Kao. 2014. Nephele streaming: Stream processing under QoS constraints at scale. Cluster Comput. 17, 1 (2014), 61--78.

Digital Library

[106]

Federico Lombardi, Leonardo Aniello, Silvia Bonomi, and Leonardo Querzoni. 2018. Elastic symbiotic scaling of operators and resources in stream processing systems. IEEE Trans. Parallel Distrib. Syst. 29, 3 (2018), 572--585.

[107]

Manisha Luthra, Boris Koldehofe, Pascal Weisenburger, Guido Salvaneschi, and Raheel Arif. 2018. TCEP: Adapting to dynamic user environments by enabling transitions between operator placement mechanisms. In Proceedings of the 12th ACM International Conference on Distributed and Event-based Systems. ACM Press, 136--147.

Digital Library

[108]

Kasper Grud Skat Madsen, Philip Thyssen, and Yongluan Zhou. 2014. Integrating fault-tolerance and elasticity in a distributed data stream processing system. In Proceedings of the 26th International Conference on Scientific and Statistical Database Management (SSDBM’14). ACM Press, 1--4.

[109]

Kasper Grud Skat Madsen, Yongluan Zhou, and Li Su. 2016. Enorm: Efficient window-based computation in large-scale distributed stream processing systems. In Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems. ACM Press, 37--48.

Digital Library

[110]

Lena Mashayekhy, Mahyar Movahed Nejad, Daniel Grosu, Quan Zhang, and Weisong Shi. 2015. Energy-aware scheduling of MapReduce jobs for big data applications. IEEE Trans. Parallel Distrib. Syst. 26, 10 (2015), 2720--2733.

Digital Library

[111]

Ruben Mayer, Boris Koldehofe, and Kurt Rothermel. 2014. Meeting predictable buffer limits in the parallel execution of event processing operators. In Proceedings of the IEEE International Conference on Big Data. IEEE, 402--411.

[112]

Ruben Mayer, Boris Koldehofe, and Kurt Rothermel. 2015. Predictable low-latency event detection with parallel complex event processing. IEEE Internet Things J. 2, 4 (2015), 274--286.

[113]

Gabriele Mencagli. 2016. A game-theoretic approach for elastic distributed data stream processing. ACM Trans. Auton. Adapt. Syst. 11, 2 (2016), 1--34.

Digital Library

[114]

Jefferson Morales, Erika Rosas, and Nicolas Hidalgo. 2014. Symbiosis: Sharing mobile resources for stream processing. In Proceedings of the IEEE Symposium on Computers and Communications. IEEE, 1--6.

[115]

Matteo Nardelli. 2016. QoS-aware deployment of data streaming applications over distributed infrastructures. In Proceedings of the 39th International Convention on Information and Communication Technology, Electronics and Microelectronics. IEEE, 736--741.

[116]

Stephen Neuendorffer and Kees Vissers. 2008. Streaming systems in FPGAs. In Embedded Computer Systems: Architectures, Modeling, and Simulation. Springer, 147--156.

[117]

Shadi A. Noghabi, Kartik Paramasivam, Yi Pan, Navina Ramesh, Jon Bringhurst, Indranil Gupta, and Roy H. Campbell. 2017. Samza: Stateful scalable stream processing at LinkedIn. Proc. VLDB Endow. 10, 12 (2017), 1634--1645.

Digital Library

[118]

Beate Ottenwälder, Boris Koldehofe, Kurt Rothermel, Kirak Hong, David Lillethun, and Umakishore Ramachandran. 2014. MCEP: A mobility-aware complex event processing system. ACM Trans. Internet Technol. 14, 1 (2014), 1--24.

Digital Library

[119]

Apostolos Papageorgiou, Ehsan Poormohammady, and Bin Cheng. 2016. Edge-computing-aware deployment of stream processing tasks based on topology-external information: Model, algorithms, and a storm-based prototype. In Proceedings of the 5th IEEE International Congress on Big Data. IEEE, 259--266.

[120]

Boyang Peng, Mohammad Hosseini, Zhihao Hong, Reza Farivar, and Roy Campbell. 2015. R-Storm: Resource-aware scheduling in Storm. In Proceedings of the 16th Annual Conference on Middleware (Middleware’15). ACM Press, 149--161.

Digital Library

[121]

Peter Pietzuch, Jonathan Ledlie, Jeffrey Shneidman, Mema Roussopoulos, Matt Welsh, and Margo Seltzer. 2006. Network-aware operator placement for stream-processing systems. In Proceedings of the 22nd International Conference on Data Engineering (ICDE’06). IEEE, 49--49.

Digital Library

[122]

T. Ralf, Muhammad Intizar Ali, Payam Barnaghi, Sorin Ganea, Frieder Ganz, Manfred Haushwirth, Brigitte Kjærgaard, K. Daniel, Alessandra Mileo, Septimiu Nechifor, Amit Sheth, and Vlasios Tsiatsis. 2014. Real time IoT stream processing and large-scale data analytics for smart city applications. In Proceedings of the European Conference on Networks and Communications. IEEE, 1--5.

[123]

Rajiv Ranjan. 2014. Streaming big data processing in datacenter clouds. IEEE Cloud Comput. 1, 1 (2014), 78--83.

[124]

Thomas Repantis, Xiaohui Gu, and Vana Kalogeraki. 2006. Synergy: Sharing-aware component composition for distributed stream processing systems. In Proceedings of the ACM/IFIP/USENIX International Conference on Middleware, Vol. 4290. Springer, 322--341.

Digital Library

[125]

Stamatia Rizou, Frank Durr, and Kurt Rothermel. 2010. Solving the multi-operator placement problem in large-scale operator networks. In Proceedings of the 19th International Conference on Computer Communications and Networks. IEEE, 1--6.

[126]

Stamatia Rizou, Frank Durr, and Kurt Rothermel. 2011. Fulfilling end-to-end latency constraints in large-scale streaming environments. In Proceedings of the IEEE International Performance Computing and Communications Conference. IEEE, 1--8.

Digital Library

[127]

Stamatia Rizou, Frank Durr, Kurt Rothermel, F. Durr, and Kurt Rothermel. 2010. Providing QoS guarantees in large-scale operator networks. In Proceedings of the 12th IEEE International Conference on High-performance Computing and Communications. IEEE, 337--345.

Digital Library

[128]

Henriette Röger and Ruben Mayer. 2019. A comprehensive survey on parallelization and elasticity in stream processing. ACM Comput. Survey 52, 2 (2019), 1--37.

Digital Library

[129]

Marek Rychly, Petr Koda, and Pavel Mr. 2014. Scheduling decisions in stream processing on heterogeneous clusters. In Proceedings of the 8th International Conference on Complex, Intelligent and Software Intensive Systems. IEEE, 614--619.

Digital Library

[130]

Marek Rychlý, Petr Škoda, and Pavel Smrž. 2015. Heterogeneity-aware scheduler for stream processing frameworks. Int. J. Big Data Intell. 2, 2 (2015), 70--82.

[131]

Mohammad Sadoghi, Rija Javed, Naif Tarafdar, Harsh Singh, Rohan Palaniappan, and Hans-Arno Jacobsen. 2012. Multi-query stream processing on FPGAs. In Proceedings of the 28th IEEE International Conference on Data Engineering. IEEE, 1229--1232.

Digital Library

[132]

Amedeo Sapio, Ibrahim Abdelaziz, Abdulla Aldilaijan, Marco Canini, and Panos Kalnis. 2017. In-network computation is a dumb idea whose time has come. In Proceedings of the 16th ACM Workshop on Hot Topics in Networks. ACM Press, 150--156.

Digital Library

[133]

Kai-Uwe Sattler and Felix Beier. 2013. Towards elastic stream processing: Patterns and infrastructure. In Proceedings of the 1st International Workshop on Big Dynamic Distributed Data. IEEE, 49--54.

[134]

Benjamin Satzger, Waldemar Hummer, Philipp Leitner, and Schahram Dustdar. 2011. Esc: Towards an elastic stream computing platform for the cloud. In Proceedings of the 4th IEEE International Conference on Cloud Computing. IEEE, 348--355.

Digital Library

[135]

Scott Schneider, Martin Hirzel, Bugra Gedik, and Kun-Lung Wu. 2012. Auto-parallelizing stateful distributed streaming applications. In Proceedings of the 21st International Conference on Parallel Architectures and Compilation Techniques (PACT’12). ACM Press, 53--64.

Digital Library

[136]

Scott Schneider and Kun-Lung Wu. 2017. Low-synchronization, mostly lock-free, elastic scheduling for streaming runtimes. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM Press, 648--661.

Digital Library

[137]

Zoe Sebepou and Kostas Magoutis. 2011. CEC: Continuous eventual checkpointing for data stream processing operators. In Proceedings of the 41st IEEE/IFIP International Conference on Dependable Systems and Networks. IEEE, 145--156.

Digital Library

[138]

Vinay Setty, Roman Vitenberg, Gunnar Kreitz, Guido Urdaneta, and Maarten Van Steen. 2014. Cost-effective resource allocation for deploying pub/sub on cloud. In Proceedings of the 34th IEEE International Conference on Distributed Computing Systems. IEEE, 555--566.

Digital Library

[139]

Zhiming Shen, Sethuraman Subbiah, Xiaohui Gu, and John Wilkes. 2011. CloudScale: Elastic resource scaling for multi-tenant cloud systems. In Proceedings of the 2nd ACM Symposium on Cloud Computing (SOCC’11). ACM Press, 1--14.

Digital Library

[140]

Ce-Kuen Shieh, Sheng-Wei Huang, Li-Da Sun, Ming-Fong Tsai, and Naveen Chilamkurti. 2017. A topology-based scaling mechanism for Apache Storm. Int. J. Netw. Manage. 27, 3 (2017), 1933--1952.

[141]

Anshu Shukla and Yogesh Simmhan. 2018. Model-driven scheduling for distributed stream processing systems. J. Parallel Distrib. Comput. 117 (2018), 98--114.

Digital Library

[142]

Anshu Shukla and Yogesh Simmhan. 2018. Toward reliable and rapid elasticity for streaming dataflows on clouds. In Proceedings of the 38th IEEE International Conference on Distributed Computing Systems. IEEE, 1096--1106.

[143]

Pavel Smirnov, Mikhail Melnik, and Denis Nasonov. 2017. Performance-aware scheduling of streaming applications using genetic algorithm. Procedia Comput. Sci. 108, 6 (2017), 2240--2249.

[144]

Dawei Sun and Rui Huang. 2016. A stable online scheduling strategy for real-time stream computing over fluctuating big data streams. IEEE Access 4, 1 (2016), 8593--8607.

[145]

Dawei Sun, Hongbin Yan, Shang Gao, Xunyun Liu, and Rajkumar Buyya. 2017. Rethinking elastic online scheduling of big data streaming applications over high-velocity continuous data streams. J. Supercomput. 74, 2 (2017), 615--636.

Digital Library

[146]

Dawei Sun, Guangyan Zhang, Chengwen Wu, Keqin Li, and Weimin Zheng. 2017. Building a fault tolerant framework with deadline guarantee in big data stream computing environments. J. Comput. Syst. Sci. 89, 1 (2017), 4--23.

[147]

Dawei Sun, Guangyan Zhang, Songlin Yang, Weimin Zheng, Samee U. Khan, and Keqin Li. 2015. Re-stream: Real-time and energy-efficient resource scheduling in big data stream computing environments. Info. Sci. 319 (2015), 92--112.

[148]

Lauritz Thamsen, Thomas Renner, and Odej Kao. 2016. Continuously improving the resource utilization of iterative parallel dataflows. In Proceedings of the 36th IEEE International Conference on Distributed Computing Systems Workshops. IEEE, 1--6.

[149]

Rafael Tolosana-Calasanz, José Ángel Bañares, Congduc Pham, and Omer F. Rana. 2016. Resource management for bursty streams on multi-tenancy cloud environments. Future Gen. Comput. Syst. 55 (2016), 444--459.

Digital Library

[150]

Ankit Toshniwal, Jake Donham, Nikunj Bhagat, Sailesh Mittal, Dmitriy Ryaboy, Siddarth Taneja, Amit Shukla, Karthik Ramasamy, Jignesh M. Patel, Sanjeev Kulkarni, Jason Jackson, Krishna Gade, and Maosong Fu. 2014. Storm@twitter. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD’14). ACM Press, 147--156.

Digital Library

[151]

Jonas Traub, Sebastian Breß, Tilmann Rabl, Asterios Katsifodimos, and Volker Markl. 2017. Optimized on-demand data streaming from sensor nodes. In Proceedings of the ACM Symposium on Cloud Computing. ACM Press, 586--597.

Digital Library

[152]

Jan Sipke van der Veen, Bram van der Waaij, Elena Lazovik, Wilco Wijbrandi, and Robert J. Meijer. 2015. Dynamically scaling Apache Storm for the analysis of streaming data. In Proceedings of the 1st IEEE International Conference on Big Data Computing Service and Applications. IEEE, 154--161.

[153]

Smita Vijayakumar, Qian Zhu, and Gagan Agrawal. 2010. Dynamic resource provisioning for data streaming applications in a cloud environment. In Proceedings of the 2nd IEEE International Conference on Cloud Computing Technology and Science. IEEE, 441--448.

Digital Library

[154]

Rohit Wagle, Henrique Andrade, Kirsten Hildrum, Chitra Venkatramani, and Michael Spicer. 2011. Distributed middleware reliability and fault tolerance support in system S. In Proceedings of the 5th ACM International Conference on Distributed Event-based Systems. ACM Press, 335--346.

Digital Library

[155]

Chunkai Wang, Xiaofeng Meng, Qi Guo, Zujian Weng, and Chen Yang. 2016. OrientStream: A framework for dynamic resource allocation in distributed data stream management systems. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. ACM Press, 2281--2286.

Digital Library

[156]

Chunkai Wang, Xiaofeng Meng, Qi Guo, Zujian Weng, and Chen Yang. 2017. Automating characterization deployment in distributed data stream management systems. IEEE Trans. Knowl. Data Eng. 29, 12 (2017), 2669--2681.

[157]

Di Wang, Elke A. Rundensteiner, Han Wang, and Richard T. Ellison. 2010. Active complex event processing: Applications in real-time health care. Proc. VLDB Endow. 3, 1--2 (2010), 1545--1548.

Digital Library

[158]

Huayong Wang and Li-Shiuan Peh. 2014. MobiStreams: A reliable distributed stream processing system for mobile devices. In Proceedings of the 28th IEEE International Parallel and Distributed Processing Symposium. IEEE, 51--60.

Digital Library

[159]

Daniel Warneke and Odej Kao. 2011. Exploiting dynamic resource allocation for efficient parallel data processing in the cloud. IEEE Trans. Parallel Distrib. Syst. 22, 6 (2011), 985--997.

Digital Library

[160]

Joel Wolf, Nikhil Bansal, Kirsten Hildrum, Sujay Parekh, Deepak Rajan, Rohit Wagle, Kun-Lung Wu, and Lisa Fleischer. 2008. SODA: An optimizing scheduler for large-scale stream-based distributed computer systems. In Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware (Middleware’08). Springer, 306--325.

Digital Library

[161]

Yingjun Wu and Kian-Lee Tan. 2015. ChronoStream: Elastic stateful stream computation in the cloud. In Proceedings of the 31st IEEE International Conference on Data Engineering. IEEE, 723--734.

[162]

Ying Xing, Jeong-Hyon Hwang, Ugur Çetintemel, and Stanley B Zdonik. 2006. Providing resiliency to load variations in distributed stream processing. In Proceedings of the 32nd International Conference on Very Large Data Bases. VLDB Endowment, 775--786.

Digital Library

[163]

Jielong Xu, Zhenhua Chen, Jian Tang, and Sen Su. 2014. T-Storm: Traffic-aware online scheduling in Storm. In Proceedings of the 34th IEEE International Conference on Distributed Computing Systems. IEEE, 535--544.

Digital Library

[164]

Le Xu, Boyang Peng, and Indranil Gupta. 2016. Stela: Enabling stream processing systems to scale-in and scale-out on-demand. In Proceedings of the IEEE International Conference on Cloud Engineering. IEEE, 22--31.

[165]

Lei Yang, Jiannong Cao, Yin Yuan, Tao Li, Andy Han, and Alvin Chan. 2013. A framework for partitioning and execution of data stream applications in mobile cloud computing. ACM SIGMETRICS Perform. Eval. Rev. 40, 4 (2013), 23--32.

Digital Library

[166]

Nikos Zacheilas, Vana Kalogeraki, Nikolas Zygouras, Nikolaos Panagiotou, and Dimitrios Gunopulos. 2015. Elastic complex event processing exploiting prediction. In Proceedings of the IEEE International Conference on Big Data. IEEE, 213--222.

Digital Library

[167]

Matei Zaharia, Tathagata Das, Haoyuan Li, Timothy Hunter, Scott Shenker, and Ion Stoica. 2013. Discretized streams: Fault-tolerant streaming computation at scale. In Proceedings of the 24th ACM Symposium on Operating Systems Principles (SOSP’13). ACM Press, 423--438.

Digital Library

[168]

Jing Zhang, Chunlin Li, Liye Zhu, and Yanpei Liu. 2016. The real-time scheduling strategy based on traffic and load balancing in Storm. In Proceedings of the 18th IEEE International Conference on High-performance Computing and Communications. IEEE, 372--379.

[169]

Zhe Zhang, Yu Gu, Fan Ye, Hao Yang, Minkyong Kim, Hui Lei, and Zhen Liu. 2010. A hybrid approach to high availability in stream processing systems. In Proceedings of the 30th IEEE International Conference on Distributed Computing Systems. IEEE, 138--148.

Digital Library

[170]

Xinwei Zhao, Saurabh Garg, Carlos Queiroz, and Rajkumar Buyya. 2017. A taxonomy and survey of stream processing systems. In Software Architecture for Big Data and the Cloud (1 ed.). Elsevier, 183--206.

[171]

Zhenhuan Gong, Xiaohui Gu, and John Wilkes. 2010. PRESS: PRedictive elastic resource scaling for cloud systems. In Proceedings of the International Conference on Network and Service Management. IEEE, 9--16.

[172]

Yongluan Zhou, Beng Chin Ooi, Kian-lee Tan, and Ji Wu. 2006. Efficient dynamic operator placement in a locally distributed continuous query system. In On the Move to Meaningful Internet Systems. Springer, 54--71.

[173]

Qian Zhu and Gagan Agrawal. 2008. Resource allocation for distributed streaming applications. In Proceedings of the 37th International Conference on Parallel Processing. IEEE, 414--421.

Digital Library

Cited By

Zhou JChen SKuang HWang X(2024)Optimal robust configuration in cloud environment based on heuristic optimization algorithmPeerJ Computer Science10.7717/peerj-cs.235010(e2350)Online publication date: 30-Sep-2024
https://doi.org/10.7717/peerj-cs.2350
Parthasaradi VKarunamurthy AHussaian Basha CSenthilkumar S(2024)Efficient Task Scheduling in Cloud Computing: A Multiobjective Strategy Using Horse Herd–Squirrel Search AlgorithmInternational Transactions on Electrical Energy Systems10.1155/2024/14444932024:1Online publication date: 15-Oct-2024
https://doi.org/10.1155/2024/1444493
Liu YChen RLi SYang JLi Sda Silva B(2024)FPGA-Based Sparse Matrix Multiplication Accelerators: From State-of-the-Art to Future OpportunitiesACM Transactions on Reconfigurable Technology and Systems10.1145/368748017:4(1-37)Online publication date: 28-Aug-2024
https://dl.acm.org/doi/10.1145/3687480
Show More Cited By

Index Terms

Resource Management and Scheduling in Distributed Stream Processing Systems: A Taxonomy, Review, and Future Directions

Recommendations

Iterative Scheduling for Distributed Stream Processing Systems
DEBS '18: Proceedings of the 12th ACM International Conference on Distributed and Event-based Systems

Nowadays data stream processing systems need to efficiently handle large volumes of data in near real-time. To achieve this, the schedulers within such systems minimise the data movement between highly communicating tasks, improving system throughput. ...
Distributed data stream processing and edge computing

Under several emerging application scenarios, such as in smart cities, operational monitoring of large infrastructure, wearable assistance, and Internet of Things, continuous data streams must be processed under very short delays. Several solutions, ...
Distributed resource allocation for stream data processing
HPCC'06: Proceedings of the Second international conference on High Performance Computing and Communications

Data streaming applications are becoming more and more common due to the rapid development in the areas such as sensor networks, multimedia streaming, and on-line data mining, etc. These applications are often running in a decentralized, distributed ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 53, Issue 3

May 2021

787 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/3403423

Editor:
Albert Zomaya
University of Sydney, Australia

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 May 2020

Accepted: 01 August 2019

Revised: 01 July 2019

Received: 01 April 2018

Published in CSUR Volume 53, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Survey
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

43
Total Citations
View Citations
1,629
Total Downloads

Downloads (Last 12 months)242
Downloads (Last 6 weeks)23

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhou JChen SKuang HWang X(2024)Optimal robust configuration in cloud environment based on heuristic optimization algorithmPeerJ Computer Science10.7717/peerj-cs.235010(e2350)Online publication date: 30-Sep-2024
https://doi.org/10.7717/peerj-cs.2350
Parthasaradi VKarunamurthy AHussaian Basha CSenthilkumar S(2024)Efficient Task Scheduling in Cloud Computing: A Multiobjective Strategy Using Horse Herd–Squirrel Search AlgorithmInternational Transactions on Electrical Energy Systems10.1155/2024/14444932024:1Online publication date: 15-Oct-2024
https://doi.org/10.1155/2024/1444493
Liu YChen RLi SYang JLi Sda Silva B(2024)FPGA-Based Sparse Matrix Multiplication Accelerators: From State-of-the-Art to Future OpportunitiesACM Transactions on Reconfigurable Technology and Systems10.1145/368748017:4(1-37)Online publication date: 28-Aug-2024
https://dl.acm.org/doi/10.1145/3687480
Souza PFerreto TCalheiros R(2024)Maintenance Operations on Cloud, Edge, and IoT Environments: Taxonomy, Survey, and Research ChallengesACM Computing Surveys10.1145/365909756:10(1-38)Online publication date: 22-Jun-2024
https://dl.acm.org/doi/10.1145/3659097
Valera HGiraldo LMu˜Noz CMarroquin ADamas LDalmau MRoose PCardinale Y(2024)Energy Optimization through a Multidimensional Distributed Scheduling Approach2024 23rd International Symposium on Parallel and Distributed Computing (ISPDC)10.1109/ISPDC62236.2024.10705392(1-7)Online publication date: 8-Jul-2024
https://doi.org/10.1109/ISPDC62236.2024.10705392
Volnes EPlagemann TGoebel V(2024)To Migrate or Not to Migrate: An Analysis of Operator Migration in Distributed Stream ProcessingIEEE Communications Surveys & Tutorials10.1109/COMST.2023.333095326:1(670-705)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/COMST.2023.3330953
Umar Mushtaq SSheikh SIdrees S(2024)Next-Gen Cloud Efficiency: Fault-Tolerant Task Scheduling With Neighboring Reservations for Improved Resource UtilizationIEEE Access10.1109/ACCESS.2024.340464312(75920-75940)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3404643
Sun DChen HGao SBuyya R(2024)Orchestrating scheduling, grouping and parallelism to enhance the performance of distributed stream computing systemExpert Systems with Applications10.1016/j.eswa.2024.124346(124346)Online publication date: Jun-2024
https://doi.org/10.1016/j.eswa.2024.124346
Li HLuo WXie WYe HDuan X(2024)Adaptive Scheduling Framework of Streaming Applications based on Resource Demand Prediction with Hybrid AlgorithmsJournal of Grid Computing10.1007/s10723-024-09756-422:1Online publication date: 9-Mar-2024
https://dl.acm.org/doi/10.1007/s10723-024-09756-4
Zhang SWang JZhang SXing YSun YGao Y(2024)Task scheduling in cloud computing systems using honey badger algorithm with improved density factor and foucault pendulum motionCluster Computing10.1007/s10586-024-04547-827:9(12411-12457)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s10586-024-04547-8
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents