More Web Proxy on the site http://driver.im/

research-article

Open access

Multi-Variate Time Series Forecasting on Variable Subsets

Authors:

Aravindan Raghuveer,

Balaraman RavindranAuthors Info & Claims

KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Pages 76 - 86

https://doi.org/10.1145/3534678.3539394

Published: 14 August 2022 Publication History

Abstract

We formulate a new inference task in the domain of multivariate time series forecasting (MTSF), called Variable Subset Forecast (VSF), where only a small subset of the variables is available during inference. Variables are absent during inference because of long-term data loss (eg. sensor failures) or high -> low-resource domain shift between train / test. To the best of our knowledge, robustness of MTSF models in presence of such failures, has not been studied in the literature. Through extensive evaluation, we first show that the performance of state of the art methods degrade significantly in the VSF setting. We propose a non-parametric, wrapper technique that can be applied on top any existing forecast models. Through systematic experiments across 4 datasets and 5 forecast models, we show that our technique is able to recover close to 95% performance of the models even when only 15% of the original variables are present.

Supplemental Material

MOV File

We discuss the overview of our work in this presentation. A new problem setup - Variable Subset Forecasting (VSF) is considered in this work. Extensive analysis first shows that SOTA forecast models exhibit significant headroom under this problem. We then propose a solution to tackle this problem based on the retrieval mechanism. Multiple experiments including ablations, comparison to baselines, qualitative as well as quantitative studies show the efficacy of the proposed solution. The code link is also provided in the presentation.

Download
408.22 MB

References

[1]

Akari Asai, Jungo Kasai, Jonathan H. Clark, Kenton Lee, Eunsol Choi, and Hannaneh Hajishirzi. 2020. XOR QA: Cross-lingual Open-Retrieval Question Answering. CoRR, Vol. abs/2010.11856 (2020).

[2]

Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. 2018. An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling. (2018).

[3]

Ankur Bapna and Orhan Firat. 2019. Non-Parametric Adaptation for Neural Machine Translation. In In Proc. of NAACL 2019 .

[4]

Ruichu Cai, Jiawei Chen, Zijian Li, Wei Chen, Keli Zhang, Junjian Ye, Zhuozhang Li, Xiaoyan Yang, and Zhenjie Zhang. 2020. Time series domain adaptation via sparse associative structure alignment. arXiv preprint arXiv:2012.11797 (2020).

[5]

Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, and Qi Zhang. 2020. Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting. In Advances in Neural Information Processing Systems .

[6]

Marco Cuturi. 2011. Fast Global Alignment Kernels. In Proceedings of the 28th International Conference on International Conference on Machine Learning (Bellevue, Washington, USA) (ICML'11). Omnipress, Madison, WI, USA, 929--936.

Digital Library

[7]

Roger Frigola. 2015. Bayesian Time Series Learning with Gaussian Processes (Doctoral thesis). University of Cambridge. https://www.repository.cam.ac.uk/handle/1810/299413

[8]

Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. 2019. Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence (2019).

Digital Library

[9]

Nur Hamid and Wahyu Catur Wibowo. 2018. Wind Speed Forecasting Using Multivariate Time-Series Radial Basis Function Neural Network. In 2018 International Conference on Advanced Computer Science and Information Systems (ICACSIS). IEEE, 423--428.

[10]

Liangzhe Han, Bowen Du, Leilei Sun, Yanjie Fu, Yisheng Lv, and Hui Xiong. 2021. Dynamic and Multi-faceted Spatio-temporal Deep Learning for Traffic Speed Forecasting. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 547--555.

Digital Library

[11]

H. V. Jagadish, Johannes Gehrke, Alexandros Labrinidis, Yannis Papakonstantinou, Jignesh M. Patel, Raghu Ramakrishnan, and Cyrus Shahabi. 2014. Big Data and Its Technical Challenges. Commun. ACM (2014).

[12]

Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. Billion-scale similarity search with gpus. IEEE Transactions on Big Data (2019).

[13]

David C. Kale, Dian Gong, Zhengping Che, Yan Liu, Gerard Medioni, Randall Wetzel, and Patrick Ross. 2014. An Examination of Multivariate Time Series Hashing with Applications to Health Care. In 2014 IEEE International Conference on Data Mining. 260--269. https://doi.org/10.1109/ICDM.2014.153

Digital Library

[14]

Tristan Karb, Niklas Kühl, Robin Hirt, and Varvara Glivici-Cotruta. 2020. A network-based transfer learning approach to improve sales forecasting of new products. arXiv preprint arXiv:2005.06978 (2020).

[15]

Urvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2021. Nearest Neighbor Machine Translation. In International Conference on Learning Representations . https://openreview.net/forum?id=7wCBOfJ8hJM

[16]

Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. 2018. Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks.

[17]

Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2018. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. In International Conference on Learning Representations .

[18]

Yonghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu, et almbox. 2018. Multivariate time series imputation with generative adversarial networks. Advances in neural information processing systems, Vol. 31 (2018).

[19]

Helmut Lütkepohl. 2005. Vector Autoregressive Moving Average Processes .Springer Berlin Heidelberg, Berlin, Heidelberg, 419--446.

[20]

Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. 2018. Deep State Space Models for Time Series Forecasting. In Advances in Neural Information Processing Systems .

[21]

Shun-Yao Shih, Fan-Keng Sun, and Hung-Yi Lee. 2018. Temporal Pattern Attention for Multivariate Time Series Forecasting. (2018).

[22]

Samya Tajmouati, Bouazza El Wahbi, Adel Bedoui, Abdallah Abarda, and Mohamed Dakkoun. 2021. Applying k-nearest neighbors to time series forecasting : two new approaches. arxiv: 2103.14200 [stat.ME]

[23]

Li Tang, Heping Pan, and Yiyong Yao. 2018. K-Nearest Neighbor Regression with Principal Component Analysis for Financial Time Series Prediction.

[24]

Xianfeng Tang, Huaxiu Yao, Yiwei Sun, Charu Aggarwal, Prasenjit Mitra, and Suhang Wang. 2020. Joint modeling of local and global temporal dynamics for multivariate time series forecasting with missing values.

[25]

Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. 2020. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks. arxiv: 2005.11650 [cs.LG]

[26]

Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. 2019. Graph WaveNet for Deep Spatial-Temporal Graph Modeling. International Joint Conferences on Artificial Intelligence Organization.

[27]

Jitao Xu, Josep Crego, and Jean Senellart. 2020. Boosting Neural Machine Translation with Similar Translations. Association for Computational Linguistics.

[28]

B. Yan and C. Domeniconi. 2004. Nearest Neighbor Ensemble. In In 17th International Conference on Pattern Recognition .

[29]

Sijie Yan, Yuanjun Xiong, and Dahua Lin. 2018. Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition. (2018).

[30]

Jennifer Yick, Biswanath Mukherjee, and Dipak Ghosal. 2008. Wireless sensor network survey. Computer networks, Vol. 52, 12 (2008), 2292--2330.

[31]

Hsiang-Fu Yu, Nikhil Rao, and Inderjit S Dhillon. 2016. Temporal Regularized Matrix Factorization for High-dimensional Time Series Prediction. In Advances in Neural Information Processing Systems .

[32]

George Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff. 2021. A Transformer-Based Framework for Multivariate Time Series Representation Learning.

[33]

Aoqian Zhang, Shaoxu Song, Yu Sun, and Jianmin Wang. 2019. Learning individual models for imputation. In 2019 IEEE 35th International Conference on Data Engineering (ICDE). IEEE, 160--171.

Cited By

Yu CWang FShao ZQian TZhang ZWei WXu YBaeza-Yates RBonchi F(2024)GinAR: An End-To-End Multivariate Time Series Forecasting Model Suitable for Variable MissingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3672055(3989-4000)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3672055
Ren XZhao KTakova KRiddle PLi LSerra ESpezzano F(2024)Periormer: Periodic Transformer for Seasonal and Irregularly Sampled Time SeriesProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679720(1973-1982)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679720
Jin MKoh HWen QZambon DAlippi CWebb GKing IPan S(2024)A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.344314146:12(10466-10485)Online publication date: Dec-2024
https://doi.org/10.1109/TPAMI.2024.3443141
Show More Cited By

Index Terms

Multi-Variate Time Series Forecasting on Variable Subsets
1. Computing methodologies
  1. Machine learning

Recommendations

Time series forecasting with the WARIMAX-GARCH method

It is well-known that causal forecasting methods that include appropriately chosen Exogenous Variables (EVs) very often present improved forecasting performances over univariate methods. However, in practice, EVs are usually difficult to obtain and in ...
Time series forecasting by a seasonal support vector regression model

The support vector regression (SVR) model is a novel forecasting approach and has been successfully used to solve time series problems. However, the applications of SVR models in a seasonal time series forecasting has not been widely investigated. This ...
Forecasting Trends in Time Series

Most time series methods assume that any trend will continue unabated, regardless of the forecast lead time. But recent empirical findings suggest that forecast accuracy can be improved by either damping or ignoring altogether trends which have a low ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 2022

5033 pages

ISBN:9781450393850

DOI:10.1145/3534678

General Chairs:
Aidong Zhang
University of Virginia
,
Huzefa Rangwala
Amazon/George Mason University

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2022

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD '22

Sponsor:

KDD '22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 14 - 18, 2022

Washington DC, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
1,994
Total Downloads

Downloads (Last 12 months)416
Downloads (Last 6 weeks)56

Reflects downloads up to 09 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yu CWang FShao ZQian TZhang ZWei WXu YBaeza-Yates RBonchi F(2024)GinAR: An End-To-End Multivariate Time Series Forecasting Model Suitable for Variable MissingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3672055(3989-4000)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3672055
Ren XZhao KTakova KRiddle PLi LSerra ESpezzano F(2024)Periormer: Periodic Transformer for Seasonal and Irregularly Sampled Time SeriesProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679720(1973-1982)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679720
Jin MKoh HWen QZambon DAlippi CWebb GKing IPan S(2024)A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.344314146:12(10466-10485)Online publication date: Dec-2024
https://doi.org/10.1109/TPAMI.2024.3443141
Zhang KWen QZhang CCai RJin MLiu YZhang JLiang YPang GSong DPan S(2024)Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and ProspectsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.338731746:10(6775-6794)Online publication date: Oct-2024
https://doi.org/10.1109/TPAMI.2024.3387317
Mao SZeng XFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Learning Visibility Attention Graph Representation for Time Series ForecastingProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615289(4180-4184)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615289
Sen SRajashekar VDharshan N(2023)Forecasting and Analysing Time Series Data Using Deep LearningIntelligent Systems10.1007/978-981-99-3932-9_25(279-291)Online publication date: 6-Oct-2023
https://doi.org/10.1007/978-981-99-3932-9_25

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents