More Web Proxy on the site http://driver.im/

research-article

Citywide Mobile Traffic Forecasting Using Spatial-Temporal Downsampling Transformer Neural Networks

Authors:

Xu ZhouAuthors Info & Claims

IEEE Transactions on Network and Service Management, Volume 20, Issue 1

Pages 152 - 165

https://doi.org/10.1109/TNSM.2022.3214483

Published: 01 March 2023 Publication History

Abstract

The efficient automated network management methods for mobile operators (for example, mobile traffic prediction) are important goals for future mobile networks. Nevertheless, accurately predicting mobile traffic across an entire city is a challenge that requires the consideration of both prediction accuracy and computational complexity, especially in terms of the thousands of regions involved in high-density and complex base station deployment scenarios for 5G/beyond 5G/6G networks. To solve this problem, this study proposed a novel deep learning network based on the transformer, i.e., a spatial-temporal downsampling neural network (STD-Net), which can dynamically and simultaneously exploit the temporal, local, and global spatial dependencies of mobile traffic. To reduce the computational complexity in spatial domains and achieve a balance between generalization (for all regions) and fitness (for each region), the model decomposes a city into patches and focuses on simultaneously exploiting the temporal and local spatial dependencies in each patch via spatial-temporal transformers. This study’s downsampling transformer is responsible for exploiting global spatial dependencies by uniformly sampling mobile traffic throughout all the regions of an entire city. Computing spatial correlations among sampled regions reduces the computational complexity even further. The superior prediction accuracy of the proposed STD-Net model over state-of-the-art baselines was confirmed by experiments on real-world mobile traffic datasets. The effectiveness of each module was tested to further validate the rationale and feasibility of the proposed STD-Net. Analyses of the computational complexity of the STD-Net revealed that its computational cost contains the same quadratic complexity as vanilla transformers.

References

[1]

W. Saad, M. Bennis, and M. Chen, “A vision of 6G wireless systems: Applications, trends, technologies, and open research problems,” IEEE Netw., vol. 34, no. 3, pp. 134–142, May/Jun. 2020.

[2]

Y. Xia, S. Dong, T. Peng, and T. Wang, “Wireless network abnormal traffic detection method based on deep transfer reinforcement learning,” in Proc. 17th Int. Conf. Mobility Sens. Netw. (MSN), 2021, pp. 528–535.

[3]

S. Talwar, D. Choudhury, K. Dimou, E. Aryafar, B. Bangerter, and K. Stewart, “Enabling technologies and architectures for 5G wireless,” in IEEE MTT-S Int. Microw. Symp. Dig., 2014, pp. 1–4.

[4]

M. Giordani, M. Polese, M. Mezzavilla, S. Rangan, and M. Zorzi, “Toward 6G networks: Use cases and technologies,” IEEE Commun. Mag., vol. 58, no. 3, pp. 55–61, Mar. 2020.

[5]

Y. Xu, G. Gui, H. Gacanin, and F. Adachi, “A survey on resource allocation for 5G heterogeneous networks: Current research, future trends, and challenges,” IEEE Commun. Surveys Tuts., vol. 23, no. 2, pp. 668–695, 2nd Quart., 2021.

[6]

S. Dong, Y. Xia, and T. Peng, “Network abnormal traffic detection model based on semi-supervised deep reinforcement learning,” IEEE Trans. Netw. Service Manag., vol. 18, no. 4, pp. 4197–4212, Dec. 2021.

[7]

H. Liu, L. T. Yang, J. Chen, M. Ye, J. Ding, and L. Kuang, “Multivariate multi-order Markov multi-modal prediction with its applications in network traffic management,” IEEE Trans. Netw. Service Manag., vol. 16, no. 3, pp. 828–841, Sep. 2019.

[8]

P. K. Agyapong, M. Iwamura, D. Staehle, W. Kiess, and A. Benjebbour, “Design considerations for a 5G network architecture,” IEEE Commun. Mag., vol. 52, no. 11, pp. 65–75, Nov. 2014.

[9]

Y. Bengio, P. Simard, and P. Frasconi, “Learning long-term dependencies with gradient descent is difficult,” IEEE Trans. Neural Netw., vol. 5, no. 2, pp. 157–166, Mar. 1994.

Digital Library

[10]

F. A. Gers, J. Schmidhuber, and F. Cummins, “Learning to forget: Continual prediction with LSTM,” Neural Comput., vol. 12, no. 10, pp. 2451–2471, 2000.

Digital Library

[11]

H. D. Trinh, L. Giupponi, and P. Dini, “Mobile traffic prediction from raw data using LSTM networks,” in Proc. IEEE 29th Annu. Int. Symp. Pers. Indoor Mobile Radio Commun. (PIMRC), 2018, pp. 1827–1832.

[12]

C.-W. Huang, C.-T. Chiang, and Q. Li, “A study of deep learning networks on mobile traffic forecasting,” in Proc. IEEE 28th Annu. Int. Symp. Pers. Indoor Mobile Radio Commun. (PIMRC), 2017, pp. 1–6.

[13]

L. Zhaoet al., “T-GCN: A temporal graph convolutional network for traffic prediction,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 9, pp. 3848–3858, Sep. 2020.

[14]

Q. Liu, J. Li, and Z. Lu, “ST-Tran: Spatial-temporal transformer for cellular traffic prediction,” IEEE Commun. Lett., vol. 25, no. 10, pp. 3325–3329, Oct. 2021.

[15]

N. Zhao, Z. Ye, Y. Pei, Y.-C. Liang, and D. Niyato, “Spatial-temporal attention-convolution network for citywide cellular traffic prediction,” IEEE Commun. Lett., vol. 24, no. 11, pp. 2532–2536, Nov. 2020.

[16]

H.-W. Kim, J.-H. Lee, Y.-H. Choi, Y.-U. Chung, and H. Lee, “Dynamic bandwidth provisioning using ARIMA-based traffic forecasting for mobile WiMAX,” Comput. Commun., vol. 34, no. 1, pp. 99–106, 2011.

Digital Library

[17]

D. Zhou, S. Chen, and S. Dong, “Network traffic prediction based on ARFIMA model,” 2013, arXiv:1302.6324.

[18]

C. Zhang, P. Patras, and H. Haddadi, “Deep learning in mobile and wireless networking: A survey,” IEEE Commun. Surveys Tuts., vol. 21, no. 3, pp. 2224–2287, 3rd Quart., 2019.

[19]

C. Zhang, H. Zhang, D. Yuan, and M. Zhang, “Citywide cellular traffic prediction based on densely connected convolutional neural networks,” IEEE Commun. Lett., vol. 22, no. 8, pp. 1656–1659, Aug. 2018.

[20]

C. Zhang, H. Zhang, J. Qiao, D. Yuan, and M. Zhang, “Deep transfer learning for intelligent cellular traffic prediction based on cross-domain big data,” IEEE J. Sel. Areas Commun., vol. 37, no. 6, pp. 1389–1401, Jun. 2019.

[21]

X. Shi, Z. Chen, H. Wang, D.-Y. Yeung, W.-K. Wong, and W.-C. Woo, “Convolutional LSTM network: A machine learning approach for precipitation nowcasting,” in Proc. Int. Conf. Adv. Neural Inf. Process. Syst., vol. 28, 2015, pp. 802–810.

[22]

A. Rago, G. Piro, G. Boggia, and P. Dini, “Anticipatory allocation of communication and computational resources at the edge using spatio-temporal dynamics of mobile users,” IEEE Trans. Netw. Service Manag., vol. 18, no. 4, pp. 4548–4562, Dec. 2021.

[23]

W. Shen, H. Zhang, S. Guo, and C. Zhang, “Time-wise attention aided convolutional neural network for data-driven cellular traffic prediction,” IEEE Wireless Commun. Lett., vol. 10, no. 8, pp. 1747–1751, Aug. 2021.

[24]

G. Bertasius, H. Wang, and L. Torresani, “Is space-time attention all you need for video understanding,” 2021, arXiv:2102.05095.

[25]

X. Wanget al., “Spatio-temporal analysis and prediction of cellular traffic in metropolis,” IEEE Trans. Mobile Comput., vol. 18, no. 9, pp. 2190–2202, Sep. 2019.

[26]

K. He, X. Chen, Q. Wu, S. Yu, and Z. Zhou, “Graph attention spatial-temporal network with collaborative global-local learning for citywide mobile traffic prediction,” IEEE Trans. Mobile Comput., vol. 21, no. 4, pp. 1244–1256, Apr. 2022.

[27]

N. Zhao, A. Wu, Y. Pei, Y.-C. Liang, and D. Niyato, “Spatial-temporal aggregation graph convolution network for efficient mobile cellular traffic prediction,” IEEE Commun. Lett., vol. 26, no. 3, pp. 587–591, Mar. 2022.

[28]

N. Wu, B. Green, X. Ben, and S. O’Banion, “Deep transformer models for time series forecasting: The influenza prevalence case,” 2020, arXiv:2001.08317.

[29]

K. Kondo, A. Ishikawa, and M. Kimura, “Sequence to sequence with attention for influenza prevalence prediction using Google trends,” in Proc. 3rd Int. Conf. Comput. Biol. Bioinf., 2019, pp. 1–7.

[30]

N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in Proc. Eur. Conf. Comput. Vis., 2020, pp. 213–229.

[31]

A. Dosovitskiyet al., “An image is worth 16×16 words: Transformers for image recognition at scale,” 2020, arXiv:2010.11929.

[32]

Z. Liuet al., “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proc. IEEE/CVF Int. Conf. Comput. Vis., 2021, pp. 10012–10022.

[33]

L. Yuanet al., “Tokens-to-token ViT: Training vision transformers from scratch on ImageNet,” in Proc. IEEE/CVF Int. Conf. Comput. Vis., 2021, pp. 558–567.

[34]

H. Ding, G. Trajcevski, P. Scheuermann, X. Wang, and E. Keogh, “Querying and mining of time series data: Experimental comparison of representations and distance measures,” Proc. VLDB Endowment, vol. 1, no. 2, pp. 1542–1552, 2008.

Digital Library

[35]

E. J. Keogh and M. J. Pazzani, “Derivative dynamic time warping,” in Proc. SIAM Int. Conf. Data Min., 2001, pp. 1–11.

[36]

A. Vaswaniet al., “Attention is all you need,” in Proc. Int. Conf. Adv. Neural Inf. Process. Syst., vol. 30, 2017, pp. 6000–6010.

[37]

J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” 2016, arXiv:1607.06450.

[38]

D. Marinet al., “Efficient segmentation: Learning downsampling near semantic boundaries,” in Proc. IEEE/CVF Int. Conf. Comput. Vis., 2019, pp. 2131–2141.

[39]

M. Li, Y. Wang, Z. Wang, and H. Zheng, “A deep learning method based on an attention mechanism for wireless network traffic prediction,” Ad Hoc Netw., vol. 107, Oct. 2020, Art. no.

[40]

G. Barlacchiet al., “A multi-source dataset of urban life in the city of milan and the province of Trentino,” Sci. Data, vol. 2, no. 1, pp. 1–15, 2015.

[41]

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2014, arXiv:1412.6980.

[42]

T. Chai and R. R. Draxler, “Root mean square error (RMSE) or mean absolute error (MAE),” Geosci. Model Develop. Discuss., vol. 7, no. 1, pp. 1525–1534, 2014.

[43]

A. C. Cameron and F. A. Windmeijer, “An R-squared measure of goodness of fit for some common nonlinear regression models,” J. Econom., vol. 77, no. 2, pp. 329–342, 1997.

[44]

C. Zhang and P. Patras, “Long-term mobile traffic forecasting using deep spatio-temporal neural networks,” in Proc. 18th ACM Int. Symp. Mobile Ad Hoc Netw. Comput., 2018, pp. 231–240.

[45]

J. Wanget al., “Spatiotemporal modeling and prediction in cellular networks: A big data enabled deep learning approach,” in Proc. IEEE INFOCOM Conf. Comput. Commun., 2017, pp. 1–9.

[46]

M. Xuet al., “Spatial-temporal transformer networks for traffic flow forecasting,” 2020, arXiv:2001.02908.

[47]

C. Feichtenhofer, A. Pinz, and A. Zisserman, “Convolutional two-stream network fusion for video action recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 1933–1941.

Cited By

Gorshenin AKozlovskaya AGorbunov SKochetkova I(2024)Mobile network traffic analysis based on probability-informed machine learning approachComputer Networks: The International Journal of Computer and Telecommunications Networking10.1016/j.comnet.2024.110433247:COnline publication date: 18-Jul-2024
https://dl.acm.org/doi/10.1016/j.comnet.2024.110433
Tuna EBaykal ASoysal A(2024)Multivariate and multistep mobile traffic prediction with SLA constraintsAd Hoc Networks10.1016/j.adhoc.2024.103594163:COnline publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1016/j.adhoc.2024.103594
Zhang JJin JTang JQu Z(2023)FPTN: Fast Pure Transformer Network for Traffic Flow ForecastingArtificial Neural Networks and Machine Learning – ICANN 202310.1007/978-3-031-44223-0_31(382-393)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1007/978-3-031-44223-0_31

Recommendations

Spatial-Temporal Convolutional Graph Attention Networks for Citywide Traffic Flow Forecasting
CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management

Traffic flow prediction plays an important role in many spatial-temporal data applications, e.g., traffic management and urban planning. Various deep learning techniques are developed to model the traffic dynamic patterns with different neural network ...
Spatial temporal incidence dynamic graph neural networks for traffic flow forecasting
Highlights
- The paper proposes a novel dynamic graph recurrent convolutional neural network model, named Dynamic-GRCNN, to deeply capture the spatio-temporal traffic flow features for more accurately predicting urban passenger traffic flows.
- The ...
Abstract
Accurate and real-time traffic passenger flows forecasting at transportation hubs, such as subway/bus stations, is a practical application and of great significance for urban traffic planning, control, guidance, etc. Recently deep learning based ...
Dynamic Multi-View Graph Neural Networks for Citywide Traffic Inference
Accurate citywide traffic inference is critical for improving intelligent transportation systems with smart city applications. However, this task is very challenging given the limited training data, due to the high cost of sensor installment and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Network and Service Management

IEEE Transactions on Network and Service Management Volume 20, Issue 1

March 2023

876 pages

ISSN:1932-4537

Issue’s Table of Contents

1932-4537 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 March 2023

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gorshenin AKozlovskaya AGorbunov SKochetkova I(2024)Mobile network traffic analysis based on probability-informed machine learning approachComputer Networks: The International Journal of Computer and Telecommunications Networking10.1016/j.comnet.2024.110433247:COnline publication date: 18-Jul-2024
https://dl.acm.org/doi/10.1016/j.comnet.2024.110433
Tuna EBaykal ASoysal A(2024)Multivariate and multistep mobile traffic prediction with SLA constraintsAd Hoc Networks10.1016/j.adhoc.2024.103594163:COnline publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1016/j.adhoc.2024.103594
Zhang JJin JTang JQu Z(2023)FPTN: Fast Pure Transformer Network for Traffic Flow ForecastingArtificial Neural Networks and Machine Learning – ICANN 202310.1007/978-3-031-44223-0_31(382-393)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1007/978-3-031-44223-0_31

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents