More Web Proxy on the site http://driver.im/

research-article

6K and 8K Effective Resolution with 4K HEVC Decoding Capability for 360 Video Streaming

Authors:

Maryam Homayouni,

Alireza Aminlou,

Miska M. Hannuksela,

Moncef GabboujAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 15, Issue 2s

Article No.: 68, Pages 1 - 22

https://doi.org/10.1145/3335053

Published: 27 July 2019 Publication History

Abstract

The recent Omnidirectional MediA Format (OMAF) standard, which specifies the delivery of 360° video content, supports only equirectangular projection (ERP) and cubemap projection and their region-wise packing with a limitation on video decoding capability to the maximum resolution of 4K (e.g., 4,096 × 2,048). Streaming of 4K ERP content allows only a limited viewport resolution, which is lower than the resolution of many current head-mounted displays (HMDs). Therefore, to take full advantage of high-resolution HMDs, delivery of 360° video content beyond 4K resolution needs to be enabled. In this regard, we propose two specific mixed-resolution packing schemes of 6K (e.g., 6,144 × 3,072) and 8K (e.g., 8,192 × 4,096) ERP content and their realization in tile-based streaming, while complying with the 4K decoding constraint and the High Efficiency Video Coding standard. The proposed packing schemes offer 6K and 8K effective resolution at the viewport. Using our proposed test methodology, experimental results indicate that the proposed layouts significantly decrease streaming bitrates when compared to mixed-quality viewport-adaptive streaming of 4K ERP. Our results further indicate that 8K-effective packing outperforms 6K-effective packing especially in high-quality videos.

References

[1]

K. Kammachi-Sreedhar, A. Aminlou, M. M. Hannuksela, and M. Gabbouj. 2016. Viewport-adaptive encoding and streaming of 360-degree video for virtual reality applications. In Proceedings of the IEEE International Symposium on Multimedia (ISM’16), 583--586.

[2]

S. Lederer. 2017. Today's and future challenges with new forms of content like 360, AR, and VR. 2017. In Proceedings of the MPEG Workshop Global Media Technology Standards for an Immersive Age. Retrieved from http://mpeg.chiariglione.org/sites/default/files/events/06_Lederer.pdf.

[3]

A. Zare, A. Aminlou, and M. M. Hannuksela. 2017. Virtual reality content streaming: Viewport-dependent and tile-based techniques. In Proceedings of the IEEE International Conference on Image Processing (ICIP’17).

[4]

Y. Sánchez, R. Skupin, and T. Schierl. 2015. Compressed domain video processing for tile-based panoramic streaming using HEVC. Proceedings of the IEEE International Conference on Image Processing (ICIP’15). 2244--2248.

[5]

A. Zare, A. Aminlou, M. M. Hannuksela, and M. Gabbouj. 2016. HEVC-compliant tile-based streaming of panoramic video for virtual reality applications. In Proceedings of the 2016 ACM on Multimedia Conference. 601--605.

Digital Library

[6]

B. Choi, Y.-K. Wang, M. M. Hannuksela, Y. Lim, and A. Murtaza. 2017. Text of ISO/IEC FDIS 23090-2 Omnidirectional media format, document: MPEG-M41922. Macau, China.

[7]

Z. CAM™ V1 Pro Cinematic VR Camera. Retrieved from http://www.z-cam.com/360-vr-camera-v1-pro/.

[8]

Comparison of virtual reality headsets. Retrieved fromhttps://en.wikipedia.org/wiki/Comparison_of_virtual_reality_headsets.

[9]

A. Zare, A. Aminlou, and M. M. Hannuksela. 2018. 6K effective resolution with 4K HEVC decoding capability for OMAF-compliant 360 video streaming. In Proceedings of the 23rd Packet Video Workshop. 72--77.

Digital Library

[10]

K. M. Misra, C. A. Segall, M. Horowitz, S. Xu, A. Fuldseth, and M. Zhou. 2013. An overview of tiles. IEEE J. Select. Topics Signal Process. 7, 6 969--977.

[11]

M. M. Hannuksela, Y. K. Wang, and M. Gabbouj. 2004. Isolated regions in video coding. IEEE Trans. Multimedia 6, 259--267.

Digital Library

[12]

R. Ghaznavi-Youvalari et al. 2017. Comparison of HEVC coding schemes for tile-based viewport-adaptive streaming of omnidirectional video. In Proceedings of the IEEE 19th International Workshop on Multimedia Signal Processing (MMSP’17), 1--6.

[13]

Information technology -- Coding of audio-visual objects -- Part 12: ISO base media file format (ISO/IEC 14496-12:2015). 2015. Retrieved from https://www.iso.org/standard/68960.html.

[14]

M. M. Hannuksela, E. B. Aksu, V. K. Malamal Vadakital, and J. Lainema. 2017. Overview of the high efficiency image file format. JCT-VC contribution, document: JCTVC-V0072. Geneva. Retrieved from http://phenix.it-sudparis.eu/jct/doc_end_user/documents/22_Geneva/wg11/JCTVC-V0072-v1.zip.

[15]

Information technology -- Dynamic adaptive streaming over HTTP (DASH) -- Part 1: Media presentation description and segment formats (ISO/IEC 23009-1:2014). 2014. Retrieved from https://www.iso.org/standard/65274.html.

[16]

D. Podborski, E. Thomas, M. M. Hannuksela, S. Oh, T. Stockhammer, and S. Pham. 2017. Virtual reality and DASH. In Proceedings of the International Broadcasting Convention (IBC’17).

[17]

M. M. Hannuksela, Y.-K. Wang, and A. Hourunranta. 2019. An overview of the OMAF standard for 360° video. In Proceedings of the Data Compression Conference.

[18]

H. Hristova, X. Corbillon, G. Simon, V. Swaminathan, and A. Devlic. 2018. Heterogeneous spatial quality for omnidirectional video. In Proceedings of the IEEE 20th International Workshop on Multimedia Signal Processing.

[19]

VR Industry Forum Guidelines. 2018. Retrieved from https://www.vr-if.org/guidelines/.

[20]

G. V. d. Auwera, M. Coban, and H. Mart. 2016. Truncated square pyramid projection (TSP) for 360 video. In Proceedings of the ITU-T Joint Video Exploration Team (JVET’16).

[21]

A. Zare, A. Aminlou, and M. M. Hannuksela. 2017. Requirements and proposed method for viewport-adaptive quality assessment. In Proceedings of the ITU-T Joint Video Exploration Team (JVET’17).

[22]

A. Aminlou, K. Kammachi-Sreedhar, A. Zare, and M. M. Hannuksela. 2016. Testing methodology for viewport-dependent encoding and streaming. In Proceedings of the ITU-T Joint Video Exploration Team (JVET’16).

[23]

M. Yu, H. Lakshman, and B. Girod. 2015. A framework to evaluate omnidirectional video coding schemes. In Proceedings of the IEEE International Symposium on Mixed and Augmented Reality. 31--36.

Digital Library

[24]

Y. Sun, A. Lu, and L. Yu. 2016. AHG8: WS-PSNR for 360 video objective quality evaluation. In Proceedings of the Joint Video Exploration Team of ITU-T (JVET’16).

[25]

A. Singla, S. Fremerey, A. Raake, P. List, and B. Feiten. 2017. Measurement of user exploration behavior for omnidirectional (360°) videos with a head mounted display. In Proceedings of the ITU-T Joint Video Exploration Team (JVET’17).

[26]

J. Boyce, E. Alshina, A. Abbas, and Y. Ye. 2016. JVET common test conditions and evaluation procedures for 360 video. In Proceedings of the ITU-T Joint Video Exploration Team (JVET’16).

[27]

G. Bjøntegard. 2001. Calculation of average PSNR differences between RD-curves.

[28]

C. Ozcinar, J. Cabrera, and A. Smolic. 2019. Visual attention-aware omnidirectional video streaming using optimal tiles for virtual reality. IEEE J. Emerg. Select. Top. Circ. Syst. 9, 1 (2019) 217--230.

Cited By

Li ZSun H(2023)Artificial intelligence-based spatio-temporal vision sensors: applications and prospectsFrontiers in Materials10.3389/fmats.2023.126999210Online publication date: 7-Dec-2023
https://doi.org/10.3389/fmats.2023.1269992
Laghari AShahid SYadav RKarim SKhan ALi HShoulin Y(2023)The state of art and review on video streamingJournal of High Speed Networks10.3233/JHS-22208729:3(211-236)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.3233/JHS-222087
Jeong JPark JLee SRyu E(2023)Fine-grained Single-layer Tiling for Viewport-Adaptive 360-degree Video Streaming2023 IEEE International Conference on Visual Communications and Image Processing (VCIP)10.1109/VCIP59821.2023.10402688(1-5)Online publication date: 4-Dec-2023
https://doi.org/10.1109/VCIP59821.2023.10402688
Show More Cited By

Index Terms

6K and 8K Effective Resolution with 4K HEVC Decoding Capability for 360 Video Streaming
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Virtual reality
2. Information systems
  1. Information systems applications
    1. Multimedia information systems
      1. Multimedia streaming

Recommendations

HEVC-compliant Tile-based Streaming of Panoramic Video for Virtual Reality Applications
MM '16: Proceedings of the 24th ACM international conference on Multimedia

Delivering wide-angle and high-resolution spherical panoramic video content entails a high streaming bitrate. This imposes challenges when panorama clips are consumed in virtual reality (VR) head-mounted displays (HMD). The reason is that the HMDs ...
6K Effective Resolution with 4K HEVC Decoding Capability for OMAF-compliant 360° Video Streaming
PV '18: Proceedings of the 23rd Packet Video Workshop

The recent Omnidirectional MediA Format (OMAF) standard specifies delivery of 360° video content. OMAF supports only equirectangular (ERP) and cubemap projections and their region-wise packing with a limitation on video decoding capability to the ...
Comparison of HEVC-based OMAF-compliant 6K effective packings for viewport-dependent 360-degree video streaming
PV '20: Proceedings of the 25th ACM Workshop on Packet Video

Omnidirectional MediA Format (OMAF), a recent standard for representation of 360° video content, supports only equirectangular projection (ERP) and cubemap projection (CMP) and their region-wise packing with a 4K decoding-capability constraint on the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 15, Issue 2s

Special Section on Cross-Media Analysis for Visual Question Answering, Special Section on Big Data, Machine Learning and AI Technologies for Art and Design and Special Section on MMSys/NOSSDAV 2018

April 2019

381 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3343360

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 July 2019

Accepted: 01 September 2018

Revised: 01 September 2018

Received: 01 October 2017

Published in TOMM Volume 15, Issue 2s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
426
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)8

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li ZSun H(2023)Artificial intelligence-based spatio-temporal vision sensors: applications and prospectsFrontiers in Materials10.3389/fmats.2023.126999210Online publication date: 7-Dec-2023
https://doi.org/10.3389/fmats.2023.1269992
Laghari AShahid SYadav RKarim SKhan ALi HShoulin Y(2023)The state of art and review on video streamingJournal of High Speed Networks10.3233/JHS-22208729:3(211-236)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.3233/JHS-222087
Jeong JPark JLee SRyu E(2023)Fine-grained Single-layer Tiling for Viewport-Adaptive 360-degree Video Streaming2023 IEEE International Conference on Visual Communications and Image Processing (VCIP)10.1109/VCIP59821.2023.10402688(1-5)Online publication date: 4-Dec-2023
https://doi.org/10.1109/VCIP59821.2023.10402688
Lin JZhang YLi NJiang H(2022)Joint Source-Channel Decoding of Polar Codes for HEVC-Based Video StreamingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/350220818:4(1-23)Online publication date: 4-Mar-2022
https://dl.acm.org/doi/10.1145/3502208
Jeong JLee SRyu E(2021)DWS-BEAM: Decoder-Wise Subpicture Bitstream Extracting and Merging for MPEG Immersive Video2021 International Conference on Visual Communications and Image Processing (VCIP)10.1109/VCIP53242.2021.9675419(1-5)Online publication date: 5-Dec-2021
https://doi.org/10.1109/VCIP53242.2021.9675419
Zare AAminlou AHannuksela M(2021)VVC Adaptive Loop Filter Optimization for Subpicture-based Viewport-adaptive Streaming2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP)10.1109/MMSP53017.2021.9733579(1-6)Online publication date: 6-Oct-2021
https://doi.org/10.1109/MMSP53017.2021.9733579
Homayouni MZare AAminlou AHannuksela MGabbouj MHellwagner HLiu YCurcio I(2020)Comparison of HEVC-based OMAF-compliant 6K effective packings for viewport-dependent 360-degree video streamingProceedings of the 25th ACM Workshop on Packet Video10.1145/3386292.3397119(15-20)Online publication date: 10-Jun-2020
https://dl.acm.org/doi/10.1145/3386292.3397119

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents