QV4: QoE-based Viewpoint-Aware V-PCC-encoded Volumetric Video Streaming

Volumetric videos allow six degrees of freedom (6DoF) movement for viewers, enabling numerous applications in domains such as entertainment, healthcare, and education. MPEG's Video-based Point Cloud Compression (V-PCC) is a recent new standard for volumetric video compression that achieves a considerable compression rate while maintaining the quality of the point cloud sequence. However, V-PCC is hard to fit into existing tiling-based volumetric video streaming framework due to the lack of proper user viewing adaptive techniques. In this paper, we propose QV4, a Quality-of-Experience (QoE) based streaming pipeline for viewpoint-aware V-PCC-encoded volumetric video. Specifically, we leverage the intermediate results produced by the V-PCC encoder to achieve effective and efficient viewpoint-aware tiling for V-PCC. We then build a QoE model and a 6DoF movement model based on real-world user data, to predict the users' viewing experience and behaviors, respectively. The proposed QoE model and 6DoF movement model are combined with viewpoint-aware V-PCC tiling to maximize the visual quality of volumetric videos. Extensive simulations show that by enabling viewpoint-aware adaptation and optimization for V-PCC-encoded volumetric videos, QV4 can achieve up to 14.67% improvement in structural similarity index (SSIM) and 7.39% improvement in video multi-method assessment fusion (VMAF) over highly dynamic viewing behaviors in a network with limited and fluctuating bandwidth.