Novel Motion-Compensated Spatio-Temporal Filtering Scheme for x265 Open-Source Video Encoder

There is a strong demand to decrease the video transmission bitrate without reducing visual quality [1]. The x265 encoder [2]-[4] is a popular open-source encoder, which generates bitstreams compliant with the H.265/MPEG-HEVC video coding standard [5]. Built on top of x264[6], the x265 encoder is integrated into several popular open-source frameworks, such as ffmpeg [7], GStreamer [8], and Handbrake [9]. In addition, the x265 is used by a variety of broadcast and streaming service providers who leverage the benefits of HEVC for streaming live and over-the-top (OTT) content. In addition to implementing nearly all the tools defined in HEVC, it implements many algorithmic optimizations that enable trading off encoder performance for quality [2]-[4]. The performance-critical kernels are implemented with hand-coded assembly kernels that use AVX2 and AVX-512 single instruction, multiple data instructions to improve performance on x86 CPUs. This flexible architecture of x265 makes it a popular choice for HEVC encoding for both on-premises and cloud services.

Recent x265 development efforts have been focused on further improving the coding gains. Specifically, the motion compensated spatio-temporal filtering (MCSTF) employed within the coding loop is especially useful for pictures that contain a high level of noise. It utilizes previously generated motion vectors across different video content resolutions to find the best temporal correspondence for low-pass filtering, while the temporal filtering is applied to the I- and P-frames. Figure 1 schematically illustrates the motion estimation process for temporal filtering in a temporal window, which consists of 5 adjacent pictures: two past, two future and one central picture used for producing a single filtered picture. Motion estimation is applied between the central picture and each future or past picture, thereby generating multiple motion-compensated predictions, which are then combined by using adaptive filtering to produce a final noise-reduced picture. Thus, a hierarchical motion estimation scheme is employed (layers L0, L1 and L2, are illustrated in Figure 2). Subsampled pictures are generated for all reference pictures

and the original picture as well: i.e., L1, while L2 is derived from L1 by using the same subsampling method. First, the motion estimation is done for each 16x16 block in L2. Then, the selected motion vector is used as an initial value for estimating the motion in L1. After that, the same is performed for estimating the motion in L0.

As a final step, the subpixel motion is estimated for each 8x8 block by using an interpolation filter on L0. Particularly, the motion of reference pictures before and after, relative to the original picture, is estimated per the 8x8 picture block. In turn, the motion compensation is applied on the pictures before and after the original picture according to the best matching motion for each block. i.e., such that pixel coordinates of the original picture in each block have the best matching coordinates within the referenced pictures. The filter is then applied to the current pixels, and after that, the filtered picture is encoded. Note that the pixels are processed one by one for the luma and chroma channels. The new sample value, is calculated by using the following equation:

[EQUATION]

where I_o is the original pixel, I_r(i) is the intensity of the corresponding pixel within the motion compensated picture i, and w_r(i, a) is the weight of the motion compensated picture where a is the number of available motion compensated pictures. The conducted extensive experimental results show significant bit-rate savings in terms of BD-BR [10].