WO2019032765A1

WO2019032765A1 - Frame-rate up conversion with reduced complexity

Info

Publication number: WO2019032765A1
Application number: PCT/US2018/045900
Authority: WO
Inventors: Yan Ye; Xiaoyu XIU; Yuwen He; Yan Zhang
Original assignee: Vid Scale, Inc.
Priority date: 2017-08-09
Filing date: 2018-08-09
Publication date: 2019-02-14
Also published as: WO2019032765A8

Abstract

When a FRUC mode is enabled for a current coding unit (CU), motion vector (MV) candidates may be derived for the current CU, One or more search MVs may be derived from the MV candidates so that an initiai motion search may be performed for the current CU using the search MVs. The search MVs, which may be fewer than the MV candidates for the CU, may be derived based on one or more attributes of the MV candidates. At the sub-CU level, sub-CU MV candidates may be determined for a current sub-CU. Sub-CU search MVs may be derived from the sub-CU MV candidates for the current sub-CU so that a motion search may be performed for the current sub-CU using the sub-CU search MVs. The number of the sub-CU search MVs may be smaller than the number of the sub-CU MV candidates.

Description

FRA E-RATE UP CONVERSION WITH REDUCED COMPLEXITY CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 62/543,060, filed August 9, 2017, U.S. Provisional Patent Application No. 62/547,539. filed August 18, 2017. U.S. Provisional Patent Application No. 62/561 ,024, filed September 20, 2017, U.S. Provisional Patent Application No. 62/599,238, filed December 15, 2017, U.S. Provisional Patent Application No. 62/622,993, filed January 29, 2018, the contents of which are incorporated by reference.

BACKGROUND

[0002] A variety of digital video compression technologies enable efficient digital video communication, distribution and consumption. Some examples of standardized video compression technologies are H.261 , MPEG-1 , MPEG-2, H.263, MPEG-4 part2 and H.284/MPEG-4 part 10 AVC. Advanced video compression technologies, such as High Efficiency Video Coding (HEVC), may provide twice the compression or half the bit rate at the same video quality compared to H.264/AVC.

SUMMARY

[0003] Systems, methods, and instrumentalities are disclosed for frame-rate up conversion. Motion derivation (e.g., for frame-rate up conversion (FRUC)) may be performed.

[0004] When a FRUC mode is enabled for a current coding unit (CU), motion vector (MV) candidates may be derived for the current CU. One or more search MVs may be derived from the MV candidates so that an initial motion search may be performed for the current CU using the search MVs. The search MVs, which may be fewer than the MV candidates for the CU, may be derived based on one or more attributes of the MV candidates.

[0005] At the sub-CU level, sub-CU MV candidates may be determined for a current sub-CU, Sub-CU search MVs may be derived from the sub-CU MV candidates for the current sub-CU so that a motion search may be performed for the current sub-CU using the sub-CU search MVs, The number of the sub- CU search MVs may be smaller than the number of the sub-CU MV candidates, [0006] To derive the search MVs from the MV candidates, the MV candidates may be grouped into one or more MV candidate sets, A search MV may be derived from each MV candidate set. For example, a representative MV of the MV candidate set may be determined and used as the search MV. The representative MV of the MV candidate set may include a centroid of the MV candidate set.

[0007] The grouping of the MV candidates may be based on one or more attributes of the MV candidates, Clustering may be used to group the MV candidates Into the MV candidate sets. Alternatively or additionally, the MV candidates may be grouped based on respective reference pictures of the MV candidates.

[0008] The one or more attributes used for MV candidate grouping may include a spatial distance between an MV candidate and a centroid MV of an MV candidate set. Spatial distances between an MV candidate and centroids of MV candidate sets may be determined, The MV candidate set whose centroid has the shortest spatial distance from the MV candidate may be identified. If the shortest spatial distance is smaller than a pre-defined threshold, the MV candidate may be grouped with the MV candidate set,

[0009] The one or more attributes used for MV candidate grouping may include MV values of the MV candidates, The MV candidates may be grouped based on the respective MV values of the MV candidates.

[0010] A representative reference picture may be selected among pictures that are referred to by the MV candidates, Each MV candidate may refer to a reference picture, A reference picture that is most frequently referred to by the MV candidates may be selected as the representative reference picture, Search MVs may be derived using the representative reference picture by scaling the MV candidates to the selected representative reference picture.

[0011] MV candidates may be scaled to refer to a selected reference picture. For example, the representative reference picture may be selected, and the MV candidates may be scaled to refer to the representative reference picture. The MV candidates that refer to a reference picture different from the selected reference picture may be scaled so that the scaled MV candidates also refer to the selected reference picture. If the MV candidates become substantially similar after the scaling, at least one of the substantially similar MV candidates may be discarded. Search MVs may be derived using the MV candidates after the MV candidates have been scaled and/or pruned,

[0012] Representative reference picture selection, MV candidate scaling and pruning, and MV grouping may be performed independently.

BRIEF DESCRIPTION OF THE DRAWINGS

[0013] FIG. 1 A is a system diagram illustrating an example communications system in which one or more disclosed embodiments may be implemented, [0014] FIG. 1 B is a system diagram illustrating an example wireless transmit/receive unit (WTRU) that may be used within the communiGations system illustrated in FiG. 1 A according to an embodiment.

[0015] FIG. 1 C is a system diagram illustrating an example radio access network (RAN) and an example core network (CN) that may be used within the communications system illustrated in FIG. 1A according to an embodiment.

[0018] FIG, 1 D is a system diagram illustrating a further example RAN and a further example CN that may be used within the communications system illustrated in FiG. 1A according to an embodiment.

[0017] FIG, 2 illustrates an example of a block-based hybrid video encoding system,

[0018] FIG, 3 illustrates an example of a block-based video decoder.

[0019] FIG, 4A illustrates an example of frame-rate up conversion (FRUC) template matching,

[0020] FIG, 4B illustrates an example of FRUC bilateral matching.

[0021] FIG. 5 illustrates an example of a motion derivation process in a FRUC mode.

[0022] FiG. 6 illustrates example positions of spatial candidates in merge mode.

[0023] FIG. 7 illustrates an example of motion field interpolation that may be used in FRUC.

[0024] FIG, 8 illustrates an example template, Tc, and corresponding prediction blocks, TO and T1 , in a current picture and two reference pictures.

[0025] FIG. 9 illustrates an example associated with low-delay configuration.

[0026] FIG. 10 illustrates an example associated with random configuration.

[0027] FIG. 1 1 illustrates an example of motion vector (MV) grouping {e.g., using MV clustering).

[0028] FIG. 12 illustrates an example of a FRUC motion derivation process with a clustering-based motion search algorithm applied at a coding unit (CU)-ievel and a sub-CU-level.

[0029] FIG. 13 illustrates an example of integer motion refinement.

[0030] FIG, 14 illustrates an example of integer motion refinement.

[0031] FIG, 15 illustrates an example of fractional motion refinement.

[0032] FIG. 16 illustrates an example of fractional motion refinement.

[0033] FIG. 17 illustrates an example of a FRUC motion derivation process with a multi-stage motion search applied at CU-ievel motion refinement and sub-CU-level motion refinement,

[0034] FIG. 18 illustrates an example of a FRUC motion derivation process with early termination of sub- CU-level motion derivation for FRUC bilateral mode.

[0035] FIG. 19 shows an example of a FRUC motion derivation process with early termination {e.g., disabling) of sub-CU-level motion derivation based on MV difference criterion.

[0036] FIG, 20 illustrates an example of a FRUC motion derivation process with early termination of sub- CU-level motion derivation based on a similarity of prediction blocks at a CU-ievel,

- - [0037] FIG. 21 illustrates an example FRUC motion derivation process with early termination of sub-CU- leve! motion derivation based on CU-level distortion for FRUC bilateral mode,

[0038] FIG. 22 illustrates an example of applying sub-CU-level motion derivation for a FRUC template CU.

[0039] FIG. 23 illustrates an example the FRUC motion derivation process with early termination of sub- CU-level motion derivation for FRUC template mode.

[0040] FIG, 24 shows an example FRUC motion derivation process when a combined early termination method is applied to conditionally skip sub-CU-level motion derivation process.

[0041] FIG, 25 illustrates an example local illumination compensation (LIC) flag encoding for FRUC bilateral mode.

[0042] FIG, 26 illustrates an example LIC flag decoding for FRUC bi-lateral mode.

[0043] FIG. 27 illustrates an example associated with deriving MVs from merge candidates (e.g., for

FRUC search).

DETAILED DESCRIPTION

[0044] A detailed description of illustrative embodiments will now be described with reference to the various Figures. Although this description provides a detailed example of possible implementations, it should be noted that the details are intended to be exemplary and in no way limit the scope of the application.

[0045] FIG, 1 A is a diagram illustrating an example communications system 100 In which one or more disclosed embodiments may be implemented. The communications system 100 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users. The communications system 100 may enable multiple wireless users to access such content through the sharing of system resources, including wireless bandwidth, For example, the communications systems 100 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), zero-tail unique-word DFT-Spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block-filtered OFDM, filter bank multicarrier (FBMC), and the like,

[0046] As shown in FIG. 1A, the communications system 100 may include wireless transmit/receive units (WTRUs) 102a, 102b, 102c, 102d, a RAN 104/113, a CN 106/1 15, a public switched telephone network (PSTN) 108, the Internet 110, and other networks 1 12, though it will be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and/or network elements. Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and/or communicate in a wireless environment, By way of example, the WTRUs 102a, 102b, 102c, 102d, any of which may be referred to as a "station" and/or a "STA", may be configured to transmit and/or receive wireless signals and may include a user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit, a pager, a cellular telephone, a personal digital assistant (PDA), a smartphone, a laptop, a netbook, a personal computer, a wireless sensor, a hotspot or Mi-Fi device, an Internet of Things (ioT) device, a watch or other wearable, a head-mounted display (HMD), a vehicle, a drone, a medical device and applications (e.g., remote surgery), an Industrial device and applications (e.g., a robot and/or other wireless devices operating in an industrial and/or an automated processing chain contexts), a consumer electronics device, a device operating on commercial and/or industrial wireless networks, and the like. Any of the WTRUs 102a, 102b, 102c and 102d may be interchangeably referred to as a UE, [0047] The communications systems 100 may also include a base station 114a and/or a base station 1 14b, Each of the base stations 1 14a, 1 14b may be any type of device configured to wireless!y interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as the CN 106/1 15, the Internet 110, and/or the other networks 1 12, By way of example, the base stations 1 14a, 1 14b may be a base transceiver station (BTS), a Node-B, an eNode B, a Home Node B, a Home eNode B, a gNB, a NR NodeB, a site controller, an access point (AP), a wireless router, and the like, While the base stations 114a, 114b are each depicted as a single element, it will be appreciated that the base stations 1 14a, 1 14b may include any number of interconnected base stations and/or network elements.

[0048] The base station 114a may be part of the RAN 104/113, which may also include other base stations and/or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc. The base station 114a and/or the base station 114b may be configured to transmit and/or receive wireless signals on one or more carrier frequencies, which may be referred to as a cell (not shown), These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage for a wireless service to a specific geographical area that may be relatively fixed or that may change over time. The cell may further be divided into cell sectors. For example, the cell associated with the base station 1 14a may be divided into three sectors, Thus, in one embodiment, the base station 114a may include three transceivers, i.e., one for each sector of the cell, In an embodiment, the base station 114a may employ multiple-input multiple output (MIMO) technology and may utilize multiple transceivers for each sector of the ceil. For example, beamforming may be used to transmit and/or receive signals in desired spatial directions,

[0049] The base stations 114a, 1 14b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 1 16, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT). [0050] More specifically, as noted above, the communications system 100 may be a muitip!e access system and may employ one or more channei access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the iike. For example, the base station 1 14a in the RAN 104/1 13 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 115/116/1 17 using wideband CDMA (WCDMA). WCDMA may include communication protocols such as High-Speed Packet Access (HSPA) and/or Evolved HSPA (HSPA+j, HSPA may include High-Speed Downlink (DL) Packet Access (HSDPA) and/or High-Speed UL Packet Access (HSUPA).

[0051] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 1 16 using Long Term Evolution (LTE) and/or LTE-Advanced (LTE-A) and/or LTE-Advanced Pro (LTE-A Pro).

[0052] in an embodiment, the base station 114a and the ^"WTRUs 102a, 102b, 102c may implement a radio technology such as NR Radio Access, which may establish the air interface 1 16 using New Radio (NR).

[0053] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may implement LTE radio access and NR radio access together, for instance using dual connectivity (DC) principles. Thus, the air interface utilized by WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and/or transmissions sent to/from multiple types of base stations (e.g., an eNB and a gNB).

[0054] In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement radio technologies such as IEEE 802.11 (i.e., Wireless Fidelity (WiFI), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evoiution (EDGE), GSM EDGE (GERAN), and the like.

[0055] The base station 1 14b in FIG. 1 A may be a wireless router, Home Node B, Home eNode B, or access point, for example, and may utilize any suitable RAT for facilitating wireless connectivity in a localized area, such as a place of business, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a roadway, and the like. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN), In an embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). in yet another embodiment, the base station 1 14b and the WTRUs 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR etc) to establish a picocell or femtocell. As shown in FIG. 1 A, the base station 1 14b may have a direct connection to the Internet 1 10, Thus, the base station 1 14b may not be required to access the Internet 1 10 via the CN 106/1 15,

[0058] The RAN 104/1 13 may be in communication with the CN 106/1 15, which may be any type of network configured to provide voice, data, applications, and/or voice over internet protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have varying quality of service (QoS) requirements, such as differing throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, and the like. The CN 106/1 15 may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, etc. , and/or perform high-level security functions, such as user authentication. Although not shown in FIG. 1A, it will be appreciated that the RAN 104/1 13 and/or the CN 106/1 15 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 104/1 13 or a different RAT. For example, in addition to being connected to the RAN 104/1 13, which may be utilizing a NR radio technology, the CN 106/1 15 may also be in communication with another RAN (not shown) employing a GSM , UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.

[0057] The CN 106/1 15 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 1 10, and/or the other networks 1 12. The PSTN 108 may include circuit- switched telephone networks that provide plain old telephone service (POTS). The internet 1 10 may include a global system of interconnected computer networks and devices that use common

communication protocols, such as the transmission control protocol (TCP), user datagram protocol (UDP) and/or the internet protocol ( IP) in the TCP/IP internet protocol suite. The networks 1 12 may include wired and/or wireless communications networks owned and/or operated by other service providers, For example, the networks 1 12 may include another CN connected to one or more RANs, which may employ the same RAT as the RAN 104/1 13 or a different RAT,

[0058] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communications system 100 may Include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links), For example, the WTRU 102c shown in FIG. 1 A may be configured to communicate with the base station 1 14a, which may employ a cellular-based radio technology, and with the base station 1 14b, which may employ an IEEE 802 radio technology,

[0059] FIG, 1 B is a system diagram illustrating an example WTRU 102. As shown in FIG. 1 B, the WTRU 102 may include a processor 1 18, a transceiver 120, a transmit/receive element 122, a

speaker/microphone 124, a keypad 126, a display/touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and/or other peripherals 138, among others, it will be appreciated that the WTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with an embodiment.

[0060] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like. The processor 118 may perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables the WTRU 102 to operate In a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit/receive element 122, While FIG. I B depicts the processor 1 18 and the transceiver 120 as separate components, it will be appreciated that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0061] The transmit/receive element 122 may be configured to transmit signals to, or receive signals from, a base station (e.g., the base station 1 14a) over the air Interface 116. For example, in one embodiment, the transmit/receive element 122 may be an antenna configured to transmit and/or receive RF signals. In an embodiment, the transmit/receive element 122 may be an emitter/detector configured to transmit and/or receive IR, UV, or visible light signals, for example. In yet another embodiment, the transmit/receive element 122 may be configured to transmit and/or receive both RF and light signals. It will be appreciated that the transmit/receive element 122 may be configured to transmit and/or receive any combination of wireless signals.

[0062] Although the transmit/receive element 122 is depicted in FIG. 1 B as a single element, the WTRU 102 may include any number of transmit/receive elements 122. More specifically, the WTRU 102 may employ MI O technology, Thus, In one embodiment, the WTRU 102 may include two or more transmit/receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air Interface 116.

[0063] The transceiver 120 may be configured to modulate the signals that are to be transmitted by the transmit/receive element 122 and to demodulate the signals that are received by the transmit/receive element 122. As noted above, the WTRU 102 may have multi-mode capabilities, Thus, the transceiver 120 may include multiple transceivers for enabling the WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11 , for example,

[0064] The processor 118 of the WTRU 102 may be coupled to, and may receive user input data from, the speaker/microphone 124, the keypad 126, and/or the display/touchpad 128 (e.g., a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit). T e processor 1 18 may also output user data to the speaker/microphone 124, the keypad 126, and/or the display/touchpad 128. in addition, the processor 1 18 may access information from, and store data in, any type of suitable memory, such as the non-removable memory 130 and/or the removable memory 132. The non-removable memory 130 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device, The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like, in other embodiments, the processor 118 may access information from, and store data in, memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).

[0065] The processor 118 may receive power from the power source 134, and may be configured to distribute and/or control the power to the other components In the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NIMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, and the like.

[0066] The processor 1 18 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102, in addition to, or in lieu of, the Information from the GPS chipset 136, the WTRU 102 may receive location information over the air interface 1 16 from a base station (e.g., base stations 1 14a, 1 14b) and/or determine its location based on the timing of the signals being received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by way of any suitable location- determination method while remaining consistent with an embodiment,

[0067] The processor 118 may further be coupled to other peripherals 138, which may include one or more software and/or hardware modules that provide additional features, functionality and/or wired or wireless connectivity. For example, the peripherals 138 may Include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs and/or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, a Virtual Reality and/or Augmented Reality (VR/AR) device, an activity tracker, and the like. The peripherals 138 may include one or more sensors, the sensors may be one or more of a gyroscope, an accelerometer, a hail effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor: a geoiocation sensor; an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and/or a humidity sensor.

[0068] The WTRU 102 may include a full duplex radio for which transmission and reception of some or all of the signals (e.g., associated with particular subframes for both the UL (e.g., for transmission) and downlink (e.g., for reception) may be concurrent and/or simultaneous. The full duplex radio may Include an interference management unit 139 to reduce and or substantially eliminate self-interference via either hardware (e.g., a choke) or signal processing via a processor (e.g., a separate processor (not shown) or via processor 1 18), In an embodiment, the WRTU 102 may include a half-duplex radio for which transmission and reception of some or all of the signals (e.g., associated with particular subframes for either the UL (e.g., for transmission) or the downlink (e.g., for reception)),

[0089] FIG, 1 C is a system diagram illustrating the RAN 104 and the CN 106 according to an

embodiment, As noted above, the RAN 104 may employ an E-UTRA radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116, The RAN 104 may also be in communication with the CN 106,

[0070] The RAN 104 may Include eNode-Bs 160a, 160b, 160c, though it will be appreciated that the IRAN 104 may include any number of eNode-Bs while remaining consistent with an embodiment. The eNode-Bs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the eNode-Bs 160a, 160b, 160c may implement MIMO technology, Thus, the eNode-B 160a, for example, may use multiple antennas to transmit wireless signals to, and/or receive wireless signals from, the WTRU 102a.

[0071] Each of the eNode-Bs 160a, 160b, 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and/or DL, and the like. As shown in FIG. 1 C, the eNode-Bs 160a, 160b, 160c may communicate with one another over an X2 Interface,

[0072] The CN 106 shown In FIG. 1 C may include a mobility management entity ( E) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (or PGW) 166. While each of the foregoing elements are depicted as part of the CN 106, It will be appreciated that any of these elements may be owned and/or operated by an entity other than the CN operator,

[0073] The MME 162 may be connected to each of the eNode-Bs 162a, 162b, 162c in the RAN 104 via an 81 interface and may serve as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer activation/deactivation, selecting a particular serving gateway during an Initial attach of the WTRUs 102a, 102b, 102c, and the like, The MME 162 may- provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM and/or WCDMA,

[0074] The SGW 164 may be connected to each of the eNode Bs 160a, 160b, 160c in the RAN 104 via the S1 interface. The SGW 164 may generally route and forward user data packets to/from the WTRUs 102a, 102b, 102c, The SGW 164 may perform other functions, such as anchoring user planes during inter- eNode B handovers, triggering paging when DL data is available for the WTRUs 102a, 102b, 102c, managing and storing contexts of the WTRUs 102a, 102b, 102c, and the like. [0075] The SGW 164 may be connected to the PGW 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the internet 1 10, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0078] The CN 106 may facilitate communications with other networks, For example, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the CN 106 may Include, or may communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to the other networks 1 12, which may include other wired and/or wireless networks that are owned and/or operated by other service providers

[0077] Although the WTRU is described in FIGS, 1A-1 D as a wireless terminal , it is contemplated that in certain representative embodiments that such a terminal may use (e.g.. temporarily or permanently) wired communication interfaces with the communication network.

[0078] In representative embodiments, the other network 1 12 may be a WLAN ,

[0079] A WLAN in Infrastructure Basic Service Set (BSS) mode may have an Access Point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may have an access or an interface to a Distribution System (DS) or another type of wired/wireless network that carries traffic in to and/or out of the BSS. Traffic to STAs that originates from outside the BSS may arrive through the AP and may be delivered to the STAs, Traffic originating from STAs to destinations outside the BSS may be sent to the AP to be delivered to respective destinations. Traffic between STAs within the BSS may be sent through the AP, tor example, where the source STA may send traffic to the AP and the AP may deliver the traffic to the destination STA. The traffic between STAs within a BSS may be considered and/or referred to as peer-to- peer traffic, The peer-to-peer traffic may be sent between (e.g., directly between) the source and destination STAs with a direct link setup (DLS). in certain representative embodiments, the DLS may use an 802.1 1 e DLS or an 802.1 1 z tunneled DLS (TDLS). A WLAN using an Independent BSS (IBSS) mode may not have an AP, and the STAs (e.g., all of the STAs) within or using the IBSS may communicate directly with each other, The IBSS mode of communication may sometimes be referred to herein as an "ad- hoc" mode of communication.

[0080] When using the 802.1 1 ac infrastructure mode of operation or a similar mode of operations, the AP may transmit a beacon on a fixed channel, such as a primary channel . The primary channel may be a fixed width (e.g., 20 MHz wide bandwidth) or a dynamically set width via signaling. The primary channel may be the operating channel of the BSS and may be used by the STAs to establish a connection with the AP. In certain representative embodiments, Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) may be implemented, for example in in 802.1 1 systems. For CSMA CA, the STAs {e.g., every STA), including the AP, may sense the primary channel. If the primary channel is sensed/detected and/or determined to be busy by a particular STA, the particular STA may back off. One STA {e.g., only one station) may transmit at any given time in a given BSS.

[0081] High Throughput (HT) STAs may use a 40 MHz wide channel for communication, for example, via a combination of the primary 20 MHz channel with an adjacent or nonadjacent 20 MHz channel to form a 40 MHz wide channel.

[0082] Very High Throughput (VHT) STAs may support 20MHz, 40 MHz, 80 MHz, and/or 160 MHz wide channels. The 40 MHz, and/or 80 MHz, channels may be formed by combining contiguous 20 MHz channels. A 160 MHz channel may be formed by combining 8 contiguous 20 MHz channels, or by combining two non-contiguous 80 MHz channels, which may be referred to as an 80+80 configuration. For the 80+80 configuration, the data, after channel encoding, may be passed through a segment parser that may divide the data into two streams, inverse Fast Fourier Transform (IFFT) processing, and time domain processing, may be done on each stream separately. The streams may be mapped on to the two 80 MHz channels, and the data may be transmitted by a transmitting STA. At the receiver of the receiving STA, the above described operation for the 80+80 configuration may be reversed, and the combined data may be sent to the Medium Access Control (MAC).

[0083] Sub 1 GHz modes of operation are supported by 802.1 1 af and 802.1 1 ah. The channel operating bandwidths, and carriers, are reduced in 802.11 af and 802.11 ah relative to those used in 802.1 1 η, and 802.1 l ac. 802.1 1 af supports 5 MHz. 10 MHz and 20 MHz bandwidths in the TV White Space (TVWS) spectrum, and 802.11 ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.1 1 ah may support Meter Type Control/Machine- Type Communications, such as MTC devices in a macro coverage area, MTC devices may have certain capabilities, for example, limited capabilities including support for {e.g., only support for) certain and/or limited bandwidths. The MTC devices may include a battery with a battery life above a threshold (e.g., to maintain a very long battery life)

[0084] WLAN systems, which may support multiple channels, and channel bandwidths, such as 802.11 n, 802.1 l ac, 802. 1 af, and 802.11 ah, include a channel which may be designated as the primary channel. The primary channel may have a bandwidth equal to the largest common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be set and/or limited by a STA, from among all STAs in operating in a BSS, which supports the smallest bandwidth operating mode. In the example of 802.11 ah, the primary channel may be 1 MHz wide for STAs (e.g., MTC type devices) that support {e.g., only support) a 1 MHz mode, even if the AP, and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and/or other channel bandwidth operating modes. Carrier sensing and/or Network Allocation Vector (NAV) settings may depend on the status of the primary channel . If the primary channel is busy, for example, due to a STA (which supports only a 1 MHz operating mode), transmitting to the AP, the entire available frequency bands may be considered busy even though a majority of the frequency bands remains idle and may be available.

[0085] in the United States, the available frequency bands, which may be used by 802.1 1 ah, are from 902 MHz to 928 MHz. In Korea, the available frequency bands are from 917.5 MHz to 923.5 MHz. In Japan, the available frequency bands are from 916,5 M Hz to 927,5 MHz. The total bandwidth available for 802, 1 1 ah is 6 MHz to 26 MHz depending on the country code

[0086] FIG. 1 D is a system diagram illustrating the RAN 1 13 and the CN 1 15 according to an embodiment. As noted above, the RAN 1 13 may employ an NR radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 1 16, The RAN 1 13 may also be in communication with the CN 1 15.

[0087] The RAN 1 13 may include gNBs 180a, 180b, 180c, though it will be appreciated that the RAN 1 13 may include any number of gNBs while remaining consistent with an embodiment. The gNBs 180a, 180b, 180c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 1 16. In one embodiment, the gNBs 180a, 180b, 180c may implement M IMO technology, For example, gNBs 180a, 108b may utilize beamforming to transmit signals to and/or receive signals from the gNBs 180a, 180b, 180c, Thus, the gNB 180a, for example, may use multiple antennas to transmit wireless signals to, and/or receive wireless signals from, the WTRU 102a. In an embodiment, the gNBs 180a, 180b, 180c may Implement carrier aggregation technology, For example, the gNB 180a may transmit multiple component carriers to the WTRU 102a (not shown). A subset of these component carriers may be on unlicensed spectrum while the remaining component carriers may be on licensed spectrum, in an embodiment, the gNBs 180a, 180b, 180c may implement Coordinated Multi-Point (CoMP) technology, For example, VVTRU 102a may receive coordinated transmissions from gNB 180a and gNB 180b (and/or gNB 180c).

[0088] The WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using transmissions associated with a scalable numerology. For example, the OFDM symbol spacing and/or OFDM subcarrier spacing may vary for different transmissions, different cells, and/or different portions of the wireless transmission spectrum, The WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using subframe or transmission time intervals (XT Is) of various or scalable lengths (e.g., containing varying number of OFDM symbols and/or lasting varying lengths of absolute time),

[0089] The gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and/or a non-standalone configuration, In the standalone configuration, WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c without also accessing other RANs (e.g., such as eNode-Bs 160a, 160b, 160c). In the standalone configuration, W'TRUs 102a, 102b, 102c may utilize one or more of gNBs 180a, 180b, 180c as a mobility anchor point, In the standalone configuration, WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using signals in an unlicensed band, In a non-standalone configuration WTRUs 102a, 102b, 102c may communicate with/connect to gNBs 180a, 180b, 180c while also communicating with/connecting to another RAN such as eNode-Bs 160a, 160b, 160c. For example, WTRUs 102a, 102b, 102c may implement DC principles to communicate with one or more gNBs 180a, 180b, 180c and one or more eNode-Bs 160a, 160b, 160c substantially simultaneously, In the non-standalone configuration, eNode-Bs 160a, 160b, 160c may serve as a mobility anchor for WTRUs 102a, 102b, 102c and gNBs 180a, 180b, 180c may provide additional coverage and/or throughput tor servicing WTRUs 102a, 102b, 102c.

[0090] Each of the gNBs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users In the UL and/or DL, support of network slicing, dual connectivity, interworking between NR and E-UTRA, routing of user plane data towards User Plane Function (UPF) 184a, 184b, routing of control plane information towards Access and Mobility Management Function (AMF) 182a, 182b and the like, As shown in FIG, 1 D, the gNBs 180a, 180b, 180c may communicate with one another over an Xn interface.

[0091] The CN 115 shown in FIG, I D may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and possibly a Data Network (DN) 185a, 185b. While each of the foregoing elements are depicted as part of the CN 115, it will be appreciated that any of these elements may be owned and/or operated by an entity other than the CN operator.

[0092] The AMF 182a, 182b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 1 13 via an N2 Interface and may serve as a control node. For example, the AMF 182a, 182b may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, support for network slicing (e.g., handling of different PDU sessions with different requirements), selecting a particular SMF 183a, 183b, management of the registration area, termination of NAS signaling, mobility management, and the like. Network slicing may be used by the AMF 182a, 182b in order to customize CN support for WTRUs 102a, 102b, 102c based on the types of services being utilized WTRUs 102a, 102b, 102c. For example, different network slices may be established for different use cases such as services relying on ultra-reliable low latency (URLLC) access, services relying on enhanced massive mobile broadband (eMBB) access, services for machine type communication (MTC) access, and/or the like, The AMF 162 may provide a control plane function for switching between the RAN 1 13 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A, LTE-A Pro, and/or non-3GPP access technologies such as WiFi. [0093] The SMF 183a, 183b may be connected to an AMF 182a, 182b in the CN 1 15 via an N11 interface. The SMF 183a, 183b may also be connected to a UPF 184a, 184b in the CN 1 15 via an N4 interface. The SMF 183a, 183b may select and control the UPF 184a, 184b and configure the routing of traffic through the UPF 184a, 184b, The SMF 183a, 183b may perform other functions, such as managing and allocating UE IP address, managing PDU sessions, controlling policy enforcement and QoS, providing downlink data notifications, and the like, A PDU session type may be IP-based, non-IP based, Ethernet-based, and the like.

[0094] The UPF 184a, 184b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 1 13 via an N3 interface, which may provide the VVTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 1 10, to facilitate communications between the VVTRUs 102a, 102b, 102c and IP-enabled devices. The UPF 184, 184b may perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, providing mobility anchoring, and the like.

[0095] The CN 115 may facilitate communications with other networks. For example, the CN 1 15 may include, or may communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an Interface between the CN 115 and the PSTN 108. In addition, the CN 115 may provide the WTRUs 102a, 102b, 102c with access to the other networks 1 12, which may include other wired and/or wireless networks that are owned and/or operated by other service providers. In one embodiment, the WTRUs 102a, 102b, 102c may be connected to a local Data Network (DN) 185a, 185b through the UPF 184a, 184b via the N3 interface to the UPF 184a, 184b and an N6 interface between the UPF 184a, 184b and the DN 185a, 185b.

[0096] In view of FIGs. 1A-1 D, and the corresponding description of FIGs. 1A-1 D, one or more, or all, of the functions described herein with regard to one or more of: WTRU 102a-d, Base Station 1 14a-b, eNode-B 160a-c, MME 162, SGW 164, PGW 166, gNB 180a-c, AMF 182a-ab, UPF 184a-b, SMF 183a-b, DN 185a- b, and/or any other device(s) described herein, may be performed by one or more emulation devices (not shown). The emulation devices may be one or more devices configured to emulate one or more, or all, of the functions described herein. For example, the emulation devices may be used to test other devices and/or to simulate network and/or WTRU functions,

[0097] The emulation devices may be designed to implement one or more tests of other devices in a lab environment and/or in an operator network environment. For example, the one or more emulation devices may perform the one or more, or all, functions while being fully or partially implemented and/or deployed as part of a wired and/or wireless communication network in order to test other devices within the

communication network, The one or more emulation devices may perform the one or more, or all, functions while being temporarily Implemented/deployed as part of a wired and/or wireless communication network. The emulation device may be directly coupled to another device for purposes of testing and/or may performing testing using over-the-air wireless communications.

[0098] The one or more emulation devices may perform the one or more, including all, functions while not being implemented/deployed as part of a wired and/or wireless communication network. For example, the emulation devices may be utilized in a testing scenario in a testing laboratory and/or a non-deployed (e.g., testing) wired and/or wireless communication network in order to implement testing of one or more components. The one or more emulation devices may be test equipment. Direct RF coupling and/or wireless communications via RF circuitry (e.g., which may Include one or more antennas) may be used by the emulation devices to transmit and/or receive data.

[0099] Video coding systems may compress digital video signals, for example, to reduce storage and/or transmission bandwidth. There are a variety of types of video coding systems, such as block-based, wavelet-based and object-based systems. Block-based video coding systems may include international video coding standards, such as (Moving Picture Experts Group) MPEG- 1/2/4 part 2, H.264/MPEG-4 part 10 Advanced Video Coding (AVC), V'C-1 and High Efficiency Video Coding (HEVC).

[0100] HEVC may, for example, provide an approximately 50% bit-rate saving or equivalent perceptual quality compared to a prior generation of video coding (e.g., H.264/MPEG AVC). Superior coding efficiency may be achieved (e.g., with additional coding tools) over HEVC. VCEG and MPEG (e.g., Joint Video Exploration Team (JV'ET)) may support coding technologies that may provide substantial enhancement of coding efficiency over HEVC, A software codebase called Joint Exploration Model (JEM) may be based on an HEVC Model (HM). Coding tools may be integrated into JEM software and may be tested, for example, using JVET common test conditions (CTCs).

[0101] JEM software may be similar to HM . JEM software may be based on a block-based hybrid video coding framework.

[0102] FIG, 2 shows an example of a block-based hybrid video encoding system, input video signal 302 may be processed block by block. Extended block sizes (e.g., a coding unit (CU)) may be used (e.g., in HEVC), for example, to efficiently compress high resolution (e.g., 1080p and beyond) video signals. A CU may be, for example, 64x64 pixels. A CU may be partitioned (e.g., Into prediction units (PU)), Separate (e.g., the same or different) prediction procedures may be applied to PUs, Spatial prediction 360 and/or temporal prediction 362 may be performed, for example, for an (e.g., each) input video block (e.g., macroblock (MB) or CU)).

[0103] Spatial prediction (e.g., intra prediction) may predict a current video block, for example, by using pixels from samples of already coded neighboring blocks (e.g., reference samples) in a (e.g., the same) video picture/slice. Spatial prediction may reduce spatial redundancy that may be inherent in a video signal. Temporal prediction (e.g., inter prediction or motion compensated prediction) may predict a current video block, for example, by using reconstructed pixels from already coded video pictures. Temporal prediction may reduce temporal redundancy that may be inherent in a video signal. A temporal prediction signal for a given video block may be signaled , for example, by one or more motion vectors (MVs), which may indicate an amount and direction of motion between a current block and a reference block. A reference picture index (e.g., for each video block) may be sent, for example, when multiple reference pictures may be supported (e.g., for H .264/AVC or HEVC). A reference index may be used to identify a reference picture [e.g.. in reference picture store 364) that a temporal prediction signal comes from .

[0104] Mode decision block 380 in an encoder may [e.g., after spatial and/or temporal prediction) choose a (e.g._., the best) prediction mode based on, for example, a rate-distortion optimization procedure. A prediction block may be subtracted from a current video block 316. A prediction residua! may be de- correlated (e.g., using transform 304) and quantized 306. Quantized residual coefficients may be inverse quantized 310 and inverse transformed 312, e.g.. to form a reconstructed residual, A reconstructed residual may be added back to prediction block 326, for example, to form a reconstructed video block. In- loop filtering 366 (e.g., de-blocking filter and/or Adaptive Loop Filters) may be applied to a reconstructed video block, for example, before it is put in reference picture store 364 and used to code future video blocks, Entropy coding unit 308 may for output video bit-stream 320, for example, by compressing and packing coding mode (e.g., inter or intra), prediction mode information, motion information and/or quantized residual coefficients. The encoder may include a WTRU (e.g. , as described herein), a processor of a WTRU, and/or the like.

[0105] FIG. 3 shows an example of a block-based video decoder. Video bit-stream 202 may be unpacked and entropy decoded at entropy decoding unit 208. Coding mode and prediction information may be sent to the spatial prediction unit 260 (e.g., if intra coded) or temporal prediction unit 262 (e.g., if inter coded), for example, to form a prediction block. Residual transform coefficients may be provided to inverse quantization unit 210 and inverse transform unit 212, e.g., to reconstruct the residua! block, A prediction block and a residua! block may be added together, e.g., at summation 226, !n-!oop filtering may be applied to a reconstructed block, for example, before it is stored in reference picture store 264. Reconstructed video in reference picture store 264 may be sent out, for example, to drive a display device and/or to predict future video blocks, The decoder may include a WTRU (e.g. , as described herein), a processor of a WTRU, and/or the like.

[0106] Motion information (e.g., MVs and reference picture indices) may be determined by an encoder and (e.g., explicitly) transmitted to a decoder. A significant amount of overhead may be spent on coding motion parameters for inter-coded blocks, Overhead of signaling motion information may be reduced, for example, by a coding mode (e.g., FRUC), The signaling of MVs and/or reference picture indices may be skipped, for example, when FRUC coding mode Is enabled for a CU. information may be derived (e.g., on the decoder side}, for example, by template-matching or bilateral-matching techniques.

[0107] FIG. 4 shows an example of frame-rate up conversion (FRUC). FIG. 4A shows an example of template-matching. FIG. 4B shows an example of bilateral-matching . FIG. 4 illustrates an example of an MV derivation process that may be used in FRUC. Template-matching (e.g., as shown in FIG. 4A) may be used to derive an MV of a current CU, for example, by finding a match (e.g., the best match) between a template (e.g., top and/or left neighboring blocks) of a current CU in a current picture and a template of a reference block in a reference picture. The best match may be. for example, an MV that may achieve the low matching cost (e.g., the lowest Sum of Absolute Difference (SAD)) between a current template and a reference template. Bilateral-matching (e.g., as shown in FIG. 4B) may be used to derive motion information of a current block, for example, by finding the best match between two blocks along a motion trajectory of a current block in two different reference pictures. A motion search process of bilateral- matching may be based on motion trajectory. In an example, motion vectors MVo and MVi pointing to two reference blocks may be proportional to temporal distances between a current picture and each of two reference pictures (e.g., To and TV). Motion vector candidates may be examined to determine motion vectors MVo and MVi that may be used in bilateral FRUC mode. A motion search may be performed (e.g., for each candidate), for example, using list-0 motion vectors and iist-1 motion vectors (e.g., separately). Motion vectors with the lowest matching cost (e.g., SAD) may be selected . Whether to use a template- matching mode or bilateral-matching mode may be determined based on , for example, rate-distortion (R-D) cost optimization . A mode leading to minimal rate-distortion cost may be selected as a FRUC mode for a current CU .

[0108] F IG. 5 shows an example of a motion derivation in a FRUC mode. Motion derivation processes for a template-matching mode and a bilateral-matching mode may include one or more of CU-leve! motion search, CU-level motion refinement, sub-CU-level motion search, and/or sub-CU-level motion refinement. An initial motion (e.g., including MVs and reference picture indices) for a CU-level motion search may be derived for a whole CU, for example, based on template-matching or bilateral-matching. In an example, a list of MV candidates may be generated. A candidate leading to a minimum matching cost may be selected as a starting MV for the CU , A local search (e.g., based on template-matching or bilateral-matching) around the starting point may be performed (e.g., at a CU-leve! motion refinement stage). An MV with a minimum matching cost may be taken as an MV for a whole CU . Granularity and accuracy of motion information may be (e.g., further) refined at a sub-CU-level motion search stage and a sub-CU-level motion refinement stage, for example, by dividing a current CU into multiple sub-CUs and deriving motion information of a (e.g., each) sub-CU with the MV derived from the CU-level as the starting search point. [0109] A CU-level motion search may be performed. A candidate-based search may be performed to derive an initial MV of a current CU , A set of unique MV candidates may be selected, for example, based on a calculated cost measurement of template-matching or bilateral-matching (e.g., depending on a FRUC mode that may be applied to a current CU), An MV candidate that may minimize the cost may be selected as an initial MV of a whole CU .

[0110] In an example, one or more of the following MV candidates may be evaluated at a CU-level motion search: MV predictors that may be derived from advanced motion vector prediction (AMVP) (e.g., when FRUC may be applied to predict MV(s) of a current CU in AMVP mode); MVs of regular merge candidates (e.g., including five spatial candidates A1 , B1 , B0, AO and B2 (e.g., as shown in FIG. 6)) and a temporal candidate that may be derived from an MV of a collocated block in a temporal neighboring picture (e.g., by temporal motion vector prediction (TMVP)); four interpolated MVs at position (0, 0), (W/2, 0), (0, H/2) and (W/2, H/2) that may be generated by a picture-level MV field interpolation process, where W and H may be width and height of a current CU; and/or MVs of the top and left neighbors of a current CU .

[0111 ] F IG. 6 shows example positions of spatial candidates in a merge mode.

[0112] A pruning operation may be performed, for example, when generating an MV candidate list at a CU-level . An MV candidate may not be added into an MV candidate list, for example, when it may be redundant (e.g., the MV candidate may have similar or the same motion as that of an MV candidate in the list). An MV of a (e.g., each) valid candidate may be used as an input to generate an MV pair (e.g., in a bilateral-matching mode), for example, when two MVs may be on the same motion trajectory as a current CU . In an example, a valid MV may be (MVa, refa) at reference list a (a = 0, 1). Reference picture refa of its paired bilateral MV may be found in another reference list b, for example, where refa and refa may be temporally symmetric with regard to a current picture. In an example, (e.g., when there may not be a reference picture refa In reference list B thai is symmetric to refa), refa may be determined, for example, by selecting a reference picture that may be different from refa and its temporal distance to a current picture may be minimized in list B. MVb may be derived, for example, by scaling MV_a (e.g., according to temporal distances between a current picture and refa and refa), for example, in accordance with Eq. 1 :

MV* _ MV* _ τ_α

MVS MV* r_b Eq . 1 where (MV£ , MV*) and (MV£, MV^) may be horizontal and vertical motion components of MVa and MVf, r_a and r_b may be temporal distances of reference pictures refa and refa to a current picture, in an example, up to 15 MVs may be included at a CU-level motion search stage. [0113] A sub-CU-level motion search may be performed. A corresponding MV may be derived at a CU- ieve! motion search. A derived MV may be used to represent the motion of a whole CU. Granularity may be insufficient, for example, given that a current CU may cover areas corresponding to different objects that may be associated with distinct motions. Precision of derived MVs may be improved, for example, by (e.g., further) splitting a [e.g., each) CU that may be coded by a FRUC mode (e.g., template-matching or bilateral-matching) into M * M sub-CUs. An individual MV for a {e.g., each) sub-CU may be derived, e.g., separately. A value of M may be calculated, for example, in accordance with Eq. 2;

where D may be a predefined splitting depth constraint (e.g., may be set to 3).

[0114] An MV of a sub-CU may be derived (e.g., similar to a CU-!eve! motion search), for example, by generating a list of MV candidates and selecting an MV candidate that may minimize a matching cost as an MV of a sub-CU, In an example, an MV candidate list at a sub-CU level may include one or more of an MV determined at a CU-level motion search; MVs from top, left, top-left and top-right spatial neighboring CUs; a scaled version of collated MVs from temporal reference pictures; up to 4 MVs that may be obtained from candidates derived by advanced temporal motion vector prediction (ATMVP); and/or (v) up to 4 MVs that may be obtained from candidates derived by spatial-temporal motion vector prediction (STMVP), [0115] During the sub-CU motion search, an MV candidate (e.g., one specific MV candidate) may be included in an MV candidate list, for example [e.g., only) when the MV candidate may point to the same reference picture indicated by a starting CU-!evel MV. An MV candidate may, e.g., otherwise, be excluded from a matching cost calculation, For example, the MVs of the sub-CUs inside a CU may point to the same reference pictures (e.g., associated with a starting CU-level MV). In an example, (e.g., only) those reference pictures may be accessed to find optimal MVs at a sub-CU-level. In an example, up to 17 MVs may be included in a sub-CU-level motion search stage.

[0118] Picture-level MV field interpolation may be performed. An (e.g., one) interpolated motion field may be generated for multiple (e.g., ail) 4x4 blocks in a picture (e.g., based on unilateral MV projection), for example, when FRUC is enabled and before each inter picture is coded, interpolated MVs may be used to search an optimal MV at a CU-level motion search.

[0117] FIG. 7 shows an example of motion field interpolation that may be used in FRUC. FIG. 7 shows an example of a process of generating interpolated MVs In a current picture, In an example, a motion field of a (e.g., each) reference picture (e.g., curRefPic) in one or more reference picture lists of a current picture (e.g., curPic) may be examined (e.g., at a 4x4 block level). An MV of a 4x4 reference block in the reference picture (e.g., when the MV passes through a 4x4 block In the current block, and the 4x4 block has not been assigned with an interpolated MV) may be scaled to a current picture, for example, according to a ratio between a temporal distance between the current picture and a reference picture [e.g., TDo) and a temporal distance between the reference picture and its own reference picture (e.g., TDi). A scaled MV may be assigned to a 4x4 block in a current picture. Motion for a block may be marked as unavailable In an interpolated motion field, for example, when a scaled MV is not assigned to a 4x4 b!ock,

[0118] CU-ievel and sub-CU-level motion refinement may be performed . There may be a limited number of searching candidates. MVs derived from a CU-level motion search and sub-CU-level motion search may not be accurate enough to represent the true motion of a current CU . Precision of derived MVs may be improved, for example, by applying MV refinement processes (e.g., after a CU-ievel motion search and a sub-CU-level motion search). MV refinement may comprise a pattern-based !ocai MV search process that minimizes the cost measurement of template-matching or bilateral-matching. In an example, two search patterns may be used, e.g., a diamond search pattern at a CU-level motion refinement and a cross search pattern at a sub-CU-level motion refinement. An MV search and local motion refinement may be performed for a CU-ievel motion refinement and for a sub-CU-leve! motion refinement. An MV search may be performed (e.g., at a quarter-sample accuracy), A local motion refinement (e.g., around the best quarter- sample MV) may be performed (e.g., at one-eighth-sample accuracy). A bi-linear interpolation filter may be used for template-matching and bilateral-matching modes, for example, when an MV may point to a fractional sample position. A search process (e.g., in CU-ievel motion refinement and sub-CU-leve! motion refinement) may be unrestricted, For example, a refinement search may be repeated until a search center may not be updated in a predefined search range, in an example, a search range may be set to 8 integer !u a samples.

[0119] Local illumination compensation (L !C) may be used to address local illumination changes (e.g., when the illumination variation is non-linear). A pair of weight and offset may be applied to a reference block to obtain a prediction block, for example, using Eq . 2A.

Pj ] = a * P_r[x + v] + β Eq. 2A

[0120] Parameter Prjx + v] may indicate the reference block that is pointed to by motion vector v, Parameters [α, β] may be the corresponding pair of weight and offset for the reference block, and parameter P[x] may be the final prediction result block. The weight and offset pair may be estimated by, for example, the least linear mean square error (LLMSE). The LLMSE may utilize the template of the current block and the template of the reference block, which may be designated by the motion vector of the current block. By minimizing the mean square difference between the template of the reference and that of the current block, Eq. 2B and 2C may derive a and β, respectively. a = -----— j —— Eq. 2B

'·∑'... U ) c X;... ;% x; r

[0121] Parameter / may represent the number of samples in the template. Parameter P_c[xi] may be the ith sample of the current block's template, and parameter Ρψο] may be the ith sample of the reference template that the corresponding motion vector may be pointed to.

[0122] To apply L!C on bi-directional prediction, the estimation process of the weight and offset pair may be applied (e.g., applied once) on each of the two reference blocks. FIG. 8 illustrates an example template, Tc, and corresponding prediction blocks' templates, To and Ti, in a current picture and reference pictures, Using motion vectors vo and vi, the template To and Ti may be fetched. By minimizing Illumination difference separately between the two pairs of templates Tc and To, Tc, and h. the corresponding two pairs of weight and offset may be derived, The two prediction blocks from the two different directions may be combined. The LiC bi-directional prediction may be provided based on, for example, Eq. 2D.

jx] = ~ (ao * Pojx + vo] + βο + ai * Pijx + vi] + βι) Eq. 2[

[0123] Parameters [αο, βο] and [αι, βι] may be the two weight pairs, and parameters vo and vi may be the corresponding motion vectors for each of the reference block.

[0124] Temporal prediction configurations {e.g., two temporal prediction configurations) may be performed. The temporal prediction configurations may evaluate the performance of different inter coding tools (e.g., comprising low-delay and random access).

[0125] A Low-delay configuration may not have a structural delay between a coding order and/or a display order {e.g., the coding order may always equal to the display order). The low-delay setting may provide for conversionai applications with a low-delay requirement, Coding configurations may be defined for a low- delay setting.

[0126] One or more of the following may apply to the coding configurations. A picture in a video sequence may be coded as an ! picture {e.g., an inter-coded picture using spatial prediction). Other pictures may be coded, for example, using {e.g., only) uni-prediction [e.g.. low-delay P (LDP) configuration) or using {e.g., only) bi-prediction (e.g., low-delay B (LDB) configuration), In LDP and/or LDB configurations, a picture {e.g., each picture) may use (e.g., may only use) reference pictures that precede the current picture in the display order, For example, the picture order counts (POCs) of the reference pictures in reference picture list L0 and LI (e.g., if LDB is applied) are smaller than that of the current picture,

[0127] FIG, 9 shows an example of low-delay configurations. As shown In FIG. 9, a group of pictures (GOP) may Include a key picture (e.g., the pictures in patterned blocks in FIG, 9) and the pictures that are temporally located in-between (e.g., two) key pictures (e.g., the pictures in blank blocks in FIG. 9). As seen in FIG, 9, the GOP size may be equal to 4. As seen in FIG. 9, four previous pictures may be used for the motion-compensated prediction of a current picture {e.g., each current picture). As seen in FIG. 9, the 4 pictures may comprise one immediately previous picture, which is closest to the current picture, and three previous key pictures, As seen in F!G, 9, a picture (e.g.. Pic11) may be predicted from other pictures (e.g., PidO, Pic8, Pic4 and PicO). it can be seen from FIG. 9 that each picture In a GOP may have different impacts on the overall coding efficiency. For example, because key pictures may be frequently used as a reference (e.g., to predict the other pictures), the coding distortion of a key picture (e.g., each key picture) may determine its own coding performance and/or may propagate [e.g., by temporal prediction) into the following pictures that refer to the key picture, Varying quantization parameter (QP) values may be assigned to different pictures in a GOP (e.g., as illustrated in FIG. 9). Smaller QPs may be used to code pictures at lower temporal layers (e.g., the key pictures) among the pictures in GOP. The pictures at lower temporal layers may have better reconstruction quality over pictures at high temporal layers.

[0128] In a random-access configuration, a hierarchical B structure may be applied. The coding efficiency achieved by bi-directlon hierarchical prediction may be improved. Random-access configurations may result in a de lay for displays, which may be due to the coding order and the display order of decoupled pictures In random-access.

[0129] As seen in FIG, 10, a hierarchical prediction structure may comprise a 4 dyadic hierarchy. A video sequence (e.g., one video sequence) may be divided into multiple GOPs in random-access. The GOPs may contain a key picture (e.g., one key picture) and/or pictures (e.g., ail the pictures) that are temporally located between two (e.g., consecutive) key pictures. The key pictures may be intra-coded (e.g., in order to enable random access) and/or inter-coded. When a key picture is Inter-coded, it may be (e.g., may only be) predicted using reference pictures (e.g., previously decoded reference pictures). The POCs of an inter key picture's (e.g., each inter key picture) reference pictures may be smaller than the POC of the key picture. As discussed herein, key pictures in random-access configurations may referred to, interchangeably, as low-delay pictures,

[0130] The pictures (e.g., the remaining pictures) in a GOP may be coded after a key picture is coded, The coding may be based on hierarchical prediction, for example, by defining different temporal layers. As shown in FIG. 10, the second and third temporal layers may include bi-predicted pictures. The bi-predicted pictures may predict the pictures at higher layers. The highest layer may contain non-referenced bi- predicted pictures (e.g., only non-referenced bi-predicted pictures). The non-referenced bi-predicted pictures may not be used to predict other pictures. Low-delay configurations may be based on aspects (e.g., the temporal layer) of pictures at a (e.g., each) GOP. Unequal QPs may be applied, for example, to code pictures in a GOP (e.g., different pictures in each GOP), Smaller QPs may be used for pictures at lower temporal layers (e.g., for better prediction quality) and higher QPs may be used for pictures at higher temporal layers (e.g., for larger bit-rate savings), [0131 ] Motion information (e.g., MVs) of a CU may be derived without signaling, for example, when a CU may be coded by a FRUC mode, Derived MVs may be refined at a sub-CU-level, which may improve granularity and accuracy of MVs that may be used in a motion compensation stage.

[0132] An optimal MV of a (e.g., one) CU/sub-CU may be derived from a set of MV candidates in a CU- level motion search and/or a sub-CU-level motion search. In an example, there may be (e.g.. up to) 15 and 17 MV candidates thai may be involved in a CU-!evel motion search and a sub-CU-level motion search , respectively. Motion compensated prediction may be performed multiple times to generate a prediction signal for an MV candidate. Reference samples may be fetched from temporal reference pictures to calculate a cost measurement. An encoder/decoder may frequently switch memory access to different reference pictures for a CU-level motion search, where MV candidates in a candidate list may come from different reference pictures.

[0133] MV refinement may be performed after CU-level and sub-CU-level motion search processes, for example, to improve the accuracy of a derived MV, For example, a pattern-based local search may be performed around a starting point. A local MV search (e.g., at CU-level and sub-CU-level) may be conducted, for example, by searching at quarter-sample accuracy In a recursive manner and by (e.g., additionally) searching at one-eighth-sample accuracy. Searching MVs may point to reference blocks that may be located at fractional sample positions in reference pictures. An interpolation filter (e.g., bi -iinear filter) may be applied, for example, to generate fractional !uma/chroma reference samples. Addition and/or multiplication operations may be used in interpolation filtering, Interpolation may need to access additional samples from an extended area of a current CU In a reference picture for a filtering process.

[0134] Sub-CU-level motion derivation (e.g. , including sub-CU-level motion search and sub-CU-level motion refinement) may compensate for different motions that may exist in sub-blocks inside a (e.g., one) CU . Sub-CU-level motion derivation (e.g., in FRUC) may be performed after a CU-level MV is derived, For example, when a CU may contain steady motion or when motions of sub-blocks inside a CU are consistent, the sub-CU-level motion derivation process in FRUC mode may be skipped .

[0135] The weight and offset of LIC may be separately derived from the template for a prediction using a least square error method, For an FRUC bilateral mode, the LIC parameters may be derived for the sub- CUs and for the prediction directions. For example, the LIC parameters may be derived for each of the sub-CUs and for each of the prediction directions. For FRUC bi-iaterai mode, the motion vectors may be derived based on reference blocks, and the two prediction blocks that are similar to each other may be identified ,

[0136] An initial FRUC MV may be derived at a CU-level and/or a sub-CU-level, for example, via a clustering-based motion search , in an example, MV candidates in an MV candidate list of a current CU/sub-CU may be clustered into several MV candidate sets. An MV (e.g., a major or a representative V) may be determined for a (e.g., each) MV candidate set for a cost measurement calculation. The tota! number of cost calculations for MV candidates may be reduced, for example, compared to calculating a cost measurement (e.g., based on template-matching or bilateral-matching) for an MV candidate in the MV candidate list for a CU-!eve! motion search and a sub-CU-level motion search.

[0137] CU-level motion refinement and sub-CU-level motion refinement in FRUC may be performed. A multi-stage searching may be performed, for example, to find an optimal local MV, An MV may be searched at integer-sample accuracy. An integer MV (e.g., the best integer MV) that minimizes cost may be refined at fractional sample accuracy.

[0138] In an example, early termination may be provided for a sub-CU-level motion derivation. One or more conditions may be provided for early termination.

[0139] FRUC bi-lateral mode motion estimation may be skipped based on whether LIC is enabled for the current CU. For example, whether to perform FRUC bi-lateral mode motion estimation may be determined based on an LIC flag associated with the current CU. if the LIC flag associated with the current CU is set to true, FRUC bi-lateral mode motion estimation may be skipped. LIC signalling may be skipped for FRUC bilateral CUs. The decoder may omit parsing the LIC parameters for a CU on a condition that the CU is a FRUC bi-lateral CU. LIC process may be skipped for FRUC bi-lateral CUs,

[0140] CU/Sub-CU-level motion search may be performed using one or more search MVs. For example, the search MVs may be derived from multiple MV candidates based on MV clustering. The number of search MVs may be less than the number of the MV candidates included in the MV candidate list(s). In an example FRUC implementation, up to 15 and 17 MV candidates may be included in MV candidate lists for a CU-level motion search and/or sub-CU-level motion search, respectively, An MV candidate in an MV candidate list may be associated with a corresponding reference picture, for example, at a CU-level motion search stage, Different MV candidates may be associated with different reference pictures,

[0141] For example, MV grouping may be performed for motion search processes (e.g., at a CU-level and/or a sub-CU-level). In an example, MV grouping may include: reference picture selection, MV scaling and pruning, and/or MV clustering. MV grouping may be performed at a CU-level and/or a sub-CU level.

[0142] As discussed herein, at a CU-level motion search, MV candidates (e.g., up to four MV candidates) may be generated via a picture-level MV field interpolation process. The MV candidates may be part of an initial candidate list. The MV candidates may be used to derive an optimal MV for the current CU. As described herein, the interpolated motion field may be generated tor a block (e.g., each 4x4 block) In the current picture based on a MV projection method (e.g., one unilateral MV projection method). An MV derivation process may be performed for each 4x4 unit within a reference picture (e.g., every reference picture) of the current picture. Checking of MV availability and/or the MV scaling may be performed. [0143] Interpolated MVs may or may not be accurate (e.g., accurate enough to provide reliable candidates for motion search), invocation [e.g., frequent invocation) of MV scaling operations may occur when the temporal distance between the current picture and the reference picture {e.g.. TDO in FiG, 7), and/or the temporal distance between the reference picture and its own reference picture (e.g. , TD1 in FIG. 7) is unequal, Rounding operations (e.g., during MV scaling) may introduce rounding errors (e.g., additional rounding errors), interpolated MVs may be derived at a granularity of 4x4 blocks. Depending on the size of the current CU (e.g., especially for large CU sizes), a derived MV and a true MV of the current CU may be less correlated (e.g., not correlated). A derived MV and a true MV may be derived at different granularities.

[0144] A CU-ievei MV candidate list may be generated such that interpolated MVs may be removed from the initial MV candidate list. The interpolated MVs may be generated by a picture-level MV field interpolation . The initial MV candidates that are evaluated for a CU-ievel motion search may include one or more of the following : the MV predictors that are derived from advanced motion vector prediction when FRUC is applied (e.g., to predict the MV(s) of the current CU in AMVP mode), the MVs of the regular merge candidates (e.g., the five spatial candidates shown In FIG. 6, A1 , B1 , BO, AO and B2), the MVs of a temporal candidate (e.g., derived from the MVs of a collocated block in the temporal neighboring picture by TMVP), and/or the MVs of the top and left neighbors of the current CU ,

[0145] Reference picture selection may be performed , A reference picture (e.g., in a given reference picture list) tor the CU-!eve! (and/or sub-CU-!evei motion search) may be selected. For example, the reference picture for the CU-ievel and/or sub-CU-!evel motion search may be selected based on reference picture indices of corresponding MV candidates in an MV candidate list, The MV candidate list may include an original candidate list,

[0146] In an example, a reference picture that is most frequently referred by the MV candidates (e.g., the MV candidates In the MV candidate list for a CU-leve! motion search and/or sub-CU-!evei motion search) may be identified as a representative reference picture for the MV candidates, A majority rule may be applied to select a reference picture index for a given (e.g., representative) reference picture that may have been used most frequently by MV candidates in the MV candidate list, For example, there may be MV candidates and M reference pictures in a reference picture list LX (X = 0, f). The K MV candidates may be associated with a collection of reference pictures with indices ro, n, rx-i in the LX, where η e

[0, M— 1] , Each MV candidate of the K MV candidates may be associated with (e.g., refer to) a reference picture of the M reference pictures. A selected reference picture index may be determined, for example, in accordance with Eq. 3:

where 1_η(τ_έ) may be an indicator function that may be defined, for example, in accordance with Eq. 4;

[0147] To illustrate, there may be three reference pictures (e.g.. M = 3) and five MV candidates in an MV candidate list (e.g., = 5) in a reference picture list for a current picture. The five MV candidates may be respectively directed to reference pictures with indices of {0, 1 , 1 , 1 , 2} (e.g.. {/¾ n, set equal to {0, 1 , 1 , 1 , 2}}, A selected reference picture index may be set to 1 , for example, in accordance with Eq. 3. A tie may be broken when calculating r^* in Eq. 3, for example, using one or more factors. In an example, a reference picture index for a reference picture with a smaller temporal distance to a current picture may be selected, for example, when two or more reference picture indices present the same usage. Usage of a reference picture in multiple lists may be taken into consideration when identifying a representative reference picture,

[0148] When the same physical reference picture may be duplicated in multiple lists, usage of a reference picture in multiple lists may be added and/or considered together. For example, a POC=30 reference picture may be used three times in L0 and two times in L1. Combined usage of the reference picture may be five times for both lists (e.g., instead of three for L0 and two for L1).

[0149] In an example, a reference picture in a reference picture list (e.g., L0/L1 ) with the smallest temporal distance to the current picture may be selected as the representative reference picture that may be used at a motion search stage. The motion search stage may include a CU-ieve! motion search stage and/or a sub-CU level motion search stage. Reference pictures with smaller temporal distance(s) may be more correlated with a current picture. Most MV candidates in a current MV candidate list may be more likely to be directed to (e.g., more likely refer to) a reference picture with the smallest distance to the current picture, in an example, a reference picture in L0/L1 with the smallest temporal distance may be selected as a reference picture (e.g., a representative reference picture) that may be used at a motion search stage. A reference picture temporally closest to the current picture may be placed at a first entry of a reference picture list L0/L1 (e.g., by default), for example, based on a reference picture list construction procedure. Reference picture list reordering may be applied to change (e.g., default) orders. In an example implementation, a reference picture positioned at a first entry of a current reference picture list of the current picture may be selected as the reference picture (e.g., a representative reference picture) that may be used at a motion search stage,

[0150] The representative reference picture for the reference picture list L0 and the representative reference picture for the reference picture list L1 may be different, A selected reference picture (e.g., representative reference picture) for LO and a selected picture (e.g., representative reference picture) tor L1 may be the same physical reference picture [e.g., the same POC), The representative reference picture for LO may be kept (e.g., in this case). The representative reference picture for LI may be updated as the second most-used reference picture. The representative reference picture for LI may be the second entry in L1 . The representative reference picture for L1 may be a picture with the second closest temporal distance from the current picture.

[0151] Reference picture selection (e.g., as in the examples herein) may determine the optimal reference picture index for a given reference picture list (e.g., LO or LI ). Reference picture selection may be applied to uni!atera!-based motion derivation (e.g., template-matching mode), where MVs in LO and LI may be derived independently. MVs may be derived in pairs (e.g., in a bilateral-matching mode), tor example, based on an assumption that the MVs may be on the same motion trajectory as a current block.

Constraints may be applied, for example, when applying the reference picture selection herein for a bilateral-matching mode. Constraints may be applied so that selected reference pictures in LO and L1 may satisfy conditions that enable the bilateral-matching mode. Such conditions may include, for example, that two reference pictures temporally symmetric with regard to a current picture.

[0152] MV scaling and/or pruning may be performed. An MV scaling may be performed, for example, based on the identified representative reference picture. Some of the MV candidates in the MV candidate list may refer to the representative reference picture. Some of the MV candidates in the MV candidate list may refer to a reference picture(s) that is different than the representative reference picture. One or more MV candidates in an MV candidate list may be updated, for example, by scaling the MV candidates (e.g., the original MV candidates) to the representative reference picture. The MV candidates that refer to the reference picture(s) that is different than the representative reference picture may be identified and scaled such that the scaled MV candidates refer to the representative reference picture. A reference picture index for the identified representative reference picture may be determined, for example, as described herein.

[0153] An MV may be scaled based on a temporal distance between the reference picture that the MV refers to and the current picture and a temporal distance between the identified representative reference picture and the current picture, A reference picture list L0 may be used to illustrate an example MV scaling process, If an MV candidate already refers to the identified representative reference picture, the MV candidate may not be updated (e.g., scaled), For example, given /-th MV candidate MVi, an updated MV candidate MVi' may be set to be MVi, when an original L0 reference picture index (e.g., r[₀) is equal to a selected L0 reference picture index r ₀. If an MV candidate refers to a reference picture that is different from the identified representative reference picture, the MV candidate may be updated. An updated MV candidate MVi' may be set to be a scaled version of MVi, for example, when r[₀ is not equal to r ₀. MV may be set to be a scaled version of MVi, which may be calculated, for example, In accordance with Eq. 5: MV/ = MV_t ·—

ti Eq. 5 where T_L may be a temporal distance between a reference picture that may be indicated by r[_Q and a current picture, and τ^* may be a temporal distance between a reference picture that may be indicated by r ₀ and a current picture. Temporal distances may be signed values. Temporal distances may be obtained by computing a delta POC of multiple (e.g., two) pictures.

[0154] MV pruning may be performed, for example, to discard duplicated entries. MV values of multiple MV candidates {e.g., in a MV candidate list) may be substantially similar (e.g., the same). For example, the MV values of different MV candidates may become the same value after an MV scaling operation, The resulted same values may be due to a precision loss. Pruning may be performed to remove one or more of the MV candidates that have the same MV value. The two MV candidates with substantially similar MV values may be duplicated entries. One or more of the duplicate entries may be removed so that different (e.g., unique) MV candidates may be maintained in the MV candidate list. For example, the MV candidate list may be the final MV candidate list for CU level and/or sub-CU level motion search.

[0155] When FRUC is enabled, search MVs may be derived (e.g., by a processor of a WTRU) from an MV candidate list, for example, to reduce the number of MVs for motion search. The derivation of the search MVs may be based on MV grouping (e.g., MV clustering). MV grouping may not be applied. For example, the derivation of the search MVs may be based on scaled and pruned MV candidates, The motion search may include an initial motion search at a CU level or a sub-CU level,

[0158] Multiple MV candidates may be derived for a current CU. One or more search MVs may be derived from the multiple MV candidates, for example, based on an attribute(s) of the MV candidates. The number of search MVs may be less than the number of the MV candidates, The attribute may include a spatial distance between a [e.g., each) MV candidate and a centroid of an MV candidate set. The attribute may include an (e.g., each) MV value of a respective MV candidate. The attribute may include one or more characteristics that are associated with one or more of the MV candidates. The attribute may include a spatial distance between a (e.g., each) pair of MV candidates.

[0157] The MV candidates may be grouped into MV candidate sets, for example, based on MV clustering, MV candidates (e.g., in a MV candidate list) may be divided into L groups (e.g., MV candidate sets).

Search MVs may be derived from the MV candidate sets. For example, a search MV may be derived from each of the L MV candidate sets. The number of search MVs, for example, the number of MVs to be used in an initial motion search, may be L. [0158] The MV candidates may be grouped into the MV candidates sets based on a within-ciuster spatial distance. For example, an MV candidate may be grouped into an MV candidates set so that a within- ciuster spatial distance is minimized. The within-ciuster spatial distance may be or Include a spatial distance between the MV candidate and a centroid of the MV candidate set. The MV candidates may be grouped into L MV candidate sets so that the average of spatial distances between each respective MV candidate and the centroid of the MV candidate set in each of the L MV candidate sets is minimized.

[0159] In an example. MV candidates may be denoted, for example, as {MVo', MVi', MVN-I), where N may be a total number of MV candidates in an MV candidate list. The MV candidates denoted as {MVo', MVi', MVN-I) may be included in an original candidate list. The MV candidates denoted as {MVo¹, MVi',

MVN-I) may be included in a modified MV candidate list after reference picture selection and/or MV scaling and pruning. N may be the total number of MV candidates in the original and/or the modified MV candidate list,

[0160] MV grouping {e.g., MV clustering) may be used to group N MVs to L {L≤ N) MV candidate sets, for example, to minimize a within-ciuster distance. The MV candidate sets may be denoted as S = {So, Si, SL-I}. The within-ciuster distance may be derived based on spatial distances of some (e.g. , all) MV candidates to centroids of respective MV candidate sets. For example, the within-ciuster distance may be minimized in accordance with Eq. 6,

where μ_ί may be a centroid of Si. A clustering optimization problem in Eq, 6 may be solved, for example, using a clustering algorithm to cluster MV candidates (e.g., in an original candidate list and/or modified candidate list), The clustering algorithm may include a classical k-means refinement algorithm (e.g., Lloyd's algorithm).

[0161] In an example, an assignment step and/or an update step may be repeated when going through the MV candidates [e.g., alternately).

[0162] In an example assignment step, a spatial distance of an MV candidate to a centroid of an MV candidate set (e.g., cluster) may be calculated. For example, spatial distances of the MV candidate to each centroid of each MV candidate set (e.g., cluster) may be calculated. The spatial distances associated with the MV candidate sets may be compared. The MV candidate set that yields the shortest spatial distance may be determined. An MV candidate may be added to the MV candidate set that yields the shortest spatial distance, for example, if the shortest spatial distance is smaller than a distance threshold. The distance threshold may be pre-defined. A different MV candidate set from the MV candidate sets used In the comparison herein may be created, for example, if the shortest spatial distance is not smaller than the distance threshold, The MV candidate may be added to a different MV candidate set.

[0163] In an example of an update step, an MV candidate may be added to a MV candidate set that has a first centroid, The MV candidate may be different from MV candidates in the MV candidate set, The first centroid of the MV candidate set may be updated to a second centroid, The second centroid may be the centroid of the MV candidates in the MV candidate set, including the added MV candidate. For example, the first centroid may be updated to the second centroid in accordance with Eq, 7:

where 5/ may represent the number of elements (e.g., MV candidates) in an MV group S_t [e.g., MV candidate set) at a f-th clustering iteration.

may represent an updated (e.g.. second) centroid of the MV candidate set S_L. The updated centroid may be selected as a representative MV of the MV candidate set and/or used to determine the search MVs.

[0184] Based on the MV candidate sets, search MVs may be derived, The search MVs may be derived from the MV candidate sets by determining centroids of the MV candidate sets. Search MVs may be used to search for a MV candidate (e.g., initial motion candidate). For example, the centroids of resulting MV candidate sets (e.g.. clusters) may be used as search MVs used for comparing cost measurements to derive an initial MV at a CU-level and/or a sub-CU-level.

[0165] FIG. 1 1 shows an example of the MV grouping (e.g., using MV clustering). Smaller circles (e.g., MV candidate 804) may represent MV candidates (e.g., original MV candidates) before MV grouping. Bigger circles (e.g., MV candidate set 822) may represent MV candidate sets based on the MV grouping. Triangles (e.g., centroid 814) may represent centroids of MV candidate sets. For example, centroid 814 may be the centroid of the MV candidate set 822, As shown in FIG. 1 1 , the MV candidate 804 may be In the MV candidate set 822 after MV grouping,

[0166] The MV grouping may be based on the spatial distance between an MV candidate and a centroid of an MV candidate set (e.g., between the MV candidate 804 and the centroid 814). The spatial distance between the MV candidate 804 and the centroid 814 may be shorter than spatial distances between the MV candidate 804 and other centroids (e.g., centroids 810, 812, 816 and 818) of other MV candidate sets (e.g., MV candidate set 824). The spatial distance between the MV candidate 804 and the centroid 814 may be smaller than a predefined distance threshold. The predefined distance threshold may be represented by the size of the bigger circles (e.g., 822). [0167] The MV grouping may be based on the spatial distance between an MV candidate and another MV candidate [e.g., between an MV candidate 804 and MV candidate 808), MV candidates may be grouped such that the average of spatial distances of MV candidate pairs (e.g., between an MV candidate 804 and MV candidate 808) may be minimized.

[0168] The MV grouping may be based on MV values of the MV candidates (e.g., MV candidate 804 and MV candidate 808). The respective MV values of MV candidate 804 and MV candidate 808 may be similar enough so that MV candidate 804 and MV candidate 808 are grouped in a same MV candidate set 822. For example, as shown in FIG. 11 , the MV values of MV candidates may be similar enough if the spatial distances between the respective MV candidates and the coordinate 806 are substantially similar (e.g., the difference between the spatial distances is less than a threshold).

[0169] MV grouping may effectively reduce the number of search MVs. As shown in FIG. 11 , triangles (e.g., centroid 814) may represent updated MV candidates after MV grouping. Triangles (e.g., centrold 814) may represent the derived search MVs for the initial motion search. A search MV may be a representative MV of the MV candidates in the MV candidate set. For example, the search MV 814 may be a representative of the MV candidates 804 and 808 in the MV candidate set 822. The number of triangles (e.g., centrolds 810-818) may be less than the number of smaller circles (e.g., MV candidate 804 and MV candidate 808).

[0170] A group merging (e.g., cluster merging process) may be performed to restrict the number of groups (e.g., clusters). For example, the number of groups (e.g., MV candidate sets) may be within a limit. MV clustering using a pre-defined MV distance threshold may generate too many clusters, A group merging process may be performed, for example, when the number of groups is greater than a threshold (e.g., a preset limit). Groups may be merged, for example, by (e.g., first) merging multiple (e.g., two) groups closest to each other. The closeness of the groups may be determine based on the distances between groups, The distance between multiple (e.g., two) groups may be measured, for example, as the distance between the centroids of the groups, For example, as shown in FIG. 11 , MV candidate set 826 and MV candidate set 828 may be the closest to each other. The distance between MV candidate set 826 and MV candidate set 828 may be measured as the distance between centroid 816 and centroid 818.

[0171] A centroid of a merged group may be updated, For example, as shown in FIG. 1 1 , MV candidate set 826 and MV candidate set 828 may be merged and the centroid of the merged MV candidate set may be different from centroid 816 and/or centroid 818. A merge process may continue, for example, until a limit (e.g., the preset limit) on a number of groups is reached, The centroid of the merged MV candidate set may be the representative of the merged MV candidate set and/or used as a search MV.

[0172] MV grouping (e.g., MV clustering procedure) may be used at a CU-level motion stage and/or a sub-CU-level motion search stage. Multiple sub-CU MV candidates may be determined for a CU (e.g., a current CU). Sub-CU search MVs may be derived from the sub-CU MV candidates (e.g., using the MV grouping described herein). The number of sub-CU search MVs may be less than the number of the sub- CU MV candidates, A motion search (e.g., an initial motion search) of the current sub-CU may be performed using the sub-CU search MVs.

[0173] Reference picture selection procedures and MV scaling/pruning procedures may be applicable to a CU-ievel motion search . At a sub-CU-levei motion search, an (e.g., one) MV candidate may be included in an MV candidate list (e.g., only) when the MV candidate points to the same reference picture of the starting CU-level MV candidate. MV candidates in a sub-CU-level candidate list may (e.g., correspondingly) be associated with the same reference picture, Reference picture selection and MV scaling/pruning may be skipped at a sub-CU motion search stage.

[0174] FIG. 12 illustrates an example motion derivation process of a FRUC applying a clustering-based motion search at both the CU-!evel and the sub-CU-leve!. As shown in FIG. 12. the motion derivation process of a FRUC may include one or more of CU-level motion search, CU-levei motion refinement. sub- CU-level motion search, or sub-CU-!evei motion refinement. In the CU-ievel motion search, an initial CU- level MV candidate list may be generated . Each MV candidate in the in the initial CU-level MV candidate list may be associated with a corresponding reference picture (e.g., of a reference picture list). A representative reference picture may be selected, for example, from the reference picture list. The MV candidates may be scaled to refer to the representative reference picture. Duplicated MV candidates (e.g., after the scaling) may be discarded, MV grouping (e.g., MV clustering) may be performed on the MV candidates after the scaling, and representative MVs may be derived based on the MV grouping . The derived representative MVs may be used as search MVs. The CU-ievel motion refinement may be performed using the representative MVs. A variable / may be set to zero. The variable / may be used to track the sub-CU-level motion search, The sub-CU-level motion search may start with generating an initial sub-CU-level MV candidate list including MV candidates for the sub-CU-level motion search , MV clustering may be performed on the MV candidates for the sub-CU-level motion search. The sub-CU-levei motion refinement may be performed . A determination may be made as to whether the sub-CU-level motion search and/or refinement was performed on the last sub-CU in the CU . If so, next steps may be performed, If not, the variable / may be increased by one, and the sub-CU-level motion search and/or refinement may be performed on another sub-CU ,

[0175] MV grouping (e.g., at a CU-level) may be independent of reference picture selection and/or MV scaling/pruning. The MV grouping may be applied on an original candidate list, for example, before reference picture selection and/or the MV scaling/pruning has been applied . The MV grouping may be applied on a modified candidate list, for example, after reference picture selection and/or the MV scaling/pruning has been applied . When reference picture selection is skipped, the MV clustering may (e.g., further) take Into account which reference pictures MV candidates may be associated with and/or grouping the MV candidates based on their respective associated reference pictures (e.g., during clustering), For example, MV clustering may (e.g., only) group MV candidates that refer to the same reference picture(s) into an (e.g.. one) MV candidate set in addition to using one or more attributes of the MV candidates for grouping.

[0178] M V clustering may be performed to reduce the number of M V candidates for which a cost measurement may be calculated for a CU-level motion search and/or a sub-CU-level motion search . For example, it may be determined that a frame rate up conversion (FRUC) prediction mode is enabled for a current CU . One or more (MV) candidates associated with one or more attribute may be determined to be used as search MVs. The number of MV candidates for which a cost measurement may be calculated tor a CU-leve! motion search and/or a sub-CU-level motion search may be reduced , The number of the search MVs may be reduced by deriving representative MVs from the MV candidates as search MVs. The MV candidates may be clustered based on the attribute of the MV candidates. A motion search may be performed using the search MVs.

[0177] Reference picture selection may be performed independently of MV grouping, in an example, representative reference picture(s) may be selected without the MV grouping. The search MVs may be derived based on MV candidates that refer to the selected representative reference picture(s). In another example, the search MVs may be derived by scaling the MV candidates to the selected representative reference picture(s) and deriving the search MVs using the scaled MV candidates. In some examples, the number of search MVs may be the same as the number of MV candidates.

[01 8] MV scaling/pruning may be performed independently of MV grouping, in an example, the scaling of the MV candidates may be performed without the MV grouping and/or without pruning . The search MVs may be derived based on the scaled and/or pruned MV candidates. The search MVs derived from the MV candidates may or may not be less than the number of the MV candidates,

[0179] When an MV candidate list (e.g., an original candidate list) contains MV candidates that point to many reference pictures, characteristics of the MV candidates may be different. If the characteristics of the MV candidates are different, MV clustering may not provide a representation (e.g., an efficient representation) of the MV candidates in the candidate list (e.g., the original candidate list).

[0180] A FRUC MV clustering-based motion search (e.g., as described herein) may be adaptively enabled/disabled . For example, whether to enable FRUC MV clustering-based motion search may be determined based on the number of reference pictures that are referred to by the MV candidates in an initial MV candidate list. If the total number of reference pictures referred to by the initial MV candidates is not larger than a pre-defined threshold, MV clustering may be enabled , For example, the MV clustering may be applied by grouping the initial MV candidates into several MV candidate sets (e.g., major MVs). This may occur before a cost measurement calculation, if the total number of reference pictures referred to by the initial MV candidates is larger than the pre-defined threshold, the initial MV candidates may be less correlated. MV clustering may be disabled, For example, an original MV search may be applied. The original MV candidates in the initial MV candidate list may be tested {e.g., exhaustively tested).

[0181] Multi-stage motion search for CU/sub-CU-levei motion refinement may be performed. Motion refinement may be performed at a CU-level and a sub-CU-leve!. Motion refinement may be applied, for example, by performing a local MV search around an initial MV that may be determined from a CU- !evel/sub-CU-leve! motion search stage. CU-level motion refinement and sub-CU-level motion refinement may be performed in an FR.UC implementation, for example, based on the following: an (e.g., one) iterative search process may be performed {e.g., at quarter-sample accuracy) starting with an initial MV; and/or a one-time local search may be performed (e.g., at one-eighth-sample accuracy) around an MV {e.g., the best MV) that may be Identified in the iterative search process to find an optimal MV. An interpolation filter may be performed, for example, to generate samples at fractional positions, Extended reference samples may be used for interpolation,

[0182] A multi-stage motion refinement may be performed to identify an optimal local MV based on a multi-stage process. A motion refinement may, for example, include one or more of integer motion refinement or fractional motion refinement. A motion refinement (e.g., during integer refinement) may search MVs at integer-sample accuracy {e.g., based on a pattern-based motion search strategy) and may maintain the integer MV (e.g., best integer MV) that minimizes cost, A motion refinement may (e.g., during fractional motion refinement) consider additional fractional sample positions adjacent to a previously determined integer MV {e.g., to improve MV precision to a desirable level).

[0183] Integer motion refinement may be performed. An integer motion refinement may find an optimal MV at integer-sample accuracy. In an example, an Initial MV derived from a CU-levei/sub-CU-!evel motion search may be rounded to integer-sample accuracy and used as a starting search center. Cost measurements of predefined integer MVs (e.g., based on a diamond-shape, cross shape, or any other search shape pattern around a search center) may be compared. An integer MV that minimizes cost may be maintained. A selected integer MV may be set as an updated (e.g., new) search center and/or a search shape pattern may be repeated, for example, if a selected integer MV is not at a search center. A search process may continue updating a search center and/or may continue a search shape pattern (e.g., diamond motion search), for example, until a search center Is not updated in a search round or reaches a maximum search round. A search process may switch to perform an (e.g., one) additional motion search based on another pattern (e.g., a cross-shape pattern or any other search pattern) that may include multiple {e.g., four) additional integer positions around a current search center that may not be covered by a previous (e.g., diamond) motion search. [0184] FIG. 13 shows an example of Integer motion refinement. In an example, an integer motion refinement process may Include one or more of the following: an initial MV (e.g., ΜΨ^η may be determined, for example, from a CU-level/sub-CU-level motion search, A starting search center MVioi ' may be calculated, for example, by rounding M¥^nit lo integer-sample accuracy. Set k= 0; a search center MVik} ' may be given. Candidate integer MVs may be determined based on predefined positions of a diamond- shape pattern; a cost of (e.g., all) MV candidates on a diamond-shape pattern may be calculated , A search center MV(k+i) ^* may be set as a candidate that minimizes cost; MV refinement may set k = k + 1 and return to calculating a cost of (e.g., all) MV candidates on a diamond-shape pattern and/or setting a search center MVik+ij ' as a candidate that minimizes cost, tor example, when MVik+v ^'≠ MViki ^" and k < kmax or, otherwise, cost of all MV candidates on a cross-shape pattern may be calculated, and/or a final integer MV of a CU/sub-CU (e.g., MV) may be set as a candidate that minimizes cost; A search center MV¾ ^* may be given. Candidate integer MVs may be determined based on predefined positions of a cross-shape pattern; cost of all MV candidates on a cross-shape pattern may be calculated . A final integer MV of a CU/sub-CU (e.g.. MV) may be set as a candidate that minimizes cost.

[0185] FIG, 14 shows an example of integer motion refinement, In an example, circles represent integer MVs checked by a diamond motion search and triangles represent integer MVs checked by a cross motion search.

[0186] As shown in FIG. 14, five diamond motion searches and one cross motion search may be applied to identify an optimal MV at an integer-sample accuracy, Although diamond and cross search patterns are used in examples, other motion search patterns (e.g., a raster search pattern, a square search pattern and a hexagon search pattern) may be applied in an integer motion search procedure,

[0187] An initial MV search may be performed at integer-sample accuracy and a searching procedure (e.g., as shown by example in FIG. 13) may have a hard restriction on a maximum value of iterations (e.g., kmax) for integer motion refinement, kmax may be related to the complexity of implementing FRUC for a hardware encoder/decoder (e.g., worst-case complexity),

[0188] Fractional motion refinement may be performed, An optimal integer MV may be determined. Precision of a derived MV may be improved, for example, by a fractional refinement. In an example, an eight-point searching procedure may be applied during a fractional MV search process, In an example (e.g.. for half-sample accuracy motion refinement), eight half-sample positions around an integer-sample position pointed by a best integer MV may be tested. A half-sample MV with a minima! cost may be used to replace an Integer MV, Eight neighboring positions adjacent to a best half-sample MV and the best quarter-sample MV may be tested, respectively, for quarter-sample and one-eighth accuracy motion refinements. A fractional MV that may further reduce cost may be selected as an output MV for an improved motion precision. [0189] FIG. 15 shows an example of fractional motion refinement. F!G. 15 shows an example of fractionai motion refinement based on an eight-point-based search, where triangles, circles, squares and hexagons represent integer-sample positions, half-sample positions, quarter-sample positions and one-eight-sample positions, respectively, In an example, eight fractional sample positions may be tested for each fractional sample accuracy, Complexity of fractionai motion refinement may be reduced, for example, by applying other search patterns, Other search patterns may include, for example, a cross search pattern, where four additional fractional positions may be tested in each round,

[0190] FIG, 16 shows an example of fractionai motion refinement, FIG, 15 shows an example of fractional motion refinement based on an eight-point-based search, where triangles, circles, squares and hexagons represent Integer-sample positions, half-sample positions, quarter-sample positions and one-eight-sample positions, respectively, FIG, 16 shows an example where a cross search pattern may be applied at a fractional motion refinement stage. A cross search pattern may significantly lower searching complexity, for example, by reducing the total number of tested Vs by half. A resulting fractional MV may sometimes deviate from an optimal MV.

[0191] In an example, fractional motion refinement may be enabled after an optimal integer MV may be derived by an integer motion refinement process. Integer motion refinement (e.g., in a multi-stage motion search procedure) may determine whether a whole motion refinement process may converge to an actual motion of a current CU/sub-CU. Additional performance benefits provided by fractional motion refinement may be limited, for example, when an integer MV derived from integer motion refinement may be distant (e.g., far away) from a true motion.

[0192] Adaptively enabling/disabling fractional motion refinement may be accomplished, for example, by comparing cost measurements before and after integer motion refinement may be applied, e.g., Coshe re and Costatter. FIG. 17 shows an example of a FRUC motion derivation process with a multi-stage motion search applied at CU-level motion refinement and sub-CU-ievel motion refinement. FIG. 17 shows an example of multi-stage motion search adaptively disabling fractional motion refinement at CU-level and sub-CU-level. In an example, integer motion refinement may be effective to approach a true motion of a current block, for example, when a cost measurement becomes smaller after integer motion refinement (e.g., Costafter < Costbefcre). Fractional motion refinement may be applied for further MV accuracy improvement. If the cost measurement does not become smaller after integer motion refinement (e.g., Costatier≥ Costbetore), a fractional motion refinement process may be skipped.

[0193] Half-sample and/or quarter-sample accuracy motion search may be performed at a fractional motion refinement stage, Fractional-sample motion refinement may be skipped.

[0194] Early termination of sub-CU-!evei motion derivation may be performed. Sub-CU-level motion derivation {e.g., sub-CU-level motion search and sub-CU-level motion refinement) in FRUC may be applied after a CU-!eve! MV is derived. Sub-CU-level motion derivation may be bypassed, for example, when motions of sub-blocks inside a CU may be (e.g., are) consistent. Sub-CU-level motion derivation may be bypassed conditionally when true bi-prediction may be derived at a CU Ievel. True bi-prediction may occur, for example, when a first MV comes from a past reference picture in display order and a second MV comes from a future reference picture in display order. Sub-CU-level motion derivation may be bypassed, for example, when true bi-prediction is derived at the CU level for a template-matching based FRUC mode, Early termination of a sub-CU-levei motion derivation process may be performed, for example, while maintaining a coding gain of FRUC,

[0195] Early termination of sub-CU-levei motion derivation for an FRUC bilateral mode may be performed for CUs that are coded by the FRUC bilateral mode. FRUC bilateral-matching may estimate MVs based on temporal symmetry along a motion trajectory between prediction blocks in forward and backward reference pictures. For example, MVs in two prediction directions may be proportional to a temporal distance between a current picture and the respective reference picture of the current picture. Bilateral-matching based motion estimation may provide reliable motion vectors, tor example, when there Is small translational motion between two reference blocks (e.g., blocks in highest temporal layers in a random access configuration). True motions of sub-CUs inside a CU may be similar, for example, when the CU is coded by a FRUC bilateral mode. A sub-CU-level motion derivation process may be skipped for CUs that are coded using the FRUC bilateral mode.

[0196] FIG. 18 shows an example of a FRUC motion derivation process with early termination of sub-CU- level motion derivation for FRUC bilateral mode. FIG. 18 shows an example of a FRUC motion derivation process after disabling sub-CU-level motion derivation for a FRUC bilateral mode. As illustrated in FIG. 18, a CU-level motion search and CU-levei motion refinement may be performed. It may be determined whether the CU is coded by the FRUC bilateral mode, If the CU is coded by the FRUC bilateral mode, sub- CU-level motion derivation may be disabled. If the CU is not coded by the FRUC bilateral mode, sub-CU- level motion derivation may be performed, for example, for each sub-CU of the CU .

[0197] Early termination of sub-CU-level motion derivation may be performed based on a motion vector difference, Sub-CU-levei motion derivation may be skipped for FRUC bilateral sub-blocks, Early termination of a sub-CU-level motion derivation process may be performed depending on an absolute difference between MVs derived at a CU-level when a CU may be bi-predicted. in an example, two prediction signals may be highly correlated, and the motions of the sub-CUs within the CU may not significantly deviate from each other, for example, when CU-!evei derived MVs of a bi-predicted CU are approximately proportional in the temporal domain. CU-level MVs may be sufficient to accurately reflect a true motion, and sub-CU-level motion derivation may be skipped . CU-level MVs may be insufficient to accurately reflect the true motion, tor example, when bi-predicted CUs whose CU-level derived MVs are far from being temporally proportional. Sub-CU-level motion derivation may be performed on top of CU-!evel Vs. True motion detection may apply to template matching based FRUC mode and bilateral matching based FRUC mode.

[0198] in an example, (MVxo, MV_yo) and (MVxi, MV_yi) may denote CU-level MVs in reference picture lists LO and L1 , and τ₀ and τ may denote signed temporal distances of reference pictures in LO and L1 to a current picture. For example, temporal distances may be calculated in terms of delta POC between two pictures along a temporal domain in display order. (MV^sxi, MV^s _yi) may be calculated as a scaled version of (MVxi, MVyi), which may be generated based on τ₀ and τ₁ , for example, In accordance with Eq. 8:

MV" = MV_xl ^■— , MV^ = MV_vl ^■—

^Ti ^{"' τ}ι Eq. 8

(MV^sxi, MV^syi) may project a motion trajectory represented by (MVxi, MVyi) onto an LO reference picture. A sub-CU-level motion derivation may be disabled for a CU, for example, when a condition (e.g., in Eq, 9) may be satisfied: max (ahs(MV_x0 - ¾), afcs(jW_y0 - ¾j)≤ thres

■^■> J jrq g where thres may be a pre-defined threshold of MV difference, MVs derived at a CU-level for a current CU may or may not be accurate, for example, when the condition [e.g., in Eq. 9) is not satisfied, and sub-CU- level motion derivation may be applied for the CU,

[0199] FIG, 19 shows an example of a FRUC motion derivation with early termination (e.g., disabling) of sub-CU-level motion derivation based on MV difference criterion. A threshold of MV difference may be used to determine whether sub-CU-level motion derivation may be skipped. Appropriate thresho!d(s) may be selected when a FRUC motion derivation process with early termination is applied.

[0200] As illustrated in FIG, 19, a CU-level motion search and CU-level motion refinement may be performed. The scaled MVs may be calculated, it may be determined whether an absolute difference between MVs is greater than a predefined threshold. If the absolute difference between MVs is greater than a predefined threshold, sub-CU-level motion derivation may be disabled. If the absolute difference between MVs is not greater than a predefined threshold, sub-CU-level motion derivation may be performed, for example, for each sub-CU of the CU.

[0201] in an example, the same threshold of MV difference may be used for multiple (e.g., all) pictures.

[0202] in an example, different thresholds may be used for different pictures, in an example [e.g., in a random access configuration), one or more small thresholds may be used for pictures at a high temporal layer (e.g., due to smaller motion) and one or more large thresholds may be used for pictures at a low temporal layer (e.g., due to larger motion). MV scaling (e.g., as may be indicated in Eq. 8 and Eq. 9}, may be applied to (MVxi, MV_yi) for calculating an MV difference, which may not be implemented for every different combination of τ₀ and τ₁. For example, when MV scaling in Eq. 8 is applied and it is assumed that T₀ > T_j ), an error of (MVxi, MVyi) may be amplified. MV scaling may be applied to an MV associated with a reference picture having a larger temporal distance to a current picture. In an example, an MV (MVxi, MVyi) may be scaled for calculating MV difference (e.g., as shown in Eq, 8 and Eq, 9), for example, when T₀ < τ_Λ , MV (MVxo, MV_yo) may be scaled for calculating an MV difference when τ₀ > τ, , for example, in accordance with Eq. 10 and Eq, 11 :

MV xO MV;₀ MV...

Eq. 10 max ( hs(MV_xl - M¾), abs(MV_yl - MV$,)) < thres ^{1 1}

[0203] MV scaling may be applied, for example, when temporal distances of (e.g., two) reference pictures to a current picture may be different (e.g., τ₀≠ τ, ). An MV difference criteria may be used to disable a sub-CU-level motion derivation process (e.g., only) when a temporal distance of the reference pictures from a current picture may be the same (e.g., τ₀ = τ-J. MV scaling may not be necessary. Sign(s) of MVx and/or MVy may be flipped, for example, without scaling the MVs,

[0204] Fixed-point implementation of MV scaling (e.g., similar to TMVP) may be used. Eq. 8 and Eq. 9 may use division, A division operation may be avoided, for example, by modifying Eq, 8 and Eq, 9, for example, in accordance with Eq. 12 and Eq. 13:

MV* = MV_xl - T₀, MV$₁ = MV_yl - T₀ ^

tq. 1 max

[0205] Similar modifications may be applied to Eq. 10 and Eq. 1 1.

[0206] Early termination of sub-CU-level motion derivation may be performed based on the similarity of prediction blocks. An MV difference may be used as a measurement to determine whether sub-CU-level motion derivation may be avoided. Two prediction blocks generated using CU-!evel MVs may be precise enough such that sub-CU-level motion derivation may be disabled, for example, when a MV difference is small, Correlation between two prediction blocks may be measured, for example, using the MV difference. Correlation between two prediction blocks may be (e.g., additionally or alternatively) measured by calculating an average difference between two prediction blocks, for example, in accordance with Eq, 14 :

where / -⁰⁾(x, y) and l⁽¹^ (x, y) may be sample values at coordinates fx, y) of motion-compensated blocks that may be derived from reference pictures in reference picture list LO and L1 , respectively; B and N may be a set of sample coordinates and a number of samples defined within a current CU, respectively: D may be a distortion measurement, such as a sum of square error (SSE), a SAD and/or a sum of absolute transformed difference (SATD).

[0207] Sub-CU-level motion derivation may be skipped (e.g., in accordance with Eq, 14), for example, when two prediction signals may be highly correlated, For example, a difference measurement between two prediction signals as derived at a CU-level may be no larger than a pre-defined threshold, e.g., Diff ≤ D_thres. If the difference measurement is larger than the pre-defined threshold, two CU-level prediction signals of a current CU may be regarded as less correlated and sub-CU-level motion derivation may be applied,

[0208] FIG, 20 shows an example of a FRUC motion derivation process with early termination of sub-CU- level motion derivation based on a similarity of prediction blocks at a CU-level. FIG, 20 shows an example of a FRUC motion derivation process after a prediction similarity criteria may be applied to disable a sub- CU-level motion derivation process,

[0209] As illustrated in FIG, 20, a CU-level motion search and CU-level motion refinement may be performed, it may be determined whether difference measurement between two prediction signals as derived at a CU-level is larger than a pre-defined threshold. If the difference measurement is no larger than the pre-defined threshold, sub-CU-level motion derivation may be disabled, If the difference measurement is larger than the pre-defined threshold, sub-CU-!evei motion derivation may be performed, for example, for each sub-CU of the CU.

[0210] When the current CU is coded by the FRUC bilateral mode, the CU-level prediction difference (e.g., as described in Eq, 14) may be used for early terminating the sub-CU-level motion derivation process, FIG. 21 illustrate an example FRUC motion derivation process after applying the distortion-based early termination of sub-CU-level motion derivation for the FRUC bilateral mode (e.g., for the FRUC bilateral mode only). Whether to perform sub-CU-level motion derivation for FRUC bilateral mode may be signaled in sequence parameter set, picture parameter set or a slice. If sub-CU-level motion derivation is to be performed, a threshold may be signaled. [0211] As illustrated in FIG. 21 , a CU-level motion search and CU-levei motion refinement may be performed. Whether to skip the sub-CU-levei motion derivation may be determined based on the coding mode of the current CU, If the current CU is not coded by FRUC bilateral mode, the sub-CU-level motion derivation may be applied. When the current CU is coded by FRUC bilateral mode, the difference between the two prediction signals obtained at the CU-level may be calculated, if the difference between the two CU-ieve! prediction signals is no larger than a pre-defined threshold, the sub-CU-level motion derivation may be skipped, If the difference measurement is larger than the pre-defined threshold, sub-CU-ievel motion derivation may be performed, for example, for each sub-CU of the current CU. The value of a difference measurement (e.g.. the value used in Eq. 14) may already be available during estimation of optimal MVs at a CU-level and may be used to determine whether sub-CU-levei motion derivation may be disabled,

[0212] Early termination of the sub-CU-level motion derivation may be performed for FRUC template mode, As described herein, in FRUC template mode, template-matching may be used to derive the MVs at CU-level and/or at sub-CU-ievel. A match {e.g., the best match) between a template {e.g., top and/or left neighboring blocks) of the current CU/sub-CU and a template of the reference block in a reference picture may be identified, The template-matching-based motion search may be enabled (e.g., only be enabled) for the sub-blocks where at least one of the spatial top and left neighbors are available.

[0213] FIG. 22 illustrates an example of applying sub-CU-level motion derivation for one FRUC template CU. As shown, the template-matching-based motion search may be enabled for the sub-blocks in the first sub-block row and/or the first sub-block column (e.g., the patterned sub-blocks in FIG. 22), and may be disabled for the other sub-blocks (e.g., the blank sub-blocks in FIG. 22). When the sub-CU-ievel motion derivation Is applied for each FRUC template CU, the MVs of the sub-blocks in the first sub-block row and in the first sub-block column of the CU (e.g., the patterned sub-blocks In FIG, 22) may be further refined. For the other sub-blocks inside the CU (e.g., the blank sub-blocks in FIG, 22), because the top neighbor and/or the left neighbor may be unavailable, the template-matching-based motion refinement may be disabled. As shown in FIG. 4A and/or FIG. 4B, the number of available template samples may affect how closely the samples of the current CU/sub-CU and the template are correlated, As shown in FIG, 22, when the sub-CU-level motion derivation is applied to one FRUC template CU. except for the sub-block in the top-left corner may have multiple spatial neighboring blocks. For other sub-blocks, a spatial neighboring block (e.g., the left neighbor for the sub-blocks In the first sub-block column and/or the top neighbor for the sub-blocks in the first sub-block row) may be available for the template matching of the other sub-blocks, [0214] The sub-CU-level motion derivation process may be disabled for the CUs that are coded by the FRUC template mode, FIG, 23 illustrates an example FRUC motion derivation process with early termination of sub-CU-level motion derivation for FRUC template mode. As illustrated in FIG. 23, a CU- level motion search and CU-levei motion refinement may be performed. It may be determined whether a current CU is coded by the FRUC template mode. If the current CU is coded by the FRUC template mode, sub-CU-level motion derivation may be disabled. If the current CU is not coded by the FRUC template mode, sub-CU-level motion derivation may be performed, for example, for each sub-CU of the CU.

[0215] An indication of skipping sub-CU-level motion derivation for FRUC template mode may be signaled in sequence parameter set, picture parameter set or slice. The correlation of sub-block and its template may be checked using the predicted block referred by CU level derived motion, if the reference block of current sub-block and the reference block of its template have different motion Information, the sub-CU- level motion derivation for current sub-block may be skipped.

[0216] In an example, the sub-CU-level motion derivation process may be bypassed, which may Include sub-CU motion search and sub-CU motion refinement (e.g., as shown in FIG. 5). in an example, (e.g., only) sub-CU motion refinement may be bypassed, and a sub-CU motion search may be performed. The various early terminations of sub-CU-level motion derivation described herein may be jointly applied. The example early termination conditions described herein may be combined, and different combinations may provide different tradeoffs in terms of the coding performance and the encoding/decoding complexity reduction. For example, sub-CU-level motion derivation process may be disabled on a condition that the current CU is coded by FRUC template mode or the current CU is coded by FRUC bilateral mode, and the distortion between two prediction signals at the CU-levei is no larger than one pre-defined threshold. If the current CU is neither coded by FRUC template mode nor Is the current CU coded by FRUC biiaterai mode, or the distortion between two prediction signals at the CU-level is larger than one pre-defined threshold, the sub-CU-!evei motion derivation may be performed. FIG. 24 shows an example FRUC motion derivation process when a combined early termination method is applied to conditionally skip sub-CU-level motion derivation process. As shown in FIG, 24 , a CU-level motion search and CU-levei motion refinement may be performed, Whether to skip the sub-CU-level motion derivation may be determined based on the coding mode of the current CU and whether the CU-level distortion criteria is satisfied. Sub-CU-level motion derivation may be disabled when the current CU is coded by FRUC template or when the current CU Is coded by FRUC bilateral and the CU-levei distortion criteria Is satisfied, Otherwise, sub-CU-level motion derivation may be performed, for example, for each sub-CU of the current CU,

[0217] LiC may be disabled for CUs coded in FRUC bilateral mode, For example, an encoder may determine whether a CU is to be coded in FRUC bilateral mode based on the associated rate distortion cost. Upon a determination that the CU is to be coded in FRUC bilateral mode, the encoder may indicate (e.g., in the bitstream) that LIC is skipped or is disabled for this CU, For example, if the CU is coded In FRUC bilateral mode (e.g., the CU is a FRUC bi-laterai CU), signaling of the LIC fiag(s) may be skipped for the CU (e.g., LIC may be turned off). FIG. 25 illustrates an example L!C flag encoding based on whether FRUC bl-iaterai mode is used to code a CU .

[0218] FIG. 26 illustrates an example LIC flag decoding based on whether FRUC bi-latera mode is used to code a CU . At the decoder side, when the current CU is coded with FRUC bi-lateral mode, the LIC flag parsing stage can be bypassed. For example, the LIC flag can be inferred to be false (e.g., for CUs coded with FRUC bi-latera! mode). The LIC process may be omitted for CUs coded with FRUC bi-!atera! mode.

[0219] LIC mode may be enabled or disabled based on the degree of similarity between the two prediction signals in the FRUC bilateral mode. For example, prediction difference may be calculated using Eq. 15.

where /^(0j (x, y) and /^(1J (x, y) may represent the sample values at the coordinate fx, y) of the motion- compensated reference blocks derived from the forward and backward reference pictures; B and N may represent the set of the sample coordinates and the number of samples as defined within the current sub- block, respectively; D may represent the distortion measurement for which different metrics may be applied, such as, SSE, SAD and SAID. If Diff is smaller than a predetermined threshold (e.g., the two predictions are sufficiently similar), LIC mode may be disabled for that given CU . If the Diff is greater than a predetermined threshold, LIC mode may be enabled for the CU .

[0220] The techniques described herein may be implemented separately, simultaneously and/or may be freely combined . The different combinations may affect coding efficiency and/or complexity. For example, some (e.g., ail) of the proposed FRUC techniques discussed herein may be combined together, As described herein, different pictures within a GOP may have a different impact on the overall coding efficiency, The impact may depend on what prediction structure is applied, in a random access configuration, low-delay pictures (e.g., the key pictures in FIG. 10) may be important (e.g., more important than the other pictures). Low-delay pictures may be used (e.g., frequently used) as a temporal reference for the motion-compensated prediction of the other pictures. The FRUC techniques described herein may be disabled (e.g., partially disabled) for low-delay pictures. A determination of whether or not a (e.g., one) current picture is a (e.g., one) low-delay picture may be made, for example, by checking the POCs of the reference pictures for a current picture. If the POCs of some (e.g., all) of the reference pictures are smaller than that of the current picture, the current picture may be a low-delay picture. If the POCs of the reference pictures are not smaller than that of the current picture, the current picture may not be a low-delay picture, This determination may be signaled at the picture/slice-level (e.g., in a slice segment header),

[0221 ] One or more of the V clustering based motion search techniques described herein may (e.g., only) be applied to the FRUC-coded CUs when the current picture is not a low-delay picture. If, for example, the current picture refers to future pictures as temporal prediction (e.g., not as a low-delay picture), MV clustering-based motion search may be applied to some (e.g., all) of the FRUC-coded CUs inside a picture. If the current picture does not refer to a future pictures as temporal prediction {e.g., the current picture is a low-delay picture), a FRUC search method, which searches (e.g., exhaustively searches all) MV candidates may be applied to identify the optimal MV at the CU-ievel and sub-CU-ieve!, If the current picture refers to future pictures as temporal prediction (e.g., not a low-delay picture), the LIC may be disabled for FRUC bilateral CUs. The signaling of an LIC flag may be bypassed by FRUC bilateral CUs. if the current picture does not refer to future pictures as temporal prediction (e.g., the current picture is a low-delay picture), the LIC may be adaptiveiy enabled/disabled (e.g.. by sending one LIC flag for each FRUC bilateral CU).

[0222] Depending on whether or not a current picture is a low-delay picture, different criteria for early termination of the sub-CU-level motion derivation process for the CUs that are coded by FRUC bilateral mode may be applied (as described herein). Different distortion thresholds, e.g., Difio and D/A LD, may be defined for low-delay pictures and non-low-delay pictures. If the difference between the two CU-leve! prediction signals of a FRUC bilateral CU is not larger than the corresponding threshold, the sub-CU-level motion derivation may be skipped, If the difference between the two CU-level prediction signals of a FRUC bilateral CU is larger than the corresponding threshold, sub-CU-level motion derivation may be applied for the CU.

[0223] Initial MVs may be derived, for example, from merge candidates for a FRUC search.

[0224] A merge mode may include one or more of the following merge candidates. The merge candidates may include a set of CU motion vectors for a CU. For example, there may be one set of CU motion vectors for the whole CU. For example, CU motion vectors may include spatial and/or TMVP merge candidates. The merge candidates may include sub-CU motion vectors (e.g., ATMVP, STMVP). For example, a sub- block {e.g.. each sub-block) within the CU may have its own motion vector. The motion field derivation process for sub-CU motion vectors (e.g., ATMVP and/or STMVP) may include deriving the motion for a sub-block within the CU (e.g., each sub-block within the CU). As described herein, an initial MV list tor a FRUC CU-level search may Include the MVs from the CU motion vectors, As described herein, the initial MV list for a FRUC sub-CU-level search may include the MVs from the sub-CU motion vectors. The merge candidate CU motion vectors and sub-CU motion vectors may be derived together, for example, before a FRUC CU-level search.

[0225] A sub-CU-level search may be skipped if, for example, FRUC fast search and/or early terminations are applied. The derivation of MVs for a CU may be performed separately and/or independently from the derivation of MVs for a sub-block of the CU. One or more of the following may apply: MVs may be derived from the merge candidates of CU motion vectors, and/or MVs may be derived from merge candidates of sub-CU motion vectors when (e.g., only when) a sub-CU-level FRUC search is needed. FIG. 27 illustrates an example associated with deriving MVs from merge candidates (e.g., for FRUC search). As seen in FIG. 27, the derivation of merge candidates with sub-block motion may be skipped when a sub-CU-level search is not performed. The total number of merge candidates may be fixed, which may improve the performance of a FRUC search . For example, the total number of merge candidates from the CU motion vectors and sub-CU motion vectors described herein may be fixed at 7. if the merge candidates are fixed at 7 and the candidates from the sub-CU motion vectors are excluded, there may be 7 MVs in the CU motion vectors. If the merge candidates from the CU motion vectors and the sub-CU motion vectors are derived together, the number of merge candidates in the CU motion vectors may be less than 7.

[0226] Examples are presented for illustration . The examples may be exclusive or non-exclusive (e.g., features In the examples may be combined).

[0227] Although features and elements are described above in particular combinations, one of ordinary skill in the art will appreciate that each feature or element can be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor, Examples of computer-readable media include electronic signals (transmitted over wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs), A processor in association with software may be used to implement a radio frequency transceiver for use in a VVTRU, UE, terminal, base station, RNC, or any host computer.

Claims

CLAjjVIS What is Claimed:

1. A wireless transmit/receive unit (WTRU) comprising:

a processor configured to:

determine that a frame rate up conversion (FRUC) prediction mode is enabled for a current coding unit (CU);

determine a plurality of motion vector (MV) candidates for the current CU;

derive a plurality of search MVs from the plurality of MV candidates based on at least one attribute associated with the MV candidates, wherein the number of search MVs is less than the number of the MV candidates; and

perform a motion search for the current CU using the search MVs.

2. The WTRU of claim 1 , wherein the motion search is an initial motion search for the current CU.

3. The WTRU of claim 1 , wherein the processor is configured to:

determine a plurality of sub-CU MV candidates for a current sub-CU;

derive a plurality of sub-CU search MVs from the plurality of sub-CU MV candidates, wherein the number of sub-CU search MVs is less than the number of the sub-CU MV candidates; and

perform a motion search for the current sub-CU using the sub-CU search MVs.

4. The WTRU of claim 1 , wherein the processor is configured to:

group the plurality of MV candidates into a plurality of MV candidate sets based on the at least one attribute associated with the MV candidates; and

derive, from each of the MV candidate sets, a search MV that is representative of MV candidates in the MV candidate set.

5. The WTRU of claim 4, wherein the search MV from the MV candidate set is derived by determining a centroid of the MV candidate set.

6. The ^"WTRU of claim 4, wherein the processor is configured to perform clustering to group the plurality of MV candidates into the MV candidate sets.

7. The WTRU of claim 4, wherein in addition to the at least one attribute associated with the MV candidates, the plurality of MV candidates are grouped further based on their respective references pictures.

8. The WTRU of claim 1 , wherein the at least one attribute is a spatial distance between a respective MV candidate and a centroid of a corresponding MV candidate set. and the processor is configured to:

determine an MV candidate of the plurality of MV candidates;

determine a first spatial distance between the MV candidate and a centroid of a first MV candidate set of a plurality of MV candidate sets and a second spatial distance between the MV candidate and a centroid MV of a second MV candidate set of the plurality of MV candidate sets; determine that the first spatial distance is smaller than the second spatial distance;

group the MV candidate with the first MV candidate set;

update the centroid of the first MV candidate set based on the MV candidate; and derive a search MV of the plurality of search MVs using the updated centroid of the first MV candidate set.

9. The WTRU of claim 1 , wherein the at least one attribute is an MV value, each MV candidate has a respective MV value, and the processor is configured to:

group MV candidates Into MV candidate sets based on the respective MV values of the MV candidates; and

derive the plurality of search MVs using the MV candidate sets,

10, The WTRU of claim 1 , wherein an MV candidate of the plurality of the MV candidates is associated with a corresponding reference picture, and the processor Is further configured to:

identify a reference picture that is most frequently referred by the plurality of MV candidates as a representative reference picture for the plurality of the MV candidates; and

perform a motion search by referring to the representative reference picture.

The WTRU of claim 1 , wherein an MV candidate of the plurality of the MV candidates refers to a reference picture of a plurality of reference pictures, and the processor is further configured to: select a representative reference picture from the plurality of reference pictures referred b the plurality of MV candidates; identify at least one MV candidate that refers to at least one reference picture different from the representative reference picture; and

scale the identified at ieast one MV candidate that refers to the at least one reference picture different from the representative reference picture such that the scaled at Ieast one MV candidate refers to the representative reference picture,

12. The WTRU of claim 11 , wherein the processor is further configured to:

determine that at Ieast two of the MV candidates referring to the representative reference picture are substantially similar; and

discard at Ieast one of the at least two of the MV candidates for the determination of the plurality of search MVs.

13. A method performed by a wireless transmit/receive unit (WTRU) comprising:

determining that a frame rate up conversion (FRUC) prediction mode is enabled for a current coding unit (CU);

determining a plurality of motion vector (MV) candidates for the current CU; deriving a plurality of search MVs from the plurality of MV candidates based on at Ieast one attribute associated with the MV candidates, wherein the number of search MVs is less than the number of the MV candidates; and

performing a motion search for the current CU using the search MVs.

14. The method of claim 13, wherein the motion search is an Initial motion search for the current CU.

15. The method of claim 13, further comprising:

grouping the plurality of MV candidates into a plurality of MV candidate sets based on the at Ieast one attribute associated with the MV candidates; and

deriving, from each of the MV candidate sets, a search MV that is representative of MV candidates in the MV candidate set.

16. The method of claim 15, wherein the search MV from the MV candidate set is derived by

determining a centroid of the MV candidate set,

17. The method of claim 13, wherein the at !east one attribute is a spatial distance between a respective MV candidate and a centroid of a corresponding MV candidate set, the method further comprising:

determining an MV candidate of the plurality of MV candidates;

determining a first spatial distance between the MV candidate and a centroid of a first MV candidate set of a plurality of MV candidate sets and a second spatial distance between the MV candidate and a centroid MV of a second MV candidate set of the plurality of MV candidate sets; determining that the first spatial distance is smaller than the second spatial distance; grouping the MV candidate with the first MV candidate set;

updating the centroid of the first MV candidate sei based on the MV candidate; and deriving a search MV of the plurality of search MVs using the updated centroid of the first MV candidate set,

18, The method of claim 13, wherein an MV candidate of the plurality of the MV candidates is

associated with a corresponding reference picture, the method further comprising:

identifying a reference picture that is most frequently referred by the plurality of MV candidates as a representative reference picture for the plurality of the MV candidates; and

performing a motion search by referring to the representative reference picture.

19, The method of claim 13, wherein an MV candidate of the plurality of the MV candidates refers to a reference picture of a plurality of reference pictures, the method further comprising:

selecting a representative reference picture from the plurality of reference pictures referred by the plurality of MV candidates;

identifying at least one MV candidate that refers to at least one reference picture different from the representative reference picture; and

scaling the Identified at least one MV candidate that refers to the at least one reference picture different from the representative reference picture such that the scaied at least one MV candidate refers to the representative reference picture,

20, The method of claim 19, the method further comprising:

determining that at least two of the MV candidates referring to the representative reference picture are substantially similar; and

discarding at least one of the at least two of the MV candidates for the determination of the plurality of search MVs.