KR20180112471A - Method and apparatus for syncronizing a plurality of videos - Google Patents

Abstract

The present invention discloses a method for synchronizing a plurality of images, and a device thereof. According to an embodiment of the present invention, the method for synchronizing a plurality of images comprises: a step of determining similarity among a plurality of images; a step of determining synchronization among the plurality of images based on the similarity of the plurality of images; and a step of synchronizing by deleting a frame of one image among images out of synchronization as much as the number of the images out of synchronization when synchronization among the plurality of images is determined to be out of synchronization. In addition, the step of determining similarity among a plurality of images can determine similarity among a plurality of images by using at least one between a method of determining similarity of a plurality of images based on variation in a brightness value and a color value of a bonding subsequence between two images that are adjacent in terms of space, and a method of determining similarity among the plurality of images based on a scene change frame of the plurality of images. Therefore, the method for synchronizing a plurality of images, and the device thereof can synchronize a plurality of images by using similarity among the images in a real-time large angle panorama play system or a multi-surface image play system.

Description

[0001] METHOD AND APPARATUS FOR SYNCRONIZING A PLURALITY OF VIDEOS [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for synchronizing a plurality of images and more particularly to a method and apparatus for synchronizing a plurality of images, A plurality of video synchronization methods and apparatus thereof.

In recent years, there has been an increasing demand for a wide viewing angle image service such as a panorama image service or a multi-view image service. Especially, there is an increasing demand for a real-image-based PR video service and experience service using a multi-projector centered on exhibition halls and experience halls. In addition, if the idol star performances are performed based on the Korean Wave, it is a reality that is commercialized through the video service movie theater.

These services are gradually evolving and are in the process of real-time services. In order to support real-time services, it is necessary to use coding / decoding technology.

In order to support a panorama image service or a multi-image service, which is provided by combining one or more HD / UHD multi-images in real time, the number of multi-image coders / decoders is used. However, when a commercial encoder / decoder is used, it is difficult to support synchronization between each multi-image. That is, the synchronization between the output multi-images of the commercial decoder to be used in the reproduction system environment can not be guaranteed to be exactly the same due to the fact that a plurality of multi-decoders are physically used.

This is because even if the same decoder is used, the input time of the multi-picture stream input to each decoder is not accurate and the decoding time of each decoder can not be exactly matched. When the output multi-image of the multi-decoders, which are not guaranteed to be synchronized, is output to the screen to provide each panorama image service or multi-image service, .

Disclosure of Invention Technical Problem [8] Accordingly, the present invention has been made to solve the above-mentioned problems occurring in the prior art, and it is an object of the present invention to provide a plurality of video synchronization methods and apparatuses for synchronizing a plurality of images using similarity between images in a real time interactive panorama playback system or a multi- .

The technical objects to be achieved by the present disclosure are not limited to the above-mentioned technical subjects, and other technical subjects which are not mentioned are to be clearly understood from the following description to those skilled in the art It will be possible.

According to one aspect of the present disclosure, a plurality of image synchronization methods includes: determining similarities between a plurality of images; Determining whether or not the plurality of images are synchronized based on the similarity between the plurality of images, and determining whether or not the plurality of images are synchronized with each other based on the degree of similarity between the plurality of images, Wherein the step of determining the degree of similarity between the plurality of images includes determining a degree of similarity between a plurality of images based on a brightness value and a variation value of a color value of a joint partial column of two spatially adjacent images, And determining a degree of similarity between the plurality of images based on the scene change frames of the plurality of images.

The features briefly summarized above for this disclosure are only exemplary aspects of the detailed description of the disclosure which follow, and are not intended to limit the scope of the disclosure.

According to the present disclosure, it is possible to provide an efficient synchronization method in a real-time interactive panorama playback system or a multi-image playback system.

The effects obtainable from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below will be.

FIG. 1 is a flowchart for explaining a plurality of image synchronization methods according to an embodiment of the present disclosure.
2 is a flowchart illustrating a method of decoding and synchronizing a plurality of images according to an embodiment of the present disclosure.
3 is a view for explaining a method of determining a similarity degree of a joint portion sequence of two spatially adjacent images according to an embodiment of the present disclosure.
4 is a diagram for explaining a method of determining a degree of similarity between a plurality of images based on a scene change frame according to an embodiment of the present disclosure.
5 is a block diagram for explaining a plurality of image synchronizing apparatuses according to an embodiment of the present disclosure;

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, which will be easily understood by those skilled in the art. However, the present disclosure may be embodied in many different forms and is not limited to the embodiments described herein.

In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present disclosure rather unclear. Parts not related to the description of the present disclosure in the drawings are omitted, and like parts are denoted by similar reference numerals.

In the present disclosure, when an element is referred to as being "connected", "coupled", or "connected" to another element, it is understood that not only a direct connection relationship but also an indirect connection relationship May also be included. Also, when an element is referred to as " comprising "or" having "another element, it is meant to include not only excluding another element but also another element .

In the present disclosure, the terms first, second, etc. are used only for the purpose of distinguishing one element from another, and do not limit the order or importance of elements, etc. unless specifically stated otherwise. Thus, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly a second component in one embodiment may be referred to as a first component .

In the present disclosure, the components that are distinguished from each other are intended to clearly illustrate each feature and do not necessarily mean that components are separate. That is, a plurality of components may be integrated into one hardware or software unit, or a single component may be distributed into a plurality of hardware or software units. Thus, unless otherwise noted, such integrated or distributed embodiments are also included within the scope of this disclosure.

In the present disclosure, the components described in the various embodiments are not necessarily essential components, and some may be optional components. Thus, embodiments consisting of a subset of the components described in one embodiment are also included within the scope of the present disclosure. Also, embodiments that include other elements in addition to the elements described in the various embodiments are also included in the scope of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

FIG. 1 is a flowchart for explaining a plurality of image synchronization methods according to an embodiment of the present disclosure.

Referring to FIG. 1, a plurality of image synchronization methods may be executed by performing steps S110 to S130.

In step S110, the degree of similarity between a plurality of images can be determined. In order to determine the degree of similarity between a plurality of images, a method of determining a similarity between a plurality of images based on the brightness value and the color value change amount of a joint partial column of two spatially adjacent images, And a method of determining the degree of similarity between a plurality of images can be used.

A method of determining the degree of similarity between a plurality of images on the basis of the brightness value and the variation amount of the color value of the joint partial column of two spatially adjacent images is as follows.

) 값을 측정하여, 제1 델타 값이 기 설정된 값(Threshold value)보다 적을 경우 유사도가 높다고 판단할 수 있다. 반면 기 설정된 값보다 클 경우 유사도가 작다고 판단할 수 있다. 여기서, 델타 값이란 두 픽셀의 밝기 값 또는 색상 값의 차이 값을 의미할 수 있다.(Y or V) of each pixel in a boundary portion of two spatially adjacent images in a plurality of images, that is, a first partial delta

). If the first delta value is less than a predetermined threshold value, it can be determined that the degree of similarity is high. On the other hand, if it is larger than the preset value, it can be judged that the degree of similarity is small. Here, the delta value may mean a brightness value of two pixels or a difference value of color values.

Specific examples thereof will be described later with reference to FIG.

On the other hand, a method of determining the degree of similarity between a plurality of images based on a scene change frame of a plurality of images is as follows.

It is possible to detect a scene change frame by measuring a second delta value of a pixel between frames according to a time sequence in each image among a plurality of images. Specifically, when the second delta value between frames according to the time order is greater than a predetermined value, it can be determined as a scene change frame.

If the scene change frames determined in each image are within the same viewpoint (or frame) or within a predetermined viewpoint interval (or a predetermined frame interval), it can be determined that the degree of similarity between the plurality of images is high. On the contrary, when the scene change frames determined in the respective images are outside the preset view difference, it can be determined that the degree of similarity between the plurality of images is low.

If the third delta value is smaller than the preset value, the degree of similarity is large. If the third delta value is larger than the predetermined value, the frame interval of the scene change frames determined in each image is assumed to be the third delta value. It can be judged that the degree of similarity is low.

A specific example of this will be described later with reference to FIG.

In step S120, it is determined whether or not the plurality of images are synchronized based on the similarity between the plurality of images. If it is determined in step S110 that the degree of similarity is high, it can be determined that the synchronization is correct by considering the same frame. However, when it is determined that the degree of similarity is small, it can be determined that the frames are not the same and the synchronization is out of order.

On the other hand, if it is determined that the synchronization between the plurality of images is out of phase (S120: NO), step S130 may be performed.

In step S130, it is possible to synchronize the frames by deleting frames of any one of images whose synchronization is out of phase by the number of image frames shifted. Specifically, it is possible to determine the number of frames up to the frame in which the degree of similarity increases again after the frame of the image with the out-of-sync has passed, as the number of frames in which the problem of synchronization has occurred. Accordingly, the synchronization can be achieved by deleting the image frame by the number of frames determined as above in the frame buffer of any one of the images whose synchronization is out of order.

On the other hand, as a method of realizing the synchronization, it is possible to sequentially delete frames of any one of the images whose synchronization is out of alignment, and to determine the similarity between images to synchronize with each other.

2 is a flowchart illustrating a method of decoding and synchronizing a plurality of images according to an embodiment of the present disclosure.

Referring to FIG. 2, a method of decoding and synchronizing a plurality of images may be performed by executing steps S210 to S250.

In step S210, a plurality of images can be respectively decoded. Here, the plurality of images may be a video stream obtained by coding images constituting a real-time dialogue panorama image or a multi-view image. Here, a decoder may be used for a plurality of images in order to decode a plurality of images. At this time, a plurality of decoders can use the same internal or external clock to support synchronization between the decoders.

However, in general, most of the plurality of decoders do not support physical external clock input, and even if a same software clock is used, it is assured that a plurality of decoded images can be accurately synchronized due to a process time difference between a plurality of decoders I can not.

Therefore, it is possible to determine whether or not to synchronize a plurality of decoded pictures by performing steps S220 to S240, and to perform synchronization.

In step S220, a method of determining a similarity between a plurality of images based on brightness values and color values of a joint partial sequence of two spatially adjacent images or a method of determining similarities between images based on a scene change frame may be used The degree of similarity between a plurality of images can be calculated.

The description thereof has been specifically described in S110 of Fig. 1, and will be omitted.

If it is determined in step S230 that the calculated delta value is greater than a predetermined value, it is determined that the synchronization is out of order. If the calculated delta value is less than the predetermined value, it is determined that the synchronization is correct. Here, the delta value indicates a first delta value when a method of determining similarity between a plurality of images based on a brightness value and a color value variation of a joint partial column of two spatially adjacent images is used, And a third delta value when a plurality of inter-image similarity determination methods are used.

If the delta value is greater than the predetermined value in step S230 (S230- Yes), step S240 is executed to realize synchronization.

The synchronization realization in step S240 has been described in detail in S130 of FIG. 1, and will be omitted.

In step S250, it is determined whether a plurality of video streams have been completed. If the video streams are finished, the entire process is terminated (S250- Yes). If not finished, a plurality of video decoding processes may be performed again (S250 -No).

3 is a view for explaining a method of determining a similarity degree of a joint portion sequence of two spatially adjacent images according to an embodiment of the present disclosure.

3, a description will be given on the assumption that an 8K panoramic image is transmitted and received in real time by combining a 4K left image and a 4K right image.

을 계산하고, 좌영상의 우측열의 색상 값인 U(L(x))와 우영상의 좌측열의 색상 값인 U(R(x))의 차이인

을 계산하여, 이들의 합인

값을 산출한다. 여기서, 와

는 수학식 1을 이용하여 계산할 수 있다.The difference between the brightness value Y (L (x)) of the right column of the left image and the brightness value Y (R (x)) of the left column of the right image

And calculates the difference between the color value U (L (x)) of the right column of the left image and the color value U (R (x)) of the left column of the right image

, And calculates the sum

Lt; / RTI > here, Wow

Can be calculated using Equation (1).

값이 클수록 공간적으로 인접한 두 영상의 접합 부분열의 유사도가 낮고,

값이 낮을수록 공간적으로 인접한 두 영상의 접합 부분열의 유사도가 높다고 할 수 있다.here,

The larger the value, the lower the similarity of the joint partial sequences of two spatially adjacent images,

The lower the value, the higher the degree of similarity of the joint partial sequences of two spatially adjacent images.

값이 기 설정된 값 (Threshold,

)보다 클 경우 좌/우 영상의 동기가 어긋난 것으로 판단할 수 있다. 여기서, 기 설정된 값과

을 비교하는 것은 수학식 2을 이용하여 나타낼 수 있다.Meanwhile,

If the value is less than the preset value (Threshold,

), It can be determined that the synchronization of the left / right image is out of order. Here,

Can be expressed using Equation (2).

4 is a diagram for explaining a method of determining a degree of similarity between a plurality of images based on a scene change frame according to an embodiment of the present disclosure.

값이 큰 것을 확인 할 수 있다. 두 영상의 장면 전환 프레임의 간격 사이에서 좌/우 영상의 유사도가 낮은 것으로 판단되어 동기가 어긋난 것으로 간주할 수 있다. 즉, 두 영상의 장면 전환 프레임을 기준으로 영상의 유사도를 판단하여 동기 여부를 확인할 수 있다.Referring to FIG. 4, when the scene change frame 24 of the two images is used as a reference,

It can be confirmed that the value is large. The similarity of the left / right images is judged to be low between the intervals of the scene change frames of the two images, so that it can be regarded that the motions are out of order. That is, the similarity of the images can be determined based on the scene change frames of the two images, and it is possible to confirm whether or not the images are synchronized.

 On the other hand, the frame of the right image can be synchronized by deleting the frame of the right image by the number of frames between the scene change frames of the two images so that the scene change frames 24 of the two images are within the same frame or predetermined frame intervals.

5 is a block diagram for explaining a plurality of image synchronizing apparatuses according to an embodiment of the present disclosure;

5, a system for reproducing a plurality of images may include a plurality of decoders 510-1 and 510-n, a plurality of image synchronizing apparatuses 520, and a reproducing unit 530. FIG.

The plurality of decoders 510-1 and 510-n can respectively decode a plurality of encoded images. Here, the number of the plurality of decoders may be the same as the number of the input images.

The plurality of image synchronization apparatuses 520 may include a similarity determination unit 521 and a synchronization unit 522.

The degree-of-similarity determination unit 521 determines a degree of similarity between images based on a brightness value and a variation value of a color value of a joint partial column of two spatially adjacent images or a method of determining similarity between images based on a scene change frame The degree of similarity between a plurality of images can be calculated using at least one.

The description thereof has been specifically described in S110 of Fig. 1, and will be omitted.

The synchronization unit 522 may determine whether a plurality of images decoded based on the similarity calculated by the similarity degree determination unit 521 are synchronized. If it is determined that the synchronization is out of phase, the synchronization unit 522 can delete the frame of any one of the images whose synchronization is out of synchronization by the number of image frames whose synchronization is out of phase and synchronize with each other. Specifically, it is possible to determine the number of frames up to the frame in which the degree of similarity increases again after the frame of the image with the out-of-sync has passed, as the number of frames in which the problem of synchronization has occurred. Accordingly, the synchronization unit 520 can synchronize the frame by deleting the image frame by the number of frames determined as above in the frame buffer of any image out of synchronization. As described above, a method of synchronizing with a method of deleting a frame is called a frame skip.

On the other hand, when synchronizing by deleting the video frame as described above, the audio corresponding to the video frame to be deleted is deleted at the same time, so that the synchronization between the video and the audio can be also synchronized.

The reproducing unit 530 can reproduce a plurality of decoded videos synchronized with each other.

[0040] A plurality of image synchronization methods and apparatuses have been described with reference to FIGS.

A plurality of image synchronization methods and apparatuses according to an embodiment of the present invention can provide an efficient synchronization method in a real-time interactive panorama playback system or a multi-image playback system

On the other hand, according to one aspect of the present disclosure, software or a computer-readable medium having executable instructions for performing a plurality of image synchronization methods may be provided. The plurality of video synchronization methods may include the steps of determining a degree of similarity between a plurality of images, determining whether or not the plurality of images are synchronized based on the degree of similarity between the plurality of images, Wherein the step of determining the degree of similarity between the plurality of images includes a step of determining a degree of similarity between two images that are spatially adjacent to each other, A method of determining a similarity degree between a plurality of images based on a brightness value of a column and a variation value of a color value and a method of determining similarity between the plurality of images based on the scene change frames of the plurality of images, And determining the degree of similarity between the images.

Although the exemplary methods of this disclosure are represented by a series of acts for clarity of explanation, they are not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. In order to implement the method according to the present disclosure, the illustrative steps may additionally include other steps, include the remaining steps except for some steps, or may include additional steps other than some steps.

The various embodiments of the disclosure are not intended to be all-inclusive and are intended to illustrate representative aspects of the disclosure, and the features described in the various embodiments may be applied independently or in a combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays A general processor, a controller, a microcontroller, a microprocessor, and the like.

The scope of the present disclosure is to be accorded the broadest interpretation as understanding of the principles of the invention, as well as software or machine-executable instructions (e.g., operating system, applications, firmware, Instructions, and the like are stored and are non-transitory computer-readable medium executable on the device or computer.

Claims (1)

In a plurality of video synchronization methods,
Determining a degree of similarity between a plurality of images;
Determining whether the plurality of images are synchronized based on the similarity between the plurality of images; And
And deleting a frame of one of the images whose synchronization is out of phase by the number of image frames whose synchronization is out of phase when it is determined that the synchronization between the plurality of images is out of phase,
Wherein the step of determining the degree of similarity between the plurality of images comprises:
A method of determining a degree of similarity between a plurality of images based on a brightness value and a variation value of a color value of a joint partial column of two spatially adjacent images and a method of determining similarities between the plurality of images based on the scene change frames of the plurality of images Wherein the degree of similarity between the plurality of images is determined using at least one of the plurality of images.

Images