TW201227608A - Method and system for rendering multi-view image - Google Patents

Abstract

A method and a system for rendering a multi-view image are provided. The method for rendering the multi-view image includes the following steps. A plurality of single-view images whose view-angle are different from each other. The single-view images are performed a resizing process, such that at least a portion of pixels of a new-resolution images are obtained. The portion of pixels of the new-resolution images are performed a view interlace process, such that a multi-view image is resulted.

Description

201227608 TW6973PA· VI. DESCRIPTION OF THE INVENTION: 1. Field of the Invention The present invention relates to a method and system for constructing an image, and more particularly to a method and system for constructing a multi-view image. [Prior Art] - Digital image has the advantages of "no waste of negative film, no space, no fading, easy storage, easy to carry, easy to edit", so that digital images have gradually replaced the photos taken by traditional negatives.

Ik is developing digital image technology, and various video secret technologies are constantly developing. Through image editing technology, you can beautify your photos, or add interesting patterns, or even edit them into a multi-view stereoscopic image. However, the construction method of multi-view stereoscopic images is quite complicated. In terms of processing technology, it is necessary to effectively increase the processing speed, so that multi-view stereoscopic images are more generally accepted. SUMMARY OF THE INVENTION According to one aspect of the present invention, a method of constructing a multi-view image is proposed. The method for constructing a multi-view image includes the following steps. Several single-view images are provided, and the viewing angles of these single-view images are different. For these single-view scenes, a resolution adjustment program (Resizing) is performed to obtain a plurality of new resolutions = at least some pixels of the image. For this new resolution image, at least the pixel is scanned for a View Interlace to open a multi-view image. Evening 201227608

TW6973PA According to another aspect of the invention, a system is proposed. The multi-view image construction system includes a construction element that provides a m-shirt image. The unit is provided to provide several single-view images: the processing angle is different. The processing unit is a single view of this: the Resizing image, the resolution of the image, and the resolution of the F (4) L new resolution image.

Draw a pixel and perform an image-like image on the face resolution image to the J staggered program (View Inte). The present invention is described in detail with reference to the drawings, and The description does not limit the scope of the invention to be protected. ^ as a feature of surgery. The present invention is directed to the first embodiment of the present invention. The multi-view image construction system 1 includes a providing unit and a processing unit 12A. The providing unit 11G is used to provide various information such as a camera, a camera, a storage device, or a connection storage medium. The processing unit 120 is configured to execute various processing and computing programs, such as a central processing unit (Central Processing Unit 201227608 TW15973PA* CPU) GPU, and one of a graphics processing unit (Graphic Processing Unit, or a stored array code) The storage medium. In the embodiment, the material 12G includes an array of the first line ΐ 2 ι and the second array of the array 122, the first thread 121 is processed in parallel processing, and the operation program 'the second thread (2) is also In the parallel processing, various processes and transitions are performed (4). Please refer to FIGS. 2 to 3, FIG. 2 is a flow chart showing the multi-view of the first embodiment, and the construction method of 04. FIG. 3 is a diagram showing the first embodiment. A schematic diagram of the construction process of the illuminating view image P1G4. The following is a more detailed description of the multi-view of the embodiment and the multi-view image of the embodiment. However, the general knowledge in the technical field of the present invention is The construction system of multi-view image P1°4 _ is not P1, and the construction method of 〇4 is not limited to (4) 建P104 construction system 1〇〇.

Pioi ^^Γρι〇+ι

= If R" stands for red, "G" stands for green, and "B" stands for blue). The resolution is 640x360x3. J, then, in step S102, the first execution 121 of the processing unit 12 performs Pixel Rendering A1A pixel interpolation light (HoleFilling) A2 in parallel processing according to the depth information to form Several new perspective images ^". μ第201227608

The TW6973PA 3 picture is taken as an example. After the original image ship passes through the pixel shifting program M and the pixel interpolation program A2, eight new viewing angle images P1G2 are formed. Each of the new 2 images P102 is also 64G, and the number of lines is also the lake color gamut channel 疋3 ("R" stands for red, "G" stands for green, % stands for blue), and its resolution is 640x360x3. Among them, the pixel shift program ^ used to the original image? Each of 1〇1 is translated to the appropriate position to construct 8 new viewing angle images 02. In the pixel shifting process, a new view image (10) may have a gap phenomenon. At this time, the gap is filled by the pixel interpolation program A2. Taking Figure 3 as an example, the single-view image (10) includes t original images P101 and 8 riding angles, so the single-view image (4) is a total of ^, the viewing angles of these single (four) images (10) are different, but the resolution ^ is 640x360x3. So 'through the steps _~si

Several single-view images P110 are shown. Then, in step S103, the first threads 121 perform a resolution adjustment procedure (Resizing) A3 on the single-view images _ in a parallel manner to obtain a plurality of new resolution images ρι〇3. . Taking Figure 3 as an example, the resolution of nine single-view (four) images like P11G is _χ36 (four), and the number of columns of the new resolution image P103 is 192 〇, the number of rows is 1080, and the gamut channel is 3. The resolution is 192 〇χΐ〇 8 〇χ 3. In the present embodiment, the pixel shifting program A1, the pixel interpolation program Μ, and the resolution adjustment program A3 are all executed in parallel by the first thread 121. Taking FIG. 3 as an example, the arrow between the original image ρι〇1 and the new video P102 indicates the operation of the steps s1〇2 of the pixel-shifting program A1 and the pixel interpolation program A2 for each of the first-execution m, so 201227608 TW6973PA. *The number of new Si Ϊ I21 is the number of new perspective images P1 02, the new perspective, like the product of p 1 〇2 | i &, the number of sheets (ie 640x8) . Furthermore, the new view image P102 and the new arrow indicate the action of the step _ 3 between the respective first-execution 121 and the resolution-type shirt P103, so this is the first step of the execution of the action, and the adjustment of Cheng Guanghong, The number of the day & the younger brother 121 is also the arrow between the number of products ^1=2 and the head of the new _piG2, and the arrow between the new resolution image P103: Early 兀 'for example, the central processing unit ^

Pr〇ceSSlngUnit, cpu) to synchronize execution. In step S104, the second thread 122 is processed in parallel (Vi = pixel of the resolution image (10) - staggered program 3_W^terlaCe) A4' to form a multi-view image P104. Taking the figure as an example, the resolution of the new resolution image P1〇3 is the same (=.=) to select the pair; *Exercising the thread 122 only needs to pick directly in the resolution image P1G3: == can form a lot Perspective image. ιTM As shown in Fig. 3, the new resolution image P10 and the multi-view movie ριη/1 ^ ^ head indicate the operation of the step S1G4 of the interleaving program A4. The number of multiphases (four) 122 is the number of columns of the multi-view image P104, χ 3). The angle image PH) is the product of the number of lines and the color gamut channel (ie 192 〇 _ as shown in Figure 3, the pixels composed of the multi-view image ρι〇4 are executed 201227608

TW6973PA ::: Qiao adjustment program, 3, so multi-view 'select ^ from a single-view image to cut 3 of each of the neighbors - like:: therefore, multi-view image pl 〇 4 each pixel two-point operation is shown t The content that the location should present. , which is precisely described with reference to Figure 4, which shows a schematic diagram of the construction process of another embodiment. The fourth image = pixel shift (PlxelRendeHm '2 material _ order (Holejr彳η. N 1 and pixel picking shoes original image P10〗) to form a number of new perspective images P102, after 4P101 and new perspective image P10 'The column program (Vi Dan directly interleaved hew Interlace) A4 to get "〇4' without the resolution adjustment procedure (see the corner image like multi-view image (10), the group image ^^ΚΓ1Ζΐη§) Α3. Program A3, more # P1ru, the pixel has not been subjected to the resolution adjustment multi-view image blue, each pixel P11 〇 pixel directly selected. Therefore, the multi-view early view / pixel can not accurately represent this position = 4 For each image, the mineral tooth phenomenon will appear on the surface. (4) The content presented. So through the anti-S3 Dan Figure 2, the two multi-view image _ each pixel, the household ΐΓ 邻近 邻近 neighboring image (four) floating point arithmetic Each pixel that obtains this view image can accurately represent the content of the presentation, and thus the phenomenon of the tooth on the screen can be referred to the fifth to seventh figures in the embodiment of the embodiment. FIG. Second 201227608 TW6973PA· Shadow Schematic diagram of the construction system of P104, the flow chart of the construction method of the multi-view image of the first embodiment, and the dry soil of the construction process of the multi-view image P1〇4 of the second embodiment: % For example, the multi-view image P104 construction system 2〇 and the embodiment of the multi-view image _ the construction system discussion method ^ the second arranging the execution mode of the thread, the rest of the same place is no longer the same

As shown in Fig. 5, the processing unit 22 of the present embodiment includes a thread 221, a plurality of second threads 222, and a plurality of / 223th. The first thread 221 is used to execute the pixel shifting program Rendering) A1 and the pixel interpolation program (H〇le Fiuing) μ ^ The second thread 222 is used to execute the resolution adjustment program ((4) motion 3, third thread 223 is used to perform the action of the interlace program ^ Interlace) Α 4. w, the following is a detailed description of the construction method of the multi-image P104 of the present embodiment in conjunction with the sixth and seventh figures. In step (4), the unit u is provided with depth information for the original image P101 and the original image P1〇1. Next, in step S202, the first thread 221 of the processing unit 22 performs pixel shifting process and pixel interpolation (4) A2 on the original image in parallel processing according to the depth information to form a plurality of new viewing angle images. P102. In the present embodiment, the pixel shifting (4) A1 and the pixel 姊 program A2 are all performed by the parallel processing in the first thread 221 by using the first thread 221 as an example. The arrow between the original image P101 and the new view image P102 indicates each first execution. 221 executes pixel shifting program A1 and pixel interpolation process 201227608

The action of step S202 of the TW6973PA sequence A2 is such that the first number of children is the product of the number of columns of the new view image P102 and the number of the 'sheets (i.e., 64 〇 x 8). - The new angle image is simplified. Then, in step S203, the second handle A is used to analyze the angle of view, and the resolution is adjusted: A3 to obtain a number of new resolution images ρι〇3. In the case of 22f 2, the resolution adjustment program A3 is executed by the second execution using the second execution. To the first? Figure = image = original image Ρ 101 and the arrow between the object indicates the action of each second execution = sequence A3 'so the image of the second execution _, ρ, =@Λ^^, 3, and (4) resolution The number of sheets is the product of the number of domain channels and the number of these new resolution images (ie 1920x1080x3x9). Next, in step S204, the third thread 2 pushes the silly sound of the new resolution image 3 by 饤4, and arranges the interlaced program ^^〇3 4 by the shape θ, θ ,, ΓΛ?.ΡΙ04°""7 degrees are the same, so ®^ 》成之纽(四) like P1G4's solution rod P103 pick 1 屮 ^ lamp 223 only need to directly in the new resolution image such as, ί should position the pixel A multi-view image pi〇4 can be formed. , arrow 223, F, listen to the second execution (10) trip ^ #View image (10) number of columns, multi-view image number of products and the number of gamut channels (ie 192 〇χ 1 〇 8 〇χ 3). 201227608

TW6973PA, please refer to FIG. 8 to FIG. 1 for the third embodiment, FIG. 8 is a schematic diagram of the construction system 300 of the multi-view image P104 of the third embodiment, and FIG. 9 is a multi-view image P104 of the third embodiment. A flowchart of the construction method, and FIG. 1 is a schematic diagram showing a construction process of the multi-view image P1〇4 of the third embodiment. The construction system 300 and method of the multi-view image P104 of the present embodiment is different from the construction system 1 and method of the multi-view image P1〇4 of the first embodiment in the arrangement of the execution mode of the thread, and the rest are the same. No longer repeat • Retelling. As shown in Fig. 8, the processing unit 32 of the present embodiment includes a plurality of first threads 321 and a plurality of second threads 322. The first thread 321 is configured to perform a Pixel Rendering A1 and a Hole Filling A2 action, and the second thread 322 is configured to perform a Resizing A3 and a staggered program ( View Interlace) The action of A4. The following is a detailed description of the construction method of the multi-view image P104 of the present embodiment in conjunction with FIGS. 9 and 10. In step S301, the providing unit 11 provides depth information of the original image P101 and the original image P101. Next, in step S302, the first thread 321 of the processing unit 32 performs Pixel Rendering on the original image ρι〇 in parallel processing according to the depth information. “And the pixel filling program (Hole Filling) A2, to form a new view image ρι〇2. In the embodiment, the 'pixel shifting program A1 and the pixel interpolation program A2 are all executed in parallel by the first thread 321 . Taking the first image as an example, the original Arrow between image P101 and new perspective image P1022 Table 11 201227608

TW6973PA The first execution executor performs the pixel shifting procedure Ai and the pixel interpolation step S302. The number 2 of the first thread % is the number of columns of the new view image P102 and the number of the new view images. The product of 640x8. Then, in step S, the second threads 322 perform a resolution adjustment procedure, a (ResiZing) A3, and a staggered program (10) (a) for the single-view images ρί1〇 in parallel to obtain directly Multi-resolution image PI04. In this embodiment, the second thread 322 performs the resolution adjustment program A3 only for a part of the pixels of the single-view image # P110, and does not actually complete any new resolution image touch, so the 1G map is indicated by a dotted line. The new resolution image p1〇3. The virtual new resolution image P103 is not complete, only some of the pixels are completed. After these pixels are completed, the interlaced program A4 is directly inserted into the multi-view image ρι〇4. Single-view image P110 (including original image P101 and new view image P102) and multi-resolution scene > image? The arrows between 1 and 4 indicate that the respective second threads 322 perform the operations of the resolution adjustment program A3 and the staggered arrangement S3〇3, so that the number of the second threads 322 is the multi-view image ρι〇4 The number of rows, the number of rows of the multi-view image P104, and the number of gamut channels (ie, 1920x1080x3). The single-view image P110 of the above embodiment is composed of the original image ρι〇1 via the Pixel Rendering and the pixel. Hole Filling A2 is used to obtain the new view image P1〇2. However, in other embodiments, the single view image p11 can be directly taken by several different angles of view without the need for pixels. Translation program A1 12 201227608 TW6973PA* and pixel interpolation program A2. The above embodiment relates to a method and system for constructing a multi-view image, which uses parallel processing to improve the processing speed of multi-view images, and uses a resolution adjustment program to improve the quality of multi-view images. In view of the above, the present invention has been disclosed in various embodiments, and is not intended to limit the present invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, the scope of the invention is defined by the scope of the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a schematic view showing the construction system of the multi-view image of the first embodiment of the present invention. Fig. 2 is a flow chart showing the construction method of the multi-view image of the first embodiment. Fig. 3 is a view showing the construction process of the multi-view image of the first embodiment. FIG. 4 is a schematic diagram showing the construction process of the multi-view image of another embodiment. Fig. 5 is a view showing the construction of the multi-view image construction system of the second embodiment. Fig. 6 is a flow chart showing the construction method of the multi-view image of the second embodiment. Fig. 7 is a view showing the construction process of the multi-view image of the second embodiment. 13 201227608

Schematic diagram of the TW6973PA. Figure 8 is a diagram showing the multi-view image of the second embodiment. Fig. 9 is a flow chart showing the construction of the multi-view image of the third embodiment. Fig. 10 is a view showing a multi-view image of the third embodiment. [Main component symbol description] 100, 200, 300: multi-view image construction system 110: providing unit 120, 220, 320: processing unit 121, 22 321 : first thread 122, 222, 322: second Thread 223: Third Thread Α 1. Pixel Rendering A2 • Pixel Interpolator (jj〇ie pilling) A3: Resizing View Repling plol, Pl 〇i' : original image P102, P102': new view image P103: new resolution image P1() 4 ' P104' : multi-view image P110. single-view image S1〇1 to S1〇4, S201 to S204, S301~ S303: Process steps

Claims (1)

201227608 TW6973PA* Seven-Patent Application Scope: A method for constructing a multi-view image, comprising: providing a plurality of single-view images, the view angles of which are not subjected to a resolution adjustment program esizing) Obtaining at least a portion of the pixels of the plurality of new resolution images, and performing at least a portion of the pixels of the Hurray new resolution image to perform a staggered 2 (VleW Interlace) to form a multi-view image. The method, and the construction of the multi-view image as described in claim 1 of the patent application, wherein the single-view images comprise an original image and a plurality of new viewing angle images corresponding to the original f. The steps include: providing an original image of the original image and depth information of the original image; and performing a pixel shifting process on the original image according to the 4 depth information (PlXel Rendering) and a one-hole interpolation (Hole Filling) to obtain the new perspective images. The construction of the multi-view image as described in claim 2, the pixel shifting program, the pixel interpolation program, and the resolution adjustment process are performed in parallel by a plurality of first-executive processes. The light is executed, and the parent misplaced program is executed in parallel by a plurality of second threads. 4. The method for constructing a multi-view image according to claim 3, wherein the number of the first threads is the number of the new view images and the number of the new view images The product, the number of the second executions 15 201227608 TW6973PA is the product of the number of columns of the multi-view image, = ', and the number of gamut channels. The number of lines of the multi-view image is as follows: 5. The method of the second aspect of the patent application scope is as follows: wherein the pixel shifting program and the constructing of the image of the image of the image of the image of the image are executed in parallel: The plurality of second threads are paralleled by the complex system: the resolution adjustment light sequence program is executed by a plurality of third execution modes, the interlaced lines. The method of parallel processing is the method of claim 5, wherein the number of the first-executors = the product of the multi-view image and the number of the new view images, the original image The number of columns is the number of columns of each new resolution image, each: the number of second executors, the number of gamut channels, and the number of third executors of the image of the resolution resolution image are the multi-view = the product of the number of sheets - the product of the number of lines and the number of gamut channels. The number of columns, the multi-view image 7 as in the patent application _ 2_ method, wherein the pixel translation program and the multi-view-to-image construct a first-execution parallel processing square-block interpolation program uses a complex number: _ Preface, plural (four): Obtain =::r Method 8. Its:: Please have some special procedures and the staggered program. The method of the seventh aspect of the patent application, wherein the number of the plurality of multi-view images is the number of the original image columns 201227608 TW6973PA* and the method The product of the number of sheets of the new perspective image, which is the product of the number of the number of the clever and fresh lines of the multi (four) image. The number of lines of the image and the color gamut pass through a multi-view image construction system. The unit provides a unit, "providing a plurality of sheets of single 1st, the angle of view of the angle image is different; and the image, the single-view-processing unit" For these single-view programs (ezing), it is called the grading adjustment of the pixels, and the new resolution of at least the staggered program (View Τη+, the main 邛 邛 像素 进行 进行 。 lac lac lac lac lac lac lac lac lac lac lac lac ) ) 'To form a multi-view image system, such as the multi-view image as described in the 10th article of the patent application scope=Single-view images including an original image and corresponding to the original == number, perspective The image processing unit performs a one-pixel shifting process (Pixel (Hole Filling) according to the original image of the original image. 12. The construction of the multi-view image as described in the scope of the patent application 糸The processing unit includes: a plurality of first-executives executing the pixel shifting program, the pixel interpolation program and the resolution adjustment program in a parallel processing manner; and a plurality of second threads The staggered program is executed in a parallel manner. 13. The multi-view image construction system described in claim 2, wherein the number of the first threads is the new view image 17 201227608 TW6973PA ' The product of the number of columns and the number of sheets of the new perspective images, the number of the second threads being the number of columns of the multi-view image, the number of rows of the multi-view image, and the number of color gamut channels 14. The construction system of the multi-view image according to claim 11, wherein the processing unit comprises: a plurality of first threads, performing the pixel shifting process and the pixel interpolation in a parallel processing manner a program; a plurality of second threads executing the resolution adjustment program in a parallel processing manner; and a plurality of third threads executing the interleaving program in a parallel processing manner. The multi-view image construction system described in the item, wherein the number of the first threads is the number of columns of the new view images and the new view angles The product of the number of the second threads is the number of columns of the new resolution image, the number of rows of the new resolution image, the number of color gamut channels, and the number of images of the new resolution images. The product of the third thread is the product of the number of columns of the multi-view image, the number of rows of the multi-view image, and the number of color gamut channels. 16. The multi-view as described in claim 11 The image construction system, wherein the processing unit comprises: a plurality of first threads, performing the pixel shifting process and the pixel interpolation program in a parallel processing manner; and a plurality of second threads in a parallel processing manner The resolution adjustment program and the interleaving program are executed. 17. The 201227608 I W6973FA system for multi-view imagery as described in claim 16 wherein the second thread performs the resolution adjustment procedure and the interleaving procedure in the same step. 18. The construction system of the multi-view image according to claim 16, wherein the number of the first threads is the number of columns of the new view images and the number of the new view images. The product, the number of the second threads is the product of the number of columns of the multi-view image, the number of rows of the multi-view image, and the number of gamut channels. 19