KR101625830B1 - Method and device for generating a depth map - Google Patents

Abstract

The present invention relates to a device and method (100) for generating a depth map for an image using monocular information, the method comprising generating (110) a first depth map for an image (205) The one depth map corresponding to an estimate of a global depth profile of the scene depicted in the image; 1. A method (120) for generating a second depth map for an image, the depth value associated with a pixel in a second depth map including depth values of a first depth map in a region spatially near the pixel and color and luminance values Generating the second depth map based on at least one of the first depth map and the second depth map; And generating (130) a third depth map for the image using depth values from the first depth map and the second depth map, wherein the generating includes generating a difference between depth values of the third and first depth maps, And the depth values of the second and first depth maps. The present invention also relates to a computer program product for use in implementing the method according to the invention.

Description

METHOD AND DEVICE FOR GENERATING A DEPTH MAP "

The present invention relates to a method and a device for generating a depth map for an image using monocular information, and a computer program product on a computer readable medium for generating a depth map for an image using monocular information .

Over the past decade, a significant amount of research has focused on the realization of 3D display technology for use in and around the home. As a result, strong winds of stereoscopic and autostereoscopic displays occurred.

In stereoscopic displays, the eyes of a spectator are generally represented, for example, by spectacles or foils located between spectators and in time-multiplexed or concurrent manner (e.g. via spectral separation) A left-eye image of the viewer's left eye, and a right-eye image of the viewer's right eye. In general, users find annoying wearing glasses, and autostereoscopic displays are similarly similarly cautious. Autostereoscopic displays, often multi-view displays, generally allow multiple virtualisations (eg, images or views 5 to 9 or more) that are multiplexed in viewing cones oriented to viewers. By observing the individual views from the cones with the left and right eyes respectively, the autostereoscopic effect is obtained by visual observation.

An important issue for stereoscopic displays and autostereoscopic displays is the delivery of content. Various approaches for delivering 3D content to a display are known. Some of these approaches explicitly encode all views, while others encode one or some views and additional depth and / or disparity information for one or both of these views. An advantage of providing depth information is that, for example, when rendering additional views based on the provided images, manipulation of the three-dimensional content is easy.

This depth information can be obtained, for example, through analysis of the disparity of stereoscopic images or using range finders, but this is generally only possible for newly acquired content. Also, stereoscopic or multi-view acquisitions generally make the cost even more expensive. As a result, considerable research has been directed towards the acquisition of depth information from monocular images, or monocular image sequences. The various applications of these algorithms can be considered from fully automated transformations for high quality content to a range of user assisted 2D to 3D conversions. In the case of a user-assisted 2D to 3D conversion, the computer-aided depth map generation can exhibit significant time savings.

Examples of approaches to acquiring a depth map from a monocular image are three-dimensional image acquisition and application VI-Vol. 5302-13, " Depth Map Generation by Image Classification "by S. Battiato et al., Published by SPIE Electronics Imaging 2004.

In the above paper, the depth map is based on the estimated global depth profile of the image, which is then combined with an additional depth map that includes the image structure. However, the resulting combination does not always provide a satisfactory depth perception in the final image.

Accordingly, it is an object of the present invention to provide an alternative method of providing an alternative depth map.

Wherein the method is achieved by a method for generating a depth map for an image using monocular information, the method comprising generating a first depth map for an image, wherein the first depth map comprises a global Generating the first depth map corresponding to an estimate of the depth profile; Wherein the depth value associated with a pixel of the second depth map is greater than the depth values of the first depth map in an area that is spatially near the pixel and at least one of the color and luminance values of the image within the area Generating the second depth map based on the second depth map; And generating a third depth map for the image using depth values from the first depth map and the second depth map, wherein the generation is such that the difference between the depth values of the third and first depth maps is greater than And the depth value of the first depth map to scale the depth difference to be scaled.

The generation of the third depth map is based on the two underlying depth map components (first and second depth maps) and both are based on an estimated global depth profile based on this first depth map . In particular, when the image is an image from an image sequence, this global depth profile is likely to be similar to the neighboring images of the image sequence, thus contributing to the temporal stability of the third depth map.

The second depth map generation then effectively integrates the local structure into the global depth profile. During generation of the second depth map, brightness and / or color similarity of the image is used to be assigned to the depth values. As a result, the depth values of objects with similar color and / or brightness in the image are made locally more similar so that structures having similarity in the image are generally more similar and often have a depth that is noticeable from the global depth profile Values.

Note that the similarity mentioned above is quantified as being within a predetermined variation of color and / or luminance values.

Finally, the third step combines the first and second depth maps generated in a manner that effectively scales some of the differences between the first and second depth maps. In one embodiment according to the present invention, this allows the introduction of more declarative differences between the local structure and the global depth profile. In this way, some subtle differences between the first and second depth maps can often be improved for the object, resulting in a more dynamic depth map. Alternatively, the local structures of the second depth map may be overly declarative, in which case scaling with a factor between 0 and 1 may improve the 3D viewing experience.

Note that when generating the depth values for the pixels of the second depth map, the depth values from the first depth map are used in the area around the corresponding pixels of the first depth map. Preferably, depth values of these approximate depth values are based on color and / or luminance similarity, even if other image attributes, such as, for example, image motion, local edginess, and / . In this manner, the depth values associated with pixels of similar color and / or brightness contribute more to the depth values of the pixels of the second depth map than less similar pixels.

Here, 'cool' is intended to imply local rather than global. It is not based on the whole image. Indeed, 'near' is often associated with pixels within the aperture. Although the hole may have a predetermined shape and / or size, it may also be possible to dynamically adapt the hole, for example, based on additional image information.

In an embodiment, the first depth map is based on a depth map template. Such a template can be a simple general template, such as a slant, i.e., a gradient from the foreground to the upper background at the bottom of the scene. An alternate slope may include a horizontal line at a constant vertical position that defines the beginning of the background. Alternatively, complex parameterizable templates such as those disclosed in "Depth Map Generation by Image Classification ", by S. Battiato, incorporated herein by reference and cited above, may be constructed. Here, an example is presented as to how a parameterizable global depth profile can be constructed based on image analysis. Generally speaking, the depth map template is made up of estimates of the depth profile of a particular type of scene, and thus can exhibit significant variations; For example, consider differences between conventional depth profiles of indoor dialogue scenes or fish-eye close-ups.

In a preferred embodiment, the generation of the second depth map comprises the application of a bilateral filter to one of the color and luminance values of the image and the depth values of the first depth map. As a result, there is no need to fragment the image, and concurrent color and / or luminance variations in the image can be used to prevent depth values from one side of the color transition from contributing to depth values of pixels on the other side of the color transition. As a result, depth values of the second depth map corresponding to regions having a particular color and / or luminance similarity will generally have depth values that are more similar than corresponding regions of the first depth map. As a result, the objects will become more pronounced from the global depth profile.

In a further preferred embodiment, the depth value for the third depth map is set to a depth value corresponding to the depth closest to the viewpoint of the scene,

- a corresponding depth value of said first depth map; And

- a candidate depth value based on a corresponding depth value from the second depth map. In practice, the depth value for the third depth map is typically set to the maximum value of the depth value of the first depth map and the depth value of the candidate based on the corresponding depth value from the second depth map. As a result of this particular measure, the depth values of the third depth map will not be placed after the global depth profile. As a result, the objects (portions thereof) placed after the global depth profile in the second depth map will be set to the global depth profile. It will be noted that additional candidate depth values are envisaged.

In advantageous embodiments, the above-mentioned candidate values include adding the depth value from the first depth map to the depth value from the first depth map and the weighted difference of the corresponding depth value from the second depth map. As a result, improvements can be coordinated and even possible under user / spectator control. Alternatively, the candidate value may include a threshold depth value based on one or more depth values that are spatially close to the depth value from the first depth map. In this way, depth values for objects with a particular color or brightness can be further improved.

In an embodiment, the image from which the depth map is generated is part of the image sequence. As a first approximation, the image sequence can be divided into screens, and subsequently the global depth profile can be selected based on images of the screen. In this way, the consistency of the generated depth maps can be further improved.

For example, in a further improved method that can be particularly advantageously combined with parametatable depth map tablets, the global depth profile may be determined based on one or more images of the scene, such as the first and last images on the screen, It is defined that there are additional variations in the scene on a large number of images / frames.

Although the above embodiments may be particularly advantageous, embodiments in which the global depth profile is selected based on all images / frames of the image sequence separately, or through evaluation of the last global depth profile and the current image / frame used more selectively And the like.

It is an object of the present invention to provide an alternative device which further provides an alternative depth map.

Wherein the above is achieved by a device for generating a depth map for an image using monocular information, the device comprising: receiving means configured to receive an image; And processing means for generating a first depth map for the image, wherein the first depth map may correspond to an estimate of a global depth profile of the scene shown in the image, (A depth value associated with a pixel of the second depth map is based on at least one of depth values of a first depth map in an area that is spatially near the pixel and color and luminance values of the image in the area) And generate a third depth map for the image using depth values from the first depth map and the second depth map, the generation being such that the difference between the depth values of the third and first depth maps is greater than the second and third depth maps, And scales the depth difference to be scaled by comparing the corresponding difference between depth values of the first depth map.

This is further achieved by a computer program product on a computer readable medium for generating a depth map for an image using monocular information.

These and other aspects, features, and advantages of the present invention will become apparent and appreciated by referring to the embodiment (s) described hereinafter.

Embodiments of the present invention will now be described by way of example with reference to the drawings.

1 is a flow chart of a method according to the present invention;
2 is a block diagram of a device according to the present invention;
3A shows vertical transverse sections of first and second depth maps;
Figure 3B shows a vertical transverse section of two depth maps.
Figure 3c shows a vertical transverse section of three depth maps;
Figure 3D shows a vertical transverse section of three depth maps.
Figure 4 shows several estimates of the global depth profile of a scene.
Figure 5 shows the output of several steps of the method according to the invention.

As described above, the generation of depth maps for an image using monocular information is more complicated than when using stereoscopic images. In the case of stereoscopic or multi-view images, it is often possible to use the disparity between each view of the scene to obtain depth information about the scene shown in the views.

Although stereoscopic and extended multi-view acquisition means are made available, there is still an additional cost associated with using such means generally as compared to monocular acquisition means. Also, in existing monocular content, it may not be possible to often re-capture in a double-view, i.e. stereo or multi-view format.

As a result, there is a need for methods for generating a depth map of an image using monocular information.

Figure 1 shows a flow diagram of a method 100 according to the present invention. The method 100 illustrates a method for generating a depth map for an image using monocular information. The method includes a first step of generating (110) a first depth map for an image. During the first step, a depth map, typically a depth map for all pixels of the image, is generated, wherein the depth values of the first depth map correspond to an estimate of the global depth profile of the scene shown in the image.

In " Depth Information by Stage Classification " by V. Nedovic et al. At IEEE 11th International Conference on Computer Vision, ICCV 2007, 2007, incorporated herein by reference, images of five global and eleven or more specific depth stages And a method of mapping images to these depth stages has been proposed. The global stages correspond, for example, to a first stage that includes sky, background, and ground, such as a landscape screen. The second stage typically includes a box suitable for indoor scenes. A third stage, called a corner stage, is typically suitable for indoor scenes. The fourth stage for one or more people in the background and finally the fifth stage include scenes that do not have depth.

It should be noted that the classification used in the above paper is particularly advantageous, but not expended, and can easily be extended by the skilled person. For example, the model presented in the paper does not provide a global depth profile corresponding to micro-screens created using fisheye lenses.

Figure 4 provides several examples of depth maps of the global depth profile that can be used for certain images. Here, the image 410 provides a gradient of the foreground of the lower portion to the background of the upper portion of the image. It will be apparent to those skilled in the art that it may be advantageous to use this profile to refine the profile by generally detecting the horizontal and then placing all pixels horizontally above the background. Alternatively, it may be advantageous to create a landscape with more ominous clouds. In the latter case, an image 420 containing two vertical gradients can be used. As a result, clouds at the top of the image are positioned closer to the viewer than they are at the horizon.

This time, the image 440 shows a depth map for a global depth profile for a fisheye micro-picture comprising a circular gradient with circles having white center and darkening towards the edge of the image. As a result, the center of the image is closest, and the edges of the image are farthest away from the point of view of the image. Finally, the image 450 provides a depth map for a simple box similar to the scene previously viewed.

Referring once again to FIG. 1, after creation of the first depth map, the next step corresponds to creating (120) a second depth map for the background. In the second depth map, the depth map associated with the pixel is based on the following three:

Depth values of a first depth map in an area that is spatially near the pixel; And

Color and / or luminance values of an image in the region;

Note that the second depth map is explicitly based on the color and / or luminance information of the first depth map and the image. As a result, the second depth map typically allows the distinction of objects with distinct colors and / or brightness from the global depth profile.

In a preferred embodiment, the generation of the second depth map involves a bidirectional filter, for example, on the one hand, the depth values of the first image and, on the other hand, the color information of the image.

To add a local structure to the depth map, crossed or joint bidirectional filters may be used.

Bidirectional filters are non-linear filters that use both (spatial) domain and (intensity) range properties. The following equation (1) shows a basic bidirectional filter. Output image O _P is determined by a weighted average of the pixels in the positions q from input image I _q near (S) of the position (P). The weighted calculation depends on the spatial distance function (s) and the intensity range function (r). Where the function (s) is a conventional 2D convolution filter kernel. The function r then decreases to larger intensity differences to deliver edge preservation properties.

(1)

(One)

이다. here,

to be.

Preferably, the functions s and r are implemented as indicated below in equations (2) and (3).

(2)

(2)

And

(3)

(3)

As noted above, a particular advantageous method of creating local depth information based on range information from the color and / or luminance information from the second images is to apply a crossover or conjugate bidirectional filter. In an embodiment of the present invention, a second depth map is generated using such a junction bidirectional filter. The conjugate bidirectional filter is used to filter the first depth map representing an estimate of the global depth profile of the scene using range information (color and / or luminance) from the input image.

Equation (4) shows how the second depth map D2 _p can be generated using the first depth map D1 _p , which represents an estimate of the depth profile of the scene, and the image content from the image I .

(4)

(4)

이다. here,

to be.

Although a particularly effective method of generating a second depth map has been provided above, the second depth map may be generated in other manners known to those skilled in the art.

Durand에 의한 "A Fast Approximation of the Bilateral Filter using a Signal Processing Approach"에 기술된 바와 같이, 더 높은-차원의 쌍방 그리드 도메인(Bilateral Grid domain)에서 공식화될 수 있다. 이 공식화는 거친 3D 그리드를 이용하여 데이터를 다운샘플(downsample)링하고, 이에 따라 매우 효율적으로 구현될 수 있다. 이 접근은 제 3 깊이 맵을 생성하기 위한 스케일링 동작(scaling operation)이 바람직한 실시예에서 업샘플링(upsampling) 이전에 다운샘플링된 3D 그리드에서 실행될 수 있기 때문에 여기에 개시된 방법과 유리하게 조합될 수 있다. 위의 논문에서 Paris 및 Durand는 다운샘플링 자체가 양방향 필터에 대해 완료될 수 있다는 것을 도시한다. 본 발명에 적용될 때, 이는 제 1 및 제 2 깊이 맵이 더 낮은 해상도로 생성될 수 있다는 것을 암시한다. The bidirectional filter is preferably implemented in Sylvain Paris, published in the Bulletin of the European Conference on Computer Vision 2006,

Can be formulated in a higher-dimensional bilateral grid domain, as described by Durand in "A Fast Approximation of the Bilateral Filter using a Signal Processing Approach ". This formulation downsamples the data using a coarse 3D grid, and thus can be implemented very efficiently. This approach can be advantageously combined with the method disclosed herein because a scaling operation for generating a third depth map can be performed in a down-sampled 3D grid prior to upsampling in a preferred embodiment . In the above paper, Paris and Durand show that the downsampling itself can be completed for the bidirectional filter. When applied to the present invention, this implies that the first and second depth maps can be generated with lower resolution.

Referring again to FIG. 1, following generation of the second depth map, the method generates 130 a third depth map for the image using depth values from the first depth map and the second depth map , So that the difference between the depth values of the third and first depth maps is compared to the corresponding difference between the depth values of the second and first depth maps.

This is once more visible in Figures 3a to 3d. 3A shows the vertical transverse section 10 of the first depth map and the vertical transverse section of the second depth map 20. FIG. As can be seen, the first depth map corresponds to a slope. This time, the second transverse portion follows a certain degree in the first depth map, but corresponds to a locally deviated curve. This second curve is an exemplary curve that can be generated using, for example, a bidirectional filter as described above.

(5)에 대응하는 수직 횡단 부(30)의 각 깊이 값들에 대응한다. Bidirectional filters are particularly advantageous, but sparse local ordering provided by, for example, T-intersection points can be achieved by using T-intersection points that can be integrated into a dense depth profile using values from a first global depth map Other approaches are foreseen, such as through the use of subsequent segmentation of the input image. Figure 3b again shows the vertical transverse section 30 corresponding to the maximum vertical (in the z-direction) above the vertical transverse section 10 and the transverse sections 10 and 20 according to the invention. Here, the depth values d3 (y)

Corresponds to the respective depth values of the vertical transverse section 30 corresponding to the vertical cross section 5 of FIG.

Where d1 (y) corresponds to each depth value of the transverse section 10,

d2 (y) corresponds to the respective depth values of the transverse section 20. [

3C shows a fourth vertical transverse section 40 according to the present invention, on the basis of the transverse section 30. Fig. The transverse section 40 is formed in the transverse section 30 so as to clarify that points on the transverse section 40 in front of the inclination (transverse section 10) are located twice as far from the incline by corresponding points on the cross section 30. [ ≪ / RTI >

(6)

(6)

By choosing an appropriate value for [omega], the depth differences can be further emphasized or reduced. When creating a depth map, this parameter allows the user to improve depth map variations based on local color and / or luminance variations. As a result, the depth map of the image will be better able to illustrate the proper correlation with the image information, and as a result will provide a more similar appearance, while the lower values of omega will provide a depth map Lt; / RTI >

It will be apparent to those skilled in the art that the choice as to whether to amplify the difference or to dampen the difference mainly depends on the nature of the second depth map. If the second depth map shows a significant difference from the first depth map, then it may be necessary to attenuate the differences. On the other hand, when the difference is typically small, amplification of the difference may be appropriate.

3D furthermore shows a fifth vertical transverse section 50 according to the invention. The fifth transverse section then shows that depth values for points on the transverse section 40 before the slope are set to the threshold value d2 _T (y). Where the threshold is determined by a vertical scan based on color and / or luminance similarity. In the case of facing a portion with a significantly higher depth map that is noticeable from the background of color and / or luminance during this vertical scan (from bottom to top), the threshold is based on the depth values encountered in the scan, Lt; / RTI > This portion (in the scan direction) above and below the depth values typically reflects the global depth profile present in the first depth map (i.e., no other portions exist). This process is also visualized in 3D by extending depth values along the background tilt to the transverse portion 50 in the upper right hand side. Here, "discontinuity" of the depth values is shown, which corresponds to the color and / or luminance discontinuity of the image.

(7)

(7)

As shown above, depth map generation is more efficient by formulating a bidirectional filter that is used to generate a second depth map in a higher-dimensional bilateral grid domain. However, this particular implementation additionally provides additional advantages; This particular formulation of the bidirectional filter provides additional separation in the additional luminance (or color) dimension. This then implies that a single scan may be used as illustrated in Figure 3D, where each portion having luminance (or color) similarity may be isolated and processed. As a result, this particular implementation provides a bonus effect because it no longer needs to explicitly detect any discontinuities.

The method from Figure 1 is further illustrated in Figure 5. Figure 5 shows a series of image portions that are processed using the method according to the present invention. The image 510 includes a portion of the input image that corresponds to a scene view of some icebergs, the lower portion of the image 510 corresponds to the foreground, and the upper portion of the image corresponds to the background.

A second depth map based on the slope is provided in the image 520 and the slope is subsequently filtered using a bidirectional filter using both the first depth map and the luminance of the image. Subsequent images 530 provide intermediate values corresponding to the maximum of the first and second depth maps to the corresponding images.

To further improve the depth impression, the present invention proposes to improve the differences between the first and second depth maps (here by factor 2) as in the example shown in FIG. 3C. The resulting image 540 shows some improvement in that the portions of the icebergs are in front of the sky behind the image 530, more specifically.

Image 550 shows the same procedure improved by factor 4. As a result, the depth impression of foreground objects is further improved.

Figure 2 shows a device 200 for generating a depth map for an image 205 using monocular information in accordance with the present invention. The device 200 comprises receiving means 210 configured to receive an image. It will be appreciated by those skilled in the art that for the sake of clarity, an example is described with reference to the generation of a depth map for only a single image, but can be modified for the generation of depth maps for sequences of images. In fact, the device according to the invention is preferably a device suitable for 2D to 3D conversion of monocular image sequences into a three-dimensional video signal comprising, for example, parallel streams of image information and depth information.

The device 200 further comprises a processing means 220. Such processing means may be implemented in a variety of ways, generally using, for example, a general purpose processing platform, using digital signal processors and / or using application specific integrated circuits. The processing means 220 is configured to generate a first depth map for the image 205. The first depth map corresponds to an estimate of the global depth profile of the scene shown in the image.

The processing means is further configured to generate a second depth map for the image. Although the resolution of the depth map is subsampled relative to the resolution of the image 205 in certain embodiments of the present invention, the depth values of the second depth map are typically associated with a single pixel of the image.

The depth value of the second depth map associated with a particular image pixel is based on the depth values of the first depth map in an area that is spatially near the image pixel. It is also based on the color or luminance values of the imaged figures within the area. As a result, the second depth map may include local depth information compared to global depth information present in the first depth map.

The processing means is further configured to generate a third depth map (215) for the image (205) using depth values from the first depth map and the second depth map. When generating the depth values of the third depth map, subtle depth differences between the first and second depth maps are scaled. As a result, certain differences between specific depth values of the third depth map and corresponding depth values of the first depth map are scaled compared to corresponding differences between depth values of the second and first depth maps.

Optionally, the device 200 may further include additional software / hardware means for handling the third depth map.

For example, the third depth map 215 may be further processed subsequently in the device 200, or alternatively may be stored on the storage means 230 for later use. Alternatively, the third depth map may preferably be encoded by the encoding means 240 with the image 205 into a three-dimensional image signal. This encoded image signal may then be stored on the storage means 230 or transmitted by a transmission means 250 for transmission by cable or RF using a protocol such as, for example, an IP protocol .

Alternatively, the third depth map may serve as an input, preferably with the image 205, for rendering one or more views by the rendering means 260. The rendered views may then be output on a three-dimensional (auto) stereoscopic display means 270.

It will be appreciated that for clarity, the above description has described embodiments of the present invention with reference to various functional units. However, it will be appreciated that any suitable distribution of functionality between different functional units (or processors) may be utilized without departing from the invention. For example, the functionality illustrated to be executed by individual processors or controllers may be executed by the same processors or controllers. Hence, references to specific functional units should be understood only as references to suitable means of providing the described functionality, rather than indicating a strictly logical or physical structure or configuration.

The invention may be implemented in any suitable form including hardware, software, firmware, or any combination thereof. The present invention may optionally be at least partially implemented as computer software for driving one or more data processors and / or digital signal processors. The elements and components of embodiments of the present invention may be implemented physically, functionally, and logically in any suitable manner. In practice, functionality may be implemented as a single unit, as a plurality of units, or as a subset of functional units. As such, the present invention may be implemented as a single unit or may be physically and functionally distributed between different units and processors.

While the invention has been described in connection with certain embodiments, it is not intended to be limited to the specific forms disclosed. Rather, the scope of the present invention is limited only by the appended claims. Additionally, it will be appreciated by those skilled in the art that although the features may appear to be described in connection with the specific embodiments, various features of the described embodiments may be combined in accordance with the present invention. In the claims, the term " comprising " does not exclude the presence of other elements or steps.

Also, a plurality of individually listed means, elements, or method steps may be implemented by, for example, a single unit or processor. Additionally, although individual features may be included in different claims, they may be advantageously combined, and the inclusion in different claims does not imply that the combination of features is not practical and / or not advantageous. The inclusion of features within one category of claims does not imply a limitation to this category, but rather indicates that this feature is suitably equally applicable to other claim categories. In addition, the order of features in the claims is not intended to imply any particular order in which the features must necessarily be operated, and in particular the order of the individual steps of the method claim does not imply that the steps must be performed in this order. Rather, the steps may be performed in any suitable order. Also, the singular reference does not exclude the plurality. Thus, references to an object's "a", "an", "first", "second", etc. do not exclude pluralities. The reference numbers in the claims are provided merely for clarifying the examples and should not be construed as limiting the scope of the claims in any way.

200: device 210: receiving means
220: Processing means 230: Storage means
240: Encoding means 250: Transmission means
260: Rendering means
270: auto stereoscopic display means

Claims (16)

A method (100) for generating a depth map for an image (205) using monocular information comprising:
Generating a first depth map for the image (205), the first depth map comprising a global depth profile (410, 420, 440, 450); generating (110) the first depth map;
(120) for a second depth map (520) for the image (205), wherein a depth value associated with a pixel of the second depth map is greater than a depth of the first depth map Generating (120) the second depth map (520) based on at least one of the values and the color and luminance values of the image in the region; And
(130) a third depth map (530, 540, 550) for the image (205) using depth values from the first depth map and the second depth map,
Wherein the generating the third depth map comprises:
- determining a depth difference between a first depth value of the first depth map and a second depth value of the second depth map, and
Scaling the depth difference so that a difference between depth values of the third and first depth maps is attenuated or amplified relative to a corresponding difference between depth values of the second and first depth maps, A method (100) for generating a depth map for an image (205) using monocular information, comprising combining a first depth value. The method according to claim 1,
Wherein the depth differences are scaled for a plurality of pixels in a spatial region having at least one of color and luminance similarity. The method according to claim 1,
Wherein the depth difference is determined between the first depth value and a maximum value of the first depth value and the second depth value. The method according to claim 1,
Wherein the generating of the second depth map comprises applying the bilateral filter to one of the color and luminance values of the image and the depth values of the first depth map, A method (100) for generating a depth map. The method according to claim 1,
Wherein the depth value for the third depth map is set to a depth value corresponding to a depth closest to the viewpoint of the scene,
- a corresponding depth value of said first depth map; And
- selected from one of candidate depth values based on corresponding depth values from said second depth map. ≪ RTI ID = 0.0 > 100. < / RTI > 6. The method of claim 5,
Wherein the candidate depth value is a value obtained by adding the depth value from the first depth map to the depth value from the first depth map plus the weighted difference of the corresponding depth value from the second depth map A method (100) for generating a depth map for an image (205). The method according to claim 6,
Wherein the candidate depth value is a critical depth value based on depth values that are spatially close to a depth value from the first depth map. 8. The method according to any one of claims 1 to 7,
Wherein the image is part of an image sequence and the estimate of the global depth profile is the same for all images in a shot. 8. The method according to any one of claims 1 to 7,
A method (100) for generating a depth map for an image (205) using monocular information, wherein the image is part of an image sequence and the estimate of the global depth profile is determined based on one or more images of the shot. A device (200) for generating a depth map for an image (205) using monocular information comprising:
Receiving means (210) configured to receive the image; And
Processing means 220,
Wherein the processing means comprises:
- generating a first depth map for the image, the first depth map corresponding to an estimate of a global depth profile (410, 420, 440, 450) of the scene shown in the image (205)
- generating a second depth map (520) for the image (205), wherein a depth value associated with a pixel of the second depth map is greater than a depth value of a first depth map in an area spatially close to the pixel, Based on at least one of the color and luminance values of the image,
- processing means configured to generate a third depth map (530, 540, 550) for the image (205) using depth values from the first depth map and the second depth map,
Creating the third depth map
Determining a depth difference between a first depth value of the first depth map and a second depth value of the second depth map; and
Scaling the depth difference so that a difference between depth values of the third and first depth maps is attenuated or amplified relative to a corresponding difference between depth values of the second and first depth maps, A device (200) for generating a depth map for an image (205) using monocular information, comprising combining a first depth value. 11. The method of claim 10,
Wherein the processing means is configured to scale the plurality of pixels of the spatial region with depth differences of at least one of color and luminance similarity, the device (200) for generating a depth map for the image (205) ). 11. The method of claim 10,
The processing means is set to a depth value corresponding to a depth whose depth value for the third depth map is closest to the viewpoint of the scene,
- a corresponding depth value of said first depth map; And
- a device (200) for generating a depth map for an image (205) using monocular information configured to be selected from candidate depth values based on corresponding depth values from the second depth map. 13. The method of claim 12,
Wherein the processing means is configured to add the depth value from the first depth map to the depth value from the first depth map and the weighted difference of the corresponding depth value from the second depth map, A device (200) for generating a depth map for an image (205) using monocular information. 14. The method according to any one of claims 10 to 13,
A device (200) for generating a depth map for an image (205) using monocular information, wherein the image is part of an image sequence and the estimate of the global depth profile is the same for all images in a shot. A computer-readable recording medium recorded with a computer program for generating a depth map for an image using monocular information,
Wherein the program comprises instructions for executing the method steps according to any one of claims 1 to 7. delete

Images