KR20090114707A - Camera and method for compounding picked-up image - Google Patents

Abstract

PURPOSE: A camera and a photographed image synthesizing method are provided to synthesize and photograph a group picture naturally by providing a guide image in real time when a photographer takes a picture of a background after he/she photographs the same background. CONSTITUTION: A photographed image synthesizing method comprises the following steps of: analyzing the first and second images which are previously photographed when entering an image synthesizing mode(S110); separating a person image from one of the first and second images according to the analyzed result(S120); and providing a guide image which induces the photographed image synthesizing by overlaying the separated person image with one of the first and second images which is not separated(S130).

Description

Camera and method for compounding picked-up image}

The present invention relates to a camera for storing a video image obtained by photographing a subject.

Due to the expansion of digital cameras and the increase in leisure life, taking pictures with digital cameras is increasing. The camera user must bring a tripod with the camera in order to take a picture of himself, but it is inconvenient and cumbersome to carry the tripod all the time.

On the other hand, if you do not bring a device such as a tripod, ask someone to take a picture. However, in this case, it is difficult to take a picture of the user under the background and composition intended by the user due to the difference in the position or the time of the photographer. Furthermore, if there is no one to ask for shooting, it is impossible to take a group picture that includes the photographer.

The present invention proposes a camera and a photographing image synthesizing method capable of naturally synthesizing a group photo.

That is, the present invention proposes a camera and a method for synthesizing a photographed image, which can be synthesized and photographed naturally in a group photograph by providing a guide image in real time when the photographed person is photographed by the photographer in the same background.

More specifically, in the photographed image compositing method according to an aspect of the present invention, when the image compositing mode is entered, analyzing the first image captured by the camera and the second image captured by the camera on the same background as the first image; According to the result, the step of dividing the person image from either the first image or the second image and overlaying the divided person image on either the first image or the second image in which the person image is not divided to induce the synthesis of the captured image. Providing a guide image.

On the other hand, the camera according to another aspect of the present invention is an image analyzer for analyzing the first image and the second image that is captured by the camera on the same background as the first image when entering the image composition mode, according to the analysis result An image divider for dividing a person image from either the first image or the second image and an image for overlaying the divided person image on either the first image or the second image in which the person image is not divided to provide a guide image. It includes a providing unit.

Hereinafter, with reference to the accompanying drawings will be described embodiments of the present invention; In the following description of the present invention, if it is determined that detailed descriptions of related well-known functions or configurations may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. In addition, terms to be described below are terms defined in consideration of functions in the present invention, which may vary according to intention or custom of a user or an operator. Therefore, the definition should be made based on the contents throughout the specification.

1 is a configuration diagram of a camera 1 according to an embodiment of the present invention.

Referring to FIG. 1, the camera 1 may include an image analyzer 10, an image divider 20, and an image provider 30, and may further include an image synthesizer 40.

The camera 1 according to an embodiment of the present invention generates a group photograph image including a photographer from one background. To this end, the camera 1 provides a guide image so that when one photographer photographs another person, he / she can photograph the photographer again on the same background. The guide image is provided in real time on the display screen in a translucent form so that the image captured on the same background and the image to be photographed can be synthesized.

First, throughout the present specification, it is assumed that the photographer photographs the first image on a specific background through the camera 1. After that, it is assumed that the photographer hands over the camera to the person included in the first image photographed by the photographer and asks for his own shooting in the same background. At this time, the second image currently captured by the camera 1 enters the image combining mode in a shooting standby state provided on the screen. The image combining mode may be performed by an operator pressing an image combining mode button.

The image analyzer 10 analyzes the first image and the second image when the image combining mode is entered. The first image is a pre-photographed image, and the second image is a real time image captured by the camera on the same background as the first image. The image information representing the characteristics of the first image and the second image may be extracted and analyzed, respectively. A detailed description of the image analysis of the image analyzer 10 will be described later with reference to FIG. 2.

The image divider 20 divides the person image from either the first image or the second image according to the result analyzed by the image analyzer 10. A detailed description of the division of the person image will be described later with reference to FIG. 3.

The image providing unit 30 overlays the image of the person divided by the image dividing unit 20 on either the first image or the second image in which the person image is not divided, and in real time, the guide image for inducing the synthesis of the captured image. to provide. Here, the first image may be a background image, and the person image divided from the second image may be a foreground image to be overlaid on the first image. Alternatively, the second image may be a background image, and the person image divided from the first image may be a foreground image to be overlaid on the second image. The background image is the background image when compositing, and the foreground image is the person image synthesized with the background image.

When the photographer photographs the guide image provided through the image providing unit 30, the image synthesizing unit 40 synthesizes the divided person image and the first image or the second image in which the person image is not divided at the position where the guide image is provided. do. Accordingly, it can be synthesized into a natural group photo including the photographer.

2 is a flowchart illustrating a method of synthesizing photographed images according to an embodiment of the present invention.

Referring to FIG. 2, when the camera enters the image combining mode (S100), the camera analyzes a first image captured in advance and a second image captured by the camera on the same background as the first image (S110).

In more detail, image information representing respective features is extracted and analyzed from the first image and the second image. The image information is information representing a feature of the image and includes feature information (feature or descriptor) for at least one interested point included in the image. The feature point refers to a point from which stable feature information can be extracted even if a view or scale of a certain image changes. In order to extract the feature points, a method such as Harrison, Hessian corner detection, or Difference of Gaussian may be used. The feature information on the detected feature point may be generated by a method such as a generalized color moment or a scale invariant feature transform (SIFT). The aforementioned methods may be used to align the first image and the second image, which will be described later in FIG. 3.

In addition, the first image and the second image may be analyzed by using image analysis techniques such as blob analysis or feature tracking. A blob is a group of adjacent pixels with the same characteristics in an image.Blob analysis counts the desired objects in an image to obtain the number of objects in a given area, and calculates statistics such as size, average, and variance of each length or area. It is an analysis method that automatically obtains. The blob analysis or feature tracking described above may be used to generate a portrait mask, which will be described later with reference to FIG. 3.

Meanwhile, the person image is divided into one of the first image and the second image according to the analyzed result (S120). A detailed description of the person image segmentation will be described later with reference to FIG. 3.

Subsequently, the divided person image is overlaid on either the first image or the second image in which the person image is not divided to provide a guide image (S130). The guide image for inducing the synthesis of the captured images is provided in real time on the display screen in a translucent form so as to synthesize the first image and the second image. In this case, the guide image may be generated by using pixel averaging.

Furthermore, when photographed from the photographer using the guide image provided in real time, the first image or the second image in which the divided person image and the person image are not divided are synthesized at the position where the guide image is generated (S140). Such compositing can be implemented through pixel replacement. The synthesized image may be displayed on the camera screen or may be stored and read and displayed in a later playback mode.

Accordingly, even without a cumbersome and difficult compositing operation using a separate program such as Photoshop, a group photo is generated by synthesizing a first image and a second image on the same background using only two photographing methods through the method according to an embodiment of the present invention. can do. This method is a way to create a group photo naturally based on the same background without dividing the image perfectly.

3 is a flowchart illustrating a method of dividing a person image of FIG. 2 in detail.

Referring to FIG. 3, a third image in which the first image and the second image are aligned according to the analyzed result is generated (S122). In this case, when the first image is set as the background image, the second image is aligned with the first image. However, when the second image is set as the background image, the first image is aligned with the second image. The reason why the third image is generated by arranging each image is because the background of the first image and the second image are taken by different photographers, and thus the background may be changed according to the shooting position or the shooting angle.

When the first image and the second image are aligned, the camera extracts feature points through the Harrison, Hessian corner detection, or DoG method described above with reference to FIG. 2, and uses a generalized color moment or SIFT method. The feature information of the extracted feature point can be obtained, but various embodiments other than the above-described method are possible.

The extracted first feature point and the feature information may be used to compare and analyze the first image and the second image. In the analysis, point information is performed on each feature point of the first image and each feature point of the second image by calculating the disparity information between the first image and the second image. Point matching may use NCC (Normalized Cross Correlation), Correlation Coefficient (Correlation Coefficient), and the like. The affine transform can be used to calculate the disparity information between the feature points matched with the point match. The disparity information includes a change amount, a movement amount, and a predetermined axis, such as enlargement or reduction, between corresponding feature points. It may include at least one information of the rotation amount for. Here, the camera may align the first image to the second image or the second image to the first image by using the calculated disparity information.

Subsequently, the camera divides and generates the portrait masks through a difference operation between the background image and the generated third image (S124). The background image is either the first image or the second image in which the person image is not divided. The difference operation is an operation of separately generating regions having mutual differences among the background image and the third image.

In operation S126, the camera detects a face region from the generated third image and extracts a face mask corresponding to the detected face region from among the segment masks. The face region may be detected using an AdaBoost method, a neural network based method, or a statistical method, but various other methods may be performed. Extraction of the person mask corresponding to the detected face region among the divided person masks may use the blob analysis described above with reference to FIG. 2. The blob analysis may include a maximum blob and a labeling method for each blob.

Subsequently, the person image is divided through an AND operation of the extracted person mask and the generated third image (S128). The AND operation is a method of extracting an image commonly included in the extracted person mask and the third image. The person image extracted in this way is the foreground image to be combined with the background image. The foreground image is combined with the background image to generate the guide image described above with reference to FIG. 2 in real time. In this case, when the photographer photographs using the generated guide image, the group picture combining the first image and the second image may be generated.

Hereinafter, a process of synthesizing photographed images according to various embodiments of the present disclosure will be described with reference to the accompanying drawings. In this case, a description will be given of a first embodiment in which a first image captured first is a background image and a second embodiment in which a second image captured later is a background image.

4A to 4D are reference diagrams for explaining a process of synthesizing photographed images according to an embodiment of the present invention.

4A to 4D are diagrams illustrating a process of synthesizing photographed images when a first image photographed first is a background image and a person image segmented from the second image is a foreground image to be overlaid on the first image.

4A to 4D, when the camera enters the image combining mode, the camera analyzes a first image (FIG. 4A) and a second image (FIG. 4B) that are previously photographed. The second image (FIG. 4B) is a real time image captured by the camera on the same background as the first image (FIG. 4A).

The person image is divided from the second image (FIG. 4B) according to the analyzed result. Subsequently, the divided person image is overlaid on the first image (FIG. 4A) to provide a guide image (FIG. 4C) for inducing the synthesis of the captured images. To this end, the pixel averaging method described above in FIG. 2 may be used. The guide image (FIG. 4C) is provided in real time on the display screen in a translucent form so as to synthesize the first image (FIG. 4A) and the second image (FIG. 4B).

Furthermore, when photographed using the provided guide image (FIG. 4C), the final composite image (FIG. 4D) is generated by synthesizing the first image (FIG. 4A) and the divided person image. This synthesis can be performed through the pixel replacement scheme described above in FIG. Accordingly, through the synthesis method according to an embodiment of the present invention, a natural group photo can be generated in real time by synthesizing the newly photographed second image to the first image, which is the background image, by only taking two photographs.

5A to 5D are reference diagrams for explaining a process of synthesizing photographed images according to another exemplary embodiment of the present invention.

5A to 5D are diagrams illustrating a process of synthesizing photographed images when the second image to be photographed is a background image and a person image divided from the first image is a foreground image to be overlaid on the second image.

5A to 5D, when the camera enters the image composition mode, the first image (FIG. 5A) and the second image (FIG. 5B) being captured by the camera on the same background as the first image (FIG. 5A). ).

Then, the person image is divided from the first image (FIG. 5A) according to the analyzed result. Subsequently, the divided person image is overlaid on the second image (FIG. 5B) to provide a guide image (FIG. 5C) for inducing the synthesis of the captured images. In this case, the pixel averaging method described above in FIG. 2 may be used. The guide image (FIG. 5C) is provided in real time on the display screen in a translucent form so as to synthesize the first image (FIG. 5A) and the second image (FIG. 5B).

Furthermore, when photographed from the photographer using the provided guide image (FIG. 5C), the final composite image (FIG. 5D) may be generated by synthesizing the second image (FIG. 5B) and the divided person image. This synthesis can be performed through the pixel replacement scheme described above in FIG. Accordingly, a group photo may be generated by real-time synthesis of a first image photographed on a second image, which is a background image, in real time by only two photographs by using the synthesis method according to an embodiment of the present invention.

6A to 6I are reference views for describing in detail the process of synthesizing the captured images of FIGS. 4A to 4D according to an exemplary embodiment of the present invention.

6A to 6I, the camera generates a third image (FIG. 6C) by aligning the second image (FIG. 6B) with the first image (FIG. 6A) according to the analysis result when entering the image synthesis mode. In this case, the feature points may be extracted through the Harrison, Hessian corner detection, or DoG method described above with reference to FIG. 2, and the feature information of the extracted feature points may be obtained through a generalized color moment or SIFT. In addition, in FIG. 3, point matching using the aforementioned NCC, correlation coefficient, and the like is performed, and disparity information is calculated between each feature point that is matched by a point transform using affine transform, and the second image ( 6B) may be aligned with the first image (FIG. 6A), but various embodiments are possible in addition to the methods described above.

Subsequently, person masks (FIG. 6D) are divided and generated by performing a difference operation between the first image (FIG. 6A), which is a background image, and the generated third image (FIG. 6C). Then, the face region (FIG. 6E) is detected from the generated third image (FIG. 6C), and the portrait mask (FIG. 6F) corresponding to the detected face region (FIG. 6E) is extracted from the divided portrait masks (FIG. 6D). do. Extraction of the person mask (FIG. 6F) corresponding to the detected face area (FIG. 6E) among the segmented person masks (FIG. 6D) may use the blob analysis described above with reference to FIG. 2.

Next, a person image (FIG. 6G) is generated by performing an AND operation on the extracted person mask (FIG. 6F) and the generated third image (FIG. 6C). The person image (FIG. 6G) generated in this manner is a foreground image to be synthesized with the background image (FIG. 6A), and is synthesized with the background image (FIG. 6A) to generate the guide image (FIG. 6H). The guide image (FIG. 6H) is an image that changes in real time. In this case, when the photographer photographs using the generated guide image (FIG. 6H), the group image (FIG. 6I) in which the first image (FIG. 6A) and the second image (FIG. 6B) are combined at the same position may be generated.

7A to 7I are reference diagrams for describing in detail the process of synthesizing the captured images of FIGS. 5A to 5D according to another exemplary embodiment of the present invention.

7A to 7I, the camera generates a third image (FIG. 7C) by aligning the first image (FIG. 7A) with the second image (FIG. 7B) according to the analysis result when entering the image synthesis mode. At this time, the feature point may be extracted through Harrison, Hessian corner detection, or DoG method described above with reference to FIG. 2, and the feature information of the extracted feature point may be obtained through a generalized color moment or SIFT. In addition, in FIG. 3, point matching using the NCC and correlation coefficients described above is performed, and the variation information between each feature point that is point matched using affine transform is calculated to use the first image ( Although FIG. 7A may be aligned with the second image (FIG. 7B), various embodiments are possible in addition to the methods described above.

Subsequently, person masks (FIG. 7D) are divided and generated by performing a difference operation between the second image (FIG. 7B), which is a background image, and the generated third image (FIG. 7C). Then, the face region (FIG. 7E) is detected from the generated third image (FIG. 7C), and the person mask (FIG. 7F) corresponding to the detected face region (FIG. 7E) is extracted from the segmented-formed portrait masks (FIG. 7D). do. Extraction of the person mask (FIG. 7F) corresponding to the detected face area (FIG. 7E) among the segmented person masks (FIG. 7D) may use the blob analysis described above with reference to FIG. 2.

Subsequently, a person image (FIG. 7G) is generated by performing an AND operation on the extracted person mask (FIG. 7F) and the generated third image (FIG. 7C). The person image generated in this manner (FIG. 7G) is a foreground image to be synthesized with the background image (FIG. 7B), and is synthesized with the background image (FIG. 7B) to generate the guide image (FIG. 7H). The guide image (FIG. 7H) is an image that changes in real time. In this case, when the photographer photographs using the generated guide image (FIG. 7H), the group image (FIG. 7I) in which the first image (FIG. 7A) and the second image (FIG. 7B) are combined at the same location may be generated.

So far, the present invention has been described with reference to the embodiments. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

1 is a block diagram of a camera according to an embodiment of the present invention,

2 is a flowchart illustrating a method of synthesizing photographed images according to an embodiment of the present invention;

3 is a flowchart illustrating in detail a method of dividing a person image of FIG. 2;

4A to 4D are reference diagrams for explaining a process of synthesizing photographed images according to an embodiment of the present invention;

5A to 5D are reference diagrams for explaining a process of synthesizing photographed images according to another embodiment of the present invention;

6A to 6I are reference diagrams for describing in detail the process of synthesizing the captured images of FIGS. 4A to 4D according to an embodiment of the present invention;

7A to 7I are reference diagrams for describing in detail the process of synthesizing the captured images of FIGS. 5A to 5D according to another exemplary embodiment of the present invention.

<Description of the symbols for the main parts of the drawings>

1: camera 10: image analysis unit

20: image segmentation unit 30: image providing unit

40: image synthesis unit

Claims (11)

Analyzing a first image photographed in advance and a second image captured by the camera on the same background as the first image when the image combining mode is entered; Dividing a person image from either the first image or the second image according to the analyzed result; And And overlaying the divided person image on either the first image or the second image in which the person image is not divided to provide a guide image for inducing the synthesis of the captured images. The method of claim 1, And when the photographer photographs the provided guide image, synthesizing the divided person image and the first image or the second image in which the person image is not divided at the position where the guide image is provided. The method of claim 1, And the first image is a background image, and the person image divided from the second image is a foreground image to be overlaid on the first image. The method of claim 1, And the second image is a background image, and the person image divided from the first image is a foreground image to be overlaid on the second image. The method of claim 1, The analyzing of each image may include extracting and analyzing image information representing characteristics of the first image and the second image, respectively. The method of claim 1, wherein dividing the person image comprises: Generating a third image in which the first image and the second image are aligned according to the analyzed result; And And dividing the person image from the generated third image. The method of claim 6, wherein dividing the person image from the generated third image comprises: Dividing and generating person masks by performing a difference operation between a background image, which is one of the first image or the second image, and the generated third image; Detecting a face region from the generated third image and extracting a face mask corresponding to the detected face region from the segmented face masks; And And dividing the person image by performing an AND operation on the extracted person mask and the generated third image. The method of claim 1, The providing of the guide image may include providing the guide image in real time as the second image captured by the camera changes. An image analyzer configured to analyze a first image photographed in advance and a second image captured by a camera on the same background as the first image when the image synthesis mode is entered; An image dividing unit dividing a person image from either the first image or the second image according to the analyzed result; And And an image provider configured to overlay the divided person image on one of the first image and the second image in which the person image is not divided to provide a guide image. The method of claim 9, And a photographing unit configured to synthesize the divided person image and the first image or the second image in which the person image is not divided, at the position where the guide image is provided when the photographer photographs the provided guide image. The method of claim 9, And the image providing unit provides the guide image in real time as the second image captured by the camera changes.

Images