JP4828788B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus such as a mobile phone or a digital camera, and particularly relates to synthesis of a template image such as a background image and a foreground image with another image.

  There is an image processing apparatus that combines a template image and a subject image to display for alignment and determines a composition (see, for example, Patent Document 1).

  In the image processing apparatus disclosed in Patent Document 1, when one template image is selected from a plurality of template images, all the plurality of template images are displayed at a low resolution for display for selection, thereby improving the response speed. ing. In addition, when aligning the selected template image and subject image, both images are combined at the high resolution, and the display for alignment is performed according to the resolution of the monitor. When the composite image of the image is stored in the memory, it is stored at a high resolution that matches the resolution of the camera.

  In addition, there is an imaging apparatus that combines two images to display for alignment and determines the composition (see, for example, Patent Document 2).

In the image pickup apparatus disclosed in Patent Document 2, when performing display for alignment of two images, the two images are combined with high resolution and displayed on the LCD. At this time, alignment of the two images is performed by coordinate calculation processing, and in order to increase the processing speed, the luminance information of one image is divided by a certain threshold value to simplify the coordinate calculation processing. Yes.
JP 2002-218212 A (page 10-11, FIG. 4 to FIG. 6) JP-A-11-196362 (pages 12-14, 19, 20 and FIGS. 7-10, 12, 13)

  By the way, in the conventional image processing apparatus described in Patent Document 1 described above, when aligning the template image and the subject image, the images are combined and displayed for alignment while maintaining the high resolution. Yes. However, in order to synthesize and display both images with high resolution, when processing only with software, there is a problem that the processing speed is slow and the usability of composition adjustment is poor. On the other hand, in order to perform processing with hardware, there is a problem that the hardware increases, resulting in an increase in the area of component mounting and an increase in price, which cannot be applied to small and inexpensive devices such as mobile phones and digital cameras.

  Further, in the conventional imaging device described in Patent Document 2 described above, when performing display for alignment of two images, the two images are combined with high resolution and displayed on the LCD. . Therefore, there is a problem similar to that of Patent Document 1.

  The present invention has been made to solve the above-described problems. When performing display for alignment of two images, the processing speed is high, the usability of composition alignment is good, and after alignment is completed. An object of the present invention is to provide an image processing apparatus capable of storing a composite image of both images in a memory at a high resolution.

In order to achieve the above object, an image processing apparatus of the present invention includes a first image storage unit that stores a first image that is a background image or a foreground image, and a second image that is combined with the first image. A second image storage means for storing an image; a low-resolution conversion means for reducing the resolution of the first image; and a low-resolution image obtained by reducing the resolution of the first image and the second image. Display combining means for creating a display composite image, display means for displaying the display composite image, and storage composite for combining the first image and the second image to create a storage composite image and means, the resolution of the low resolution image is being lower than the resolution of the second image.

According to the present invention, when display for alignment of two images is performed, the resolution of one template image is lower than the resolution of the current image during photographing of the other subject, so that the other image can be displayed without increasing hardware . While the current image being photographed remains in high resolution, a composite image for display can be created by hardware and displayed at high speed. As a result, the usability of composition adjustment is good.

  Embodiments of the present invention will be described below with reference to the drawings.

  1 to 3 show an embodiment in which the present invention is applied to a mobile phone, FIG. 1 is a block diagram of a main part relating to image processing of a mobile phone having a camera, and FIG. 2 is an explanatory view for creating a template image. 3 is a diagram for explaining the synthesis of the template image and the subject image.

  FIG. 1 is a block diagram of a main part relating to image processing of a mobile phone having a camera. The main parts relating to image processing of the cellular phone are a camera unit 1, a shutter button 2, a control unit 3, an LCD controller 4, an LCD 5, a guide frame memory 6, a template image memory 7, a current image memory 8, a confirmed composite image memory 9, and the like. It is configured.

  Further, the control unit 3 includes a CPU, a program memory, an I / O, and the like (not shown), and performs image processing, communication processing, user interface processing, and the like by software. The LCD controller 4 includes a main image memory 41, a small-capacity sub-image memory 42, an LCD control unit 43, and the like, and performs image processing for displaying on the LCD 5.

  The interface 4a between the LCD controller 4 and the LCD 5 has a large number of control lines and signal lines. For this purpose, the LCD controller 4 and the LCD 5 are mounted in the same LCD unit. In a small image processing apparatus such as a mobile phone or a digital camera, since the LCD unit cannot be made large, the memory of the LCD controller 4 cannot have a large capacity. For this reason, the capacity of the sub-image memory 42 used for secondary purposes is small.

  Next, the function of each part of the mobile phone will be described with reference to FIGS.

(Create a template image)
First, a function for creating a template image which is one of two images to be combined will be described. A user (user) operates a key (not shown) to set the mobile phone to a shooting mode. Then, the camera of the mobile phone is pointed at the subject A (FIG. 2A).

  At this time, the subject A is photographed by the camera unit 1, and a high-resolution subject image is sent to the control unit 3. This subject image is a current image that changes constantly when the user changes the orientation of the camera. The control unit 3 sends this subject image to the main image memory 41 of the LCD controller 4. The LCD controller 43 of the LCD controller 4 reads out the subject image in the main image memory 41, performs conversion according to the pixel arrangement of the LCD 5, and sends the subject image to the LCD 5. The LCD 5 displays this.

  The composition of this display image changes depending on the orientation of the camera. The user presses the shutter button 2 when the desired composition is obtained. The control unit 3 sends a subject A (FIG. 2A), which is a subject image from the camera unit 1 when the shutter button 2 is pressed, to an external device such as a current image memory 8 or another SD card (not shown). Save to memory.

  Next, the user operates the key to enter the guide frame display mode. The control unit 3 reads a plurality of guide frame images (not shown) in the guide frame memory 6 and displays them on the LCD 5 via the LCD controller 4. For the purpose of extracting the face portion of the subject A (FIG. 2 (a)), the user operates a key on a guide frame (FIG. 2 (b)) close to the contour of the face from a plurality of guide frame images. To select.

  The contour extraction processing unit (not shown) of the control unit 3 is based on the subject A (FIG. 2A) stored in the current image memory 8 and the selected guide frame (FIG. 2B). Then, the contour extraction process is performed to extract the contour portion of the face. For this contour extraction process, various methods are disclosed, for example, Japanese Patent Laid-Open No. 2001-188910.

  Next, a template image creation unit (not shown) of the control unit 3 removes the extracted face contour from the subject A (FIG. 2A), and a hollow template image (FIG. 2C). Create At this time, the hollowed face portion is converted to pink. This hollow template image (FIG. 2 (c)) is an image used as a frame (outer frame or background), and is used for the purpose of synthesizing another image with a pink hollow face portion. The

  Further, the template image creation unit of the control unit 3 creates an outside template image (FIG. 2D) in which the outside of the face contour is removed. At this time, the outside portion is converted into pink. This outside template image (FIG. 2 (d)) is an image used as a stamp (foreground), and is used for the purpose of synthesizing another image with the pink outside portion.

  The control unit 3 stores the hollow template image (FIG. 2C) and the outside template image (FIG. 2D) in the template image memory 7.

  The hollow template image (FIG. 2C) and the outer template image (FIG. 2D) are high-resolution images, similar to the subject A (FIG. 2A) photographed by the camera unit 1. It is.

(Composite display of template image and subject image)
Next, a combined display of the subject image that is the other of the two images to be combined and the template image described above will be described. The case where the template image is an outside template image (FIG. 2D) will be described.

  The user operates the key to set the mobile phone to the image composition mode. The control unit 3 reads a plurality of template images stored in the template image memory 7 and displays them on the LCD 5. The user selects an outside template image (FIG. 2D) from among a plurality of template images.

  The control unit 3 sends the high-resolution outside-extracted template image (FIG. 2D) stored in the template image memory 7 to the sub-image memory 42 of the LCD controller 4. As high resolution, for example, color information of 8 bits per dot of image is assumed. However, the sub-image memory 42 is a small-capacity memory, and has a capacity for storing, for example, 4-bit color information per dot of image.

  At this time, the connection hardware of the control unit 3 and the LCD controller 4 is configured so that only the upper 4 bits of the color information of 8 bits are written in the sub-image memory 42. As a result, the color information is reduced. However, the outside pink color portion is a single gradation only of the upper bits, and the pink color is maintained as it is.

  As a result, the sub-image memory 42 stores the color-reduced low-resolution externally-extracted template image shown in FIG.

  Note that the process of reducing 8 bits to the upper 4 bits is performed not only by hardware but also by software processing to reduce a high-resolution exterior template image (FIG. 2D) as shown in FIG. 3B. Alternatively, the low-resolution external template image may be temporarily converted, and the low-resolution external template image (FIG. 3B) may be sent to the sub-image memory 42 of the LCD controller 4.

  Further, the conversion to low resolution is not limited to the color information reduction, but may be another conversion method such as area conversion of one dot of the image, and the amount of information only needs to be reduced.

  Next, the user points the camera of the mobile phone toward the subject B (FIG. 3A).

  At this time, the subject B is photographed by the camera unit 1, and a high-resolution subject image is constantly sent to the control unit 3. The subject image is a current image in which the composition changes constantly when the user changes the camera direction. The control unit 3 sends this subject image to the main image memory 41 of the LCD controller 4.

  The LCD control unit 43 of the LCD controller 4 includes the current image of the subject B (FIG. 3A) that is constantly written in the main image memory 41 and the low-resolution externally-extracted template image (FIG. and b)) are synthesized by the method described below.

  The LCD control unit 43 has a pink color detecting means for the outside template image (FIG. 3B), and performs pink color detection for each pixel. Then, with respect to the pixels for which the pink color is detected, the corresponding pixels of the current image of the subject B (FIG. 3A) in the main image memory 41 are sent to the LCD 5.

  Further, the LCD control unit 43 sets the corresponding pixels of the outside template image (FIG. 3B) in the sub-image memory 42 for the pixels other than pink in the outside template image (FIG. 3B). To send. That is, the face portion of the outside template image (FIG. 3B) is sent to the LCD 5.

  This synthesis processing hardware is simply configured by a pink color detection decoder (not shown) and a multiplexer (not shown) for selecting one of the two images. Moreover, since it is hardware, the composition processing and display are performed at high speed.

  As a result, the composite image shown in FIG. The user changes the angle of the subject B by changing the direction of the camera while viewing the composite image. At this time, since the high-resolution current image of the subject B that constantly changes by changing the angle and the low-resolution fixed template image are displayed at high speed, the user feels comfortable and has good usability. Alignment can be performed. Although the template image has a low resolution, there is no problem for the purpose of alignment.

  Then, as shown in FIG. 3D, the user matches the position of the face part of the outside template image (FIG. 3B) with the face part of the subject B (FIG. 3A). When the desired composition is obtained, the shutter button 2 is pressed to confirm the alignment.

(Composite save of template image and subject image)
Next, a description will be given of combining and saving the current image when the shutter button 2 is pressed and the template image described above. When the shutter button 2 is pressed, the control unit 3 temporarily stores the subject B (FIG. 3A) that is the current image from the camera unit 1 at that time in the current image memory 8.

  Next, the control unit 3 stores the high-quality image of the determined subject B (FIG. 3A) once stored in the current image memory 8 and the high-quality outside template stored in the template image memory 7. The image (FIG. 2 (d)) is read out. Then, they are synthesized by software processing as described below.

  The control unit 3 performs pink color detection for each pixel of the outside template image (FIG. 2D). Then, with respect to the pixel in which the pink color is detected, the corresponding pixel of the subject B (FIG. 3A) in the current image memory 8 is stored in the confirmed composite image memory 9.

  Further, the control unit 3 stores the corresponding pixels of the outside template image (FIG. 2D) in the confirmed composite image memory 9 for pixels other than pink in the outside template image (FIG. 2D). To do. That is, the face portion of the outside template image (FIG. 2D) is stored in the confirmed composite image memory 9. By performing this operation for all the pixels, a high-definition fixed composite image (FIG. 3 (e)) is stored in the final composite image memory 9.

  Next, for the display of the high-resolution confirmed composite image, the control unit 3 reads the confirmed composite image (FIG. 3E) stored in the confirmed composite image memory 9 to display the main image memory of the LCD controller 4. 41. The LCD control unit 43 of the LCD controller 4 reads the confirmed composite image in the main image memory 41, performs conversion according to the pixel arrangement of the LCD 5, and sends the subject image to the LCD 5. The LCD 5 displays this.

  In the above-described embodiment, the template image (FIG. 2 (d)) is used as the template image. However, when the template image (FIG. 2 (c)) is used, the template image can be similarly processed. it can.

  In the above-described embodiment, the current image is displayed in high quality and the template image is displayed in low quality. Conversely, the current image may be displayed in low quality and the template image may be displayed in high quality. For this purpose, the control unit 3 may send the current image to the sub image memory 42 of the LCD controller 4 and send the template image to the main image memory 41 of the LCD controller 4.

  As described above, according to the embodiment, the two images can be synthesized and displayed at high speed without increasing hardware, and the user can perform alignment at high speed. Images can be saved.

  In the above-described embodiment, one of the two images to be combined is the current image (FIG. 3A) in which the composition changes and the image constantly changes when the user changes the camera direction, but is fixed. It may be an image. For example, when a fixed image of “wig” which is a plurality of products is already provided as a composition already aligned with the other template image (FIG. 3B), the fixed image of “wig” and the other template The face portion of the image (FIG. 3B) can be synthesized and displayed at a high speed, and can be sequentially switched to a plurality of “wig” fixed images for display. Then, the user can select a favorite “wig”.

  In this way, two images to be combined can be combined and displayed at high speed without an increase in hardware. Further, after the composite image is determined, it can be stored as a high-resolution composite image.

The block diagram of the principal part regarding the image processing of the mobile telephone which has a camera which concerns on the Example of this invention. Explanatory drawing which creates the template image which concerns on each Example of this invention. The figure explaining the synthesis | combination of the template image and subject image which concern on each Example of this invention.

Explanation of symbols

1 Camera unit 2 Shutter button 3 Control unit 4 LCD controller 5 LCD
6 Guide frame memory 7 Template image memory 8 Current image memory 9 Final composite image memory 41 Main image memory 42 Sub-capacity memory 43 with a small capacity LCD controller

Claims (6)

First image storage means for storing a first image that is a background image or a foreground image;
Second image storage means for storing a second image to be combined with the first image;
Low-resolution conversion means for reducing the resolution of the first image;
Display combining means for combining the low-resolution image with the resolution of the first image lowered and the second image to create a display combined image;
Display means for displaying the composite image for display;
; And a storage synthesizing means for creating a store combined image by combining the first image and the second image,
The image processing apparatus according to claim 1, wherein the resolution of the low-resolution image is lower than the resolution of the second image . First image storage means for storing a first image that is a background image or a foreground image;
Second image storage means for storing a second image to be combined with the first image;
Low-resolution conversion means for reducing the resolution of the second image;
Display compositing means for compositing the first image with a low resolution image in which the resolution of the second image is lowered;
Display means for displaying the composite image for display;
; And a storage synthesizing means for creating a store combined image by combining the first image and the second image,
An image processing apparatus according to claim 1, wherein the resolution of the low-resolution image is lower than the resolution of the first image . Template image storage means for storing a template image that is a background image or a foreground image;
Photographing means for outputting a current image of a subject constantly changing according to a camera angle;
Low resolution conversion means for reducing the resolution of the template image;
Display combining means for combining the current image with a low-resolution template image with a reduced resolution of the template image to create a display combined image;
Display means for displaying the composite image for display;
A confirmed current image storage means for storing the current image when the camera angle is confirmed;
; And a storage synthesizing means for creating a store combined image by combining the determined current image and the previous term template image,
The image processing apparatus according to claim 1, wherein the resolution of the low-resolution image is lower than the resolution of the current image . Template image storage means for storing a template image that is a background image or a foreground image;
Photographing means for outputting a current image of a subject constantly changing according to a camera angle;
Low resolution conversion means for reducing the resolution of the current image;
Display compositing means for composing the template image with the low-resolution current image with a reduced resolution of the current image to create a composite image for display;
Display means for displaying the composite image for display;
A confirmed current image storage means for storing the current image when the camera angle is confirmed;
; And a storage synthesizing means for creating a store combined image by combining the determined current image and the previous term template image,
The image processing apparatus according to claim 1, wherein the resolution of the low-resolution image is lower than the resolution of the template image . And further comprising third image storage means for storing the low resolution image,
The low resolution conversion means includes
Of the number of bits representing the resolution of the first image stored in the first image storage means, the first image corresponding to a plurality of higher-order bits is converted into the low resolution in accordance with the capacity of the third image storage means. As an image, it is stored in the third image storage means,
The display combining means includes:
The image processing apparatus according to claim 1, wherein the low-resolution image and the second image are combined and displayed on the display unit. And further comprising third image storage means for storing the low resolution image,
The low resolution conversion means includes
Of the number of bits representing the resolution of the template image stored in the template image storage unit, a plurality of higher-order template images corresponding to the capacity of the third image storage unit are used as the low resolution image. Stored in the image storage means,
The display combining means includes:
4. The image processing apparatus according to claim 3, wherein the low resolution image and the current image are combined and displayed on the display means.

Images