JP2018173885A - Image processing device, image processing method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device, an image processing method, and a computer program capable of changing a color of a portion for each portion in a region of a person to be photographed with abundant expressions without being affected by parameters such as hue, brightness and saturation of an original color of the photographed image of the person.SOLUTION: An image processing device includes identifying means that identifies for each one of a plurality of images, a region corresponding to a portion of a photographed person in the one image; receiving means that receives a color for a region identified by the identifying means; first creating means that creates a uniformed image where the color in the region has been changed to a predefined color; second creating means that creates a colored image where a range corresponding to the identified area has been colored with the color selected by the receiving means; first synthesizing means that synthesizes the colored image with the uniformed image; and second synthesizing means that synthesizes the image synthesized by the synthesizing means with the one image.SELECTED DRAWING: Figure 11

Description

  The present invention relates to an image processing apparatus, an image processing method, and a computer program in a shooting game machine that can take a photograph of a user who is a person as a subject and obtain a predetermined photo sticker sheet or photo data in which the shooting result is output.

  An apparatus for discharging a photo sticker sheet obtained by a user photographing himself / herself is provided as a photographing game machine. A user of a photographing amusement machine has a desire to obtain a photographic image that is different from the usual self as if it were a game. It is expected to have a function that allows you to easily get yourself in a digital image by applying hair and makeup that you cannot do in real life.

  In many of today's photographing amusement machines, image processing is performed in which makeup (makeup) is applied to the face (Patent Documents 1, 2, etc.). In any case, it aims to make the image that does not cause a sense of incongruity by improving the familiarity with the person area in which the user himself / herself is photographed while making the image so that the user himself / herself can get an uplifting feeling as a result of the transformation. is there.

JP 2016-110416 A JP, 2014-131160, A

  In image processing for obtaining an image with makeup applied, processing is performed for each part of a different face such as an eye, eyebrows, lips and the like to be applied. In order to improve the familiarity with the rest of the area after processing and avoid creating a sense of incongruity, the eyes, eyebrows, lips, etc. use the texture of the original image as it is or superimpose a transparent image with a changed color. A process of combining the original color with the color to be changed is performed. When a transparent image superimposing process and a composition process with the original color are performed, depending on the original color of the target portion, the influence of the original color may appear strongly and the effect of the color change may be weakened. In particular, when processing for low-lightness areas such as hair, if a transparent image is superimposed on the original color, or if the modified color is calculated based on the original color, the result as directed Is likely not to be obtained. For example, it has been difficult to change the hair color of a black hair user to a light color such as light pink.

  The present invention has been made in view of such circumstances, and without being affected by parameters such as hue, brightness, and saturation of the original color of a captured image obtained by photographing the user, for each portion of the subject person area, An object of the present invention is to provide an image processing device, an image processing method, and a computer program capable of changing the color of the portion with rich expressions.

The image processing apparatus according to the present invention includes means for acquiring a series of a plurality of images, and specifying means for specifying a face of a human subject imaged from the acquired images and an organ region within the face. An image processing apparatus for performing processing on an area corresponding to the person subject,
A specifying unit that specifies a region corresponding to a part of the person subject in the one image for each one of the plurality of images, and a receiving unit that receives a color for the region specified by the specifying unit A first creating unit that creates a uniform image in which the color in the identified region is changed to a predetermined color, and the region selected by the color selected by the receiving unit A second creation means for creating a colored image with a colored range, a first synthesis means for synthesizing the colored image with the uniformized image, and an image synthesized by the synthesis means are synthesized into the one image. Second synthesizing means.

  In the image processing apparatus according to the present invention, the specifying unit specifies a hair region as a part of the human subject, and the second generation unit receives a selection of a hair color and generates a hair color image as a colored image. To do.

  In the image processing apparatus according to the present invention, the specified color is light brown.

  In the image processing apparatus according to the present invention, the specified color is different for each color option received by the second generation unit, and the first generation unit has a predetermined color corresponding to the color received by the reception unit. A color is determined, and the uniform image is created with the determined color.

  In the image processing apparatus according to the present invention, the second creating unit receives an input of a trajectory including a plurality of locations on the image, and the range corresponding to the accepted trajectory is selected in the specified area. Draw in color.

  In the image processing apparatus according to the present invention, the first synthesizing unit performs blend synthesis for performing a predetermined calculation between pixel values for pixels corresponding to the case where the colored image is arranged on the uniform image. .

  In the image processing apparatus according to the present invention, the first creating unit separates color information indicating a color in the specified area and texture information indicating a texture in the area, and the color information after the separation is The color to be displayed is changed to the specified color, and the first combining unit sequentially combines the colored image and the image based on the texture information on the image based on the color information after the color change of the uniformized image. To do.

  In the image processing apparatus according to the present invention, a plurality of faces specified in each of the plurality of images on a screen separated to receive operations from a plurality of persons, respectively. The processing by the first and second creating means and the first and second synthesizing means is executed for one of the areas and the hair area in any of the plurality of face areas.

  In the image processing method according to the present invention, a processor for acquiring a series of a plurality of images specifies a face of a human subject photographed in the acquired plurality of images and an area of an organ in the face, and In the image processing method for performing processing on a corresponding region, for each image of the plurality of images, a region corresponding to a part of the person subject in the one image is specified, and the specified region Create a uniform image in which the color in the specified area is changed to a predetermined color, create a colored image in which the area corresponding to the area is colored with the received color, The colored image is combined with the uniform image, and the combined image is further combined into the one image.

  A computer program according to the present invention is a computer program that causes a computer to acquire a series of a plurality of images and to perform processing on the acquired plurality of images. Each time an area corresponding to a part of the person subject in the one image is specified, a color for the specified area is accepted, and the color in the specified area is changed to a predetermined color. Creating a colored image, coloring a region corresponding to the region with the received color, combining the colored image with the uniformized image, and further combining the combined image with the one image A process of combining the image is executed.

  In the present invention, the first synthesis is performed by superimposing a colored image in which the corresponding range is colored with the selected color on the uniform image obtained by changing the range corresponding to a part of the human subject to the specified color. The first combined image is superimposed on the image to be processed and combined. As a result, it becomes possible to change to the selected color by weakening the influence of the original color, and the color change is effective even if the region to be processed has a relatively high saturation or low brightness color. become.

  In the present invention, by combining with a uniform image that has been changed to a specified color, particularly for a hair region as a part of a human subject, it is possible to change to a lighter and brighter color that was difficult in the past. Become. Note that the order of creation of the uniform image and creation of the colored image is not limited.

  In the present invention, the specified color is not an achromatic color such as white but a light brown color, so that even if the selected color is a light color such as light pink, light blue, or light purple, it is familiar. Good color expression is possible. Note that light brown is, for example, a pixel value (R = 155, G = 110, B = 80). In the case of the HSL value, for example, H (hue) = 16 to 30/360, S (saturation) = 30 to 50/100, and L (luminance) = 36 to 50/100. Can be light brown.

  In the present invention, the specified color differs depending on the selected color, and the uniformized image is created by changing to the specified color corresponding to the selected color. This enables rich color expression that is not affected by the color of the original image, but is not limited to the light color change that has been difficult in the past.

  In the present invention, by using a colored image in which a locus on an image based on a handwriting operation is drawn with a selected color, it is possible to change the color of an arbitrary place in a part of the specified human subject to a light color. It becomes possible.

  In the present invention, the process for synthesizing the colored image with the uniform image is realized by a so-called blend synthesis such as soft light synthesis or overlay synthesis. Coloring becomes effective by combining the lightness appropriately while reflecting the color of the image to be overlaid.

  In the present invention, the uniformized image is separated into color information (low frequency component) and texture information (high frequency component) based on the frequency of Fourier transform or based on wavelet transform, and the color of the color information is changed to a prescribed color. Created by changing. Thereby, it is possible to change only the color without losing the original texture such as the edge portion extracted as the high frequency component. When the images are combined, the images are overlapped with the texture information at the top, so that a combined image in which the color is effectively changed while utilizing the texture can be obtained.

  In the present invention, when a plurality of human subjects are captured in one image, one of a plurality of face areas and a whole face area may be selected as a color change target for each image. Selection is possible, and the process is executed for the selected change target. The degree of freedom of operation is improved by making color changes uniform or changing individually according to user preferences.

  According to the present invention, it is difficult in the past even when processing is performed on a portion of a person subject area of a user in a photographed image, such as an eyebrow or a hair region, which has a color with relatively higher saturation or lightness than others. The expression of the color that has been made possible while taking advantage of the original texture. As a result, it is possible to obtain an image as if the user was actually subjected to hair and make-up, and the result as produced in the shooting game machine was obtained, and the satisfaction of the user who selected the shooting game machine Can be improved.

It is a perspective view which shows the external appearance of the imaging | photography game machine in this Embodiment. It is a block diagram which shows typically the structure of the imaging | photography game machine in this Embodiment. It is a flowchart which shows an example of the whole process performed with the imaging | photography game machine of this Embodiment. It is a flowchart which shows an example of the whole process performed with the imaging | photography game machine of this Embodiment. It is explanatory drawing which shows typically the example of the content of the mask image of the area | region of the hair. It is a flowchart which shows an example of the creation processing procedure of a hair color uniform image. It is explanatory drawing which shows an example of a captured image typically. It is explanatory drawing which shows typically an example of the hair color equalization image produced from the image of FIG. It is explanatory drawing which shows an example of a graffiti operation screen typically. It is a flowchart which shows an example of the process sequence which changes the hair color at the time of one-touch makeup. It is explanatory drawing which shows the outline | summary of the process which changes the hair color at the time of one-touch makeup. It is a flowchart which shows an example of the process sequence which changes the hair color at the time of hair color makeup. It is explanatory drawing which shows the outline | summary of the process which changes the hair color at the time of hair color makeup. It is explanatory drawing which shows an example of the image after a hair color change typically.

  Hereinafter, the present invention will be specifically described with reference to the drawings showing embodiments thereof.

  FIG. 1 is a perspective view showing an external appearance of a photographing amusement machine according to the present embodiment. A photographing amusement machine as a whole is a box-shaped device having a substantially rectangular parallelepiped shape that a plurality of users can enter. A shooting game machine can be roughly divided into a shooting area A and an editing area B, and a plurality of devices are provided in each of the areas A and B.

  In the shooting area A, devices related to the pre-service section 1 and the shooting section 2 are provided. The pre-service section 1 includes a coin slot, a touch panel display, a speaker, and the like that are provided toward the outside of the shooting area A. Accept the operation. The photographing unit 2 is configured to include a camera 21, a display 22, a lighting device group 23, and a speaker 24 (see FIG. 2) installed on the wall surface inside the photographing area A so as to photograph the user. A guide is displayed, and an image is taken in a plurality of times while being guided by voice, and processing for outputting the taken image to the editing unit is performed.

  In the editing area B, devices related to the first editing unit 3a, the second editing unit 3b (see FIG. 2), and the printing unit 4 are provided. The first editing unit 3a is configured to include a device in which a multi-touch panel display 30a for displaying an operation screen is exposed at a central portion of the editing area B at a height that is easy for a user to operate. The first editing unit 3a further includes a speaker 33a, a lighting device, and touch pens 31a and 32a (see FIG. 2) for the display 30a. The first editing unit 3a accepts a graffiti operation on an image based on a photographed image taken by the user using the display 30a and the touch pens 31a and 32a, and executes image processing corresponding to the accepted operation. The second editing unit 3b is configured in the same manner as the first editing unit 3a (not shown), and is provided back to back with the first editing unit 3a. The printing unit 4 includes a printer provided in a housing of an apparatus including the first editing unit 3a and the second editing unit 3b in the editing area B, and includes a photo sticker sheet on which an image after image processing is printed. It exhales from the outlet provided on the surface of the body.

  FIG. 2 is a block diagram illustrating a configuration of the shooting game machine according to the present embodiment. The shooting game machine includes the above-described pre-serving section 1, the shooting section 2, the first editing section 3a, the second editing section 3b, and the printing section 4 in a range that is visible to the user in the shooting area A and the editing area B. . In addition, the shooting game machine includes the communication unit 5 and a control mechanism at a location hidden on the wall surface in the shooting area A, or a position that cannot be seen by the user, such as inside the housing of the device in the editing area B. .

  The communication unit 5 is connected to an external network such as the Internet and realizes transmission / reception of data including images taken. The communication unit 5 is connected to a control mechanism, and the control mechanism can communicate with a server device on an external network via the communication unit 5 to transmit and receive data. In the shooting game machine in the present embodiment, the communication unit 5 transmits the image data obtained by shooting to a server device managed by the manufacturer of the shooting game machine. As a result, the user can download the captured image from the server device to an information terminal device such as a so-called smartphone, tablet terminal, PC (Personal Computer), etc., and store it in a storage device in the information terminal device.

  The control mechanism includes a PC that controls each hardware described above, a control board including the bus 100 to which each hardware is connected, and a power supply unit (not shown) that supplies power to each hardware. The PC may be composed of a single PC, but in order to facilitate processing, the PC that controls the pre-serving section 1, the PC that controls the photographing section 2, the PC that controls the first editing section 3a, the second Different PCs may be used for each control target, such as a PC that controls the editing unit 3b and a PC that controls the printing unit 4 and the communication unit 5. In the following description, image processing based on graffiti operations in the first editing unit 3a and the second editing unit 3b will be mainly described. For this reason, the description will be made by virtually separating the first PC and the second PC. The first PC corresponds to the control unit 6 that controls the processes in the prior customer service unit 1 and the photographing unit 2. The second PC controls image processing including predetermined processing for the image obtained by the photographing unit 2, acceptance of editing by the first editing unit 3a and the second editing unit 3b, and synthesis based on these, and printing unit 4 corresponds to the image processing unit 7 instructing the print output in 4 and the data transmission from the communication unit 5.

  The control unit 6 includes a CPU (Central Processing Unit) 60, a memory 61, an HDD (Hard Disk Drive) 62, and a capture / chroma keyboard 63 provided in the first PC. The memory 61 may be a RAM such as a DRAM (Dynamic Random Access Memory) or an SDRAM (Synchronous DRAM), and may be a flash memory or another nonvolatile storage device such as an SSD (Solid State Drive) instead of the HDD 62.

  The CPU 60 reads the control program 6P stored in the HDD 62, and executes the control program 6P while reading / writing various information in the memory 61, thereby realizing the following functions.

  The CPU 60 inputs a control signal output from the pre-service unit 1 when receiving a predetermined charge at the coin insertion slot of the pre-service unit 1, causes the touch panel display to output an operation screen, and guides from the speaker. Output audio. Further, the CPU 60 inputs information corresponding to the operation on the operation screen displayed on the display of the pre-service unit 1 from the pre-service unit 1. Specifically, the CPU 60 inputs a user name, information related to a transmission destination address from the communication unit 5, and the like by an operation on a touch panel display.

  The CPU 60 controls the capture unit 2 to input the monitor video signal obtained from the camera 21 to the capture / chroma keyboard 63 and acquires the image data of each frame included in the video signal from the capture / chroma keyboard 63. Then, a video signal of an image for a live screen is created based on the acquired frame image data, and the created image signal is output to the display 22. Further, the CPU 60 outputs an image signal of a shooting guide for notifying the user of shooting start and guidance to the display 22 and outputs an audio signal to the speaker 24.

  The CPU 60 performs photographing by synchronizing photographing by the camera 21 of the photographing unit 2 and flash lighting by the lighting device group 23. The CPU 60 executes photographing with the camera 21 a plurality of times. The CPU 60 acquires a plurality of image data of an image obtained by photographing from the camera 21, assigns identification information so that each can be distinguished, stores it in the HDD 62, duplicates it, and outputs it to the image processing unit 7.

  The HDD 62 stores various information referred to by the CPU 60. The HDD 62 stores a control program 6P executed by the CPU 60 in advance. In addition to the control program 6P, the HDD 62 displays images and font data to be displayed on the display of the pre-service section 1 and the display 22 of the photographing section 2, and sounds and music output from the speaker of the pre-service section 1 and the speaker 24 of the photographing section 2. Such data is stored in advance. The HDD 62 stores image data obtained by taking a picture in the photographing unit 2.

  The capture / chroma keyboard 63 is a circuit board having a capture function and a chroma key function. The capture / chroma keyboard 63 has a capture function of acquiring image data of 30 (29.97) frames per second from a video signal and outputting the image data as a still image. The capture / chroma keyboard 63 extracts pixels determined to have a predetermined color tone and brightness within a certain range from a still image, and sets an α value (transparency) for the extracted pixels. It has a chroma key function to make the color transparent and output. The capture / chroma keyboard 63 sequentially captures frames from the video signal for monitoring from the camera 21, executes chroma key processing on the still image obtained by the capture as necessary, and writes it in the built-in memory. The CPU 60 sequentially reads still images from a memory built in the capture / chroma keyboard 63 and creates an image signal of an image for a live screen to be displayed on the display 22 of the photographing unit 2.

  The image processing unit 7 includes a CPU 70, a memory 71, an HDD 72, and a graphic board 73 provided in the second PC. The memory 71 uses a RAM such as DRAM or SDRAM.

  The CPU 70 reads the image processing program 7P stored in the HDD 72, and executes image processing based on the image processing program 7P while reading / writing various information from / to the memory 71. Further, the CPU 70 stores the image data output from the control unit 6 in the memory 71 or the HDD 72 together with the assigned identification information, and performs processing based on the image data. The processing includes processing to make the face of the person in the image a small face, brightening the color of the cheeks of the face (blushing), processing to gloss the hair, processing to brighten the hair color, large eyes Various processes are included, such as a process for making the body line appear thin, and a process for making the leg look longer. The CPU 70 stores the image data copied for each type of processing in the memory 71 or the HDD 72, executes the processing on each image data, and stores the plurality of processed image data in the memory 71 or the HDD 72.

  Further, the CPU 70 performs a graffiti operation in the first editing unit 3a or the second editing unit 3b based on a plurality of image data before and after the above-described processing processing obtained by photographing stored in the memory 71 or the HDD 72. Accept. The CPU 70 creates image data such as a handwritten character or a stamp including makeup to be displayed on a preview display portion of a graffiti operation screen, which will be described later, according to the content of the operation accepted as the graffiti operation. The CPU 70 outputs the created image data to the display of the first editing unit 3a or the second editing unit 3b. Further, the CPU 70 creates composite image data according to the confirmed editing content displayed in the preview display section of the graffiti operation screen. Details of the acceptance of the graffiti operation and the creation of the composite image data will be described later. Further, the CPU 70 executes processing for printing out an image on a photographic sticker sheet based on the generated composite image data, and processing for transmitting the composite image data or image data before the composite processing from the communication unit 5 to the external server device.

  The HDD 72 stores various information referred to by the CPU 70. The HDD 72 stores an image processing program 7P executed by the CPU 70 in advance. The HDD 72 stores in advance data such as guidance character images to be displayed on the displays of the first editing unit 3a and the second editing unit 3b, characters, font data or images that are composition materials, music and voices to be output from a speaker, and the like. I remember it. In an image that is a composition material, makeup information 720 is stored in advance so as to realize a function of applying makeup. The makeup information 720 includes material images such as lip pattern information, cheek images, colored contact images, eyeline images, false eyelash images, hair color pattern information, and hair accessory images. The makeup information 720 can be rewritten. Also, the HDD 72 stores image data obtained by image processing based on the accepted editing and image data of the combined image.

  The graphic board 73 is hardware that draws an image based on input image data and a drawing command and outputs image data of the drawn image. The CPU 70 uses the graphic board 73 to perform image processing on the image data acquired and stored from the control unit 6. Specifically, the CPU 70 gives image data and a drawing command for obtaining a target image to the graphic board 73 to perform processing, and inputs the image data obtained by the drawing processing from a memory built in the graphic board 73. To do. The drawing process includes enlargement / reduction, rotation, movement, chroma key function, trimming function, alpha function, brightness or color tone adjustment function, filter function, composition function, and the like.

  FIG. 3 and FIG. 4 are flowcharts showing an example of the overall processing executed in the shooting game machine of the present embodiment. The processing procedure performed by the control unit 6 and the processing procedure performed by the image processing unit 7 will be described separately. FIG. 3 is a flowchart illustrating an example of a shooting processing procedure executed by the control unit 6.

  The processing procedure shown in the flowchart of FIG. 3 is started when the CPU 60 of the control unit 6 detects an operation and a coin insertion on the touch panel type display of the pre-service unit 1.

  The CPU 60 accepts input of a user name, selection of a shooting course, selection of an output sheet design, and the like on the touch panel type display of the customer service section 1 (step S101). The CPU 60 confirms the availability of the shooting area A. If the shooting area A is empty, the CPU 60 displays a movement guidance screen for prompting movement on the display and outputs a guidance voice (step S102).

  Next, the CPU 60 causes the display 22 of the photographing unit 2 to display a guidance screen for photographing contents in the photographing area A, and outputs guidance voice and BGM from the speaker 24 (step S103). The guidance screen displayed at this time may include a live image based on the monitor output of the camera 21 of the photographing unit 2 and a pose sample for each photographing time. Depending on the shooting course and sheet design selected in step S101, the hair color processing described later is performed in real time on the live video displayed at this time so that the processed image is displayed as a video. Also good.

  The CPU 60 controls the camera 21 and the illuminating device group 23 of the photographing unit 2 according to the photographing mode or the sheet design selected in Step S101, and executes plural times of photographing (Step S104). The CPU 60 stores the image data obtained by each shooting in the HDD 62 so as to be mutually distinguishable (step S105). In the multiple shootings in step S104, it is preferable that the up-up shooting from the waist and the whole-body shooting including the toes are distinguished and executed according to the shooting mode or the seat design selected in step S101.

  When shooting is completed, the CPU 60 confirms the availability of the editing area B, and if it is available, the guidance screen prompts the user to move the editing area B to either the first editing unit 3a or the second editing unit 3b. Is displayed on the display 22 and a guidance voice is output from the speaker 24 (step S106), and the process in the control unit 6 is terminated.

  Next, the process proceeds to the processing in the image processing unit 7. FIG. 4 is a flowchart illustrating an example of a processing procedure of image processing, graffiti operation reception, and printing executed by the image processing unit 7. In the following description, the user is urged to move to the first editing unit 3a out of the first editing unit 3a and the second editing unit 3b to perform a graffiti operation. However, in the second editing unit 3b, Since the same process is performed when a graffiti operation is performed, detailed description thereof is omitted.

  The CPU 70 of the image processing unit 7 acquires the image data stored in the HDD 62 by the CPU 60 of the control unit 6 (step S201). The CPU 70 executes a process of extracting one or a plurality of person subject areas for each of the acquired plurality of image data (step S202). Specifically, in the extraction process in step S202, the graphic board 73 given the image data creates image data of a mask image in which pixels corresponding to the person subject area are α (transparent) from the image based on the image data. This is realized.

  For each of a plurality of image data, the CPU 70 applies a face area (contour) to each of one or a plurality of person subject areas extracted by the extraction process in step S202, as well as the eyes, noses, And the process which specifies the feature point of each organ area | region, such as a mouth, is performed (step S203). The processing in step S203 may be performed using various existing techniques such as skin color detection and template matching. In step S203, the CPU 70 specifies the feature points of the face area and the organ area using information on the coordinates of the pixels in the image corresponding to the respective contours. Specifically, in the case of a face, a plurality of coordinates on the temple, the apex of the forehead, the chin, and the cheek outline are specified. In the case of an eye, the coordinates of the head, the corner of the eye, a plurality of locations on the upper eyelid contour, and a plurality of locations on the lower eyelid contour are specified. In the case of the mouth, both ends, that is, the mouth angle, the center of the mouth area, and the coordinates in the image of a plurality of locations on the contours of the upper and lower lips are specified. The identified coordinate information is stored in the memory 71 or the HDD 72 in association with each image data. In addition, information of coordinates specifying the outline of the nose and the position of the nose head, the eyebrows of the eyebrows, the buttocks, and the like is stored.

  Then, the CPU 70 executes a process of specifying a hair region for each of one or a plurality of person subject regions (step S204). Various methods are envisaged for specifying the hair region. For example, the CPU 70 may use the feature points of the face region and each organ region specified in step S203. Specifically, based on the information on the vertex of the forehead among the feature points of the identified face area, the CPU 70 acquires color information on a specific pixel located several pixels above the vertex of the forehead, and the specific pixel Are used to refer to the color information of adjacent pixels of interest in turn. When the color indicated by the color information of the target pixel is a color that is similar within a predetermined range from the color indicated by the color information of the specific pixel (can be determined to be a matching color), the target pixel is the hair of the hair including the specific pixel. The region of the hair may be identified as a set of pixels of interest that are recognized as being within the region and repeated to determine that the colors are similar. Note that the area specified as the hair area may be a range inside a predetermined pixel such as 0.0 to 2.0 pixels from the range corresponding to the set of target pixels determined to be similar. This is to avoid erroneously specifying the skin part side near the boundary with the skin part as the hair region. Then, the identified hair region is coordinate information indicating an outline, or information for identifying a human subject region in the image as a mask image obtained by alpha-changing (transparenting) the corresponding pixels in the region. It is good to store in association. FIG. 5 is an explanatory diagram schematically showing an example of the contents of a mask image of a hair region. The white area in FIG.

  Returning to FIG. 4, the description will be continued. The CPU 70 creates a hair color uniform image in which the hair color is uniformed based on the identified hair region (step S205). Details of the creation of the hair color uniform image will be described later.

  Next, the CPU 70 executes processing for the extracted human subject area, the specified face area, and each organ area in the face area for each of a plurality of image data (step S206). In step S <b> 206, the CPU 70 performs processing suitable for the shooting method (up-shooting, whole-body shooting, etc.) on each of the image data obtained by a plurality of shootings. As described above, the processing includes processing to make the face small, processing to enlarge the eyes, processing to make the body line appear thin, processing to make the legs look longer, and the like.

  The processing up to step S206 is started in the background with respect to the image data stored for each photographing each time a plurality of photographing processes in step S104 in the flowchart of FIG. 3 are performed. Then, the process may be executed so as to proceed while the guidance screen is displayed in the editing area B in step S106.

  Next, the CPU 70 displays a graffiti operation screen on the display 30a of the first editing unit 3a, and outputs a guidance voice and BGM from the speaker 33a (step S207). Then, the CPU 70 accepts a graffiti operation including stamping, character addition, and makeup using the display 30a and the touch pens 31a and 32a on the graffiti operation screen (step S208). Details of the graffiti operation acceptance process will be described later.

  Further, the CPU 70 accepts selection of a layout or the like in the composite photo sticker sheet (step S209). The CPU 70 executes the composition process based on the graffiti operation accepted in step S208 and the layout accepted in step S209 (step S210). Specifically, in step S210, the CPU 70 synthesizes an image such as a stamp, a handwritten character, a superimposing makeup image, etc. received by a graffiti operation with an image after the processing on the image based on each image data, and forms a photo sticker sheet Execute the composition process to lay out.

  The CPU 70 displays a screen including a preview screen of the photo sticker sheet after the composition processing on the display 30a (step S211). The CPU 70 confirms whether or not the printer of the printing unit 4 is in use, and if it is not in use, causes the printer of the printing unit 4 to start printing the photo sticker sheet (step S212). While the print output process is being performed, the CPU 70 performs operations such as selecting an image that can be obtained from the information terminal device possessed by the user from among a plurality of photographed images and inputting an information destination address. Accepted by the display 30a of the first editing unit 3a (step S213). The CPU 70 transmits the image data of the image selected according to the accepted operation to the external server device (step S214). The CPU 70 displays a standby screen on the display 30a of the first editing unit 3a until the print output process is completed. When the print output process is completed, the CPU 70 displays a guidance screen that prompts the user to move to the front of the print unit 4. At the same time, a guidance voice is output (step S215), and the process is terminated.

  FIG. 6 is a flowchart showing an example of a procedure for creating a hair color uniform image. The following processing is performed on a plurality of image data obtained by a series of photographing.

  The CPU 70 selects one image data (step S51), and selects one face area in the image based on the selected image data (step S52). The CPU 70 extracts information (mask image) of the hair region identified in step S204 in the flowchart of FIG. 4 corresponding to the selected face region (step S53). The CPU 70 separates the portion corresponding to the hair region extracted in step S53 in the image based on the selected image data into a color information image and a texture information image using the graphic board 73, and stores each of them. (Step S54). The color information and texture information may be separated by frequency separation using Fourier transform, for example, or other means such as wavelet transform may be used.

  The CPU 70 changes the color of the hair color portion in the color information image stored in step S54 to a predetermined color (step S55), and the changed color information image and the separated texture information image are respectively converted into image data. And stored in association with the face area (step S56). In step S56, the changed color information image and the separated texture information image may be combined and returned.

  The CPU 70 determines whether or not all face areas have been selected (step S57), and if it is determined that all have not been selected (S57: NO), the process returns to step S52. If it is determined in step S57 that all the face areas have been selected (S57: YES), the CPU 70 determines whether or not all the image data of the series of captured images has been selected (step S58). If it is determined that all image data has not been selected (S58: NO), the CPU 70 returns the process to step S51. If it is determined that all image data has been selected (S58: YES), the CPU 70 returns to hair. The process for creating the color uniform image is terminated, and the process returns to step S206 in FIG.

  The processing procedure shown in the flowchart of FIG. 6 will be described with a specific example. FIG. 7 is an explanatory diagram schematically illustrating an example of a captured image, and FIG. 8 is an explanatory diagram schematically illustrating an example of a hair color uniform image created from the image of FIG. In either case, a photographic image is schematically shown by a diagram. In the image example shown in the explanatory diagram of FIG. 7, two human subjects are photographed, and one of the hair color regions of the human subjects for two people is darker and the other is brighter. In the example schematically showing the image of FIG. 7 by a diagram, the darker the density of the hatching, the lower the lightness of the hair color, that is, the darker the color, the closer to the black hair, and the lower the density is the hair. It shows that the lightness of the color is higher than the other, that is, it is close to so-called light brown hair. The hair color uniform image shown in the explanatory diagram of FIG. 8 shows an image in which color information after color change and texture information are combined. In the image example shown in the explanatory diagram of FIG. 8, as is clear when compared with the image example shown in the explanatory diagram of FIG. 7, the lightness of the hair color is unified and the flow of the hair is represented. It can be seen that the texture is maintained as it is. The hue and saturation are also defined colors without depending on the original color. For example, if expressed by pixel values of the three primary colors (red: R, green: G, blue: B) in the image signal, light brown (R = 155, G = 110, B = 80). When expressed in terms of HSL values, the color is light brown if H (hue) = 16 to 30/360, S (saturation) = 30 to 50/100, and L (luminance) = 36 to 50/100. obtain. At this time, by setting the color to light brown instead of the achromatic color white, a result as the effect can be obtained when changing to a light color described later. Note that this specified color may differ depending on the type of synthesis process of the hair color uniform image and the hair color image in the hair color changing process described later. Further, since the specified color may vary depending on whether the color after change is a light color or a bright color, the specified color may be set differently depending on the selected color.

  The creation of the hair color uniform image described with reference to FIGS. 6 to 8 is not limited to being performed in advance in step S205 in the flowchart of FIG. As will be described later, when a graffiti operation is accepted, it may be executed when an operation for changing the hair color is performed. When creating a hair color uniform image when accepting a graffiti operation, a hair color uniform image uniformed with a specified color set in association with the selected color may be created. .

  Next, details of acceptance of the graffiti operation will be described. FIG. 9 is an explanatory diagram schematically illustrating an example of a graffiti operation screen. The graffiti operation screen is displayed on the display 30a of the first editing unit 3a, and the user can perform various selections, image movement and enlargement / reduction or rotation, handwriting input, drawing, and the like on the graffiti operation screen with the touch pen. it can. Since the same applies to the graffiti operation in the second editing unit 3b, detailed description thereof is omitted.

  The graffiti operation screen has preview display sections 301 and 302 for images taken with a user as a human subject on the left and right. The images displayed on the preview display units 301 and 302 are images after a predetermined processing (step S206 in FIG. 4), and operations such as handwriting input on the preview display units 301 and 302 are performed by the touch pens 31a and 32a. It is accepted specially. Each of the two preview display sections 301 and 302 has selection sections 3011 and 3012 for selecting which image data to edit from a plurality of images based on a plurality of image data. .

  The graffiti operation screen has a pen palette section 303 for selecting a pen color, pen type, and pen thickness for handwriting input at the center between the preview display section 301 and the preview display section 302. The pen pallet unit 303 identifies and accepts a selection operation from the touch pen 31 a for operation on the preview display unit 301 and a selection operation from the touch pen 32 a for operation on the preview display unit 302. The pen palette unit 303 includes a pen preview unit 3031 and a pen preview unit 3032 for displaying a sample of handwritten text according to the selected pen color, pen type, and thickness.

  The graffiti operation screen includes a stamp palette 304 for selecting a pre-stored material image such as a stamp image and a message image so as to correspond to the preview display unit 301 and the preview display unit 302, respectively. The stamp palette 304 includes a tab selection unit 101 for selecting a category of an image such as a stamp to be superimposed on two preview images displayed on the preview display units 301 and 302, respectively. In the tab selection unit 101, for example, four types of tabs of “SPECIAL”, “STAMP”, “MESSAGE”, and “MAKE” are displayed so as to be selectable. “SPECIAL” is a tab for displaying a screen for accepting selection of a special material associated with a special event period or season. “STAMP” is a tab for displaying a screen for selecting an image such as an illustration stored in advance. “MESSAGE” is a tab for displaying a screen for selecting a message image including characters stored in advance. “MAKE” is a tab for displaying a screen for accepting selection of an image based on makeup information stored in the HDD 72.

  The graffiti operation screen shown in FIG. 9 shows an example of the contents when the “MAKE” tab is selected in both the tab selection units 101 corresponding to the preview display units 301 and 302. The stamp palette 304 when the “MAKE” tab is selected has a makeup palette 310 corresponding to each of the preview display portions 301 and 302.

  Makeup palette 310 includes makeup selection unit 102 at the top. Makeup selection unit 102 includes a plurality of buttons corresponding to a plurality of makeup types located at the top of makeup palette 310. In the example shown in FIG. 9, the types of makeup are “one touch”, “lip”, “blush”, “color control”, and “hair color”.

  As shown in FIG. 9, a makeup palette 310 corresponding to the preview display unit 301 shows an example of contents displayed when “one touch” is selected by the makeup selection unit 102. A makeup palette 310 corresponding to the preview display unit 302 shows an example of contents displayed when “hair color” is selected by the makeup selection unit 102. At the bottom of makeup palette 310, makeup target display section 103 and makeup target switching button 104 are displayed when any makeup type is selected.

  The makeup target display unit 103 is an image in which an image of a rectangular area including a face area extracted for a plurality of image data is displayed. When a plurality of face areas are included in the image displayed on the preview display section 301 (or the preview display section 302), the makeup target switching button 104 is selected as a makeup target one by one from the plurality of face areas. This button is used to cyclically switch between selecting the entire face area. For example, if two face areas are included in the image, each time the makeup target switching button 104 is selected, the makeup target face area is the first face area, the second face area, or both faces. The area is cyclically switched to the area, the first person's face area, and so on.

  When “one touch” is selected as the makeup type, as shown in FIG. 9, a theme selection icon 711 corresponding to each different theme and an OFF button 712 for erasing the one-touch makeup are arranged. The theme selection icon 711 is a makeup and decoration image selection button corresponding to each of the three themes “MILKY PINK”, “FANCY BLUE”, and “FAIRY PURPLE” in the present embodiment. When “MILKY PINK” is selected, hair makeup based on a warm color including a change in hair color is automatically performed in association with the theme of “MILKY PINK”. Similarly, in the case of “FANCY BLUE”, hair makeup based on a light color based on a cold color matching the theme name is automatically applied. In this way, “one touch” applies makeup associated with a theme. The arrangement of the hair accessory image may be performed.

  Processing when one of the plurality of theme selection icons 711 is selected in a state where “one touch” is selected by the makeup selecting unit 102 in the makeup palette 310 corresponding to the preview display unit 301 of FIG. Will be described. FIG. 10 is a flowchart illustrating an example of a processing procedure for changing the hair color during one-touch makeup. The process shown in the flowchart of FIG. 10 will be described as a process performed on the preview display unit 301. However, the same applies to the preview display unit 302, and a detailed description thereof will be omitted.

  The CPU 70 specifies the face area to be made based on the selection state of the make object switching button 104 (step S301), and extracts the hair area corresponding to the specified face area to be made (step S302). The CPU 70 creates a hair color image in which the entire range of the hair region extracted using the graphic board 73 is colored with the hair color corresponding to the selected theme selection icon 711 (step S303). The hair color image may be transparent except for the hair region.

  Next, the CPU 70 extracts a hair color uniform image corresponding to the specified face area to be made (step S304). When the hair color uniform image is not created in advance in step S205 of FIG. 4, the CPU 70 executes steps S53 to S56 in the processing procedure of FIG. 6 at this time. That is, the order of the creation of the hair color image in step S303 and the creation of the hair color uniform image may be any first. When creating a hair color uniform image in step S304, a hair color uniform image that is uniformized to different colors according to the hair color pattern information corresponding to the selected theme may be created. At this time, the CPU 70 uses the graphic board 73 to change the hair color to the corresponding color in step S55.

  The CPU 70 uses the graphic board 73 to synthesize the hair color image created in step S303 with the hair color uniform image extracted in step S304 (step S305). The synthesis process in step S305 performs, for example, a process called soft light synthesis. That is, in step S305, a hair color image is superimposed on a hair color uniformized image (combined separated color information and texture information), and corresponding pixels (pixels having the same coordinates) when they are superimposed are predetermined. Perform the operation. For example, the graphic board 73 performs the soft light composition, for each color component (R, G, B), the ratio of the pixel value to the maximum value (for example, 255) of the pixel of the hair color uniform image is the bottom, and the pixel of the hair color image For example, a 冪 operation using a value based on a ratio of the value to the maximum value as a 冪 index is performed. In the composition in step S305, not only the soft light composition but also other blending processing such as overlay composition for performing an operation based on the multiplication value of pixel values, etc., for reflecting the color of the hair color image in the hair color uniform image. You may make it perform a synthesis | combination.

  Next, the CPU 70 uses the graphic board 73 to superimpose the synthesized image obtained by changing the hair color obtained in step S305 on the original image (the image after the processing in step S206 in FIG. 4) (step S306). ). The CPU 70 displays the combined image on the preview display unit 301 (step S307), and ends the process.

  FIG. 11 is an explanatory diagram showing an outline of processing for changing the hair color during one-touch makeup. Each rectangle in FIG. 11 represents a layer and represents an overlap of images. A conventional method for synthesizing hair color is shown as a comparative example in the lower part of FIG. In the image processing unit 7 according to the present embodiment, the CPU 70 selects all the images corresponding to the color information in the hair color uniformized image and the range of the hair color region in the layer above the original image. The images corresponding to the texture information of the hair color image colored in the color and the hair color uniform image are superimposed. Conventionally, when a hair color image is soft-light synthesized with the original image, in the image processing unit 7 according to the present embodiment, the CPU 70 applies hair to the image corresponding to the color information of the hair color uniformized image. The color image is synthesized by soft light, the texture information is synthesized by addition, and the synthesized image is synthesized with the original image.

  Conventionally, as shown in FIG. 11, it is difficult to express a light hair color such as “MILKY PINK” set for “one touch” makeup. However, by the processing in the image processing unit 7 of the present embodiment, as shown in FIG. 11, the hair color is expressed without being affected by the hue, lightness, saturation, etc. of the color of the hair region in the original image. It became possible.

  Next, processing in a state where “hair color” is applied in makeup selecting section 102 in FIG. 9 will be described. When “hair color” is selected as the makeup type in FIG. 9, a plurality of types of hair colors are displayed below the makeup selection section 102 of the makeup palette 310 corresponding to the preview display section 302 as shown in FIG. , An OFF button 753 for erasing makeup of “hair color”, and an eraser selection button 754 for erasing a part thereof.

  The makeup information 720 stored in the HDD 72 of the image processing unit 7 is a hair color for identifying the color, texture, and gradation pattern for hair color corresponding to the hair color selection icon 751 as shown in FIG. Includes multiple types of color pattern information. As shown in FIG. 9, the plurality of types of hair color pattern information include information on hair colors such as purple, silver blue, blonde, and pink. In addition, specific color information and gradation pattern information that changes the color tone or shading between the top and the tip of the hair in association with “FAIRLY”, “MILKY PINK”, or “SODA” representing a color image are displayed. May be included. The information on the gradation pattern includes, for example, information on a plurality of colors to be mixed, information on a position where the color is changed, transparency of each color, and the like.

  When “hair color” is selected in makeup selecting section 102 in FIG. 9 in makeup palette 310 shown in this manner, one of a plurality of hair color selection icons 751 is selected. Processing will be described. FIG. 12 is a flowchart illustrating an example of a processing procedure for changing the hair color during hair color makeup. The processing shown in the flowchart of FIG. 12 will be described as processing performed on the preview display unit 302, but the same applies to the preview display unit 301, and detailed description thereof will be omitted.

  The CPU 70 specifies a face area to be made based on the selection state of the make object switching button 104 (step S401), and extracts a hair area corresponding to the specified face area to be made (step S402).

  Then, the CPU 70 identifies hair color pattern information corresponding to the selected hair color selection icon 751 (step S403). The CPU 70 determines the line type, color, and thickness of the drawing brush based on the identified hair color pattern information (step S404), detects contact of the touch pen 32a with the preview display unit 302 (step S405), and determines it. The contact portion is drawn with the drawn line type, color, and thickness drawing brush (step S406).

  The CPU 70 creates a hair color image in which the portion drawn in step S406 is drawn with the line type, color, and thickness determined in step S404 within the range corresponding to the mask image of the hair region extracted in step S402. (Step S407). The CPU 70 extracts a part from the hair color uniform image corresponding to the face area specified in step S401 within the range drawn in step S407 (step S408).

  The CPU 70 combines the hair color image created in step S407 with a part of the hair color uniform image extracted in step S408 (step S409). The composition process in step S409 is the same as step S305 in the flowchart of FIG.

  The CPU 70 superimposes the synthesized image obtained by changing the hair color obtained in step S409 on the original image (step S410), displays the synthesized image on the preview display unit 302 (step S411), and performs processing. Exit.

  Note that the processing procedure shown in FIG. 12 is continuously executed while “hair color” is selected by the makeup selecting unit 102, for example, every time a predetermined time elapses. While the graffiti operation using the touch pen 32 a is being performed in a state where “hair color” is selected, processing for changing the hair color in real time is executed and displayed on the preview display unit 302.

  FIG. 13 is an explanatory diagram showing an outline of a process for changing the hair color during hair color makeup. Each rectangle in FIG. 13 represents a layer and represents an overlap of images. In the process at the time of hair color makeup, unlike the process at the time of one-touch makeup shown in FIG. 11, the soft light composition and the composition process for the original image are performed only in the drawing range based on the operation using the touch pen 32a. The processing is the same as that described with reference to FIG. 11 except that the drawing range is limited.

  FIG. 14 is an explanatory diagram schematically illustrating an example of an image after the hair color is changed. The screen example shown in FIG. 14 shows a state in which the hair color is changed to a light color with respect to the original image shown in FIG. Also in FIG. 14, a photographic image is schematically shown by a diagram. For the human subject on the left side in FIG. 14, it is shown that the color of the entire hair region has been changed by “one-touch makeup”, and for the human subject on the right side in FIG. "Color make" shows that the color is changed only in the drawing range. As described above, by the processing of the image processing unit 7 in the present embodiment, the color of the hair region in the original image is not affected by the hue, lightness, saturation, etc. It has become possible to express hair colors that are completely different from the hair color.

  The above-described synthesis process related to the hair color change using the hair color uniform image is also executed in step S210 in the flowchart of FIG. The CPU 70 stores hair color pattern information and drawing range corresponding to the hair color corresponding to the theme selection icon 711 selected by “one-touch makeup” or the hair color selection icon 751 selected by “hair color makeup”. Keep it. The CPU 70 uses the stored information so as to be the same as the image displayed on the preview display sections 301 and 302 when the graffiti operation is finished, and the processing shown in the flowchart of FIG. 10 or FIG. A synthesis process for changing the hair color by executing the procedure is executed.

  In this embodiment, a uniform image is created for a hair region, and a color image for determining a color and a soft light are combined, and then the original image is combined. However, the color change target is not limited to the hair region. As a region corresponding to a part of a human subject, for example, a portion of hair such as eyebrows, a lip portion painted with lipstick in the original image, a cheek portion subjected to teak, an eye shadow The same processing may be performed to change the original color to another different color even at the eye portion.

  It should be understood that the embodiment disclosed above is illustrative in all respects and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the meanings described above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

7 Image processing unit 70 CPU (computer, processor)
71 Memory 72 HDD
720 information for makeup 73 graphic board (first and second creation means, first and second synthesis means)
7P image processing program

Claims (10)

An area corresponding to the person subject, comprising: means for acquiring a series of images; and specifying means for specifying a face of a human subject photographed from the acquired images and an area of an organ in the face. In an image processing apparatus that performs processing on
Specifying means for specifying an area corresponding to a part of the person subject in the one image for each one of the plurality of images;
Receiving means for receiving a color for the area specified by the specifying means;
First creating means for creating a uniform image in which the color in the specified region is changed to a predetermined color;
Second creation means for creating a colored image in which a range corresponding to the specified area is colored with the color selected by the reception means;
First combining means for combining the colored image with the uniformed image;
An image processing apparatus comprising: a second combining unit configured to combine the image combined by the combining unit into the one image. The specifying means specifies a hair region as a part of the person subject,
The image processing apparatus according to claim 1, wherein the second creation unit accepts selection of a hair color and creates a hair color image as a colored image. The image processing apparatus according to claim 2, wherein the specified color is light brown. The specified color is different for each color option received by the second creating means,
The first creation means includes
Determining a prescribed color corresponding to the color received by the receiving means;
The image processing apparatus according to claim 1, wherein the uniformized image is created with the determined color. The second creation means receives an input of a trajectory including a plurality of locations on the image,
5. The image processing apparatus according to claim 1, wherein a range corresponding to a received trajectory is drawn with the selected color within the specified region. The first synthesis unit performs blend synthesis in which a predetermined calculation is performed on pixel values for pixels corresponding to the case where the colored image is arranged on the uniformized image. 6. The image processing apparatus according to any one of 5 above. The first creation means separates color information indicating a color in the specified area into texture information indicating a texture in the area, and changes the color indicated by the color information after separation to the specified color. change,
The said 1st synthetic | combination means superimposes the said color image and the image based on texture information on the image based on the color information after the color change of the said uniform image in order, and synthesize | combines them. Image processing device. On each of the screens separated to accept operations from a plurality of persons, for each one of the plurality of images, one of a plurality of face regions specified in the image, and the The processing by the first and second creating means and the first and second synthesizing means is executed on the hair area in any of the plurality of face areas. An image processing apparatus according to any one of the above. An image in which a processor that acquires a series of images specifies a face of a human subject and an area of an organ in the face captured in the acquired images, and performs processing on the area corresponding to the human subject In the processing method,
For each one of the plurality of images, an area corresponding to a part of the person subject in the one image is specified,
Accept colors for the identified area,
Create a uniform image that changes the color in the specified area to a predefined color,
Create a colored image by coloring the area corresponding to the area with the accepted color,
The colored image is synthesized with the uniform image,
An image processing method, further comprising combining the combined image into the one image. In a computer program for causing a computer to acquire a series of a plurality of images and executing processing on the acquired plurality of images,
In the computer,
For each one of the plurality of images, an area corresponding to a part of the person subject in the one image is specified,
Accept colors for the identified area,
Create a uniform image that changes the color in the specified area to a predefined color,
Create a colored image by coloring the area corresponding to the area with the accepted color,
The colored image is synthesized with the uniform image,
A computer program for executing a process of further combining a combined image into the one image.