WO2024071884A1

WO2024071884A1 - Apparatus and method for generating image of bald head person, virtual hair styling experience apparatus comprising apparatus for generating bald head person image, and virtual hair styling method using same

Info

Publication number: WO2024071884A1
Application number: PCT/KR2023/014604
Authority: WO
Inventors: 유제정; 정재민; 이종하; 김영신
Original assignee: 주식회사 미러로이드
Priority date: 2022-09-27
Filing date: 2023-09-25
Publication date: 2024-04-04
Also published as: KR102538783B1

Abstract

The present specification relates to: an apparatus and method for generating an image of a bald head person; a virtual hair styling experience apparatus comprising an apparatus for generating the bald head person image; and a virtual hair styling method using same. The method for generating the bald head person image comprises: generating a background image by filling, by inpainting, a background mask in which a person object area has been removed from a person image; generating a bald head image by inputting a face area extracted from the person image into an artificial neural network that generates a head image in a face area; generating a bald head background image by synthesizing the background image and the bald head image; and then generating the bald head person image on the basis of the bald head background image and a mask in which a hair area has been removed from the person image, and thus has the effect of generating a natural person image in which only the hair has been removed from a face image of a person with hair.

Description

Device and method for generating a shaved head person image, and virtual hair styling experience device including a shaved person image generating device, and virtual hair styling method using the same

This specification relates to an apparatus and method for generating a bald head person image for a virtual hair styling experience, a virtual hair styling experience device including a device for generating the bald person image, and a virtual hair styling method using the same.

Recently, in the beauty field, a virtual hair styling experience service has been provided that allows customers to try out various hairstyles they want in advance and decide which hairstyle to use. The virtual hair styling experience service is a service that provides the experience as if the experience had been previously done with hair styling by compositing or overlapping a virtually created hair image with the experiencer's face photo and outputting it. The hair object image was designed by a person using computer graphics, and a virtual experience was provided by superimposing the designed hair object image on the experiencer's face image and adjusting the size and shape as necessary. .

The content described above is only intended to help understand the background technology of the technical ideas of the present invention, and therefore, it cannot be understood as content corresponding to prior art known to those skilled in the art of the present invention.

However, the virtual hair image overlaid on the experiencer's existing hair did not naturally overlap the experiencer's head or face, causing awkwardness, and the existing hair protruded below the virtual hair image, which resulted in a very low level of satisfaction with the experience. .

In addition, it is possible to remove the existing hair from the experiencer's face image and then superimpose a virtual hair image, but there is a problem in that the face image with the existing hair removed is unnatural, lowering the experiencer's satisfaction with the experience.

The present specification is intended to solve the above-described problem, and an embodiment of the present specification aims to generate a natural bald-headed person image by removing only the hair from the facial image of a person with hair.

In addition, the purpose of this specification is to solve the problem that virtual hair images superimposed on existing hair do not naturally overlap the customer's head or face, resulting in awkwardness.

Additionally, the purpose of this specification is to provide a method of automatically generating a bald face image of a person by removing only the hair from the human face image through an artificial neural network (AI algorithm).

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood by those skilled in the art from the description below.

This specification presents a method of generating a bald person image using a bald person image generating device. The method of generating a bald person image includes extracting a background mask from which the person object area is removed by segmenting the person image, and filling the background mask with the removed person object area by inpainting to create a background image. ; generating a face image by extracting a face area from the person image, and inputting the face image into an artificial neural network that generates a head from the face area to generate a bald head image; generating a bald head background image by combining the background image and the bald head image; Extracting a non-hair mask from which hair regions are removed by performing segmentation on the person image; and generating a bald head person image based on the non-hair mask and the bald head background image.

The method for generating a bald person image and other embodiments may include the following features.

According to an embodiment, the step of generating a face image by extracting a face area from the person image and inputting the face image into an artificial neural network that generates a head from the face area to generate a bald head image includes: extracting a face image excluding the hair portion based on the feature points of; and generating the bald head image by inputting the face image into an artificial neural network trained to generate a head portion from any input facial image.

Depending on the embodiment, the face image may be extracted based on a feature point corresponding to the face area among feature points predicted using a Dlib image processing tool.

According to an embodiment, the step of generating a bald person image based on the non-hair mask and the bald head background image may include overlapping the non-hair mask on the bald head background image to form the bald head background image. It may include: generating a person image.

According to an embodiment, the step of generating a bald head person image based on the non-hair mask and the bald head background image may include the hair from the bald head background image based on the non-hair mask. Extracting a bald area corresponding to the area; and generating the bald person image by overlapping the bald area with the person image.

Meanwhile, this specification presents a device for generating images of people with bald heads. The device for generating an image of a bald head person includes: a storage unit; and a control unit functionally connected to the storage unit, wherein the storage unit stores data generated by the control unit, and the control unit performs segmentation on the person image to extract a background mask from which the person object area is removed. , creating a background image by filling in the area from which the person object area was removed in the background mask by inpainting, generating a face image by extracting the face area from the person image, and generating a head from the face area of the face image. A bald head image is generated by inputting it into an artificial neural network, a bald head background image is created by combining the background image and the bald head image, and a non-hair mask is created by segmenting the person image to remove the hair area. may be extracted, and a bald person image may be generated based on the non-hair mask and the bald head background image.

The bald person image generating device and other embodiments may include the following features.

According to an embodiment, the control unit extracts the face image excluding the hair portion based on the feature point corresponding to the face area among the feature points of the face in the person image and learns to generate the head portion from a random input face image. The bald head image can be generated by inputting the face image into an artificial neural network.

Depending on the embodiment, the control unit may generate the bald person image by overlapping the non-hair mask on the bald head background image.

According to an embodiment, the control unit extracts a bald head area corresponding to the hair area from the bald head background image based on the non-hair mask, and overlaps the bald head area with the person image to You can create an image of a person with a bald head.

On the other hand, this specification presents a virtual hair styling experience method. The virtual hair styling experience method includes obtaining a person image from an input image; performing segmentation on the person image to extract a background mask from which the person object area has been removed, and filling a portion of the background mask from which the person object area has been removed by inpainting to create a background image; generating a face image by extracting a face area from the person image, and inputting the face image into an artificial neural network that generates a head from the face area to generate a bald head image; generating a bald head background image by combining the background image and the bald head image; Extracting a non-hair mask from which hair regions are removed by performing segmentation on the person image; generating a bald head person image based on the non-hair mask and the bald head background image; and outputting a virtual hairstyle experience image by combining the selected hairstyle object image with the bald person image.

The virtual hair styling experience method and other embodiments may include the following features.

According to an embodiment, the step of generating a bald person image based on the non-hair mask and the bald head background image includes overlapping the non-hair mask on the bald head background image to create the bald person image. It may include a step of generating a.

According to an embodiment, the step of generating a face image by extracting a face area from the person image and inputting the face image into an artificial neural network that generates a head from the face area to generate a bald head image includes, extracting a face image excluding the hair portion based on facial feature points; and generating the bald head image by inputting the face image into the artificial neural network that has been trained to generate a head portion from any input facial image.

According to an embodiment, the step of outputting a virtual hairstyle experience image by combining the selected hairstyle object image with the bald person image may include detecting facial feature points in the bald face image; extracting a center point of each first eye of the bald face image based on the facial feature points; transforming the hairstyle object image based on the center point of each first eye and the center point of each second eye of the person image from which the hairstyle object is extracted included in meta information of the hairstyle object image; and combining the modified hairstyle object image with the bald head person image to output the virtual hairstyle experience image.

According to an embodiment, the hairstyle object image is further modified based on the center point of each first eye and the center point of each second eye of the person image from which the hairstyle object included in the meta information of the hairstyle object image is extracted. The step of calculating a distance between two eyes and a tilt of the two eyes of the bald face image based on the center point of each first eye; calculating a distance between two eyes and a tilt of the two eyes of the person image from which the hairstyle object is extracted based on the center point of each second eye; adjusting the size of the hairstyle object image to fit the bald face image based on the distance between the eyes of the bald face image and the distance between the eyes of the person image from which the hairstyle object is extracted; And adjusting the rotation angle of the hairstyle object image to fit the bald face image based on the tilt of the eyes of the bald face image and the tilt of the eyes of the person image from which the hairstyle object is extracted. You can.

According to an embodiment, the artificial neural network includes a face area image generated by extracting only the face area from a bald person image with a background, and a bald person without a background generated by removing the background of the bald person image with a background using a segmentation model. When a random face area image is input as an image set consisting of images, it can be learned to generate a head part in the face area and create a bald head image without a background.

Embodiments disclosed in this specification have the effect of creating a natural bald-headed person image by removing only the hair from the facial image of a person with hair.

Additionally, the embodiment disclosed in the present specification has the effect of generating a natural hair composite face image by generating a bald face image from an input person image and then synthesizing the hair object image.

In addition, the method according to the embodiment disclosed in this specification can extract high-quality hair images from images taken directly of the customer's head, thereby providing a virtual hair styling experience with hair images that naturally overlap the customer's head or face. There is a possible effect.

In addition, this specification has the effect of automatically outputting a bald image with hair removed from a person's face image through an AI algorithm, thereby reducing the user's sense of heterogeneity in the hair styling experience due to existing hair.

Meanwhile, the effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

The following drawings attached to this specification illustrate preferred embodiments of the present invention, and serve to further understand the technical idea of the present invention along with specific details for carrying out the invention. Therefore, the present invention is described in such drawings. It should not be interpreted as limited to the specific details.

1 is a diagram conceptually explaining a method of generating a bald person image using a bald person image generating device according to an embodiment.

Figure 2 shows an example of training data for a face segmentation deep learning model.

Figure 3 shows an example of a facial feature point extraction model.

Figure 4 shows an example of a method for generating a training data set for an artificial neural network that generates an image of a bald head.

Figures 5 and 6 are flowcharts explaining a method of generating a bald person image using a bald person image generating device according to an embodiment.

Figure 7 is a configuration diagram of an apparatus for generating an image of a bald-headed person according to an embodiment.

Figure 8 shows a schematic configuration of a virtual hair styling experience device according to an embodiment.

FIG. 9 is a diagram illustrating the operation of a virtual hair styling experience device and the concept of a virtual hair styling experience method using the virtual hair styling experience device according to an embodiment.

FIG. 10 sequentially shows examples of step-by-step states in which a virtual hair styling experience is performed by a virtual hair styling experience device according to an embodiment.

FIG. 11 illustrates an example of a method in which a virtual hair styling experience device combines a hair object image with a bald face image according to an embodiment.

Figure 12 shows an example of transforming a hair object image to be synthesized using facial feature points extracted from a face image.

Figure 13 is a block diagram of an AI device that can be applied to an embodiment of the present invention.

The technology disclosed in this specification can be applied to an apparatus and method for generating a bald person image for a virtual hair styling experience, and a virtual hair styling experience device including a bald person image generating device and a virtual hair styling method using the same. However, the technology disclosed in this specification is not limited to this, and can be applied to all devices and methods to which the technical idea of the technology can be applied.

It should be noted that the technical terms used in this specification are only used to describe specific embodiments and are not intended to limit the spirit of the technology disclosed in this specification. In addition, the technical terms used in this specification, unless specifically defined in a different way in this specification, should be interpreted as meanings generally understood by those skilled in the art in the field to which the technology disclosed in this specification belongs. It should not be interpreted in a very comprehensive sense or in an excessively reduced sense. In addition, if the technical term used in this specification is an incorrect technical term that does not accurately express the idea of the technology disclosed in this specification, it is a technical term that can be correctly understood by a person with ordinary knowledge in the field to which the technology disclosed in this specification belongs. It should be understood and replaced with . Additionally, general terms used in this specification should be interpreted as defined in the dictionary or according to the context, and should not be interpreted in an excessively reduced sense.

Terms containing ordinal numbers, such as first, second, etc., used in this specification may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component without departing from the scope of the present invention.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the attached drawings. However, identical or similar components will be assigned the same reference numerals regardless of the reference numerals, and duplicate descriptions thereof will be omitted.

Additionally, when describing the technology disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the technology disclosed in this specification, the detailed description will be omitted. In addition, it should be noted that the attached drawings are only intended to facilitate easy understanding of the spirit of the technology disclosed in this specification, and should not be construed as limiting the spirit of the technology by the attached drawings.

Throughout the specification, a device or terminal includes a communication terminal or communication device capable of wired or wireless communication with a server or other device. The form of the device or terminal may be various, such as a mobile phone, smartphone, smart pad, laptop computer, desktop computer, wearable device, mirror-shaped display device, smart mirror, etc. Wearable devices can be diverse, such as watch-type terminals, glass-type terminals, HMDs, etc. Additionally, the terminal is not limited to this form and can be implemented with various electronic devices.

Hereinafter, embodiments will be described in detail with reference to the attached drawings.

1 is a diagram conceptually explaining a method of generating a bald person image using a bald person image generating device according to an embodiment. Figure 2 shows an example of training data for a facial segmentation deep learning model, and Figure 3 shows an example of a facial feature point extraction model.

Referring to FIGS. 1 to 3, the bald person image generating device according to the embodiment subdivides the person image 10 input into the device into parts, extracts a background mask from which the object area corresponding to the person in the person image is removed, and , the background image 20 is created by filling the background of the extracted background mask with the removed object area using image inpainting. Here, the method of segmenting the input person image 10 into parts can be performed using a face segmentation deep learning model. The face segmentation deep learning model used in the embodiment may be a BiSeNet model that has been pre-trained so that when a face image is input, a mask is generated for each part. In this embodiment, CelebAMask shown in FIG. 2 can be used as training data for the BiSeNet model.

In addition, the bald person image generating device generates a face image 30 by extracting the face area from the person image 10, and then inputs the generated face image 30 into an artificial neural network that generates a head from the face area to create a bald head. Generate a head image 40. Here, the method of generating the face image 30 by extracting the face area from the person image 10 is to predict 68 facial keypoints using Dlib, an image processing tool, as shown in Figure 3. , In order to remove unnecessary noise in creating a bald head, a face image is created by cropping the face area using feature points 0 to 26 among the predicted feature points. Meanwhile, the artificial neural network that generates the bald head image 40 can use NVIDIA's pix2pixHD model, called Image to Image Translation technology, as a deep learning model for generating a virtual head from images of the face area.

Next, the device for generating a bald head image generates a bald head image 50 with a background by combining the background image 20 and the bald head image 40.

Additionally, the bald person image generating device extracts a non-hair mask 60 by removing the area corresponding to hair (hereinafter referred to as the hair area 61) from the previously segmented person image 10.

Finally, the device for generating a bald-headed person image may generate a bald-headed person image 70 based on the bald-headed head image 50 with a non-hair mask 60 and a background. For example, the device for generating a bald-headed person image may generate a bald-headed person image 70 by overlapping or combining a non-hair mask 60 and a bald head image 50 with a background. For another example, the device for generating a bald head person image may select an area corresponding to the hair area 61 (a head area consisting of only the bald head) from the background image 50 of a bald head with a background based on the non-hair mask 60. corresponds to ) is extracted, and the corresponding area extracted from the background image 50 of the bald head with the background is overlaid on the person image 10 to create a bald person image 70.

Below, with reference to the attached drawings, a learning method of an artificial neural network that generates a bald head image will be described.

Figure 4 shows an example of a method for generating a training data set for an artificial neural network that generates an image of a bald head. Figure 5 shows an example of bald face image data used for learning an artificial neural network.

Referring to Figures 1 and 4, the bald person image generating device generates a face area image 81 without a background by extracting only the face area based on the facial feature points described above from the bald person image 80 with a background, The background is removed from the bald person image 80 with the background using the facial segmentation deep learning model to generate a bald person image 82 without the background. Afterwards, an image set consisting of a face area image 81 and a bald head person image 82 without a background is included in the training data for an artificial neural network that generates a bald head image from the face area image. Next, the bald person image generating device trains an artificial neural network to generate a bald head image 40 by generating a head portion from the face area when a random face area image 10 is input with the generated learning data. Here, facial image data of a person with a bald head can be used as the bald person image 80 with a background. Here, the bald person image 80 is an image obtained by photographing the face of a person who was originally bald, that is, the face of a person who was originally bald, and the face images of these people who were originally bald can be collected from bald face image providers such as public data. . Since the bald face images of the various people are the face images of the originally bald person, that is, the person in the face image acquired from public data is likely to be a relatively old person, the effect of aging in these face images is high. It may be a corrected image with wrinkles, etc. removed.

5 and 6 are flowcharts illustrating a method of generating a bald person image using a bald person image generating device according to an embodiment.

Referring to FIGS. 1 and 5 to 6, an apparatus for generating a bald person image according to an embodiment performs segmentation on the person image 10 to extract a background mask from which the person object area is removed, and extracts the person object from the background mask. The background image 20 is created by filling the removed area with inpainting (S100). Here, the device for generating a bald head person image does not directly use the input person image, but uses a face cropping algorithm to produce a face crop image in which only the face part is cut into a rectangle from the input person image as the person image 10. By using it, deep learning performance can be maximized.

Next, the bald person image generating device generates a face image 30 by extracting a face area from the person image 10, and inputs the face image 30 into an artificial neural network that generates a head from the face area to create a bald head. A head image 40 is created (S110). Here, the process of generating the bald head image 40 (S110) includes the process of extracting the face image 30 excluding the hair part based on the characteristic points of the face from the person image (S111) and the process of extracting the face image 30 excluding the hair portion from the person image (S111). It may be performed by a method comprising a process (S112) of generating the bald head image 40 by inputting the face image 30 into an artificial neural network trained to generate a head portion from an image. Here, the face image 30 can be extracted from the person image 10 based on the feature points corresponding to the face area among the feature points predicted using the Dlib image processing tool. In addition, the artificial neural network consists of a face area image generated by extracting only the face area from a bald person image with a background and a bald person image without a background generated by removing the background of the bald person image with a background using a segmentation model. When a random face area image is input as an image set, it can be learned to generate a head part from the face area to create a bald head image.

Next, the bald head person image generating device combines the background image 20 and the bald head image 40 to generate a bald head background image 50 (S120). Here, the bald head background image 50 refers to an image created by combining the background image 20 with the background of the bald head image 40.

Next, the device for generating a bald head person image performs segmentation on the person image 10 and then extracts a mask 60 in which only the hair region 61 is removed from the person image 10 (S130). A mask that removes only the hair area from the person image 10 may be called a non-hair mask.

Finally, the bald person image generating device generates a bald person image 70 based on the non-hair mask 60 and the bald head background image 50 (S140). According to an embodiment, the bald person image generating device may generate the bald person image 70 by superimposing the non-hair mask 60 on the bald head background image 50. According to another embodiment, the bald head person image generating device extracts a bald head region corresponding to the hair region 61 from the bald head background image 50 based on the non-hair mask 60, The bald head area may be overlaid on the person image 10 to create the bald person image 70. Specifically, the device for generating a bald head person image extracts a bald area corresponding to the hair area 61 by superimposing the non-hair mask 60 on the bald head background image 50, and extracts the bald head area corresponding to the hair area 61. The bald head area can be overlaid on the person image 10 to create a bald person image 70. Here, the method of extracting the bald area corresponding to the hair area generates an overlapping image by overlapping the non-hair mask 60 on the bald head background image 50, and then creates an overlapping image in the overlapping image. The bald area corresponding to the hair area 61 can be extracted.

Here, if a face crop image in which only the face part is cut out from the original person image was used as the person image 10, the created bald person image 70 is superimposed on the original person image to make the person in the original person image bald. Images can be created.

In the above description, steps, processes or operations may be further divided into additional steps, processes or operations, or may be combined into fewer steps, processes or operations, depending on the implementation example of the invention. Additionally, some steps, processes, or operations may be omitted, or the order between steps or operations may be switched, as needed. Additionally, each step or operation included in the above-described method of generating an image of a bald head person may be implemented as a computer program and stored in a computer-readable recording medium, and each step, process, or operation may be executed by a computer device.

Hereinafter, an apparatus for generating an image of a bald-headed person will be described with reference to the attached drawings.

Referring to FIGS. 1 and 7 , the device 100 for generating a bald person image may be configured to include a communication unit 110, a storage unit 120, and a control unit 130. The illustrated components are not essential, and the bald person image generating apparatus 100 may be implemented with more components or fewer components. These components may be implemented in hardware or software, or through a combination of hardware and software.

The communication unit 110 can transmit and receive data with an external device through a network.

The storage unit 120 may store data generated by the control unit 130 or data received through the communication unit 110.

The control unit 130 is functionally connected to the communication unit 110 and the storage unit 120.

In addition, the control unit 130 performs segmentation on the person image 10 to extract a background mask from which the person object area has been removed, and fills the background mask with the part from which the person object area has been removed by inpainting the background image 20. ) can be created.

In addition, the control unit 130 extracts the face area from the person image 10 to generate a face image 30, and inputs the face image 30 into an artificial neural network that generates a head from the face area to create a bald head image 40. ) can be created.

Additionally, the control unit 130 may generate a bald head background image 50 by combining the background image 20 and the bald head image 40.

Additionally, the control unit 130 may perform segmentation on the person image 10 to extract a non-hair mask 60 from which the hair region 61 is removed.

In addition, the control unit 130 generates the bald person image 70 by combining the non-hair mask 60 and the bald head background image 50, or generates the bald head person image 70 based on the non-hair mask 60. A bald head area corresponding to the hair area 61 may be extracted from the bald head background image 50 and the extracted bald area may be overlaid on the person image 10 to generate a bald person image 70. Here, the control unit 130 creates an overlapping image by overlapping the non-hair mask 60 on the bald head background image 50, and extracts the bald area corresponding to the hair area 61 from the overlapping image. You can.

In addition, the control unit 130 extracts the face image 30 excluding the hair portion based on the feature points corresponding to the face area among the feature points of the face predicted using the Dlib image processing tool in the person image 10, and extracts the face image 30 excluding the hair portion. A bald head image 40 can be generated by inputting the face image 30 into an artificial neural network that has been trained to generate a head part from a random face image. Here, the artificial neural network consists of a face area image created by extracting only the face area from a bald person image with a background and a bald person image without a background created by removing the background of the bald person image with a background using a segmentation model. When a random face area image is input as an image set (described in the description of FIG. 4), the control unit 130 learns to create a head part in the face area to create a bald head image.

Networks disclosed herein include, for example, wireless networks, wired networks, public networks such as the Internet, private networks, Global System for Mobile communication networks (GSM) networks, general packet wireless networks (General Packet Radio Network (GPRN), Local Area Network (LAN), Wide Area Network (WAN), Metropolitan Area Network (MAN), Cellular Network, Public Switched Telephone Network ; PSTN), Personal Area Network, Bluetooth, Wi-Fi Direct, Near Field communication, Ultra-Wide band, combinations thereof, or any It may be a different network, but is not limited to these.

Meanwhile, when the face image of a real person with hair is input as input data, the bald person image generating device 100 generates a bald face image of a real person input through an artificial neural network, and allows the user to experience the bald face image. It can be used in a virtual hair styling experience service by overlapping or compositing desired hair object images.

The virtual hair styling experience service may be performed by a virtual hair styling experience device, and the virtual hair styling experience device may include the bald person image generating device 100.

In this specification, a hairstyle may refer to hair having a specific shape and color, and a hair object image may refer to an image of hair having a specific shape and color.

The virtual hair styling experience device 1000 may include all of the components of the bald head person image generating device 100 of FIG. 7 described above. Additionally, the virtual hair styling experience device 1000 may further include a photographing unit 140. The illustrated components are not essential, and a virtual hair styling experience device for providing a virtual hair styling experience service may be implemented with more components or fewer components.

The photographing unit 140 may acquire image data of the person including hair by photographing the person.

Additionally, the control unit 130 may process the acquired facial image data and data stored in the storage unit 120 according to a user's command or a predetermined method or process. The control unit 130 may execute computer-readable code (eg, software) stored in the storage unit 120 and/or instructions triggered by the control unit 130 .

The control unit 130 may be a data processing device implemented in hardware that has a circuit with a physical structure for executing desired operations. For example, the intended operations may include code or instructions included in the program.

For example, data processing devices implemented in hardware include microprocessors, central processing units, processor cores, multi-core processors, and multiprocessors. , ASIC (Application-Specific Integrated Circuit), and FPGA (Field Programmable Gate Array).

The storage unit 120 may store instructions (or programs) that can be executed by the control unit 130. For example, the instructions may include instructions for executing the operation of the control unit 130 and/or the operation of each component of the control unit 130.

The storage unit 120 may be implemented as a volatile memory device or a non-volatile memory device.

Volatile memory devices may be implemented as dynamic random access memory (DRAM), static random access memory (SRAM), thyristor RAM (T-RAM), zero capacitor RAM (Z-RAM), or twin transistor RAM (TTRAM).

Non-volatile memory devices include EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, MRAM (Magnetic RAM), Spin-Transfer Torque (STT)-MRAM (MRAM), and Conductive Bridging RAM (CBRAM). , FeRAM (Ferroelectric RAM), PRAM (Phase change RAM), Resistive RAM (RRAM), Nanotube RRAM (Nanotube RRAM), Polymer RAM (PoRAM), Nano Floating Gate Memory (NFGM), holographic memory, molecular electronic memory device, or insulation resistance change memory.

Depending on the embodiment, the virtual hair styling experience device 1000 may be configured without including the photographing unit 140. In this case, face image data for a person may be retrieved from an internal storage device as separate image data or received from an external device, etc. through the communication network. The communication unit 110 may receive facial image data from the outside through the communication network, or may transmit facial image data processed by the control unit 130 to the outside through the communication network.

The virtual hair styling experience device 1000 may be implemented with a printed circuit board (PCB) such as a motherboard, an integrated circuit (IC), or a system on chip (SoC). For example, the virtual hair styling experience device 100 may be implemented with an application processor.

Additionally, the virtual hair styling experience device 1000 may be implemented as a personal computer (PC), a data server, a portable device, or a smart mirror including the PC.

Portable devices include laptop computers, mobile phones, smart phones, tablet PCs, mobile internet devices (MIDs), personal digital assistants (PDAs), and enterprise digital assistants (EDAs). , digital still camera, digital video camera, portable multimedia player (PMP), personal navigation device or portable navigation device (PND), handheld game console, e-book ( It can be implemented as an e-book) or a smart device. A smart device may be implemented as a smart watch, smart band, or smart ring.

The communication network used by the communication unit 110 to transmit and receive data with external devices or servers includes, for example, a wireless network, a wired network, a public network such as the Internet, a private network, and a global system for mobile communication networks (Global System for Mobile). communication network (GSM) network, General Packet Radio Network (GPRN), Local Area Network (LAN), Wide Area Network (WAN), Metropolitan Area Network (MAN) , cellular network, Public Switched Telephone Network (PSTN), Personal Area Network, Bluetooth, Wi-Fi Direct, Near Field communication, ultra-wideband (Ultra-Wide band), a combination thereof, or any other network, but is not limited to these.

Hereinafter, a virtual hair styling experience method using the virtual hair styling experience device 1000 will be described in detail with reference to the attached FIGS. 9 to 12.

The virtual hair styling experience device 1000 includes the above-described bald person image generating device, and trains an artificial neural network with training data consisting of pairs of bald face images of various people prepared in advance and face area images extracted from the bald face images. It has been ordered.

Referring to FIG. 9, when an experiencer who wants to experience virtual hair styling films himself or herself through the video capture unit of the virtual hair styling experience device 1000 or directly inputs his or her image data, the virtual hair styling experience device 1000 A person image 910 is acquired from the input image data (S901).

Next, the virtual hair styling experience device 1000 inputs the acquired person image 910 into the artificial neural network for generating the bald head image described above to generate a bald face image 920 in which hair is removed from the person image 910 of the experiencer. Do (S902). Here, the bald face image 920 may be generated by the bald head person image generation method described in FIG. 1 .

Meanwhile, the virtual hair styling experience device 1000 transmits the person image data acquired from the person image 910 to the biometric characteristic analysis model to analyze the experiencer's biometric characteristics (S903), and transmits it to the fashion feature analysis model to analyze the experiencer's biometric characteristics. Analyze fashion features (S904). The analyzed biometric characteristic results and fashion characteristic results are input into a preference score prediction model to predict a preference score according to the experiencer's biometric characteristic results and fashion characteristic results (S905), and the virtual hair styling experience device 1000 predicts the preference score prediction result. Accordingly, several hairstyles (930) that are judged to suit the experiencer are recommended (S906). Here, the virtual hair styling experience device 1000 displays the person's skin color and tone, eyebrow shape, beard shape, eye size, pupil size, eye color, ear size, nose size, etc. in the image of the person before being input to the artificial neural network. Analyze biometric characteristics such as face shape, race, age, gender, or fashion characteristics such as clothing type and accessory type, and then recommend at least one hairstyle based on the results of the analysis of biometric characteristics and/or fashion characteristics. You can. Biometric characteristic analysis and fashion feature analysis by the virtual hair styling experience device 1000 may be performed using a biometric feature analysis model and a fashion feature analysis model created using an artificial intelligence model, respectively. The biometric specific analysis model is a CNN-based learning model for numerically expressing the characteristics of facial images, and includes a feature extractor that inputs facial images, learns and outputs biometric information such as age, gender, and skin color. It can be. Additionally, this feature extractor may include a facial feature point extraction model to better capture facial features such as face shape, eye size, and color by regressing the face image into feature points. The fashion feature analysis model is a CNN-based learning model for numerically expressing the fashion features of a person video. It uses human segmentation to classify the person in the input video into arms, legs, face, hair, upper part, and lower part. It is possible to accurately capture the fashion style worn by a person. When a full-body image of a person is input, the model can extract the human mask and extract elements that affect hairstyle, such as tops, bottoms, and accessories.

The virtual hair styling experience device 1000 is based on biometric characteristic information of the input face image of the person expressed numerically in the biometric specific analysis model and/or fashion features of the face image of the person expressed numerically in the fashion feature analysis model. Based on the analysis results, at least one hairstyle can be recommended.

The virtual hair styling experience device 1000 generates a hair style preference prediction model according to biometric characteristics and fashion characteristics based on preference score information for each hairstyle based on biometric characteristics and fashion characteristics given by experts such as hair stylists. Next, the hairstyle preference prediction model is trained with preference score data for each hairstyle predetermined based on biometric characteristics and fashion features, and a biometric characteristic information analysis model for the person to be predicted is output to the trained hairstyle preference prediction model. By inputting the result and/or the fashion feature analysis model output result, at least one hairstyle for the person may be recommended based on the preference prediction result output from the hairstyle preference prediction model.

Next, the virtual hair styling experience device 1000 synthesizes a hair style selected from among several recommended hair styles 930 on the experiencer's shaved head face image 920 (S907) to create a virtual hair styling experience face image 940. Outputs .

Meanwhile, the virtual hair styling experience device 1000 may generate a hair object image 950 by extracting only the area corresponding to the hair from the person image 910 (S908). The generated hair object image 950 may be stored in a hair material database and used as a recommended hairstyle for other experiencers. The virtual hair styling experience device 1000 receives at least one hairstyle selected from among at least one recommended hairstyle, then synthesizes the selected at least one hairstyle with the bald face image of the person output from the artificial neural network and outputs it. can do.

In the above-described embodiment, facial image data of a person captured with a camera was used as facial image data input to the artificial neural network, but the virtual hair styling experience device 1000 without a camera receives it from the outside through a data communication unit or stores it in memory. You can also use the face image data of a person who wants to experience virtual hair styling.

When face image data 1010 of a person is input, the virtual hair styling experience device 1000 inputs the face image data 1010 input into an artificial neural network to determine the original existence of the face image data 1010 for the input person. The bald face image 1020 is created by removing the existing hair 1011, and the hairstyle 1031 selected by the user (experiencer) is composited or overlaid on the bald face image 1020 to create a virtual image with the user-selected hairstyle applied. A face image 1030 is output.

Meanwhile, when the virtual hair styling experience device 1000 generates a hair style 1031 for the user's experience, that is, a hair object image, the hair object image is based on the center point of each eye of the person from the image of the person from which the hair object is extracted. The hair object image is transformed so that it can be appropriately synthesized into the bald face images of various people. At this time, the size of the hair object image may be enlarged or reduced, and the rotation angle, that is, the degree to which the hair object is tilted, may be adjusted.

Figure 11 shows an example of a method by which a virtual hair styling experience device according to an embodiment synthesizes a hair object image into a bald face image, and Figure 12 shows a hair object image to be synthesized using facial feature points extracted from the face image. An example of transformation is shown.

Referring to FIGS. 11 and 8 , the control unit 130 of the virtual hair styling experience device 1000 extracts facial feature points 1102 from the bald face image 1101 and displays them by overlapping them with the bald face image 1101 to create a face. A face image 1103 with a feature point 1102 displayed is generated. Additionally, the control unit 130 extracts facial feature points 1112 from the face image 1111 of a person with hair and generates a face image 1113 in which the facial feature points 1112 are displayed. The control unit 130 can extract feature points from a face image using the dlib facial feature point detection model. The control unit 130 extracts the hair object 1115 from the face image 1111 of a person with hair.

The control unit 130 uses the detected facial feature points to find the center point of the eyes of the person in the bald face image 1101 and the face image 1111 of the person with hair, and determines the coordinates of the center points of the eyes of the two people. Using this, you can find the distance between each person's eyes and the slope of the eyes. Here, the slope of the two eyes can be obtained as the slope of a straight line connecting the center points of the two eyes. Information about the center points of the two eyes extracted from the face image 1111 of a person with hair is stored in the meta information of the extracted hair object image and is used when composited into the hair object image, that is, the bald face image. The center points of the two eyes can be calculated using the facial feature points of FIG. 3. Referring to FIG. 3, in an example of facial feature points extracted from a face image, the feature points of the left eye are a group 301 composed of points 36 to 41, and the feature points of the right eye are a group 301 composed of points 36 to 41. Since it is a group 302 composed of (47), the center point of the left eye can be calculated as the average of the coordinates of points 36 to 41, and the center point of the right eye can be calculated as the average of the coordinates of points 42 to 47. It can be calculated as the average of coordinates.

Referring again to FIG. 11, the control unit 130 uses the coordinates of the center points of the two eyes extracted by the facial feature point extraction method of FIG. 3 to determine the distance between the two eyes of the bald face image 1101 and the tilt of the two eyes. And the hair object image 1115 extracted from the face image 1111 of the person with hair using the distance between the eyes and the tilt of the eyes of the face image 1111 of the person with hair is a bald face image ( 1101), adjust the size and rotation angle of the hair object image so that it overlaps appropriately. At this time, the control unit 130 moves the position of the hair object image so that the hair object image accurately overlaps the face image 1111 of the two people using the center points of the two eyes of the two people. Through the above-described process, a hair composite image 1120, that is, a virtual hair styling experience image, can be generated in which the hair object image extracted from the face image 1111 of a person with hair is appropriately overlapped with the bald face image 1101. there is.

Referring to FIG. 12, FIG. 12(a) is a case where the size of the hair object image 1202 to be synthesized into the bald face image 1201 is smaller than the size of the bald face image 1201, and the enlarged hair object image ( 12(b) shows an example of compositing 1203) into a bald face image 1201, and FIG. 12(b) shows that the size of the hair object image 1205 to be synthesized into the bald face image 1204 is the size of the bald face image 1204. In a larger case, an example in which a reduced hair object image 1206 is synthesized with a bald face image 1204 is shown.

The artificial neural network, biometric characteristic analysis model, fashion feature analysis model, and preference score prediction model of the virtual hair styling experience device according to the embodiment are based on artificial intelligence. Hereinafter, with reference to the attached FIG. 13, the embodiments of the present specification will be described. About a learning data generating device and learning data generating method for learning an artificial neural network for a virtual hair styling experience according to the following, and an artificial intelligence processing device applicable to the virtual hair styling experience device and method including the learning data generating device. Explain.

The AI device 1300 may include an electronic device including an AI module capable of performing AI processing or a server including the AI module. In addition, the AI device 1300 is included as at least a portion of the bald head person image generating device 100 and the virtual hair styling experience device 1000 shown in FIGS. 7 and 8 and performs at least some of the AI processing together. It may be equipped to do so.

The AI processing procedure of the AI device 1300 includes the bald head person image generating device 100 shown in FIG. 7 and the virtual hair styling experience device 1000 shown in FIG. 8, and all operations/steps related to their control. It can include all operations/steps for generating a bald face image through a field and deep learning network. For example, the bald head person image generating device 100 can AI process the collected and generated learning data set to process/judge and learn, and then generate a bald face image with only the hair removed from the input face image with hair. You can perform any action. The AI device 1300 may be included as a component of the control unit 130 of FIGS. 7 and 8 or may be replaced with the control unit 130.

The AI device 1300 may include an AI processor 1301, a memory 1305, and/or a communication unit 1307.

The AI device 1300 is a computing device capable of learning a neural network, and may be implemented as various electronic devices such as a server, desktop PC, laptop PC, tablet PC, etc., or may be implemented as a single chip. In the present invention, the AI device 1300 may be a bald head person image generating device 100 and a virtual hair styling experience device 1000 implemented in any one of the various electronic devices.

The AI processor 1301 can learn a neural network using a program stored in the memory 1305. In particular, the AI processor 1301 can learn a neural network for recognizing device-related data. Here, a neural network for recognizing device-related data may be designed to simulate the structure of the human brain on a computer, and may include a plurality of network nodes with weights that simulate neurons of a human neural network. Multiple network modes can exchange data according to their respective connection relationships to simulate the synaptic activity of neurons sending and receiving signals through synapses. Here, the neural network may include a deep learning model developed from a neural network model. In a deep learning model, multiple network nodes are located in different layers and can exchange data according to convolutional connection relationships. Examples of neural network models include deep neural networks (DNN), convolutional deep neural networks (CNN), recurrent neural networks (RNN), Restricted Boltzmann Machine (RBM), and deep trust. It includes various deep learning techniques such as deep belief networks (DBN) and Deep Q-Network, and can be applied to fields such as computer vision (CV), speech recognition, natural language processing, and voice/signal processing. You can.

Meanwhile, the processor that performs the above-described functions may be a general-purpose processor (e.g., CPU), or may be an AI-specific processor (e.g., GPU) for artificial intelligence learning.

The memory 1305 can store various programs and data necessary for the operation of the AI device 1300. The memory 1305 can be implemented as non-volatile memory, volatile memory, flash memory, hard disk drive (HDD), or solid state drive (SDD). The memory 1305 is accessed by the AI processor 1301, and reading/writing/modifying/deleting/updating data by the AI processor 1301 can be performed. Additionally, the memory 1305 may store a neural network model (eg, deep learning model 1306) generated through a learning algorithm for data classification/recognition according to an embodiment of the present invention.

Meanwhile, the AI processor 1301 may include a data learning unit 1302 that learns a neural network for data classification/recognition. The data learning unit 1302 can learn standards regarding which learning data to use to determine data classification/recognition and how to classify and recognize data using the learning data. The data learning unit 1302 can learn a deep learning model by acquiring learning data to be used for learning and applying the acquired learning data to the deep learning model.

The data learning unit 1302 may be manufactured in the form of at least one hardware chip and mounted on the AI device 1300. For example, the data learning unit 1302 may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as part of a general-purpose processor (CPU) or a graphics processor (GPU) to be used in the AI device 1300. It may be mounted. Additionally, the data learning unit 1302 may be implemented as a software module. When implemented as a software module (or a program module including instructions), the software module may be stored in a non-transitory computer readable media that can be read by a computer. In this case, at least one software module may be provided by an operating system (Operating System) or an application (application program).

The data learning unit 1302 may include a learning data acquisition unit 1303 and a model learning unit 1304.

The learning data acquisition unit 1303 may acquire learning data required for a neural network model for classifying and recognizing data. For example, the learning data acquisition unit 1303 may acquire image data and/or sample data for bald faces and faces with hair to be input to a neural network model as learning data.

The model learning unit 1304 can use the acquired training data to train the neural network model to have a judgment standard on how to classify certain data. At this time, the model learning unit 1304 can learn a neural network model through supervised learning that uses at least some of the learning data as a judgment standard. Alternatively, the model learning unit 1304 can learn a neural network model through unsupervised learning, which discovers a judgment standard by learning on its own using training data without guidance. Additionally, the model learning unit 1304 can learn a neural network model through reinforcement learning using feedback on whether the result of situational judgment based on learning is correct. Additionally, the model learning unit 1304 may train a neural network model using a learning algorithm including error back-propagation or gradient descent.

When the neural network model is learned, the model learning unit 1304 may store the learned neural network model in memory. The model learning unit 1304 may store the learned neural network model in the memory of a server connected to the AI device 1300 through a wired or wireless network.

The data learning unit 1302 further includes a learning data pre-processing unit (not shown) and a learning data selection unit (not shown) to improve the analysis results of the recognition model or save the resources or time required for generating the recognition model. You may.

The learning data preprocessor may preprocess the acquired data so that the acquired data can be used for learning to determine the situation. For example, the learning data preprocessor may process the acquired data into a preset format so that the model learning unit 1304 can use the acquired learning data for learning to recognize image data for the transmitter.

Additionally, the learning data selection unit may select data necessary for learning among the learning data acquired by the learning data acquisition unit 1303 or the learning data pre-processed by the pre-processing unit. The selected learning data may be provided to the model learning unit 1304. For example, the learning data selection unit may recognize a specific field among the data sets collected through the network and select only data included in the specific field as learning data.

Additionally, the data learning unit 1302 may further include a model evaluation unit (not shown) to improve the analysis results of the neural network model.

The model evaluation unit inputs evaluation data into the neural network model, and when the analysis result output from the evaluation data does not satisfy a predetermined standard, the model learning unit 1302 can learn again. In this case, the evaluation data may be predefined data for evaluating the recognition model. As an example, the model evaluation unit may evaluate that, among the analysis results of the learned recognition model for the evaluation data, if the number or ratio of the evaluation data for which the analysis result is inaccurate exceeds a preset threshold, a predetermined standard is not satisfied. .

The communication unit 1307 can transmit the results of AI processing by the AI processor 1301 to an external electronic device. Here, the external electronic device may be defined as a bald head person image generating device 100 and a virtual hair styling experience device 1000. Meanwhile, the AI device 1300 may be implemented by being functionally embedded in the control unit 130 provided in the bald head person image generating device 100 and the virtual hair styling experience device 1000.

Meanwhile, the AI device 1300 shown in FIG. 13 has been described as functionally divided into an AI processor 1301, a memory 1305, and a communication unit 1307, but the above-described components are integrated into one module to form an AI module. Please note that it may also be referred to as .

The term “unit” (e.g., control unit, etc.) used herein may mean, for example, a unit including one or a combination of two or more of hardware, software, or firmware. “Part” may be used interchangeably with terms such as unit, logic, logical block, component, or circuit, for example. A “part” may be the minimum unit of an integrated part or a part thereof. “Part” may be the minimum unit or part of one or more functions. The “part” may be implemented mechanically or electronically. For example, a “part” may be an Application-Specific Integrated Circuit (ASIC) chip, Field-Programmable Gate Arrays (FPGAs), or programmable-logic device, known or to be developed in the future, that performs certain operations. It can contain at least one.

At least a portion of the device (e.g., modules or functions thereof) or method (e.g., operations) according to various embodiments may be stored in a computer-readable storage media, e.g., in the form of a program module. It can be implemented with instructions stored in . When the instruction is executed by a processor, the one or more processors may perform the function corresponding to the instruction. Computer-readable media includes all types of recording devices that store data that can be read by a computer system. Computer-readable storage media/computer-readable recording media include hard disks, floppy disks, magnetic media (e.g. magnetic tape), and optical media (e.g. CD-ROM (compact disc read only memory, digital versatile disc (DVD), magneto-optical media (e.g., floptical disk), hardware devices (e.g., read only memory (ROM), random disk (RAM)) access memory, or flash memory, etc.), and may also include those implemented in the form of a carrier wave (e.g., transmission via the Internet). In addition, program instructions include machine language such as that created by a compiler. In addition to code, it may include high-level language code that can be executed by a computer using an interpreter, etc. The above-described hardware device may be configured to operate as one or more software modules to perform the operations of various embodiments, and vice versa. Same thing.

A module or program module according to various embodiments may include at least one of the above-described components, some of them may be omitted, or may further include other additional components. Operations performed by modules, program modules, or other components according to various embodiments may be executed sequentially, in parallel, iteratively, or in a heuristic manner. Additionally, some operations may be executed in a different order, omitted, or other operations may be added.

As used herein, the term “one” is defined as one or more than one. Additionally, the use of introductory phrases such as “at least one” and “one or more” in a claim may mean that the same claim contains introductory phrases such as “at least one” and “one or more” and ambiguous phrases such as “an.” The introduction of another claim element by the ambiguous phrase "a", if any, shall be construed to mean that any particular claim containing the claim element so introduced is limited to an invention containing only one such element. It shouldn't be.

In this document, expressions such as “A or B” or “at least one of A and/or B” may include all possible combinations of the items listed together.

Unless otherwise specified, terms such as “first” and “second” are used to optionally distinguish between the elements described by such terms. Accordingly, these terms are not necessarily intended to indicate temporal or other priority of such elements, and the mere fact that particular means are recited in different claims does not indicate that a combination of such means cannot be advantageously used. . Accordingly, these terms are not necessarily intended to indicate temporal or other priority of such elements. The mere fact that a particular measure is cited in different claims does not indicate that a combination of these measures cannot be used usefully.

The arrangement of components to achieve the same function is effectively “related” so that the desired function is achieved. Accordingly, any two components combined to achieve particular functionality may be considered to be “related” to each other such that the desired functionality is achieved, regardless of structure or intervening components. Likewise, two such associated components may be considered “operably connected” or “operably coupled” to each other to achieve a desired function.

Additionally, those skilled in the art will recognize that the boundaries between the functionality of the foregoing operations are illustrative only. Multiple operations may be combined into a single operation, a single operation may be distributed into additional operations, and the operations may be executed at least partially overlapping in time. Additionally, alternative embodiments may include multiple instances for a particular operation, and the order of the operations may vary in various other embodiments. However, other modifications, variations and alternatives are also possible. Accordingly, the detailed description and drawings are to be regarded in an illustrative rather than a restrictive sense.

The phrase “may be X” indicates that condition X may be met. This phrase also indicates that condition X may not be met. For example, a reference to a system containing a specific component should also include scenarios in which the system does not contain the specific component. For example, a reference to a method that includes a specific behavior should also include scenarios in which the method does not include that specific component. However, as another example, a reference to a system configured to perform a specific action should also include scenarios in which the system is not configured to perform a specific task.

The terms “comprising,” “having,” “consisting of,” “consisting of,” and “consisting essentially of” are used interchangeably. For example, any method may include at least the operations included in the drawings and/or the specification, or may include only the operations included in the drawings and/or the specification. Alternatively, the word “comprising” does not exclude the presence of elements or acts listed in a claim.

A system, apparatus or device referred to in this specification includes at least one hardware component.

In the above, preferred embodiments of the technology of this specification have been described with reference to the attached drawings. Here, the terms or words used in this specification and claims should not be construed as limited to their usual or dictionary meanings, but should be construed as meanings and concepts consistent with the technical idea of the present invention. The scope of the present invention is not limited to the embodiments disclosed in this specification, and the present invention may be modified, changed, or improved in various forms within the scope described in the spirit and claims of the present invention.

The bald person image generation technology of the present invention has been explained with a focus on examples of application to a bald person image generating device for a virtual hair styling experience and a virtual hair styling experience device based on the bald person image generation technology, but in addition to this, virtual hair styling It is possible to apply it to a variety of devices where you can experience various modifications.

Claims

In a method of generating a bald person image using a bald person image generating device,

Extracting a background mask from which the person object area is removed by performing segmentation on the person image, and filling the part of the background mask from which the person object area has been removed with inpainting to create a background image;

generating a face image by extracting a face area from the person image, and inputting the face image into an artificial neural network that generates a head from the face area to generate a bald head image;

generating a bald head background image by combining the background image and the bald head image;

Extracting a non-hair mask from which hair regions are removed by performing segmentation on the person image; and

Generating a bald head person image based on the non-hair mask and the bald head background image; comprising:

How to create an image of a person with a bald head.
According to claim 1,

The step of generating a face image by extracting a face area from the person image, and inputting the face image into an artificial neural network that generates a head from the face area to generate a bald head image,

extracting a face image excluding hair from the person image based on facial feature points; and

Including the step of generating the bald head image by inputting the face image into an artificial neural network learned to generate a head portion from an input random face image;

How to create an image of a person with a bald head.
The method of claim 2, wherein the artificial neural network is:

A random face as an image set consisting of a face area image created by extracting only the face area from a bald person image with a background and a bald person image without a background created by removing the background of the bald person image with a background using a segmentation model. When the area image is input, the head part is created from the face area and learned to create a bald head image without a background.

How to create an image of a person with a bald head.
The method of claim 3, wherein the face image is:

Extracted based on the feature points corresponding to the face area among the feature points predicted using the Dlib image processing tool.

How to create an image of a person with a bald head.
According to claim 1,

The step of generating a bald head person image based on the non-hair mask and the bald head background image,

Comprising: generating the bald person image by superimposing the non-hair mask on the bald head background image.

How to create an image of a person with a bald head.
According to claim 1,

The step of generating a bald head person image based on the non-hair mask and the bald head background image,

extracting a bald area corresponding to the hair area from the bald head background image based on the non-hair mask; and

Comprising: generating the bald person image by overlapping the bald area with the person image.

How to create an image of a person with a bald head.
storage unit; and

Including a control unit functionally connected to the storage unit,

The storage unit stores data generated by the control unit,

The control unit,

Segmentation is performed on the person image to extract a background mask from which the person object area is removed, and the part of the background mask from which the person object area has been removed is filled with inpainting to create a background image;

Generate a face image by extracting the face area from the person image, input the face image into an artificial neural network that generates a head from the face area, and generate a bald head image,

Generating a bald head background image by combining the background image and the bald head image,

Segmentation is performed on the person image to extract a non-hair mask from which the hair region is removed,

Generating a bald head person image based on the non-hair mask and the bald head background image.

A device for creating images of people with bald heads.
The method of claim 7, wherein the control unit,

In the person image, the face image excluding the hair is extracted based on the feature point corresponding to the face area among the facial feature points, and the face image is input to an artificial neural network trained to generate a head portion from a random input face image. To generate the bald head image

A device for creating images of people with bald heads.
The method of claim 7, wherein the control unit,

Creating the bald person image by superimposing the non-hair mask on the bald head background image.

A device for creating images of people with bald heads.
The method of claim 7, wherein the control unit,

Extracting a bald area corresponding to the hair area from the bald head background image based on the non-hair mask, and overlapping the bald area on the person image to generate the bald person image.

A device for creating images of people with bald heads.
The method of claim 8, wherein the artificial neural network is:

A random face as an image set consisting of a face area image created by extracting only the face area from a bald person image with a background and a bald person image without a background created by removing the background of the bald person image with a background using a segmentation model. When the area image is input, the head part is created from the face area and learned to create a bald head image without a background.

A bald head person image generating device, characterized in that:
Obtaining a person image from an input image;

performing segmentation on the person image to extract a background mask from which the person object area has been removed, and filling a portion of the background mask from which the person object area has been removed by inpainting to create a background image;

generating a face image by extracting a face area from the person image, and inputting the face image into an artificial neural network that generates a head from the face area to generate a bald head image;

generating a bald head background image by combining the background image and the bald head image;

Extracting a non-hair mask from which hair regions are removed by performing segmentation on the person image;

generating a bald head person image based on the non-hair mask and the bald head background image; and

Comprising: combining the selected hairstyle object image with the bald person image and outputting a virtual hairstyle experience image;

How to experience virtual hair styling.
According to claim 12,

The step of generating a bald head person image based on the non-hair mask and the bald head background image,

Comprising: generating the bald person image by superimposing the non-hair mask on the bald head background image.

How to experience virtual hair styling.
According to claim 12,

The step of generating a bald head person image based on the non-hair mask and the bald head background image,

extracting a bald area corresponding to the hair area from the bald head background image based on the non-hair mask; and

Comprising: generating the bald person image by overlapping the bald area with the person image.

How to experience virtual hair styling.
According to claim 12,

The step of generating a face image by extracting a face area from the person image, and inputting the face image into an artificial neural network that generates a head from the face area to generate a bald head image,

extracting a face image excluding hair from the person image based on facial feature points; and

Including the step of generating the bald head image by inputting the face image into the artificial neural network learned to generate a head part from an input random face image;

How to experience virtual hair styling.
According to claim 11,

The step of combining the selected hairstyle object image with the bald head person image and outputting a virtual hairstyle experience image,

Detecting facial feature points from the bald face image;

extracting a center point of each first eye of the bald face image based on the facial feature points;

transforming the hairstyle object image based on the center point of each first eye and the center point of each second eye of the person image from which the hairstyle object is extracted included in meta information of the hairstyle object image; and

Comprising: combining the modified hairstyle object image with the bald head person image and outputting the virtual hairstyle experience image;

How to experience virtual hair styling.
According to claim 16,

The step of transforming the hairstyle object image based on the center point of each eye and the center point of each second eye of the person image from which the hairstyle object is extracted included in the meta information of the hairstyle object image,

calculating a distance between two eyes and a tilt of the two eyes of the bald face image based on the center point of each first eye;

calculating a distance between two eyes and a tilt of the two eyes of the person image from which the hairstyle object is extracted based on the center point of each second eye;

adjusting the size of the hairstyle object image to fit the bald face image based on the distance between the eyes of the bald face image and the distance between the eyes of the person image from which the hairstyle object is extracted; and

Adjusting the rotation angle of the hairstyle object image to fit the bald face image based on the tilt of the two eyes of the bald face image and the tilt of the two eyes of the person image from which the hairstyle object is extracted; comprising;

How to experience virtual hair styling.
The method of claim 12, wherein the artificial neural network is:

A random face as an image set consisting of a face area image created by extracting only the face area from a bald person image with a background and a bald person image without a background created by removing the background of the bald person image with a background using a segmentation model. When the area image is input, the head part is created from the face area and learned to create a bald head image without a background.

How to experience virtual hair styling.