KR102434051B1 - Skin condition analyzing system using moire phenomenon - Google Patents

Abstract

According to an exemplary embodiment of the present disclosure, a skin condition analyzing system may include: a filming device including a filter unit including a plurality of filming modes and a camera unit; and an analysis device configured to analyze a face image for each mode received from the filming device. The analysis device may include: a mode selection module selecting any one among the plurality of filming modes; a change amount analysis module comparing results for each of the plurality of filming modes based on the face image for each mode and calculating a change amount according to a result of the comparison; and a procedure recommendation module recommending a procedure for each state of patients based on the change amount.

Description

Skin analysis system using moire phenomenon {SKIN CONDITION ANALYZING SYSTEM USING MOIRE PHENOMENON}

The present disclosure relates to a system for measuring and analyzing a skin condition using a moire pattern phenomenon and recommending a dermatological procedure.

In the current trend of increasing interest in personal beauty, patients visiting dermatology hospitals and dermatology clinics (hereinafter referred to as dermatology clinics) are continuously increasing. However, the reality is that most patients simply receive popular procedures regardless of their skin condition without clearly diagnosing their skin condition.

With the increase in the performance of imaging devices such as resolution, and the diversification of skin analysis methods, a new skin condition measurement and procedure management system that recommends the most necessary treatment to the current patient by comparing the individual patient's past skin condition with the current skin condition is required.

Laid-open Patent Publication 10-2014-0128698 (2014.11.06.)

An object of the present disclosure is to provide a system for measuring a skin condition using a moire pattern phenomenon, and for managing and recommending a dermatological procedure.

The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

A skin analysis system for solving the above problems includes a photographing device including a filter unit including a plurality of photographing modes and a camera unit, and an analysis apparatus configured to analyze a face image for each mode received from the photographing apparatus, The analysis device includes a mode selection module for selecting any one of the plurality of photographing modes, a change amount analysis module for comparing results for each of the plurality of photographing modes based on the face image for each mode, and calculating a change amount according to the comparison of the results , and a procedure recommendation module for recommending a procedure for each skin condition of the patient based on the change amount.

In an exemplary embodiment, the plurality of photographing modes include a moire mode configured to generate a moire pattern by measuring the curvature of the skin based on a contour line, a high-resolution mode for checking skin conditions, and fine invisible skin troubles Includes ultraviolet (UV) mode to check, polarized mode to measure facial contours by shooting based on a polarizing filter, light mode to check face brightness, and red light mode to check skin texture including wrinkles It can be characterized as

In an exemplary embodiment, the mode selection module receives the face image for each mode, generates a mode change signal instructing a change to a photographing mode corresponding to user procedure information, and captures the mode change signal It may be characterized in that it is transmitted to the device.

In an exemplary embodiment, the change amount analysis module may calculate the change amount by calculating the difference between the face image for each mode and the past face image.

In an exemplary embodiment, the change amount analysis module is configured to: face symmetry, skin volume, skin wrinkles, sagging and off, changes in the height of the face surface, skin brightness, blemishes, blemishes, freckles, pores, It may be characterized by generating skin analysis information including pigmentation, oil, moisture, elasticity, skin tone, and skin temperature.

In an exemplary embodiment, the treatment recommendation module may provide, based on the skin analysis information and the calculated change amount, user preference treatment information and recommended treatment information reflecting public preference treatment.

In an exemplary embodiment, the photographing device includes a front image, a first left image and a first right image that are distorted at a first angle from the front, and a second left image and a second right image that are distorted at a second angle from the front. It can be characterized by obtaining.

In an exemplary embodiment, the change amount analysis module is configured to calculate the difference between the front image, the first left image, the first right image, the second left image, and the second right image, and the past face image. By doing so, it can be characterized in that the amount of change is calculated.

In an exemplary embodiment, a skin analysis method performed by a skin analysis system including a photographing apparatus and an analysis apparatus includes: acquiring, by the photographing apparatus, a face image for each mode classified according to a plurality of photographing methods; receiving, by a device, a face image for each mode, and providing a mode change signal instructing a change to a shooting mode corresponding to user procedure information to the imaging device; providing, by the analysis device, a first mode face image to the analysis device; calculating, by the analysis device, a difference between the first mode facial image and a past facial image; skin analysis including facial symmetry, skin volume, skin wrinkles, sagging and off, changes in the height of the face surface, skin brightness, blemishes, blemishes, freckles, pores, pigmentation, oil, moisture, elasticity, skin tone, and skin temperature Generating the information, and the analysis device, based on the skin analysis information and the calculated change amount, may include the step of providing the user preference treatment information and the recommended treatment information reflecting the public preference treatment.

In an exemplary embodiment, the plurality of photographing methods include a moire mode configured to generate a moire pattern by measuring the curvature of the skin based on a contour line, a high-resolution mode for checking skin conditions, and fine invisible skin troubles Includes ultraviolet (UV) mode to check, polarized mode to measure facial contours by shooting based on a polarizing filter, light mode to check face brightness, and red light mode to check skin texture including wrinkles It can be characterized as

Other specific details of the present disclosure are included in the detailed description and drawings.

The skin analysis system according to an exemplary embodiment of the present disclosure can visually confirm a skin condition change by accurately analyzing the skin condition and facial condition based on the moire phenomenon, and can provide a customized treatment according to the skin condition to the patient. have.

In addition, the skin analysis system according to an exemplary embodiment of the present disclosure may analyze the skin and facial conditions of a patient in various ways by utilizing a photographing mode for measuring various skin conditions in addition to a moire mode for forming a moire pattern.

In addition, the skin analysis system according to an exemplary embodiment of the present disclosure may recommend the most necessary treatment to the current patient by comparing the individual patient's past skin condition with the current skin condition.

Effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a conceptual diagram conceptually illustrating a skin care system according to an exemplary embodiment of the present disclosure.
2 is a block diagram illustrating a skin care system according to an exemplary embodiment of the present disclosure.
3 is a block diagram illustrating a photographing apparatus according to an exemplary embodiment of the present disclosure.
4 is a diagram illustrating a photographing mode of a photographing apparatus according to an exemplary embodiment of the present disclosure.
5 is a block diagram illustrating an analysis apparatus according to an exemplary embodiment of the present disclosure.
6 is a diagram illustrating in detail an operation of an analysis device according to an exemplary embodiment of the present disclosure.
7 is a diagram illustrating visual effects according to a photographing mode according to an exemplary embodiment of the present disclosure.
8 is a diagram illustrating a multi-angle face image according to an exemplary embodiment of the present disclosure.
9 is a flowchart illustrating a skin analysis method according to an exemplary embodiment of the present disclosure.

Advantages and features of the present disclosure and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present disclosure to be complete, and those of ordinary skill in the art to which the present disclosure pertains. It is provided to fully understand the scope of the present disclosure to those skilled in the art, and the present disclosure is only defined by the scope of the claims.

The terminology used in the present disclosure is for the purpose of describing the embodiments and is not intended to limit the present disclosure. In this disclosure, the singular may also include the plural unless specifically stated otherwise in the phrase. As used in the disclosure, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout the disclosure, and "and/or" may include each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present disclosure.

The word "exemplary" is used in this disclosure in the sense of "used as an illustration or illustration." Any embodiment described as “exemplary” in this disclosure should not necessarily be construed as preferred or advantageous over other embodiments.

Embodiments of the present disclosure may be described in terms of a function or a block performing a function. Blocks, which may be referred to as 'parts' or 'modules' of the present disclosure, etc., include logic gates, integrated circuits, microprocessors, microcontrollers, memories, passive electronic components, active electronic components, optical components, hardwired circuits, and the like. It may be physically implemented by analog or digital circuitry, such as, and optionally driven by firmware and software. Also, the term “unit” as used in the disclosure refers to a hardware element such as software, Field Programmable Gate Array (FPGA), or Application Specific Integrated Circuits (ASIC), and “unit” may perform certain roles. However, "part" is not meant to be limited to software or hardware. A “unit” may be configured to reside on an addressable storage medium and may be configured to refresh one or more processors. Thus, by way of example, “part” refers to elements such as software elements, object-oriented software elements, class elements, and task elements, and processes, functions, properties, procedures, subroutines, and programs. It may include segments of code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functionality provided within elements and “parts” may be combined into a smaller number of elements and “parts” or further separated into additional elements and “parts”.

Embodiments of the present disclosure may be implemented using at least one software program running on at least one hardware device and may perform a network management function to control an element.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between a component and other components. A spatially relative term may be understood as a term that includes different directions of components during use or operation in addition to the directions shown in the drawings. For example, when a component shown in the drawing is turned over, a component described as "beneath" or "beneath" of another component is placed "above" of the other component. can get Accordingly, the exemplary term “below” may include both directions below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

In each step, the identification code is used for convenience of description, and the identification code does not describe the order of each step, and each step may be performed differently from the specified order unless the specific order is clearly stated in the context. have.

Unless otherwise defined, all terms (including technical and scientific terms) used in this disclosure may be used with the meanings commonly understood by those of ordinary skill in the art to which this disclosure belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly specifically defined.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram conceptually illustrating a skin care system 10 according to an exemplary embodiment of the present disclosure.

The skin care system 10 may measure and analyze the skin condition using various photographing modes such as moire pattern phenomenon, and recommend treatments that can be treated at dermatology hospitals and dermatology clinics (hereinafter, dermatology clinics). According to an exemplary embodiment, the skin care system 10 can accurately measure the current skin condition by comparing the past skin condition of each patient with the current skin condition according to the increase in the performance of the imaging device such as resolution, and the diversification of skin analysis methods. Based on the measurement and analysis results, it is possible to recommend the most necessary procedure for the current patient.

The skin care system 10 according to an exemplary embodiment of the present disclosure includes an imaging device and an analysis device. The photographing apparatus may acquire a facial image including a skin state and a facial skeleton state from a patient, and may provide the obtained image to the analysis apparatus through a wired communication method or a wireless communication method. The analysis device analyzes and analyzes the overall degree of change on the face, such as checking the condition of the skin before and after the procedure from the received face image, measuring the degree of change, or measuring the change in the overall three-dimensional effect of the face from the face image for each angle. data can be generated.

The skin care system 10 according to an exemplary embodiment of the present disclosure may further include a user terminal or a display device. In an exemplary embodiment, when the facial image acquired from the photographing device or the analysis data generated from the analysis device is provided to the user terminal, the patient can directly check the face and skin condition changes according to his or her procedure through the user terminal You can manage your own treatment progress with data. In an exemplary embodiment, when the facial image acquired from the imaging device, or the analysis data generated from the analysis device is provided to the display device, the doctor who performed the procedure gives the treated patient before and after the procedure based on the facial image and the analysis data It can explain changes in the condition, and can suggest and recommend appropriate additional treatments and prescriptions according to the degree of change.

The skin analysis system according to an exemplary embodiment of the present disclosure can visually confirm a skin condition change by accurately analyzing the skin condition and facial condition based on the moire phenomenon, and can provide a customized treatment according to the skin condition to the patient. have.

In addition, the skin analysis system according to an exemplary embodiment of the present disclosure may analyze the skin and facial conditions of a patient in various ways by utilizing a photographing mode for measuring various skin conditions in addition to a moire mode for forming a moire pattern.

In addition, the skin analysis system according to an exemplary embodiment of the present disclosure may recommend the most necessary treatment to the current patient by comparing the individual patient's past skin condition with the current skin condition.

2 is a block diagram illustrating a skin care system 10 according to an exemplary embodiment of the present disclosure.

The skin care system 10 according to an exemplary embodiment of the present disclosure may include a photographing apparatus 100 and an analysis apparatus 200 .

The photographing apparatus 100 may convert an optical signal of an object incident through an optical lens into an electrical signal or an image (ie, image data). The photographing apparatus 100 may include, for example, a pixel array including a plurality of two-dimensionally arranged pixels and a sensing circuit, and the pixel array may convert received optical signals into electrical signals. For example, the pixel array may be implemented as a photoelectric conversion device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and may be implemented with various types of photoelectric conversion devices. The sensing circuit may convert an electrical signal provided from the pixel array into an image, and output the converted image as raw data. The imaging apparatus 100 may be implemented as a semiconductor chip including a pixel array and a sensing circuit.

The photographing apparatus 100 may include a first communication module 110 and a photographing module 130 .

The photographing module 130 refers to a function or means for converting an object into an electrical signal or image data in a photographing apparatus, or a combination thereof. The photographing module 130 will be described in more detail with reference to FIG. 3 .

The first communication module 110 may be a communication means for providing an electrical signal or image data for the sensed object to the analysis apparatus 200 . In the present disclosure, for convenience of description, the first communication module 110 and the second communication module 210 will be described together.

The skin analysis system 10 may transmit and receive information through the first communication module 110 and the second communication module 210 . The first communication module 110 and the second communication module 210 are a Wi-Fi (WiFi; Wireless-Fidelity) chip, a Bluetooth (Bluetooth??) chip, a wireless communication chip, a Near Field Communication (NFC) chip, a Radio Frequency (RFID) chip. identification) may be included.

Here, communication, that is, transmission and reception of data may be performed by wire or wirelessly. To this end, the communication unit includes a wired communication module that accesses the Internet through a local area network (LAN), a mobile communication module that accesses a mobile communication network through a mobile communication base station and transmits and receives data, and a wireless local area network (WLAN) such as Wi-Fi. A short-distance communication module using an area network communication method or a wireless personal area network (WPAN) communication method such as Bluetooth or Zigbee, or a global navigation satellite system (GNSS) such as GPS (Global Positioning System) ) using a satellite communication module or a combination thereof.

The first communication module 110 and the second communication module 210 are technical standards or communication methods (eg, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access (CDMA2000) Access 2000), Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (EV-DO), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE) ) and LTE-A (Long Term Evolution-Advanced), etc.) can transmit and receive radio signals with at least one of a base station, an external terminal, and an external server on a mobile communication network.

In addition, as the wireless technology of the present disclosure, for example, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband) , WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), and the like.

In addition, the communication technology of the present disclosure is Bluetooth (Bluetooth??), RFID (Radio Frequency Identification), infrared communication (Infrared Data Association; IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi- Fi (Wireless-Fidelity), Wi-Fi Direct, Wireless USB (Wireless Universal Serial Bus), TTL (Transistor-Transistor Logic), USB, IEEE1394, Ethernet, MIDI (Musical Instrument Digital Interface), RS232, RS422, RS485, optical communication (Optical Communication), using at least one of a coaxial cable communication (Coaxial Cable Communication) technology, may include a technology for supporting communication.

Analysis device 200 is a laptop computer (laptop computer), digital broadcasting terminal, PDA (personal digital assistants), PMP (portable multimedia player), navigation, slate PC (slate PC), tablet PC (tablet PC), ultrabook ( ultrabook), wearable devices (eg, smartwatch, smart glass, head mounted display), workstation, server, cloud server, mobile phone, smart phone ), but is not limited thereto, and collectively refers to an electronic processing device that processes data, analyzes skin, and provides a computing function to suggest a recommended treatment.

The analysis apparatus 200 may include a second communication module 210 , a processor 230 , a second communication module 210 , a memory 250 , and a database 270 . The second communication module 210 has been described above.

The processor 230 processes one or more commands required for control of the analysis device 200 , operates according to the commands, and performs organic operations of various functional units of the analysis device 200 , such as performing a determination according to program logic. is configured to control. The processor 230 processes signals, data, information, etc. input or output through an input/output interface (not shown) or the second communication module 210 or by driving an application program stored in the memory 250, thereby providing information appropriate to the user. or may provide or process functions. The processed data may be stored in the memory 250 , used to build the database 270 , or transmitted to the outside through an input/output interface (not shown) or the second communication module 210 . Such a processor 230 may be implemented as a general-purpose processor, a dedicated processor, or an application processor. In an exemplary embodiment, the processor 230 includes a computational processor (eg, a central processing unit (CPU)) including dedicated logic circuits (eg, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.) , GPU (Graphic Processing Unit), AP (Application Processor), etc.), but is not limited thereto. In an exemplary embodiment, the processor 230 is a digital signal processor (DSP) capable of converting an analog signal to digital and high-speed processing, a micro controller unit (MCU), or a neural processing unit (NPU) specialized for processing an artificial neural network. It is not excluded that it is implemented as such.

The memory 250 may be a local storage medium supporting various functions of the processor 230 . The memory 250 may store an application program that can be driven by the processor 230 , data for the operation of the processor 230 , and instructions. At least some of these applications may be downloaded from the outside through wireless communication. The application program may be stored and installed in the memory 250 and driven to perform an operation (or function) by the processor 230 .

Memory 250 is DDR SDRAM (Double Data Rate Synchronous Dynamic Random Access Memory), LPDDR (Low Power Double Data Rate) SDRAM, GDDR (Graphics Double Data Rate) SDRAM, RDRAM (Rambus Dynamic Random Access Memory), DDR2 SDRAM, DDR3 It may be a volatile memory including a dynamic random access memory (DRAM) such as SDRAM or DDR4 SDRAM.

However, embodiments of the present disclosure need not be limited thereto. In an exemplary embodiment, the memory 250 should retain data even when the power supplied to the processor 230 is cut off, and may be provided as a writable non-volatile memory to reflect the change. have. However, the present invention is not limited thereto, and the memory 250 may include flash memory or resistive memory cells such as EPROM or EEPROM, resistive RAM (ReRAM), phase change RAM (PRAM), magnetic RAM (MRAM), and MRAM (MRAM). Spin-Transfer Torgue MRAM), Conductive bridging RAM (CBRAM), Ferroelectric RAM (FeRAM), and various other types of memory may be applied. Alternatively, the memory 250 is an embedded multimedia card (eMMC), a universal flash storage (UFS), or a Compact Flash (CF), Secure Digital (SD), Micro-SD (Micro Secure Digital) , Mini-SD (Mini Secure Digital), xD (extreme digital), or memory stick (Memory Stick) can be implemented in various types of devices. In the present disclosure, it is described that all instruction information is stored in one memory 250 for convenience of description, but the present disclosure is not limited thereto, and the memory 250 may include a plurality of memories.

The database 270 refers to a set of data that is integrated and managed for the purpose of being shared and used by storing various types of information. The database 270 may temporarily or semi-permanently store data. For example, the database 270 stores an operating program (OS) for driving at least one device, data for hosting a website, or data related to an application (eg, a web application). can be In addition, the database may store the modules in the form of computer code as described above. The database 270 is managed through middleware separate from the application program.

Examples of the database 270 include a hard disk drive (HDD), a solid state drive (SSD), a flash memory, a read-only memory (ROM), a random access memory (RAM), and the like. This can be. Such a database may be provided in a built-in type or a detachable type.

The database 270 includes a relational database (RDB), a key-value type database, an object type database, a document type database, a memory database, and the like.

The skin care system 10 according to an exemplary embodiment of the present disclosure further includes a plurality of user terminals, including a first user terminal 301 , a second user terminal 302 , and an N-th user terminal 303 . may include

Each of the first user terminal 301 and the Nth user terminal 303 may receive a face image and/or an analysis image from the second communication module 210 . According to an exemplary embodiment, each of the first user terminal 301 and the N-th user terminal 303 receives a face image obtained from a photographing device, or analysis data generated from an analysis device, and performs facial image and/or analysis The data may be provided to the user (ie the patient) immediately, or stored until presented to the user. The user (patient) can directly check the change of the face and skin condition according to the user's procedure through the facial image and/or analysis data, and can manage the procedure of the user as data. That is, the user can visually check the skin condition change by accurately analyzing the user's skin condition and facial condition through the skin analysis system according to an exemplary embodiment of the present disclosure.

3 is a block diagram illustrating a photographing apparatus 100 according to an exemplary embodiment of the present disclosure. A description overlapping with FIG. 2 will be omitted.

The photographing module 130 may include a filter unit 131 and a camera unit 139 , and the filter unit 131 includes a first mode 133 , a second mode 135 , and an N-th mode 137 . It may include a plurality of shooting modes, such as.

According to an exemplary embodiment of the present disclosure, the plurality of photographing modes include a moire mode configured to generate a moire pattern by measuring the curvature of the skin based on a contour line, a high-resolution mode for checking skin conditions, and fine skin invisible to the naked eye. Ultraviolet (UV) mode to check for troubles, polarized mode to measure the outline of the face by shooting based on a polarizing filter, light mode to check the brightness of the face, and red light mode to check skin texture including wrinkles may include.

According to an exemplary embodiment, the first mode 133 may be a moire mode, the second mode 135 may be a high-resolution mode, and the N-th mode 137 may be a red light mode, but is not limited thereto. to N-th mode.

The camera unit 139 may further include a mode selector (not shown) for receiving a mode change signal for information on the operation mode and changing the operation mode of the image sensor. The mode selector may generate signals for changing the operation mode of the photographing apparatus 100 based on an external signal. The mode change signal will be described in more detail with reference to FIG. 6 .

The camera unit 139 may be a complementary metal-oxide-semiconductor (CMOS) image sensor or a charge coupled devices (CCD) image sensor.

The camera unit 139 may further include an image signal processor (ISP) (not shown). The ISP may image-process raw data provided from the photographing apparatus 100 to generate a converted image. For example, the ISP may image-process raw data based on a set white balance, parameter, and color space. The converted image may be a color space image such as RGB, YUV, or the like.

According to an exemplary embodiment, the photographing apparatus 100 may acquire a front face image and a plurality of side images of the recipient (patient). The photographing apparatus 100 includes a front face image of the recipient, a first left image and a first right image twisted at a first angle (eg, 30 degrees) from the central axis of the front image, and a second angle (eg, 45 degrees) from the central axis. ), a second left image and a second right image, and a third left image and a third right image distorted at a third angle (eg, 60 degrees) from the central axis may be obtained.

4 is a diagram illustrating a photographing mode of the photographing apparatus 100 according to an exemplary embodiment of the present disclosure.

The photographing mode of the photographing apparatus 100 includes a moire mode configured to generate a moire pattern by measuring the curvature of the skin based on a contour line, a high-resolution mode for checking the skin condition, and ultraviolet light for checking microscopic skin troubles that are invisible to the eye It may include a (UV) mode, a polarization mode for measuring the outline of the face by shooting based on a polarization filter, a light mode for checking the brightness of the face, and a red light mode for checking the skin texture including wrinkles.

Referring to FIG. 4A , the photographing apparatus 100 may operate in a high-resolution mode for checking a skin condition. The high-resolution mode takes high-resolution images most similar to human observation, and you can see the closest thing to the real thing without the fisheye effect. Through the high-resolution mode, the operator or the operator may check an objective image of the object to be treated, and consider the image to be treated in precise judgment for the operation.

Referring to FIG. 4B , the photographing apparatus 100 may operate in a moire mode configured to generate a moire pattern by measuring the curvature of the skin based on the contour line. The Moire phenomenon is one of light scattering phenomena, in which light passes through a lattice pattern with a specific brightness and specific wavelength. is a phenomenon that occurs In a moire pattern, lines or figures of the same shape and thickness are drawn at regular and regular intervals on a transparent plate and overlapped in two layers. Moire fringes are a kind of intensity interference effect using incoherent light sources. Moire pattern not only appears much larger than the original grid pattern, but also moves much faster than the movement of a single grid, so it is widely used in industries such as minute displacement, rotation, distortion of curved surfaces, and three-dimensional shape recognition of objects. It is widely used for mimicking wave interference.

According to an exemplary embodiment of the present disclosure, the photographing apparatus 100 generates contour lines on the skin and the face through the moire pattern, so that the symmetry of the face and the skin, volume, wrinkles, sagging and off, and changes in the height of the face surface can be checked intuitively.

Referring to FIG. 4(c) , the photographing apparatus 100 may operate in an ultraviolet (UV) mode for checking fine invisible skin troubles. According to an exemplary embodiment of the present disclosure, the photographing apparatus 100 may check the size, location and distribution of invisible blemishes, blemishes, and freckles, and the location and degree of expansion and distribution of pores through the ultraviolet mode. have.

Although not shown in the drawings, the photographing apparatus 100 can clearly confirm the facial contour by using a polarizing filter, and accurately measure the polarization mode configured to check the pigmentation of the skin such as melasma and redness, and the brightness level of the face. It can operate through a comparable light mode, and a red light mode suitable for checking skin texture conditions such as wrinkles.

5 is a block diagram illustrating an analysis apparatus 200 according to an exemplary embodiment of the present disclosure. A description overlapping with FIG. 2 will be omitted.

The analysis apparatus 200 may be configured to analyze the face image received from the photographing apparatus 100 . According to an exemplary embodiment, the face image may include a face image photographed from the front, a face image for each mode photographed for each mode, and a face image for each angle photographed from the front and side.

The analysis device 200 may include a second communication module 210 , a processor 230 , a memory 250 , and a database 270 . The processor 230 includes a mode selection unit 231 , a mode-specific result comparison unit 233 , a variation analysis unit 235 , and a procedure recommendation unit 237 , and the memory 250 includes a mode selection module 251 . , a result comparison module 253 for each mode, a change amount analysis module 255 , and a procedure recommendation module 257 . The various modules 251 to 257 stored in the memory 250 may be fetched by the processor 230 to perform an operation for performing a specific function through the core of the processor 230 .

The mode selection module 251 may select any one of a plurality of shooting modes. The mode selection module 251 receives the skin analysis information generated from the variation analysis module 255, generates a mode change signal capable of changing to the most appropriate photographing mode for analyzing the patient's skin and face conditions, and the photographing apparatus 100 ) to transmit a mode change signal.

The mode-specific result comparison module 253 may check the face images for each mode acquired from the photographing apparatus 100 , and set the photographing mode having the greatest difference from the past face images as the main photographing mode. The photographing apparatus 100 may photograph an object according to the photographing mode determined as the main photographing mode.

The change amount analysis module 255 may compare the results for each of the plurality of shooting modes based on the face images for each mode, and calculate the amount of change according to the comparison of the results. The amount of change includes a first amount of change indicating a change in skin condition and face shape by differentially calculating a face image for each mode from a past face image, and a front image, the first left image, the first right image, the second left image, and the A second change amount representing a face and skin condition for each angle, and a change in a three-dimensionality of the face may be included by performing a difference operation between each of the second right images and the past face images.

The procedure recommendation module 257 may recommend a procedure for each skin condition of the patient based on the amount of change.

Various face images (a front face image, a face image for each mode, and a face image for each angle) processed by the processor 230 may be stored in the database 270 . The skin analysis information, recommended treatment information, and mode change signal history generated by the processor 230 may be stored in the database 270 . In addition, the database 270 stores user treatment information, past face images, user preferred treatment information, and public preferred treatment information, and according to the request of the processor 230, a mode selection unit 231, a result comparison unit for each mode ( 233 ), the change amount analyzer 235 , and/or the procedure recommendation unit 237 may provide necessary information.

A detailed operation of the analysis device 200 will be described in more detail with reference to FIG. 6 .

6 is a diagram illustrating in detail the operation of the analysis apparatus 200 according to an exemplary embodiment of the present disclosure.

The mode selection module 251 may receive face images for each mode from the photographing apparatus 100 , and load user procedure information from the database 270 . According to an exemplary embodiment, the mode selection module 251 generates a mode change signal instructing a change to a shooting mode suitable for analyzing the treatment result received by the subject based on the user procedure information, and generates the mode change signal. may be transmitted to the photographing device 100 . For example, the mode selection module 251 receives a mode change signal for changing the shooting mode of the imaging device 100 to the most appropriate moire mode for confirming the lifting treatment effect when the user (subject) treatment information is lifting treatment. may be transmitted to the photographing device 100 . For example, the mode selection module 251 changes the imaging mode of the imaging device 100 to the most appropriate UV mode to confirm the blemish removal treatment effect when the user (subject) treatment information is blemish removal treatment through laser. may transmit a mode change signal to the photographing apparatus 100 .

The photographing apparatus 100 may obtain a face image suitable for analysis by switching to a photographing mode suitable for the subject according to the mode change signal, and transmit the face image back to the analysis apparatus 200 . The mode selection module 251 may confirm that the user procedure information and the face image match, and transmit the face image to the change amount analysis module 255 . Although not shown in the drawings, it will be understood that the mode selection module 251 may receive a face image photographed in a single mode or a face image for each angle.

The change amount analysis module 255 may receive the face image from the mode selection module 251 and load the past face image from the database 270 . According to an exemplary embodiment, the change amount analysis module 255 may compare a face image obtained in a photographing mode suitable for confirming a result of a treatment performed by a person to be treated with a past face image.

According to an exemplary embodiment, the change amount analysis module 255 may calculate a first change amount representing a change in a skin condition and a face shape by calculating the difference between the face image for each mode and the past face image. The change amount analysis module 255 performs facial symmetry, skin volume, skin wrinkles, sagging and off, changes in the height of the face surface, skin brightness, freckles, blemishes, freckles, pores, pigmentation, oiliness, Skin analysis information including moisture, elasticity, skin tone, and skin temperature can be generated.

According to an exemplary embodiment, the change amount analysis module 255 separates the front image, the first left image, the first right image, the second left image, and the second right image, respectively, and the past face image. By performing the calculation, it is possible to calculate a second change amount representing the face and skin condition for each angle, and the change in the three-dimensionality of the face. The change amount analysis module 255 may generate the above-described skin analysis information based on the calculated first change amount.

The procedure recommendation module 257 may receive the skin analysis information, the first change amount and/or the second change amount from the change amount analysis module 255, and load the user preference treatment information and the public preference treatment information from the database 270 have. User preference treatment information may mean the type and content of dermatology treatment and treatment preferred by the user entered by the user. It may be information about dermatological procedures and treatment methods that have a large number or interest soaring. The treatment recommendation module 257 may provide user preference treatment information and recommended treatment information reflecting public preference treatment based on the skin analysis information and the calculated first or second change amount.

7 is a diagram illustrating visual effects according to a photographing mode according to an exemplary embodiment of the present disclosure.

According to an exemplary embodiment, FIG. 7 is a face image photographed in a moire mode among a plurality of photographing modes.

The left image of FIG. 7 shows a past facial image before the dermatological procedure and treatment of the recipient. The skin analysis system 10 may observe the skeleton and shape of the face based on the contour line by photographing the face of the recipient in the moire mode. Through the left image of FIG. 7 , it can be confirmed that the skin texture, skin height, and forehead portion are turned off on the right side of the past face image.

The right image of FIG. 7 shows a dermatological procedure and a facial image after treatment of the recipient. Compared with the left image of FIG. 7 , changes in the skin texture, skin height, and wrinkle state on the other side of the face of the recipient can be confirmed through the skin analysis system 10, and a round contour line is drawn due to the improvement of the forehead part. that can be ascertained.

8 is a diagram illustrating a multi-angle face image according to an exemplary embodiment of the present disclosure. 8A is a view showing the face image for each angle as a whole, and FIGS. 8B to 8G are views showing the face image before and after the treatment for each angle.

Referring to FIG. 8A , the photographing apparatus 100 includes a front image, a first left image and a first right image that are distorted at a first angle (eg, 30 degrees) from a central axis of the front image, and a second angle from the central axis. It is possible to obtain a second left image and a second right image that are twisted (eg, 45 degrees), and a third left image and a third right image that are shifted at a third angle (eg, 60 degrees) from the central axis. The change amount analysis module 255 includes a front image, a first left image and a first right image that are distorted at a first angle (eg, 30 degrees) from the central axis of the front image, and a second angle (eg, 45 degrees) from the central axis. ), the second left image and the second right image, and the third left image and the third right image each twisted at a third angle (eg, 60 degrees) from the central axis, respectively, and the past face image. A second change amount representing a change in the face and skin state of each star and the three-dimensionality of the face may be calculated.

Referring to FIGS. 8B, 8C, and 8D , the past facial images before the dermatological treatment and treatment of the recipient are shown as face images for each angle. Fig. 8b is a first right image distorted at a first angle to the right from the central axis of the face image, Fig. 8c is a front face image, and Fig. 8d is a first left image distorted at a first angle to the left from the central axis of the face image to be.

Referring to FIGS. 8E, 8F, and 8G , the face images after the dermatological treatment and treatment of the recipient are shown as face images for each angle. Fig. 8e is a first right image twisted at a first angle to the right from the central axis of the face image, Fig. 8c is a front face image, and Fig. 8d is a first left image twisted at a first angle to the left from the central axis of the face image to be.

By referring to FIGS. 8B and 8E together, it is possible to check the skin and facial condition changes on the right half of the face through the first right image, and by referring to FIGS. 8C and 8F together, the skin and facial condition changes in the front face image can be confirmed. Also, by referring to FIGS. 8D and 8G , changes in skin and facial conditions on the left side of the face can be confirmed through the first left image.

9 is a flowchart illustrating a skin analysis method according to an exemplary embodiment of the present disclosure.

In step S110 , the photographing apparatus 100 may acquire face images for each mode classified according to a plurality of photographing methods.

In step S120 , the photographing apparatus 100 may transmit a face image for each mode to the analysis apparatus 200 .

In step S130, a, a face image for each mode may be received, and a mode change signal instructing a change to a photographing mode corresponding to the user procedure information may be generated.

In step S140 , the analysis apparatus 200 may transmit a mode change signal to the photographing apparatus 100 .

In step S150 , the photographing apparatus 100 may determine a first mode face image corresponding to the user procedure information.

In operation S160 , the photographing apparatus 100 may transmit the first mode face image to the analysis apparatus 200 .

In step S170 , the analysis apparatus 200 may calculate a change amount by performing a difference operation between the first mode face image and the past face image;

In step S180, the analysis device 200 performs facial symmetry, skin volume, skin wrinkles, sagging and off, changes in the height of the face surface, skin brightness, blemishes, blemishes, freckles, pores, pigmentation, Skin analysis information including oil, moisture, elasticity, skin tone, and skin temperature can be generated.

In step S190 , the analysis apparatus may provide user preference treatment information and recommended treatment information reflecting public preference treatment based on the skin analysis information and the calculated change amount.

The skin analysis method according to an exemplary embodiment of the present disclosure may be implemented as a source code using a programming language. For example, source code is Python, R, C, C++, C#, Objective-C, Java, Javascript, Python, Visuak Basic, Visual Basic .NET, PHP, SQL, PL/SQL, Perl, Grooovy, Ruby, Go , MATLAB, Swift, Assembly, and TensorFlow-based application languages can be written through a combination of multiple functions that make up the syntax expressed in various programming languages. A function referred to in the present disclosure means any logical operation for implementing the semantics indicated by the syntax.

The steps of a method or algorithm described in relation to an embodiment of the present disclosure may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. A software module may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present disclosure pertains.

Various embodiments of the present disclosure may be implemented as software including one or more instructions stored in a storage medium (eg, memory) readable by a machine. For example, the processor (eg, the processor 230 ) may call at least one of one or more instructions stored from a storage medium and execute it. This may enable the device to be operated to perform at least one function according to the called at least one instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the 'non-transitory storage medium' is a tangible device and only means that it does not include a signal (eg, electromagnetic wave), and this term is used when data is semi-permanently stored in the storage medium. and temporary storage. For example, the 'non-transitory storage medium' may include a buffer in which data is temporarily stored.

According to an exemplary embodiment, the method according to various embodiments disclosed in the present disclosure may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a device-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store™) or on two user devices. It can be distributed (eg downloaded or uploaded) directly or online between devices (eg smartphones). In the case of online distribution, at least a portion of the computer program product (eg, a downloadable app) is stored at least on a machine-readable storage medium, such as a memory of a manufacturer's server, an application store's server, or a relay server's memory. It may be temporarily stored or temporarily created.

The skin analysis method according to the exemplary embodiment of the present disclosure described above may be implemented as a program (or application) and stored in a medium to be executed in combination with a server that is hardware. Here, the program is coded in a computer language such as C, C++, JAVA, or machine language, which the control unit (CPU) of the computer can read through the device interface of the computer in order for the computer to read the program and execute the methods implemented as the program. Code may be included. Such code may include functional code related to functions defining functions necessary for executing the methods, etc. can In addition, the code may further include additional information or a storage unit reference related code for which location (address address) in the internal or external storage unit of the computer to be referenced for additional information necessary for the control unit of the computer to execute the above functions. . In addition, if the control unit of the computer needs to communicate with any other computer or server located remotely in order to execute the above functions, how should the code communicate with any other computer or server remotely using the communication module of the computer? It may further include a communication-related code for information or media to be transmitted/received during communication.

The storage medium is not a medium that stores data for a short moment, such as a register, a cache, a storage unit, etc., but a medium that stores data semi-permanently and can be read by a device. Specifically, examples of the storage medium include, but are not limited to, a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. That is, the program may be stored in various recording media on various servers that the computer can access or in various recording media on the user's computer. In addition, the medium may be distributed in a computer system connected by a network, and computer-readable codes may be stored in a distributed manner.

Above, although embodiments of the present disclosure have been described with reference to the accompanying drawings, those of ordinary skill in the art to which the present disclosure pertains can realize that the present disclosure can be embodied in other specific forms without changing its technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

Claims (10)

In the skin analysis system,
a photographing apparatus including a filter unit including a plurality of photographing modes, and a camera unit; and
and an analysis device configured to analyze the face image for each mode received from the photographing device,
The analysis device is
a mode selection module for selecting any one of the plurality of shooting modes;
a change amount analysis module for comparing results for each of the plurality of shooting modes based on the face images for each mode, and calculating a change amount according to the comparison of the results; and
and a treatment recommendation module for recommending a treatment for each skin condition of the patient based on the amount of change,
The mode selection module,
receiving the face image for each mode, generating a first mode change signal instructing a change to a photographing mode corresponding to user procedure information, and transmitting the first mode change signal to the photographing device;
The change amount analysis module,
Calculate a first change amount representing a change in skin condition and face shape by calculating the difference between the face image for each mode and the past face image,
The change amount analysis module,
Based on the calculated amount of change, facial symmetry, skin volume, skin wrinkles, sagging and off, changes in the height of the face surface, skin brightness, freckles, blemishes, freckles, pores, pigmentation, oil, moisture, elasticity, skin tone, skin Generate skin analysis information including temperature,
The mode selection module receives the skin analysis information as feedback and transmits a second mode change signal instructing a change to a photographing mode corresponding to a current skin condition to the photographing device. According to claim 1,
The plurality of shooting modes,
a moire mode configured to generate a moire fringe by measuring the curvature of the skin based on the contour;
High resolution mode to check skin condition;
Ultraviolet (UV) mode to check for microscopic skin troubles that are invisible to the naked eye;
Polarization mode for measuring facial contours by shooting based on a polarization filter;
light mode to check the brightness of the face; and
A skin analysis system, characterized in that it includes a red light mode for checking skin texture conditions including wrinkles. delete delete delete According to claim 1,
The procedure recommendation module is
Based on the skin analysis information and the calculated change amount, the skin analysis system, characterized in that it provides user-preferred treatment information and recommended treatment information reflecting the public preferred treatment. According to claim 1,
The photographing device is
Acquiring a front image, a first left image and a first right image skewed at a first angle from the central axis of the front image, and a second left image and a second right image skewed at a second angle from the central axis , skin analysis system. 8. The method of claim 7,
The change amount analysis module,
Changes in face and skin conditions and facial stereoscopicity by angle by differential calculation of each of the front image, the first left image, the first right image, the second left image, and the second right image, and the past face image A skin analysis system, characterized in that calculating a second variation representing A skin analysis method made by a skin analysis system including an imaging device and an analysis device, the method comprising:
obtaining, by the photographing apparatus, face images for each mode classified according to a plurality of photographing methods;
receiving, by the analysis device, the face image for each mode, and providing a mode change signal instructing a change to a photographing mode corresponding to user procedure information to the photographing apparatus;
providing, by the photographing device, a face image of a first mode corresponding to the user procedure information to the analysis device;
calculating, by the analysis device, a change amount by performing a difference operation between the face image in the first mode and the face image in the past;
Facial symmetry, skin volume, skin wrinkles, sagging and off, changes in the height of the face surface, skin brightness, blemishes, blemishes, freckles, pores, pigmentation, oiliness, moisture, elasticity , generating skin analysis information including skin tone and skin temperature; and
and providing, by the analysis device, recommended treatment information reflecting user preference treatment information and public preference treatment based on the skin analysis information and the calculated change amount,
The step of providing the mode change signal to the photographing device comprises:
receiving face images for each mode;
generating a first mode change signal instructing a change to a photographing mode corresponding to user procedure information; and
transmitting the first mode change signal to the photographing device;
The skin analysis system further comprising the step of receiving the skin analysis information as feedback and transmitting a second mode change signal instructing a change to the photographing mode corresponding to the current skin condition to the photographing device. 10. The method of claim 9,
The plurality of shooting methods are
a moire mode configured to generate a moire fringe by measuring the curvature of the skin based on the contour;
High resolution mode to check skin condition;
Ultraviolet (UV) mode to check for microscopic skin troubles that are invisible to the naked eye;
Polarization mode for measuring facial contours by shooting based on a polarization filter;
light mode to check the brightness of the face; and
A skin analysis system, characterized in that it includes a red light mode for checking skin texture conditions including wrinkles.

Images