KR101691806B1

KR101691806B1 - Method and apparatus for displaying images using pre-processing

Info

Publication number: KR101691806B1
Application number: KR1020150099176A
Authority: KR
Inventors: 김재철; 정진욱
Original assignee: 주식회사 시어스랩
Priority date: 2015-07-13
Filing date: 2015-07-13
Publication date: 2017-01-02

Abstract

Disclosed is an image display method using pre-processing and a device thereof. According to an embodiment of the present invention, the image display method using pre-processing comprises the following steps of: detecting a specific object in real time from a subject photographed by a camera; extracting feature points of the detected specific object; forming a plurality of meshes by using the extracted feature points; adjusting the plurality of formed meshes to perform pre-processing correction with respect to the specific object; and displaying the subject including the specific object in which the pre-processing correction is performed.

Description

[0001] METHOD AND APPARATUS FOR DISPLAYING IMAGES USING PRE-PROCESSING [0002]

The present invention relates to an image display using preprocessing, and more particularly, to an image display method and apparatus using a preprocessing method capable of correcting a subject by preprocessing in real time before a subject photographed by a camera is displayed on a screen.

Recently, digital cameras have been used in various fields. It is used in a wide variety of fields from general cameras for taking pictures to shooting digital movies.

Such a digital camera captures an image using a light-receiving element formed of a semiconductor, processes the image taken by the light-receiving element and displays it on the screen, thereby visually providing the user with the captured image or the image to be captured .

Since a digital camera displays an image photographed by a camera module including a light receiving element on a screen only through a basic digital image processing, a user image can be obtained when the user's face is photographed.

Embodiments of the present invention provide an image display method and apparatus using a preprocessing method capable of preprocessing a subject photographed by a camera in real time before being displayed on a screen to correct the subject and providing a corrected subject.

Embodiments of the present invention provide a method and an apparatus for displaying an image using a preprocessing method capable of providing a corrected face desired by a user by allowing the degree of correction of a user's face and a correction region to be set through user setting.

According to an embodiment of the present invention, there is provided an image display method using preprocessing, comprising: detecting a specific object in real time from a subject photographed by a camera; Extracting feature points of the detected specific object; Forming a plurality of meshes using the extracted minutiae; Adjusting the plurality of formed meshes to perform preprocessing correction on the specific object; And displaying the object including the specific object corrected by the preprocessing.

Wherein the preprocessing correcting step identifies at least one of eye, nose, jaw line, face contour, mouth, and eye line when the specific object is a person and adjusts the mesh for the identified one of the plurality of meshes You can pre-calibrate specific objects.

The preprocessing correction may modify the mesh shape for the identified at least one to pre-process the particular object.

The pre-processing step may include preprocessing the specific object by adjusting the color, material, and brightness of the texture applied to the at least one mesh.

The preprocessing correction may pre-process the specific object by adjusting the identified mesh for at least one based on preset correction information by a user.

The preprocessing correction may pre-process the specific object by adjusting the meshes for the identified at least one based on a preset degree of correction for each face part.

An image display apparatus using a preprocess according to an embodiment of the present invention includes an object detection unit for detecting a specific object in real time from a subject photographed by a camera; A feature point extracting unit for extracting feature points of the detected specific object; A mesh forming unit that forms a plurality of meshes using the extracted feature points; A preprocessing corrector for performing preprocessing correction of the specific object by adjusting the plurality of meshes formed; And a display unit for displaying the subject including the specific object corrected by the preprocessing.

Wherein the preprocess correction unit identifies at least one of an eye, a nose, a jaw line, a facial contour, a mouth, and an eye line when the specific object is a person, and adjusts a mesh for the identified at least one of the plurality of meshes, It is possible to perform preprocessing correction.

The preprocessing corrector may modify the mesh shape of the identified at least one to pre-process the specific object.

The pre-processing correction unit may perform preprocessing correction of the specific object by adjusting color, material, and brightness of a texture applied to the at least one mesh.

The preprocessing correction unit may perform preprocessing correction of the specific object by adjusting the meshes for the identified at least one based on correction information preset by the user.

The pre-processing correction unit may perform pre-processing correction of the specific object by adjusting the mesh for the identified at least one based on a preset degree of correction for each face part.

Embodiments of the present invention provide a method and apparatus for correcting a photographed or photographed subject in a real subject by, for example, correcting the subject in real time before displaying the photographed subject on the screen and providing the corrected subject, You can provide a face.

In the embodiments of the present invention, the face of a person to be photographed is corrected in a clear and beautiful manner by correcting the eyes, nose, jaw line, face contour, mouth, eye line and the like among the faces of a person to be photographed, Display, and can be applied to both picture taking and moving picture taking. In particular, embodiments of the present invention can pre-correct eye, nose, jaw line, and face contours of a face.

Embodiments of the present invention allow the user to set the degree of correction and the correction region of the user's face, so that the user can display a desired corrected face on the screen.

The present invention can be applied to all fields related to images such as a field of photographing such as photographing or moving picture photographing, a field displaying a photographed picture or moving picture on the screen.

FIG. 1 shows an example for explaining the present invention.
2 is a flowchart illustrating an image display method using preprocessing according to an embodiment of the present invention.
FIG. 3 shows an operational flow diagram of an embodiment of step S240 shown in FIG.
FIG. 4 illustrates another exemplary operational flow diagram for step S240 shown in FIG.
5 is a diagram illustrating an image display method using a preprocess according to an exemplary embodiment of the present invention.
6 illustrates a configuration of an image display apparatus using a preprocess according to an embodiment of the present invention.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to or limited by the embodiments. In addition, the same reference numerals shown in the drawings denote the same members.

The present invention is characterized in that a pre-processing correction is performed before a subject photographed by a video image photographing device is displayed on a screen so that the face of the subject can be clearly and beautifully corrected and displayed on the screen.

Particularly, in the embodiments of the present invention, the face of a person to be photographed can be corrected in a clear and beautiful manner by pre-processing correction of the eyes, nose, jaw line, face contour, mouth, eye line and the like among human faces included in the photographed subject .

FIG. 1 shows an example for explaining the present invention.

As shown in FIG. 1, the present invention can be applied to a device 100 having a camera, for example, a smart phone, and is installed in an application form on a smart phone or the like, , Motion picture shooting, stored picture or moving picture, it is possible to perform pre-processing correction before displaying the photographed object or the photographed subject on the screen, and display the pre-processed corrected subject on the screen.

At this time, the preprocessing correction is performed by detecting a specific object, that is, a person's face, from the subject photographed by the camera, extracting the feature points of the detected face, forming meshes using the extracted feature points, , The face can be preprocessed and corrected. Furthermore, the preprocessing correction according to the present invention is not limited to the one in which the object is photographed in real time at the time of photographing the object by the camera, and a correction process may be performed before displaying the already stored photograph or moving image on the screen. Of course, the preprocessing correction for the already stored photograph or moving image may or may not be performed through the setting, and this can be provided by the provider providing the present invention.

Hereinafter, for convenience of explanation, it is assumed that the present invention is performed in a smart phone having a camera according to the present invention. Of course, it is apparent to those skilled in the art that the present invention is not limited to being applied to a smart phone, and that the present invention can be applied to all devices on which the present invention can be mounted.

2 is a flowchart illustrating an image display method using preprocessing according to an embodiment of the present invention.

Referring to FIG. 2, an image display method using preprocessing according to an exemplary embodiment of the present invention executes an application for capturing an image, and when a subject is photographed by a camera, a predetermined specific object, A human face is detected (S210).

The specific object detected in step S210 is not limited to a human face, and may include an animal face, a man-made structure, for example, a statue face, and the type of a specific object to be detected may be determined by a vendor. And may be set by a user through a user setting item as necessary.

At this time, in step S210, a human face can be detected by recognizing a face contour of a person in the subject.

If a specific object is detected in step S210, the feature points of the detected specific object are extracted, and a plurality of meshes are formed using the extracted feature points (S220, S230).

The feature points to be extracted are points corresponding to characteristics of a specific object and feature points can be extracted from the entire face region such as eyes, nose, mouth, eye line, lip, chin line, outline, And the feature points can be extracted by tracking the photographed face in real time.

Step S230 may form a plurality of meshes for a specific object, for example, a face portion of a human being, and may form a plurality of meshes using all the methods of forming meshes using minutiae.

If a plurality of meshes for a specific object are formed in step S230, the plurality of meshes formed are adjusted to perform preprocessing correction on the specific object (S240).

Here, the step S240 identifies at least one of eye, nose, jaw line, face contour, mouth, and eye line when the specific object is a person, adjusts the mesh for at least one of the plurality of meshes, For example, by performing 3D rendering on the eye, nose, jaw line, and facial contour to perform correction such as cutting the jaw, raising the nose, enlarging the eyes, etc., It is possible to perform preprocessing correction.

In step S240, the degree of correction and the correction region for performing the preprocessing for a specific object may be preset by the business operator, but the present invention is not limited thereto and may be set by the user. Thus, step S240 can perform preprocessing correction of the human face by adjusting the meshes for the human face based on the preset degree of correction and the corrected region.

That is, in step S240, the specific object may be subjected to preprocessing correction by adjusting the mesh for at least one identified based on the correction information preset by the user, for example, based on the corrected region and the corrected degree for each corrected region, The specific object may be pre-processed by adjusting the mesh for at least one identified based on the per-site correction level.

An exemplary method of performing preprocessing correction of a specific object in step S240 may include: 1) modifying a mesh shape for a plurality of meshes to perform preprocessing correction on a specific object; and 2) You can also pre-calibrate specific objects by adjusting the brightness. At this time, the meshes to be preprocessed may be meshes for at least one predetermined region among the regions constituting the specific object.

1, a process S240 of modifying a mesh shape by performing a preprocessing correction on a specific object may be performed in the same manner as in the example shown in FIG. 3, if a plurality of meshes are formed for a specific object, for example, Identifying predetermined regions of the human face, e.g., eye, nose, jaw line, face contour, mouth, and eye line, and changing the mesh shape for the identified eye, nose, jaw line, face contour, mouth, , Thereby preprocessing the human face (S310, S320).

At this time, the mesh shape changed in step S320 may be changed based on the degree of correction preset by the business operator or the user, or may be changed based on the predetermined site-specific correction data or the correction ratio.

(S240) of adjusting the color, material, and brightness of the texture by adjusting the texture of the texture by performing preprocessing correction on the specific object as shown in FIG. 4, the specific object, for example, The eye, the nose, the jaw line, the face contour, the mouth and the eye line are identified from the human face and the identified eye, nose, jaw line, face contour, mouth, By adjusting the color, material, and brightness of the texture applied to the mesh, the human face is subjected to preprocessing correction (S410, S420).

In this case, the color, the material, and the brightness of the texture adjusted in step S420 may be adjusted based on the face color, brightness, texture of the object to be photographed, and the preset degree of correction, And may be adjusted based on a predetermined correction degree without any limitation. In addition, the step S420 may adjust the color, material, and brightness of the texture to be applied to the meshes of the facial region identified in consideration of the facial atmosphere or entertainer's face set by the user. In this case, the user may select a model, an entertainer, an idol photograph, etc. to be resembled and perform a preprocessing correction on the face using the face of the selected photo.

Referring again to FIG. 2, when the specific object of the subject is preprocessed and corrected in step S240, the subject including the specific object subjected to the preprocessing correction is displayed on the screen (S250).

If the subject subjected to the preprocessing correction in step S250 is displayed on the screen and the subject photographing command is received by user interaction, the subject displayed on the screen, i.e., the subject subjected to the pre-processing, is photographed (S270).

At this time, the photographed or moving picture photographed in step S270 may be stored or shared with others.

For example, as shown in FIG. 5, the subject including the specific object subjected to the preprocessing correction is displayed on the display area 510, and when the photographing or moving picture photographing button 520 is selected by the user, You can shoot a subject or take a movie.

As described above, the image display method using the preprocessing according to the embodiment of the present invention allows the face of a person to be displayed more beautifully than the real one by performing pre-processing correction before photographing or photographing a subject to be photographed on the screen .

In the method according to the embodiment of the present invention, the setting for correcting the face by the preprocessing may be determined by the provider providing the present invention, but the user can set the information about the degree of correction, the corrected region, the atmosphere, .

In addition, although this preprocessing correction function is basically applied to the application to which the present invention is applied, it may or may not be applied according to the user setting. For example, if the user does not want to use the preprocessing function, the uncorrected person's face may be displayed on the screen.

The present invention can be applied to moving picture shooting by performing preprocessing correction in real time for each frame of a moving picture to be photographed when moving pictures are taken.

Furthermore, the method according to an exemplary embodiment of the present invention can be applied not only to photographing an image in real time, but also to a photograph or moving image already captured and stored.

For example, a face-corrected photograph may be displayed on the screen after performing face detection, feature point extraction, mesh formation, and preprocess correction on a photograph before displaying a pre-stored photograph on the screen. Of course, the preprocessing correction function for the stored picture or moving picture can be set by the user.

In the image display method using the preprocess according to the embodiment of the present invention, the preprocessing processes are performed before the subject is photographed on the viewfinder screen. As described above, the pre- So that it can be corrected and displayed.

FIG. 6 illustrates a configuration of an image display apparatus using preprocessing according to an exemplary embodiment of the present invention, and shows a configuration of an apparatus for performing the image display method of FIGS. 2 to 5 described above.

6, an image display device 600 using a preprocess according to an embodiment of the present invention includes an object detecting unit 610, a feature point extracting unit 620, a mesh forming unit 630, a preprocessing correction unit 640, A display unit 650, and a storage unit 660.

The object detection unit 610 detects a specific object in real time from a subject photographed by a camera.

Here, the object detecting unit 610 can detect not only a human face but also an animal face, a man-made structure, for example, an in-phase face from a subject, and the type of a specific object to be detected by the object detecting unit 610 is But may be set by a user through a user setting item as needed.

At this time, the object detecting unit 610 can detect the human face by recognizing the human face contour in the object.

The feature point extracting unit 620 extracts the feature points of the detected specific object.

In this case, the feature point extracting unit 620 may extract feature points from all areas of the face such as eyes, nose, mouth, eye line, lip, chin line, outline, The feature points can be extracted by tracking the photographed face in real time.

The mesh forming unit 630 forms a plurality of meshes using the extracted feature points.

Here, the mesh-forming unit 630 can form a plurality of meshes for a specific object, for example, a human face region, and can form a plurality of meshes using all the methods for forming meshes using minutiae points .

The preprocessing correction unit 640 preprocesses the specific object by adjusting a plurality of formed meshes.

Here, the preprocessing corrector 640 identifies at least one of eye, nose, jaw line, face contour, mouth, and eye line when the specific object is a person, and adjusts the mesh for at least one of the plurality of meshes You can pre-calibrate the object, and especially adjust the mesh for the eye, nose, jaw line, and face contour. For example, 3D rendering to reduce the jaw, raise the nose, enlarge the eyes, etc. , The human face can be preprocessed and corrected.

The preprocessing correction unit 640 can perform preprocessing correction of a specific object by adjusting a plurality of meshes using a degree of preprocessing correction and a correction site for a specific object set by a business operator or a user providing the device.

At this time, the preprocessing correction unit 640 may perform preprocessing correction of the specific object by adjusting the mesh for at least one identified based on the correction information preset by the user, for example, And may pre-process the specific object by adjusting the mesh for at least one identified based on the predetermined degree of correction for each face part.

Specifically, the preprocessing correction unit 640 may modify a mesh shape of a plurality of meshes to perform preprocessing correction of a specific object, or may adjust a color, a material, and a brightness of a texture applied to a plurality of meshes It is also possible to perform preprocessing correction.

For example, the preprocessing corrector 640 may determine a specific object, for example, a plurality of meshes for a human face, if the plurality of meshes for the human face are formed, to determine predetermined portions of the human face such as eyes, nose, The human face can be preprocessed corrected by identifying the line and changing the mesh shape for the identified eye, nose, jaw line, face contour, mouth, and eye line.

In another example, the preprocessing corrector 640 may determine that a particular object, e. G., A plurality of meshes for a human face, has been formed, such as the eye, nose, jaw line, face contour, The human face can be preprocessed and corrected by identifying the eye line and adjusting the color, material, and brightness of the texture on the mesh for the identified eye, nose, jaw line, face contour, mouth, and eye line.

In this case, the preprocessing correction unit 640 may adjust the color, material, and brightness of the texture to be imposed on the mesh based on the facial color, brightness, texture, and preset degree of correction of the subject, , And the color, material, and brightness of the texture can be adjusted based on the preset degree of correction irrespective of the texture.

In addition, the preprocessing corrector 640 may adjust the color, material, and brightness of the texture applied to the meshes of the facial region identified in consideration of the facial atmosphere or the entertainer's face set by the user.

The display unit 650 displays a subject including the specific object subjected to the preprocessing correction.

In this case, the display unit 650 may be means for displaying data related to the image, and may be means for displaying all the data related to the present invention, such as a pre-processed corrected object, a video related user interface, May be a touch screen.

The storage unit 660 is a means for storing data related to the present invention, and stores applications, algorithms, pictures, moving pictures, and the like stored by the present invention.

The system or apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the systems, devices, and components described in the embodiments may be implemented in various forms such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array ), A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to embodiments may be implemented in the form of a program instruction that may be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims

delete

Detecting a specific object in real time from a subject photographed by a camera;
Extracting feature points of the detected specific object;
Forming a plurality of meshes using the extracted minutiae;
Adjusting the plurality of formed meshes to perform preprocessing correction on the specific object; And
A step of displaying the object including the specific object corrected by the preprocessing
Lt; / RTI >
The preprocessing correcting step
Identifying at least one of an eye, a nose, a jaw line, a facial contour, a mouth, and an eye line if the specific object is a person, and identifying at least one of the plurality of meshes Corrects the specific object by preprocessing,
The detecting step and the preprocessing correcting step
A method of displaying an image using a pre-processing performed before displaying the object on a viewfinder screen.

3. The method of claim 2,
The preprocessing correcting step
And modifying a mesh shape of the identified at least one object to perform a preprocessing correction of the specific object.

3. The method of claim 2,
The preprocessing correcting step
And preprocessing the specific object by adjusting a color, a material, and a brightness of a texture of the at least one identified mesh.

delete

An object detecting unit that detects a specific object in real time from a subject photographed by a camera;
A feature point extracting unit for extracting feature points of the detected specific object;
A mesh forming unit that forms a plurality of meshes using the extracted feature points;
A preprocessing corrector for performing preprocessing correction of the specific object by adjusting the plurality of meshes formed; And
A display unit for displaying the subject including the specific object corrected by the preprocessing;
Lt; / RTI >
The pre-
Identifying at least one of eye, nose, jaw line, face contour, mouth, and eye line if the specific object is a person based on a preset degree of correction and a correction region, Corrects the specific object by preprocessing,
Wherein the pre-processing is performed on the specific object before displaying the object on the viewfinder screen.

10. The method of claim 9,
The pre-
And modifying the mesh shape of the identified at least one object to perform pre-processing correction of the specific object.

10. The method of claim 9,
The pre-
Processing means for performing preprocessing correction of the specific object by adjusting a color, a material, and a brightness of a texture of the at least one identified mesh.

delete